Understanding and Improving the use of LLMs for Education

Abstract: AI has the potential to extend the capabilities of educators and improve learning outcomes. In this talk, I will describe new metrics for understanding how students use LLMs, new methods that expand the reach of an active pedagogy technique, and techniques that still drive models off-topic. Along the way, we’ll see implications for the idea of AI-resilient user interfaces, identify limitations in the ways most LLM-based education systems are evaluated, and learn about how students are balancing AI-based tools with traditional resources. To conclude, I’ll present a lightning round of key findings from other work in my group, spanning hallucinations in code, benchmarks for visualisation generation, human-AI teaming methods for clinical coding, and more.