Decoding and Encoding Expertise: Toward Human-Centered AI for Software Engineering

Abstract: As Artificial Intelligence becomes a fundamental partner in the software development lifecycle, a critical gap has emerged: while modern models excel at predicting code tokens, they remain “cognitively blind” to the high-level reasoning, intentionality, and memory constraints of human programmers. This cognitive misalignment results in AI models that often fail to match human judgment, leading to diminishing returns as task complexity increases. This talk presents a research roadmap for bridging this divide by treating human cognition not only as an external user requirement, but as a formal blueprint for engineering the next generation of machine intelligence.

The first half of the talk focuses on decoding the developer. Leveraging medical imaging (fMRI/fNIRS) and eye-tracking, I reveal the neurological and cognitive underpinnings of software engineering. By establishing a systematic methodology to model developers’ cognitive processes, I uncover the complex relationship between programming and natural language processing while providing a rigorous framework to quantify human expertise for software engineering. The second half focuses on encoding this expertise to drive measurable gains in AI for SE. I discuss the evolution of our methodology, moving from data-centric augmentation techniques to a cognitive theory-grounded approach. Our results indicate that by integrating human cognitive signals, such as expert attention patterns, we can significantly surpass the performance of traditional, “cognitively-unaware” AI models. Specifically, this alignment leads to substantial improvements in tasks like code summarization and comprehension. Ultimately, this work establishes a closed loop between understanding human cognition and enhancing automated models for SE, providing a scalable foundation for the era of human-centered AI for software engineering.

Bio: Dr. Yu Huang is an Assistant Professor of Computer Science at Vanderbilt University, where she leads the MIND Lab (Mixed INtelligence Development for programming). She also holds a secondary appointment in the Department of Teaching and Learning at the Peabody College of Education and Human Development. Her research focuses on human factors and human-centered AI for software engineering, aiming to bridge the divide between human cognition and automated programming models. Her work leverages medical imaging (fMRI/fNIRS), eye tracking, and cognitive architectures to decode the neural underpinnings of human expertise in software engineering and encode those signals into the next generation of AI tools. Her work also extends to optimizing computer science education and the long-term sustainability of open-source software ecosystems.

Dr. Huang has received four ACM SIGSOFT Distinguished Paper Awards, the 2025 ICPC Vaclav Rajlich Early Career Achievement Award and 2021 EECS Rising Star. Her research is supported by the NSF, GitHub, ARPA-H, and Vanderbilt University. She holds a PhD in Computer Science and Engineering from the University of Michigan.

This seminar is both online and in-person: Zoom Link