Associate Professor of Informatics Recognized with NSF CAREER Award
Dr. Joshua Garcia, Associate Professor of Informatics, was recognized this year with a National Science Foundation CAREER Award. The CAREER Award, one of the most distinguished awards NSF grants to early-career faculty, highlights awardees’ high potential to serve as research and education-oriented role models in their fields.
Garcia was awarded over $700k for his project, “Enhancing Software Testing and Debugging for Autonomous Driving.” The project, awarded under NSF’s Division of Computing and Communication Foundations, is focused on identifying existing bugs in autonomous vehicles driving software, and producing support that makes future debugging easier.
Autonomous vehicles, or AVs as they are called in the research area, have become ubiquitous in recent years. Garcia’s work emphasizes this fact, referencing over fifty corporations currently developing AVs, many of which are already on the market and on our roads. The future view on AVs leans optimistic, with experts Garcia cites predicting that these vehicles will have a significant positive impact on society, in no small part because of their role in reducing overall accident rates. But Garcia points to a long list of incidents over the years, some still very recent in public memory, that indicate how far we actually are from that future. It is also still accepted practice for AVs to undergo what are referred to as “field operational tests,” where AVs are tested by driving out in the world on their own, a practice that Garcia says is both unnecessarily expensive as well as dangerous. Even more, he says it fails to capture “critical testing scenarios.”
Instead, Garcia’s approach has AVs undergo virtual tests that take place in software simulations, which are not only safer but more efficient in the number of scenarios they can generate – and importantly (the linchpin to Garcia’s approach) provide the ability to generate autonomous driving scenarios that maximize the degree to which the AV itself is responsible for the violation.
Garcia’s career, founded on his study of software architecture and design during his PhD in Computer Science at USC, previously focused on research into finding vulnerabilities in mobile phone security software. Organizations like IBM and government agencies like the Department of Homeland Security have used Garcia’s approaches to software vulnerability identification.
But after a number of years, as the international research community grew, Garcia felt the research area of mobile phone security – particularly with Android apps, which was his primary focus – was growing oversaturated, and he decided to pivot to something fresh. With early collaborator Qi Alfred Chen (Assistant Professor of Computer Science, ICS), Garcia started focusing on software analysis for autonomous driving vehicles, applying his software analysis techniques to a current yet still emerging domain. In his work up until that point, Garcia had always included some forms of program analysis (programs written for the purpose of analyzing other programs for certain properties, such as bugs or vulnerabilities), so the shift in focus was more of a shift in application than an entirely new research area.
Garcia’s awarded research identifies six key tasks in its approach to identifying and debugging software in autonomous driving vehicles. One main task Garcia wants the testing software to focus on is generating autonomous driving scenarios that maximize the responsibility of the autonomous vehicle as the source of the error. “Prior to our work,” Garcia explains, “the research was heavily ignoring the importance of the vehicle itself being responsible for a violation. It’s not that hard to generate scenarios that result in some kind of violation, including collisions – to, for example, generate scenarios where you have vehicles that will try to run into the autonomous vehicle, and crash into it. Plenty of preexisting work did that, but focusing on that doesn’t necessarily show you a bug in the autonomous driving system that you would want to fix. For developers of autonomous driving systems, that really makes more sense to focus on.”
An important aspect of Garcia’s project is the educational component. The NSF CAREER award specifically recognizes awardees’ potential for significant contributions in education and mentorship, and Garcia is well poised to meet this bar. His project includes plans for teaching modules on autonomous driving system (ADS) testing at multiple educational levels—graduate, undergraduate, and high school—designed to engage students interested in computer science and software architecture early and support them throughout their educational and research journeys. The project also emphasizes expanding local access and participation through a quarterly educational program developed in collaboration with Professor Emeritus Debra Richardson and Vinh Luong of the UCI Office of Outreach, Access and Inclusion. Offered in partnership with community colleges in Southern California, the program provides students with opportunities to explore computing concepts, engage in ADS research, and receive support as they transition to four-year universities. At the high school level, the project includes summer research experiences that allow students to participate directly in ADS testing.
The other main tasks in Garcia’s approach include a total analysis of the context of any given testing driving scenario, including precedence of traffic laws versus safety (for example: if an autonomous vehicle needs to perform a small traffic violation to avoid an accident, the hope is that the vehicle has the capability to make that kind of nuanced decision), and testing to determine avoidability of collisions. There’s also the issue of repetition and predictability. “You can put the same vehicle in the same scenario on the same map, and it might not do the same thing each time,” Garcia explains. “Simulation itself is very expressive already, and having to run tests over and over to get the bug to appear is very expensive. So we’re focusing on taking non-deterministic tests and trying to make them deterministic, so they can actually debug these things. Actually locate the place in the code where the bug is happening so you can fix it.”
The idea of all this work, which also leverages machine learning, Garcia says, is for developers to “get some confidence that a software system is well tested. Has sufficient quality. You want to be able to test as many parts of the system as possible, [including] the code. A lot of the code inside the autonomous driving system actually isn’t covered by existing tests.” Ultimately, Garcia hopes for his research to reduce “safety-critical errors” that increasingly threaten the public as more and more companies and government organizations adopt the use of autonomous vehicles.
Garcia credits his graduate students for their support in producing this line of work, emphasizing that his current students, Yuqi Huai and Yuntianyi Chen, made significant contributions in the form of consistent input and producing preliminary research. Garcia further credits collaborators willing to support the work through the Autoware Foundation and its Autoware Centers of Excellence (e.g., the head of the centers, Professor Rahul Mangaram, at UCI (Asst. Prof. Qi Alfred Chen and Assoc. Professor Yasser Shoukry), and at companies affiliated with the Autoware Foundation, i.e., Tier IV (Simon Thompson and Maxime Clement) and Whale Dynamic (David Chang and Lingzi Sun). He also acknowledges UC Irvine’s NSF CAREER Institute and the many people who provided advice, support and feedback on the proposal.
For Garcia, who recently earned tenure, the CAREER Award is more than just an important stamp of approval from the U.S. research community – it has a personal meaning to Garcia as an academic in both his professional and personal life. “I’m surrounded by lots of people, in my research area, at the university, outside the university, and also in my academic family, who have won this award,” Garcia says. “So it means a lot to me to now be able to celebrate and be recognized myself in the same way with my community.” The funding is also valuable to Garcia because of what it means for his ability to support his students – the ability to fund a graduate student summer salary, purchase equipment, and fund student travel. “I’m grateful in many ways,” he says.
The official award abstract for the project is available to view online. To learn more about Dr. Garcia and his recent work, visit his website.
– Jenna Abrams