Radiance Fields Advancing 3D Scene Understanding for Robotics and Autonomous Driving

Abstract: Since emerging in 2020, neural radiance fields (NeRFs) have marked a transformative breakthrough in representing photorealistic 3D scenes. In the years that followed, numerous variants have evolved, enhancing performance, enabling the capture of dynamic changes over time, and tackling challenges in large-scale environments. Among these, NVIDIA’s Instant-NGP stood out, earning recognition as one of TIME Magazine’s Best Inventions of 2022. Radiance fields now facilitate advanced 3D scene understanding, leveraging large language models (LLMs) and diffusion models to enable sophisticated scene editing and manipulation. Their applications span robotics, where they support planning, navigation, and manipulation. In autonomous driving, they serve as immersive simulation systems or can be used as digital twins for video compression integrated in edge computing architectures. This lecture explores the evolution, capabilities, and practical impact of radiance fields in these cutting-edge domains.

Bio: Matúš Dopiriak is a 3rd-year PhD candidate at the Technical University in Košice, Department of Computers and Informatics, advised by Professor Ing. Juraj Gazda, PhD. His research explores the integration of radiance fields in autonomous mobility within edge computing architectures. Additionally, he studies the application of Large Vision-Language Models (LVLMs) to address edge-case scenarios in traffic through simulations that generate and manage these uncommon and hazardous conditions.