Multidisciplinary Collaborators Set their Sights on Color Vision in the Dark
A multidisciplinary collaboration between ophthalmology researchers and computer scientists at UCI has shown promising results in revealing color in the dark. Distinguished Professor of Computer Science Pierre Baldi and students from the Donald Bren School of Information and Computer Sciences (ICS) teamed up with Clinical Assistant Professor of Ophthalmology Andrew Browne and others from the School of Medicine (SOM) to leverage deep learning (DL) in recreating color images from infrared input images. “Deep Learning to Enable Color Vision in the Dark,” the PLOS ONE paper that Browne and Baldi co-authored with computer science Ph.D. students Ekaterina Deyneka, Francesco Ceccarelli and Siwei Chen, and SOM researchers Josiah To, Jianing Tang and Anderson Vu, outlines their findings.
“I met Andrew Browne a couple of years ago, and since then we have been collaborating intensively on several projects,” says Baldi. “His laboratory provides the data; we develop and deploy all the solutions.”
For this project, the solution involved combining infrared illuminants in the red and near-infrared (NIR) spectrum and processing them using deep learning. The goal was to recompose an NIR image to present it as if it were visualized with visible spectrum light, testing the hypothesis that “deep learning could render human-visible scenes using NIR illumination.”
Their conclusion? “This proof-of-principle study using printed images with a limited optical pigment context supports the notion that the deep learning approach could be refined and deployed in more practical scenarios.”
As outlined in the IEEE Spectrum article, “Color Night Vision Brought to You by AI,” there are many practical applications for this work. “You can imagine this having military applications, or just recreational applications to see at night,” Browne says. “You can also imagine this finding being used in chemical processing, where people need to work in places that are sensitive to visible light, or in medical or research situations where one has to work with light-sensitive retinal tissue.”
For example, one long-term goal of this collaboration is to enable eye surgeries that occur with low-energy light (that is, light in the red/infrared portion of the spectrum). “If you shine normal visible light on the retina for long periods of time (often required for surgery), the retina can get damaged,” explains Baldi. “So we want to shine light that has low energy to avoid damaging the retina, but we want the surgeon to be able to see everything as if under regular light. So we are developing AI that can reconstruct images/videos taken with light in the red/infrared portion of the spectrum so they appear as if they were taken under regular light.”
— Shani Murray