Novel AR/VR Theater Class Builds Foundation for New Traditions in Theater

“What new experiences can we create?” Informatics Professor Josh Tanenbaum presents this question to the 22 students enrolled in Informatics 295/190: AR/VR Theater, a new course he is teaching in collaboration with Broadway/film producer and director and UCI alumnus Tim Kashani. The question isn’t theoretical or merely an exercise in imagination. The experimental class will develop three augmented and virtual reality theater projects for use by Apples & Oranges Studios, a company founded by Kashani and his wife, Pamela, that develops and produces new musicals in cutting-edge ways.

Apples & Oranges combines Kashani’s passion for both technology and the arts. Kashani, who formed a rock band in his younger years, changed course after high school and went on to receive a B.S. in information and computer science and an MBA from UCI. His wife is a Broadway actress who played the original Rapunzel in Into the Woods, which just happens to be the musical that first sparked Tanenbaum’s own interest in theater. So when Kashani floated the idea of collaborating on a class that would create innovative prototypes for real-world use, Tanenbaum didn’t hesitate.

The class spans winter and spring, requiring a two-quarter commitment from the mix of undergraduate and graduate students. However, according to Tanenbaum, this is actually a longer-term project that involves building “collective literacy” around new media forms: “I view this as a 100-year project in which we’re laying the groundwork for new traditions in theater.”

Intertwining Technology and the Arts
The class includes students from both the Donald Bren School of Information and Computer Sciences and the Claire Trevor School of the Arts, and the reading list follows suit, covering topics ranging from interface design to acting techniques. At times, the class feels more like an acting workshop than an informatics course, with students performing various improv activities, yet Tanenbaum clearly threads the relationship to computing throughout.

The lesson for week four started with a micro-lecture on method acting, but the focus was on “outside-in” approaches, which leveraged external factors such as costumes for character transformation. The students then formed a circle to play two “Ninja Slap!” games — with the only difference between the two versions of the turn-based tag-like game being that the second required participants to wear a black mask draped across their faces.

In the discussion that followed, students talked about how they took the game more seriously when wearing the mask; clearly, the small piece of fabric influenced the user experience. Tanenbaum explains how masks can break actors out of inhibitions and habits as they take on a new character and influence how they’re perceived by others. The class considers how avatars act as masks, noting that their depiction (as strong or wounded, for example) can serve as a form of communication, eliciting a variety of effects and emotions. VR headsets can play a similar role, but Tanenbaum warns his students that the fun must extend beyond the excitement of trying something different: “People like new experiences, and VR is still new. Don’t hide behind the novelty.”

Designing the Experience
For the projects, Tanenbaum has divided the students into three teams that draw on the collective strengths of the class. Each week, the teams must provide certain deliverables. During week four, the team members presented a draft of their concept documents and shared storyboards as they pitched their ideas to Kashani and his colleague Zach Anderson. The projects ranged from a current Broadway show to immersive new endeavors in storytelling, and each team not only conveyed its vision, but also outlined the software and hardware specifications. Tanenbaum is pleased to hear students discuss the academic and theoretical reasoning behind the practical applications envisioned: “It’s rewarding to see you applying what we’re learning here!” Kashani likes that the proposals pull audiences further into the theater experience and integrate well with existing practices and workflows.

Future weeks will involve further designing, testing and developing the experimental software, which exists at the intersection of games, virtual reality, augmented reality and theater. The hands-on design process, resulting in prototypes that will later be realized for use in Broadway productions, is a real-world experience unlike any other in academia. As Kashani explains, “I want you to touch unfinished pieces that will live beyond this class.”

— Shani Murray