Introduction
TheatAR is an exploration project focusing on the use of augmented reality in live performance. In collaboration with the CMU School of Drama, the team adapted an existing story to the stage, featuring a fully realized animated character that performs alongside a human actor.
The end result is a short performance piece — a proof of concept for AR’s expanding applications in the entertainment industry.
Highlights
Approach
Blending a virtual performer with a live human actor raised unique challenges in timing, spatial alignment and stagecraft:
- Choreographing the animated character to respond to the live actor’s cues.
- Aligning the AR performer convincingly within the physical stage space.
- Designing for an audience experience that felt magical rather than technical.
My contribution
As lead programmer, I turned the performance concept into a working real-time mixed-reality system on HoloLens. I explored spatial occlusion, lighting and persistence so the virtual character felt grounded on the physical stage, and networked multiple devices to support a synchronized audience experience.
The software also had to respect the rhythms of live theater. Technical cues, animation timing and shared spatial state needed to stay aligned with a human performer, making reliability and graceful recovery as important as the visual effect itself. The completed prototype was performed for live audiences at CMU.