Introduction

NeuroACT is a series of interactive experiences designed for a cutting-edge neural-interface device. Operating on electromyography (EMG) technology that measures the electrical activities produced by our skeletal muscles, this wearable device lets us interact with the computer applications through thin air. We use machine learning to develop our prototypes in 3 specific areas that showcase the device’s strengths in gaming, training, and location-based entertainment. We believe our designs shine a light to what was considered as science fiction and enable guests to experience the device in the most fun and meaningful way.

My Contributions

I was a programmer in this project. NeuroACT was an exploratory project where the team together built several prototypes to understand and find out use-cases for the armband. My responsibilities included :

• Providing feedback on the documentation and the device API.
• Troubleshooting network and data transfer issues with the device.
• Creating interfaces for the device API to work with multiple technologies like the Kinect and web.
• Assisted in designing experiences for the armband which takes into consideration the training and calibration time for every user.
• Prototyped various experiences that make use of different hand gestures and created learning models for them, which allowed us to narrow down on the best gestures that the device worked with.

Final Prototypes

Sound Mosaic - An Interactive Music Synthesizer

Sound Mosaic is an experimental interactive application that uses the EMG armband to control sound parameters and the Kinect to track the user position on the CAVE floor to add new layers to the music track. CAVE is a 3-sided projection screen which contains a walkable space in its center.

Cake Master

The game uses the player's squeeze force to control the cream filling into the cake mould. They are awarded stars based on their accuracy.

Snowboard

Control the snowboarder to make him move through the gates. Pick any gesture to move him left and right.

Send