Performances tend to be higher quality when there is strong nonverbal sensorimotor communication between the conductor and musicians.


From watching tutorials to reading academic papers, we gained valuable insight into how conductors communicate through their gestures. Unfortunately, the current remote environment hinders this nonverbal communication since rehearsal is often asynchronous and requires multiple pieces of technology. 
In the case study below, the conductor recorded her gestures and created a click track for musicians to listen to which helps them stay on the same beat. Orchestra members then recorded themselves playing while juggling multiple visual and audio signals.

Our new concept featured a tangible device that translates conductor’s gestures into light patterns through a device in the musician’s space.

The use of light is partly inspired by the Maluma-Takete Effect which is the mapping between visual shapes and musical sounds. The following artifacts explore the interactions between a conductor (Ava) and a musician (Darien) who are remotely rehearsing for an upcoming performance using our light device.

We designed the device to communicate the tempo the conductor is setting as well as dynamic gestures such as play louder or softer (crescendo/decrescendo). The device also serves as a helpful tool when the musician is practicing by themselves.