SyncUp

Wearable Device for Remote Music Collaboration






OVERVIEW

Our challenge was to design a remote collaboration tool that utilizes multisensory interactions.


The coronavirus pandemic has drastically shifted the way many of us work by pushing us to collaborate remotely, a  trend that is expected to persist as companies adjust to new culture shifts. Given this timely topic, our brief was to design a multisensory experience that improves collaboration between remote users.

Team
With Hannah Kim, Minkyoung Lee, & Yu Chuan Shan

Advised by Dina El Zanfaly & Kyuha Shim
Time
7 weeks (Fall 2020)
CMU Interaction Design Studio
Role
Design Research, Conceptual Design, Interaction Design, 2D/3D Prototyping
Tools
Figma, Arduino, Adobe Premier Pro, Vectary


WHY MULTISENSORY?

Screen-based interactions make it difficult for people to communicate.


According to the PsychiatricTimes, flat screen interactions eliminate nonverbal cues such as body language and eye contact which then requires users to put in more cognitive and emotional effort to communicate. This is why we may experience “Zoom fatigue.” To address this problem space, our goals were to:
  • Research a remote collaboration context to understand existing problems and design opportunities
  • Design a multisensory experience that utilizes intermodality (e.g., input: touch, gesture, speech; output: visual, auditory, haptic feedback)



OUTCOME

SyncUp is a multisensory experience that includes a wearable device to enhance nonverbal communication between a conductor and musicians in a remote setting.



Wearable device
During rehearsal, the device translates the conductor’s directions into real-time haptic and light feedback on a musician’s wrist.


AI-informed dashboard
After rehearsal, the conductor is able to review the recording with AI-informed data such as which musicians were off-tempo. 

PRODUCT DEMOS


Scenario 1:  Synchronous Remote Rehearsal
A conductor meets with her ensemble to rehearse online. The musicians rely on their wearable device to know the tempo the conductor is setting as well as when they are being cued in.

Starting Rehearsal
Cuing In 


Scenario 2: Rehearsal Review & Individual Practice
The conductor reviews the rehearsal recording and gives asynchronous feedback to musicians on how they can improve. The musicians rely on the wearable device to help them practice according to the conductor’s feedback.

Individual Practice