Multimodal AI companion for Dance Practice: Reimagining Dance Practice using Real Time AI feedback
Product Designer · 2024
Multimodal AI companion for Dance Practice: Reimagining Dance Practice using Real Time AI feedback
Product Designer · 2024
Multimodal AI companion for Dance Practice: Reimagining Dance Practice using Real Time AI feedback
Product Designer · 2024
TL;DR
Natya.AI, is an AI companion for dance practice.
I crafted the vision for an AI-powered practice companion for dancers, making long-distance learning affordable and accessible.
TL;DR
Natya.AI, is an AI companion for dance practice.
I crafted the vision for an AI-powered practice companion for dancers, making long-distance learning affordable and accessible.
Context
Bharatnatyam is a 2500 year old Indian classical danceform with a highly developed codification of movements.
Bharatnatyam is one of the seven classical dance styles originating in India, tracing its roots to 5000 BC, making it one of the oldest art forms in the world!
Like all Indian classical dance forms, it is based on the Natya Shastra, a 2500 year-old treaty on dramatics, which gives a very precise and highly developed codification of dance, music and theater.
1/7
Indian Classical Dance Form
2500
Year Old Dance Form
Context
Bharatnatyam is a 2500 year old Indian classical danceform with a highly developed codification of movements.
Bharatnatyam is one of the seven classical dance styles originating in India, tracing its roots to 5000 BC, making it one of the oldest art forms in the world!
Like all Indian classical dance forms, it is based on the Natya Shastra, a 2500 year-old treaty on dramatics, which gives a very precise and highly developed codification of dance, music and theater.
1/7
Indian Classical Dance Form
2500
Year Old Dance Form
Challenge
Bharatnatyam Dancers record themselves while practicing to visualize their movements in space. However, recording & reviewing is cumbersome, time consuming, and breaks the flow of practice.
97%
dancers record themselves while practicing.
They face 2 main challenges:
No live feedback
Dancer’s miss their guru’s live prompting.
Review Fatigue
Reviewing recordings to find areas of improvement is very time consuming.
Challenge
Bharatnatyam Dancers record themselves while practicing to visualize their movements in space. However, recording & reviewing is cumbersome, time consuming, and breaks the flow of practice.
97%
dancers record themselves while practicing.
They face 2 main challenges:
No live feedback
Dancer’s miss their guru’s live prompting.
Review Fatigue
Reviewing recordings to find areas of improvement is very time consuming.
Hypothesis
What if Bharatnatyam dancers could be live prompted, mimicking the in-class experience
providing actionable feedback on posture & timing.
making the recording process plug and play.
decreasing interruptions and reducing time spent reviewing by at least 50%.
Hypothesis
What if Bharatnatyam dancers could be live prompted, mimicking the in-class experience
providing actionable feedback on posture & timing.
making the recording process plug and play.
decreasing interruptions and reducing time spent reviewing by at least 50%.
MEET NATYA.AI
Natya.AI is an app for Bharatnatyam dancers that enhances practice time by providing real time voice and visual feedback on your live practice recording using AI vision and data.
Choose you settings
Open the app.
Connect Music.
Select your Controls
Begin!
Get live feedback during practice
The yellow overlay, maps your body movement, while the red one is telling the correct way of doing something.
Reduce finding fatigue
This interface makes reviewing hassle free. It splits the entire video into short snippets of good and bad.
Splits allow dancers to find just the perfect length of video for review.
Review at 1x, 2x or 1/2x
This allows dancers to review faster and practice along.
Movement Library
Auto captures and classifies unique micro sequences.
Actionable Feedback
The dual focus on providing visual feedback using color and listing out the moments that need practice, makes the summary very actionable.
Compare Movements
As a dancer, see how your movements have improved over time.
Process
I explored techniques and methods around design ethnography and participatory design.
Through this collaborative process I aimed to democratise design: working together with the dance community and local dance meetups, affecting the project with creative and critical input.
I strongly believe that real impact is made when we, as designers, strengthen peoples capacity to contribute.
17
Subject Matter Experts
300
Dancers Surveyed
3
Participatory Sessions
2
Role Play Sessions
Ethnographic Research
Getting out and ‘snowballing’: talking with dancers and SMEs, sharing stories and pulling threads – all without agenda.
Co Creation/ Participatory Workshops
Engaging users and stakeholders in research materials, co-creating to learn from their years of experience.
Prototype Testing
First with sketches to get feedback on UI and usability and then with a developed prototype for concept.
Role Play
Acting out scenarios to qiuckly reach the human, emotional level of how they want to interact with the prototypes.
Process
I explored techniques and methods around design ethnography and participatory design.
Through this collaborative process I aimed to democratise design: working together with the dance community and local dance meetups, affecting the project with creative and critical input.
I strongly believe that real impact is made when we, as designers, strengthen peoples capacity to contribute.
17
Subject Matter Experts
300
Dancers Surveyed
3
Participatory Sessions
2
Role Play Sessions
Ethnographic Research
Getting out and ‘snowballing’: talking with dancers and SMEs, sharing stories and pulling threads – all without agenda.
Co Creation/ Participatory Workshops
Engaging users and stakeholders in research materials, co-creating to learn from their years of experience.
Prototype Testing
First with sketches to get feedback on UI and usability and then with a developed prototype for concept.
Role Play
Acting out scenarios to qiuckly reach the human, emotional level of how they want to interact with the prototypes.
Impact
92% of Bharatnatyam dancers surveyed would use the Natya.AI practice companion for their regular practice time.
87% of Bharatnatyam dancers surveyed felt that Natya.AI would greatly enhance their practice time and reduce interruptions and review fatigue by 50%.
“Can you bring this to life...”
- Jeevika Bhatt, performer & choreographer
“This should’ve been built 10 years ago, thanks for thinking of us Bharatnatyam dancers...”
- Sri Thina, teacher & performer
Impact
92% of Bharatnatyam dancers surveyed would use the Natya.AI practice companion for their regular practice time.
87% of Bharatnatyam dancers surveyed felt that Natya.AI would greatly enhance their practice time and reduce interruptions and review fatigue by 50%.
“Can you bring this to life...”
- Jeevika Bhatt, performer & choreographer
“This should’ve been built 10 years ago, thanks for thinking of us Bharatnatyam dancers...”
- Sri Thina, teacher & performer
Retrospective & Next Steps
Co-Creation and Roleplay are great in understanding user mental models.
Co-Creation and Roleplay are great in understanding the mental models and the deeper reasoning for the ‘Why’ and how for the users. They may not be able to tell but co-creation helps them think and make!
Hand drawn sketches were great for feedback on the usability and high fidelity prototypes were great for concept testing. For this concept, there were gaps in a user being able to understand the concept without seeing a high fidelity prototype.
Role playing with a dance teacher to get feedback on voice commands was great. This not only helped me kink out the voice based scenarios but also able to get feedback on the type of feedback helpful for dancers.
Working with the users helped me understand and further innovate the review recordings splits view.
I’m currently speaking with engineers who work with the AI to train an AI model to detect a few basic steps of Bharatnatyam and see where we can take this.
Retrospective & Next Steps
Co-Creation and Roleplay are great in understanding user mental models.
Co-Creation and Roleplay are great in understanding the mental models and the deeper reasoning for the ‘Why’ and how for the users. They may not be able to tell but co-creation helps them think and make!
Hand drawn sketches were great for feedback on the usability and high fidelity prototypes were great for concept testing. For this concept, there were gaps in a user being able to understand the concept without seeing a high fidelity prototype.
Role playing with a dance teacher to get feedback on voice commands was great. This not only helped me kink out the voice based scenarios but also able to get feedback on the type of feedback helpful for dancers.
Working with the users helped me understand and further innovate the review recordings splits view.
I’m currently speaking with engineers who work with the AI to train an AI model to detect a few basic steps of Bharatnatyam and see where we can take this.