Teaching Robots to Dance: The Future of Choreo-Robotics

Teaching robots to dance enhances their emotional intelligence, making them better companions and caregivers in our everyday lives.

### Choreo-Robotics: Choreographing the Future of Intelligent Machines
* 🕺  Choreographers understand the impact of movement on humans, making it essential to teach robots how to move with **intention, agility, and emotional expression**.
* 🧠  AI technologies like GPT and Gemini are giving robots the brains to become **much smarter**, and we are just beginning to explore their potential in the **physical world**.
* 🤖  The emerging field of choreo-robotics combines robotics and dance to create **general-purpose robots** that can be **caregivers, teachers, and companions**.
* 🎭  Dancing with robots is no longer just about art or entertainment; it is about **applying the lessons of how movement changes emotions** and the **importance of contextual movement** to robots in our everyday lives.
* 🏫  At Everyday Robots, an AI moonshot at Google, an AI agent was trained to teach **15 robots how to move together as a flock** in an interactive performance and installation work called Project Starling.
* 🎵  Each joint on the robot in Project Starling was mapped to a different sound, creating a **unique symphony** as the robots moved.
* 🌍  General-purpose robots are coming, and how they move will determine whether we welcome them into our homes and workplaces, shaping the **future of human-robot interaction** in an increasingly AI world.

I have an unusual job. I teach robots how to dance. In 2017, I transitioned from dancing
on stages with humans to dancing with robots. Big robots, small robots, single-arms, and roving robots. Robots that fly, vacuuming robots. A lot of robots that look like humans. And a whole bunch that don't. I'm pretty sure I've choreographed and danced with more
different types of robots than anyone on Earth. Why do I do this? Because robots move, and choreographers understand
how movement affects us. If a robot slides politely
out of a doorway to let you pass, it might make you feel seen and acknowledged. If a robot marches quickly towards you
and avoids you at the last second, it might cause revulsion and fear. Robots are beginning to show up
in our everyday environments, from sidewalks to offices,
backyards to hospitals. And they will be threatening
and confusing to us if we do not carefully
examine how they move.

I'm also a roboticist and an engineer. I study this connection between
robot motion and emotion. I combine my expertise
in robotics and dance in the emerging field of choreo-robotics, the intersection of dance and robotics. Right now, we're giving AI a robot body
with physical intelligence. AI technologies like GPT and Gemini
are becoming the robot's brains, which means that robots
are getting smarter. Much smarter. Think of what AI has already done
to text and video generation. We are just scratching the surface
on seeing what AI can do in the physical world. We are going to be interacting with these new intelligent
robots more and more, interacting with them through movement. Roboticists already spend a lot of time
thinking about movement, but they think about it
in terms of efficiency, optimizing for speed or success. That's not enough. We need to teach robots how to dance. If a robot can dance, make every movement
with intention, agility, balance.

emotional expression and grace. I believe they will also become
better caregivers, teachers, and companions. It will help us build better
general-purpose robots that we actually want to live amongst us. If we teach robots to dance, we expand the future
of intelligent machines. Choreographers and engineers have made
dances with robots since the 1990s, and these highly scripted performances
have captured the public's imagination. They've made us feel wonder, made us laugh, made us want to dance along, and sometimes made us never want
to meet a robot in real life. Dancing with robots is no longer
just about art or entertainment. It is about transferring the lessons
that one, all movement changes emotions, and two, contextual movement
is critically important, and applying those lessons to robots
in our everyday lives. Robotics engineers are beginning
to consider emotion and context when designing intelligent
machines that interact with us. The choices we make now are going to dictate
how we live our lives with robots.

If we get this right, robots will be more welcomed,
safe, and delightful. Humans will be more empowered
and comfortable. Before AI, programmers
needed hours to script a simple dance sequence for robot to perform. Just like they needed hours to script
the robot to open a single door. With AI, you can teach the robot
to open just a few specific doors, and it will learn to open all of them, even ones it hasn't seen before. It's also true for dance. You can teach the robot
to dance with a specific person, and it will learn how to dance and move with many others in many different
environments and circumstances. This is what I did at Everyday Robots, then a robot AI moonshot at Google. But rather than teach one robot, I used AI to teach 15 robots
how to move together as a flock. We imagined a world where you could walk
down a hallway filled with robots, and they would part to make space for you.

like a flock of doves or a crowd
of people on a city street. Where a robot could navigate seamlessly
and even beautifully through a busy, chaotic Times Square. The world we live in is complicated
for robots to understand, so it was impossible to script
or choreograph interactions like these step by step. We needed to use the magic of AI to teach the robots
how to improvise and react. This was a massive challenge
that had never been done before, and the result is an interactive
performance and installation work called Project Starling. This took us two years
of hardcore engineering to complete. We trained an AI agent
on the preferences of a choreographer, me. I chose the most engaging
patterns and sequences. We also taught the robots how to recognize and respond to human gestures. While we were at it, we turned the robots
into musical instruments. Each joint on the robot
was mapped to a different sound. When the robot's torso moved,
triggered a bass sound.

Its hand opening and closing
triggered a bell sound. The robot's actions created
its own unique symphony. This is what Project Starling looks like. Our AI generated movement
and our robot generated music. (Music) (Music ends) (Applause). I wanted it to feel like stepping out
of your normal reality and into a waking dream, one that was so far
from where we stand today. Yet, all the same felt like a world where your kids could
grow up and feel calm, where you could be surrounded by machines
and it felt natural and safe. People were drawn to just wander
amongst these robots. You know, some were a bit tentative at first, but then they began to engage and explore with curiosity and playfulness. People kept coming back. What we learned about gesture, navigation, human interaction, and expression
can be applied to robots more generally. Think of the following future scenario. A robot in a senior living facility.

waiting in a crowded cafe. You sit at your table. You wave over to it,
make a sipping gesture, and the robot understands
that you would like a glass of water. And when it drives over to you,
with its glass of water, it nods at the other people it passes and pauses graciously to let them go by. And when it arrives at your table,
it places the glass gracefully down. And when you thank it,
it nods in acknowledgment. Maybe it even performs a small
celebratory dance in place. We can choose to build and program
robots that we welcome, robots that make us feel understood, robots that we want to spend time with. The promise of robots that will live
and work amongst us has existed for a very long time, from Karel Capek's 1920 play
Rossum's Universal Robots to The Jetsons' Rosey the Robot to C-3PO of Star Wars
and even Terminator.

these stories have been deeply impactful
in what we imagine and build. Yes, we can have robots
that do our chores. But we can also have robots
be our dance partners. Humans do a range of things, ceramics to diplomacy, singing in choirs to vertical farming. Imagine a world where robots
can do a myriad of things that are functional or creative, and everything in between. General-purpose robots are coming, and they will help us to address
big demographic challenges like aging populations
and massive labor shortages. And how they move will unlock whether we let them into our homes and welcome them in our workplaces. Because we want to feel
like we're still marvelously human in this increasingly AI world. And we can, because we have the power
to choreograph the future we want. Thank you. (Applause)

Next Up for AI? Dancing Robots | Catie Cuan | TED

The TED Talks channel features the best talks and performances from the TED Conference, where the world's leading thinkers and doers give the talk of their lives in 18 minutes (or less). Look for talks on Technology, Entertainment and Design -- plus science, business, global issues, the arts and more. You're welcome to link to or embed these videos, forward them to others and share these ideas with people you know.

TED's videos may be used for non-commercial purposes under a Creative Commons License, Attribution–Non Commercial–No Derivatives (or the CC BY – NC – ND 4.0 International) and in accordance with our TED Talks Usage Policy: https://www.ted.com/about/our-organization/our-policies-terms/ted-talks-usage-policy. For more information on using TED for commercial purposes (e.g. employee learning, in a film or online course), please submit a Media Request at https://media-requests.ted.com

Teaching Robots to Dance: The Intersection of Dance and Robotics

Robots' movement can evoke emotions in humans, and studying this connection is the focus of choreo-robotics.

AI technologies are advancing, giving robots physical intelligence and the ability to interact with humans through movement.

The goal is to teach robots to dance with intention, agility, and balance, going beyond just efficiency and speed in their motion.

Teaching robots to dance can improve their emotional expression and interaction with humans.

This integration of AI and dance can potentially shape the future of robotics, making them more adaptable to various environments and social interactions.

A former Google AI project demonstrated the ability to teach multiple robots to move together, showcasing a potential future where robots can coexist with humans in a shared space.

Teaching Robots to Improvise: The Project Starling Story

Robots were taught to navigate chaotic environments like Times Square, culminating in the interactive performance Project Starling.

The AI agent was trained on the choreographer's preferences to create engaging movement patterns, while also turning the robots into musical instruments.

The project aimed to create a world where humans and machines could naturally and safely coexist, with the insights gained being applicable to robots in various settings, such as senior living facilities.

Robots are being designed to not only perform tasks but also to interact with humans in a way that makes them feel understood and welcomed.

The concept of general-purpose robots is emerging to address demographic challenges and labor shortages.

The future of robotics involves creating a world where robots can engage in a wide range of functional and creative activities, potentially becoming our dance partners and addressing the need for human connection in an AI-driven world.