research
Two new AI systems, called ALOHA Unleashed and DemoStart, help robots learn how to perform complex tasks that require delicate movements.
People perform many tasks every day, such as tying shoelaces or tightening screws. However, for robots, it is very difficult to learn these highly dexterous tasks correctly. To make robots more useful in people’s lives, they need to become more adept at interacting with physical objects in dynamic environments.
Today we present two new papers that explore the latest advances in artificial intelligence (AI) in the field of robotic dexterity research. ALOHA Unleashed helps robots learn how to perform complex and novel two-arm manipulation tasks. DemoStart uses simulation to improve the real-world performance of multi-fingered robotic hands.
These systems open the way for robots to perform a wide range of useful tasks by helping them learn from human examples and turn images into actions.
Improving imitation learning using two robotic arms
Until now, most advanced AI robots have been able to pick up and place objects using one arm. In a new paper, we present ALOHA Unleashed, which achieves a high level of dexterity in bilateral manipulation. With this new method, the robot learned to tie shoelaces, hang shirts, repair other robots, insert gears, and even clean the kitchen.
An example of a two-armed robot that straightens shoelaces and ties them with ribbons.
A two-armed robot spreads out a polo shirt on a table, hangs it on a hanger, and then hangs it back on the hanger.
An example of a two-arm robot repairing another robot.
The ALOHA Unleashed approach is built on the ALOHA 2 platform, which is based on the original ALOHA (a low-cost open-source hardware system for dual-hand remote operation) from Stanford University.
ALOHA 2 is much more agile than previous systems, featuring two hands that can be easily teleoperated for training and data collection purposes, allowing the robot to learn how to perform new tasks with fewer demonstrations.
We also improved the ergonomics of the robot hardware and enhanced the learning process in modern systems. First, we collected demo data by remotely manipulating the robot’s behavior by performing difficult tasks such as tying shoelaces and hanging a T-shirt. Next, we applied a diffusion method to predict robot behavior from random noise, similar to how the Imagen model generates images. This allows the robot to learn from the data and perform the same tasks on its own.
Learn robot behavior through some simulation demos.
Controlling a dexterous robotic hand is a complex task, and it becomes even more complex as fingers, joints, and sensors are added. In another new paper, we present DemoStart, which uses reinforcement learning algorithms to help robots learn dexterous behaviors in simulation. These learned behaviors are particularly useful for complex implementations, such as multi-fingered hands.
DemoStart starts learning in easy conditions, and over time starts learning in more difficult conditions to master the task as much as possible. For the same purpose, it requires 100x fewer simulation demos to learn how to solve a task in simulation than is typically required when learning from real-world cases.
The robot achieved a success rate of over 98% in a variety of tasks, including rearranging cubes of a specific color in the simulation, tightening nuts and bolts, and organizing tools. In the real world, it achieved a 97% success rate in rearranging and lifting cubes, and a 64% success rate in a plug-and-socket insertion task that requires high finger coordination and precision.
Examples of a robotic arm that has learned to successfully insert a yellow connector in simulation (left) and real-world conditions (right).
Here’s an example of a robotic arm learning how to tighten a screw in a simulation.
We developed DemoStart together with MuJoCo, an open source physics simulator. After mastering various tasks in simulation and using standard techniques to bridge the gap between simulation and reality, such as domain randomization, our approach was able to translate to the physical world with almost zero shots.
Robot learning in simulation can reduce the cost and time required to run real-world physical experiments. However, designing these simulations is difficult and does not always translate successfully into real-world performance. By combining reinforcement learning with learning from a few demos, DemoStart’s incremental learning automatically generates a curriculum that bridges the gap between simulation and reality, making it easier to transfer knowledge from simulation to physical robots and reducing the cost and time required to run physical experiments.
To further advance robotic learning through intensive experiments, we tested this novel approach on a three-fingered robotic hand called DEX-EE, developed in collaboration with Shadow Robot.
Image of the DEX-EE dexterous robotic hand developed by Shadow Robot in collaboration with the Google DeepMind robotics team (Source: Shadow Robot).
The future of robot dexterity
Robotics is a unique area of ​​AI research that shows how well our approaches work in the real world. For example, large-scale language models can tell you how to tighten a bolt or tie a shoelace, but even when implemented in a robot, they cannot do those tasks on their own.
One day, AI robots will help people do all sorts of things at home, at work, and beyond. Agility research, including the efficient and general learning approach we described today, will help make that future possible.
While we still have a long way to go before robots can grasp and manipulate objects as easily and precisely as people, we are making significant progress, and each breakthrough innovation is another step in the right direction.