What is imitation learning?
Imitation learning is a machine learning approach in which a robot learns how to perform a task from recordings of a person demonstrating it. Unlike conventional robot teaching, where positions and sequences are entered one by one, learning from demonstrations gives the robot motions that still work when conditions change slightly.
It is also called learning from demonstration (LfD). Imitation learning is one of the main ways to teach motions to robots with physical AI.
Demonstrate, record, learn
Imitation learning moves through these three steps, after which the robot can act on its own. What a person teaches is not a procedure, but the motion itself.
Demonstrate
- A person moves the robot with a leader arm
- Repeated tens to hundreds of times
Record
- Camera images
- Joint angles
- Forces on the gripper
Learn
- An AI model learns "in this situation, it moved like this"
Act on its own
- Decides the next motion from camera images
Where it struggles, add demonstrations and retrain
01Demonstrate
A person operates the robot to show the task. Teleoperation with a leader arm or VR controllers is common, and in some setups the person moves the robot arm directly by hand.
02Record
During each demonstration, camera images, joint angles, and the forces on the gripper are recorded over time. Collecting tens to hundreds of demonstrations per task is common.
03Learn
An AI model learns from the recorded pairs of "in this situation, it moved like this." Once trained, the robot decides its next motion on its own from camera images and other inputs.
Main methods
Methods differ in how they learn from demonstrations.
01Behavior cloning
The most basic method: it learns situation-action pairs directly as the correct answers. It is simple and easy to use, but once the robot drifts into situations the demonstrations never covered, small errors tend to compound.
02Human corrections during training
When the robot in training gets stuck, a person adds the correct action for that situation, which keeps errors from compounding. DAgger (Dataset Aggregation) is the best-known example.
03Inverse reinforcement learning
Infers what the demonstrator is trying to achieve, the reward, and learns motions that serve that goal. It copies the goal rather than the motion itself.
04Learning motions in chunks
Recent methods have become widespread: ACT (Action Chunking with Transformers) predicts several steps of motion at once, and Diffusion Policy generates motions with a diffusion model. Both can learn smooth motions from relatively few demonstrations.
How it compares with other ways to teach robots
There are three main ways to teach a robot a motion. In practice, they are often combined depending on the task.
| Compared | Robot teaching | Reinforcement learning | Imitation learning |
|---|---|---|---|
| How it is taught | Positions and sequences are entered one by one | The robot finds its own way through trial, error, and rewards | A person demonstrates |
| What it needs | Time and effort to program motions | Large numbers of trials, usually in simulation | Demonstration data |
| When conditions change | Must be re-taught | Adapts within what it has learned | Adapts within what was demonstrated |
| Best suited for | Exactly the same motion, repeated | Motions like walking and balancing, where the right answer is hard to put into words | Tasks that are easy for people but hard to write down as steps |
Frequently asked questions
- How many demonstrations does imitation learning need?
- It depends on the task, but tens to hundreds of demonstrations per task are common. Building on a robot foundation model (VLA) pretrained on large amounts of task data is expected to reduce how many demonstrations a new task needs.
- What are the weaknesses of imitation learning?
- It struggles with situations the demonstrations did not cover. Small errors at each step can compound until the robot drifts outside the range of the demonstrations. Common remedies are to vary the conditions while collecting demonstrations, to have a person add corrections during training, and to generate extra demonstrations in simulation.
- Can imitation learning be combined with reinforcement learning?
- Yes. Learning rough motions by imitation and then refining them with reinforcement learning has been widely studied. AlphaGo, the Go program that first learned from human game records and then improved through self-play, followed this pattern. It usually needs fewer trials than learning from scratch, but performance can temporarily drop right after switching to reinforcement learning.
- What manufacturing tasks is imitation learning suited for?
- Tasks that are easy for people but hard to write down as steps: loading and unloading parts whose orientation or position varies slightly each time, handling objects without a fixed shape, and routing cables. If the motion is exactly the same every time, conventional robot teaching or minimal AI is faster and more reliable.
- How does imitation learning relate to VLAs?
- A VLA (vision-language-action model) is a large model pretrained on demonstration data from many robots, and imitation learning is at the core of that training. When a VLA is adapted to a specific task (fine-tuning), demonstration data for that task is used as well.
Related terms
- What is physical AI?
What physical AI is, how it works, and how it differs from generative AI and industrial robots
References
- DAgger (human corrections during training)Ross et al. (2011). A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. AISTATS
- Inverse reinforcement learningNg, Russell (2000). Algorithms for Inverse Reinforcement Learning. ICML
- ACT (learning motions in chunks)Zhao et al. (2023). Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. RSS
- Diffusion PolicyChi et al. (2023). Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. RSS
- AlphaGo (learning from human games, then refining with reinforcement learning)Silver et al. (2016). Mastering the game of Go with deep neural networks and tree search. Nature 529
- Reinforcement learning for a robot hand, aided by human demonstrationsRajeswaran et al. (2018). Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations. RSS
- The performance drop right after switching to reinforcement learning, and a remedyNair et al. (2020). AWAC: Accelerating Online Reinforcement Learning with Offline Datasets
- VLAs (robot foundation models)Brohan et al. (2023). RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. CoRL
- VLAs (an open-source model)Kim et al. (2024). OpenVLA: An Open-Source Vision-Language-Action Model. CoRL