0% found this document useful (0 votes)

40 views76 pages

Robot Learning Tutorial Guide

Q: In what ways does the Action Chunking with Transformers (ACT) method improve learning in robotic tasks?

The Action Chunking with Transformers (ACT) method improves learning in robotic tasks by utilizing transformers to process sequences of actions as chunks, thereby minimizing the compounding of errors during prediction . ACT models expect a set number of actions as targets, allowing the model to learn coherent sequences that adapt over different tasks, reducing the likelihood of inconsistent actions . Furthermore, ACT's use of attention mechanisms helps to integrate context across sequences, enhancing the robustness of actions undertaken during robot operations .

Q: How do Behavior Cloning (BC) techniques complement Reinforcement Learning (RL) in robotics?

Behavior Cloning (BC) techniques complement Reinforcement Learning (RL) by providing a framework to learn from previously recorded trajectories, thereby reducing exploration in real environments, which can be erratic and dangerous . BC is particularly useful in scenarios where reward functions are sparse or complex, as it allows leveraging human-provided demonstrations to guide learning. Additionally, BC methods serve as a foundation for generalist policies, which can then be integrated with RL approaches to refine and improve task performance in various robotic applications .

Q: What are the main limitations of Reinforcement Learning (RL) when applied to real-world robotics tasks?

Reinforcement Learning (RL) faces several limitations in real-world robotics applications. One primary challenge is the general unavailability of complex tasks' dense reward functions, which require human expertise and trial-and-error to develop, making tasks with sparse rewards more challenging to learn . Additionally, simulating high-fidelity environments is often computationally constrained, especially for contact-rich manipulations . Moreover, RL requires numerous interactions with the environment, which is time-consuming and resource-intensive when working with physical robots. Sample-efficient methods are necessary to overcome these challenges, as real-world training is inherently time-bottlenecked .

Q: How does π0 leverage modern VLA techniques to enhance robotic control?

π0 leverages modern Vision-Language-Action (VLA) techniques by using unified transformer models with separate sub-components for perception and action, facilitating cross-task language conditioning and enhancing control accuracy . It uses a mixture of experts (MoE) approach for efficient computation while maintaining specialized sub-components for action prediction. This structural design allows π0 to execute actions more efficiently across various tasks, leveraging visual-semantic understanding to predict and perform complex actions autonomously .

Q: What advantages do decentralized datasets offer for robotic learning, and how do they differ from centralized datasets?

Decentralized datasets offer the advantage of diversity, as they encompass data collected from various sources and environments, enhancing the generalization capabilities of models . Unlike centralized datasets which are typically gathered through coordinated efforts across institutions, decentralized datasets allow individual researchers to contribute, often filling gaps in less-explored or unique applications. This can lead to broader benchmarking opportunities and more tailored training sets for specific robotic constraints or tasks .

Q: What role does entropy play in the DORAEMON method proposed by Tiboni et al., 2024?

The DORAEMON method proposed by Tiboni et al., involves explicitly maximizing the entropy of a parametric Beta distribution during training, which allows for more flexibility compared to a uniform distribution . By using an outer-loop max-entropy objective, DORAEMON effectively increases the entropy levels of the training distribution dynamically, facilitating enhanced exploration and robustness of policies developed within the constraints of the inner-loop RL problem . This approach aims to address challenges in domain randomization by aligning training and real-world task conditions as closely as possible .

Q: How are self-supervised Vision-Language Models (VLMs) integrated into Vision-Language-Action (VLA) architectures?

Self-supervised Vision-Language Models (VLMs) are integrated into Vision-Language-Action (VLA) architectures by serving as the backbone for visual-semantic understanding . They feed visual and language tokens into a joint attention mechanism that allows for shared information processing with action-specific modules. This architecture enables efficient cross-modal learning, improving the ability to interpret both visual and linguistic inputs simultaneously, which enhances the model's generalization across diverse robotic tasks . This integration also leverages disjoint sets of weights for perception and action, optimizing the model's performance in resource-constrained environments .

Q: Why is it challenging to design reward functions for complex robotic tasks, and how do sparse rewards affect learning?

Designing reward functions for complex robotic tasks is challenging because it often requires intricate human expertise and significant trial-and-error . Dense rewards are difficult to craft but provide clear guidance during training. Conversely, sparse rewards, while simpler, only inform whether a specific goal is achieved without detailing the path to success, making learning more difficult as agents receive less frequent feedback, leading them to exploration inefficiency and potentially hindered performance .

Q: How do Vision-Language-Action (VLA) models enhance robot learning across different tasks and environments?

Vision-Language-Action (VLA) models enhance robot learning by integrating vision, language, and action modules within a unified framework, leveraging pre-trained Vision-Language Models (VLMs) for visual perception and action prediction . They employ action chunking and mixture of experts (MoE) architectures to handle multimodal data distributions effectively, improving cross-task performance and enabling few-shot adaptation to novel tasks . This allows robots to generalize across different tasks and environments while benefitting from compute-efficient modular architectures that can handle both perception and action planning .

Q: How do Soft Actor-Critic (SAC) algorithms improve sample efficiency compared to on-policy methods in RL?

Soft Actor-Critic (SAC) algorithms improve sample efficiency by using a replay buffer that allows past experiences to be reused for training, thus reducing the number of interactions needed with the environment . In contrast, on-policy methods discard past data after a few updates, requiring more environment interactions to learn effectively. SAC’s replay buffer also provides flexibility to include expert demonstrations, facilitating learning under data-scarce conditions and improving convergence speed in real-world applications .

Uploaded by

hj7.com

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

40 views76 pages

Robot Learning Tutorial Guide

Uploaded by

hj7.com

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

Robot Learning: A Tutorial

Francesco Capuano Caroline Pascal Adil Zouitine Thomas Wolf Michel Aractingi

University of Oxford, Hugging Face

Abstract
Robot learning is at an inflection point, driven by rapid advancements in machine learning and the growing
arXiv:2510.12403v1 [[Link]] 14 Oct 2025

availability of large-scale robotics data. This shift from classical, model-based methods to data-driven,
learning-based paradigms is unlocking unprecedented capabilities in autonomous systems. This tutorial
navigates the landscape of modern robot learning, charting a course from the foundational principles of
Reinforcement Learning and Behavioral Cloning to generalist, language-conditioned models capable of
operating across diverse tasks and even robot embodiments. This work is intended as a guide for researchers
and practitioners, and our goal is to equip the reader with the conceptual understanding and practical tools
necessary to contribute to developments in robot learning, with ready-to-use examples implemented in lerobot.

Code: [Link]
Date: October 15, 2025

Contents

1 Introduction 3
1.1 LeRobotDataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1.1 The dataset class design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Code Example: Batching a (Streaming) Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Code Example: Collecting Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Classical Robotics 9
2.1 Explicit and Implicit Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Different Types of Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3 Example: Planar Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3.1 Adding Feedback Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4 Limitations of Dynamics-based Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3 Robot (Reinforcement) Learning 16

3.1 A (Concise) Introduction to RL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2 Real-world RL for Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2.1 Code Example: Real-world RL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.2 Limitations of RL in Real-World Robotics: Simulators and Reward Design . . . . . . . . . . . 32

4 Robot (Imitation) Learning 33

4.1 A (Concise) Introduction to Generative Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.1.1 Variational Auto-Encoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.1.2 Diffusion Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.1.3 Flow Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.2 Action Chunking with Transformers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.2.1 Code Example: Training and Using ACT in Practice . . . . . . . . . . . . . . . . . . . . . . . . 46
4.3 Diffusion Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.3.1 Code Example: Training and Using Diffusion Policies in Practice . . . . . . . . . . . . . . . . 50

1
4.4 Optimized Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.4.1 Code Example: Using Async Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

5 Generalist Robot Policies 57

5.1 Preliminaries: Models and Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5.2 VLAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.2.1 VLMs for VLAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.3 π0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.3.1 Code Example: Using π0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.4 SmolVLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.4.1 Code Example: Using SmolVLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

6 Conclusions 67

Foreword
Robotics is an inherently multidisciplinary field, which is witnessing unprecedented advancements since its inception
in the 1960s. Yet, more than sixty years after the debut of Unimate, robots have still not fully integrated into the rich,
unstructured, and dynamic world we humans inhabit. Over the decades, numerous disciplines have shown immense
promise in tackling the challenges of creating autonomous robotic systems. This tutorial takes a clear stance in the
debate on whether modern Machine Learning can play a pivotal role in the development of autonomous robots: we
believe this to be the case.
Nonetheless, we also hold that the wealth of research from both academia and industry in classical robotics over the
past six decades is, simply put, too valuable to be cast aside in favor of purely learning-based methods. However,
the interplay between classical robotics and modern machine learning is still in its nascent stages, and the path to
integration yet to be clearly defined. In turn our goal here is to present what we consider to be the most relevant
approaches within robot learning today, while warmly extending an invite to collaborate to expand the breadth of
this work! Start contributing today here.
This tutorial. . .

• Does not aim to be a comprehensive guide to general field of robotics, manipulation or underactuated systems: Si-
ciliano and Khatib (2016) and Tedrake (a,b) do this better than we ever could.
• Does not aim to be an introduction to statistical or deep learning: Shalev-Shwartz and Ben-David (2014) and Prince
(2023) cover these subjects better than we ever could.
• Does not aim to be a deep dive into Reinforcement Learning, Diffusion Models, or Flow Matching: invaluable works
such as Sutton and Barto (2018), Nakkiran et al. (2024), and Lipman et al. (2024) do this better than we ever
could.

Instead, our goal here is to provide an intuitive explanation as per why these disparate ideas have converged to form
the exciting field of modern robot learning, driving the unprecedented progress we see today. In this spirit, we follow
the adage: "a jack of all trades is a master of none, but oftentimes better than a master of one."
We sincerely hope this tutorial serves as a valuable starting point for your journey into robot learning.

2
Figure 1 | lerobot is the open-source library for end-to-end robotics developed by Hugging Face. The library is vertically
integrated on the entire robotics stack, supporting low-level control of real-world robot devices, advanced data and inference
optimizations, as well as SOTA robot learning methods with simple implementations in pure Pytorch.

1 Introduction
Autonomous robotics holds the premise of relieving humans from repetitive, tiring or dangerous manual tasks.
Consequently, the field of robotics has been widely studied since its first inception in the 1950s. Lately, advancements
in Machine Learning (ML) have sparked the development of a relatively new class of methods used to tackle robotics
problems, leveraging large amounts of data and computation rather than human expertise and modeling skills to
develop autonomous systems.
The frontier of robotics research is indeed increasingly moving away from classical model-based control paradigm,
embracing the advancements made in ML, aiming to unlock (1) monolithic perception-to-action control pipelines
and (2) multi-modal data-driven feature extraction strategies, together with (3) reduced reliance on precise models
of the world and (4) a better positioning to benefit from the growing availability of open robotics data. While
central problems in manipulation, locomotion and whole-body control demand knowledge of rigid-body dynamics,
contact modeling, planning under uncertainty, recent results seem to indicate learning can prove just as effective as
explicit modeling, sparking interest in the field of robot learning. This interest can be largely justified considering the
significant challenges related to deriving accurate models of robot-environment interactions.
Moreover, since end-to-end learning on ever-growing collections of text and image data has historically been at the
core of the development of foundation models capable of semantic reasoning across multiple modalities (images, text,
audio, etc.), deriving robotics methods grounded in learning appears particularly consequential, especially as the
number of openly available datasets continues to grow.
Robotics is, at its core, an inherently multidisciplinary field, requiring a wide range of expertise in both software and
hardware. The integration of learning-based techniques further broadens this spectrum of skills, raising the bar for
both research and practical applications. lerobot is an open-source library designed to integrate end-to-end with
the entire robotics stack. With a strong focus on accessible, real-world robots (1) lerobot supports many, openly
available, robotic platforms for manipulation, locomotion and even whole-body control. lerobotalso implements a
(2) unified, low-level approach to reading/writing robot configurations to extend support for other robot platforms
with relatively low effort. The library introduces LeRobotDataset, (3) a native robotics dataset’s format currently
being used by the community to efficiently record and share datasets. lerobot also supports many state-of-the-art
(SOTA) algorithms in robot learning—mainly based on Reinforcement Learning (RL) and Behavioral Cloning (BC)
techniques—with efficient implementations in Pytorch, and extended support to experimentation and experiments
tracking. Lastly, lerobot defines a custom, optimized inference stack for robotic policies decoupling action planning
from action execution, proving effective in guaranteeing more adaptability at runtime.
This tutorial serves the double purpose of providing useful references for the Science behind—and practical use
of—common robot learning techniques. To this aim, we strike to provide a rigorous yet concise overview of the core
concepts behind the techniques presented, paired with practical examples of how to use such techniques concretely,
with code examples in lerobot, for researchers and practitioners interested in the field of robot learning. This tutorial
is structured as follows:

• Section 2 reviews classical robotics foundations, introducing the limitations of dynamics-based approaches to
robotics.

3
• Section 3 elaborates on the limitations of dynamics-based methods, and introduce RL as a practical approach to
solve robotics problems, considering its upsides and potential limitations.
• Section 4 further describes robot learning techniques that aim at solving single-tasks learning, leveraging BC
techniques to autonomously reproduce specific expert demonstrations.
• Section 5 presents recent contributions on developing generalist models for robotics applications, by learning from
large corpora of multi-task & multi-robot data (robotics foundation models).

Our goal with this tutorial is to provide an intuitive explanation of the reasons various disparate ideas from Machine
Learning (ML) have converged and are powering the current evolution of Robotics, driving the unprecedented progress
we see today. We complement our presentation of the most common and recent approaches in robot learning with
practical code implementations using lerobot, and start here by presenting the dataset format introduced with
lerobot.

1.1 LeRobotDataset
LeRobotDataset is one of the most impactful features of lerobot, developed in keeping with the observation that
robotics data is increasingly central in robot learning. Thus, lerobot defines a standardized dataset format designed to
address the specific needs of robot learning research, providing a unified and convenient access to robotics data across
modalities, including sensorimotor readings, multiple camera feeds and teleoperation status. LeRobotDataset also
accommodates for storing general information regarding the data being collected, including textual descriptions of the
task being performed by the teleoperator, the kind of robot used, and relevant measurement specifics like the frames
per second at which the recording of both image and robot state’s streams are proceeding.
In this, LeRobotDataset provides a unified interface for handling multi-modal, time-series data, and it is designed to
seamlessly integrate with the PyTorch and Hugging Face ecosystems. LeRobotDataset can be easily extended by
users and it is highly customizable by users, and it already supports openly available data coming from a variety of
embodiments supported in lerobot, ranging from manipulator platforms like the SO-100 arm and ALOHA-2 setup,
to real-world humanoid arm and hands, as well as entirely simulation-based datasets, and self-driving cars. This
dataset format is built to be both efficient for training and flexible enough to accommodate the diverse data types
encountered in robotics, while promoting reproducibility and ease of use for users.

1.1.1 The dataset class design

A core design choice behind LeRobotDataset is separating the underlying data storage from the user-facing API.
This allows for efficient storage while presenting the data in an intuitive, ready-to-use format.
Datasets are always organized into three main components:

• Tabular Data: Low-dimensional, high-frequency data such as joint states, and actions are stored in efficient memory-
mapped files, and typically offloaded to the more mature datasets library by Hugging Face, providing fast with
limited memory consumption.
• Visual Data: To handle large volumes of camera data, frames are concatenated and encoded into MP4 files. Frames
from the same episode are always grouped together into the same video, and multiple videos are grouped together
by camera. To reduce stress on the file system, groups of videos for the same camera view are also broke into
multiple sub-directories, after a given threshold number.
• Metadata A collection of JSON files which describes the dataset’s structure in terms of its metadata, serving as the
relational counterpart to both the tabular and visual dimensions of data. Metadata include the different feature
schema, frame rates, normalization statistics, and episode boundaries.

For scalability, and to support datasets with potentially millions of trajectories (resulting in hundreds of millions
or billions of individual camera frames), we merge data from different episodes into the same high-level structure.
Concretely, this means that any given tabular collection and video will not typically contain information about one
episode only, but rather a concatenation of the information available in multiple episodes. This keeps the pressure on
the file system limited, both locally and on remote storage providers like Hugging Face, though at the expense of
leveraging more heavily relational-like, metadata parts of the dataset, which are used to reconstruct information such
as at which position, in a given file, an episode starts or ends. An example struture for a given LeRobotDataset would
appear as follows:

• meta/[Link]: This metadata is a central metadata file. It contains the complete dataset schema, defining all

4
features (e.g., [Link], action), their shapes, and data types. It also stores crucial information like
the dataset’s frames-per-second (fps), lerobot’s version at the time of capture, and the path templates used to
locate data and video files.
• meta/[Link]: This file stores aggregated statistics (mean, std, min, max) for each feature across the entire
dataset, used for data normalization for most policy models and accessible externally via [Link].
• meta/[Link]: This file contains the mapping from natural language task descriptions to integer task indices,
which are useful for task-conditioned policy training.
• meta/episodes/* This directory contains metadata about each individual episode, such as its length, the corre-
sponding task, and pointers to where its data is stored in the dataset’s files. For scalability, this information is
stored in files rather than a single large JSON file.
• data/*: Contains the core frame-by-frame tabular data, using parquet files to allow for fast, memory-mapped
access. To improve performance and handle large datasets, data from multiple episodes are concatenated into larger
files. These files are organized into chunked subdirectories to keep the size of directories manageable. A single file
typically contains data for more than one single episode.
• videos/*: Contains the MP4 video files for all visual observation streams. Similar to the data/ directory, the
video footage from multiple episodes is concatenated into single MP4 files. This strategy significantly reduces the
number of files in the dataset, which is more efficient for modern filesystems.

1.2 Code Example: Batching a (Streaming) Dataset

This section provides an overview of how to access datasets hosted on Hugging Face using the LeRobotDataset class.
Every dataset on the Hugging Face Hub containing the three main pillars presented above (Tabular, Visual and
relational Metadata), and can be assessed with a single instruction.
In practice, most reinforcement learning (RL) and behavioral cloning (BC) algorithms tend to operate on stack of
observation and actions. For the sake of brevity, we will refer to joint spaces, and camera frames with the single term
of frame. For instance, RL algorithms may use a history of previous frames ot−Ho :t to mitigate partial observability,
and BC algorithms are in practice trained to regress chunks of multiple actions (at+t+Ha ) rather than single controls.
To accommodate for these specifics of robot learning training, LeRobotDataset provides a native windowing operation,
whereby users can define the seconds of a given window (before and after) around any given frame, by using the
delta_timestemps functionality. Unavailable frames are opportunely padded, and a padding mask is also returned
to filter out the padded frames. Notably, this all happens within the LeRobotDataset, and is entirely transparent to
higher level wrappers commonly used in training ML models such as [Link].
Conveniently, by using LeRobotDataset with a Pytorch DataLoader one can automatically collate the individual
sample dictionaries from the dataset into a single dictionary of batched tensors for downstream training or inference.
LeRobotDataset also natively supports streaming mode for datasets. Users can stream data of a large dataset
hosted on the Hugging Face Hub, with a one-line change in their implementation. Streaming datasets supports
high-performance batch processing (ca. 80-100 it/s, varying on connectivity) and high levels of frames randomization,
key features for practical BC algorithms which otherwise may be slow or operating on highly non-i.i.d. data. This
feature is designed to improve on accessibility so that large datasets can be processed by users without requiring large
amounts of memory and storage.

Code 1: Batching a (Streaming) Dataset

[Link]

1 import torch
2 from lerobot . datasets . l er o bo t_ d at as e t import LeRobo tDatas et
3 from lerobot . datasets . s t r e a m i n g _ d a t a s e t import S t r e a m i n g L e R o b o t D a t a s e t
4
5 delta_timestamps = {
6 # 0.2 , and 0.1 seconds * before * each frame
7 " observation . images . wrist_camera " : [ -0.2 , -0.1 , 0.0]
8 }
9
10 # Optionally , use S t r e a m i n g L e R o b o t D a t a s e t to avoid downloading the dataset
11 dataset = LeRob otData set (

5
12 " lerobot / s v l a _ s o 1 0 1 _ p i c k p l a c e " ,
13 d e l t a _ t im e s t a m p s = d e l t a _ t i m e s t a m p s
14 )
15
16 # Streams frames from the Hugging Face Hub without loading into memory
17 streaming_dataset = StreamingLeRobotDataset (
18 " lerobot / s v l a _ s o 1 0 1 _ p i c k p l a c e " ,
19 d e l t a _ t im e s t a m p s = d e l t a _ t i m e s t a m p s
20 )
21
22 # Get the 100 th frame in the dataset by
23 sample = dataset [100]
24 print ( sample )
25 # {
26 # ' observation . state ': tensor ([...]) ,
27 # ' action ': tensor ([...]) ,
28 # ' observation . images . wrist_camera ': tensor ([3 , C , H , W ]) , for delta timesteps
29 # ...
30 # }
31
32 batch_size =16
33 # wrap the dataset in a DataLoader to use process it batches for training purposes
34 data_loader = torch . utils . data . DataLoader (
35 dataset ,
36 batch_size = batch_size
37 )
38
39 # Iterate over the DataLoader in a training loop
40 num_epochs = 1
41 device = " cuda " if torch . cuda . is_available () else " cpu "
42
43 for epoch in range ( num_epochs ):
44 for batch in data_loader :
45 # Move data to the appropriate device ( e . g . , GPU )
46 observations = batch [ " observation . state " ]. to ( device )
47 actions = batch [ " action " ]. to ( device )
48 images = batch [ " observation . images . wrist_camera " ]. to ( device )
49
50 # Next , you can do amazing_model . forward ( batch )
51 ...

1.3 Code Example: Collecting Data

Code 2: Record a Dataset

[Link]

1 """
2 You can also use the CLI to record data . To see the required arguments , run :
3 lerobot - record -- help
4 """
5 from lerobot . cameras . opencv . c o n f i g u r a t i o n _ o p e n c v import O p e n C V C a m e r a C o n f i g
6 from lerobot . datasets . l er o bo t_ d at as e t import LeRobo tDatas et
7 from lerobot . datasets . utils import h w _ t o _ d a t a s e t _ f e a t u r e s
8 from lerobot . robots . so 100_fo llower import SO100Follower , S O 1 0 0 F o l l o w e r C o n f i g
9 from lerobot . teleoperators . so100_leader . c o n f i g _ s o 1 0 0 _ l e a d e r import S O 1 0 0 L e a d e r C o n f i g
10 from lerobot . teleoperators . so100_leader . so100_leader import SO100Leader
11 from lerobot . utils . control_utils import i n i t _ k e y b o a r d _ l i s t e n e r
12 from lerobot . utils . utils import log_say
13 from lerobot . utils . v i s u a l i z a t i o n _ u t i l s import init_rerun
14 from lerobot . scripts . l erobot _recor d import record_loop
15
16 NUM_EPISODES = 5
17 FPS = 30
18 E P I S O D E _ T I M E _ S E C = 60
19 RES ET_TIM E_SEC = 10

6
20 T A S K _ D E S C R I P T I O N = ... # provide a task description
21
22 HF_USER = ... # provide your Hugging Face username
23
24 follower_port = ... # find your ports running : lerobot - find - port
25 leader_port = ...
26 follower_id = ... # to load the calibration file
27 leader_id = ...
28
29 # Create the robot and teleoperator confi gurati ons
30 camera_config = { " front " : O p e n C V C a m e r a C o n f i g (
31 index_or_path =0 , width =640 , height =480 , fps = FPS )
32 }
33 robot_config = S O 1 0 0 F o l l o w e r C o n f i g (
34 port = follower_port ,
35 id = follower_id ,
36 cameras = camera_config
37 )
38 teleop_config = S O 1 0 0 L e a d e r C o n f i g (
39 port = leader_port ,
40 id = leader_id
41 )
42
43 # Initialize the robot and teleoperator
44 robot = SO100Follower ( robot_config )
45 teleop = SO100Leader ( teleop_config )
46
47 # Configure the dataset features
48 ac ti o n_ fe a tu re s = h w _ t o _ d a t a s e t _ f e a t u r e s ( robot . action_features , " action " )
49 obs_features = h w _ t o _ d a t a s e t _ f e a t u r e s ( robot . observation_features , " observation " )
50 d a t a s e t _ f e a t u r e s = {** action_features , ** obs_features }
51
52 # Create the dataset where to store the data
53 dataset = LeRob otData set . create (
54 repo_id = f " { HF_USER }/ robot - learning - tutorial - data " ,
55 fps = FPS ,
56 features = dataset_features ,
57 robot_type = robot . name ,
58 use_videos = True ,
59 i m a g e _ w r i t e r _ t h r e a d s =4 ,
60 )
61
62 # Initialize the keyboard listener and rerun visualization
63 _ , events = i n i t _ k e y b o a r d _ l i s t e n e r ()
64 init_rerun ( session_name = " recording " )
65
66 # Connect the robot and teleoperator
67 robot . connect ()
68 teleop . connect ()
69
70 episode_idx = 0
71 while episode_idx < NUM_EPISODES and not events [ " stop _recor ding " ]:
72 log_say ( f " Recording episode { episode_idx + 1} of { NUM_EPISODES } " )
73
74 record_loop (
75 robot = robot ,
76 events = events ,
77 fps = FPS ,
78 teleop = teleop ,
79 dataset = dataset ,
80 control_ti me_s = EPISODE_TIME_SEC ,
81 single_task = TASK_DESCRIPTION ,
82 display_data = True ,
83 )
84
85 # Reset the environment if not stopping or re - recording
86 if ( not events [ " stop_ record ing " ]) and \
87 ( episode_idx < NUM_EPISODES - 1 or events [ " r e r e c o r d _ e p i s o d e " ]):
88 log_say ( " Reset the environment " )
89 record_loop (

7
90 robot = robot ,
91 events = events ,
92 fps = FPS ,
93 teleop = teleop ,
94 co ntrol_ time_s = RESET_TIME_SEC ,
95 single_task = TASK_DESCRIPTION ,
96 display_data = True ,
97 )
98
99 if events [ " r e r e c o r d _ e p i s o d e " ]:
100 log_say ( " Re - recording episode " )
101 events [ " r e r e c o r d _ e p i s o d e " ] = False
102 events [ " exit_early " ] = False
103 dataset . c l e a r _ e p i s o d e _ b u f f e r ()
104 continue
105
106 dataset . save_episode ()
107 episode_idx += 1
108
109 # Clean up
110 log_say ( " Stop recording " )
111 robot . disconnect ()
112 teleop . disconnect ()
113 dataset . push_to_hub ()

8
Figure 2 | Overview of methods to generate motion (clearly non-exhausitve, see Bekris et al. (2024)). The different methods
can be grouped based on whether they explicitly (dynamics-based ) or implicitly (learning-based ) model robot-environment
interactions.

2 Classical Robotics

Know your enemy [...]

Sun Tzu

TL;DR

Learning-based approaches to robotics are motivated by the need to (1) generalize across tasks and embodiments
(2) reduce dependency on human expertise (3) leverage historical trends on the production of data—all
traditionally overlooked by dynamics-based techniques.

2.1 Explicit and Implicit Models

Robotics is concerned with producing artificial motion in the physical world in useful, reliable and safe fashion. Thus,
robotics is an inherently multi-disciplinar domain: producing autonomous motion in the physical world requires, to the
very least, interfacing different software (motion planners) and hardware (motion executioners) components. Further,
knowledge of mechanical, electrical, and software engineering, as well as rigid-body mechanics and control theory
have therefore proven quintessential in robotics since the field first developed in the 1950s. More recently, Machine
Learning (ML) has also proved effective in robotics, complementing these more traditional disciplines (Connell and
Mahadevan, 1993). As a direct consequence of its multi-disciplinar nature, robotics has developed as a rather wide
array of methods, all concerned with the main purpose of producing artificial motion in the physical world.
Methods to produce robotics motion range from traditional explicit models—dynamics-based1 methods, leveraging
precise descriptions of the mechanics of robots’ rigid bodies and their interactions with eventual obstacles in the
environment—to implicit models—learning-based methods, treating artificial motion as a statistical pattern to learn
given multiple sensorimotor readings (Agrawal; Bekris et al., 2024). A variety of methods have been developed
between these two extrema. For instance, Hansen et al. (2022) show how learning-based systems can benefit from
information on the physics of problems, complementing a traditional learning method such as Temporal Difference
(TD)-learning Sutton and Barto (2018) with Model-Predictive Control (MPC). Conversely, as explicit models may
be relying on assumptions proving overly simplistic—or even unrealistic—in practice, learning can prove effective
to improve modeling of complex phenomena or complement perception (McCormac et al., 2016). Such examples
1 In here, we refer to both kinematics and dynamics-based control.

9
Figure 3 | Different kinds of motions are achieved with potentially very different robotic platforms. From left to right, top to
bottom: ViperX, SO-100, Boston Dynamics’ Spot, Open-Duck, 1X’s NEO, Boston Dynamics’ Atlas. This is an example list of
robotic platforms and is (very) far from being exhaustive.

aim at demonstrating the richness of approaches to robotics, and Figure 2 graphically illustrates some of the most
relevant techniques. Such a list is clearly far from being exhaustive, and we refer to Bekris et al. (2024) for a more
comprehensive overview of both general and application-specific methods for motion generation. In this section, we
wish to introduce the inherent benefits of learning-based approaches to robotics—the core focus on this tutorial.

2.2 Different Types of Motion

In the vast majority of instances, robotics deals with producing motion via actuating joints connecting nearly
entirely-rigid links. A key distinction between focus areas in robotics is based on whether the generated motion
modifies (1) the absolute state of the environment (via dexterity), (2) the relative state of the robot with respect to
its environment (exercising mobility skills), or (3) a combination of the two (Figure 3).
Effects such as (1) are typically achieved through the robot, i.e. generating motion to perform an action inducing
a desirable modification, effectively manipulating the environment (manipulation). Motions like (2) may result in
changes in the robot’s physical location within its environment. Generally, modifications to a robot’s location within
its environment may be considered instances of the general locomotion problem, further specified as wheeled or legged
locomotion based on whenever a robot makes use of wheels or leg(s) to move in the environment. Lastly, an increased
level of dynamism in the robot-environment interactions can be obtained combining (1) and (2), thus designing
systems capable to interact with and move within their environment. This category is problems is typically termed
mobile manipulation, and is characterized by a typically much larger set of control variables compared to either
locomotion or manipulation alone.
The traditional body of work developed since the very inception of robotics is increasingly complemented by learning-
based approaches. ML has indeed proven particularly transformative across the entire robotics stack, first empowering
planning-based techniques with improved state estimation used for traditional planning (Tang et al., 2023) and
then end-to-end replacing controllers, effectively yielding perception-to-action methods (Kober et al.). Work in
producing robots capable of navigating a diverse set of terrains demonstrated the premise of both dynamics and
learning-based approaches for locomotion (Griffin et al., 2017; Ji et al., 2023; Lee et al., 2020; Margolis et al., 2022),
and recent works on whole-body control indicated the premise of learning-based approaches to generate rich motion
on complex robots, including humanoids (Zhang et al., 2024; Bjorck et al., 2025). Manipulation has also been widely
studied, particularly considering its relevance for many impactful use-cases ranging from high-risk applications for
humans (Fujita et al., 2020; Alizadeh and Zhu, 2024) to manufacturing (Sanneman et al., 2020). While explicit models
have proven fundamental in achieving important milestones towards the development of modern robotics, recent
works leveraging implicit models proved particularly promising in surpassing scalability and applicability challenges
via learning (Kober et al.).

2.3 Example: Planar Manipulation

Robot manipulators typically consist of a series of links and joints, articulated in a chain finally connected to an
end-effector. Actuated joints are considered responsible for generating motion of the links, while the end effector is
instead used to perform specific actions at the target location (e.g., grasping/releasing objects via closing/opening a
gripper end-effector, using a specialized tool like a screwdriver, etc.).

10
Figure 4 | Cheaper, more accessible robots are starting to rival traditional platforms like the Panda arm platforms in adoption
in resource-constrained scenarios. The SO-100, in particular, has a cost in the 100s of Euros, and can be entirely 3D-printed in
hours, while the industrially-manufactured Panda arm costs tens of thousands of Euros and is not openly available.

Figure 5 | The SO-100 arm is a 6-dof manipulator arm. Preventing some of its joints (shoulder pane, wrist flex and wrist roll)
from actuating, it can be represented as a traditional 2-dof planar manipulator (the gripper joint in the end-effector is not
considered towards the count of the degrees of freedom used to produce motion).

Recently, the development of low-cost manipulators like the ALOHA (Zhao et al., 2023) ALOHA-2 (Aldaco et al.)
and SO-100/SO-101 (Knight et al.) platforms significantly lowered the barrier to entry to robotics, considering the
increased accessibility of these robots compared to more traditional platforms like the Franka Emika Panda arm
(Figure 4).
Deriving an intuition as per why learning-based approaches are gaining popularity in the robotics community requires
briefly analyzing traditional approaches for manipulation, leveraging tools like forward and inverse kinematics (FK,
IK) and control theory. Providing a detailed overview of these methods falls (well) out of the scope of this tutorial,
and we refer the reader to works including Siciliano and Khatib (2016); Lynch and Park (2017); Tedrake (a,b) for a
much more comprehensive description of these techniques. Here, we mostly wish to highlight the benefits of ML over
these traditional techniques
Consider the (simple) case where a SO-100 is restrained from actuating (1) the shoulder pane and (2) the wrist flex
and roll motors. This effectively reduces the degrees of freedom of the SO-100 from the original 5+1 (5 joints + 1
gripper) to 2+1 (shoulder lift, elbow flex + gripper). As the end-effector does not impact motion in this model,
the SO-100 is effectively reduced to the planar manipulator robot presented in Figure 5, where spheres represent
actuators, and solid lines indicate length-l links from the base of the SO-100 to the end-effector (ee).
Further, let us make the simplifying assumption that actuators can produce rotations up to 2π radians. In practice,
this is seldom the case due to movement obstructions caused by the robot body itself (for instance, the shoulder lift
cannot produce counter-clockwise movement due to the presence of the robot’s base used to secure the SO-100 to its
support and host the robot bus), but we will introduce movement obstruction at a later stage.
All these simplifying assumptions leave us with the planar manipulator of Figure 6a, free of moving its end-
effector by controlling the angles θ1 and θ2 , jointly referred to as the robot’s configuration, and indicated with
q = [θ1 , θ2 ] ∈ [−π, +π]2 . The axis attached to the joints indicate the associated reference frame, whereas circular

11
(a) | Free to move (b) | Constrained by the surface (c) | Constrained by surface and (fixed)
obstacle

Figure 6 | Planar, 2-dof schematic representation of the SO-100 manipulator under diverse deployment settings. From left
to right: completely free of moving; constrained by the presence of the surface; constrained by the surface and presence of
obstacles. Circular arrows around each joint indicate the maximal rotation feasible at that joint.

arrows indicate the maximal feasible rotation allowed at each joint. In this tutorial, we do not cover topics related to
spatial algebra, and we instead refer the reader to Lynch and Park (2017, Chapter 2) and Tedrake (a, Chapter 3) for
excellent explanations of the mechanics and theoretical foundations of producing motion on rigid bodies.
Considering the (toy) example presented in Figure 6a, then we can analytically write the end-effector’s position p ∈ R2
as a function of the robot’s configuration, p = p(q), p : Q 7→ R2 . In particular, we have:
! !
px (θ1 , θ2 ) l cos(θ1 ) + l cos(θ1 + θ2 )
p(q) = = ∈ Sln=2
1 +l2
= {p(q) ∈ R2 : ∥p(q)∥22 ≤ (2l)2 , ∀q ∈ Q}
py (θ1 , θ2 ) l sin(θ1 ) + l sin(θ1 + θ2 )

Deriving the end-effector’s pose—position and orientation—in some m-dimensional space p ∈ P ⊂ Rm starting from
the configuration q ∈ Q ⊂ Rn of a n-joints robot is referred to as forward kinematics (FK), whereas identifying the
configuration corresponding to any given target pose is termed inverse kinematics (IK). In that, FK is used to map a
robot configuration into the corresponding end-effector pose, whereas IK is used to reconstruct the configuration(s)
given an end-effector pose.
In the simplified case here considered (for which p ≡ p, as the orientation of the end-effector is disregarded for
simplicity), one can solve the problem of controlling the end-effector’s location to reach a goal position p∗ by solving
analytically for q : p(q) = fFK (q) = p∗ . However, in the general case, one might not be able to solve this problem
analytically, and can typically resort to iterative optimization methods comparing candidate solutions using a loss
function (in the simplest case, ∥p(q) − p∗ ∥22 is a natural candidate), yielding:

min ∥p(q) − p∗ ∥22 . (1)

q∈Q

Exact analytical solutions to IK are even less appealing when one considers the presence of obstacles in the robot’s
workspace, resulting in constraints on the possible values of q ∈ Q ⊆ [−π, +π]n ⊂ Rn in the general case of n-links
robots.
For instance, the robot in Figure 6b is (very naturally) obstacled by the presence of the surface upon which it rests: θ1
can now exclusively vary within [0, π], while possible variations in θ2 depend on θ1 (when θ1 → 0 or θ1 → π, further
downwards movements are restricted). Even for a simplified kinematic model, developing techniques to solve eq. 1 is
in general non-trivial in the presence of constraints, particularly considering that the feasible set of solutions Q may
change across problems. Figure 6c provides an example of how the environment influences the feasible set considered,
with a new set of constraints deriving from the position of a new obstacle.
However, IK—solving eq. 1 for a feasible q—only proves useful in determining information regarding the robot’s
configuration in the goal pose, and crucially does not provide information on the trajectory to follow over time to
reach a target pose. Expert-defined trajectories obviate to this problem providing a length-K succession of goal
poses τK = [p∗0 , p∗1 , . . . p∗K ] for tracking. In practice, trajectories can also be obtained automatically through motion
planning algorithms, thus avoiding expensive trajectory definition from human experts. However, tracking τK via
IK can prove prohibitively expensive, as tracking would require K resolutions of eq. 1 (one for each target pose).
Differential inverse kinematics (diff-IK) complements IK via closed-form solution of a variant of eq. 1. Let J(q) denote

12
the Jacobian matrix of (partial) derivatives of the FK-function fFK : Q 7→ P, such that J(q) = ∂fF∂q K (q)
. Then, one
can apply the chain rule to any p(q) = fFK (q), deriving ṗ = J(q)q̇, and thus finally relating variations in the robot
configurations to variations in pose, thereby providing a platform for control.
Given a desired end-effector trajectory ṗ∗ (t) (1) indicating anchor regions in space and (2) how much time to spend
in each region, diff-IK finds q̇(t) solving for joints’ velocities instead of configurations,

q̇(t) = arg min ∥J(q(t))ν − ṗ∗ (t)∥22 (2)

Unlike eq. 1, solving for q̇ is much less dependent on the environment (typically, variations in velocity are constrained
by physical limits on the actuators). Conveniently, eq. 2 also often admits the closed-form solution q̇ = J(q)+ ṗ∗ ,
where J + (q) denotes the Moore-Penrose pseudo-inverse of J(q). Finally, discrete-time joint configurations q can be
reconstructed from joint velocities q̇ using forward-integration on the continuous-time joint velocity , qt+1 = qt + ∆t q̇t
for a given ∆t, resulting in tracking via diff-IK.
Following trajectories with diff-IK is a valid option in well-controlled and static environments (e.g., industrial
manipulators in controlled manufacturing settings), and relies on the ability to define a set of target velocities to
track [ṗ∗0 , ṗ∗1 , . . . , ṗ∗k ]—an error-prone task largely requiring human expertise. Furthermore, diff-IK relies on the ability
to (1) access J(q) ∀q ∈ Q and (2) compute its pseudo-inverse at every iteration of a given control cycle—a challenging
assumption in highly dynamical settings, or for complex kinematic chains.

2.3.1 Adding Feedback Loops

While very effective when a goal trajectory has been well specified, the performance of diff-IK can degrade significantly
in the presence of modeling/tracking errors, or in the presence of non-modeled dynamics in the environment.
One such case is presented in Figure 7, where another rigid body other than
the manipulator is moving in the environment along the horizontal axis, with
velocity ẋB . Accounting analytically for the presence of this disturbance—for
instance, to prevent the midpoint of the link from ever colliding with the
object—requires access to ẋB at least, to derive the equation characterizing
the motion of the environment.
Less predictable disturbances however (e.g., ẋB ← ẋB + ε, ε ∼ N (0, 1)) may
prove challenging to model analytically, and one could attain the same result of
Figure 7 | Planar manipulator robot
preventing link-object collision by adding a condition on the distance between in the presence of a moving obstacle.
the midpoint of l and xB , enforced through a feedback loop on the position
of the robot and object at each control cycle.
To mitigate the effect of modeling errors, sensing noise and other disturbances,
classical pipelines indeed do augment diff-IK with feedback control looping back quantities of interest. In practice,
following a trajectory with a closed feedback loop might consist in backwarding the error between the target and
measured pose, ∆p = p∗ − p(q), hereby modifying the control applied to q̇ = J(q)+ (ṗ∗ + kp ∆p), with kp defined as
the (proportional) gain.
More advanced techniques for control consisting in feedback linearization, PID control, Linear Quatratic Regulator
(LQR) or Model-Predictive Control (MPC) can be employed to stabilize tracking and reject moderate perturbations,
and we refer to Siciliano and Khatib (2016, Chapter 8) for in-detail explanation of these concepts, or (Tedrake, a,
Chapter 8) for a simple, intuitive example in the case of a point-mass system. Nonetheless, feedback control presents
its challenges as well: tuning gains remains laborious and system-specific. Further, manipulation tasks present
intermittent contacts inducing hybrid dynamics (mode switches) and discontinuities in the Jacobian, challenging
the stability guarantees of the controller and thus often necessitating rather conservative gains and substantial
hand-tuning.
We point the interested reader to Siciliano and Khatib (2016, Chapter 2,7,8), Lynch and Park (2017, Chapter 6,11),
and Tedrake (a, Chapter 3,8) for extended coverage of FK, IK, diff-IK and control for (diff-)IK.

2.4 Limitations of Dynamics-based Robotics

Despite the last 60+ years of robotics research, autonomous robots are still largely incapable of performing tasks at
human-level performance in the physical world generalizing across (1) robot embodiments (different manipulators,

13
Figure 8 | Dynamics-based approaches to robotics suffer from several limitations: (1) orchestrating multiple components poses
integration challenges; (2) the need to develop custom processing pipelines for the sensing modalities and tasks considered
hinders scalability; (3) simplified analytical models of physical phenomena (here friction at the gripper; credits to Antonova
et al. (2017)) limit real-world performance. Lastly, (4) dynamics-based methods overlook trends in the availability and growth
of robotics data.

different locomotion platforms, etc.) and (2) tasks (tying shoe-laces, manipulating a diverse set of objects). While
essential in the early development of robotics, the aforementioned methods require significant human expertise to be
used in practice, and are typically specific to a particular applicative problem.
Dynamics-based robotics pipelines have historically been developed sequentially, engineering the different blocks now
within most architectures for specific purposes. That is, sensing, state estimation, mapping, planning, (diff-)IK, and
low-level control have been traditionally developed as distinct modules with fixed interfaces. Pipelining these specific
modules proved error-prone, and brittleness emerges—alongside compounding errors—whenever changes incur (e.g.,
changes in lighting for sensing, occlusion/failure of sensors, control failures). Adapting such a stack to new tasks
or robotic platforms often entails re-specifying objectives, constraints, and heuristics at multiple stages, incurring
significant engineering overhead.
Moreover, classical planners operate on compact, assumed-sufficient state representations; extending them to reason
directly over raw, heterogeneous and noisy data streams is non-trivial. This results in a limited scalability to
multimodal data and multitask settings, as incorporating high-dimensional perceptual inputs (RGB, depth, tactile,
audio) traditionally required extensive engineering efforts to extract meaningful features for control. Also, the large
number of tasks, coupled with the adoption of per-task planners, goal parameterizations, and safety constraints,
results in an explosion in design and validation options, with little opportunity to reuse solutions across tasks.
Setting aside integration and scalability challenges: developing accurate modeling of contact, friction, and compliance
for complicated systems remains difficult. Rigid-body approximations are often insufficient in the presence of
deformable objects, and relying on approximated models hinders real-world applicability of the methods developed.

14
In the case of complex, time-dependent and/or non-linear dynamics, even moderate mismatches in parameters,
unmodeled evolutions, or grasp-induced couplings can qualitatively affect the observed dynamics.
Lastly, dynamics-based methods (naturally) overlook the rather recent increase in availability of openly-available
robotics datasets. The curation of academic datasets by large centralized groups of human experts in robotics (O’Neill
et al., 2025; Khazatsky et al., 2025) is now increasingly complemented by a growing number of robotics datasets
contributed in a decentralized fashion by individuals with varied expertise. If not tangentially, dynamics-based
approaches are not posed to maximally benefit from this trend, which holds the premise of allowing generalization in
the space of tasks and embodiments, like data was the cornerstone for advancements in vision (Alayrac et al., 2022)
and natural-language understanding (Brown et al., 2020).
Taken together, these limitations (Figure 8) motivate the exploration of learning-based approaches that can (1)
integrate perception and control more tightly, (2) adapt across tasks and embodiments with reduced expert modeling
interventions and (3) scale gracefully in performance as more robotics data becomes available.

15
Figure 9 | Learning-based robotics streamlines perception-to-action by learning a (1) unified high-level controller capable to
take (2) high-dimensional, unstructured sensorimotor information. Learning (3) does not require a dynamics model and instead
focuses on interaction data, and (4) empirically correlates with the scale of the data used.

3 Robot (Reinforcement) Learning

Approximate the solution, not the
problem [...]

Richard Sutton

TL;DR

The need for expensive, high-fidelity simulators can be obviated learning from real-world data, using sample-
efficient algorithms that can safely train directly on hardware.

Learning-based techniques for robotics naturally address the limitations presented in Section 2 (Figure 9). In particular,
learning-based techniques typically rely on monolithich prediction-to-action pipelines (visuomotor policies) which
do directly map sensorimotor inputs to predicted actions, streamlining control policies by removing the need to
interface multiple components. Mapping sensory inputs to actions also makes it possible to incorporate diverse
input modalities, leveraging the automatic feature extraction capabilities of modern learning systems. Moreover,
learning-based approaches can, in principle, bypass explicit modeling altogether and instead rely solely on interaction
data—an advantage that proves transformative when dynamics are difficult to model or entirely unknown. Lastly,
learning for robotics (robot learning) is naturally well posed to leverage the growing amount of robotics data openly
available, just as computer vision and natural language processing did historically benefit from large-scale corpora of
data, in great part overlooked by dynamics-based approaches.
Being a field at its relative nascent stages, no prevalent technique(s) proves distinctly better than any other in the

16
Figure 11 | Examples of two different robotics tasks performed using RL. In the manipulation task (A) an agent learns to reach
for a yellow plastic block in its environment, and to put it inside of a box. In the locomotion task (B) an agent learns to move
its center of mass sideways without falling.

domain of robot learning. Still, two major classes of methods gained prominence: Reinforcement Learning (RL) and
Behavioral Cloning (BC) (Figure 10). In this section, we provide a conceptual overview of applications of RL to
robotics, as well as introduce practical examples of how to use RL within lerobot. We then introduce the major
limitations RL suffers from, to introduce BC techniques in Section 4 and Section sec:learning-foundation.
In Figure 10 we deliberately include generalist robot models (Black et al.,
2024; Shukor et al., 2025) alongside task-specific BC methods. While signif-
icantly different in spirit—generalist models are language-conditioned and use
instructions to generate motion valid across many tasks, while task-specific
models are typically not language-conditioned and used to perform a single
task—foundation models are still largely trained to reproduce trajectories
contained in a (large) training set of input demonstrations. Thus, we argue
generalist policies can indeed be grouped alongside other task-specific BC
methods, as they both leverage similar training data and schemas. Figure 10 il-
lustrates this categorization graphically, explicitly listing all the robot learning
policies currently available in lerobot: Action Chunking with Transformers
(ACT) (Zhao et al., 2023), Diffusion Policy (Chi et al., 2024), Vector-Quantized
Behavior Transformer (VQ-BeT) (Lee et al., 2024), π0 (Black et al., 2024),
SmolVLA (Shukor et al., 2025), Human-in-the-loop Sample-efficient RL (HIL-
SERL) (Luo et al., 2024) and TD-MPC (Hansen et al., 2022).
Applications of RL to robotics have been studied long enough that the rela- Figure 10 | Overview of the
robot learning methods implemented
tionship between these two disciplines has been compared to that of physics
in lerobot. All algorithms are imple-
and matematics (Kober et al.). Indeed, due to their inherently interactive mented in Pytorch. References: Zhao
and sequential nature, robotics control problems can be directly cast as RL et al. (2023); Chi et al. (2024); Lee et al.
problems. Figure 11 presents two of such cases. Reaching for an object to (2024); Black et al. (2024); Shukor et al.
then move it somewhere else in the scene is a sequential problem where over (2025); Luo et al. (2024); Hansen et al.
time the controller needs to adjust the position of the robot arm based on (2022) (top-to-bottom, left-to-right).
the current configuration and the (possibly varying) position of the object.
Figure 11 also shows an example of a locomotion problem, where sequentiality is inherent in the problem formulation:
while sliding to the side, the controller needs to keep adjusting to the robot’s to avoid failure (falling).

3.1 A (Concise) Introduction to RL

The RL framework (Sutton and Barto, 2018), which we briefly introduce here, has often been used to tackle robotics
problems (Kober et al.). RL is a subfield within ML fundamentally concerned with the development of autonomous
systems (agents) capable to continuously behave in an evolving environment, developing (ideally, well-performing)
control strategies (policies). Crucially for robotics, RL agents improve through trial and error, bypassing explicit
models of the problem dynamics in favor of interaction data. In RL, this feedback loop between actions and outcomes
(Figure 12) is established through the agent sensing a scalar quantity (reward ) measuring how desirable a given
transition is for the accomplishment of its goal.
Formally, interactions between an agent and its environment are typically modeled via a Markov Decision Pro-

17
Figure 12 | Agent-Environment interaction diagram (image credits to Sutton and Barto (2018)).

cess (MDP) (Bellman, 1957). Representing robotics problems via MDPs offers several advantages, including (1)
incorporating uncertainty through MDP’s inherently stochastic formulation and (2) providing a theoretically-sound
framework for learning without an explicit model of the environment dynamics. While accommodating a continuous
time formulation too, MDPs are typically considered in discrete time in RL, assuming interactions to atomically take
place at discrete timestep t = 0, 1, 2, 3, . . . , T . MDPs allowing for an unbounded number of interactions (T → +∞)
are termed infinite-horizon, and opposed to finite-horizon MDPs in which T is finite. Unless diversely specified, we
will only be referring to discrete-time finite-horizon (episodic) MDPs.
Formally, a lenght-T Markov Decision Process (MDP) is a tuple M = ⟨S, A, D, r, γ, ρ, T ⟩, where:

• S is the state space; st ∈ S denotes the (possibly non-directly observable) environment state at time t. In robotics,
states often comprise robot configuration and velocities (qt , q̇t ), and can also accomodate sensor readings such as
camera or audio streams.
• A is the action space; at ∈ A may represent joint torques, joint velocities, or even end-effector commands at
timestep t. In general, actions correspond to commands intervenings on the configuration of the robot.
• D represents the (possibly non-deterministic) environment dynamics, with D : S × A × S 7→ [0, 1], D (st , at , st+1 ) =
P(st+1 |st , at ). For instance, for a planar manipulator dynamics could be considered deterministic when the
environment is fully described (Figure 6a), and stochastic when unmodeled disturbances depending on non-
observable parameters intervene (Figure 7).
• r : S × A × S → R is the reward function, weighing the transition (st , at , st+1 ) in the context of the achievement of
an arbitrary goal. For instance, a simple reward function for quickly moving along the x axis (Figure 11) could
be based on the absolute position of the robot along the x axis (pxt ), present negative penalties for falling over
(measured from pzt ) and a introduce bonuses ṗxt for speed, r(st , at , st+1 ) ≡ r(st ) = pxt · ṗxt − p1z .
t

Lastly, γ ∈ [0, 1) represent the discount factor regulating preference for immediate versus long-term reward (with an
effective horizon equal to 1−γ
1
), and ρ is the distribution over S for the MDP’s initial, s0 ∼ ρ.
Therefore, a length-T trajectory is the (random) sequence
τ = (s0 , a0 , r0 , s1 , a1 , r1 , . . . , sT −1 , aT −1 , rT −1 , sT ), (3)
with per-step rewards defined as rt = r(st , at , st+1 ) for ease of notation. Interestingly, assuming both the environment
dynamics and conditional distribution over actions given states—i.e., the policy—to be Markovian:
P(st+1 |st , at , st−1 , at−1 , . . . s0 , a0 ) = P(st+1 |st , at ) (4)
P(at |st , at−1 , st−1 , s0 , a0 ) = P(at |st ), (5)
the probability of observing a given trajectory τ factorizes into:
−1
TY
P(τ ) = P(s0 ) P(st+1 |st , at ) P(at |st ). (6)
t=0

Policies P(at |st ) are typically indicated as π(at |st ), often parametrized via θ, yielding πθ (at |st ), and are traine by
optimizing the (discounted) return associated to a given τ , i.e. the (random) sum of measured rewards over an
arbitrary trajectory,
T
X −1
G(τ ) = γ t rt .
t=0

18
Figure 13 | Popular RL algorithms. See Achiam (2018) for a complete list of citations.

In that, agents seek to learn control strategies (policies, πθ ) maximizing the expected return Eτ ∼πθ G(τ ). For a given
dynamics D—i.e., for a given problem—taking the expectation over the (possibly random) trajectories resulting from
acting according to a certain policy provides a direct, goal-conditioned ordering in the space of all the possible policies
Π, yielding the (maximization) target J : Π 7→ R

J(πθ ) = Eτ ∼Pθ;D [G(τ )], (7)

−1
TY
Pθ;D (τ ) = ρ D(st , at , st+1 ) πθ (at |st ). (8)
t=0

Crucially, in the RL framework the agent is assumed to only observe the environment dynamics and not to intervene
on them, and thus eq. 7 varies exclusively with the policy followed. In turn, MDPs naturally provide a framework
to optimize over the space of the possible behaviors an agent might enact (π ∈ Π), searching for the optimal policy
π ∗ = arg maxθ J(πθ ), where θ is the parametrization adopted by the policy set Π : πθ ∈ Π, ∀θ. Besides providing a
target for policy search, G(τ ) can also be used to discriminate between states st and st , at pairs. Given any state
s ∈ S—e.g., given a configuration q of a robot—the state-value function

Vπ (s) = Eτ ∼π G(τ ) s0 = s

can be used to discriminate between desirable and undesirable state in terms of long-term (discounted) reward
maximization, under a given policy π. Similarily, the state-action value function also conditions the cumulative
discounted reward on selecting action a when in s, and thereafter act according to π,

Qπ (s, a) = Eτ ∼π G(τ ) s0 = s, a0 = a .

Importantly, value functions are interrelated:

Qπ (st , at ) = Est+1 ∼P(•|st ,at ) [rt + γVπ (st+1 )] (9)

Vπ (st ) = Eat ∼π(•|st ) [Qπ (st , at )], (10)

inducing an ordering over states and state-action pairs under π, and value functions are thus central to most RL
algorithms. A variety of algorithms have been developed in RL attempting to find (approximate) solutions to the
problem of maximizing cumulative reward (we report some in Figure 13).
Popular approaches to continuous state and action space—such as those studied within robotics—include Schulman
et al. (2017a, TRPO), Schulman et al. (2017b, PPO) and Haarnoja et al. (2018, SAC). Across manipulation (Akkaya

19
Figure 14 | Simulated (left) vs. real-world (right) OpenDuck. Discrepancies in the simulation dynamics (reality gap) pose risks
to policy transfer.

et al., 2019) and locomotion problems (Lee et al., 2020), RL proved extremely effective in providing a platform to (1)
leverage a unified, streamlined perception-to-action pipeline, (2) natively integrate propioperception with multi-modal
high-dimensional sensory streams (3) disregard a description of the environment dynamics, by focusing on observed
interaction data rather than modeling, and (4) anchor policies in the experience collected and stored in datasets. For
a more complete survey of applications of RL to robotics, we refer the reader to Kober et al.; Tang et al. (2025).

3.2 Real-world RL for Robotics

Streamlined end-to-end control pipelines, data-driven feature extraction and a disregard for explicit modeling in favor
of interaction data are all features of RL for robotics. However, RL still suffers from limitations concerning safety and
learning efficiency, particularly pressing for real-world robotics applications.
First, especially early in training, actions are typically explorative, and thus may be erractic. On physical systems,
untrained policies may command high velocities, self-collisiding configurations, or torques exceeding joint limits,
leading to wear and potential hardware damage. Mitigating these risks requires external safeguards (e.g., watchdogs,
safety monitors, emergency stops), often incuring in a high degree of human supervision. Further, in the typical
episodic setting considered in most robotics problems, experimentation is substantially slowed down by the need to
manually reset the environment over the course of training, a time-consuming and error-prone process. Second, learning
efficiently remains problematic in RL, limiting the applicability of RL in real-world robotics due to consequently
prohibitive timescales of training. Even strong algorithms such as SAC (Haarnoja et al., 2018) typically require a
large numbers of transitions {(st , at , rt , st+1 )}N
t=1 . On real-world hardware, generating this data is time-consuming.

Training RL policies in simulation (Tobin et al., 2017) addresses both issues, eliminating physical risk and dramatically
increasing throughput. Yet, simulators require significant modeling effort, and rely on assumptions (simplified physical
modeling, instantaneous actuation, static environmental conditions, etc.) limiting the possibilities to transfer the
policies learned in simulation, due the discrepancy between real and simulated environments (reality gap, Figure 14).
Domain randomization (Tobin et al., 2017) (DR) is a popular technique to overcome the reality gap, and consists in
randomizing the parameters of the simulated environment during training, aiming at inducing robustness to specific
disturbances. In this, DR is typically employed to increase the diversity of scenarios over the course of training,
improving on the performace sim-to-real transferred policies (Akkaya et al., 2019; Antonova et al., 2017; Ji et al.,
2023). In practice, DR is performed training in simulation on simulated dynamics D, further parametrized as D ≡ Dξ ,
with a dynamics (random) vector ξ drawn an arbitrary distribution, ξ ∼ Ξ. For instance, one could decide to
randomize the friction coefficient of the surface in a locomotion task (Figure 15), or the center of mass of an object
for a manipulation task. Over the course of training—typically at each episode’s reset—a new ξ is drawn, and used to
specify the environment’s dynamics for that episode.
While effective in transfering policies across the reality gap in real-world robotics (Tobin et al., 2017; Akkaya et al.,
2019; Ji et al., 2023; Tiboni et al., 2024), DR often requires extensive manual engineering. First, identifying which

20
Figure 15 | The same locomotion task can be carried out in different (simulated) domains (exemplified by the difference in
terrains) at training time, resulting to increased robustness over diverse environment dynamics.

parameters to randomize—i.e., the support supp(Ξ) of Ξ—is an inherently task specific process. When locomoting over
different terrains, choosing to randomize the friction coefficient is a reasonable choice, yet not completely resolutive as
other factors (lightning conditions, external temperature, joints’ fatigue, etc.) may prove just as important in practice,
making selecting these parameters yet another source of brittlness.
Selecting the dynamics distribution Ξ is also non-trivial. On the one hand, distributions with low entropy might risk
to cause failure at transfer time, due to the limited robustness induced over the course of training. On the other hand,
excessive randomization may cause over-regularization and hinder performance (Margolis et al., 2022). Consequently,
the research community investigated approaches to automatically select the randomization distribution Ξ, using
signals from the training process or tuning it to reproduce observed real-world trajectories. Akkaya et al. (2019)
use a parametric uniform distribution U(a, b) as Ξ, widening the bounds a, b as training progresses and the agent’s
performance improves (AutoDR). While effective, AutoDR requires significant tuning—the bounds are widened by a
fixed, pre-specified amount ∆ along—and may disregard data when performance does not improve after a distribution
update (Tiboni et al., 2024). Tiboni et al. (2024) propose a similar method to AutoDR (DORAEMON) to evolve Ξ
based on the training signal, but with the key difference of explicitly maximizing the entropy of a parametric Beta
distribution—inherently more flexible than uniform distributions—with learned updates instead of fixed ∆. In this,
DORAEMON proves particularly effective at dynamically increasing the entropy levels of the training distribution by
employing an outer-loop max-entropy objective, tackled under performance constraints in the inner-loop RL problem.
Other approaches to automatically perform DR consist in specifically tuning Ξ to align as much as possible the
simulation and real-world domains. For instance, Chebotar et al. (2019) interleave in-simulation policy training with
repeated real-world policy rollouts used to adjust Ξ based on real-world data, while Tiboni et al. (2023) leverage a
single, pre-collected set of real-world trajectories and tune Ξ under a simple likelihood objective.
While DR has shown promise, it does not address the main limitation that, even under the assumption that an ideal
distribution Ξ was available, many robotics problems cannot be simulated with high-enough fidelity under practical
computational constraints. Simulating contact-rich manipulation of possibly deformable or soft materials—i.e., folding
a piece of clothing—can prove time-intensive, limiting the benefits of in-simulation training.
A perhaps more foundamental limitation of RL for robotics is the general unavailability of complicated tasks’ dense
reward function, the design of which is essentially based on human expertise, ingenuity and trial-and-error. In
practice, sparse reward functions can be used to conclude whether one specific goal has been attained—has this t-shirt
been correctly folded? —but unfortunately incur in more challenging learning. As a result, despite notable successes,
deploying RL directly on real-world robots at scale remains challenging.
To make the most of (1) the growing number of openly available datasets and (2) relatively inexpensive robots like the
SO-100, RL could (1) be anchored in already-collected trajectories—limiting erratic and dangerous exploration—and
(2) train in the real-world directly—bypassing the aforementioned issues with low-fidelity simulations. In such a
context, sample-efficient learning is also paramount, as training on the real-world is inherently time-bottlenecked.
Off-policy algorithms like Soft Actor-Critic (SAC) (Haarnoja et al., 2018) tend to be more sample efficient then their
on-policy counterpart (Schulman et al., 2017b), due to the presence a replay buffer used over the course of training.
Other than allowing to re-use past transitions (st , at , rt , st+1 ), the replay buffer can also accomodate for the injection
of previously-collected data in the training process (Ball et al., 2023). Using expert demonstrations to guide learning
together with learned rewards, RL can be effectively carried out in the real-world (Luo et al., 2025). Interestingly,
when complemented with in-training human interventions, real-world RL agents have been shown to learn policies
with near-perfect success rates on challenging manipulation tasks in 1-2 hours (Luo et al., 2024).

21
Sample-efficient RL In an MDP, the optimal policy π ∗ can be derived from its associated Q-function, Q∗ ≡ Qπ∗ ,
and in particular the optimal action(s) µ(st ) can be selected maximizing the optimal Q-function over the action space,

µ(st ) = max Q∗ (st , at ).

at ∈A

Interestingly, the Q∗ -function satisfies a recursive relationship (Bellman equation) based on a very natural intuition2 :
[...] If the optimal value Q∗ (st+1 , at+1 ) of the [state] st+1 was known for all possible actions at+1 , then
the optimal strategy is to select the action at+1 maximizing the expected value of rt + γQ∗ (st+1 , at+1 )

Q∗ (st , at ) = Est+1 ∼P(•|st ,at ) rt + γ max Q∗ (st+1 , at+1 ) st , at
at+1 ∈A

In turn, the optimal Q-function is guaranteed to be self-consistent by definition. Value-iteration methods exploit this
relationship (and/or its state-value counterpart, V ∗ (st ) ) by iteratively updating an initial estimate of Q∗ , Qk using
the Bellman equation as update rule (Q-learning):

Qi+1 (st , at ) ← Est+1 ∼P(•|st ,at ) rt + γ max Qi (st+1 , at+1 ) st , at , i = 0, 1, 2, . . . , K
at+1 ∈A

Then, one can derive the (ideally, near-optimal) policy by explicitly maximizing over the action space the final (ideally,
near-optimal) estimate QK ≈ Q∗ at each timestep. Indeed, one can show that under certain assumptions on the MDP
considered, QK → Q∗ as K → ∞.
Effective in its early applications to small-scale discrete problems, vanilla Q-learning was found complicated to scale
to large S × A problems, in which storing Q : S × A 7→ R alone might result prohibitive. Also, vanilla Q-learning is
not directly usable for continuous, unstructured state-action space MPDs, such as those considered in robotics. In
their seminal work on Deep Q-Learning (DQN), Mnih et al. (2013) propose learning Q-values using deep convolutional
neural networks, thereby accomodating for large and even unstructured state spaces. DQN parametrizes the Q-function
using a neural network with parameters θ, updating the parameters by sequentially minimizing the expected squared
temporal-difference error (TD-error, δi ):

L(θi ) = E(st ,at )∼χ(•) (yi − Qθi (st , at ))2 , (11)

| {z }
δi

(12)

yi = Est+1 ∼P(•|st ,at ) rt + γ max Qθi−1 (st+1 , at+1 ) ,
at ∈A

where χ represents a behavior distribution over state-action pairs. Crucially, χ can in principle be different from the
policy being followed, effectively allowing to reuse prior data stored in a replay buffer D in the form of (st , at , rt , st+1 )
transitions, used to form the TD-target yi , TD-error δi and loss function eq. 11 via Monte-Carlo (MC) estimates.
While effective in handling large, unstructured state spaces for discrete action-space problems, DQN’s application to
continous control problems proved challenging. Indeed, in the case of high-capacity function approximators such as
neural networks, solving maxat ∈A Qθ (st , at ) at each timestep is simply unfeasible due to the (1) continous nature
of the action space (A ⊂ Rn for some n) and (2) impossibility to express the policy with a cheap (ideally, even
closed-form) formulation, so that max Qθ could be solved analytically. Silver et al. (2014) tackle these fundamental
challenges by using a deterministic function of the state st as policy, µϕ (st ) = at , parametrized by ϕ. Thus, policies
can be iteratively refined updating ϕ along the direction:

(13)

dϕ = Est ∼P(•) ∇ϕ Q(st , at )|at =µϕ (st ) = Est ∼P(•) ∇at Q(st , at )|at =µϕ (st ) · ∇ϕ µ(st )

Provably, eq. 13 is the deterministic policy gradient (DPG) of the policy µϕ (Silver et al., 2014), so that updates
ϕk+1 ← ϕk + αdϕ are guaranteed to increase the (deterministic) cumulative discounted reward, J(µϕ ). Lillicrap
et al. (2019) extended DPG to the case of (1) high-dimensional unstructured observations and (2) continuous action
spaces, introducing Deep Deterministic Policy Gradient (DDPG), an important algorithm in RL and its applications
to robotics. DDPG adopts a modified TD-target compared to eq. 12, by maintaining a policy network used to select
actions, yielding
(14)

yi = Est+1 ∼P(•|st ,at ) rt + γQθi−1 (st+1 , µϕ (st+1 )) .
2 Quote from Mnih et al. (2013). The notation used has slightly been adapted for consistency with the rest of this tutorial.

22
Similarily to DQN, DDPG also employs the same replay buffer mechanism, reusing past transitions over training for
increased sample efficiency and estimate the loss function via MC-estimates.
Soft Actor-Critic (SAC) (Haarnoja et al., 2018) is a derivation of DDPG in the max-entropy (MaxEnt) RL framework,
in which RL agents are tasked with maximizing the discounted cumulative reward, while acting as randomly as
possible. MaxEnt RL (Haarnoja et al., 2017) has proven particularly robust thanks to the development of diverse
behaviors, incentivized by its entropy-regularization formulation. In that, MaxEnt revisits the RL objective J(π) to
specifically account for the policy entropy H(π(•|st )),
T
X
J(π) = E(st ,at )∼χ [rt + αH(π(•|st ))]. (15)
t=0

This modified objective results in the soft TD-target:

(16)

yi = Est+1 ∼P(•|st ,at ) rt + γ Qθi−1 (st+1 , at+1 ) − α log πϕ (at+1 |st+1 ) , at+1 ∼ πϕ (•|st )

Similarily to DDPG, SAC also maintains an explicit policy, trained under the same MaxEnt framework for the
maximization of eq. 15, updated using:

exp(Qπk (st , •))
πk+1 ← arg min D KL π ′
(•|s t ) (17)
π ′ ∈Π Zπk (st )

The update rule provided in eq. 17 optimizes the policy while projecting it on a set Π of tractable distributions (e.g.,
Gaussians, Haarnoja et al. (2017)).

Sample-efficient, data-driven RL Sampling (st , at , rt , st+1 ) from the replay buffer D conveniently allows to approx-
imate expectations for TD-target and TD-error through Monte-Carlo (MC) estimates. The replay buffer D also
proves extremely useful in maintaining a history of previous transitions and using it for training, improving on sample
efficiency. Furthermore, it also naturally provides an entry point to inject offline trajectories recorded by a human
demonstrator into the training process.
Reinforcement Learning with Prior Data (RLPD) (Ball et al., 2023) is an Offline-to-Online RL algorithm leveraging
prior data to effectively accelerate the training of a SAC agent. Unlike previous works on Offline-to-Online RL, RLPD
avoids any pre-training and instead only uses the available offline data Doffline to improve online-learning from scratch.
During each training step, transitions from both the offline and online replay buffers are sampled in equal proportions,
and used in the underlying SAC routine. Together with other implementation details (using LayerNorm layers to
prevent value overestimation, and the use of ensembles techniques to form the TD-target), RLPD proves a particularly
simple yet effective approach to use Doffline for Offline-to-Online RL.

Sample-efficient, data-driven, real-world RL Despite the possibility to leverage offline data for learning, the
effectiveness of real-world RL training is still limited by the need to define a task-specific, hard-to-define reward
function. Further, even assuming to have access to a well-defined reward function, typical robotics pipelines rely
on augmenting propioperceptive inputs with camera streams, and thus even well-defined rewards would need to be
defined starting from unstructured observation—a challenging assumption in practice. In their technical report, Luo
et al. (2025) empirically address the needs (1) to define a reward function and (2) to use it starting from unstructured,
image observations. In particular, Luo et al. (2025, SERL) introduces a suite of tools streamlining training of reward
classifiers c, as well as jointly learn forward-backward controllers to speed up real-world RL.
Reward classifiers are particularly useful in treating complex, dynamic tasks—e.g., folding a t-shirt—for which a
precise reward formulation is arbitrarily complex to obtain, or that do require significant shaping and are more easily
learned directly from demonstrations of success (e+ ) or failure (e− ) states, rather than from a precise formulation
of rt , with a natural target for the reward classifier being r(s) = log c(e+ verts). Furthermore, Luo et al. (2025)
demonstrate the benefits of learning separate (1) forward and (2) backward controllers—parametrized by separate
policies—where (1) the former learns to execute a task to completion and (2) the latter learns to reset the environment
to its initial state from terminal states, thereby aiding training in real-world episodic settings.
Lastly, in order to improve on the robustness of their approach to different goals while maintaing practical scalabil-
ity, Luo et al. (2025) introduced a modified state and action space, expressing proprioperceptive configurations q and
actions q̇ in the frame of the end-effector pose at t = 0. Randomizing the initial pose of the end-effector (s0 ), Luo
et al. (2025) achieved a similar result to that of manually randomizing the environment at every timestep, but with

23
Figure 16 | (A) HIL-SERL allows for real-world training of high performance RL agents by building on top advancements
presented by of SAC, RLPD and SERL. (B) Example of human intervention during a HIL-SERL training process on a real-world
SO-100.

the benefit of maintaining the environment in the same condition across multiple training episodes, achieving higher
scalability of their method thanks to the increased practicality of their approach.
Building on off-policy deep Q-learning with replay buffers, entropy regularization for better exploration, expert
demonstrations to guide learning, and a series of tools and recommendations for real-world training using reward
classifiers (Figure 16), Luo et al. (2024) introduce human interactions during training, learning near-optimal policies
in challenging real-world manipulation tasks in 1-2 hours.
Human-in-the-Loop, Sample Efficient Robot reinforcement Learning (HIL-SERL) (Luo et al., 2024) augments offline-
to-online RL with targeted human corrections during training, and employs prior data to (1) train a reward classifier
and (2) bootstrap RL training on expert trajectories. While offline demonstrations provide the initial dataset seeding
learning and constraining early exploration, interactive, online corrections allow a human supervisor to intervene on
failure modes and supply targeted interventions, greatly aiding the learning process (Luo et al., 2024). Crucially,
human intervention data is stored in both the offline and online replay buffers, differently from the autonomous
transitions generated at training time and stored in the online buffer only. In turn, given an intervention timestep
k ∈ (0, T ), length-K human intervention data {shumank , ahuman
k k+1 , }k=1 is more likely to be sampled
, rkhuman , shuman K

than the data generated online during training, providing stronger supervision to the agent while still allowing for
autonomous learning. Empirically, HIL-SERL attains near-perfect success rates (99%+) on diverse manipulation
tasks within 1-2 hours of training (Luo et al., 2024), underscoring how offline datasets with online RL can markedly
improve stability and data efficiency, and ultimately even allow real-world RL-training.

3.2.1 Code Example: Real-world RL

This example shows how to use the HIL-SERL implementation supported by lerobot. This code example is organized
into four parts: we first show how to train a reward classifier from a custom set of demonstrations, then define
the Actor and Learner components, and finally, we bring them together in a complete script showing how to use
HIL-SERL in practice.
At a higher level, the HIL-SERL architecture (Figure 17) relies on two main components:

• An Actor, running a frozen policy network used to interact with the environment and obtain observations.
Observations are used to both condition the frozen actor in selecting the action to enact, and to form (st , at , rt , st+1 )
transitions that are shared with the Learner. Rewards are inferred using a custom, learned reward classifier trained
on a dataset of offline demonstrations.
• A Learner, used to optimize the policy’s parameters θ for maximum expected return. The learner samples batches

24
Figure 17 | HIL-SERL is a SOTA RL algorithm for training control policies directly in the real-world. Its implementation in
lerobot relies on a decoupled actor-learner architecture, communicating over processes (and possibly networks) with queues
used to share (1) transitions (st , at , rt , st+1 ) and (2) parameters θ.

of offline data from online and offline buffers in equal proportion (Ball et al., 2023), and shares updated parameters
with the Actor.

The HIL-SERL architecture presented in this example can be exclusively run locally, but the implementation in
lerobot also allows the Actor and Learner to run on two separate machines connected by the network.

Code 3: Training a Reward Classifier

[Link]

1 import torch
2
3 from lerobot . datasets . l er o bo t_ d at as e t import LeRobo tDatas et
4 from lerobot . policies . factory import make_policy , m a k e _ p r e _ p o s t _ p r o c e s s o r s
5 from lerobot . policies . sac . reward_model . c o n f i g u r a t i o n _ c l a s s i f i e r import R e w a r d C l a s s i f i e r C o n f i g
6
7 # Device to use for training
8 device = " mps " # or " cuda " , or " cpu "
9
10 # Load the dataset used for training
11 repo_id = " lerobot / e x a m p l e _ h i l _ s e r l _ d a t a s e t "
12 dataset = LeRob otData set ( repo_id )
13
14 # Configure the policy to extract features from the image frames
15 camera_keys = dataset . meta . camera_keys
16
17 config = R e w a r d C l a s s i f i e r C o n f i g (
18 num_cameras = len ( camera_keys ) ,
19 device = device ,
20 # backbone model to extract features from the image frames
21 model_name = " microsoft / resnet -18 " ,
22 )
23
24 # Make policy , preprocessor , and optimizer
25 policy = make_policy ( config , ds_meta = dataset . meta )
26 optimizer = config . g e t _ o p t i m i z e r _ p r e s e t (). build ( policy . parameters ())
27 preprocessor , _ = m a k e _ p r e _ p o s t _ p r o c e s s o r s ( policy_cfg = config , dataset_stats = dataset . meta . stats )

25
28
29
30 # your HF username and model repo id for the reward classifier
31 classifier_id = " lerobot / r e w a r d _ c l a s s i f i e r _ h i l _ s e r l _ e x a m p l e "
32
33 # Instantiate a dataloader
34 dataloader = torch . utils . data . DataLoader ( dataset , batch_size =16 , shuffle = True )
35
36 # Training loop
37 num_epochs = 5
38 for epoch in range ( num_epochs ):
39 total_loss = 0
40 total_ accura cy = 0
41 for batch in dataloader :
42 # Preprocess the batch and move it to the correct device .
43 batch = preprocessor ( batch )
44
45 # Forward pass
46 loss , output_dict = policy . forward ( batch )
47
48 # Backward pass and optimization
49 optimizer . zero_grad ()
50 loss . backward ()
51 optimizer . step ()
52
53 total_loss += loss . item ()
54 total_accu racy += output_dict [ " accuracy " ]
55
56 avg_loss = total_loss / len ( dataloader )
57 avg_accuracy = t otal_a ccurac y / len ( dataloader )
58 print (
59 f " Epoch { epoch + 1}/{ num_epochs } , Loss : { avg_loss :.4 f } , Accuracy : { avg_accuracy :.2 f }% "
60 )
61
62 print ( " Training finished ! " )
63
64 # You can now save the trained policy .
65 policy . push_to_hub ( classifier_id )

Code 4: Defining the Actor

[Link]

1 import m u lt ip r oc e ss in g as mp
2 from queue import Empty
3
4 import torch
5 from pathlib import Path
6
7 from lerobot . envs . configs import H I L S e r l R o b o t E n v C o n f i g
8 from lerobot . policies . sac . modeling_sac import SACPolicy
9 from lerobot . policies . sac . reward_model . m o d e l i n g _ c l a s s i f i e r import Classifier
10 from lerobot . rl . gy m_ m an i pu la t or import mak e_robo t_env
11 from lerobot . teleoperators . utils import TeleopEvents
12
13 MAX_EPISODES = 5
14 M A X _ S T E P S _ P E R _ E P I S O D E = 20
15
16 def ma ke _ po l ic y_ o bs ( obs , device : torch . device = " cpu " ):
17 return {
18 " observation . state " : torch . from_numpy ( obs [ " agent_pos " ]). float (). unsqueeze (0). to ( device ) ,
19 **{
20 f " observation . image .{ k } " :
21 torch . from_numpy ( obs [ " pixels " ][ k ]). float (). unsqueeze (0). to ( device )
22 for k in obs [ " pixels " ]
23 },
24 }

26
25
26 def run_actor (
27 t r a n s i t i o n s _ q u e u e : mp . Queue ,
28 p a r a m e t er s _ q u e u e : mp . Queue ,
29 shutdo wn_eve nt : mp . Event ,
30 policy_actor : SACPolicy ,
31 r e w a r d _ c l a s s i f i e r : Classifier ,
32 env_cfg : HILSerlRobotEnvConfig ,
33 device : torch . device = " mps " ,
34 o u t p u t _ di r e c t o r y : Path | None = None
35 ):
36 " " " The actor process - interacts with environment and collects data .
37 The policy is frozen and only the parameters are updated , popping the most recent ones
38 from a queue . " " "
39 policy_actor . eval ()
40 policy_actor . to ( device )
41
42 r e w a r d _ c l a s s i f i e r . eval ()
43 r e w a r d _ c l a s s i f i e r . to ( device )
44
45 # Create robot environment inside the actor process
46 env , teleop_device = mak e_robo t_env ( env_cfg )
47
48 try :
49 for episode in range ( MAX_EPISODES ):
50 if shu tdown_ event . is_set ():
51 break
52
53 obs , _info = env . reset ()
54 ep isode_ reward = 0.0
55 step = 0
56 e p i s o d e _ t r a n s i t i o n s = []
57
58 print ( f " [ ACTOR ] Starting episode { episode + 1} " )
59
60 while step < M A X _ S T E P S _ P E R _ E P I S O D E and not shutdo wn_eve nt . is_set ():
61 try :
62 new_params = p a r a m e t e r s _ q u e u e . get_nowait ()
63 policy_actor . lo a d_ st a te _d i ct ( new_params )
64 print ( " [ ACTOR ] Updated policy parameters from learner " )
65 except Empty : # No new updated parameters available from learner , waiting
66 pass
67
68 # Get action from policy
69 policy_obs = ma ke _ po li c y_ o bs ( obs , device = device )
70 # predicts a single action , not a chunk of actions !
71 action_tensor = policy_actor . select_action ( policy_obs )
72 action = action_tensor . squeeze (0). cpu (). numpy ()
73
74 # Step environment
75 next_obs , _env_reward , terminated , truncated , _info = env . step ( action )
76 done = terminated or truncated
77
78 # Predict reward
79 p ol i cy _n e xt _o b s = ma k e_ p ol ic y _o bs ( next_obs , device = device )
80 reward = r e w a r d _ c l a s s i f i e r . predic t_rewa rd ( po l ic y_ n ex t _o bs )
81
82 if reward >= 1.0: # success detected ! halt episode
83 if not done :
84 terminated = True
85 done = True
86
87 # In HIL - SERL , human interventions come from the teleop device
88 i s_ i nt er v en ti o n = False
89 if hasattr ( teleop_device , " g e t _ t e l e o p _ e v e n t s " ):
90 # Real intervention detection from teleop device
91 teleop_events = teleop_device . g e t _ t e l e o p _ e v e n t s ()
92 is _i n te rv e nt io n = teleop_events . get ( TeleopEvents . IS_INTERVENTION , False )
93
94 # Store transition with intervention metadata

27
95 transition = {
96 " state " : policy_obs ,
97 " action " : action ,
98 " reward " : float ( reward ) if hasattr ( reward , " item " ) else reward ,
99 " next_state " : policy_next_obs ,
100 " done " : done ,
101 " truncated " : truncated ,
102 " complementary_info ": {
103 " i s_ i nt er v en ti o n " : is_intervention ,
104 },
105 }
106
107 e p i s o d e _ t r a n s i t i o n s . append ( transition )
108
109 episo de_rew ard += reward
110 step += 1
111
112 obs = next_obs
113
114 if done :
115 break
116
117 # Send episode transitions to learner
118 t r a n s i t i o n s _ q u e u e . put_nowait ( e p i s o d e _ t r a n s i t i o n s )
119
120 except K e y b o a r d I n t e r r u p t :
121 print ( " [ ACTOR ] Interrupted by user " )
122 finally :
123 # Clean up
124 if hasattr ( env , " robot " ) and env . robot . is_connected :
125 env . robot . disconnect ()
126 if teleop_device and hasattr ( teleop_device , " disconnect " ):
127 teleop_device . disconnect ()
128 if o u t p u t _ d i r e c t o r y is not None :
129 policy_actor . s av e_ p re tr a in ed ( o u t p u t _ d i r e c t o r y )
130 print ( f " [ ACTOR ] Latest actor policy saved at : { o u t p u t _ d i r e c t o r y } " )
131
132 print ( " [ ACTOR ] Actor process finished " )

Code 5: Defining the Learner

[Link]

1 import m u lt ip r oc e ss in g as mp
2 from queue import Empty , Full
3
4 import torch
5 import torch . optim as optim
6
7 from lerobot . policies . sac . modeling_sac import SACPolicy
8 from lerobot . rl . buffer import ReplayBuffer
9
10 LOG_EVERY = 10
11 SEND_EVERY = 10
12
13 def run_learner (
14 t r a n s i t i o n s _ q u e u e : mp . Queue ,
15 p a r a m e t er s _ q u e u e : mp . Queue ,
16 shutdo wn_eve nt : mp . Event ,
17 policy _learn er : SACPolicy ,
18 online_buffer : ReplayBuffer ,
19 offlin e_buff er : ReplayBuffer ,
20 lr : float = 3e -4 ,
21 batch_size : int = 32 ,
22 device : torch . device = " mps " ,
23 ):
24 " " " The learner process - trains SAC policy on transitions streamed from the actor ,

28
25 updating parameters for the actor to adopt . " " "
26 policy _learn er . train ()
27 policy _learn er . to ( device )
28
29 # Create Adam optimizer from scratch - simple and clean
30 optimizer = optim . Adam ( p olicy_ learne r . parameters () , lr = lr )
31
32 print ( f " [ LEARNER ] Online buffer capacity : { online_buffer . capacity } " )
33 print ( f " [ LEARNER ] Offline buffer capacity : { offl ine_bu ffer . capacity } " )
34
35 training_step = 0
36
37 while not sh utdown _event . is_set ():
38 # retrieve incoming transitions from the actor process
39 try :
40 transitions = t r a n s i t i o n s _ q u e u e . get ( timeout =0.1)
41 for transition in transitions :
42 # HIL - SERL : Add ALL transitions to online buffer
43 online_buffer . add (** transition )
44
45 # HIL - SERL : Add ONLY human intervention transitions to offline buffer
46 i s_ i nt er v en ti o n = \
47 transition . get ( " c o m p l e m e n t a r y _ i n f o " , {}). get ( " i s_ i nt er v en ti o n " , False )
48 if i s_ in t er ve n ti o n :
49 off line_b uffer . add (** transition )
50 print (
51 f " [ LEARNER ] Human intervention detected ! "
52 f " Added to offline buffer ( now { len ( offl ine_bu ffer )} transitions ) "
53 )
54
55 except Empty :
56 pass # No transitions available , continue
57
58 # Train if we have enough data
59 if len ( online_buffer ) >= p olicy_ learne r . config . o n l i n e _ s t e p _ b e f o r e _ l e a r n i n g :
60 # Sample from online buffer ( autonomous + human data )
61 online_batch = online_buffer . sample ( batch_size // 2)
62
63 # Sample from offline buffer ( human demo nstrat ions only )
64 offline_batch = o ffline _buffe r . sample ( batch_size // 2)
65
66 # Combine batches - this is the key HIL - SERL mechanism !
67 batch = {}
68 for key in online_batch . keys ():
69 if key in offline_batch :
70 batch [ key ] = torch . cat ([ online_batch [ key ] , offline_batch [ key ]] , dim =0)
71 else :
72 batch [ key ] = online_batch [ key ]
73
74 loss , _ = pol icy_le arner . forward ( batch )
75
76 optimizer . zero_grad ()
77 loss . backward ()
78 optimizer . step ()
79 training_step += 1
80
81 if training_step % LOG_EVERY == 0:
82 print (
83 f " [ LEARNER ] Training step { training_step } , Loss : { loss . item ():.4 f } , "
84 f " Buffers : Online ={ len ( online_buffer )} , Offline ={ len ( o ffline _buffe r )} "
85 )
86
87 # Send updated parameters to actor every 10 training steps
88 if training_step % SEND_EVERY == 0:
89 try :
90 state_dict = { k : v . cpu () for k , v in pol icy_le arner . state_dict (). items ()}
91 p a r a m e t e r s _ q u e u e . put_nowait ( state_dict )
92 print ( " [ LEARNER ] Sent updated parameters to actor " )
93 except Full :
94 # Missing write due to queue not being consumed ( should happen rarely )

29
95 pass
96
97 print ( " [ LEARNER ] Learner process finished " )

Code 6: Using HIL-SERL

[Link]

1 import m u lt ip r oc e ss in g as mp
2 import signal
3 from typing import Callable
4 from pathlib import Path
5
6 from lerobot . datasets . l er o bo t_ d at as e t import LeRobo tDatas et
7 from lerobot . datasets . utils import h w _ t o _ d a t a s e t _ f e a t u r e s
8 from lerobot . envs . configs import HILSerlProcessorConfig , H I L S e r l R o b o t E n v C o n f i g
9 from lerobot . policies . sac . c o n f i g u r a t i o n _ s a c import SACConfig
10 from lerobot . policies . sac . modeling_sac import SACPolicy
11 from lerobot . policies . sac . reward_model . m o d e l i n g _ c l a s s i f i e r import Classifier
12 from lerobot . rl . buffer import ReplayBuffer
13 from lerobot . rl . gy m_ m an i pu la t or import mak e_robo t_env
14 from lerobot . robots . so 100_fo llower import S O 1 0 0 F o l l o w e r C o n f i g
15 from lerobot . teleoperators . so100_leader import S O 1 0 0 L e a d e r C o n f i g
16
17
18 run_learner : Callable = ... # use / modify the functions defined earlier
19 run_actor : Callable = ...
20
21 " " " Main function - coordinates actor and learner processes . " " "
22
23 device = " mps " # or " cuda " or " cpu "
24 o u t p u t _ d i r e c t o r y = Path ( " outputs / r o b o t _ l e a r n i n g _ t u t o r i a l / hil_serl " )
25 o u t p u t _ d i r e c t o r y . mkdir ( parents = True , exist_ok = True )
26
27 # find ports using lerobot - find - port
28 follower_port = ...
29 leader_port = ...
30
31 # the robot ids are used the load the right calibration files
32 follower_id = ...
33 leader_id = ...
34
35 # A pretrained model ( to be used in - distribution !)
36 r e w a r d _ c l a s s i f i e r _ i d = " lerobot / r e w a r d _ c l a s s i f i e r _ h i l _ s e r l _ e x a m p l e "
37 r e w a r d _ c l a s s i f i e r = Classifier . f r om _ pr et r ai ne d ( r e w a r d _ c l a s s i f i e r _ i d )
38
39 r e w a r d _ c l a s s i f i e r . to ( device )
40 r e w a r d _ c l a s s i f i e r . eval ()
41
42 MAX_EPISODES = 5
43 M A X _ S T E P S _ P E R _ E P I S O D E = 20
44
45 # Robot and environment configuration
46 robot_cfg = S O 1 0 0 F o l l o w e r C o n f i g ( port = follower_port , id = follower_id )
47 teleop_cfg = S O 1 0 0 L e a d e r C o n f i g ( port = leader_port , id = leader_id )
48 processor_cfg = H I L S e r l P r o c e s s o r C o n f i g ( control_mode = " leader " )
49
50 env_cfg = H I L S e r l R o b o t E n v C o n f i g ( robot = robot_cfg , teleop = teleop_cfg , processor = processor_cfg )
51
52 # Create robot environment
53 env , teleop_device = make_r obot_e nv ( env_cfg )
54
55 obs_features = h w _ t o _ d a t a s e t _ f e a t u r e s ( env . robot . observation_features , " observation " )
56 ac ti o n_ fe a tu re s = h w _ t o _ d a t a s e t _ f e a t u r e s ( env . robot . action_features , " action " )
57
58 # Create SAC policy for action selection
59 policy_cfg = SACConfig (

30
60 device = device ,
61 input_ featur es = obs_features ,
62 o ut pu t _f ea t ur es = action_features ,
63 )
64
65 policy_actor = SACPolicy ( policy_cfg )
66 pol icy_le arner = SACPolicy ( policy_cfg )
67
68 d e m o n s t r a t i o n s _ r e p o _ i d = " lerobot / e x a m p l e _ h i l _ s e r l _ d a t a s e t "
69 of fl i ne _d a ta se t = LeRob otData set ( repo_id = d e m o n s t r a t i o n s _ r e p o _ i d )
70
71 # Online buffer : initialized from scratch
72 o n l i n e _ r e p l a y _ b u f f e r = ReplayBuffer ( device = device , state_keys = list ( obs_features . keys ()))
73 # Offline buffer : Created from dataset ( pre - populated it with d emonst ration s )
74 o f f l i n e _ r e p l a y _ b u f f e r = ReplayBuffer . f r o m _ l e r o b o t _ d a t a s e t (
75 l er ob o t_ da t as et = offline_dataset , device = device , state_keys = list ( obs_features . keys ())
76 )
77
78 # Create communication channels between learner and actor processes
79 t r a n s i t i o n s _ q u e u e = mp . Queue ( maxsize =10)
80 p a r a m e t e r s _ q u e u e = mp . Queue ( maxsize =2)
81 shu tdown_ event = mp . Event ()
82
83
84 # Signal handler for graceful shutdown
85 def si gnal_handler ( sig ):
86 print ( f " \ nSignal { sig } received , shutting down ... " )
87 shutdo wn_eve nt . set ()
88
89
90 signal . signal ( signal . SIGINT , s ignal_ handle r )
91 signal . signal ( signal . SIGTERM , signa l_hand ler )
92
93 # Create processes
94 le ar n er _p r oc es s = mp . Process (
95 target = run_learner ,
96 args =(
97 transitions_queue ,
98 parameters_queue ,
99 shutdown_event ,
100 policy_learner ,
101 online_replay_buffer ,
102 offline_replay_buffer ,
103 ),
104 kwargs ={ " device " : device } , # can run on accelerated hardware for training
105 )
106
107 actor_process = mp . Process (
108 target = run_actor ,
109 args =(
110 transitions_queue ,
111 parameters_queue ,
112 shutdown_event ,
113 policy_actor ,
114 reward_classifier ,
115 env_cfg ,
116 output_directory ,
117 ),
118 kwargs ={ " device " : " cpu " } , # actor is frozen , can run on CPU or accelerate for inference
119 )
120
121 le ar n er _p r oc es s . start ()
122 actor_process . start ()
123
124 try :
125 # Wait for actor to finish ( it controls the episode loop )
126 actor_process . join ()
127 s hutdow n_even t . set ()
128 l ea rn e r_ pr o ce ss . join ( timeout =10)
129

31
130 except K e y b o a r d I n t e r r u p t :
131 print ( " Main process interrupted " )
132 s hutdow n_even t . set ()
133 actor_process . join ( timeout =5)
134 l ea rn e r_ pr o ce ss . join ( timeout =10)
135
136 finally :
137 if le a rn er _ pr oc e ss . is_alive ():
138 l e arne r _p ro c es s . terminate ()
139 if actor_process . is_alive ():
140 actor_process . terminate ()

3.2.2 Limitations of RL in Real-World Robotics: Simulators and Reward Design

Despite the advancements in real-world RL training, training RL agents for real-world tasks still suffers from the
following limitations:

• In those instances where real-world training experience is prohibitively expensive to gather (e.g., Tokamak
control (Degrave et al., 2022), Autonomous Stratospehere Navigation (Bellemare et al., 2020))in-simulation training
is often the only viable option. However, high-fidelity simulators for real-world problems can be difficult to build
and maintain, especially for contact-rich manipulation and tasks involving deformable or soft materials.
• Reward design is a fundamental source of brittleness in real-world RL pipelines. While shaping dense rewards
is often necessary to guide exploration in long-horizon tasks, the process is error-prone and heavily reliant on
human expertise and intuition. Poorly tuned terms can lead to specification gaming or convergence to local optima,
making reward shaping a critical challenge for applying RL in practice. Sparse rewards that only signal successful
trajectories can avoid these pitfalls but typically result in much slower learning due to reduced supervision.

Advances in learning to act from potentially large corpora of human demonstrations via Behavioral Cloning (BC)
address both of these concerns. Although suffering from an inherent suboptimality—imitation learning can at most
match the performance level of the demonstrator—learning to reproduce expert demonstrations via BC has proven
increasingly competitive and practical, bypassing the need for simulated environments and hard-to-define reward
functions.

32
Figure 18 | (A) Average (with standard deviation) evolution of the actuation levels over the first 5 recorded episodes in
lerobot/svla_so101_pickplace. Proprioperceptive states provide invaluable to determine the robot’s state during an episode.
(B) Camera frames are also recorded alongside measurements on the robot’s state, capturing information about the robot’s
interaction with its environment.

4 Robot (Imitation) Learning

The best material model for a cat
is another, or preferably the same
cat

Norbert Wiener

TL;DR

Behavioral Cloning provides a natural platform to learn from real-world interactions without the need to
design any reward function, and generative models prove more effective than point-wise policies at dealing with
multimodal demonstration datasets.

Learning from human demonstrations provides a pragmatic alternative to the RL pipeline discussed in Section 3.
Indeed, especially in real-world robotics, online exploration is typically costly and potentially unsafe, and designing
(dense) reward signals is a brittle and task-specific process. Further, even success detection itself often requires bespoke
instrumentation, while episodic training demands reliable resets—all factors complicating training RL algorithms
on hardware at scale. Behavioral Cloning (BC) sidesteps these constraints by casting control an imitation learning
problem, leveraging previously collected expert demonstrations to anchor the learned autonomous behavior. Most
notably, by learning-to-imitate, autonomous systems naturally adhere to the objectives, preferences, and success
criteria implicitly encoded in the data, which reduces early-stage exploratory failures and obviates hand-crafted reward
shaping altogether.
(i) (i)
Formally, let D = {τ (i) }Ni=1 be a set of expert trajectories, with τ
(i)
= {(ot , at )}Tt=0
i
representing the i-th length-Ti
trajectory in D, ot ∈ O denoting observations (e.g., images and proprioception altogether), and at ∈ A the expert
actions. Typically, observations o ∈ O consist of both image and proprioperceptive information, while actions a ∈ A
represent control specifications for the robot to execute, e.g. a joint configuration. Note that differently from Section 3,
in the imitation learning context D denotes an offline dataset collecting N length-Ti reward-free (expert) human
trajectories τ (i) , and not the environment dynamics. Similarily, in this section τ (i) represent a length-Ti trajectory
of observation-action pairs, which crucially omits entirely any reward information. Figure 18 graphically shows
trajectories in terms of the average evolution of the actuation on the 6 joints of a teleoperated SO-100 manipulator.
Notice how proprioperceptive states are captured jointly with camera frames over the course of the recorded episodes,
providing a unified high-frame rate collection of both image and joint teleoperation data. Figure 19 shows (ot , at )-pairs
for the same dataset, with the actions performed by the human expert illustrated alongside the corresponding
observation. In principle, (expert) trajectories τ (i) can have different lengths since demonstrations might exhibit
multi-modal strategies to attain the same goal, resulting in multiple, different behaviors.
Behavioral Cloning (BC) (Pomerleau, 1988) aims at producing synthetic behaviors by learning the mapping from

33
Figure 19 | Sample observations and action pairs over the course of a given trajectory recorded in lerobot/svla_so101_
pickplace. Observations, comprising of both proprioperceptive and visual information, are recorded alongside the configuration
of a second, leader robot controlled by a human expert, providing complete information for regressing actions given observations.

observations to actions, and in its most natural formulation can be effectively tackled as a supevised learning problem,
consisting of learning the (deterministic) mapping f : O 7→ A, at = f (ot ) by solving

min E(ot ,at )∼p(•) L(at , f (ot )), (18)

given an arbitrary risk function L : A × A 7→ R, L(a, a′ ).

Typically, the expert’s joint observation-action distribution p : O × A 7→ [0, 1] is assumed to be unknown, in keeping
with a classic Supervised Learning (SL) framework3 . However, differently from standard SL assumptions, the samples
collected in D—realizations of the underlying p—are not i.i.d., as expert demonstrations are collected sequentially in
the form of trajectories. In practice, this aspect can be partially mitigated by considering pairs in a non-sequential
order—shuffling the samples in D—so that the expected risk under p can be approximated using MC estimates,
although these estimates may in general be less accurate. Another strategy to mitigate the impact of regressing
over non-i.i.d. samples relies on the possibility of interleaving BC and data collection (Ross et al., 2011, DAgger),
aggregating multiple datasets iteratively. However, because we only consider the case where a single offline dataset D
of trajectories is available and no more data can be collected, DAgger falls out of our scope.
Despite the inherent challenges of learning from non-i.i.d. data, the BC formulation presents several operational
advantages in robotics. First, training happens offline and naturally accomodates for expert, demonstration data,
hereby severily limiting exploration risks by preventing the robot from performing dangerous actions altogether,
by anchoring action in imitation. Second, reward design is entirely unnecessary in BC, as demonstrations already
reflect human intent. The absence of rewards also prevents the risk of misalignment and specification gaming (reward
hacking), otherwise inherent in purely reward-based RL (Heess et al., 2017). Third, because expert trajectories encode
terminal conditions, success detection and resets are implicit in the dataset. Finally, empirical evidence suggests the
performance of BC scales naturally with growing corpora of demonstrations collected across tasks, embodiments, and
environments. Nonetheless, BC can, in principle, only reproduce behaviors that are at best as good as those of the
demonstrator, and therefore offers no remedy for the suboptimal decisions that humans may enact. This limitation is
particularly problematic in sequential decision-making tasks where expert demonstrations are scarce–—either because
data collection is costly or because human performance is inherently suboptimal. Yet, many robotics applications still
benefit from relatively inexpensive pipelines for collecting high-quality human-generated trajectories, justifying the
use of BC in such settings.
While conceptually elegant, point-estimate policies f : O 7→ A learned by solving eq. 18 have been observed to
suffer from (1) compounding errors (Ross et al., 2011) and (2) poor fit to multimodal distributions (Florence et al.,
2022; Ke et al., 2020). Figure 20 illustrates these two key issues related to learning explicit policies (Florence
et al., 2022). Besides sequentiality in D, compounding errors due to covariate shift may also prove catastrophic,
as even small ϵ-prediction errors 0 < ∥µ(ot ) − at ∥ ≤ ϵ can quickly drive the policy into out-of-distribution states,
incuring in less confident generations and thus compounding errors (Figure 20, left). Moreover, point-estimate policies
typically fail to learn multimodal targets, which are very common in human demonstrations solving real-world robotics
problems, as multiple trajectories can be equally as good towards the accomplishment of a goal (e.g., symmetric
grasps, Figure 20, right). In particular, unimodal regressors tend to average across modes, yielding indecisive or
3 Throughout, we will adopt the terminology and notation for SL used in Shalev-Shwartz and Ben-David (2014)

34
Figure 20 | Point-wise policies suffer from limitations due to (A) covariate shifts and (B) poor approximation of multimodal
demonstrations. (A) Small errors may drive the policy out of distribution, incuring in a vicious circle ultimately resulting in
failure. (B) Both modes of reaching for a target object in the scene—either left or right-first—are equally as good and thus
equally as likely to be present in a dataset of human demonstrations, ultimately resulting in multimodal demonstrations.

Figure 21 | Intuitively, latent variable in a single latent model may contain information regarding the task being performed,
which directly results in the likelihood of the same observation-action pair being different for two different tasks. When (A)
picking a block the likelihood of a wide gripper’s opening should be higher than narrower one, while it should be the opposite
when (B) pushing the block.

even unsafe commands (Florence et al., 2022). To address poor multimodal fitting, Florence et al. (2022) propose
learning the generative model p(o, a) underlying the samples in D, rather than explicitly learning a prediction function
f : a = f (o).

4.1 A (Concise) Introduction to Generative Models

Generative Models (GMs) aim to learn the stochastic process underlying the very generation of the data collected, and
typically do so by fitting a probability distribution that approximates the unknown data distribution, p. In keeping
with the GM literature, p(x) ← P(x), x ∼ p. In the case of BC, the unknown data distribution p may represent
the expert’s joint distribution over (o, a)-pairs. Thus, given a finite set of N pairs D = {(o, a)i }N
i=0 available as an
imitation learning target (and thus assumed to be i.i.d.), GMs seek to learn a parametric distribution pθ (o, a) such
that (1) new samples (o, a) ∼ pθ (•) resemble those stored in D, and (2) high likelihood is assigned to the observed
regions of the unobservable p. Likelihood-based learning provides a principled training objective to achieve both goals,
and it is thus extensively used in GMs (Prince, 2023).

35
Figure 22 | (A) The latent variable model in a robotics application regulates influence between observed (o, a) variables and
an unobservable latent variable. (B) VAEs approximate exact latent variable models by means of variational inference.

4.1.1 Variational Auto-Encoders

A common inductive bias used in GM posits samples (o, a) are influenced from an unobservable latent variable z ∈ Z,
resulting in: Z
p(o, a) = p(o, a|z)p(z) (19)
supp(Z)

Intuitively, in the case of observation-action pairs (o, a) for a robotics application, z could be interpreted as some high
level representation of the underlying task being performed by the human demonstrator. In such case, treating p(o, a)
as a marginalization over supp(Z) of the complete joint distribution p(o, a, z) natively captures the effect different
tasks have on the likelihood of observation-action pairs. Figure 21 graphically illustrates this concept in the case of a
(A) picking and (B) pushing task, for which, nearing the target object, the likelihood of actions resulting in opening
the gripper—the higher q6 , the wider the gripper’s opening—should intuitively be (A) high or (B) low, depending
on the task performed. While the latent space Z typically has a much richer structure than the set of all actual
tasks performed, eq. 19 still provides a solid framework to learn joint distribution conditioned on unobservable yet
relevant factors. Figure 22 represents this latent-variable framework in the context of a robotics application: the true,
z-conditioned generative process assigns likelihood p((o, a)|z) to the single (o, a)-pair. Using Bayes’ theorem, one can
reconstruct the posterior distribution on supp(Z), qθ (z|o, a) from the likelihood pθ (o, a|z), prior pθ (z) and evidence
pθ (o, a). VAEs approximate the latent variable model presented in eq. 19 using an approximate posterior qϕ (z|o, a)
while regressing parameters for a parametric likelihood, pθ (o, a|z) (Figure 22).
Given a dataset D consisting of N i.i.d. observation-action pairs, the log-likelihood of all datapoints under θ (in
Bayesian terms, the evidence pθ (D)) can be written as:
N
X
log pθ (D) = log pθ ((o, a)i ) (20)
i=0
N Z
X
= log pθ ((o, a)i |z)p(z) (21)
i=0 supp(Z)
N Z
X qθ (z|(o, a)i )
= log · pθ ((o, a)i |z)p(z) (22)
i=0 supp(Z) θ
q (z|(o, a)i )
N
X p(z)
= log Ez∼qθ (•|(o,a)i ) · pθ ((o, a)i |z) , (23)
i=0
qθ (z|(o, a)i )

qθ (z|(o,a)i )
where we used eq. 19 in eq. 20, multiplied by 1 = qθ (z|(o,a)i ) in eq. 21, and used the definition of expected value in
eq. 23.
In the special case where one assumes distributions to be tractable, pθ (D) is typically tractable too, and maxθ log pθ (D)
provides a natural target for (point-wise) infering the unknown parameters θ of the generative model. Unfortunately,
eq. 23 is rarely tractable when the distribution p is modeled with approximators such as neural networks, especially
for high-dimensional, unstructured data.
In their seminal work on Variational Auto-Encoders (VAEs), Kingma and Welling (2013) present two major contribu-
tions to learn complex latent-variable GMs from unstructured data, proposing (1) a tractable, variational lower-bound

36
to eq. 23 as an optimization target to jointly learn likelihood and posterior and (2) using high-capacity function
approximators to model the likelihood pθ (o, a|z) and (approximate) posterior distribution qϕ (z|o, a) ≈ qθ (z|o, a).
In particular, the lower bound on eq. 23 (Evidence LOwer Bound, ELBO) can be derived from eq. 23 applying
Jensen’s inequality—log E[•] ≥ E[log(•)]—yielding:
N
X p(z)
(24)

log pθ (D) ≥ Ez∼qθ (•|(o,a)i ) log pθ ((o, a)i |z) + Ez∼qθ (•|(o,a)i ) log
i=0
qθ (z|(o, a)i )
N
X
Ez∼qθ (•|(o,a)i ) log pθ ((o, a)i |z) − DKL qθ (z|(o, a)i )∥p(z) (25)

=
i=0

The true, generally intractable, posterior qθ (z|o, a) prevents computing both the expectation and KL divergence
terms in eq. 25, and therefore Kingma and Welling (2013) propose deriving the ELBO using an approximate posterior
qϕ (z|o, a), resulting in the final, tractable, ELBO objective,
N
X
ELBOD (θ, ϕ) = Ez∼qϕ (•|(o,a)i ) log pθ ((o, a)i |z) − DKL qϕ (z|(o, a)i )∥p(z) (26)

i=0

From Jensen’s inequality, maximizing ELBO results in maximizing the log-likelihood of the data too, thus providing a
natural, tractable optimization target. Indeed, expectations can be estimated using MC estimates from the learned
distributions in eq. 26, while the KL-divergence term can typically be computed in closed-form (1) modeling qϕ as a
Gaussian qϕ (z|o, a) = N µϕ (o, a), Σϕ (o, a) with learned mean vector µϕ (o, a) and learned variance-covariance matrix

Σϕ (o, a) and (2) imposing a standard Gaussian prior on the latent space, p(z) = N (0, I).
An intuitive explanation of the learning dynamics of VAEs can be given considering the equivalent case of minimizing
the negative ELBO, which admits the particularly interpretable factorization (considering, without loss of generality,
only one (o, a) ∼ D):

min −ELBO(o,a)∼D (θ, ϕ) = min Lrec (θ) + Lreg (ϕ), (27)

θ,ϕ θ,ϕ

Lrec (θ) = Ez∼qϕ (•|o,a ) log pθ (o, a|z) (28)

Lreg (ϕ) = DKL qϕ (z|o, a)∥p(z) . (29)

For any given (o, a) pair, the expected value term in eq. 28 is typically computed via MC estimates, resulting in
n
1X
−Ez∼qϕ (•|o,a) log pθ (o, a|z) = Lrec ≈ −

log pθ (o, a|zi ).
n i=0

Assuming pθ (o, a|z) to be parametrized with an isotropic Gaussian distribution with mean µθ (z) ∈ Rd and variance
σ 2 , the log-likelihood thus simplifies to:
n
1 2 d 1X 2
log p(o, a|zi ) = − (o, a) − µθ (zi ) − log(2πσ 2 ) =⇒ Lrec ≈ (o, a) − µθ (zi )
2σ 2 2 2 n i=0 2

In practice, it is common to approximate the learned likelihood pθ (o, a|z) with a parametric distribution (e.g.,
Gaussian) whose parameters are given by a learned coefficient vector derived from µθ (z), z ∼ p(•). Under this
formulation, learning a VAE amounts to (1) reconstructing the examples in D by minimizing (1) the reconstruction
loss Lrec —a standard supervised learning objective for regression—while (2) regularizing the latent representation by
minimizing Lreg . The latter enforces information compression, since with the common prior choice p(z) = N (0, I) in
eq. 29, the regularizer constrains the posterior and thereby limits the expressivity of qϕ (z|o, a).

4.1.2 Diffusion Models

VAEs approximate probability distributions via a single latent variable model, assuming the underlying unknown
distribution can be factored according to eq. 19, and solve the variational-inference problem of jointly learning
the likelihood pθ and (approximate) posterior qϕ for such model. In that, the unknown data distribution p(o, a) is
effectively approximated via Z p(z)pθ (o, a|z), and the underlying generative process reproduced by (1) sampling a
R

37
Figure 23 | HMLV models posit the data generation process is influenced by a stack of Markov-dependent latent variables,
with samples from the posterior distribution being progressively higher up in the hierarchy.

latent variable and (2) learning to decode it into a high-likelihood sample under the (unknown) p(o, a). Diffusion
Models (DMs) (Ho et al., 2020) are another class of GMs which treat the similar problem of approximating an
underlying unknown data distribution—variational inference—by partially extending VAEs to the case where multiple
latent variables influence each other and the generative process underlying o, a itself. In particular, DMs posit the
generative process can be decomposed to a series of piece-wise (Markovian) interactions between (latent) variables
(Figure 23), resulting in
Z Z Z
p( o, a ) = ... p(z0 , z1 , . . . zT ) (30)
|{z} supp(Z0 ) supp(Z1 ) supp(ZT )
=z0
T
Y
p(z0 , z1 , . . . zT ) = p(zT ) p(zt−1 |zt ), (31)
t=1

where we explicitly showed the marginalization over the multiple latents in eq. 30, and used the law of conditional
probability and Markov property in eq. 31. Also, for ease of notation, we will refer to observation-action pairs o, a as
z0 .
Similar to VAEs, it is generally not possible to assign an exact interpretation to the latent variables. Nevertheless, a
reasonable application-driven intuition is that Hierarchical Markov Latent Variable (HMLV) models, by capturing
hierarchical and decoupled interactions among latent variables, can reflect the different resolutions at which conditioning
factors intervene. For example, in a robotics setting, one might naturally distinguish between high-level trajectory
planning (higher up in the hierarchy, t → T ) and fine-grained motion adjustments (closer to empirical observations,
t → 0). In that, HMLV models thus provide a framework to perform variational inference via multiple, sequential
sampling steps from different higher level distributions instead of approximating the generative process with a
single-latent variable
√ model. DMs are a particular instantiation of HMLV models for which the posterior is fixed to
q(zt |zt−1 ) = N (zt 1 − βt , βt I), for a given βt ∈ R+ . In practice, βt is used to iteratively reduce the signal-to-noise
ratio along the latents’ hierarchy, similarily to how a diffusion process influences the information of a physical system.
Just like VAEs, DMs attemp to learn to reproduce an underlying data distribution p(o, a) given a collection of
i.i.d. samples approximating the model posited to have generated the data in the first place (eq. 30). Similarily to
VAEs, DMs approximate the process of sampling from the unknown p(o, a) by (1) sampling from an easy-to-sample
distribution (e.g., Gaussian) and (2) learning to reconstruct high-likelihood samples under the unknown distribution.
However, in stark contrast with VAEs, the easy-to-sample distribution contains no mutual information regarding the
data distribution p(o, a). Crucially, as no information from the sample (o, a) (denoted as z0 ≡ (o, a) for simplicity of
notation) is assumed to be propagated throughout the chain of latents, the posterior q(zt |zt−1 ) assumes a relatively
amicable structure in DMs, reducing complexity. The true likelihood p(zt−1 |zt ) is instead typically approximated using
the parametrization pθ (zt−1 |zt ). In that, the information contained in the unknwon data distribution is reconstructed
via a process in which samples from a fixed distribution are iteratively turned into (ideally) high-likelihood samples
under p(o, a)—a process referred to as denoising.

38
Under such model, we can express the log-likelihood of an arbitrary sample z0 as:
Z
log pθ (z0 ) = log pθ (z0 , z1 , z2 , . . . zT ) (32)
supp(Z1 )×supp(Z2 )×···×supp(ZT ) | {z }
z0:T
pθ (z0:T ) · q(z1:T |z0 )
Z
= log (33)
supp(Z1:T ) q(z1:T |z0 )

pθ (z0:T )
= log Ez1:T ∼q(•|z0 ) (34)
q(z1:T |z0 )

pθ (z0:T )
≥ Ez1:T ∼q(•|z0 ) log (35)
q(z1:T |z0 )
QT
p(zT ) t=1 pθ (zt−1 |zt )
= Ez1:T ∼q(•|z0 ) log QT (36)
t=1 q(zt |zt−1 )
QT
p(zT ) · pθ (z0 |z1 ) t=2 pθ (zt−1 |zt )
= Ez1:T ∼q(•|z0 ) log QT −1 (37)
q(zT |zT −1 ) t=1 q(zt |zt−1 )
QT −1
p(zT ) · pθ (z0 |z1 ) t=1 pθ (zt |zt+1 )
= Ez1:T ∼q(•|z0 ) log QT −1 (38)
q(zT |zT −1 ) t=1 q(zt |zt−1 )
TY−1
p(zT ) · pθ (z0 |z1 ) pθ (zt |zt+1 )
= Ez1:T ∼q(•|z0 ) log + Ez1:T ∼q(•|z0 ) log (39)
q(zt |zt−1 ) t=1
q(zt |zt−1 )
TX −1
p(zT ) pθ (zt |zt+1 )
(40)

= Ez1:T ∼q(•|z0 ) log pθ (z0 |z1 ) + Ez1:T ∼q(•|z0 ) log + Ez1:T ∼q(•|z0 ) log
q(zT |zT −1 ) t=1
q(zt |zt−1 )
TX −1
p(zT ) pθ (zt |zt+1 )
= Ez1 ∼q(•|z0 ) log pθ (z0 |z1 ) + EzT −1:T ∼q(•|z0 ) log + Ezt−1:t+1 ∼q(•|z0 ) log
q(zT |zT −1 ) t=1
q(zt |zt−1 )
(41)
= Ez1 ∼q(•|z0 ) log pθ (z0 |z1 ) − EzT −1 ∼q(•|z0 ) DKL (q(zT |zT −1 )∥p(zT )) (42)

T
X −1
E(zt−1 ,zt+1 )∼q(•|z0 ) DKL (q(zt |zt−1 )∥pθ (zt |zt+1 )) ,

−
t=1

1:T |z0 )
where we: used eq. 30 and multiplied by 1 = q(z
q(z1:T |z0 ) in eq. 33; used Jensen’s inequality in eq. 35; used the law of
conditional probability for both numerator and denominator in eq. 36; stepped forward and backward the products in
the numerator and denominator products in eq. 37, respectively; reindexed the product terms in eq. 38; removed
out-of-expectation variables in eq. 41; used the defintion of KL-divergence in eq. 42. In turn, eq. 42 provides an
optimization target to learn pθ solving maxθ log pθ (D).
In their seminal work on using DMs for variational inference, Ho et al. (2020) introduce major contributions regarding
solving minθ − log pθ (z0 ). In particular,
√ Ho et al. (2020) exclusively adopt a fixed, isotropic Gaussian posterior
in the form of q(zt |zt−1 ) = N ( 1 − βt zt−1 , βt I). The choice of adopting Gaussians has profound implications
on the generative process modeled. Indeed, under the (mild) assumption that the variance is sufficiently small
βt ≤ η, η ∈ R+ , Sohn et al. (2015) proved that the likelihood p(zt−1 |zt ) is Gaussian as well, which allows for the
particularly convenient parametrization of the approximate likelihood pθ (zt−1 |zt ) = N (µθ (zt , t), Σθ (zt , t)), t ∈ [1, T ],
as well as for closed-form tractability of the KL-divergence terms in eq. 42. Further, the posterior’s
√ structure also
enables the analytical
Qt description of the distribution of the t-th latent variable, q(zt |z0 ) = N ( ᾱt z0 , (1 − ᾱt )I), with
αt = 1 − βt , ᾱt = k=1 αk , conveniently preventing iterative posterior sampling simplifying computing eq. 42. It
follows:
T
X −1
∇θ log pθ (z0 ) = Ez1 ∼q(•|z0 ) ∇θ log pθ (z0 |z1 ) − Ezt−1 ,zt+1 ∼q(•|z0 ) ∇θ DKL (q(zt |zt−1 )∥pθ (zt |zt+1 ), (43)
t=1

where the former term is equivalent to the reconstruction term in eq. 27 and the latter term can be obtained in closed
form.
Besides mathematical tractability of eq. 43, adopting Gaussian posteriors allows for a particularly intuitive inter-
pretation of the training dynamics of DMs (Permenter and Yuan, 2024). As the hierarchical latent variables are

39
Figure 24 | DMs iteratively corrupt samples (left) from an unknown distribution into a quasi-standard Gaussian (center),
learning the displacement field (right) that permits to reconstruct samples from the unknown target distribution by iteratively
denoising samples of a tractable, easy-to-sample distribution.

40
Figure 25 | A joint action-observation distribution, in the simplified case where the observation is the elbow-flex actuation in
a SO-100, and the action is the recorded position for the same joint from the teleoperator arm. The motion recorded being
teleoperated, the points distribute along a the diagonal.

repeatedly corrupted by applying increasingly more Gaussian noise, they progressively lose information about the
original (unknown) sample z0 , converging toward a standard Gaussian which eventually contains no information at all
(Figure 24). Figure 24 illustrates this process on a simplified, bidimensional observation-action distribution, where we
considered o = q2 and a = q2h , with q2 denoting the robot’s elbow flex actuation and q2h the corresponding human
teleoperator’s elbow flex. Because the recorded behavior is teleoperated, measurements mostly distribute along the
line a = o + η, η ∼ N (0, 1), with η-variability accouting for minor control inconsistencies (Figure 25). Notice how
corrupted samples distribute differently from the most reasonable structure a ≃ o, further underscoring how diffusion
corrupts both the individual samples and the global distribution (Figure 24, left and center). In this, using Gaussian
posteriors—i.e., adding Gaussian noise—effectively simulates a Brownian motion for the elements in the distribution’s
support (in Figure 24, O × A), whereby information diffuses away from the samples. Comparing the diffused samples
to the original data points, one can derive an estimate of the total displacement induced by the diffusion process,
and, under the assumption that the likelihood of the totally diffused samples is low under the original unknown data
distribution, one can effectively approximate the unkwown distribution by learning to reverse such displacement. This
key intuition allows to write a simplified training objective4 :
√ √
L(θ) = Et,z0 ,ϵ ∥ϵ − ϵθ ( ᾱt z0 + ϵ 1 − ᾱt , t)∥2 , t ∼ U({1, . . . , T }), z0 ∼ D, ϵ ∼ N (0, I). (44)

In this simplified (minimization) objective, the optimization process differs from eq. 42 in that, rather than maximizing
pθ directly, the parameters θ of the pairwise likelihood pθ (zt−1 |zt ) are adjusted to predict the total displacement ϵ for
a randomly long (t ∼ U ({1, . . . , T })) diffusion process starting from a sample of the target distribution.
By learning the total displacement from a generally, uninformative corrupted sample obtained diffusing information
and a sample from an unknown distribution Ho et al. (2020) show that one can approximate the underlying distribution
reversing the displacement, denoising samples. Interestingly, under the hypothesis that real-world data belongs to a
single, higher-dimensional manifold (Manifold Hypothesis), Permenter and Yuan (2024) show that diffusion learns the
gradient of a distance function from any off-point manifold (such as perturbed, uniformative samples), and the data
manifold itself. Following this gradient—i.e., denoising a sample from an uninformative distribution—corresponds to
projecting back into the manifold, yielding a procedure to sample from unknown distributions by means of Euclidean
projection. Indeed, under the assumption that pθ (zt−1 |zt ) is Gaussian, sampling zt−1 ∼ pθ (•|zt ) corresponds to
computing:

1 βt
zt−1 = √ zt − √ ϵθ (zt , t) + σt ϵ, ϵ ∼ N (0, I), (45)
αt 1 − ᾱt
thus showing that the lower-level latent variables in a DM can be obtained by iteratively removing noise from the
one-step higher order variable, using the noise regressor ϵθ (zt , t) learned minimizing eq. 44.

4.1.3 Flow Matching

The posterior parametrization adopted by DMs proved traditionally effective, yet it raised concerns circa its efficiency
at inference time, where a possibly large number (hundreds) of compute-expensive denoising steps are needed in
order to recover a sample from the target distribution. Flow Matching (FM) (Lipman et al., 2023) extends DMs
4 See Luo (2022, "Three equivalent interpretations") for a complete derivation

41
Figure 26 | Probability distributions can be modified differently by applying different vector fields, inducing different
flows of mass across
√ the same support (top versus bottom, using two different time-invariant 2D-fields u1 (x, y) = (x, 0) and
√
u2 (x, y) = (x/ 2, y/ 2)). Notice time flows continuously in [0, 1]. FM models learn to approximate a target vector field,
thereby producing arbitrary (goal) transformations of an easy-to-sample initial distribution.

to the general case of arbitrary likelihood and posteriors, and in this defines a superseding class of GMs providing
a unified framework for learning continuous transformations between distributions, encompassing and generalizing
DMs. Instead of a stochastic, discrete, multi-step denoising process, FM aims to learn a deterministic, continuous,
differentiable flow ψ : [0, 1]×Z 7→ Z, formalized starting from a (possibly time-dependent) vector field v : [0, 1]×Z 7→ Z
transporting over time samples from a simple prior distribution p0 —e.g., a standard Gaussian—to a more complex,
typically unknown data distribution p1 . In this, FM accomodates for arbitrary intermediate distributions, breaking
free from the particular case where posterior and likelihood are exclusively Gaussians. Note also how FM models time
t ∈ [0, 1] to be varying continuously while moving away from an easy-to-sample distribution p0 towards the unknown
data-distribution, p1 . This results in a continuous (and deterministic) trajectory at inference, which is in practice
more efficient compared to following stochastic paths like in DMs. Formally, FM can be fully characterized by an
ordinary differential equation (ODE) relating instantaneous variations of flows with the underlying vector field, and
hence providing complete trajectories over the distributions’ support when integrating over time,
d
ψ(z, t) = v(t, ψ(t, z)), (46)
dt
ψ(0, z) = z. (47)

In practice, flow models learn to approximate these dynamics by estimating a vector field v that matches the true,
unknown u, so that the induced flows ψ can approximate the ideal trajectories ψ ∗ .
FM proved very effective in a variety of applications, ranging from image (Esser et al., 2024) and video genera-
tion (Polyak et al., 2025) to robotics control (Black et al., 2024). Most notably, in their introductory work on FM for
GM, Lipman et al. (2023) show how DMs can be seen as a specific instance of FM where the conditional target vector
field v learned by the noise regressor εθ corresponds to:
d
dt α(1− t) 1
Rt
u(t, z|z0 ) = (α(1 − t)z − z0 ), α(t) = e− 2 0
β(s)ds
, ∀z0 ∈ D. (48)
1 − (α(1 − t))2

Conditional vector fields are defined not only over their argument z and time t, but do also vary with respect to an
auxiliary variable z0 , thereby extending the standard notion of a vector field to incorporate additional conditioning.
Note that the traditional discrete-time noise-scheduler {βt }Tt=0 is now generalized to a continuous map β : [0, 1] 7→ R+ .
Crucially, Lipman et al. (2023) prove that by exclusively optimizing the vector field for individual data points z0 ∈ D,
one also retrieves the optimal flow to morph the entire support of the initial distribution p0 into p1 s.t.D ∼ p1 .
While the noising schedule of DMs results in a stochastic resembling a random (Brownian) walk, FM allows for more
general—potentially, deterministic—likelihood and posterior parametrization. In the FM literature the likelihood and
posterior probabilty densities defined along a HMLV model are typically referred to as a probability path, where the
distributions for successive adjacent transitions in the HMLV model are related by the (normalized) flow between them
(Figure 26). The inherent flexibility of FM is one of their key advantages over DMs, as it opens up the possibility of

42
Figure 27 | Compared to diffusion, flow matching distorts distribution along a less randomic pattern, resulting in a clearer
interpolation between source and target distribution. The visualization shows an example comparison between these two
methods on joint distribution of robot observations and actions over T = 50 steps.

learning more efficient paths. For instance, one can design probability paths inspired by Optimal Transport (OT), a
mathematical framework concerned with characterizing the most efficient morphings between probability distributions.
Probability paths obtained through OT paths tend to be straighter than diffusion paths (Figure 27), which can lead
to faster and more stable training, as well as empirically result in higher-quality generations with fewer denoising
steps at inference time. In particular, by avoiding unnecessary backtracking associated with the inherent stochastic
nature of both the noising and denoising process in DMs, test-time compute is typically significantly reduced in FM,
while retaining comparable results (Lipman et al., 2023).
In practice, FM can be applied to generative modeling by learning a vector field regressor vθ (z, t) to approximate a
given target vector field u(t, z). In the particular case of DMs, u(t, z) is defined as in eq. 48, while in priciple the
target vector field can be learned to induce an arbitrary mass displacement, or fixed according to OT. Given a sample
from the data distribution z1 ∼ p1 and a sample from an easy-to-sample prior z0 ∼ p0 , Conditional FM (CFM) defines
a simple path between them using linear interpolation between samples zt = (1 − t)z0 + tz1 , which in turn results in
the target vector field u(t, zt ) = z1 − z0 . FM models can then be trained with a simple regression objective defined as:

L(θ) = Et,z0 ,z1 ∥vθ ((1 − t)z0 + tz1 , t) − (z1 − z0 )∥2 , t ∼ U([0, 1]), (49)

where z0 ∼ p0 (•) and z1 ∼ p1 (•). Note how in eq. 49—differently from eq. 44—time is assumed to be varying
continuously t ∼ U([0, 1]) rather than discretely t ∼ U({0, ∆t, 2∆t, . . . , 1}), a key property of flow-based models.
Therefore, the objective in eq. 49 directly regresses the learned vector field onto the simple, straight path connecting
a point from the prior and a point from the data, providing a simulation-free training procedure that is both stable
and efficient. At inference time, samples are generated by starting with z0 ∼ p0 and iteratively refined according to
dt = vθ (zt , t) for t ∈ [0, 1]—an operation that can be numerically carried out with standard ODE solvers, and that in
dz

practice is often carried out numerically via forward-Euler integrating over tens of denoising steps.

4.2 Action Chunking with Transformers

While GMs prove useful in learning complex, high-dimensional multi-modal distributions, they do not natively address
the compouding errors problem characteristic of modeling online, sequential predictions. In Action Chunking with
Transformers (ACT), Zhao et al. (2023) present an application of VAEs to the problem of learning purely from
offline trajectories, and introduce a simple, yet effective method to mitigate error compounding, learning high-fidelity
autonomous behaviors via BC. Drawing inspiration from how humans plan to enact sequences of actions at:t+k instead
of single actions at , Zhao et al. (2023) propose learning a GM on a dataset of input demonstrations by modeling
chunks of multiple actions directly. Besides contributions to learning high-performance autonomous behaviors, Zhao
et al. (2023) also introduce hardware contributions in the form of a low-cost bimanual robot setup (ALOHA) capable
of performing fine-grained manipulation tasks, such as opening a lid, slotting a battery in its allotment or even prepare
tape for application. Notably, ALOHA bimanual setup costs just as much as a mono-arm Franka arm and can be
assembled from easy-to-source parts, underscoring its higher accessibility.

43
Zhao et al. (2023) do also present significant algorithmic contributions related to synthetizing performant autonomous
behaviors for the ALOHA setup, adopting transformers as the architectural backbone to learn a Conditional VAE (Sohn
et al., 2015) from demonstrations. Conditional VAEs are a variation of the standard VAE introducing an arbitrary
conditioning on sampling from the latent prior, modeling one-to-many relationships between latent and data samples.
Further, in stark contrast with previous work (Florence et al., 2022; Janner et al., 2022), Zhao et al. (2023) do not
learn a full joint pθ (o, a) on observation and actions, and rather focus on the conditional pθ (a|o). While the policy
distribution pθ (a|o) can in principle be entirely described from the joint pθ (o, a), conditional distributions are often
intractable when using function approximators, as pθ (a|o) = R pθp(o,a)
(o,a)
, and the integral in the denominator is typically
A θ
intractable. Thus, instead of modeling the full joint using a vanilla VAE, Zhao et al. (2023) propose learning a
conditional VAE (Sohn et al., 2015) modeling the policy distribution directly, hence approximating p(a|o).
In practice, when learning from demonstrations adopting CVAEs results in a slight modification to the VAE objective
in eq. 26, which is adapted to:
N
X
ELBOD (θ, ϕ, ω) = Ez∼qϕ (·|oi ,ai ) log pθ (ai |z, oi ) − DKL qϕ (z|oi , ai )∥pω (z|oi ) (50)

i=0

Notice how in eq. 50 we are now also learning a new set of parameters ω for the prior distribution in the latent space.
Effectively, this enables conditioning latent-space sampling (and thus reconstruction) during training (and potentially
inference too), providing useful when learning inherently conditional distributions like policies. Further, ACT is
trained as a β-CVAE (Higgins et al., 2017), weighing the KL regularization term in eq. 50 with an hyperparameter
β ∈ R+ regulating the information condensed in the latent space, where higher β results in a less expressive latent
space.
In their work, Zhao et al. (2023) ablated using a GM to learn from human demonstrations compared to a simpler,
supervised objective, L1 (a, a′ ) = ∥a − a′ ∥1 . Interestingly, they found the performance of these two approaches to be
comparable when learning from scripted demonstrations. That is, when learning from data collected rolling out a
predetermined set of commands [q0c , q1c , . . . ], GM did not prove competitive compared to standard supervised learning.
However, when learning from human demonstrations—i.e., from data collected executing commands coming from a
human controller [q0h , q1h , . . . ]— Zhao et al. (2023) found performance (defined as the success rate on a downstream
task) to be severily (-33.3%) hindered from adopting a standard supervised learning objective compared to a richer,
potentially more complex to learn variational objective. The result of such ablation reflects from the multimodal
nature of human demonstrations data, and is consistent with the findings presented by Florence et al. (2022). The
authors also ablate the action chunking paradigm, reporting significant performance gains deriving from using action
chunking (1% vs. 44% success rate). To reduce acting open-loop, Zhao et al. (2023) also design an inference process
consisting in performing inference at every timestep t and then aggregate multiple chunks using an exponential moving
average (EMA) on the overlapping chunks.
In ACT (Figure 30), inference for a given observation o ∈ O could be performed by (1) defining a prior pω (z|o) for
the latent variable z and (2) decoding an action chunk from a sampled latent z ∼ pω (•|o), similarily to how sampling
from standard VAEs takes place, with the exception that vanilla VAEs typically pose p(z|o) ≡ p(z) ∼ N (0, I) and
thus skip (1).
However, the authors claim that using a deterministic procedure to sample z benefits policy evaluation, and thus avoid
using the conditional prior at all at inference time, effectively using the CVAE framework exclusively to train a more
expressive decoder. At test time, Zhao et al. (2023) propose simply using z = 0, as the conditional prior on z used in
training is set to be a standard Gaussian. Further, conditioning on the observation o is achieved through explicitly
feeding proprioperceptive and visual observations to the decoder, pθ (a|z, o) at test time. If at inference z is sampled
from a standard Gaussian, during training z is sampled from an approximate posterior distribution qϕ (z|o, a), which,
however, disregards image observations and exclusively uses proprioperceptive states to form o for efficiency reasons.

44
Figure 28 | The CVAE encoder used in ACT. Input action chunks are first embedded and aggregated with positional
embeddings, before being processed alongside embedded proprioperceptive information, and a learned [CLS] token used to
aggregate input level information, and predict the style variable z. The encoder is exclusively used to train the decoder, and it
is entirely disregarded at inference time.

Figure 29 | The CVAE decoder used in ACT, comprising of a full encoder-decoder Transformer architecture. Camera
observations from all n camera views are first embedded using pre-trained visual encoders, and then aggregated with the
corresponding positional embeddings. Then, the proprioperceptive information and style variable z retrieved from the CVAE
encoder, are fed to the encoder-decoder Transformer for inference. The encoder shares the matrices K, V with the decoder, and
is trained to decode fixed position embeddings into action chunks.

45
Figure 30 | Action Chunking with Transformer (ACT), as in Zhao et al. (2023). ACT introduces an action chunking paradigm
to cope with high-dimensional multi-modal demonstration data, and a transformer-based CVAE architecture.

4.2.1 Code Example: Training and Using ACT in Practice

Code 7: Training ACT

[Link]

1 from pathlib import Path

2
3 import torch
4
5 from lerobot . configs . types import FeatureType
6 from lerobot . datasets . l er o bo t_ d at as e t import LeRobotDataset , L e R o b o t D a t a s e t M e t a d a t a
7 from lerobot . datasets . utils import d a t a s e t _ t o _ p o l i c y _ f e a t u r e s
8 from lerobot . policies . act . c o n f i g u r a t i o n _ a c t import ACTConfig
9 from lerobot . policies . act . modeling_act import ACTPolicy
10 from lerobot . policies . factory import m a k e _ p r e _ p o s t _ p r o c e s s o r s
11
12
13 def m a k e _ d e l t a _ t i m e s t a m p s ( delta_indices : list [ int ] | None , fps : int ) -> list [ float ]:
14 if delta_indices is None :
15 return [0]
16
17 return [ i / fps for i in delta_indices ]
18
19
20 o u t p u t _ d i r e c t o r y = Path ( " outputs / r o b o t _ l e a r n i n g _ t u t o r i a l / act " )
21 o u t p u t _ d i r e c t o r y . mkdir ( parents = True , exist_ok = True )
22
23 # Select your device
24 device = torch . device ( " mps " ) # or " cuda " or " cpu "
25
26 dataset_id = " lerobot / s v l a _ s o 1 0 1 _ p i c k p l a c e "
27
28 # This specifies the inputs the model will be expecting and the outputs it will produce
29 d a t a s e t _ m e t a d a t a = L e R o b o t D a t a s e t M e t a d a t a ( dataset_id )
30 features = d a t a s e t _ t o _ p o l i c y _ f e a t u r e s ( d a t a s e t _ m e t a d a t a . features )
31
32 ou tp u t_ fe a tu re s = { key : ft for key , ft in features . items () if ft . type is FeatureType . ACTION }
33 inp ut_fea tures = { key : ft for key , ft in features . items () if key not in ou t pu t_ f ea tu r es }
34
35 cfg = ACTConfig ( in put_fe atures = input_features , ou tp u t_ fe a tu re s = ou tp u t_ fe a tu re s )
36 policy = ACTPolicy ( cfg )
37 preprocessor , postprocessor = m a k e _ p r e _ p o s t _ p r o c e s s o r s (

46
38 cfg , dataset_stats = d a t a s e t _ m e t a d a t a . stats
39 )
40
41 policy . train ()
42 policy . to ( device )
43
44 # To perform action chunking , ACT expects a given number of actions as targets
45 delta_timestamps = {
46 " action " : m a k e _ d e l t a _ t i m e s t a m p s ( cfg . action_delta_indices , d a t a s e t _ m e t a d a t a . fps ) ,
47 }
48
49 # add image features if they are present
50 d e l t a _ t i m e s t a m p s |= {
51 k : m a k e _ d e l t a _ t i m e s t a m p s ( cfg . observation_delta_indices , d a t a s e t _ m e t a d a t a . fps )
52 for k in cfg . ima ge_fea tures
53 }
54
55 # Instantiate the dataset
56 dataset = LeRob otData set ( dataset_id , d e l t a _ t i m e s t a m p s = d e l t a _ t i m e s t a m p s )
57
58 # Create the optimizer and dataloader for offline training
59 optimizer = cfg . g e t _ o p t i m i z e r _ p r e s e t (). build ( policy . parameters ())
60 batch_size = 32
61 dataloader = torch . utils . data . DataLoader (
62 dataset ,
63 batch_size = batch_size ,
64 shuffle = True ,
65 pin_memory = device . type != " cpu " ,
66 drop_last = True ,
67 )
68
69 # Number of training steps and logging frequency
70 tra ining_ steps = 1
71 log_freq = 1
72
73 # Run training loop
74 step = 0
75 done = False
76 while not done :
77 for batch in dataloader :
78 batch = preprocessor ( batch )
79 loss , _ = policy . forward ( batch )
80 loss . backward ()
81 optimizer . step ()
82 optimizer . zero_grad ()
83
84 if step % log_freq == 0:
85 print ( f " step : { step } loss : { loss . item ():.3 f } " )
86 step += 1
87 if step >= trai ning_s teps :
88 done = True
89 break
90
91 # Save the policy checkpoint , alongside the pre / post processors
92 policy . s a ve _p r et ra i ne d ( o u t p u t _ d i r e c t o r y )
93 preprocessor . sa v e_ pr e tr ai n ed ( o u t p u t _ d i r e c t o r y )
94 postprocessor . s av e_ p re t ra in e d ( o u t p u t _ d i r e c t o r y )
95
96 # Save all assets to the Hub
97 policy . push_to_hub ( " fracapuano / r o b o t _ l e a r n i n g _ t u t o r i a l _ a c t _ e x a m p l e _ m o d e l " )
98 preprocessor . push_to_hub ( " fracapuano / r o b o t _ l e a r n i n g _ t u t o r i a l _ a c t _ e x a m p l e _ p i p e l i n e " )
99 postprocessor . push_to_hub ( " fracapuano / r o b o t _ l e a r n i n g _ t u t o r i a l _ a c t _ e x a m p l e _ p i p e l i n e " )

47
Code 8: Using ACT
[Link]

1 import torch
2
3 from lerobot . cameras . opencv . c o n f i g u r a t i o n _ o p e n c v import O p e n C V C a m e r a C o n f i g
4 from lerobot . datasets . l er o bo t_ d at as e t import L e R o b o t D a t a s e t M e t a d a t a
5 from lerobot . policies . act . modeling_act import ACTPolicy
6 from lerobot . policies . factory import m a k e _ p r e _ p o s t _ p r o c e s s o r s
7 from lerobot . policies . utils import build_inference_frame , m a k e _ r o b o t _ a c t i o n
8 from lerobot . robots . so 100_fo llower . c o n f i g _ s o 1 0 0 _ f o l l o w e r import S O 1 0 0 F o l l o w e r C o n f i g
9 from lerobot . robots . so 100_fo llower . so10 0_foll ower import SO100Follower
10
11 device = torch . device ( " mps " ) # or " cuda " or " cpu "
12 model_id = " fracapuano / r o b o t _ l e a r n i n g _ t u t o r i a l _ a c t _ e x a m p l e _ m o d e l "
13 model = ACTPolicy . f ro m _p r et ra i ne d ( model_id )
14
15 dataset_id = " lerobot / s v l a _ s o 1 0 1 _ p i c k p l a c e "
16 # This only downloads the metadata for the dataset , ~10 s of MB even for large - scale datasets
17 d a t a s e t _ m e t a d a t a = L e R o b o t D a t a s e t M e t a d a t a ( dataset_id )
18 preprocess , postprocess = m a k e _ p r e _ p o s t _ p r o c e s s o r s (
19 model . config , dataset_stats = d a t a s e t _ m e t a d a t a . stats
20 )
21
22 # # find ports using lerobot - find - port
23 follower_port = ... # something like "/ dev / tty . u s b m o d e m 5 8 7 6 0 4 3 1 6 3 1 "
24
25 # # the robot ids are used the load the right calibration files
26 follower_id = ... # something like " fol lower_ so100 "
27
28 MAX_EPISODES = 5
29 M A X _ S T E P S _ P E R _ E P I S O D E = 20
30
31 # Robot and environment configuration
32 # Camera keys must match the name and resolutions of the ones used for training !
33 # You can check the camera keys expected by a model in the info . json card on the Hub
34 camera_config = {
35 " side " : O p e n C V C a m e r a C o n f i g ( index_or_path =1 , width =640 , height =480 , fps =30) ,
36 " up " : O p e n C V C a m e r a C o n f i g ( index_or_path =1 , width =640 , height =480 , fps =30) ,
37 }
38
39 robot_cfg = S O 1 0 0 F o l l o w e r C o n f i g ( port = follower_port , id = follower_id , cameras = camera_config )
40 robot = SO100Follower ( robot_cfg )
41 robot . connect ()
42
43 for _ in range ( MAX_EPISODES ):
44 for _ in range ( M A X _ S T E P S _ P E R _ E P I S O D E ):
45 obs = robot . ge t_ o bs er v at io n ()
46 obs_frame = b u i l d _ i n f e r e n c e _ f r a m e ( obs , d a t a s e t _ m e t a d a t a . features , device )
47
48 obs = preprocess ( obs_frame )
49
50 action = model . select_action ( obs )
51 action = postprocess ( action )
52
53 action = m a k e _ r o b o t _ a c t i o n ( action , d a t a s e t _ m e t a d a t a . features )
54
55 robot . send_action ( action )
56
57 print ( " Episode finished ! Starting new episode ... " )

4.3 Diffusion Policy

DMs have proven very effective in approximating complex highly dimensional distributions, such as distributions over
images (Ho et al., 2020) or videos (Polyak et al., 2025), thanks to their inherent capability to deal with multimodal
data, and their training stability. In Diffusion Policy (DP), Chi et al. (2024) present an application of DMs the

48
Figure 31 | The Diffusion Policy archicture, as in Chi et al. (2024). A stack of Ho previous observations is used as external
conditioning to denoise a group of Ha actions. Conditioning is performed at every layer of a U-Net block. Diffusion Policy
allows to obtain fully-formed action chunks with as little as T = 10 denoising steps.

field of robot learning, leveraging diffusion to model expert demonstrations in a variety of simulated and real-world
tasks. Similarily to ACT (Zhao et al., 2023), Chi et al. (2024) (1) adopt a modified observation-conditioned target
distribution instead of the full joint p(o, a), and (2) predict multiple actions into the future instead of a single action.
Besides the intractability of the observations’ marginal pθ (o) given pθ (o, a), DP’s choice to model the data distribution
through pθ (a|o) also stems from the computational burden of diffusion at test time: generating actions together with
observations would require a large number of denoising steps—an unnecessarily slow and ultimately unhelpful process,
given that robotics focuses on producing controls rather than reconstructing observations.
In practice, conditioning on observation data is achieved conditioning the noise regressor ϵθ introduced in eq. 44 on a
stack of Ho observations, resulting in the conditional, simplified diffusion objective:
√ √
L(θ) = Et,at:t+Ha ,ϵ ∥ϵ − ϵθ ( ᾱt at:t+Ha + ϵ 1 − ᾱt , t, ot−Ho :t )∥2 , (51)

t ∼ U({1, . . . , T }), at:t+Ha , ot−Ho :t ∼ D, ϵ ∼ N (0, I).

Note how in eq. 51 the noise regressor is conditioned on both the latent variable rank t and on a stack of previous
observations ot−Ho :t . Chi et al. (2024) claim the combination of (1) conditioning on a horizon of previous observations
and (2) predicting multiple actions into the future allows DP to commit to specific modes in the data at inference
time, which proves essential for good performance and avoiding undecisiveness.
Figure 31 shows the convolution-based version of the architecture proposed by Chi et al. (2024), illustrating inference
on a single sample drawn from D, for simplicity. The starting, arbitrarily noisy chunk of Ha actions ãt:t+Ha is first
mapped to a (learned) high-dimensional space. Similarily, both image observations and poses are also embedded
before being aggregated to the action embeddings. Then, a U-Net (Ronneberger et al., 2015) is trained to regress
the noise added into ãt:t+Ha , conditioned on observation information at every layer, thus seeking to optimize eq. 51.
At inference time, the noise predictor is used to predict the quantity of noise at every t ∈ [T, . . . , 0] and iteratively
subtract it from ãt:t+Ha , reversing the diffusion process simulated in training conditioned on ot−Ho :t to predict at:t+Ha .
DP can be trained with as little as 50-150 demos (ca. 15-60 minutes of teleoperation data), and exhibit strong
performance on a variety of simulated and real-world tasks, including dexterous and deformable manipulation tasks
such as sauce pouring and yoga-mat unrolling. Notably, the authors ablated the relevance of using RGB camera streams
as input to their policy, and observed how high frame-rate visual observations can be used to attain performance
(measured as success rate) comparable to that of state-based policies, which are typically trained in simulation with
priviledged information not directly available in real-world deployments. As high-frame rate RGB inputs naturally
accomodate for dynamic, fast changing environments, Chi et al. (2024)’s conclusion offers significant evidence for
learning streamlined control policies directly from pixels. In their work, Chi et al. (2024) also ablate the performance
of DP against the size of the dataset collected, showing that DP reliably outperforms the considered baseline for

49
all benchmark sizes considered. Further, in order accelerate inference, Chi et al. (2024) employ Denoising Diffusion
Implicit Models (Song et al., 2022), a variant of Denoising Diffusion Probabilistic Models (Ho et al., 2020) (DDPM)
adopting a strictly deterministic denoising paradigm (differently from DDPM’s natively stochastic one) inducing
the same final distribution’s as DDPM’s, and yet resulting in 10x less denoising steps at inference time (Chi et al.,
2024). Across a range of simulated and real-world tasks, Chi et al. (2024) find DPs particularly performant when
modeling ϵθ with a transformer-based network, although the authors note the increased sensitivity of transformer
networks to hyperparameters. Thus, Chi et al. (2024) explicitly recommend starting out with a simpler, convolution-
based architecture for diffusion (Figure 31), which is however reported to be biased towards learning low-frequency
components (Tancik et al., 2020), and thus may prove more challenging to train with non-smooth action sequences.

4.3.1 Code Example: Training and Using Diffusion Policies in Practice

Code 9: Training Diffusion Policy

[Link]
py

1 from pathlib import Path

2
3 import torch
4
5 from lerobot . configs . types import FeatureType
6 from lerobot . datasets . l er o bo t_ d at as e t import LeRobotDataset , L e R o b o t D a t a s e t M e t a d a t a
7 from lerobot . datasets . utils import d a t a s e t _ t o _ p o l i c y _ f e a t u r e s
8 from lerobot . policies . diffusion . c o n f i g u r a t i o n _ d i f f u s i o n import Di ff u si on C on fi g
9 from lerobot . policies . diffusion . m o d e l i n g _ d i f f u s i o n import D i ff us i on Po l ic y
10 from lerobot . policies . factory import m a k e _ p r e _ p o s t _ p r o c e s s o r s
11
12
13 def m a k e _ d e l t a _ t i m e s t a m p s ( delta_indices : list [ int ] | None , fps : int ) -> list [ float ]:
14 if delta_indices is None :
15 return [0]
16
17 return [ i / fps for i in delta_indices ]
18
19
20 o u t p u t _ d i r e c t o r y = Path ( " outputs / r o b o t _ l e a r n i n g _ t u t o r i a l / diffusion " )
21 o u t p u t _ d i r e c t o r y . mkdir ( parents = True , exist_ok = True )
22
23 # Select your device
24 device = torch . device ( " mps " ) # or " cuda " or " cpu "
25
26 dataset_id = " lerobot / s v l a _ s o 1 0 1 _ p i c k p l a c e "
27
28 # This specifies the inputs the model will be expecting and the outputs it will produce
29 d a t a s e t _ m e t a d a t a = L e R o b o t D a t a s e t M e t a d a t a ( dataset_id )
30 features = d a t a s e t _ t o _ p o l i c y _ f e a t u r e s ( d a t a s e t _ m e t a d a t a . features )
31
32 ou tp u t_ fe a tu re s = { key : ft for key , ft in features . items () if ft . type is FeatureType . ACTION }
33 inp ut_fea tures = { key : ft for key , ft in features . items () if key not in ou t pu t_ f ea tu r es }
34
35 cfg = D if f us i on Co n fi g ( input _featu res = input_features , o ut pu t _f e at ur e s = o ut pu t _f e at ur e s )
36 policy = D iffu s io nP o li c y ( cfg )
37 preprocessor , postprocessor = m a k e _ p r e _ p o s t _ p r o c e s s o r s (
38 cfg , dataset_stats = d a t a s e t _ m e t a d a t a . stats
39 )
40
41 policy . train ()
42 policy . to ( device )
43
44 # To perform action chunking , ACT expects a given number of actions as targets
45 delta_timestamps = {
46 " observation . state " : m a k e _ d e l t a _ t i m e s t a m p s (
47 cfg . observation_delta_indices , d a t a s e t _ m e t a d a t a . fps
48 ),
49 " action " : m a k e _ d e l t a _ t i m e s t a m p s ( cfg . action_delta_indices , d a t a s e t _ m e t a d a t a . fps ) ,

50
50 }
51
52 # add image features if they are present
53 d e l t a _ t i m e s t a m p s |= {
54 k : m a k e _ d e l t a _ t i m e s t a m p s ( cfg . observation_delta_indices , d a t a s e t _ m e t a d a t a . fps )
55 for k in cfg . ima ge_fea tures
56 }
57
58 # Instantiate the dataset
59 dataset = LeRob otData set ( dataset_id , d e l t a _ t i m e s t a m p s = d e l t a _ t i m e s t a m p s )
60
61 # Create the optimizer and dataloader for offline training
62 optimizer = cfg . g e t _ o p t i m i z e r _ p r e s e t (). build ( policy . parameters ())
63 batch_size = 32
64 dataloader = torch . utils . data . DataLoader (
65 dataset ,
66 batch_size = batch_size ,
67 shuffle = True ,
68 pin_memory = device . type != " cpu " ,
69 drop_last = True ,
70 )
71
72 # Number of training steps and logging frequency
73 tra ining_ steps = 1
74 log_freq = 1
75
76 # Run training loop
77 step = 0
78 done = False
79 while not done :
80 for batch in dataloader :
81 batch = preprocessor ( batch )
82 loss , _ = policy . forward ( batch )
83 loss . backward ()
84 optimizer . step ()
85 optimizer . zero_grad ()
86
87 if step % log_freq == 0:
88 print ( f " step : { step } loss : { loss . item ():.3 f } " )
89 step += 1
90 if step >= trai ning_s teps :
91 done = True
92 break
93
94 # Save the policy checkpoint , alongside the pre / post processors
95 policy . s a ve _p r et ra i ne d ( o u t p u t _ d i r e c t o r y )
96 preprocessor . sa v e_ pr e tr ai n ed ( o u t p u t _ d i r e c t o r y )
97 postprocessor . s av e_ p re t ra in e d ( o u t p u t _ d i r e c t o r y )
98
99 # Save all assets to the Hub
100 policy . push_to_hub ( " fracapuano / r o b o t _ l e a r n i n g _ t u t o r i a l _ d i f f u s i o n _ e x a m p l e _ m o d e l " )
101 preprocessor . push_to_hub ( " fracapuano / r o b o t _ l e a r n i n g _ t u t o r i a l _ d i f f u s i o n _ e x a m p l e _ m o d e l " )
102 postprocessor . push_to_hub ( " fracapuano / r o b o t _ l e a r n i n g _ t u t o r i a l _ d i f f u s i o n _ e x a m p l e _ m o d e l " )

Code 10: Using Diffusion Policy

[Link]

51
10
11 device = torch . device ( " mps " ) # or " cuda " or " cpu "
12 model_id = " fracapuano / r o b o t _ l e a r n i n g _ t u t o r i a l _ d i f f u s i o n _ e x a m p l e _ m o d e l "
13
14 model = Di f fu si o nP ol i cy . fr o m_ pr e tr ai n ed ( model_id )
15
16 dataset_id = " lerobot / s v l a _ s o 1 0 1 _ p i c k p l a c e "
17 # This only downloads the metadata for the dataset , ~10 s of MB even for large - scale datasets
18 d a t a s e t _ m e t a d a t a = L e R o b o t D a t a s e t M e t a d a t a ( dataset_id )
19 preprocess , postprocess = m a k e _ p r e _ p o s t _ p r o c e s s o r s (
20 model . config , model_id , dataset_stats = d a t a s e t _ m e t a d a t a . stats
21 )
22
23 MAX_EPISODES = 5
24 M A X _ S T E P S _ P E R _ E P I S O D E = 20
25
26
27 # # find ports using lerobot - find - port
28 follower_port = ... # something like "/ dev / tty . u s b m o d e m 5 8 7 6 0 4 3 1 6 3 1 "
29
30 # # the robot ids are used the load the right calibration files
31 follower_id = ... # something like " fol lower_ so100 "
32
33 # Robot and environment configuration
34 # Camera keys must match the name and resolutions of the ones used for training !
35 # You can check the camera keys expected by a model in the info . json card on the Hub
36 camera_config = {
37 " side " : O p e n C V C a m e r a C o n f i g ( index_or_path =1 , width =640 , height =480 , fps =30) ,
38 " up " : O p e n C V C a m e r a C o n f i g ( index_or_path =1 , width =640 , height =480 , fps =30) ,
39 }
40
41 robot_cfg = S O 1 0 0 F o l l o w e r C o n f i g ( port = follower_port , id = follower_id , cameras = camera_config )
42 robot = SO100Follower ( robot_cfg )
43 robot . connect ()
44
45
46 for _ in range ( MAX_EPISODES ):
47 for _ in range ( M A X _ S T E P S _ P E R _ E P I S O D E ):
48 obs = robot . ge t_ o bs er v at io n ()
49 obs_frame = b u i l d _ i n f e r e n c e _ f r a m e ( obs , d a t a s e t _ m e t a d a t a . features , device )
50
51 obs = preprocess ( obs_frame )
52
53 action = model . select_action ( obs )
54 action = postprocess ( action )
55 action = m a k e _ r o b o t _ a c t i o n ( action , d a t a s e t _ m e t a d a t a . features )
56 robot . send_action ( action )
57
58 print ( " Episode finished ! Starting new episode ... " )

4.4 Optimized Inference

Modern visuomotor policies output action chunks–sequences π(ot ) = at , at+1 , . . . , at+Ha = At with At a sequence

of Ha ≫ 1 low-level commands scheduled for execution in an action queue, all originating from a single environment
observation, ot . Predicting series of actions instead of single commands proved essential in learning complex, multi-
modal behavior (Zhao et al., 2023; Chi et al., 2024), and it also holds the premise to be useful to optimize how
inference is carried out in practice.
A robot may indeed execute an entire action chunk At before a new observation ot+Ha is passed to the policy π to
predict the next chunk, which would result in open-loop control between observations captured every Ha timesteps.
Zhao et al. (2023) adopt a different strategy, whereby the robot controller interleaves chunk prediction At ← π(ot )
and chunk consumption at ← PopFront(At ), and computes a new chunk of actions at every timestep t, to then
aggregate the predicted chunks on overlapping sections. While adaptive—every observation at every timestep ot is
processed—such an approach relies on running inference continuously, which can be prohibitive in resource-constrained
scenarios, such as edge deployments. A less resource-intensive approach is to entirely exhaust the chunk A before
predicting a new chunk of actions, a strategy we refer to as synchronous (sync) inference. Sync inference allocates

52
Figure 32 | Asynchronous inference. Illustration of the asynchronous inference stack. Note that the policy can be run on a
remote server, possibly with GPUs.
computation every Ha timesteps, resulting in a reduced computational burden (on average) at control time. In
contrast, sync inference also inherently hinders the responsiveness of robot systems, introducing blind lags due to the
robot being idle while computing A.
One can use the fact that policies output multiple actions at the same time to directly (1) the lack of adaptiveness and
(2) the presence of lags at runtime by decoupling action chunk prediction A from action execution at ← PopFront(At ).
This decoupled stack, which we refer to as asynchronous (async) inference (1), also enables optimized inference by
allowing action-chunk inference to run on a separate machine, typically equipped with better computational resources
than the ones onboard a robot. In async inference, a RobotClient sends an observation ot to a PolicyServer,
receiving an action chunk At once inference is complete (Figure 32). In this, we avoid execution lags by triggering
chunk prediction while the control loop is still consuming a previously available chunk, aggregating the previous
and incoming chunks whenever the latter is available to the RobotClient. In turn, async-inference tightens the
loop between action prediction and action execution efficienty, by increasing the frequency at which observations are
processed for chunk prediction while not running inference at every timestep. Crucially, decoupling action prediction
from action execution also allows to allocate more computational resources on a remote policy server sending actions
to the robot client over the network.

Algorithm 1 Asynchronous inference control-loop

1: Input: horizon T , chunk size Ha , threshold g ∈ [0, 1]
2: Init: capture o0 ; send o0 to PolicyServer; receive A0 ← π(o0 )
3: for t to Ha do
4: at ← PopFront(At )
5: Execute(at ) ▷ execute action at step t
|At |
6: if Ha < g then ▷ queue below threshold
7: capture new observation, ot+1
8: if NeedsProcessing (ot+1 ) then ▷ similarity filter, or triggers direct processing
9: async_handle ← AsyncInfer(ot+1 ) ▷ Trigger new chunk prediction (non blocking)
10: Ãt+1 ← π(ot+1 ) ▷ New queue is predicted with the policy
11: At+1 ← f (At , Ãt+1 ) ▷ aggregate overlaps (if any)
12: end if
13: end if
14: if NotCompleted(async_handle) then
15: At+1 ← At ▷ No update on queue (inference is not over just yet)
16: end if
17: end for

53
Figure 33 | Action queue size evolution at runtime for various levels of g when (A) not filtering out observation based on
joint-space similarity and (B) filtering out near-duplicates observation, measuring their similarity in joint-space.

In practice, async inference (1) tightens the control loop by capturing observations more often, eliminating idle gaps
at runtime (2) and directly allows to run inference on more powerful computational resources than the ones typically
available onboard autonomous robotic platforms. Algorithmically, one can attain (1) on the RobotClient-side
by consuming actions from a readily available queue until a given condition on the number of remaining actions
in the queue (|At |/Ha < g) is met. When this condition is triggered, a new observation of the environment is
captured and sent to the (possibly remote) PolicyServer. To avoid redundant server calls and erratic behavior at
runtime observations are compared in joint-space, and near-duplicates are dropped. Two observations are considered
near-duplicates if their distance in joint-space falls under a predetermined threshold, dlim ∈ R+ . Importantly, should
the queue available to the robot client eventually empty out, the most recent observation is processed regardless of
similarity.
Interestingly, the behavior of async inference can be studied analytically. First, let ℓ be a random variable modeling the
time needed to receive an action chunk A after sending an observation o, i.e. the sum of (1) the time to send across the
observation o between the RobotClient and PolicyServer, tC→S (2) the inference latency on the PolicyServer,
ℓS and (3) the time to send A between the PolicyServer and RobotClient, tS→C . Under the (reasonable)
assumption of independence, E[ℓ] = E[tC→S ] + E[ℓS ] + E[tS→C ], which can be further simplified to E[ℓ] ≃ E[ℓS ],
assuming communication time is (1) equal in both directions and (2) negligible with respect to the inference latency.
Second, let ∆t be the environment’s control cycle. With a real-world frame-rate of 30 frames-per-second (fps),
∆t = 33ms. Consequently, exhausted queues at runtime—i.e. being idle awaiting for a new chunk—are avoided for
S ]/∆t
g ≥ E[ℓH a
. In this, the action queue threshold g below which to capture and send a new observation for processing
plays a major role relatively to the availability of actions to the RobotClient.
Figure 33 illustrates how the size of the action chunk |At | evolves over time for three representative values of g,
detailing the following key scenarios:

• Sequential limit (g = 0). The client drains the entire chunk before forwarding a new observation to the server.
During the round-trip latency needed to compute the next chunk, the queue is empty, leaving the robot incapable
of acting. This reproduces the behavior of a fully sequential deployment and results in an average of E[ℓS ] idle
seconds.
• Asynchronous inference (g ∈ (0, 1)). Allowing the client to consume a 1 − g fraction of its available queue At−1
before triggering inference for a new action queue At , computation is amortized while keeping the queue from
emptying. The overlap between successive chunks provides a buffer against modeling errors without the full cost of
the g = 1 regime. The updated queue At is obtained aggregating queues on the overlapping timesteps between
At−1 and the incoming Ãt .
• Sync-inference limit (g = 1). As an extreme case, and in keeping with Zhao et al. (2023), an observation is sent
at every timestep. The queue is therefore almost always filled, with only a minor saw-tooth due to ∆t/E[ℓs ] < 1.
While maximally reactive, this setting incurs one forward pass per control tick and can prove prohibitively expensive
on limited hardware. Importantly, because the client is consuming actions while the server computes the next
chunk, the available queue never gets entirely filled.

Figure 33 emphasizes the trade-off governed by g: small values of g result in idle periods, whereas g ≈ 1 assumes a

54
highly accurate model and pays a significant compute price. In practice, choosing g ∈ (0, 1) allows to strike a balance
between reactivity against resource budgets. If not for the aforementioned similarity filter, the RobotClient would
send observations for processing every (1−g)Ha ·∆t seconds, receiving a new chunk of actions every (1−g)Ha ·∆t+E[ℓS ],
on average. The presence of the filter for observation similarity dilates this processing time, and serves the scope
of avoiding the robot stalling due to the queue being constantly integrated with an incoming, nearly identical,
action chunk. In particular, Figure 33 results in a queue which is filled with incoming actions unless near-duplicate
observations are filtered out from the processing pipeline. For clarity, the red arrow in 33 highlights a timestep where
the observation similarity mechanism is bypassed, forcing a (nearly identical) observation to be processed as the queue
results empty.

4.4.1 Code Example: Using Async Inference

Code 11: Spinning up a Remote Server

[Link]

1 from lerobot . a sy nc _ in fe r en ce . configs import P o l i c y S e r v e r C o n f i g

2 from lerobot . a sy nc _ in fe r en ce . policy_server import serve
3
4 host = ... # something like "[Link]" if you ' re exposing to localhost
5 port = ... # something like 8080
6
7 config = P o l i c y S e r v e r C o n f i g (
8 host = host ,
9 port = port ,
10 )
11 serve ( config )

Code 12: Attaching a Robot Client

[Link]

1 import threading
2 from lerobot . robots . so 100_fo llower import S O 1 0 0 F o l l o w e r C o n f i g
3 from lerobot . cameras . opencv . c o n f i g u r a t i o n _ o p e n c v import O p e n C V C a m e r a C o n f i g
4 from lerobot . a sy nc _ in fe r en ce . configs import R o b o t C l i e n t C o n f i g
5 from lerobot . a sy nc _ in fe r en ce . robot_client import RobotClient
6 from lerobot . a sy nc _ in fe r en ce . helpers import v i s u a l i z e _ a c t i o n _ q u e u e _ s i z e
7
8 # these cameras must match the ones expected by the policy ( use lerobot - find - cameras )
9 # check the config . json on the Hub for the policy you are using to see the expected camera specs
10 camera_cfg = {
11 " top " : O p e n C V C a m e r a C o n f i g ( index_or_path =0 , width =640 , height =480 , fps =30) ,
12 " side " : O p e n C V C a m e r a C o n f i g ( index_or_path =1 , width =640 , height =480 , fps =30)
13 }
14
15 # # find ports using lerobot - find - port
16 follower_port = ... # something like "/ dev / tty . u s b m o d e m 5 8 7 6 0 4 3 1 6 3 1 "
17
18 # # the robot ids are used the load the right calibration files
19 follower_id = ... # something like " fol lower_ so100 "
20
21 robot_cfg = S O 1 0 0 F o l l o w e r C o n f i g (
22 port = follower_port ,
23 id = follower_id ,
24 cameras = camera_cfg
25 )
26
27 ser ver_ad dress = ... # something like 127.0. 0.1:80 80 if using localhost
28
29 # 3. Create client configuration
30 client_cfg = R o b o t C l i e n t C o n f i g (
31 robot = robot_cfg ,

55
32 server _addre ss = server_address ,
33 policy_device = " mps " ,
34 policy_type = " smolvla " ,
35 p r e t r a i n e d _ n a m e _ o r _ p a t h = " fracapuano / smolvla_async " ,
36 c h u n k _ s i z e _ t h r e s h o l d =0.5 , # g
37 a c t i o n s _ p e r _ c h u n k =50 , # make sure this is less than the max actions of the policy
38 )
39
40 # 4. Create and start client
41 client = RobotClient ( client_cfg )
42
43 # 5. Provide a textual description of the task
44 task = ...
45
46 if client . start ():
47 # Start action receiver thread
48 a c t i o n _ r e c e i v e r _ t h r e a d = threading . Thread ( target = client . receive_actions , daemon = True )
49 a c t i o n _ r e c e i v e r _ t h r e a d . start ()
50
51 try :
52 # Run the control loop
53 client . control_loop ( task )
54 except K e y b o a r d I n t e r r u p t :
55 client . stop ()
56 a c t i o n _ r e c e i v e r _ t h r e a d . join ()
57 # ( Optionally ) plot the action queue size
58 v i s u a l i z e _ a c t i o n _ q u e u e _ s i z e ( client . a c t i o n _ q u e u e _ s i z e )

56
Figure 34 | Fields within ML such as Computer Vision and NLP converged on the development of foundation models, trained
on a variety of large scale models and capable to perform multiple downstream tasks (top). Conversely, robotics suffered from
limited standardization in terms of the architectures used, and siloed, task specific datasets, incurring in a high degree of
fragmentation which traditionally hindered the development of generalist models for robotics in favour of task-specific models
(bottom).

5 Generalist Robot Policies

Specialization is for insects

Robert A. Heinlein

TL;DR

Openly available, large-scale datasets and the development of stable-to-train, expressive and efficient architectures
fostered research on the development of generalist robot policies that can operate across embodiment and tasks.

The advent of large models trained on internet-scale datasets has drastically influenced fields like Computer Vision
(CV) and Natural Language Processing (NLP), shifting the previously task-specific paradigm towards combining (1) an
initial, task-agnostic large-scale pre-training stage and a (2) task-specific, adjustment phase. This pre-train-and-adaptat
paradigm has now largely replaced more classic approaches consisting of task-specific data collection, curation and
model training in many subdomains within CV and NLP, and it is motivated by the main drawback of limited
scalability for task-specific approaches, which have been traditionally more labor intensive. Factors including (1)
the advancements in generalist models learned with self-supervision for perception (Oquab et al., 2024) or semantic
understanding (Devlin et al., 2019) and (2) the popularization of collective efforts to aggregate large-scale openly
available datasets (O’Neill et al., 2025; Khazatsky et al., 2025) are increasingly pushing the field of robot learning
towards the pre-train-and-adapt paradigm. This shift taps into the long-standing challenge of developing generalist
robot policies, and holds the premise to surpass traditionally siloed approaches to robotics problems and develop
a foundation robotics model. While Section 4 introduced methods for learning single-task policies such as ACT
or Diffusion Policy, in this section we present advancements in developing generalist, multi-task, policies, capable
of performing a wide range of tasks across different environments and embodiments, and guided by unstructured
instructions typically given in plain, natural language.

57
Figure 35 | Early efforts in the development of generalist models for robotics include BC-Zero (Jang et al., 2022), RT-1 (Brohan
et al., 2023b), and RT-2 (Brohan et al., 2023a): large scale models trained on thousands of demonstrations. The open release
of the Open-X (O’Neill et al., 2025) and DROID datasets (Khazatsky et al., 2025) fostered the development of open source
models: OpenVLA (Kim et al., 2024), π0 (Black et al., 2024) and SmolVLA (Shukor et al., 2025).

5.1 Preliminaries: Models and Data

The remarkable success of foundation models in NLP and CV seems to be increasingly predicated on two core
principles: architectural innovation and (joint) data-compute scaling. Indeed, the transformer architecture proved very
effective in capturing long-range dependencies in a variety of data formats, and its stability and expressivity made it
the de facto standard for modern large-scale models trained on internet-scale datasets. However, in stark contrast with
large-scale NLP and CV datasets (Raffel et al., 2023; Deng et al., 2009), robotics has historically developed around
small, task-specific datasets. In turn, this traditionally hindered scalability across problems as well as results, posing
concrete challenges to developing general-purpose robot learning algorithms. Indeed, differently from the wealth
of relatively readily-available task-agnostic text and images datasets on the internet, robotics data is intrinsically
embodied and thus task-specific: datasets collected for manipulation differ significantly from locomotion. In particular,
since each expert trajectory is tied to a specific robot platform and the operating conditions of its environment
and task, data heterogeneity has long posed a methodological challenge for scaling robotics datasets via aggregation.
Further, datasets consisting of expert demonstrations are (1) intrinsically more expensive to collect and (2) notoriously
heterogeneous—different human experts may perform the same task in very different. Beyond this, heterogeneity also
raises conceptual issues: naively mixing data across embodiments can induce negative transfer, as control strategies
developed in isolation for different robot systems in different environments may even conflict when combined. Thus,
the high degree of fragmentation of robotics datasets and tasks has traditionally led to the development of specialist
policies, trained on small, task-specific datasets, developed to perform well at their designated task but that fail to
generalize to new deployment scenarios (Figure 34).
Driven by the goal of developing generalist robot policies, the research community has increasingly explored how
insights and techniques from other areas of ML can be integrated into robotics. Figure 35 shows a timeline of some
of the most popular contributions attempting at developing generalist policies. Starting from BC-Zero, a latent
variable model trained on 25k+ demonstrations, the field has now evolved into π0 , a transformer-based model trained
on 10M+ demonstrations and exhibiting strong few-shot capabilities across tasks and embodiments. In between,
Robotics Transformer 1 (RT-1) (Brohan et al., 2023b) represented a significant step in the direction of developing a
generalist robot policies over prior work including (1) BC-Zero (Jang et al., 2022) and (2) Gato (Reed et al., 2022), in
that Brohan et al. (2023b) use a much larger and diverse set of training tasks compared to both BC-Zero and Gato.
In particular, RT-1 uses a transformer architecture, and is trained on as many as 130k human-recorded trajectories
collected over 13 robots and over 17 months. RT-1 learns to process a history of camera images and a natural language
instruction, and feeds the resulting sequence of high-dimensional tokens to a transformer, trained using a classification
loss on a discretized actions space consisting of six different 256-bins, one for each joint of a 6-dof robotic arm.
In a follow-up work, the same group of authors propose a modified method to learn generalist models, leveraging (1)
a more powerful architecture and (2) scaling up the dataset used (Brohan et al., 2023a, RT-2). In RT-2, Brohan et al.
(2023a) propose inheriting internet-scale semantic knowledge from large-scale multi-modal datasets to learn a single,
unified model for robotics control. Such a model, termed Vision-Language-Action (VLA) in the original RT-2 paper,
effectively casts robot control as a language-modeling problem, and in particular as a Visual Question-Answering
(VQ&A) task, in which the output token space used to represent textual tokens is shared with the 8-bits tokens used

58
Figure 36 | Robot learning is undergoing a paradigmatic shift: centralized data collections (A, left) are increasingly larger,
often comprising millions of demonstrations, while (A, right) decentralized data collection efforts are becoming an alternative for
large scale data collection. (B) Generalist models are also becoming increasingly smaller and easier to run on limited hardware.

to represent the 256 (28 ) actuation levels of a 6-dof robot. In their work, Brohan et al. (2023a) propose co-fine-tuning
large-scale VLMs such as PaLIX (Chen et al., 2023) or PaLM-E (Driess et al., 2023) on a mix of (1) web and (2)
robotics data, complementing VQ&A training with robotics-specific signal, and learning to directly output robot
actions in a shared token space for visual and language inputs. In their work, the authors claim using large models
trained on internet-scale data as backbones for VLAs allows models to tap into the rich semantic knowledge embedded
in the VLM’s parameters, interpreting instructions and unseen objects by connecting them to concepts acquired while
pre-training. For instance, Brohan et al. (2023a) show that while RT-2 has never been explicitly trained to repurpose
tools for a hammering task, it can still combine its semantic understanding of images, so that when asked which
object between (1) a piece of paper, (2) a pair of headphones or (3) a rock may be used instead of a hammer, it
correctly answers (3).
Traditionally, research efforts revolved around not only training models, but also proposing datasets for the community,
a costly and time-consuming process. Due to the aforementioned embodiment gap, the data used in research efforts in
robot learning have traditionally proved rather fragmented, tailored to the specific task considered by the specific group
of researchers who collected it, which ultimately hindered integration. The Open X-Embodiment project (O’Neill et al.,
2025) was a landmark collaboration effort to address data fragmentation, by curating the aggregation of 60 existing
robotics datasets from 22 different robot embodiments and 21 institutions across the world, and resulted in a total 1.4M
of cross-embodiments, cross-tasks, openly-available trajectories. Besides the contribution of an aggregate, large scale
dataset, O’Neill et al. (2025) also demonstrated significant positive transfer across tasks and embodiments, showing
that a single model trained on multi-embodiment data can outperform specialist models trained on their respective
single-embodiment datasets. The Distributed Robot Interaction Dataset (DROID) (Khazatsky et al., 2025) represents
another significant step towards addressing the problem of scarse and disaggregated data in robot learning, providing
a unique dataset consisting of 75k+ human demonstrations collected in realistic (in-the-wild ) manipulation settings,
providing another cornerstone for building general-purpose robot policies. Recently, foundational datasets curated
through large, centralized efforts, are increasingly complemented by decentralized, community-driven contributions of
robotics data. Software libraries like lerobot have been instrumental in enabling decentralized collection of large
amounts of data, providing the infrastructure for researchers and practitioners to easily contribute trajectories from a
wide range of embodiments, democratizing data access via distributed collection.
Despite these advancements, the success of large, proprietary models like RT-1 and RT-2, highlighted a growing
accessibility gap in robotics research, as training and deploying large-scale robotics foundation models requires
computational resources simply unattainable for most research institutions. The OpenVLA project (Kim et al.,
2024) emerged in direct contrast to traditionally closed-source efforts to develop VLAs. In particular, Kim et al.
(2024) trained OpenVLA by exclusively leveraging openly available data (970k+ trajectories from the Open-X
dataset), and openly shared their training recipes alongside the model weights. Architecturally, OpenVLA integrates
a pre-trained vision encoder to project visual tokens into the embedding space of the Llama2-7B (Touvron et al.,
2023) language-model backbone. The language model backbone is then used to predict discrete action tokens over
256 activation levels.
Figure 36 shows the current trends in robot learning in terms of size and nature of the robotics datasets contributed,
together with the size and accessibility of the available models. As datasets collected via centralized, cross-institutions
cooperation of increasing size are made available for the research community, decentralized datasets collected by
individual researchers and practitioners also gained traction, closing the gap with academic benchmarks thanks to

59
community-contributed datasets. Further, models used across tasks and embodiments are increasingly becoming much
more compute-efficient, and as a result the models’ size has been consistently reducing over time, with consequent
gains for autonomous robots in real-world, resource-constrained environments.

5.2 VLAs
Modern recipes to train large scale VLAs extend early efforts to learn foundation models from large amounts of data via
BC, introducing significant advancements concerning both architectural and procedural aspects. From an architectural
perspective, modern VLAs such as π0 (Black et al., 2024) leverage a unified transformer model for efficiency of
computation, while maintaining specialized sub-components within the model for visual perception and action
prediction, enabling cross-task performance via language conditioning. Crucially, modern VLAs includingπ0 (Black
et al., 2024) and SmolVLA (Shukor et al., 2025) adopt unified transformer models employing disjoint set of weights
(experts) for both compute-efficient visual-semantic understanding as well as control. Procedurally, VLAs complement
advanced Vision-Language Model (VLM) backbones with action-specific modules (1) adopting mid-sized action experts
to model continuous actions distributions p(at:t+Ha |ot )—avoiding discrete action tokens entirely—and (2) relying
on action chunking (Zhao et al., 2023, Section 4) as a strategy to reduce error compounding when predicting multiple
actions learning from inherently non-i.i.d. data, such as demonstration data.
These architectural and procedural innovations present three benefits over task-specific methods. First, developing
architectures that exploit internet-scale pre-trained backbones allows to fully capitalize on the vast world knowledge
and skills state-of-the-art VLMs exhibit, preventig models from needing to learn visual, linguistic and semantic
concepts from scratch. Second, using generative models for continuous action distributions allows to learn rich,
multimodal data distributions, a much more likely scenario in the big-data regime which is typically tackled while
developing generalist policies. Further, introducing separate components for perception and action planning enable
using Mixture of Experts (MoE) architectures (Fedus et al., 2022), which are often more efficient to run—a key
feature for models deployed in real-world scenarios. This new paradigm has been at the core of some of the most
capable generalist policies developed to date, capable to few-shot adapt to novel tasks and to perform highly dexterous
manipulation tasks ranging from end-to-end folding laundry to bussing tables (Black et al., 2024).

5.2.1 VLMs for VLAs

VLMs are designed to handle both visual and textual modalities, most commonly by taking both images and text as
inputs, generating text conditioned on the visual context. Recent advances in VLMs have been driven by the success
of LLMs, with many approaches building upon pretrained LLMs and adopting similar training paradigms to the
ones used in language modeling. Typically, VLMs (Alayrac et al., 2022; Laurençon et al., 2024; Lin et al., 2024) are
constructed by integrating a pretrained vision encoder (Radford et al., 2021; Zhai et al., 2023; Fini et al., 2024) with
a pretrained LLM (Grattafiori et al., 2024; Jiang et al., 2023). Training then proceeds in multiple multimodal stages,
beginning with a large-scale pretraining on datasets containing image-text pairs (Schuhmann et al., 2022; Byeon et al.,
2022) and interleaved vision-language corpora (Laurençon et al., 2023; Zhu et al., 2023), all followed by a supervised
fine-tuning stage on instruction-tuning datasets (Liu et al., 2023; Tong et al., 2024; Laurençon et al., 2024). The
inherent multimodal nature of VLMs enables them to jointly reason over vision and language. Pre-training on vast
internet-scale datasets allows these models to associate visual patterns with textual descriptions, thereby acquiring a
rich semantic understanding of the world—knowledge about objects, their properties, and relationships—without
explicit supervision for each concept. In turn, integrating VLMs as the perceptual backbone for VLAs allows the
latter to inherit rich, contextual world knowledge from the VLM, sidestepping the need to re-learn visual and semantic
representations. In principle, this also allows the robot to ground high-level natural language instructions in its visual
context, and possibly recognize objects by connecting them to the pre-trained concepts absorbed during pre-training,
improving on the possibility to generalize to novel scenarios.
Recently, compute efficiency has also become a central focus in multi-modal research. Several works aim to reduce
training costs by using smaller, more diverse datasets (Liu et al., 2023; Dai et al., 2023; Bai et al., 2025; Zhu et al.,
2024; Tong et al., 2024), training smaller-scale models (Marafioti et al., 2025; Korrapati, 2024; Yao et al., 2024), or by
adapting pretrained unimodal models by tuning only a small subset of parameters (Shukor et al., 2023; Vallaeys et al.,
2024; Mañas et al., 2023; Koh et al., 2023; Tsimpoukelli et al., 2021; Li et al., 2023). While the majority of VLM
research focuses on image and text modalities, recent work has also demonstrated that similar techniques can be
extended to integrate additional modalities, such as video and audio (Wang et al., 2025; Liu et al., 2024; Zhang et al.,
2025; Kong et al., 2024)—a particularly promising direction of research for robotics applications, where multiple
sensor modalities can be integrated effectively. This trend towards efficiency is paramount for robotics applications,
where policies must operate under the stringent constraints of real-world deployment.

60
Figure 37 | The π0 architecture, as in Black et al. (2024). Vision and language tokens are routed to a VLM backbone which
is prevented from attending robot proprioperceptive states and action tokens, which are instead routed to a smaller subset
of weights within the architecture referred to as "action expert". The architecture is trained with Flow Matching on 10M+
trajectories from a mixture of closed and openly available datasets.

5.3 π0
π0 (Black et al., 2024) introduce a VLA consisting of a MoE architecture consisting of (1) a pre-trained VLM backbone
(Gemma 2.6B (Team et al., 2024)) and (2) a dedicated action expert used to generate continuous actions via flow
matching. Images and language are embedded with PaliGemma, a VLM merging independently encoded visual and
textual features deep in the network (late-fusion), while proprioceptive state and actions chunks are routed to a
smaller action expert, initialized from scratch. The two separate experts communicate via self-attention layers, but
maintain disjoint weights to obtain query, key and values matrices at each layer, maintaining specialization while
efficiently allocating computation.
Concretely, π0 is a single, unified transformer with two disjoint sets of weights ϕ, θ. A larger VLM backbone fϕ
initialized from Gemma 2.6B processes multiple image frames obtained from multiple cameras points [{It }nt=1 ], as
well as a language instruction [ℓt ] used to describe the task considered. Concurrently, a 300M-parameter action expert
based on a similar transformer architecture is used to process both the robot proprioperceptive state qt and an action
chunk at:t+Ha (Figure 37). The different expert networks operate separately in processing the respective inputs and
turn them into query, key and value matrices, and only share information between each other via self-attention layers.
The outputs from the VLM backbone are disregarded, while the vector field regressed by the action expert is used to
iteratively refine the action process. In particular, π0 uses a blockwise causal attention mask over tokens belonging to
three separate blocks: (1) image and language tokens Ti obtained from [{It }nt=1 , ℓt ], (2) proprioperceptive tokens Tq
obtained from qt , and (3) the action tokens Ta for items in the chunk aτt:t+Ha at time τ in the flow-matching process.
Notably, within each block the attention operations are bidirectional, while across blocks, future blocks are masked
out. Formally, this corresponds to using an attention mask like:
T Tq Ta
 i 
Ti 1 0 0
A = Tq  1 1 0 , 1 : Bidirectional Attention, 0 : Masked Attention
Ta 1 1 1
Note how intra-block directional attention allows tokens to communicate freely, while inter -block communication is
mediated by the attention mask A. Blockwise causal masking effectively prevents the pre-trained perception-language
tokens from attending to robotics-tokens, likely out of distribution for VLM backbones traditionally trained on large
corpora of internet, non-robotics, data. Crucially, because communication is obstructed between image-language
tokens, proprioperceptive tokens and action tokens, one can cache keys and values across denoising steps at runtime
time, incuring in a reduced computational footprint and faster inference.

61
In π0 , both the VLM backbone and action expert are update using a flow matching loss, and in particular are updated
minimizing:
h i
2
L(ϕ, θ) = Eτ,ϵ,ot ,at:t+Ha vθ (τ at:t+Ha + (1 − τ )ϵ, ot , τ ) − (ϵ − at:t+Ha ) , (52)
| {z }
ãt:t+Ha

τ ∼ Beta[0,s] (1.5, 1), ϵ ∼ N (0, I), ot , at:t+Ha ∼ D

where the two experts parametrized by the separate weights ϕ, θ interact with each other via self-attention layers only, so
that the action expert vθ internal computations also depend on the VLM backbone’s parameters ϕ. Importantly, Black
et al. (2024) minimize eq. 52 over both the multimodal backbone and action expert parameters, thus updating both
the internal representations of the VLM and action-expert weights using BC-specific gradients. In contrast, Driess
et al. (2025) later show that failing to insulate the VLM knowledge from the flow matching gradients actually harms
performance.
At runtime, inference is performed iteratively refining action chunks while numerically forward-integrating the vector
field predicted by the action expert,

aτt:t+H
+δ
a
= aτt:t+Ha + δvθ (aτt:t+Ha , ot ) (53)

Flow matching (Lipman et al., 2023, Section4.1.3) can be seen as a continuous time, deterministic generalization
of diffusion processes, and has proven effective in modeling highly complex multi-modal distributions, including
those over images and video. In turn, the application of flow matching to large-scale datasets of multiple human
behaviors across tasks and embodiments appears rather consequential, particularly considering how it can enable
faster inference via a limited number of denoising steps at test time—as few as 10, in π0 . In particular, the action
expert is implemented as a conditional flow matching model. Each action token embeds a noisy action aτi ∈ aτt:t+Ha ,
alongside a sinusoidal encoding of the flow process timestep τ . The action expert then leverages full bidirectional
attention across the Ha action tokens provided, and also attends to previous proprioperceptive and image-language
tokens. Interestingly, differently from a standard flow matching pipeline (Lipman et al., 2023), τ is not sampled from
a uniform distribution τ ∼ U([0, 1]), but rather obtained from τ ∼ Beta(1.5, 1) defined on the [0, s], s < 1 support
(Figure 38).
Using such Beta distribution emphasizes higher noise levels dur-
ing training, a choice Black et al. (2024) argue allows π0 to fo-
cus on learning to reconstruct the mean of the data distribution
E[at:t+Ha |ot ] over an identity map during training, in keeping
with Esser et al. (2024). To further optimize performance and
reduce inference time, Black et al. (2024) propose reducing the
support of the timestep distribution to [0, s], s < 1, as for any
forward-integration step size δ = 1 − s timesteps above s are never
sampled at inference time.
Besides adopting a MoE architecture with a VLM backbone initial-
ized from a pre-trained model and trained jointly with an action
expert via flow matching, π0 also relies on a unique pre-training
corpus comprising of a mix of proprietary and open data totaling Figure 38 | Unlike more traditional flow-matching
10M+ trajectories, which in their work Black et al. (2024) claim algorithms, π0 uses a modified distribution to sam-
to be the largest dataset used to develop a foundational robotics ple the timestep τ from during training and infer-
model to date. The dataset used to train π0 —referred to as "the ence, favouring earlier timestamps corresponding
π dataset"—comprises a private, undisclosed portion obtained via to noisier chunks.
expert teleoperation as well as openly available datasets including
Open-X and DROID, with only ≈ 9.1% of the π being openly
available. In the π dataset, open datasets such as DROID and Open-X are complemeneted with expert trajectories
consisting of dexterous demonstrations tasks spanning 7 robot configurations and 68 different tasks. Crucially, Black
et al. (2024) show that pre-training on the π dataset yields a broadly capable base model, which can be adapted
via fine-tuning on narrower, higher-quality task data, which induces a fluent multi-stage behavior while retaining
robustness. In particular, Black et al. (2024) report that, across a variety of benchmarks, the version of π0 pretrained
on the π dataset and fine-tuned on extra high-quality data demonstrations consistently outperforms a π0scratch baseline
trained entirely from scratch for a given specific task, which further underscores the relevance of pretraining on the π
dataset. Black et al. (2024) do also offer an intuition behind this finding: high-quality demonstrations of a given task

62
tend to omit failure data, which inherently prevents an autonomous agent to learn how to recover from near-failure
states. In turn, robot trained on high-quality data exclusively with BC may as well be entirely incapable to recover
from failure. Conversely, large scale collections of human demonstrations are typically much more diverse (if anything,
for their sheer scale), and typically contain rich and diverse information, which may prove suboptimal for any given
task when considered in isolation but which proves invaluable in coupling with a small, narrower set of demonstrations.
Lastly, Black et al. (2024) present cross-embodiment experiments where they demonstrate π0 ’s ability to control
both mobile and static manipulator robots with varying arm embodiments. The emergence of cross-embodiment
capabilities is largely to be attributed to the presence of large scale cross-embodiment data in π data mixture, which
is in practice handled by π0 outputting actions with maximal configuration size across the whole π dataset, and
zero-padding robots with fewer dofs. π0 does also rely on exactly three camera views at both training and test time,
and uses masked image slots for training and deployment scenarios with fewer cameras.

5.3.1 Code Example: Using π0

Code 13: Using π0

[Link]

1 import torch
2
3 from lerobot . cameras . opencv . c o n f i g u r a t i o n _ o p e n c v import O p e n C V C a m e r a C o n f i g
4 from lerobot . datasets . utils import h w _ t o _ d a t a s e t _ f e a t u r e s
5 from lerobot . policies . factory import m a k e _ p r e _ p o s t _ p r o c e s s o r s
6 from lerobot . policies . pi0 . modeling_pi0 import PI0Policy
7 from lerobot . policies . utils import build_inference_frame , m a k e _ r o b o t _ a c t i o n
8 from lerobot . robots . so 100_fo llower . c o n f i g _ s o 1 0 0 _ f o l l o w e r import S O 1 0 0 F o l l o w e r C o n f i g
9 from lerobot . robots . so 100_fo llower . so10 0_foll ower import SO100Follower
10
11 MAX_EPISODES = 5
12 M A X _ S T E P S _ P E R _ E P I S O D E = 20
13
14 device = torch . device ( " mps " ) # or " cuda " or " cpu "
15 model_id = " lerobot / pi0_base "
16
17 model = PI0Policy . f ro m _p r et ra i ne d ( model_id )
18
19 preprocess , postprocess = m a k e _ p r e _ p o s t _ p r o c e s s o r s (
20 model . config ,
21 model_id ,
22 # This overrides allows to run on MPS , otherwise defaults to CUDA ( if available )
23 p r e p r o c e s s o r _ o v e r r i d e s ={ " d e v i c e _ p r o c e s s o r " : { " device " : " mps " }} ,
24 )
25
26 # find ports using lerobot - find - port
27 follower_port = ... # something like "/ dev / tty . u s b m o d e m 5 8 7 6 0 4 3 1 6 3 1 "
28
29 # the robot ids are used the load the right calibration files
30 follower_id = ... # something like " fol lower_ so100 "
31
32 # Robot and environment configuration
33 # Camera keys must match the name and resolutions of the ones used for training !
34 # You can check the camera keys expected by a model in the info . json card on the Hub
35 camera_config = {
36 " base_0_rgb " : O p e n C V C a m e r a C o n f i g ( index_or_path =0 , width =640 , height =480 , fps =30) ,
37 " l e f t _ w r i s t _ 0 _ r g b " : O p e n C V C a m e r a C o n f i g ( index_or_path =1 , width =640 , height =480 , fps =30) ,
38 " r i g h t _ w r i s t _ 0 _ r g b " : O p e n C V C a m e r a C o n f i g ( index_or_path =2 , width =640 , height =480 , fps =30) ,
39 }
40
41 robot_cfg = S O 1 0 0 F o l l o w e r C o n f i g ( port = follower_port , id = follower_id , cameras = camera_config )
42 robot = SO100Follower ( robot_cfg )
43 robot . connect ()
44
45 task = ... # something like " pick the red block "
46 robot_type = ... # something like " so100 _follo wer " for multi - embodiment datasets
47

63
Figure 39 | The SmolVLA architecture, as in Shukor et al. (2025). SmolVLA is a compact MoE model trained with flow
matching to denoise action chunks. Vision and language tokens are fed to a VLM backbone, and share information with the
proprioperceptive and action tokens via the attention mechanism. The attention expert interleaves SA and CA layers for
further conditioning on the visual features from the VLM backbone. SmolVLA skips computations and reduces the visual
tokens, resulting in 7x less memory usage than π0 (450M parameters vs. π0 ’s 3.3B).

48 # This is used to match the raw observation keys to the keys expected by the policy
49 ac ti o n_ fe a tu re s = h w _ t o _ d a t a s e t _ f e a t u r e s ( robot . action_features , " action " )
50 obs_features = h w _ t o _ d a t a s e t _ f e a t u r e s ( robot . observation_features , " observation " )
51 d a t a s e t _ f e a t u r e s = {** action_features , ** obs_features }
52
53 for _ in range ( MAX_EPISODES ):
54 for _ in range ( M A X _ S T E P S _ P E R _ E P I S O D E ):
55 obs = robot . ge t_ o bs er v at io n ()
56 obs_frame = b u i l d _ i n f e r e n c e _ f r a m e (
57 obs , dataset_features , device , task = task , robot_type = robot_type
58 )
59
60 obs = preprocess ( obs_frame )
61
62 action = model . select_action ( obs )
63 action = postprocess ( action )
64 action = m a k e _ r o b o t _ a c t i o n ( action , d a t a s e t _ f e a t u r e s )
65 robot . send_action ( action )
66
67 print ( " Episode finished ! Starting new episode ... " )

5.4 SmolVLA
With VLAs in the early stage of development compared to more mature LLMs and VLMs, much of the progress
made on VLAs remains proprietary, with many releases exclusively sharing the weights while withholding the data
used, full experimental details and essential methodological components of training. In constrast with this closed
approach, SmolVLA (Shukor et al., 2025) is an entirely open-source research effort, which aims at democratizing
the developments of robotics foundation models by open sourcing the model alongside the data used as well as the
training recipes.
While encouraging efforts like π0 (Black et al., 2024) demonstrate the feasibility of open VLA systems, they remain
(1) large and compute-intensive and (2) dependent on closed datasets collected via centralized efforts on costly robotic
platforms, which ultimately hinders the accessibility of the method altogether. SmolVLA mitigates both these issues
by (1) prioritizing a compact, compute-efficient VLA design and (2) targeting community-contributed datasets on

64
accessible robotic platforms such as the SO-100 and SO-101 arms. Similarly to π0 , SmolVLA (Figure 39) employs
a MoE architecture combining a pretrained VLM backbone with a dedicated action expert, and trains with flow
matching. To ensure efficiency and accessibility, SmolVLA adopts SmolVLM-2 (Marafioti et al., 2025) as its VLM
backbone, considering SmolVLM-2’s reduced size and capability to process multiple image inputs alongside text items.
SmolVLM-2 uses SigLIP (Zhai et al., 2023) as vision encoder, producing visual features for a SmolLM2 language
decoder (Allal et al., 2025). Further, SmolVLA adopts a smaller action expert consisting of ∼100M parameters and
an interleaved stack of self and cross-attention layers. To improve efficiency, the action expert adopts a reduced
embedding dimension compared to the VLM backbone, resulting in dvθ = 0.75dVLM . Shukor et al. (2025)’s design
choices thus result in a much smaller size model compared to π0 , consisting of ca. 450M parameters versus π0 ’s 3.3B
parameters.
In practice, SmolVLA consumes multi-view RGB images, a natural-language instruction, and projected sensorimotor
state token as inputs, together with the noised action chunk ãt:t+Ha the action expert vθ is trained to denoise. The
robot proprioperceptive states are projected to a shared token space with the VLM to match dVLM , and successively
projected into the expert’s token space. Similarily to π0 , SmolVLA adopts separate experts communicating exclusively
through self-attention layers, which however do not employ blockwise causal attention masking and rather favour
simple causal masking.
In contrast with π0 , the action expert interleaves cross-attention (CA) and self-attention (SA) layers, a choice shown
to yield higher success and smoother action chunks in practice. While in the expert SA layers tokens are used to
obtain queries, keys and values, CA layers use action tokens only as queries, and instead project visual, language
and proprioperceptive tokens from the VLM backbone to a shared embedding space to then obtain keys and values.
Notably, keys and values can be cached here as well, resulting in performance gains at inference time.
SmolVLA also trims down both token and layer compute. First, it reduces visual tokens via pixel shuffling to a
fixed budget of 64 tokens per frame, foregoing the tiling used during VLM pretraining for the sake of runtime
efficiency. Second, it skips upper VLM layers, as only features from the first N decoder layers, with N = L/2, are
consumed, which provides a good speed-performance trade-off and effectively halves compute needs for the larger
part of SmolVLA. Beyond model compactness, SmolVLA also contributes an inference stack that decouples action
prediction from execution for responsiveness on modest hardware (Section 4.4).
Departing from reliance on proprietary datasets, SmolVLA pretrains exclusively on 450+ community datasets, totaling
20k+ trajectories. Because instructions in community contributed dataset can be noisy or missing, the authors
re-annotate tasks with a small off-the-shelf VLM using frames sampled from the dataset, and standardize camera
viewpoints by mapping sources to a consistent top/wrist/side ordering. At test time, similarily to π0 , SmolVLA
forward-integrates flow over 10 steps, resulting in fast inference. SmolVLA proves effective across a range of both real-
world and simulated environments, rivaling π0 while being close to 40% faster and consuming 6x less memory (Shukor
et al., 2025).

5.4.1 Code Example: Using SmolVLA

Code 14: Using SmolVLA

[Link]

65
17 model = SmolVLAPolicy . f ro m _p r et ra i ne d ( model_id )
18
19 preprocess , postprocess = m a k e _ p r e _ p o s t _ p r o c e s s o r s (
20 model . config ,
21 model_id ,
22 # This overrides allows to run on MPS , otherwise defaults to CUDA ( if available )
23 p r e p r o c e s s o r _ o v e r r i d e s ={ " d e v i c e _ p r o c e s s o r " : { " device " : " mps " }} ,
24 )
25
26 # find ports using lerobot - find - port
27 follower_port = ... # something like "/ dev / tty . u s b m o d e m 5 8 7 6 0 4 3 1 6 3 1 "
28
29 # the robot ids are used the load the right calibration files
30 follower_id = ... # something like " fol lower_ so100 "
31
32 # Robot and environment configuration
33 # Camera keys must match the name and resolutions of the ones used for training !
34 # You can check the camera keys expected by a model in the info . json card on the Hub
35 camera_config = {
36 " camera1 " : O p e n C V C a m e r a C o n f i g ( index_or_path =0 , width =640 , height =480 , fps =30) ,
37 " camera2 " : O p e n C V C a m e r a C o n f i g ( index_or_path =1 , width =640 , height =480 , fps =30) ,
38 }
39
40 robot_cfg = S O 1 0 0 F o l l o w e r C o n f i g ( port = follower_port , id = follower_id , cameras = camera_config )
41 robot = SO100Follower ( robot_cfg )
42 robot . connect ()
43
44 task = ... # something like " pick the red block "
45 robot_type = ... # something like " so100 _follo wer " for multi - embodiment datasets
46
47 # This is used to match the raw observation keys to the keys expected by the policy
48 ac ti o n_ fe a tu re s = h w _ t o _ d a t a s e t _ f e a t u r e s ( robot . action_features , " action " )
49 obs_features = h w _ t o _ d a t a s e t _ f e a t u r e s ( robot . observation_features , " observation " )
50 d a t a s e t _ f e a t u r e s = {** action_features , ** obs_features }
51
52 for _ in range ( MAX_EPISODES ):
53 for _ in range ( M A X _ S T E P S _ P E R _ E P I S O D E ):
54 obs = robot . ge t_ o bs er v at io n ()
55 obs_frame = b u i l d _ i n f e r e n c e _ f r a m e (
56 obs , dataset_features , device , task = task , robot_type = robot_type
57 )
58
59 obs = preprocess ( obs_frame )
60
61 action = model . select_action ( obs )
62 action = postprocess ( action )
63 action = m a k e _ r o b o t _ a c t i o n ( action , d a t a s e t _ f e a t u r e s )
64 robot . send_action ( action )
65
66 print ( " Episode finished ! Starting new episode ... " )

66
6 Conclusions
This tutorial has charted the paradigmatic shift transforming robotics, tracing the evolution of robotics from
structured, model-based methods to the dynamic, data-driven approaches that define modern robot learning. We
began by examining the limitations of traditional dynamics-based control, namely its brittleness and significant
engineering overhead, which motivate the adoption of more flexible, learning-based alternatives. Unlike scalable,
data-driven techniques, conventional explicit models demand extensive human expertise, hindering wider accessibility
and scalability of robotics.
Our exploration traced a clear trajectory of progress, beginning with Reinforcement Learning (RL). While RL offers
a powerful paradigm for learning through interaction, its application in robotics is complicated by challenges such
as sample inefficiency, safety concerns in real-world training, and the complexities of reward design. We saw how
modern approaches like HIL-SERL make real-world RL more feasible by incorporating training-time human guidance,
datasets of previously collected data as well as learned reward classifiers.
Nonetheless, the inherent difficulties of RL increasingly motivate approaches based on imitation learning, capable to
safely learns from limited numbers of real-world, reward-free expert demonstrations. In turn, the wider adoption of
imitation learning led to the development of single-task policies, where advanced Behavioral Cloning techniques—
implemented as state-conditioned generative models like Action Chunking with Transformers and Diffusion Policy—have
demonstrated the ability to learn complex, multimodal behaviors from human demonstrations. These advancements
laid the groundwork for the current frontier: generalist, language-conditioned Vision-Language-Action models capable
to perform few- and zero-shot a variety of different real-world tasks. By leveraging powerful pre-trained backbones
and sophisticated generative methods like flow matching, models such as π0 and SmolVLA represent a significant leap
towards foundational models for robotics capable of generalizing across diverse tasks, and even robot embodiments.
A central theme of this work is the critical role of openness in accelerating this progress. The recent explosion in
capability is inseparable from the advent of large-scale, openly available datasets, standardized, stable and accessible
model architectures, and accessible, open-source software like lerobot. We argue this convergence on open-source
robotics is not a mere trend but a fundamental enabler, democratizing access to research and unlocking the potential
of large, decentralized efforts to advance the field.
The journey detailed in this tutorial, from first principles to the state-of-the-art, aims to equip researchers and
practitioners with the context and tools to begin their own explorations in open-source robot learning.

References
Joshua Achiam. Spinning up in deep reinforcement learning. 2018.
Pulkit Agrawal. Computational Sensorimotor Learning.
Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias
Plappert, Glenn Powell, Raphael Ribas, Jonas Schneider, Nikolas Tezak, Jerry Tworek, Peter Welinder, Lilian Weng, Qiming
Yuan, Wojciech Zaremba, and Lei Zhang. Solving Rubik’s Cube with a Robot Hand, October 2019.
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie
Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei,
Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj
Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karen Simonyan. Flamingo: A Visual Language Model
for Few-Shot Learning, November 2022.
Jorge Aldaco, Travis Armstrong, Robert Baruch, Jeff Bingham, Sanky Chan, Debidatta Dwibedi, Chelsea Finn, Pete Florence,
Spencer Goodrich, Wayne Gramlich, Alexander Herzog, Jonathan Hoech, Thinh Nguyen, Ian Storz, Baruch Tabanpour,
Jonathan Tompson, Ayzaan Wahid, Ted Wahrburg, Sichun Xu, Sergey Yaroshenko, and Tony Z Zhao. ALOHA 2: An
Enhanced Low-Cost Hardware for Bimanual Teleoperation.
Mohammad Alizadeh and Zheng H. Zhu. A comprehensive survey of space robotic manipulators for on-orbit servicing. Frontiers
in Robotics and AI, 11, October 2024. ISSN 2296-9144. doi: 10.3389/frobt.2024.1470950.
Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guilherme Penedo, Lewis Tunstall, Andrés
Marafioti, Hynek Kydlíček, Agustín Piqueres Lajarín, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan-Son Nguyen,
Clémentine Fourrier, Ben Burtenshaw, Hugo Larcher, Haojun Zhao, Cyril Zakka, Mathieu Morlon, Colin Raffel, Leandro von
Werra, and Thomas Wolf. SmolLM2: When Smol Goes Big – Data-Centric Training of a Small Language Model, February
2025.

67
Rika Antonova, Silvia Cruciani, Christian Smith, and Danica Kragic. Reinforcement Learning for Pivoting Task, March 2017.

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang,
Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu,
Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-VL
technical report, 2025.

Philip J. Ball, Laura Smith, Ilya Kostrikov, and Sergey Levine. Efficient Online Reinforcement Learning with Offline Data,
May 2023.

Kostas E. Bekris, Joe Doerr, Patrick Meng, and Sumanth Tangirala. The State of Robot Motion Generation, October 2024.

Marc G. Bellemare, Salvatore Candido, Pablo Samuel Castro, Jun Gong, Marlos C. Machado, Subhodeep Moitra, Sameera S.
Ponda, and Ziyu Wang. Autonomous navigation of stratospheric balloons using reinforcement learning. Nature, 588(7836):
77–82, December 2020. ISSN 1476-4687. doi: 10.1038/s41586-020-2939-8.

Richard Bellman. A Markovian Decision Process. Journal of Mathematics and Mechanics, 6(5):679–684, 1957. ISSN 0095-9057.

Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi "Jim" Fan, Yu Fang, Dieter Fox,
Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu
Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed,
You Liang Tan, Guanzhi Wang, Zu Wang, Jing Wang, Qi Wang, Jiannan Xiang, Yuqi Xie, Yinzhen Xu, Zhenjia Xu,
Seonghyeon Ye, Zhiding Yu, Ao Zhang, Hao Zhang, Yizhou Zhao, Ruijie Zheng, and Yuke Zhu. GR00T N1: An Open
Foundation Model for Generalist Humanoid Robots, March 2025.

Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol
Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri,
Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky.
$π_0$: A Vision-Language-Action Flow Model for General Robot Control, October 2024.

Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny
Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan,
Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry
Kalashnikov, Yuheng Kuang, Isabel Leal, Lisa Lee, Tsang-Wei Edward Lee, Sergey Levine, Yao Lu, Henryk Michalewski,
Igor Mordatch, Karl Pertsch, Kanishka Rao, Krista Reymann, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Pierre
Sermanet, Jaspiar Singh, Anikait Singh, Radu Soricut, Huong Tran, Vincent Vanhoucke, Quan Vuong, Ayzaan Wahid,
Stefan Welker, Paul Wohlhart, Jialin Wu, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. RT-2:
Vision-Language-Action Models Transfer Web Knowledge to Robotic Control, July 2023a.

Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan,
Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil J.
Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Kuang-Huei Lee, Sergey Levine, Yao Lu, Utsav Malla,
Deeksha Manjunath, Igor Mordatch, Ofir Nachum, Carolina Parada, Jodilyn Peralta, Emily Perez, Karl Pertsch, Jornell
Quiambao, Kanishka Rao, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Kevin Sayed, Jaspiar Singh, Sumedh Sontakke,
Austin Stone, Clayton Tan, Huong Tran, Vincent Vanhoucke, Steve Vega, Quan Vuong, Fei Xia, Ted Xiao, Peng Xu, Sichun
Xu, Tianhe Yu, and Brianna Zitkovich. RT-1: Robotics Transformer for Real-World Control at Scale, August 2023b.

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav
Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon
Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz
Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and
Dario Amodei. Language Models are Few-Shot Learners, July 2020.

Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Baek, and Saehoon Kim. COYO-700M: Image-text
pair dataset, 2022.

Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan Ratliff, and Dieter Fox. Closing
the sim-to-real loop: Adapting simulation randomization with real world experience. In 2019 International Conference on
Robotics and Automation (ICRA), pages 8973–8979. IEEE, 2019.

Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Carlos Riquelme Ruiz, Sebastian
Goodman, Xiao Wang, Yi Tay, Siamak Shakeri, Mostafa Dehghani, Daniel Salz, Mario Lucic, Michael Tschannen, Arsha
Nagrani, Hexiang Hu, Mandar Joshi, Bo Pang, Ceslee Montgomery, Paulina Pietrzyk, Marvin Ritter, A. J. Piergiovanni,
Matthias Minderer, Filip Pavetic, Austin Waters, Gang Li, Ibrahim Alabdulmohsin, Lucas Beyer, Julien Amelot, Kenton Lee,
Andreas Peter Steiner, Yang Li, Daniel Keysers, Anurag Arnab, Yuanzhong Xu, Keran Rong, Alexander Kolesnikov, Mojtaba
Seyedhosseini, Anelia Angelova, Xiaohua Zhai, Neil Houlsby, and Radu Soricut. PaLI-X: On Scaling up a Multilingual Vision
and Language Model, May 2023.

68
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion
Policy: Visuomotor Policy Learning via Action Diffusion, March 2024.

Jonathan H. Connell and Sridhar Mahadevan, editors. Robot Learning. Springer US, Boston, MA, 1993. ISBN 978-1-4613-6396-5
978-1-4615-3184-5. doi: 10.1007/978-1-4615-3184-5.

Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi.
InstructBLIP: Towards general-purpose vision-language models with instruction tuning. In Thirty-Seventh Conference on
Neural Information Processing Systems, 2023.

Jonas Degrave, Federico Felici, Jonas Buchli, Michael Neunert, Brendan Tracey, Francesco Carpanese, Timo Ewalds, Roland
Hafner, Abbas Abdolmaleki, Diego de las Casas, Craig Donner, Leslie Fritz, Cristian Galperti, Andrea Huber, James Keeling,
Maria Tsimpoukelli, Jackie Kay, Antoine Merle, Jean-Marc Moret, Seb Noury, Federico Pesamosca, David Pfau, Olivier
Sauter, Cristian Sommariva, Stefano Coda, Basil Duval, Ambrogio Fasoli, Pushmeet Kohli, Koray Kavukcuoglu, Demis
Hassabis, and Martin Riedmiller. Magnetic control of tokamak plasmas through deep reinforcement learning. Nature, 602
(7897):414–419, February 2022. ISSN 1476-4687. doi: 10.1038/s41586-021-04301-9.

J. Deng, K. Li, M. Do, H. Su, and L. Fei-Fei. Construction and analysis of a large scale image ontology. Vision Sciences Society,
2009.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers
for Language Understanding, May 2019.

Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan
Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine,
Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. PaLM-E:
An Embodied Multimodal Language Model, March 2023.

Danny Driess, Jost Tobias Springenberg, Brian Ichter, Lili Yu, Adrian Li-Bell, Karl Pertsch, Allen Z. Ren, Homer Walke, Quan
Vuong, Lucy Xiaoyang Shi, and Sergey Levine. Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast,
Generalize Better, May 2025.

Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz,
Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and
Robin Rombach. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis, March 2024.

William Fedus, Jeff Dean, and Barret Zoph. A Review of Sparse Expert Models in Deep Learning, September 2022.

Enrico Fini, Mustafa Shukor, Xiujun Li, Philipp Dufter, Michal Klein, David Haldimann, Sai Aitharaju, Victor Guilherme Turrisi
da Costa, Louis Béthune, Zhe Gan, Alexander T. Toshev, Marcin Eichner, Moin Nabi, Yinfei Yang, Joshua M. Susskind, and
Alaaeldin El-Nouby. Multimodal Autoregressive Pre-training of Large Vision Encoders, November 2024.

Pete Florence, Corey Lynch, Andy Zeng, Oscar A. Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee, Igor
Mordatch, and Jonathan Tompson. Implicit Behavioral Cloning. In Proceedings of the 5th Conference on Robot Learning,
pages 158–168. PMLR, January 2022.

Jun Fujita, Daisuke Soda, Chotaro Murata, and Hiroyuki Tsuhari. Development of Robots for Nuclear Power Plants and Their
Application to New Fields. 57(4), 2020.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman,
Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang,
Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen
Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak,
Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton
Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu,
Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy,
Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve,
Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell,
Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan
Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet
Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu,
Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng
Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone,
Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary, Laurens
van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas
Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria

69
Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh,
Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier
Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik
Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy,
Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel,
Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana
Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath
Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman,
Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas
Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta,
Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei
Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaofang Wang, Xiaoqing Ellen Tan,
Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song,
Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya
Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand,
Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo,
Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan,
Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel,
Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth
Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence,
Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang,
Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia
Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David,
Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine
Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar,
Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet,
Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory
Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha,
Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor
Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice
Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy
Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres,
Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich,
Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin
Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca
Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias
Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle
Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso,
Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa,
Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg
Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner,
Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad
Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond
Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu,
Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma,
Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha,
Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith
Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer
Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara
Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun
Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad Ionescu, Vlad
Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable,
Xiaocheng Tang, Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi,
Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li,
Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The Llama 3
Herd of Models, November 2024.

Robert J. Griffin, Georg Wiedebach, Sylvain Bertrand, Alexander Leonessa, and Jerry Pratt. Walking Stabilization Using
Step Timing and Location Adjustment on the Humanoid Robot, Atlas. In 2017 IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS), pages 667–673, September 2017. doi: 10.1109/IROS.2017.8202223.

Tuomas Haarnoja, Haoran Tang, Pieter Abbeel, and Sergey Levine. Reinforcement Learning with Deep Energy-Based Policies.
In Proceedings of the 34th International Conference on Machine Learning, pages 1352–1361. PMLR, July 2017.

70
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft Actor-Critic: Off-Policy Maximum Entropy Deep
Reinforcement Learning with a Stochastic Actor, August 2018.

Nicklas Hansen, Xiaolong Wang, and Hao Su. Temporal Difference Learning for Model Predictive Control, July 2022.

Nicolas Heess, Dhruva TB, Srinivasan Sriram, Jay Lemmon, Josh Merel, Greg Wayne, Yuval Tassa, Tom Erez, Ziyu Wang,
S. M. Ali Eslami, Martin Riedmiller, and David Silver. Emergence of Locomotion Behaviours in Rich Environments, July
2017.

Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and
Alexander Lerchner. Beta-vae: Learning basic visual concepts with a constrained variational framework. In International
Conference on Learning Representations, 2017.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models, December 2020.

Eric Jang, Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, and Chelsea Finn. BC-Z:
Zero-Shot Task Generalization with Robotic Imitation Learning, February 2022.

Michael Janner, Yilun Du, Joshua B. Tenenbaum, and Sergey Levine. Planning with Diffusion for Flexible Behavior Synthesis,
December 2022.

Yandong Ji, Gabriel B. Margolis, and Pulkit Agrawal. DribbleBot: Dynamic Legged Manipulation in the Wild, April 2023.

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian
Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock,
Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7B, October 2023.

Liyiming Ke, Jingqiang Wang, Tapomayukh Bhattacharjee, Byron Boots, and Siddhartha Srinivasa. Grasping with Chopsticks:
Combating Covariate Shift in Model-free Imitation Learning for Fine Manipulation, November 2020.

Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany,
Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, Peter David Fagan, Joey Hejna, Masha Itkina, Marion Lepert,
Yecheng Jason Ma, Patrick Tree Miller, Jimmy Wu, Suneel Belkhale, Shivin Dass, Huy Ha, Arhan Jain, Abraham Lee,
Youngwoon Lee, Marius Memmel, Sungjae Park, Ilija Radosavovic, Kaiyuan Wang, Albert Zhan, Kevin Black, Cheng Chi,
Kyle Beltran Hatch, Shan Lin, Jingpei Lu, Jean Mercat, Abdul Rehman, Pannag R. Sanketi, Archit Sharma, Cody Simpson,
Quan Vuong, Homer Rich Walke, Blake Wulfe, Ted Xiao, Jonathan Heewon Yang, Arefeh Yavary, Tony Z. Zhao, Christopher
Agia, Rohan Baijal, Mateo Guaman Castro, Daphne Chen, Qiuyu Chen, Trinity Chung, Jaimyn Drake, Ethan Paul Foster,
Jensen Gao, Vitor Guizilini, David Antonio Herrera, Minho Heo, Kyle Hsu, Jiaheng Hu, Muhammad Zubair Irshad, Donovon
Jackson, Charlotte Le, Yunshuang Li, Kevin Lin, Roy Lin, Zehan Ma, Abhiram Maddukuri, Suvir Mirchandani, Daniel
Morton, Tony Nguyen, Abigail O’Neill, Rosario Scalise, Derick Seale, Victor Son, Stephen Tian, Emi Tran, Andrew E.
Wang, Yilin Wu, Annie Xie, Jingyun Yang, Patrick Yin, Yunchu Zhang, Osbert Bastani, Glen Berseth, Jeannette Bohg, Ken
Goldberg, Abhinav Gupta, Abhishek Gupta, Dinesh Jayaraman, Joseph J. Lim, Jitendra Malik, Roberto Martín-Martín,
Subramanian Ramamoorthy, Dorsa Sadigh, Shuran Song, Jiajun Wu, Michael C. Yip, Yuke Zhu, Thomas Kollar, Sergey
Levine, and Chelsea Finn. DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset, April 2025.

Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster,
Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine,
Percy Liang, and Chelsea Finn. OpenVLA: An Open-Source Vision-Language-Action Model, September 2024.

Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.

Rob Knight, Pepijn Kooijmans, Thomas Wolf, Simon Alibert, Michel Aractingi, Dana Aubakirova, Adil Zouitine, Russi Martino,
Steven Palma, Caroline Pascal, and Remi Cadene. Standard Open SO-100 & SO-101 Arms.

Jens Kober, J Andrew Bagnell, and Jan Peters. Reinforcement Learning in Robotics: A Survey.

Jing Yu Koh, Ruslan Salakhutdinov, and Daniel Fried. Grounding language models to images for multimodal inputs and
outputs, 2023.

Zhifeng Kong, Arushi Goel, Rohan Badlani, Wei Ping, Rafael Valle, and Bryan Catanzaro. Audio flamingo: A novel audio
language model with few-shot learning and dialogue abilities. In International Conference on Machine Learning, pages
25125–25148. PMLR, 2024.

Vik Korrapati. Moondream. Online, 2024.

Hugo Laurençon, Lucile Saulnier, Leo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth
Karamcheti, Alexander M Rush, Douwe Kiela, Matthieu Cord, and Victor Sanh. OBELICS: An open web-scale filtered
dataset of interleaved image-text documents. In Thirty-Seventh Conference on Neural Information Processing Systems
Datasets and Benchmarks Track, 2023.

71
Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models?, May
2024.

Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, and Marco Hutter. Learning Quadrupedal Locomotion over
Challenging Terrain. Science Robotics, 5(47):eabc5986, October 2020. ISSN 2470-9476. doi: 10.1126/scirobotics.abc5986.

Seungjae Lee, Yibin Wang, Haritheja Etukuru, H. Jin Kim, Nur Muhammad Mahi Shafiullah, and Lerrel Pinto. Behavior
Generation with Latent Actions, June 2024.

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pre-training with frozen image
encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, ICML’23, ,
Honolulu, Hawaii, USA„ 2023. [Link].

Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan
Wierstra. Continuous control with deep reinforcement learning, July 2019.

Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, and Song
Han. VILA: On Pre-training for Visual Language Models, May 2024.

Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow Matching for Generative Modeling,
February 2023.

Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky T. Q. Chen, David Lopez-Paz,
Heli Ben-Hamu, and Itai Gat. Flow Matching Guide and Code, December 2024.

Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In NeurIPS 2023
Workshop on Instruction Tuning and Instruction Following, 2023.

Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu.
Kangaroo: A powerful video-language model supporting long-context video input. arXiv preprint arXiv:2408.15542, 2024.

Calvin Luo. Understanding Diffusion Models: A Unified Perspective, August 2022.

Jianlan Luo, Charles Xu, Jeffrey Wu, and Sergey Levine. Precise and Dexterous Robotic Manipulation via Human-in-the-Loop
Reinforcement Learning, October 2024.

Jianlan Luo, Zheyuan Hu, Charles Xu, You Liang Tan, Jacob Berg, Archit Sharma, Stefan Schaal, Chelsea Finn, Abhishek
Gupta, and Sergey Levine. SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning, March 2025.

Kevin M. Lynch and Frank C. Park. Modern Robotics: Mechanics, Planning, and Control. Cambridge University Press, 1
edition, May 2017. ISBN 978-1-316-66123-9 978-1-107-15630-2 978-1-316-60984-2. doi: 10.1017/9781316661239.

Oscar Mañas, Pau Rodriguez Lopez, Saba Ahmadi, Aida Nematzadeh, Yash Goyal, and Aishwarya Agrawal. MAPL:
Parameter-efficient adaptation of unimodal pre-trained models for vision-language few-shot prompting. In Andreas Vlachos
and Isabelle Augenstein, editors, Proceedings of the 17th Conference of the European Chapter of the Association for
Computational Linguistics, pages 2523–2548, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics. doi:
10.18653/v1/[Link]-main.185.

Andrés Marafioti, Orr Zohar, Miquel Farré, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton
Lozhkov, Nouamane Tazi, Vaibhav Srivastav, Joshua Lochner, Hugo Larcher, Mathieu Morlon, Lewis Tunstall, Leandro von
Werra, and Thomas Wolf. SmolVLM: Redefining small and efficient multimodal models, April 2025.

Gabriel B. Margolis, Ge Yang, Kartik Paigwar, Tao Chen, and Pulkit Agrawal. Rapid Locomotion via Reinforcement Learning,
May 2022.

John McCormac, Ankur Handa, Andrew Davison, and Stefan Leutenegger. SemanticFusion: Dense 3D Semantic Mapping with
Convolutional Neural Networks, September 2016.

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller.
Playing Atari with Deep Reinforcement Learning, December 2013.

Preetum Nakkiran, Arwen Bradley, Hattie Zhou, and Madhu Advani. Step-by-Step Diffusion: An Elementary Tutorial, June
2024.

72
Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee,
Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Herzog, Alex Irpan, Alexander
Khazatsky, Anant Rai, Anchit Gupta, Andrew Wang, Andrey Kolobov, Anikait Singh, Animesh Garg, Aniruddha Kembhavi,
Annie Xie, Anthony Brohan, Antonin Raffin, Archit Sharma, Arefeh Yavary, Arhan Jain, Ashwin Balakrishna, Ayzaan
Wahid, Ben Burgess-Limerick, Beomjoon Kim, Bernhard Schölkopf, Blake Wulfe, Brian Ichter, Cewu Lu, Charles Xu,
Charlotte Le, Chelsea Finn, Chen Wang, Chenfeng Xu, Cheng Chi, Chenguang Huang, Christine Chan, Christopher Agia,
Chuer Pan, Chuyuan Fu, Coline Devin, Danfei Xu, Daniel Morton, Danny Driess, Daphne Chen, Deepak Pathak, Dhruv
Shah, Dieter Büchler, Dinesh Jayaraman, Dmitry Kalashnikov, Dorsa Sadigh, Edward Johns, Ethan Foster, Fangchen Liu,
Federico Ceola, Fei Xia, Feiyu Zhao, Felipe Vieira Frujeri, Freek Stulp, Gaoyue Zhou, Gaurav S. Sukhatme, Gautam Salhotra,
Ge Yan, Gilbert Feng, Giulio Schiavi, Glen Berseth, Gregory Kahn, Guangwen Yang, Guanzhi Wang, Hao Su, Hao-Shu
Fang, Haochen Shi, Henghui Bao, Heni Ben Amor, Henrik I. Christensen, Hiroki Furuta, Homanga Bharadhwaj, Homer
Walke, Hongjie Fang, Huy Ha, Igor Mordatch, Ilija Radosavovic, Isabel Leal, Jacky Liang, Jad Abou-Chakra, Jaehyung Kim,
Jaimyn Drake, Jan Peters, Jan Schneider, Jasmine Hsu, Jay Vakil, Jeannette Bohg, Jeffrey Bingham, Jeffrey Wu, Jensen
Gao, Jiaheng Hu, Jiajun Wu, Jialin Wu, Jiankai Sun, Jianlan Luo, Jiayuan Gu, Jie Tan, Jihoon Oh, Jimmy Wu, Jingpei
Lu, Jingyun Yang, Jitendra Malik, João Silvério, Joey Hejna, Jonathan Booher, Jonathan Tompson, Jonathan Yang, Jordi
Salvador, Joseph J. Lim, Junhyek Han, Kaiyuan Wang, Kanishka Rao, Karl Pertsch, Karol Hausman, Keegan Go, Keerthana
Gopalakrishnan, Ken Goldberg, Kendra Byrne, Kenneth Oslund, Kento Kawaharazuka, Kevin Black, Kevin Lin, Kevin
Zhang, Kiana Ehsani, Kiran Lekkala, Kirsty Ellis, Krishan Rana, Krishnan Srinivasan, Kuan Fang, Kunal Pratap Singh,
Kuo-Hao Zeng, Kyle Hatch, Kyle Hsu, Laurent Itti, Lawrence Yunliang Chen, Lerrel Pinto, Li Fei-Fei, Liam Tan, Linxi "Jim"
Fan, Lionel Ott, Lisa Lee, Luca Weihs, Magnum Chen, Marion Lepert, Marius Memmel, Masayoshi Tomizuka, Masha
Itkina, Mateo Guaman Castro, Max Spero, Maximilian Du, Michael Ahn, Michael C. Yip, Mingtong Zhang, Mingyu Ding,
Minho Heo, Mohan Kumar Srirama, Mohit Sharma, Moo Jin Kim, Muhammad Zubair Irshad, Naoaki Kanazawa, Nicklas
Hansen, Nicolas Heess, Nikhil J. Joshi, Niko Suenderhauf, Ning Liu, Norman Di Palo, Nur Muhammad Mahi Shafiullah,
Oier Mees, Oliver Kroemer, Osbert Bastani, Pannag R. Sanketi, Patrick "Tree" Miller, Patrick Yin, Paul Wohlhart, Peng
Xu, Peter David Fagan, Peter Mitrano, Pierre Sermanet, Pieter Abbeel, Priya Sundaresan, Qiuyu Chen, Quan Vuong, Rafael
Rafailov, Ran Tian, Ria Doshi, Roberto Martín-Martín, Rohan Baijal, Rosario Scalise, Rose Hendrix, Roy Lin, Runjia Qian,
Ruohan Zhang, Russell Mendonca, Rutav Shah, Ryan Hoque, Ryan Julian, Samuel Bustamante, Sean Kirmani, Sergey
Levine, Shan Lin, Sherry Moore, Shikhar Bahl, Shivin Dass, Shubham Sonawani, Shubham Tulsiani, Shuran Song, Sichun
Xu, Siddhant Haldar, Siddharth Karamcheti, Simeon Adebola, Simon Guist, Soroush Nasiriany, Stefan Schaal, Stefan Welker,
Stephen Tian, Subramanian Ramamoorthy, Sudeep Dasari, Suneel Belkhale, Sungjae Park, Suraj Nair, Suvir Mirchandani,
Takayuki Osa, Tanmay Gupta, Tatsuya Harada, Tatsuya Matsushima, Ted Xiao, Thomas Kollar, Tianhe Yu, Tianli Ding,
Todor Davchev, Tony Z. Zhao, Travis Armstrong, Trevor Darrell, Trinity Chung, Vidhi Jain, Vikash Kumar, Vincent
Vanhoucke, Vitor Guizilini, Wei Zhan, Wenxuan Zhou, Wolfram Burgard, Xi Chen, Xiangyu Chen, Xiaolong Wang, Xinghao
Zhu, Xinyang Geng, Xiyuan Liu, Xu Liangwei, Xuanlin Li, Yansong Pang, Yao Lu, Yecheng Jason Ma, Yejin Kim, Yevgen
Chebotar, Yifan Zhou, Yifeng Zhu, Yilin Wu, Ying Xu, Yixuan Wang, Yonatan Bisk, Yongqiang Dou, Yoonyoung Cho,
Youngwoon Lee, Yuchen Cui, Yue Cao, Yueh-Hua Wu, Yujin Tang, Yuke Zhu, Yunchu Zhang, Yunfan Jiang, Yunshuang Li,
Yunzhu Li, Yusuke Iwasawa, Yutaka Matsuo, Zehan Ma, Zhuo Xu, Zichen Jeff Cui, Zichen Zhang, Zipeng Fu, and Zipeng
Lin. Open X-Embodiment: Robotic Learning Datasets and RT-X Models, May 2025.

Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel
Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao
Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal,
Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning Robust Visual Features without Supervision,
February 2024.

Frank Permenter and Chenyang Yuan. Interpreting and Improving Diffusion Models from an Optimization Perspective, June
2024.

Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao
Ma, Ching-Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra,
Ji Hou, Jialiang Wang, Kiran Jagadeesh, Kunpeng Li, Luxin Zhang, Mannat Singh, Mary Williamson, Matt Le, Matthew Yu,
Mitesh Kumar Singh, Peizhao Zhang, Peter Vajda, Quentin Duval, Rohit Girdhar, Roshan Sumbaly, Sai Saketh Rambhatla,
Sam Tsai, Samaneh Azadi, Samyak Datta, Sanyuan Chen, Sean Bell, Sharadh Ramaswamy, Shelly Sheynin, Siddharth
Bhattacharya, Simran Motwani, Tao Xu, Tianhe Li, Tingbo Hou, Wei-Ning Hsu, Xi Yin, Xiaoliang Dai, Yaniv Taigman,
Yaqiao Luo, Yen-Cheng Liu, Yi-Chiao Wu, Yue Zhao, Yuval Kirstain, Zecheng He, Zijian He, Albert Pumarola, Ali Thabet,
Artsiom Sanakoyeu, Arun Mallya, Baishan Guo, Boris Araya, Breena Kerr, Carleigh Wood, Ce Liu, Cen Peng, Dimitry
Vengertsev, Edgar Schonfeld, Elliot Blanchard, Felix Juefei-Xu, Fraylie Nord, Jeff Liang, John Hoffman, Jonas Kohler, Kaolin
Fire, Karthik Sivakumar, Lawrence Chen, Licheng Yu, Luya Gao, Markos Georgopoulos, Rashel Moritz, Sara K. Sampson,
Shikai Li, Simone Parmeggiani, Steve Fine, Tara Fowler, Vladan Petrovic, and Yuming Du. Movie Gen: A Cast of Media
Foundation Models, February 2025.

Dean A. Pomerleau. ALVINN: An Autonomous Land Vehicle in a Neural Network. In Advances in Neural Information
Processing Systems, volume 1. Morgan-Kaufmann, 1988.

73
Simon J.D. Prince. Understanding Deep Learning. The MIT Press, 2023.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell,
Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural
Language Supervision, February 2021.

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J.
Liu. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer, September 2023.

Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai
Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas
Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, and Nando de Freitas. A Generalist Agent, November
2022.

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation,
May 2015.

Stephane Ross, Geoffrey J. Gordon, and J. Andrew Bagnell. A Reduction of Imitation Learning and Structured Prediction to
No-Regret Online Learning, March 2011.

Lindsay Sanneman, Christopher Fourie, and Julie A. Shah. The State of Industrial Robotics: Emerging Technologies, Challenges,
and Key Research Directions, October 2020.

C Schuhmann, A Köpf, R Vencu, T Coombes, and R Beaumont. Laion coco: 600m synthetic captions from laion2b-en. URL
[Link] 2022.

John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, and Pieter Abbeel. Trust Region Policy Optimization, April
2017a.

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms,
August 2017b.

Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University
Press, 1 edition, May 2014. ISBN 978-1-107-05713-5 978-1-107-29801-9. doi: 10.1017/CBO9781107298019.

Mustafa Shukor, Corentin Dancette, and Matthieu Cord. Ep-alm: Efficient perceptual augmentation of language models. In
Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22056–22069, 2023.

Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi,
Caroline Pascal, Martino Russi, Andres Marafioti, Simon Alibert, Matthieu Cord, Thomas Wolf, and Remi Cadene. SmolVLA:
A Vision-Language-Action Model for Affordable and Efficient Robotics, June 2025.

Bruno Siciliano and Oussama Khatib, editors. Springer Handbook of Robotics. Springer Handbooks. Springer International
Publishing, Cham, 2016. ISBN 978-3-319-32550-7 978-3-319-32552-1. doi: 10.1007/978-3-319-32552-1.

David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Riedmiller. Deterministic policy gradient
algorithms. In Eric P. Xing and Tony Jebara, editors, Proceedings of the 31st International Conference on Machine Learning,
volume 32 of Proceedings of Machine Learning Research, pages 387–395, Bejing, China, June 2014. PMLR.

Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning Structured Output Representation using Deep Conditional Generative
Models. In Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015.

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models, October 2022.

Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. Adaptive Computation and Machine
Learning Series. The MIT Press, Cambridge, Massachusetts, second edition edition, 2018. ISBN 978-0-262-03924-6.

Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ra-
mamoorthi, Jonathan T. Barron, and Ren Ng. Fourier Features Let Networks Learn High Frequency Functions in Low
Dimensional Domains, June 2020.

Chen Tang, Ben Abbatematteo, Jiaheng Hu, Rohan Chandra, Roberto Martín-Martín, and Peter Stone. Deep Reinforcement
Learning for Robotics: A Survey of Real-World Successes. Annual Review of Control, Robotics, and Autonomous Systems, 8
(Volume 8, 2025):153–188, May 2025. ISSN 2573-5144. doi: 10.1146/annurev-control-030323-022510.

Yang Tang, Chaoqiang Zhao, Jianrui Wang, Chongzhen Zhang, Qiyu Sun, Weixing Zheng, Wenli Du, Feng Qian, and Juergen
Kurths. Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey. IEEE Transactions on Neural
Networks and Learning Systems, 34(12):9604–9624, December 2023. ISSN 2162-237X, 2162-2388. doi: 10.1109/TNNLS.2022.
3167688.

74
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot,
Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon,
Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stanczyk, Sertan
Girgin, Nikola Momchev, Matt Hoffman, Shantanu Thakoor, Jean-Bastien Grill, Behnam Neyshabur, Olivier Bachem,
Alanna Walton, Aliaksei Severyn, Alicia Parrish, Aliya Ahmad, Allen Hutchison, Alvin Abdagic, Amanda Carl, Amy Shen,
Andy Brock, Andy Coenen, Anthony Laforge, Antonia Paterson, Ben Bastian, Bilal Piot, Bo Wu, Brandon Royal, Charlie
Chen, Chintu Kumar, Chris Perry, Chris Welty, Christopher A. Choquette-Choo, Danila Sinopalnikov, David Weinberger,
Dimple Vijaykumar, Dominika Rogozińska, Dustin Herbison, Elisa Bandy, Emma Wang, Eric Noland, Erica Moreira, Evan
Senter, Evgenii Eltyshev, Francesco Visin, Gabriel Rasskin, Gary Wei, Glenn Cameron, Gus Martins, Hadi Hashemi, Hanna
Klimczak-Plucińska, Harleen Batra, Harsh Dhand, Ivan Nardini, Jacinda Mein, Jack Zhou, James Svensson, Jeff Stanway,
Jetha Chan, Jin Peng Zhou, Joana Carrasqueira, Joana Iljazi, Jocelyn Becker, Joe Fernandez, Joost van Amersfoort, Josh
Gordon, Josh Lipschultz, Josh Newlan, Ju-yeong Ji, Kareem Mohamed, Kartikeya Badola, Kat Black, Katie Millican, Keelin
McDonell, Kelvin Nguyen, Kiranbir Sodhia, Kish Greene, Lars Lowe Sjoesund, Lauren Usui, Laurent Sifre, Lena Heuermann,
Leticia Lago, Lilly McNealus, Livio Baldini Soares, Logan Kilpatrick, Lucas Dixon, Luciano Martins, Machel Reid, Manvinder
Singh, Mark Iverson, Martin Görner, Mat Velloso, Mateo Wirth, Matt Davidow, Matt Miller, Matthew Rahtz, Matthew
Watson, Meg Risdal, Mehran Kazemi, Michael Moynihan, Ming Zhang, Minsuk Kahng, Minwoo Park, Mofi Rahman, Mohit
Khatwani, Natalie Dao, Nenshad Bardoliwalla, Nesh Devanathan, Neta Dumai, Nilay Chauhan, Oscar Wahltinez, Pankil
Botarda, Parker Barnes, Paul Barham, Paul Michel, Pengchong Jin, Petko Georgiev, Phil Culliton, Pradeep Kuppala,
Ramona Comanescu, Ramona Merhej, Reena Jana, Reza Ardeshir Rokni, Rishabh Agarwal, Ryan Mullins, Samaneh Saadat,
Sara Mc Carthy, Sarah Perrin, Sébastien M. R. Arnold, Sebastian Krause, Shengyang Dai, Shruti Garg, Shruti Sheth, Sue
Ronstrom, Susan Chan, Timothy Jordan, Ting Yu, Tom Eccles, Tom Hennigan, Tomas Kocisky, Tulsee Doshi, Vihan Jain,
Vikas Yadav, Vilobh Meshram, Vishal Dharmadhikari, Warren Barkley, Wei Wei, Wenming Ye, Woohyun Han, Woosuk
Kwon, Xiang Xu, Zhe Shen, Zhitao Gong, Zichuan Wei, Victor Cotruta, Phoebe Kirk, Anand Rao, Minh Giang, Ludovic
Peran, Tris Warkentin, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, D. Sculley, Jeanine Banks, Anca Dragan,
Slav Petrov, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Sebastian
Borgeaud, Noah Fiedel, Armand Joulin, Kathleen Kenealy, Robert Dadashi, and Alek Andreev. Gemma 2: Improving Open
Language Models at a Practical Size, August 2024.

Russ Tedrake. Robotic Manipulation. Perception, Planning and Control., a.

Russ Tedrake. Underactuated Robotics. Algorithms for Walking, Running, Swimming, Flying, and Manipulation, b.

Gabriele Tiboni, Karol Arndt, and Ville Kyrki. DROPO: Sim-to-Real Transfer with Offline Domain Randomization, January
2023.

Gabriele Tiboni, Pascal Klink, Jan Peters, Tatiana Tommasi, Carlo D’Eramo, and Georgia Chalvatzaki. Domain Randomization
via Entropy Maximization, March 2024.

Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain Randomization for
Transferring Deep Neural Networks from Simulation to the Real World, March 2017.

Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang,
Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms.
Advances in Neural Information Processing Systems, 37:87310–87356, 2024.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra,
Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David
Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony
Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem
Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao,
Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi
Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh
Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan,
Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2:
Open Foundation and Fine-Tuned Chat Models, July 2023.

Maria Tsimpoukelli, Jacob L Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. Multimodal few-shot learning
with frozen language models. Advances in Neural Information Processing Systems, 34:200–212, 2021.

Théophane Vallaeys, Mustafa Shukor, Matthieu Cord, and Jakob Verbeek. Improved baselines for data-efficient perceptual
augmentation of llms. arXiv preprint arXiv:2403.13499, 2024.

Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei
Gao, et al. InternVideo2. 5: Empowering video mllms with long and rich context modeling. arXiv preprint arXiv:2501.12386,
2025.

75
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He,
Qianyu Chen, Huarong Zhou, Zhensheng Zou, Haoye Zhang, Shengding Hu, Zhi Zheng, Jie Zhou, Jie Cai, Xu Han, Guoyang
Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. MiniCPM-v: A GPT-4V level MLLM on your phone, 2024.

Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid Loss for Language Image Pre-Training,
September 2023.

Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang
Zhang, Xin Li, et al. VideoLLaMA 3: Frontier multimodal foundation models for image and video understanding. arXiv
preprint arXiv:2501.13106, 2025.

Chong Zhang, Wenli Xiao, Tairan He, and Guanya Shi. WoCoCo: Learning Whole-Body Humanoid Control with Sequential
Contacts, November 2024.

Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning Fine-Grained Bimanual Manipulation with Low-Cost
Hardware, April 2023.

Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. MiniGPT-4: Enhancing vision-language understanding
with advanced large language models. In The Twelfth International Conference on Learning Representations, 2024.

Wanrong Zhu, Jack Hessel, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Alex Fang, Youngjae Yu, Ludwig Schmidt,
William Yang Wang, and Yejin Choi. Multimodal C4: An open, billion-scale corpus of images interleaved with text. In
Thirty-Seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.

Common questions