Frame count is now derived from the upstream .npy length

docs/source/_toctree.yml

@@ -3,8 +3,6 @@
     title: LeRobot
   - local: installation
     title: Installation
-  - local: cheat-sheet
-    title: Cheat sheet
   title: Get started
 - sections:
   - local: il_robots
@@ -39,12 +37,8 @@
     title: Porting Large Datasets
   - local: using_dataset_tools
     title: Using the Dataset Tools
-  - local: language_and_recipes
-    title: Language Columns and Recipes
-  - local: tools
-    title: Tools
-  - local: video_encoding_parameters
-    title: Video encoding parameters
+  - local: dataset_subtask
+    title: Using Subtasks in the Dataset
   - local: streaming_video_encoding
     title: Streaming Video Encoding
   title: "Datasets"
@@ -73,8 +67,6 @@
 - sections:
   - local: sarm
     title: SARM
-  - local: topreward
-    title: TOPReward
   title: "Reward Models"
 - sections:
   - local: inference
@@ -147,8 +139,6 @@
     title: OMX
   - local: openarm
     title: OpenArm
-  - local: rebot_b601
-    title: reBot B601-DM
   title: "Robots"
 - sections:
   - local: phone_teleop

docs/source/act.mdx

@@ -90,6 +90,6 @@ lerobot-record \
   --dataset.single_task="Your task description" \
   --dataset.streaming_encoding=true \
   --dataset.encoder_threads=2 \
-  # --dataset.camera_encoder.vcodec=auto \
+  # --dataset.vcodec=auto \
   --policy.path=${HF_USER}/act_policy
 ```

docs/source/cheat-sheet.mdx

@@ -1,139 +0,0 @@
-# Cheat sheet
-
-All of the LeRobot commands in one place. If you forgot how to use a specific command or want to learn about a new one you can do it here.
-
-> [!WARNING]
-> For all of the commands listed below remember to change the ports/names/ids to your own values!
-
-> [!TIP]
-> Another great way to look at all the commands and get them configured for your specific setup is to use this [Jupyter Notebook](https://github.com/huggingface/lerobot/blob/main/examples/notebooks/quickstart.ipynb).
-
-### Setup and installation
-
-For installation please look at [LeRobot Installation](https://huggingface.co/docs/lerobot/main/en/installation).
-
-### Useful tools
-
-###### Find port
-
-Use this to identify which serial ports your robots are connected to. Follow the instructions in your terminal: you will be asked to unplug the USB cable and press Enter. The script will then detect and print the correct serial port for that robot.
-
-```bash
-lerobot-find-port
-```
-
-###### Find cameras
-
-Quickly find camera indices and verify their output. This command prints camera information to the terminal and saves test frames from each detected camera to `lerobot/outputs/captured_images`
-
-```bash
-lerobot-find-cameras
-```
-
-### Calibration
-
-In most cases you will need to perform calibration just once for each robot and teleoperation device. Before performing the calibration make sure that all the joints are roughly in the middle position.
-
-```bash
-lerobot-calibrate \
-    --robot.type=so101_follower \
-    --robot.port=/dev/ttyACM0 \
-    --robot.id=my_follower_arm
-```
-
-Make sure that you use the same IDs used during calibration later for the other scripts. That's how LeRobot finds the calibration files.
-
-### Teleoperation
-
-Teleoperating with two cameras and displaying the data with Rerun.
-
-```bash
-lerobot-teleoperate \
-    --robot.type=so101_follower \
-    --robot.port=/dev/ttyACM0 \
-    --robot.id=my_follower_arm \
-    --robot.cameras="{ top: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30} }" \
-    --teleop.type=so101_leader \
-    --teleop.port=/dev/ttyACM1 \
-    --teleop.id=my_leader_arm \
-    --display_data=true
-```
-
-### Recording a dataset
-
-The dataset is automatically uploaded to the server and saved under repo_id, make sure you are logged in to your HF account with CLI:
-`hf auth login`
-
-You can get the token from: [https://huggingface.co/settings/tokens](https://huggingface.co/settings/tokens)
-
-```bash
-lerobot-record \
-    --robot.type=so101_follower \
-    --robot.port=/dev/ttyACM0 \
-    --robot.id=my_follower_arm \
-    --robot.cameras="{ top: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30} }" \
-    --teleop.type=so101_leader \
-    --teleop.port=/dev/ttyACM1 \
-    --teleop.id=my_leader_arm \
-    --dataset.repo_id=${HF_USER}/so101_dataset_test \
-    --dataset.num_episodes=30 \
-    --dataset.single_task="put the red brick in a bowl" \
-    --dataset.streaming_encoding=true \
-    --display_data=true
-```
-
-While collecting the dataset you can control the process with your keyboard:
-Control the data recording flow using keyboard shortcuts:
-
-- Press **Right Arrow (`→`)**: Save episode and move to the next.
-- Press **Left Arrow (`←`)**: Delete current episode and retry.
-- Press **Escape (`ESC`)**: Stop, encode videos, and upload.
-
-### Training
-
-Depending on your hardware training the policy might take a few hours. That's how you train simple `ACT` policy:
-
-```bash
-lerobot-train \
-    --dataset.repo_id=${HF_USER}/so101_dataset_test \
-    --policy.type=act \
-    --output_dir=outputs/train/act_so101_test \
-    --job_name=act_so101_test \
-    --policy.device=cuda \
-    --wandb.enable=true \
-    --policy.repo_id=${HF_USER}/policy_test \
-    --steps=20000
-```
-
-- Policy Types: `act`, `diffusion`, `smolvla`, `pi05`
-- Devices: `cuda` (NVIDIA), `mps` (Apple Silicon), `cpu`
-
-If you want to fine-tune a specific model you can provide the path to the model. In this case path is enough and type can be skipped.
-
-```bash
-lerobot-train \
-    --dataset.repo_id=${HF_USER}/so101_dataset_test \
-    --policy.path=username/the_policy_to_finetune \
-    --policy.device=cuda \
-    --policy.repo_id=${HF_USER}/policy_test \
-    --output_dir=outputs/train/act_so101_test \
-    --steps=20000
-```
-
-### Inference
-
-Inference means running the trained policy/model on a robot. For that we use `lerobot-rollout`. You will need to provide a path to your policy. It can be a local path or a path to Hugging Face for example "lerobot/folding_latest". Your cameras configuration needs to match what was used when collecting the dataset. Duration is in seconds if unspecified, it will run forever.
-
-> [!TIP]
-> If you are using the previous release V0.5.1 instead of `lerobot-rollout` you need to use `lerobot-record`. More information [here](https://huggingface.co/docs/lerobot/v0.5.1/en/il_robots#run-inference-and-evaluate-your-policy).
-
-```bash
-lerobot-rollout \
-    --strategy.type=base \
-    --policy.path=${HF_USER}/my_policy \
-    --robot.type=so101_follower \
-    --robot.port=/dev/ttyACM1 \
-    --robot.cameras="{ up: {type: opencv, index_or_path: /dev/video1, width: 640, height: 480, fps: 30}, side: {type: opencv, index_or_path: /dev/video5, width: 640, height: 480, fps: 30}}" \
-    --task="Put lego brick into the transparent box" \
-    --duration=60
-```

docs/source/dataset_subtask.mdx

@@ -0,0 +1,277 @@
+# Using Subtasks in LeRobot Datasets
+
+Subtask support in robotics datasets has proven effective in improving robot reasoning and understanding. Subtasks are particularly useful for:
+
+- **Hierarchical policies**: Building policies that include subtask predictions to visualize robot reasoning in real time
+- **Reward modeling**: Helping reward models understand task progression (e.g., SARM-style stage-aware reward models)
+- **Task decomposition**: Breaking down complex manipulation tasks into atomic, interpretable steps
+
+LeRobotDataset now supports subtasks as part of its dataset structure, alongside tasks.
+
+## What are Subtasks?
+
+While a **task** describes the overall goal (e.g., "Pick up the apple and place it in the basket"), **subtasks** break down the execution into finer-grained steps:
+
+1. "Approach the apple"
+2. "Grasp the apple"
+3. "Lift the apple"
+4. "Move to basket"
+5. "Release the apple"
+
+Each frame in the dataset can be annotated with its corresponding subtask, enabling models to learn and predict these intermediate stages.
+
+<img
+  src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/lerobot/subtask-asset.png"
+  alt="An overview of subtask annotation showing how frames are labeled with intermediate subtask stages"
+  width="80%"
+/>
+
+<p>
+  <em>Figure: Overview of subtask annotation.</em>
+</p>
+
+**Reference:** _Subtask-learning based for robot self-assembly in flexible collaborative assembly in manufacturing_, Original Article, Published: 19 April 2022.
+
+## Dataset Structure
+
+Subtask information is stored in the dataset metadata:
+
+```
+my-dataset/
+├── data/
+│   └── ...
+├── meta/
+│   ├── info.json
+│   ├── stats.json
+│   ├── tasks.parquet
+│   ├── subtasks.parquet      # Subtask index → subtask string mapping
+│   └── episodes/
+│       └── ...
+└── videos/
+    └── ...
+```
+
+### Subtasks Parquet File
+
+The `meta/subtasks.parquet` file maps subtask indices to their natural language descriptions:
+
+| subtask_index | subtask (index column) |
+| ------------- | ---------------------- |
+| 0             | "Approach the apple"   |
+| 1             | "Grasp the apple"      |
+| 2             | "Lift the apple"       |
+| ...           | ...                    |
+
+### Frame-Level Annotations
+
+Each frame in the dataset can include a `subtask_index` field that references the subtasks parquet file:
+
+```python
+# Example frame data in the parquet file
+{
+    "index": 42,
+    "timestamp": 1.4,
+    "episode_index": 0,
+    "task_index": 0,
+    "subtask_index": 2,  # References "Lift the apple"
+    "observation.state": [...],
+    "action": [...],
+}
+```
+
+## Annotating Datasets with Subtasks
+
+We provide a HuggingFace Space for easily annotating any LeRobotDataset with subtasks:
+
+**[https://huggingface.co/spaces/lerobot/annotate](https://huggingface.co/spaces/lerobot/annotate)**
+
+After completing your annotation:
+
+1. Click "Push to Hub" to upload your annotated dataset
+2. You can also run the annotation space locally by following the instructions at [github.com/huggingface/lerobot-annotate](https://github.com/huggingface/lerobot-annotate)
+
+## Loading Datasets with Subtasks
+
+When you load a dataset with subtask annotations, the subtask information is automatically available:
+
+```python
+from lerobot.datasets import LeRobotDataset
+
+# Load a dataset with subtask annotations
+dataset = LeRobotDataset("jadechoghari/collect-fruit-annotated")
+
+# Access a sample
+sample = dataset[100]
+
+# The sample includes both task and subtask information
+print(sample["task"])        # "Collect the fruit"
+print(sample["subtask"])     # "Grasp the apple"
+print(sample["task_index"])  # tensor(0)
+print(sample["subtask_index"])  # tensor(2)
+```
+
+### Checking for Subtask Support
+
+You can check if a dataset has subtask annotations:
+
+```python
+# Check if subtasks are available
+has_subtasks = (
+    "subtask_index" in dataset.features
+    and dataset.meta.subtasks is not None
+)
+
+if has_subtasks:
+    print(f"Dataset has {len(dataset.meta.subtasks)} unique subtasks")
+    print("Subtasks:", list(dataset.meta.subtasks.index))
+```
+
+## Using Subtasks for Training
+
+### With the Tokenizer Processor
+
+The `TokenizerProcessor` automatically handles subtask tokenization for Vision-Language Action (VLA) models:
+
+```python
+from lerobot.processor import TokenizerProcessorStep
+
+# Create a tokenizer processor step
+tokenizer_processor = TokenizerProcessorStep(
+    tokenizer_name_or_path="google/paligemma-3b-pt-224",
+    padding="max_length",
+    max_length=64,
+)
+
+# The processor will automatically tokenize subtasks if present in the batch
+# and add them to the observation under:
+# - "observation.subtask.tokens"
+# - "observation.subtask.attention_mask"
+```
+
+When subtasks are available in the batch, the tokenizer processor adds:
+
+- `observation.subtask.tokens`: Tokenized subtask text
+- `observation.subtask.attention_mask`: Attention mask for the subtask tokens
+
+### DataLoader with Subtasks
+
+```python
+import torch
+from lerobot.datasets import LeRobotDataset
+
+dataset = LeRobotDataset("jadechoghari/collect-fruit-annotated")
+
+dataloader = torch.utils.data.DataLoader(
+    dataset,
+    batch_size=16,
+    shuffle=True,
+)
+
+for batch in dataloader:
+    # Access subtask information in the batch
+    subtasks = batch["subtask"]  # List of subtask strings
+    subtask_indices = batch["subtask_index"]  # Tensor of subtask indices
+
+    # Use for training hierarchical policies or reward models
+    print(f"Batch subtasks: {set(subtasks)}")
+```
+
+## Example Datasets with Subtask Annotations
+
+Try loading a dataset with subtask annotations:
+
+```python
+from lerobot.datasets import LeRobotDataset
+
+# Example dataset with subtask annotations
+dataset = LeRobotDataset("jadechoghari/collect-fruit-annotated")
+
+# Explore the subtasks
+print("Available subtasks:")
+for subtask_name in dataset.meta.subtasks.index:
+    print(f"  - {subtask_name}")
+
+# Get subtask distribution
+subtask_counts = {}
+for i in range(len(dataset)):
+    sample = dataset[i]
+    subtask = sample["subtask"]
+    subtask_counts[subtask] = subtask_counts.get(subtask, 0) + 1
+
+print("\nSubtask distribution:")
+for subtask, count in sorted(subtask_counts.items(), key=lambda x: -x[1]):
+    print(f"  {subtask}: {count} frames")
+```
+
+## Use Cases
+
+### 1. Hierarchical Policy Training
+
+Train policies that predict both actions and current subtask:
+
+```python
+class HierarchicalPolicy(nn.Module):
+    def __init__(self, num_subtasks):
+        super().__init__()
+        self.action_head = nn.Linear(hidden_dim, action_dim)
+        self.subtask_head = nn.Linear(hidden_dim, num_subtasks)
+
+    def forward(self, observations):
+        features = self.encoder(observations)
+        actions = self.action_head(features)
+        subtask_logits = self.subtask_head(features)
+        return actions, subtask_logits
+```
+
+### 2. Stage-Aware Reward Modeling (SARM)
+
+Build reward models that understand task progression:
+
+```python
+# SARM predicts:
+# - Stage: Which subtask is being executed (discrete)
+# - Progress: How far along the subtask (continuous 0-1)
+
+class SARMRewardModel(nn.Module):
+    def forward(self, observations):
+        features = self.encoder(observations)
+        stage_logits = self.stage_classifier(features)
+        progress = self.progress_regressor(features)
+        return stage_logits, progress
+```
+
+### 3. Progress Visualization
+
+Monitor robot execution by tracking subtask progression:
+
+```python
+def visualize_execution(model, observations):
+    for t, obs in enumerate(observations):
+        action, subtask_logits = model(obs)
+        predicted_subtask = subtask_names[subtask_logits.argmax()]
+        print(f"t={t}: Executing '{predicted_subtask}'")
+```
+
+## API Reference
+
+### LeRobotDataset Properties
+
+| Property                    | Type                   | Description                                |
+| --------------------------- | ---------------------- | ------------------------------------------ |
+| `meta.subtasks`             | `pd.DataFrame \| None` | DataFrame mapping subtask names to indices |
+| `features["subtask_index"]` | `dict`                 | Feature spec for subtask_index if present  |
+
+### Sample Keys
+
+When subtasks are available, each sample includes:
+
+| Key             | Type           | Description                          |
+| --------------- | -------------- | ------------------------------------ |
+| `subtask_index` | `torch.Tensor` | Integer index of the current subtask |
+| `subtask`       | `str`          | Natural language subtask description |
+
+## Related Resources
+
+- [SARM Paper](https://arxiv.org/pdf/2509.25358) - Stage-Aware Reward Modeling for Long Horizon Robot Manipulation
+- [LeRobot Annotate Space](https://huggingface.co/spaces/lerobot/annotate) - Interactive annotation tool
+- [LeRobotDataset v3.0](./lerobot-dataset-v3) - Dataset format documentation

docs/source/earthrover_mini_plus.mdx

@@ -194,7 +194,7 @@ lerobot-record \
     --dataset.single_task="Navigate around obstacles" \
     --dataset.streaming_encoding=true \
     --dataset.encoder_threads=2 \
-    # --dataset.camera_encoder.vcodec=auto \
+    # --dataset.vcodec=auto \
     --display_data=true
 ```
 

docs/source/groot.mdx

@@ -123,7 +123,7 @@ lerobot-record \
   --dataset.single_task="Grab and handover the red cube to the other arm" \
   --dataset.streaming_encoding=true \
   --dataset.encoder_threads=2 \
-  # --dataset.camera_encoder.vcodec=auto \
+  # --dataset.vcodec=auto \
   --policy.path=<user>/groot-bimanual \ # your trained model
   --dataset.episode_time_s=30 \
   --dataset.reset_time_s=10

docs/source/hope_jr.mdx

@@ -232,7 +232,7 @@ lerobot-record \
     --dataset.private=true \
     --dataset.streaming_encoding=true \
     --dataset.encoder_threads=2 \
-    # --dataset.camera_encoder.vcodec=auto \
+    # --dataset.vcodec=auto \
     --display_data=true
 ```
 
@@ -278,6 +278,6 @@ lerobot-record \
   --dataset.num_episodes=10 \
   --dataset.streaming_encoding=true \
   --dataset.encoder_threads=2 \
-  # --dataset.camera_encoder.vcodec=auto \
+  # --dataset.vcodec=auto \
   --policy.path=outputs/train/hopejr_hand/checkpoints/last/pretrained_model
 ```

docs/source/il_robots.mdx

@@ -193,7 +193,7 @@ lerobot-record \
     --dataset.num_episodes=5 \
     --dataset.single_task="Grab the black cube" \
     --dataset.streaming_encoding=true \
-    # --dataset.camera_encoder.vcodec=auto \
+    # --dataset.vcodec=auto \
     --dataset.encoder_threads=2
 ```
 </hfoption>

docs/source/language_and_recipes.mdx

@@ -1,147 +0,0 @@
-# Language columns and recipes
-
-Most LeRobot datasets ship with a single `task` string per episode — fine for
-short, single-instruction skills, but not enough for the longer-horizon,
-multi-modal robot policies the field is moving toward (high-level planning,
-memory, interjections, VQA, tool use). To support those policies without
-forking the dataset format, LeRobot extends `LeRobotDataset` with two optional
-language columns and a small recipe layer that turns those rows into
-chat-style training samples on the fly.
-
-The design splits cleanly into three layers:
-
-1. **Data in the dataset** — language annotations stored next to frames in
-   `data/chunk-*/file-*.parquet` as two optional columns (`language_persistent`
-   and `language_events`). Datasets without these columns keep their existing
-   behavior.
-2. **Recipe** — a YAML file that declares which annotation rows to bind and
-   how to lay them out as chat turns (`role`, `content`, optional images,
-   optional tool calls). Recipes are pure config; no Python required to add a
-   new one.
-3. **Training format** — at sample time, `RenderMessagesStep` resolves the
-   recipe against the per-frame annotations and emits HF-style `messages` plus
-   LeRobot-specific sidecars (`message_streams`, `target_message_indices`)
-   that policy processors consume.
-
-This page describes each layer in turn.
-
-## Layer 1 — language columns in the dataset
-
-The two optional columns live next to frame data in
-`data/chunk-*/file-*.parquet`:
-
-- `language_persistent`: a list of rows broadcast across every frame in an episode for state that remains active, such as `subtask`, `plan`, and `memory`.
-- `language_events`: a list of rows only on the exact frame where an event was emitted, such as `interjection`, `vqa`, and speech tool calls.
-
-Both columns share the same row shape (event rows omit `timestamp` because the
-frame the row sits on already provides it):
-
-```text
-role: string
-content: string | null
-style: string | null
-timestamp: float32        # persistent rows only
-camera: string | null     # observation.images.* feature key, view-dependent rows only
-tool_calls: list[Json] | null
-```
-
-The `camera` field tags rows whose `content` is grounded in a specific camera
-view. Rows of view-dependent styles (`vqa` and `trace`) MUST set `camera` to
-the matching `observation.images.*` feature key. Rows of every other style —
-including `motion`, which describes robot-frame primitives in joint / Cartesian
-terms — MUST leave `camera` as `null`. Pipeline writers and the validator
-enforce this via `validate_camera_field(style, camera)`.
-
-`meta/tasks.parquet` remains the canonical source for the task. The special `${task}` recipe binding always reads that task string and does not depend on language annotations.
-
-### Architecture
-
-The language stack itself has three internal modules backing layer 1:
-
-1. `lerobot.datasets.language` defines the schema, style registry, and `column_for_style`.
-2. `lerobot.datasets.language_render` resolves rows and renders messages.
-3. `RenderMessagesStep` turns dataset samples into `messages`, `message_streams`, and `target_message_indices`.
-
-`LeRobotDataset` stays recipe-agnostic. It passes `language_persistent` and `language_events` through when present, and unannotated datasets keep their existing behavior.
-
-## Layer 2 — recipe anatomy
-
-Recipes are YAML files backed by `TrainingRecipe` and `MessageTurn`. They
-declare which annotation rows to pull (via `bindings`) and how to compose them
-into chat turns (`messages`).
-
-```yaml
-messages:
-  - { role: user, content: "${task}", stream: high_level }
-  - { role: assistant, content: "${subtask}", stream: low_level, target: true }
-```
-
-A recipe can also branch into a weighted **blend** of sub-recipes. At sample
-time, exactly one branch is selected deterministically from the sample index,
-so different frames train different objectives (e.g. memory updates vs.
-low-level execution vs. VQA) without any Python wiring.
-
-### Temporal semantics
-
-Persistent styles are active after emission until replaced:
-
-- `active_at(t, style=subtask)`
-- `nth_prev(style=memory, offset=1)`
-- `nth_next(style=subtask, offset=1)`
-
-Event styles only exist on their exact timestamp:
-
-- `emitted_at(t, style=interjection)`
-- `emitted_at(t, style=vqa, role=user, camera=observation.images.top)`
-- `emitted_at(t, role=assistant, tool_name=say)`
-
-Exact event matching has no tolerance window, so writers must stamp event rows with frame timestamps from the parquet data.
-
-### View-dependent resolution
-
-For view-dependent styles (`vqa` and `trace`), the resolver gains a
-`camera=` filter parallel to `role=` and `tool_name=`. Datasets with multiple
-cameras typically emit one (`vqa`, `user`) + (`vqa`, `assistant`) pair per
-camera at the same timestamp; without `camera=`, those resolvers see two
-matches and raise an ambiguity error. Recipes consume each camera through its
-own binding plus a matching image block, e.g.
-
-```yaml
-ask_vqa_top:
-  bindings:
-    vqa_query: "emitted_at(t, style=vqa, role=user, camera=observation.images.top)"
-    vqa: "emitted_at(t, style=vqa, role=assistant, camera=observation.images.top)"
-  messages:
-    - role: user
-      stream: high_level
-      if_present: vqa_query
-      content:
-        - { type: image, feature: observation.images.top }
-        - { type: text, text: "${vqa_query}" }
-    - {
-        role: assistant,
-        content: "${vqa}",
-        stream: high_level,
-        target: true,
-        if_present: vqa,
-      }
-```
-
-Add one such sub-recipe per camera the dataset records.
-
-## Layer 3 — training format
-
-Rendered samples use HF-style chat messages plus LeRobot sidecars:
-
-```python
-sample["messages"]
-sample["message_streams"]
-sample["target_message_indices"]
-```
-
-The renderer does not apply a tokenizer chat template. Policy processors decide how to serialize the messages for their backbone, which keeps the same dataset usable across SmolVLA, Pi0.5, and any future VLM that expects OpenAI-style chat messages.
-
-## Graceful absence
-
-If both language columns are missing, `None`, or empty, `RenderMessagesStep` is a no-op.
-If an event-scoped branch is selected on a frame without the required event row, rendering returns `None`, allowing a loader to retry another sample.

docs/source/lerobot-dataset-v3.mdx

@@ -10,7 +10,6 @@ This docs will guide you to:
 - Stream datasets without downloading using `StreamingLeRobotDataset`
 - Apply image transforms for data augmentation during training
 - Migrate existing `v2.1` datasets to `v3.0`
-- Experiment with other `LeRobotDataset` formats and implementations like Lance
 
 ## What’s new in `v3`
 
@@ -44,7 +43,7 @@ lerobot-record \
   --dataset.num_episodes=5 \
   --dataset.single_task="Grab the black cube" \
   --dataset.streaming_encoding=true \
-  # --dataset.camera_encoder.vcodec=auto \
+  # --dataset.vcodec=auto \
   --dataset.encoder_threads=2
 ```
 
@@ -316,39 +315,3 @@ Dataset v3.0 uses incremental parquet writing with buffered metadata for efficie
 - Ensures the dataset is valid for loading
 
 Without calling `finalize()`, your parquet files will be incomplete and the dataset won't load properly.
-
-## Other formats and implementations
-
-### Lance
-
-Lance is a useful format for multimodal AI datasets, especially for large-scale training requiring high performance IO and random access.
-
-The `lerobot-lancedb` package implements `LeRobotLanceDataset` (for JPEG images) and `LeRobotLanceVideoDataset` (for mp4 videos).
-Those two storage layouts both subclass LeRobotDataset and can provide data loading speed ups.
-
-`LeRobotLanceDataset` is a drop-in replacement for `LeRobotDataset`:
-
-```python
-from lerobot.datasets import LeRobotDatasetMetadata
-from lerobot.policies.diffusion.configuration_diffusion import DiffusionConfig
-from lerobot_lancedb import LeRobotLanceDataset, LeRobotLanceVideoDataset
-
-cfg = DiffusionConfig(...)
-meta = LeRobotDatasetMetadata(root=local_dataset_path)  # or use repo_id=... to load metadata from the Hub
-delta_timestamps = {...}
-
-# Use LeRobotLanceDataset for image datasets
-dataset = LeRobotLanceDataset(
-    root=local_dataset_path,                            # or use repo_id=... to stream from the Hub
-    delta_timestamps=delta_timestamps,
-    return_uint8=True,
-)
-# Or use LeRobotLanceVideoDataset for video datasets:
-dataset = LeRobotLanceVideoDataset(
-    root=local_dataset_path,                            # or use repo_id=... to stream from the Hub
-    delta_timestamps=delta_timestamps,
-    return_uint8=True,
-)
-```
-
-Join the discussion on [Github](https://github.com/huggingface/lerobot/issues/3608) and explore the `lerobot-lancedb` documentation [here](https://lancedb.github.io/lerobot-lancedb/).

docs/source/reachy2.mdx

@@ -161,7 +161,7 @@ lerobot-record \
     --dataset.private=true \
     --dataset.streaming_encoding=true \
     --dataset.encoder_threads=2 \
-    # --dataset.camera_encoder.vcodec=auto \
+    # --dataset.vcodec=auto \
     --display_data=true
 ```
 
@@ -203,7 +203,7 @@ lerobot-record \
     --dataset.private=true \
     --dataset.streaming_encoding=true \
     --dataset.encoder_threads=2 \
-    # --dataset.camera_encoder.vcodec=auto \
+    # --dataset.vcodec=auto \
     --display_data=true
 ```
 

docs/source/rebot_b601.mdx

@@ -1,186 +0,0 @@
-# reBot B601-DM
-
-[reBot B601-DM](https://wiki.seeedstudio.com/rebot_arm_b601_dm_lerobot/) is an open-source, low-cost robot arm from Seeed Studio for embodied-AI and imitation learning. It comes as a **follower** arm (the `B601-DM`, a 6-DOF arm plus gripper driven by Damiao CAN motors) and a **leader** arm (the `StarArm102` / `reBot Arm 102`, driven by FashionStar UART smart servos) used to teleoperate it.
-
-This page covers **calibration** and **teleoperation** for both single-arm and bimanual (dual-arm) setups.
-
-<div style="display: flex; align-items: center; gap: 10px;">
-  <img
-    src="https://files.seeedstudio.com/wiki/robotics/projects/lerobot/b601dm_zeroposition.jpg"
-    alt="reBot B601-DM follower arm at its zero position"
-    width="48%"
-  />
-  <img
-    src="https://files.seeedstudio.com/wiki/robotics/projects/lerobot/102_zeroposition.jpg"
-    alt="reBot Arm 102 leader arm at its zero position"
-    width="48%"
-  />
-</div>
-
-_Left: the B601-DM follower at its zero position. Right: the reBot Arm 102 leader at its zero position. Images courtesy of [Seeed Studio](https://wiki.seeedstudio.com/rebot_arm_b601_dm_lerobot/)._
-
-## Install LeRobot 🤗
-
-Follow our [Installation Guide](./installation), then install the reBot support:
-
-```bash
-pip install -e ".[rebot]"
-```
-
-This pulls in `motorbridge` (CAN motor control for the B601-DM follower) and `motorbridge-smart-servo` (FashionStar UART servos for the reBot Arm 102 leader).
-
-## Registered device types
-
-| Type                     | Kind                                         |
-| ------------------------ | -------------------------------------------- |
-| `rebot_b601_follower`    | single-arm B601-DM follower robot            |
-| `bi_rebot_b601_follower` | bimanual (dual-arm) follower robot           |
-| `rebot_102_leader`       | single-arm reBot Arm 102 leader teleoperator |
-| `bi_rebot_102_leader`    | bimanual (dual-arm) leader teleoperator      |
-
-The bimanual types compose two single-arm instances and namespace each arm's
-observation/action keys with a `left_` / `right_` prefix. Per-arm settings are
-passed through nested `left_arm_config.*` / `right_arm_config.*` arguments.
-
-## Find the USB ports
-
-For each device, find the USB port associated with its motor bus using:
-
-```bash
-lerobot-find-port
-```
-
-<Tip warning={true}>
-  On Linux, remove `brltty` (`sudo apt remove brltty`) so it does not hold the
-  leader's USB serial port. You may also need to grant access to the serial
-  devices: `sudo chmod 666 /dev/ttyACM* /dev/ttyUSB*`.
-</Tip>
-
-## Calibration
-
-Neither arm stores a persistent hardware calibration: every time it connects, the motors are re-zeroed against the pose the arm is physically holding. Calibration simply records that zero pose. When prompted, **manually move the arm to its zero position** (the default sit-down pose shown above, gripper fully closed) and press <kbd>ENTER</kbd>.
-
-### Follower (B601-DM)
-
-<hfoptions id="calibrate-follower">
-<hfoption id="Single arm">
-
-```bash
-lerobot-calibrate \
-    --robot.type=rebot_b601_follower \
-    --robot.port=/dev/ttyACM0 \
-    --robot.id=follower \
-    --robot.can_adapter=damiao
-```
-
-</hfoption>
-<hfoption id="Dual arm">
-
-Connect the bimanual follower; calibration runs for the left arm, then the right arm.
-
-```bash
-lerobot-calibrate \
-    --robot.type=bi_rebot_b601_follower \
-    --robot.id=bi_follower \
-    --robot.left_arm_config.port=/dev/ttyACM0 \
-    --robot.left_arm_config.can_adapter=damiao \
-    --robot.right_arm_config.port=/dev/ttyACM1 \
-    --robot.right_arm_config.can_adapter=damiao
-```
-
-Per-arm calibration files are saved with `_left` / `_right` suffixes on the id.
-
-</hfoption>
-</hfoptions>
-
-### Leader (reBot Arm 102)
-
-<hfoptions id="calibrate-leader">
-<hfoption id="Single arm">
-
-```bash
-lerobot-calibrate \
-    --teleop.type=rebot_102_leader \
-    --teleop.port=/dev/ttyUSB0 \
-    --teleop.id=leader
-```
-
-</hfoption>
-<hfoption id="Dual arm">
-
-```bash
-lerobot-calibrate \
-    --teleop.type=bi_rebot_102_leader \
-    --teleop.id=bi_leader \
-    --teleop.left_arm_config.port=/dev/ttyUSB0 \
-    --teleop.right_arm_config.port=/dev/ttyUSB1
-```
-
-</hfoption>
-</hfoptions>
-
-## Teleoperation
-
-Once both arms are calibrated, drive the follower with the leader. The follower talks to its CAN bus through a Damiao serial bridge (`can_adapter=damiao`, the default) or a SocketCAN adapter (`can_adapter=socketcan`). See the [OpenArm page](./openarm) for more details on the SocketCAN adapter configuration.
-
-<hfoptions id="teleoperate">
-<hfoption id="Single arm">
-
-```bash
-lerobot-teleoperate \
-    --robot.type=rebot_b601_follower \
-    --robot.port=/dev/ttyACM0 \
-    --robot.id=follower \
-    --robot.can_adapter=damiao \
-    --teleop.type=rebot_102_leader \
-    --teleop.port=/dev/ttyUSB0 \
-    --teleop.id=leader
-```
-
-</hfoption>
-<hfoption id="Dual arm">
-
-The bimanual leader and follower reuse the single-arm classes; each arm is
-configured through nested `left_arm_config.*` / `right_arm_config.*` arguments,
-so a bimanual reBot Arm 102 leader drives a bimanual B601-DM follower.
-
-```bash
-lerobot-teleoperate \
-    --robot.type=bi_rebot_b601_follower \
-    --robot.id=bi_follower \
-    --robot.left_arm_config.port=/dev/ttyACM0 \
-    --robot.left_arm_config.can_adapter=damiao \
-    --robot.right_arm_config.port=/dev/ttyACM1 \
-    --robot.right_arm_config.can_adapter=damiao \
-    --teleop.type=bi_rebot_102_leader \
-    --teleop.id=bi_leader \
-    --teleop.left_arm_config.port=/dev/ttyUSB0 \
-    --teleop.right_arm_config.port=/dev/ttyUSB1
-```
-
-</hfoption>
-</hfoptions>
-
-<Tip>
-  The leader and follower share the same joint names (`shoulder_pan,
-  shoulder_lift, elbow_flex, wrist_flex, wrist_yaw, wrist_roll, gripper`), so
-  leader actions map directly onto the follower.
-</Tip>
-
-If the motion of a joint is reversed, flip its sign in the leader's `joint_directions` (the gripper also carries a scale to widen its range to the follower):
-
-```bash
-lerobot-teleoperate \
-    --robot.type=rebot_b601_follower \
-    --robot.port=/dev/ttyACM0 \
-    --robot.can_adapter=damiao \
-    --teleop.type=rebot_102_leader \
-    --teleop.port=/dev/ttyUSB0 \
-    --teleop.joint_directions='{"shoulder_pan":-1,"shoulder_lift":-1,"elbow_flex":1,"wrist_flex":1,"wrist_yaw":1,"wrist_roll":-1,"gripper":-6}'
-```
-
-## Recording datasets
-
-Swap `lerobot-teleoperate` for `lerobot-record` (with the same `--robot.*` / `--teleop.*` arguments, plus `--dataset.*`) to record demonstrations for training. See [Imitation Learning for Robots](./il_robots) for the full workflow.
-
-For hardware assembly and wiring, see the [Seeed Studio reBot wiki](https://wiki.seeedstudio.com/rebot_arm_b601_dm_lerobot/).

docs/source/smolvla.mdx

@@ -108,7 +108,7 @@ lerobot-record \
   --dataset.num_episodes=10 \
   --dataset.streaming_encoding=true \
   --dataset.encoder_threads=2 \
-  # --dataset.camera_encoder.vcodec=auto \
+  # --dataset.vcodec=auto \
   # <- Teleop optional if you want to teleoperate in between episodes \
   # --teleop.type=so100_leader \
   # --teleop.port=/dev/ttyACM0 \

docs/source/streaming_video_encoding.mdx

@@ -17,9 +17,9 @@ This makes `save_episode()` near-instant (the video is already encoded by the ti
 | Parameter               | CLI Flag                          | Type          | Default       | Description                                                       |
 | ----------------------- | --------------------------------- | ------------- | ------------- | ----------------------------------------------------------------- |
 | `streaming_encoding`    | `--dataset.streaming_encoding`    | `bool`        | `True`        | Enable real-time encoding during capture                          |
-| `vcodec`                | `--dataset.camera_encoder.vcodec` | `str`         | `"libsvtav1"` | Video codec. `"auto"` detects best HW encoder                     |
+| `vcodec`                | `--dataset.vcodec`                | `str`         | `"libsvtav1"` | Video codec. `"auto"` detects best HW encoder                     |
 | `encoder_threads`       | `--dataset.encoder_threads`       | `int \| None` | `None` (auto) | Threads per encoder instance. `None` will leave the vcoded decide |
-| `encoder_queue_maxsize` | `--dataset.encoder_queue_maxsize` | `int`         | `30`          | Max buffered frames per camera (~1s at 30fps). Consumes RAM       |
+| `encoder_queue_maxsize` | `--dataset.encoder_queue_maxsize` | `int`         | `60`          | Max buffered frames per camera (~2s at 30fps). Consumes RAM       |
 
 ## 3. Performance Considerations
 
@@ -48,7 +48,7 @@ This parameter controls how many threads each encoder instance uses internally:
 
 ### Backpressure and Frame Dropping
 
-Each camera has a bounded queue (`encoder_queue_maxsize`, default 30 frames). When the encoder can't keep up:
+Each camera has a bounded queue (`encoder_queue_maxsize`, default 60 frames). When the encoder can't keep up:
 
 1. The queue fills up (consuming RAM)
 2. New frames are **dropped** (not blocked) — the capture loop continues uninterrupted
@@ -82,15 +82,15 @@ Use HW encoding when:
 
 ### Available HW Encoders
 
-| Encoder             | Platform      | Hardware                                                                                         | CLI Value                                           |
-| ------------------- | ------------- | ------------------------------------------------------------------------------------------------ | --------------------------------------------------- |
-| `h264_videotoolbox` | macOS         | Apple Silicon / Intel                                                                            | `--dataset.camera_encoder.vcodec=h264_videotoolbox` |
-| `hevc_videotoolbox` | macOS         | Apple Silicon / Intel                                                                            | `--dataset.camera_encoder.vcodec=hevc_videotoolbox` |
-| `h264_nvenc`        | Linux/Windows | NVIDIA GPU                                                                                       | `--dataset.camera_encoder.vcodec=h264_nvenc`        |
-| `hevc_nvenc`        | Linux/Windows | NVIDIA GPU                                                                                       | `--dataset.camera_encoder.vcodec=hevc_nvenc`        |
-| `h264_vaapi`        | Linux         | Intel/AMD GPU                                                                                    | `--dataset.camera_encoder.vcodec=h264_vaapi`        |
-| `h264_qsv`          | Linux/Windows | Intel Quick Sync                                                                                 | `--dataset.camera_encoder.vcodec=h264_qsv`          |
-| `auto`              | Any           | Probes the system for available HW encoders. Falls back to `libsvtav1` if no HW encoder is found | `--dataset.camera_encoder.vcodec=auto`              |
+| Encoder             | Platform      | Hardware                                                                                         | CLI Value                            |
+| ------------------- | ------------- | ------------------------------------------------------------------------------------------------ | ------------------------------------ |
+| `h264_videotoolbox` | macOS         | Apple Silicon / Intel                                                                            | `--dataset.vcodec=h264_videotoolbox` |
+| `hevc_videotoolbox` | macOS         | Apple Silicon / Intel                                                                            | `--dataset.vcodec=hevc_videotoolbox` |
+| `h264_nvenc`        | Linux/Windows | NVIDIA GPU                                                                                       | `--dataset.vcodec=h264_nvenc`        |
+| `hevc_nvenc`        | Linux/Windows | NVIDIA GPU                                                                                       | `--dataset.vcodec=hevc_nvenc`        |
+| `h264_vaapi`        | Linux         | Intel/AMD GPU                                                                                    | `--dataset.vcodec=h264_vaapi`        |
+| `h264_qsv`          | Linux/Windows | Intel Quick Sync                                                                                 | `--dataset.vcodec=h264_qsv`          |
+| `auto`              | Any           | Probes the system for available HW encoders. Falls back to `libsvtav1` if no HW encoder is found | `--dataset.vcodec=auto`              |
 
 > [!NOTE]
 > In order to use the HW accelerated encoders you might need to upgrade your GPU drivers.
@@ -100,15 +100,15 @@ Use HW encoding when:
 
 ## 5. Troubleshooting
 
-| Symptom                                                            | Likely Cause                                 | Fix                                                                                                                                                                                                                                                                                                 |
-| ------------------------------------------------------------------ | -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| System freezes or choppy robot movement or Rerun visualization lag | CPU starved (100% load usage)                | Close other apps, reduce encoding throughput, lower `encoder_threads`, use `h264`, use `display_data=False`. If the CPU continues to be at 100% then it might be insufficient for your setup, consider `--dataset.streaming_encoding=false` or HW encoding (`--dataset.camera_encoder.vcodec=auto`) |
-| "Encoder queue full" warnings or dropped frames in dataset         | Encoder can't keep up (Queue overflow)       | If CPU is not at 100%: Increase `encoder_threads`, increase `encoder_queue_maxsize` or use HW encoding (`--dataset.camera_encoder.vcodec=auto`).                                                                                                                                                    |
-| High RAM usage                                                     | Queue filling faster than encoding           | `encoder_threads` too low or CPU insufficient. Reduce `encoder_queue_maxsize` or use HW encoding                                                                                                                                                                                                    |
-| Large video files                                                  | Using HW encoder or H.264                    | Expected trade-off. Switch to `libsvtav1` if CPU allows                                                                                                                                                                                                                                             |
-| `save_episode()` still slow                                        | `streaming_encoding` is `False`              | Set `--dataset.streaming_encoding=true`                                                                                                                                                                                                                                                             |
-| Encoder thread crash                                               | Codec not available or invalid settings      | Check `vcodec` is installed, try `--dataset.camera_encoder.vcodec=auto`                                                                                                                                                                                                                             |
-| Recorded dataset is missing frames                                 | CPU/GPU starvation or occasional load spikes | If ~5% of frames are missing, your system is likely overloaded — follow the recommendations above. If fewer frames are missing (~2%), they are probably due to occasional transient load spikes (often at startup) and can be considered expected.                                                  |
+| Symptom                                                            | Likely Cause                                 | Fix                                                                                                                                                                                                                                                                                  |
+| ------------------------------------------------------------------ | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| System freezes or choppy robot movement or Rerun visualization lag | CPU starved (100% load usage)                | Close other apps, reduce encoding throughput, lower `encoder_threads`, use `h264`, use `display_data=False`. If the CPU continues to be at 100% then it might be insufficient for your setup, consider `--dataset.streaming_encoding=false` or HW encoding (`--dataset.vcodec=auto`) |
+| "Encoder queue full" warnings or dropped frames in dataset         | Encoder can't keep up (Queue overflow)       | If CPU is not at 100%: Increase `encoder_threads`, increase `encoder_queue_maxsize` or use HW encoding (`--dataset.vcodec=auto`).                                                                                                                                                    |
+| High RAM usage                                                     | Queue filling faster than encoding           | `encoder_threads` too low or CPU insufficient. Reduce `encoder_queue_maxsize` or use HW encoding                                                                                                                                                                                     |
+| Large video files                                                  | Using HW encoder or H.264                    | Expected trade-off. Switch to `libsvtav1` if CPU allows                                                                                                                                                                                                                              |
+| `save_episode()` still slow                                        | `streaming_encoding` is `False`              | Set `--dataset.streaming_encoding=true`                                                                                                                                                                                                                                              |
+| Encoder thread crash                                               | Codec not available or invalid settings      | Check `vcodec` is installed, try `--dataset.vcodec=auto`                                                                                                                                                                                                                             |
+| Recorded dataset is missing frames                                 | CPU/GPU starvation or occasional load spikes | If ~5% of frames are missing, your system is likely overloaded — follow the recommendations above. If fewer frames are missing (~2%), they are probably due to occasional transient load spikes (often at startup) and can be considered expected.                                   |
 
 ## 6. Recommended Configurations
 
@@ -146,7 +146,7 @@ On very constrained systems, streaming encoding may compete too heavily with the
 # 2camsx 640x480x3 @30fps: Requires some tuning.
 
 # Use H.264, disable streaming, consider batching encoding
-lerobot-record --dataset.camera_encoder.vcodec=h264 --dataset.streaming_encoding=false ...
+lerobot-record --dataset.vcodec=h264 --dataset.streaming_encoding=false ...
 ```
 
 ## 7. Closing note

docs/source/tools.mdx

@@ -1,210 +0,0 @@
-# Tools
-
-LeRobot v3.1 supports **tool calls** in policies — assistant messages can
-emit structured invocations like `say(text="OK, starting now")` that the
-runtime dispatches to a real implementation (TTS, controller, logger, …).
-
-This page covers:
-
-1. Where the tool catalog lives.
-2. How the annotation pipeline produces tool-call atoms.
-3. How to add your own tool.
-
-## Where tools are declared
-
-Two layers.
-
-**The catalog** — a list of OpenAI-style function schemas — lives at
-`meta/info.json["tools"]` on each dataset. Example:
-
-```json
-{
-  "features": { "...": "..." },
-  "tools": [
-    {
-      "type": "function",
-      "function": {
-        "name": "say",
-        "description": "Speak a short utterance to the user via the TTS executor.",
-        "parameters": {
-          "type": "object",
-          "properties": {
-            "text": {
-              "type": "string",
-              "description": "The verbatim text to speak."
-            }
-          },
-          "required": ["text"]
-        }
-      }
-    }
-  ]
-}
-```
-
-Read it via the dataset metadata accessor:
-
-```python
-from lerobot.datasets.dataset_metadata import LeRobotDatasetMetadata
-
-meta = LeRobotDatasetMetadata(repo_id="pepijn/super_poulain_final_annotations")
-tools = meta.tools     # list[dict] — OpenAI tool schemas
-```
-
-If the dataset's `info.json` doesn't declare any tools, `meta.tools`
-returns `DEFAULT_TOOLS` from `lerobot.datasets.language` — currently a
-single-entry list with the canonical `say` schema. So unannotated
-datasets and chat-template consumers keep working without any
-configuration:
-
-```python
-prompt_str = tokenizer.apply_chat_template(
-    sample["messages"],
-    tools=meta.tools,                 # works either way
-    add_generation_prompt=False,
-    tokenize=False,
-)
-```
-
-**The implementations** — runnable Python — will live under
-`src/lerobot/tools/`, one file per tool. The runtime dispatcher and
-the canonical `say` implementation (wrapping Kyutai's pocket-tts) are
-not part of the catalog layer described here; today this layer ships
-only the schema storage and the `DEFAULT_TOOLS` fallback constant.
-
-## Per-row tool _invocations_
-
-The catalog above describes _what can be called_. The actual _call_ — the
-function name plus the argument values — is stored per-row, on the
-assistant atoms in `language_events`:
-
-```python
-{
-  "role": "assistant",
-  "content": null,
-  "style": null,
-  "timestamp": 12.4,
-  "camera": null,
-  "tool_calls": [
-    { "type": "function",
-      "function": { "name": "say", "arguments": { "text": "On it." } } }
-  ]
-}
-```
-
-Recipes splice these into rendered messages via `tool_calls_from`:
-
-```yaml
-user_interjection_response:
-  bindings:
-    speech: "emitted_at(t, role=assistant, tool_name=say)"
-  messages:
-    - { role: user, content: "${task}", stream: high_level }
-    - {
-        role: assistant,
-        content: "${current_plan}",
-        stream: high_level,
-        target: true,
-        tool_calls_from: speech,
-      }
-```
-
-The model's training target is one assistant turn that carries both the
-plan text _and_ the `say` tool call. At inference, the runtime parses
-the generated text back into structured `tool_calls` and dispatches to
-the matching implementation.
-
-## How to add your own tool
-
-> **Note:** Steps 2 and 3 below describe the runtime layer
-> (`src/lerobot/tools/`, the `Tool` protocol, `TOOL_REGISTRY`,
-> `get_tools(meta)`) which is not part of the catalog layer shipped
-> today — those modules don't yet exist in the tree. Step 1 alone is
-> enough to make the tool visible to the chat template via
-> `meta.tools` so the model can learn to _generate_ the call;
-> executing the call at inference requires the runtime layer.
-
-Three steps. Concrete example: a `record_observation` tool the policy
-can call to capture an extra observation outside the regular control
-loop.
-
-### Step 1 — declare the schema
-
-Add an entry under `meta/info.json["tools"]`. Either edit the file
-directly on disk _before_ running the annotation pipeline (it'll be
-preserved) or hand it to `lerobot-annotate` via a config flag.
-
-```json
-{
-  "tools": [
-    { "type": "function", "function": { "name": "say", "...": "..." } },
-    {
-      "type": "function",
-      "function": {
-        "name": "record_observation",
-        "description": "Capture a high-resolution still image for the user.",
-        "parameters": {
-          "type": "object",
-          "properties": {
-            "label": {
-              "type": "string",
-              "description": "Short label for the saved image."
-            }
-          },
-          "required": ["label"]
-        }
-      }
-    }
-  ]
-}
-```
-
-The schema follows OpenAI's function-calling convention exactly, so the
-chat template can render it natively.
-
-### Step 2 — implement the call
-
-Create `src/lerobot/tools/record_observation.py`:
-
-```python
-from .base import Tool
-from typing import Any
-
-RECORD_OBSERVATION_SCHEMA: dict[str, Any] = { "...": "..." }   # mirrors the JSON above
-
-
-class RecordObservationTool:
-    name = "record_observation"
-    schema = RECORD_OBSERVATION_SCHEMA
-
-    def __init__(self, schema: dict | None = None, output_dir: str = "."):
-        self.output_dir = output_dir
-
-    def call(self, arguments: dict) -> str:
-        label = arguments["label"]
-        # ... save the latest camera frame to <output_dir>/<label>.png ...
-        return f"saved {label}.png"
-```
-
-One file per tool keeps dependencies isolated — `record_observation`
-might pull `pillow`, while `say` pulls `pocket-tts`. Users installing
-only the tools they need avoid heavy transitive deps.
-
-### Step 3 — register it
-
-Add to `src/lerobot/tools/registry.py`:
-
-```python
-from .record_observation import RecordObservationTool
-
-TOOL_REGISTRY["record_observation"] = RecordObservationTool
-```
-
-That's it. At runtime `get_tools(meta)` looks up each schema in
-`meta.tools`, instantiates the matching registered class, and returns
-a name → instance dict the dispatcher can route into.
-
-If you want to use a tool _without_ writing an implementation (e.g. for
-training-time chat-template formatting only), step 1 alone is enough —
-the model still learns to _generate_ the call. Steps 2 and 3 are only
-needed to actually _execute_ it at inference.

docs/source/topreward.mdx

@@ -1,177 +0,0 @@
-# TOPReward
-
-TOPReward is a **zero-shot reward model** that extracts token log-probabilities from an off-the-shelf vision-language model (VLM) as a robotic reward signal. Given a video trajectory and a task instruction, it returns the VLM's log-likelihood that the instruction is true — no fine-tuning required.
-
-**Paper**: [TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics](https://arxiv.org/abs/2602.19313)
-**Project**: [topreward.github.io](https://topreward.github.io/webpage/)
-**Original code**: [github.com/TOPReward/TOPReward](https://github.com/TOPReward/TOPReward)
-**Default backbone**: [Qwen/Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct)
-
-## Overview
-
-TOPReward asks a generic VLM how likely a task instruction is, **conditioned on the video** of a robot trying to complete that task. Concretely, given:
-
-- A trajectory video (a sequence of frames).
-- A task instruction (e.g. _"open the drawer"_).
-
-it builds a chat prompt of the form
-
-```text
-<video>
-"The above video shows a robot manipulation trajectory that completes the
- following task: <instruction> Decide whether the above statement is True
- or not. The answer is: True"
-```
-
-forwards it through the VLM, label-masks everything except the very last token, and reads back the log-probability of that token — by default the literal `"True"` that closes the suffix template. The resulting `log P("True" | video + prompt + instruction)` is the reward.
-
-Because the method only depends on a frozen VLM, TOPReward is **zero-shot**: there are no fine-tuned weights to host. The "model" in LeRobot is a small wrapper around `transformers`' `Qwen3VLForConditionalGeneration` plus the label-masking logic. The processor owns the tokeniser and builds the full chat prompt (EO-1/Robometer pattern).
-
-## What the LeRobot integration covers
-
-- Standard `reward_model.type=topreward` configuration through LeRobot.
-- VLM loading via the `transformers` `Qwen3VLForConditionalGeneration` API.
-- Prompt assembly + tokenisation in the processor (matching upstream `QwenClient.compute_instruction_reward`).
-- `compute_reward()` returns one scalar log-prob per sample.
-- LeRobot reward-model save/load — `save_pretrained` writes only `config.json` (the VLM is identified by `vlm_name`).
-- An offline labeling script that writes a `topreward_progress.parquet` (SARM-compatible schema) for RA-BC and overlay.
-
-The current LeRobot port supports the **Qwen3-VL client only**. Other upstream clients (Gemini, OpenAI, Gemma, Molmo) can be added as follow-up extras.
-
-## Installation Requirements
-
-1. Install LeRobot following the [Installation Guide](./installation).
-2. Install the TOPReward optional extra:
-
-```bash
-pip install -e ".[topreward]"
-```
-
-or, with `uv` from a source checkout:
-
-```bash
-uv sync --extra topreward
-```
-
-This pulls in `transformers`. The first time you run TOPReward, Hugging Face will also download the VLM weights from the Hub (~16 GB for Qwen3-VL-8B-Instruct). A GPU is strongly recommended.
-
-## Model Inputs and Outputs
-
-TOPReward expects:
-
-- A trajectory video or sequence of frames.
-- A natural-language task description.
-
-In LeRobot datasets the preprocessor reads:
-
-| Config field              | Default                     | Meaning                                       |
-| ------------------------- | --------------------------- | --------------------------------------------- |
-| `reward_model.image_key`  | `observation.images.top`    | Camera observation used by TOPReward          |
-| `reward_model.task_key`   | `task`                      | Key in complementary data for the task string |
-| `reward_model.max_frames` | `16`                        | Cap on frames per sample                      |
-| `reward_model.fps`        | `2.0`                       | Metadata passed to the Qwen video processor   |
-| `reward_model.vlm_name`   | `Qwen/Qwen3-VL-8B-Instruct` | Hugging Face Hub id of the underlying VLM     |
-
-The model returns:
-
-- `compute_reward(batch)`: one log-probability per sample. Higher = better task-video alignment. When `success_threshold` is finite, returns the binary thresholded value instead.
-
-## Usage
-
-### Load the reward model directly
-
-```python
-from lerobot.rewards.topreward import TOPRewardConfig, TOPRewardModel
-
-cfg = TOPRewardConfig(
-    vlm_name="Qwen/Qwen3-VL-8B-Instruct",
-    device="cuda",
-)
-reward_model = TOPRewardModel(cfg)
-```
-
-### Use the reward factory
-
-```python
-from lerobot.rewards import make_reward_model, make_reward_model_config, make_reward_pre_post_processors
-
-cfg = make_reward_model_config(
-    "topreward",
-    vlm_name="Qwen/Qwen3-VL-8B-Instruct",
-    device="cuda",
-    image_key="observation.images.top",
-)
-reward_model = make_reward_model(cfg)
-preprocessor, postprocessor = make_reward_pre_post_processors(cfg)
-```
-
-The preprocessor tokenises the full prompt (video + prefix + instruction suffix), writes Qwen-VL tensors + `prompt_length` under `observation.topreward.*`. The model reads those tensors, label-masks based on `prompt_length`, and extracts the log-prob reward.
-
-### Offline dataset labeling
-
-Write a `topreward_progress.parquet` for RA-BC training and overlay videos:
-
-```bash
-# Sparse-dense (15 anchors per episode, matches upstream)
-uv run python -m lerobot.rewards.topreward.compute_rabc_weights \
-    --dataset-repo-id lerobot/libero_10_image \
-    --num-samples 15 \
-    --device cuda
-```
-
-Then render the progress overlay for any episode:
-
-```bash
-uv run examples/dataset/create_progress_videos.py \
-    --repo-id lerobot/libero_10_image \
-    --episode 0 \
-    --progress-file topreward_progress.parquet \
-    --gif
-```
-
-## Configuration Notes
-
-### Prompt knobs
-
-The default prompt mirrors the upstream paper:
-
-```text
-prompt_prefix = "The above video shows a robot manipulation trajectory that completes the following task: "
-prompt_suffix_template = "{instruction} Decide whether the above statement is True or not. The answer is: True"
-```
-
-Both are exposed on `TOPRewardConfig` for ablation. The suffix template **must** contain `{instruction}`.
-
-### Chat template
-
-`add_chat_template=True` wraps the full prompt (including instruction) with the tokenizer's chat template before tokenisation. Default is `False`, matching the upstream paper's main experiments.
-
-## Limitations
-
-- The current LeRobot port is **inference-only and zero-shot**; `forward()` is not overridden and `is_trainable` returns `False`.
-- Only the **Qwen3-VL family** is supported; other upstream clients are out of scope.
-- TOPReward inherits the underlying VLM's biases.
-
-## References
-
-- [TOPReward project page](https://topreward.github.io/webpage/)
-- [TOPReward paper](https://arxiv.org/abs/2602.19313)
-- [Original TOPReward code](https://github.com/TOPReward/TOPReward)
-- [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct)
-
-## Citation
-
-```bibtex
-@article{chen2026topreward,
-  title={TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics},
-  author={Chen, Shirui and Harrison, Cole and Lee, Ying-Chun and Yang, Angela Jin and
-          Ren, Zhongzheng and Ratliff, Lillian J and Duan, Jiafei and Fox, Dieter and
-          Krishna, Ranjay},
-  journal={arXiv preprint arXiv:2602.19313},
-  year={2026}
-}
-```
-
-## License
-
-The original TOPReward codebase is MIT-licensed. The LeRobot port follows the LeRobot Apache 2.0 license; the wrapped Qwen3-VL weights are subject to the original Qwen license.

docs/source/using_dataset_tools.mdx

@@ -117,10 +117,10 @@ lerobot-edit-dataset \
     --repo_id lerobot/pusht_image \
     --operation.type convert_image_to_video \
     --operation.output_dir outputs/pusht_video \
-    --operation.camera_encoder.vcodec libsvtav1 \
-    --operation.camera_encoder.pix_fmt yuv420p \
-    --operation.camera_encoder.g 2 \
-    --operation.camera_encoder.crf 30
+    --operation.vcodec libsvtav1 \
+    --operation.pix_fmt yuv420p \
+    --operation.g 2 \
+    --operation.crf 30
 
 # Convert only specific episodes
 lerobot-edit-dataset \
@@ -147,7 +147,11 @@ lerobot-edit-dataset \
 **Parameters:**
 
 - `output_dir`: Custom output directory (optional - by default uses `new_repo_id` or `{repo_id}_video`)
-- `camera_encoder`: Video encoder settings — all sub-fields accessible via `--operation.camera_encoder.<field>. See [Video Encoding Parameters](./video_encoding_parameters) for more details.
+- `vcodec`: Video codec to use - options: `h264`, `hevc`, `libsvtav1` (default: `libsvtav1`)
+- `pix_fmt`: Pixel format - options: `yuv420p`, `yuv444p` (default: `yuv420p`)
+- `g`: Group of pictures (GOP) size - lower values give better quality but larger files (default: 2)
+- `crf`: Constant rate factor - lower values give better quality but larger files, 0 is lossless (default: 30)
+- `fast_decode`: Fast decode tuning option (default: 0)
 - `episode_indices`: List of specific episodes to convert (default: all episodes)
 - `num_workers`: Number of parallel workers for processing (default: 4)
 

docs/source/video_encoding_parameters.mdx

@@ -1,117 +0,0 @@
-# Video encoding parameters
-
-When video storage is enabled, LeRobot stores each camera stream as an **MP4** file instead of saving one image file per timestep. Video encoding compresses across time, which usually cuts dataset size and I/O compared to a pile of PNG, while keeping MP4 — a format every player and loader understands.
-
-Encoding frames into an MP4 is a full FFmpeg pipeline: choice of encoder, pixel format, GOP/keyframes, quality vs. speed, and optional extra encoder flags. Most of these knobs are user-tunable through `camera_encoder`, a nested `VideoEncoderConfig` (`lerobot.configs.video.VideoEncoderConfig`) passed through PyAV.
-
-You can set these parameters from the CLI with `--dataset.camera_encoder.<field>` (e.g. with `lerobot-record` or `lerobot-rollout`). The same block applies to every camera video stream in that run.
-
-<Tip>
-  Video storage must be on for `camera_encoder` to have any effect —
-  `use_videos=True` in Python APIs, or `--dataset.video=true` on the CLI (the
-  recording default). With video off, inputs stay as images and `camera_encoder`
-  is ignored.
-</Tip>
-
-For details on **when** frames are written vs. encoded (streaming vs. post-episode), queues, and other top-level `--dataset.*` switches, see [Streaming Video Encoding](./streaming_video_encoding). For an encoding-parameter comparison and experiments, see the [video-benchmark Space](https://huggingface.co/spaces/lerobot/video-benchmark).
-
----
-
-## Example
-
-```bash
-lerobot-record \
-    --robot.type=so100_follower \
-    --robot.port=/dev/tty.usbmodem58760431541 \
-    --robot.cameras="{laptop: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
-    --robot.id=black \
-    --teleop.type=so100_leader \
-    --teleop.port=/dev/tty.usbmodem58760431551 \
-    --teleop.id=blue \
-    --dataset.repo_id=<my_username>/<my_dataset_name> \
-    --dataset.num_episodes=2 \
-    --dataset.single_task="Grab the cube" \
-    --dataset.streaming_encoding=true \
-    --dataset.encoder_threads=2 \
-    --dataset.camera_encoder.vcodec=h264 \
-    --dataset.camera_encoder.preset=fast \
-    --dataset.camera_encoder.extra_options={"tune": "film", "profile:v": "high", "bf": 2} \
-    --display_data=true
-```
-
----
-
-## Tuning parameters
-
-<Tip warning={true}>
-The defaults are tuned to balance **compression ratio**, **visual quality**, and **decoding/seek speed** for typical robotics datasets. Changing them can affect both recording (CPU load, frame drops) and training (decoding throughput, image quality).
-
-Only override these parameters if you have a specific reason to, and measure the impact on your pipeline before relying on the new settings.
-
-</Tip>
-
-All flags below are prefixed with `--dataset.camera_encoder.` on the CLI.
-
-| Parameter       | Type             | Default       | Description                                                                                                                                                                            |
-| --------------- | ---------------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `vcodec`        | `str`            | `"libsvtav1"` | Video codec name. `"auto"` picks the first available hardware encoder from a fixed preference list, falling back to `libsvtav1`.                                                       |
-| `pix_fmt`       | `str`            | `"yuv420p"`   | Output pixel format. Must be supported by the chosen codec in your FFmpeg build.                                                                                                       |
-| `g`             | `int`            | `2`           | GOP size — a keyframe every `g` frames. Emitted as FFmpeg option `g`.                                                                                                                  |
-| `crf`           | `int` or `float` | `30`          | Abstract quality value, mapped per codec (see the [mapping](#mapping-videoencoderconfig--ffmpeg-options) below). Lower → higher quality / larger output where the mapping is monotone. |
-| `preset`        | `int` or `str`   | `12` \*       | Encoder speed preset; meaning depends on the codec. <br/>\* When unset and `vcodec=libsvtav1`, LeRobot defaults to `12`.                                                               |
-| `fast_decode`   | `int`            | `0`           | `libsvtav1`: `0–2`, passed via `svtav1-params`. <br/>`h264` / `hevc` (software): if `>0`, sets `tune=fastdecode`. <br/>Other codecs: usually unused.                                   |
-| `video_backend` | `str`            | `"pyav"`      | Only `"pyav"` is currently implemented for video encoding.                                                                                                                             |
-| `extra_options` | `dict`           | `{}`          | Extra FFmpeg or codec specific options merged after the structured fields above. Cannot override keys already set by those fields.                                                     |
-
----
-
-## Persistence in dataset metadata
-
-After the first episode of a video stream is encoded, the encoder configuration is **persisted into the dataset metadata** (`meta/info.json`) under each video feature, alongside the values probed from the file itself. For a video feature `observation.images.<camera>`, the layout in `info.json` is:
-
-```json
-{
-  "features": {
-    "observation.images.laptop": {
-      "dtype": "video",
-      "shape": [480, 640, 3],
-      "info": {
-        "video.height": 480,
-        "video.width": 640,
-        "video.codec": "h264",
-        "video.pix_fmt": "yuv420p",
-        "video.fps": 30,
-        "video.channels": 3,
-        "video.is_depth_map": false,
-        "video.g": 2,
-        "video.crf": 30,
-        "video.preset": "fast",
-        "video.fast_decode": 0,
-        "video.video_backend": "pyav",
-        "video.extra_options": { "tune": "film", "profile:v": "high", "bf": 2 }
-      }
-    }
-  }
-}
-```
-
-Two sources contribute to the `info` block:
-
-- **Stream-derived** (read back from the encoded MP4 with PyAV): `video.height`, `video.width`, `video.codec`, `video.pix_fmt`, `video.fps`, `video.channels`, `video.is_depth_map`, plus `audio.*` if an audio stream is present.
-- **Encoder-derived** (taken from `VideoEncoderConfig`): `video.g`, `video.crf`, `video.preset`, `video.fast_decode`, `video.video_backend`, `video.extra_options`.
-
-<Tip>
-  This block is populated **once**, from the **first** episode. It assumes every
-  episode in the dataset was encoded with the same `camera_encoder`. Changing
-  encoder settings partway through a recording is not supported — the
-  `info.json` will only reflect the parameters used for the first episode.
-</Tip>
-
----
-
-## Merging datasets
-
-When aggregating datasets with `merge_datasets`, video files are concatenated as-is (no re-encoding), and encoder fields in `info.json` are merged per-key:
-
-- **Stream-derived fields must match** across sources: `video.codec`, `video.pix_fmt`, `video.height`, `video.width`, `video.fps`. Otherwise FFmpeg's concat demuxer fails.
-- **Encoder-tuning fields are merged loosely**: `video.g`, `video.crf`, `video.preset`, `video.fast_decode`, `video.extra_options`. If every source agrees, the value is kept; if not, it's set to `null` (or `{}` for `video.extra_options`) and a warning is logged.

examples/dataset/create_robometer_progress_videos.py

@@ -0,0 +1,244 @@
+#!/usr/bin/env python
+
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Create videos with a Robometer progress overlay for one LeRobot dataset episode.
+
+This is a lightweight smoke-test utility for Robometer checkpoints. It downloads
+one episode video, samples a small number of frames, runs Robometer on those
+frames, and reuses the progress overlay renderer from
+``examples/dataset/create_progress_videos.py``.
+
+Example:
+
+    uv run python examples/dataset/create_robometer_progress_videos.py \\
+        --repo-id lerobot/aloha_mobile_cabinet \\
+        --episode 0 \\
+        --reward-model-path lilkm/robometer-4b \\
+        --device cuda
+"""
+
+from __future__ import annotations
+
+import argparse
+import logging
+from pathlib import Path
+
+import cv2
+import numpy as np
+import torch
+
+from examples.dataset.create_progress_videos import (
+    composite_progress_video,
+    convert_mp4_to_gif,
+    download_episode_metadata,
+    download_video_file,
+    load_episode_meta,
+)
+from lerobot.rewards.robometer import RobometerConfig, RobometerRewardModel
+from lerobot.rewards.robometer.modeling_robometer import decode_progress_outputs
+from lerobot.rewards.robometer.processor_robometer import RobometerEncoderProcessorStep
+from lerobot.utils.utils import init_logging
+
+
+def _default_device() -> str:
+    return "cuda" if torch.cuda.is_available() else "cpu"
+
+
+def sample_episode_frames(
+    video_path: Path,
+    *,
+    from_timestamp: float,
+    to_timestamp: float,
+    fps: float,
+    num_frames: int,
+) -> tuple[np.ndarray, np.ndarray]:
+    """Sample RGB frames uniformly from an episode video segment.
+
+    Returns:
+        ``(frames, frame_indices)`` where ``frames`` is ``(T,H,W,C)`` uint8 RGB
+        and ``frame_indices`` are local episode frame indices used for overlay.
+    """
+    if num_frames <= 0:
+        raise ValueError(f"num_frames must be positive, got {num_frames}")
+
+    duration_seconds = to_timestamp - from_timestamp
+    total_frames = max(int(round(duration_seconds * fps)), 1)
+    frame_indices = np.linspace(0, total_frames - 1, num=min(num_frames, total_frames), dtype=int)
+
+    capture = cv2.VideoCapture(str(video_path))
+    frames: list[np.ndarray] = []
+    try:
+        for frame_idx in frame_indices:
+            timestamp = from_timestamp + frame_idx / fps
+            capture.set(cv2.CAP_PROP_POS_MSEC, timestamp * 1000)
+            ret, frame_bgr = capture.read()
+            if not ret:
+                logging.warning("Could not read frame %d at %.3fs", frame_idx, timestamp)
+                continue
+            frames.append(cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2RGB))
+    finally:
+        capture.release()
+
+    if not frames:
+        raise RuntimeError(f"No frames could be sampled from {video_path}")
+
+    return np.stack(frames), frame_indices[: len(frames)]
+
+
+def predict_robometer_progress(
+    frames: np.ndarray,
+    *,
+    task: str,
+    reward_model_path: str,
+    device: str,
+) -> list[float]:
+    """Run Robometer and return per-sampled-frame progress predictions."""
+    config = RobometerConfig(pretrained_path=reward_model_path, device=device, max_frames=None)
+    model = RobometerRewardModel.from_pretrained(reward_model_path, config=config)
+
+    encoder = RobometerEncoderProcessorStep(
+        base_model_id=model.config.base_model_id,
+        use_multi_image=model.config.use_multi_image,
+        use_per_frame_progress_token=model.config.use_per_frame_progress_token,
+        max_frames=None,
+    )
+    batch = encoder.encode_samples([(frames, task)])
+
+    model_device = next(model.model.parameters()).device
+    inputs = {key: value.to(model_device) if hasattr(value, "to") else value for key, value in batch.items()}
+
+    model.eval()
+    with torch.no_grad():
+        progress_logits, success_logits = model._compute_rbm_logits(inputs)
+
+    decoded = decode_progress_outputs(
+        progress_logits,
+        success_logits,
+        is_discrete_mode=model.config.use_discrete_progress,
+    )
+    return decoded["progress_pred"][0]
+
+
+def process_dataset(
+    repo_id: str,
+    episode: int,
+    reward_model_path: str,
+    device: str,
+    camera_key: str | None,
+    output_dir: Path,
+    num_frames: int,
+    task: str | None = None,
+    create_gif: bool = False,
+) -> Path:
+    safe_name = repo_id.replace("/", "_")
+    logging.info("Processing %s episode %d with Robometer %s", repo_id, episode, reward_model_path)
+
+    local_path = download_episode_metadata(repo_id, episode)
+    episode_meta = load_episode_meta(local_path, episode, camera_key)
+    video_path = download_video_file(repo_id, local_path, episode_meta["video_rel"])
+
+    task_name = task or episode_meta.get("task_name", "")
+    if not task_name:
+        raise ValueError("No task found in dataset metadata. Pass --task explicitly.")
+
+    frames, frame_indices = sample_episode_frames(
+        video_path,
+        from_timestamp=episode_meta["from_ts"],
+        to_timestamp=episode_meta["to_ts"],
+        fps=episode_meta["fps"],
+        num_frames=num_frames,
+    )
+    logging.info("Sampled %d frames for Robometer inference", len(frames))
+
+    progress = predict_robometer_progress(
+        frames,
+        task=task_name,
+        reward_model_path=reward_model_path,
+        device=device,
+    )
+    progress_data = np.stack([frame_indices, np.asarray(progress, dtype=np.float32)], axis=1)
+    logging.info("Progress predictions: %s", [round(float(value), 3) for value in progress])
+
+    output_path = output_dir / f"{safe_name}_ep{episode}_robometer_progress.mp4"
+    final_path = composite_progress_video(
+        video_path=video_path,
+        from_timestamp=episode_meta["from_ts"],
+        to_timestamp=episode_meta["to_ts"],
+        progress_data=progress_data,
+        output_path=output_path,
+        fps=episode_meta["fps"],
+        task_name=task_name,
+    )
+
+    if create_gif:
+        final_path = convert_mp4_to_gif(final_path)
+    return final_path
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Create MP4/GIF videos with Robometer progress overlay for dataset episodes."
+    )
+    parser.add_argument("--repo-id", required=True, help="Hugging Face LeRobot dataset repo id.")
+    parser.add_argument("--episode", type=int, required=True, help="Episode index to visualize.")
+    parser.add_argument(
+        "--reward-model-path",
+        default="lilkm/robometer-4b",
+        help="Robometer checkpoint path or Hub repo id (e.g. lilkm/robometer-4b).",
+    )
+    parser.add_argument("--device", default=_default_device(), help="Torch device for Robometer inference.")
+    parser.add_argument(
+        "--camera-key",
+        default=None,
+        help="Camera observation key (e.g. observation.images.top). Auto-selects first camera if omitted.",
+    )
+    parser.add_argument(
+        "--task", default=None, help="Task description override if dataset metadata lacks one."
+    )
+    parser.add_argument(
+        "--num-frames",
+        type=int,
+        default=8,
+        help="Number of episode frames to sample for Robometer inference.",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=Path,
+        default=Path("progress_videos"),
+        help="Directory to write output files.",
+    )
+    parser.add_argument("--gif", action="store_true", help="Also generate a GIF from the MP4 output.")
+    args = parser.parse_args()
+
+    init_logging()
+    args.output_dir.mkdir(parents=True, exist_ok=True)
+
+    result = process_dataset(
+        repo_id=args.repo_id,
+        episode=args.episode,
+        reward_model_path=args.reward_model_path,
+        device=args.device,
+        camera_key=args.camera_key,
+        output_dir=args.output_dir,
+        num_frames=args.num_frames,
+        task=args.task,
+        create_gif=args.gif,
+    )
+    logging.info("Output: %s", result)
+
+
+if __name__ == "__main__":
+    main()

examples/notebooks/quickstart.ipynb

@@ -80,7 +80,7 @@
     "}\n",
     "\n",
     "# Dataset\n",
-    "HF_USER = \"your_hf_username\"  # `hf auth whoami` to find your username\n",
+    "HF_USER = \"your_hf_username\"  # `huggingface-cli whoami` to find your username\n",
     "DATASET_NAME = \"my_so101_dataset\"\n",
     "TASK_DESCRIPTION = \"pick and place the block\"\n",
     "NUM_EPISODES = 10\n",
@@ -291,34 +291,7 @@
     "\n",
     "Uses `POLICY_PATH` from the Configuration cell (defaults to the Hub repo ID). You can also put there the `LAST_CHECKPOINT_PATH`.\n",
     "\n",
-    "See the [inference docs](https://huggingface.co/docs/lerobot/il_robots#run-inference-and-evaluate-your-policy) for details.\n",
-    "\n",
-    "Recently ```lerobot-rollout``` was introduced, you can [read more about it here](https://huggingface.co/docs/lerobot/main/en/il_robots?eval=Base+mode+%28no+recording%29#run-inference-and-evaluate-your-policy)."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "print_cmd(\n",
-    "    \"lerobot-rollout\",\n",
-    "    \"--strategy.type=base\",\n",
-    "    f\"--policy.path={POLICY_PATH}\",\n",
-    "    f\"--robot.type={ROBOT_TYPE}\",\n",
-    "    f\"--robot.port={ROBOT_PORT}\",\n",
-    "    CAMERAS_FLAG,\n",
-    "    f'--task=\"{TASK_DESCRIPTION}\"',\n",
-    "    \"--duration=60\",\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "if you are using the V0.5.1 release you should use ```lerobot-record``` instead of rollout"
+    "See the [inference docs](https://huggingface.co/docs/lerobot/il_robots#run-inference-and-evaluate-your-policy) for details."
    ]
   },
   {

pyproject.toml

@@ -95,7 +95,7 @@ dependencies = [
 
 # ── Feature-scoped extras ──────────────────────────────────
 dataset = [
-    "datasets>=4.7.0,<5.0.0",
+    "datasets>=4.0.0,<5.0.0",
     "pandas>=2.0.0,<3.0.0", # NOTE: Transitive dependency of datasets
     "pyarrow>=21.0.0,<30.0.0", # NOTE: Transitive dependency of datasets
     "lerobot[av-dep]",
@@ -151,8 +151,6 @@ pyserial-dep = ["pyserial>=3.5,<4.0"]
 deepdiff-dep = ["deepdiff>=7.0.1,<9.0.0"]
 pynput-dep = ["pynput>=1.7.8,<1.9.0"]
 pyzmq-dep = ["pyzmq>=26.2.1,<28.0.0"]
-motorbridge-dep = ["motorbridge>=0.3.2,<0.4.0"]
-motorbridge-smart-servo-dep = ["motorbridge-smart-servo>=0.0.4,<0.1.0"]
 
 # Motors
 feetech = ["feetech-servo-sdk>=1.0.0,<2.0.0", "lerobot[pyserial-dep]", "lerobot[deepdiff-dep]"]
@@ -176,9 +174,6 @@ unitree_g1 = [
     "lerobot[pygame-dep]",
 ]
 reachy2 = ["reachy2_sdk>=1.0.15,<1.1.0"]
-# Seeed Studio reBot B601-DM follower (motorbridge / CAN) + StarArm102 / reBot Arm 102
-# leader (motorbridge-smart-servo / FashionStar UART servos).
-rebot = ["lerobot[motorbridge-dep]", "lerobot[motorbridge-smart-servo-dep]"]
 kinematics = ["lerobot[placo-dep]"]
 intelrealsense = [
     "pyrealsense2>=2.55.1.6486,<2.57.0 ; sys_platform != 'darwin'",
@@ -209,7 +204,7 @@ groot = [
     "flash-attn>=2.5.9,<3.0.0 ; sys_platform != 'darwin'"
 ]
 sarm = ["lerobot[transformers-dep]", "pydantic>=2.0.0,<3.0.0", "faker>=33.0.0,<35.0.0", "lerobot[matplotlib-dep]", "lerobot[qwen-vl-utils-dep]"]
-topreward = ["lerobot[transformers-dep]"]
+robometer = ["lerobot[transformers-dep]", "lerobot[qwen-vl-utils-dep]", "lerobot[peft-dep]"]
 xvla = ["lerobot[transformers-dep]"]
 eo1 = ["lerobot[transformers-dep]", "lerobot[qwen-vl-utils-dep]"]
 hilserl = ["lerobot[transformers-dep]", "lerobot[dataset]", "gym-hil>=0.1.13,<0.2.0", "lerobot[grpcio-dep]", "lerobot[placo-dep]"]
@@ -266,7 +261,6 @@ all = [
     "lerobot[lekiwi]",
     "lerobot[openarms]",
     "lerobot[reachy2]",
-    "lerobot[rebot]",
     "lerobot[kinematics]",
     "lerobot[intelrealsense]",
     "lerobot[diffusion]",
@@ -287,7 +281,6 @@ all = [
     "lerobot[libero]; sys_platform == 'linux'",
     "lerobot[metaworld]",
     "lerobot[sarm]",
-    "lerobot[topreward]",
     "lerobot[peft]",
     # "lerobot[unitree_g1]", TODO: Unitree requires specific installation instructions for unitree_sdk2
 ]
@@ -310,6 +303,7 @@ lerobot-imgtransform-viz="lerobot.scripts.lerobot_imgtransform_viz:main"
 lerobot-edit-dataset="lerobot.scripts.lerobot_edit_dataset:main"
 lerobot-setup-can="lerobot.scripts.lerobot_setup_can:main"
 lerobot-rollout="lerobot.scripts.lerobot_rollout:main"
+lerobot-export-robometer="lerobot.scripts.lerobot_export_robometer:main"
 
 # ---------------- Tool Configurations ----------------
 

scripts/debug_robometer_embed_diff.py

@@ -0,0 +1,164 @@
+#!/usr/bin/env python
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+"""Pinpoint exactly which rows of ``embed_tokens`` / ``lm_head`` differ.
+
+Useful follow-up to ``scripts/verify_robometer_export.py`` when the verifier
+reports a small tail of differing keys but you want to know whether the
+diff is:
+
+1. Concentrated in the 5 special-token rows added by ``resize_token_embeddings``
+   (expected non-determinism: mean-resize sampling differs between runs).
+2. Spread across the full vocabulary (would point to a real loading bug).
+
+Also confirms whether ``apply_upstream_checkpoint`` actually overwrites the
+embed/lm-head tensors when loading the upstream state dict (vs. silently
+skipping them due to a key mismatch).
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+
+import torch
+from safetensors.torch import load_file
+
+from lerobot.configs.rewards import RewardModelConfig
+from lerobot.rewards.robometer import RobometerConfig, RobometerRewardModel
+from lerobot.rewards.robometer._upstream_loader import (
+    _download_robometer_snapshot,
+    _remap_state_dict_keys,
+    _resolve_checkpoint_safetensors_files,
+    apply_upstream_checkpoint,
+)
+
+EMBED_KEY = "model.model.language_model.embed_tokens.weight"
+LMHEAD_KEY = "model.lm_head.weight"
+
+
+def _load_upstream(path: str) -> RobometerRewardModel:
+    cfg = RobometerConfig(pretrained_path=path, device="cpu")
+    model = RobometerRewardModel(cfg)
+    apply_upstream_checkpoint(model, path)
+    model.eval()
+    return model
+
+
+def _load_lerobot(path: str) -> RobometerRewardModel:
+    cfg = RewardModelConfig.from_pretrained(path)
+    if not isinstance(cfg, RobometerConfig):
+        raise TypeError(f"Expected RobometerConfig, got {type(cfg)}")
+    cfg.pretrained_path = path
+    cfg.device = "cpu"
+    return RobometerRewardModel.from_pretrained(path, config=cfg)
+
+
+def _inspect_upstream_state_dict(upstream_path: str, model: RobometerRewardModel) -> None:
+    """Dump the upstream state-dict view of the embed/lm-head tensors.
+
+    Loads the raw upstream safetensors (pre-remap), runs the remapper, and
+    reports whether the embed/lm-head keys survive into the merged dict that
+    eventually hits ``model.load_state_dict``.
+    """
+    snapshot_dir = _download_robometer_snapshot(upstream_path)
+    files = _resolve_checkpoint_safetensors_files(snapshot_dir)
+    merged: dict[str, torch.Tensor] = {}
+    for path in files:
+        merged.update(load_file(str(path)))
+    remapped = _remap_state_dict_keys(merged, model)
+
+    print(f"\n=== Upstream state-dict inspection (snapshot at {snapshot_dir}) ===")
+    print(f"raw keys (before remap)  : {len(merged)}")
+    print(f"keys after remap         : {len(remapped)}")
+    print(f"model expects (state_dict): {len(model.state_dict())}")
+
+    expected = set(model.state_dict())
+    present_after_remap = set(remapped) & expected
+    print(f"keys present after remap : {len(present_after_remap)}")
+
+    missing_keys = expected - set(remapped)
+    print(f"keys missing from remap  : {len(missing_keys)}")
+    if missing_keys:
+        sample = list(missing_keys)[:10]
+        print(f"  sample missing keys    : {sample}")
+
+    unexpected_keys = set(remapped) - expected
+    print(f"keys unexpected by model : {len(unexpected_keys)}")
+    if unexpected_keys:
+        sample = list(unexpected_keys)[:10]
+        print(f"  sample unexpected keys : {sample}")
+
+    for key in (EMBED_KEY, LMHEAD_KEY):
+        present = key in remapped
+        shape = tuple(remapped[key].shape) if present else None
+        print(f"  {key:60s}  present={present}, shape={shape}")
+
+
+def _diff_embed(name: str, a: torch.Tensor, b: torch.Tensor, special_token_count: int) -> None:
+    a = a.float()
+    b = b.float()
+    if a.shape != b.shape:
+        print(f"❌ {name} shape mismatch: {tuple(a.shape)} vs {tuple(b.shape)}")
+        return
+
+    abs_diff = (a - b).abs()
+    per_row_max = abs_diff.max(dim=1).values
+    nz_rows = (per_row_max > 0).nonzero(as_tuple=True)[0].tolist()
+    print(f"\n=== {name} (shape {tuple(a.shape)}) ===")
+    print(f"global max|Δ|         = {abs_diff.max().item():.3e}")
+    print(f"rows with any diff    = {len(nz_rows)}")
+    if nz_rows:
+        first = nz_rows[:10]
+        last = nz_rows[-10:]
+        print(f"  first nonzero rows  = {first}")
+        print(f"  last nonzero rows   = {last}")
+        vocab_size = a.shape[0]
+        base_vocab = vocab_size - special_token_count
+        special_rows = list(range(base_vocab, vocab_size))
+        in_special = [r for r in nz_rows if r in special_rows]
+        out_special = [r for r in nz_rows if r not in special_rows]
+        print(
+            f"  diffs in special-token rows ({base_vocab}..{vocab_size - 1}): {len(in_special)}/{special_token_count}"
+        )
+        print(f"  diffs in base-vocab rows  (0..{base_vocab - 1})           : {len(out_special)}")
+        for r in special_rows:
+            print(
+                f"    row {r}: max|Δ|={per_row_max[r].item():.3e}, "
+                f"upstream_norm={a[r].norm().item():.3e}, lerobot_norm={b[r].norm().item():.3e}"
+            )
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+    parser.add_argument("--upstream", required=True)
+    parser.add_argument("--lerobot", required=True)
+    parser.add_argument(
+        "--special-token-count",
+        type=int,
+        default=5,
+        help="Number of special tokens Robometer adds. Defaults to len(ROBOMETER_SPECIAL_TOKENS)=5.",
+    )
+    args = parser.parse_args()
+
+    print(f"Loading upstream:        {args.upstream}")
+    upstream = _load_upstream(args.upstream)
+    print(f"Loading LeRobot-format:  {args.lerobot}")
+    lerobot = _load_lerobot(args.lerobot)
+
+    _inspect_upstream_state_dict(args.upstream, upstream)
+
+    sd_u, sd_l = upstream.state_dict(), lerobot.state_dict()
+
+    for key in (EMBED_KEY, LMHEAD_KEY):
+        if key not in sd_u or key not in sd_l:
+            print(f"❌ key missing: {key} (upstream={key in sd_u}, lerobot={key in sd_l})")
+            continue
+        _diff_embed(key, sd_u[key], sd_l[key], args.special_token_count)
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/extract_libero_episode_for_parity.py

@@ -0,0 +1,168 @@
+#!/usr/bin/env python
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+"""Extract one LIBERO episode for Robometer parity testing.
+
+Loads a LeRobot LIBERO (or any video-bearing LeRobot) dataset, picks one
+episode, samples ``--num-frames`` frames uniformly across its duration
+(matching upstream Robometer's default of 8 frames), and saves them to
+``.npz`` plus a sidecar ``.txt`` task file.
+
+The ``.npz`` layout (``frames`` key, ``(T, H, W, C) uint8``) is what upstream
+``example_inference_local.py`` consumes, so the same file feeds both pipelines
+and frame sampling cannot drift.
+
+Workflow:
+
+1. Run this script (LeRobot env) to produce ``frames.npz`` + ``task.txt``.
+2. Pass them to upstream ``scripts/example_inference_local.py``
+   (upstream env) to produce reference progress / success outputs.
+3. Pass the same ``frames.npz`` to ``scripts/parity_robometer.py``
+   (LeRobot env) to compare both sides.
+
+Example:
+
+    uv run python scripts/extract_libero_episode_for_parity.py \\
+        --repo-id lerobot/libero_10_image \\
+        --episode 0 \\
+        --num-frames 8 \\
+        --out-dir /tmp/libero_ep0
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+import numpy as np
+import torch
+
+from lerobot.configs.types import FeatureType
+from lerobot.datasets.lerobot_dataset import LeRobotDataset
+
+
+def _pick_visual_feature(features: dict, requested: str | None) -> str:
+    """Return a visual feature key, preferring ``requested`` when given."""
+    visual_keys = [
+        key
+        for key, ft in features.items()
+        if getattr(ft, "type", None) == FeatureType.VISUAL or ft.get("dtype", "") == "video"
+    ]
+    if not visual_keys:
+        raise ValueError(f"Dataset has no visual feature; available: {list(features)}")
+    if requested is not None:
+        if requested not in visual_keys:
+            raise ValueError(f"Camera key {requested!r} not in dataset visual features {visual_keys}")
+        return requested
+    return visual_keys[0]
+
+
+def _frame_uint8_hwc(tensor: torch.Tensor) -> np.ndarray:
+    """Convert a LeRobotDataset video frame to ``uint8`` ``(H, W, C)`` RGB."""
+    arr = tensor.detach().cpu().numpy()
+    if arr.ndim == 3 and arr.shape[0] in (1, 3):
+        arr = arr.transpose(1, 2, 0)
+    if arr.dtype != np.uint8:
+        arr = np.clip(arr * 255.0 if arr.max() <= 1.0 + 1e-3 else arr, 0, 255).astype(np.uint8)
+    if arr.shape[-1] == 1:
+        arr = np.repeat(arr, 3, axis=-1)
+    return arr
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description=__doc__,
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+    )
+    parser.add_argument(
+        "--repo-id",
+        default="lerobot/libero_10_image",
+        help="LeRobot LIBERO (or other) dataset repo id (default: lerobot/libero_10_image).",
+    )
+    parser.add_argument("--episode", type=int, default=0, help="Episode index.")
+    parser.add_argument(
+        "--camera-key",
+        default=None,
+        help="Visual feature key (e.g. observation.images.image). Auto-selects first if omitted.",
+    )
+    parser.add_argument(
+        "--num-frames",
+        type=int,
+        default=8,
+        help="Number of frames to sample uniformly (default: 8 — Robometer's training-time default).",
+    )
+    parser.add_argument(
+        "--out-dir",
+        type=Path,
+        default=Path("outputs/robometer_parity/libero"),
+        help="Directory to write frames.npz / task.txt / frame_indices.npy.",
+    )
+    args = parser.parse_args()
+
+    print(f"Loading {args.repo_id} (episode {args.episode})...")
+    dataset = LeRobotDataset(args.repo_id, episodes=[args.episode])
+
+    camera_key = _pick_visual_feature(dataset.features, args.camera_key)
+    print(f"Using camera key: {camera_key}")
+
+    ep_from = int(dataset.episode_data_index["from"][0].item())
+    ep_to = int(dataset.episode_data_index["to"][0].item())
+    total_frames = ep_to - ep_from
+    if total_frames <= 0:
+        print(f"ERROR: episode {args.episode} has no frames.", file=sys.stderr)
+        return 1
+    print(f"Episode has {total_frames} frames; sampling {args.num_frames} uniformly.")
+
+    indices = np.linspace(0, total_frames - 1, num=min(args.num_frames, total_frames), dtype=int)
+    frames: list[np.ndarray] = []
+    task: str = ""
+    for offset in indices:
+        sample = dataset[ep_from + int(offset)]
+        frame_tensor = sample[camera_key]
+        frames.append(_frame_uint8_hwc(frame_tensor))
+        if not task:
+            task = sample.get("task", "") or ""
+
+    if not task:
+        print("ERROR: episode has no task description in metadata.", file=sys.stderr)
+        return 1
+
+    frames_array = np.stack(frames)
+
+    args.out_dir.mkdir(parents=True, exist_ok=True)
+    frames_path = args.out_dir / "frames.npz"
+    task_path = args.out_dir / "task.txt"
+    indices_path = args.out_dir / "frame_indices.npy"
+
+    np.savez(frames_path, frames=frames_array)
+    task_path.write_text(task + "\n", encoding="utf-8")
+    np.save(indices_path, indices)
+
+    print()
+    print(f"Wrote {frames_path} (shape={frames_array.shape}, dtype={frames_array.dtype})")
+    print(f"Wrote {task_path}   (task={task!r})")
+    print(f"Wrote {indices_path} (frame_indices={indices.tolist()})")
+    print()
+    print("Next steps:")
+    print("  # in upstream env (where `robometer` is importable):")
+    print(
+        f"  python third_party/robometer/scripts/example_inference_local.py \\\n"
+        f"      --model-path robometer/Robometer-4B \\\n"
+        f"      --video {frames_path} \\\n"
+        f'      --task "{task}" \\\n'
+        f"      --out {args.out_dir / 'upstream.npy'}"
+    )
+    print()
+    print("  # back in LeRobot env:")
+    print(
+        f"  uv run python scripts/parity_robometer.py \\\n"
+        f"      --frames {frames_path} \\\n"
+        f'      --task "{task}" \\\n'
+        f"      --upstream-progress {args.out_dir / 'upstream.npy'} \\\n"
+        f"      --upstream-success  {args.out_dir / 'upstream_success_probs.npy'}"
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/parity_robometer.py

@@ -0,0 +1,232 @@
+#!/usr/bin/env python
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+"""Functional parity check: LeRobot Robometer vs. upstream Robometer.
+
+Runs the in-tree :class:`RobometerRewardModel` on the same frames + task that
+upstream Robometer was run on, and compares per-frame progress / success
+predictions against reference outputs saved by upstream's
+``scripts/example_inference_local.py``.
+
+Workflow:
+
+1. In the upstream Robometer environment (where ``robometer`` is importable),
+   run::
+
+       python third_party/robometer/scripts/example_inference_local.py \\
+           --model-path robometer/Robometer-4B \\
+           --video /path/to/episode.mp4 \\
+           --task "Open the drawer" \\
+           --fps 1.0 \\
+           --out /tmp/robometer_upstream.npy
+
+   This produces:
+   - ``/tmp/robometer_upstream.npy``               (progress predictions)
+   - ``/tmp/robometer_upstream_success_probs.npy`` (success probabilities)
+
+2. Extract the exact same frames the upstream script used, save as ``.npz``::
+
+       # quick helper: extract frames at the same fps and save as .npz
+       python -c "
+       from third_party.robometer.scripts.example_inference_local import load_frames_input
+       import numpy as np
+       frames = load_frames_input('/path/to/episode.mp4', fps=1.0, max_frames=512)
+       np.savez('/tmp/robometer_frames.npz', frames=frames)
+       "
+
+3. In this LeRobot env, run this script::
+
+       uv run python scripts/parity_robometer.py \\
+           --frames /tmp/robometer_frames.npz \\
+           --task "Open the drawer" \\
+           --upstream-progress /tmp/robometer_upstream.npy \\
+           --upstream-success  /tmp/robometer_upstream_success_probs.npy \\
+           --lerobot-model     lilkm/robometer-4b
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+
+import numpy as np
+import torch
+
+from lerobot.configs.rewards import RewardModelConfig
+from lerobot.rewards.robometer import RobometerConfig, RobometerRewardModel
+from lerobot.rewards.robometer.modeling_robometer import decode_progress_outputs
+from lerobot.rewards.robometer.processor_robometer import RobometerEncoderProcessorStep
+
+
+def _load_frames(path: str) -> np.ndarray:
+    """Load frames from .npy/.npz. Expects (T, H, W, C) uint8."""
+    if path.endswith(".npy"):
+        frames = np.load(path)
+    elif path.endswith(".npz"):
+        with np.load(path, allow_pickle=False) as npz:
+            frames = npz["frames"].copy() if "frames" in npz else next(iter(npz.values())).copy()
+    else:
+        raise ValueError(f"Frames must be .npy or .npz (got {path!r}).")
+
+    if frames.dtype != np.uint8:
+        frames = np.clip(frames, 0, 255).astype(np.uint8)
+    if frames.ndim != 4:
+        raise ValueError(f"Frames must be 4D (T,H,W,C); got shape {frames.shape}.")
+    if frames.shape[-1] not in (1, 3):
+        # Probably (T,C,H,W) — transpose
+        if frames.shape[1] in (1, 3):
+            frames = frames.transpose(0, 2, 3, 1)
+        else:
+            raise ValueError(f"Cannot interpret frame channel layout: {frames.shape}.")
+    return frames
+
+
+def _run_lerobot(
+    frames: np.ndarray,
+    task: str,
+    model_path: str,
+    device: str,
+) -> tuple[np.ndarray, np.ndarray]:
+    """Run LeRobot's Robometer on the given frames; return (progress, success)."""
+    cfg = RobometerConfig(pretrained_path=model_path, device=device, max_frames=None)
+    model = RobometerRewardModel.from_pretrained(model_path, config=cfg)
+
+    encoder = RobometerEncoderProcessorStep(
+        base_model_id=model.config.base_model_id,
+        use_multi_image=model.config.use_multi_image,
+        use_per_frame_progress_token=model.config.use_per_frame_progress_token,
+        max_frames=None,
+    )
+    batch = encoder.encode_samples([(frames, task)])
+
+    model_device = next(model.model.parameters()).device
+    inputs = {key: value.to(model_device) if hasattr(value, "to") else value for key, value in batch.items()}
+
+    model.eval()
+    with torch.no_grad():
+        progress_logits, success_logits = model._compute_rbm_logits(inputs)
+
+    decoded = decode_progress_outputs(
+        progress_logits,
+        success_logits,
+        is_discrete_mode=model.config.use_discrete_progress,
+    )
+    progress = np.asarray(decoded["progress_pred"][0], dtype=np.float32)
+    success = (
+        np.asarray(decoded["success_probs"][0], dtype=np.float32)
+        if decoded["success_probs"]
+        else np.array([], dtype=np.float32)
+    )
+    return progress, success
+
+
+def _compare(name: str, lerobot: np.ndarray, upstream: np.ndarray, atol: float, rtol: float) -> bool:
+    print(f"\n=== {name} ===")
+    if lerobot.shape != upstream.shape:
+        print(f"shape mismatch: lerobot={lerobot.shape}  upstream={upstream.shape}")
+        return False
+
+    abs_diff = np.abs(lerobot - upstream)
+    rel_diff = abs_diff / (np.abs(upstream) + 1e-12)
+    print(f"shape        : {lerobot.shape}")
+    print(f"max |Δ|      : {abs_diff.max():.3e}")
+    print(f"mean |Δ|     : {abs_diff.mean():.3e}")
+    print(f"max rel |Δ|  : {rel_diff.max():.3e}")
+    print(f"lerobot[:5]  : {lerobot[:5]}")
+    print(f"upstream[:5] : {upstream[:5]}")
+
+    within_tol = bool(np.allclose(lerobot, upstream, atol=atol, rtol=rtol))
+    print(f"allclose(atol={atol}, rtol={rtol}) -> {within_tol}")
+    return within_tol
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description=__doc__,
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+    )
+    parser.add_argument(
+        "--frames",
+        required=True,
+        help=".npy / .npz file with the exact frames upstream was run on (T,H,W,C uint8).",
+    )
+    parser.add_argument("--task", required=True, help="Task instruction string.")
+    parser.add_argument(
+        "--upstream-progress",
+        required=True,
+        help="Reference progress .npy saved by upstream example_inference_local.py.",
+    )
+    parser.add_argument(
+        "--upstream-success",
+        default=None,
+        help="Optional reference success_probs .npy. If omitted, success comparison is skipped.",
+    )
+    parser.add_argument(
+        "--lerobot-model",
+        default="lilkm/robometer-4b",
+        help="LeRobot-format Robometer Hub repo id or local path.",
+    )
+    parser.add_argument(
+        "--device",
+        default="cuda" if torch.cuda.is_available() else "cpu",
+        help="Device for the LeRobot model (default: cuda if available).",
+    )
+    parser.add_argument(
+        "--atol",
+        type=float,
+        default=1e-3,
+        help="Absolute tolerance for allclose (default: 1e-3; bf16 round-trip headroom).",
+    )
+    parser.add_argument(
+        "--rtol",
+        type=float,
+        default=1e-2,
+        help="Relative tolerance for allclose (default: 1e-2).",
+    )
+    parser.add_argument(
+        "--out-prefix",
+        default="lerobot_robometer_outputs",
+        help="Save the LeRobot outputs as <prefix>_progress.npy / <prefix>_success.npy.",
+    )
+    args = parser.parse_args()
+
+    # 0. Sanity: confirm the LeRobot config is a RobometerConfig.
+    cfg = RewardModelConfig.from_pretrained(args.lerobot_model)
+    if not isinstance(cfg, RobometerConfig):
+        print(f"ERROR: {args.lerobot_model!r} does not resolve to a RobometerConfig.", file=sys.stderr)
+        return 2
+
+    # 1. Load frames + task + upstream reference outputs.
+    frames = _load_frames(args.frames)
+    upstream_progress = np.load(args.upstream_progress).astype(np.float32)
+    upstream_success = (
+        np.load(args.upstream_success).astype(np.float32) if args.upstream_success is not None else None
+    )
+
+    print(f"Loaded {frames.shape[0]} frames at {frames.shape[1:]}, task={args.task!r}")
+    print(f"LeRobot model: {args.lerobot_model}  device: {args.device}")
+
+    # 2. Run LeRobot pipeline.
+    progress, success = _run_lerobot(frames, args.task, args.lerobot_model, args.device)
+    np.save(f"{args.out_prefix}_progress.npy", progress)
+    if success.size > 0:
+        np.save(f"{args.out_prefix}_success.npy", success)
+    print(f"Saved LeRobot outputs to {args.out_prefix}_progress.npy / _success.npy")
+
+    # 3. Compare to upstream references.
+    progress_ok = _compare("progress", progress, upstream_progress, args.atol, args.rtol)
+    if upstream_success is not None and success.size > 0:
+        success_ok = _compare("success_probs", success, upstream_success, args.atol, args.rtol)
+    else:
+        success_ok = True
+        print("\n(skipping success comparison — upstream success file not provided)")
+
+    print()
+    if progress_ok and success_ok:
+        print("Parity check passed.")
+        return 0
+    print("Parity check FAILED.")
+    return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/parity_robometer_upstream_examples.py

@@ -0,0 +1,362 @@
+#!/usr/bin/env python
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+"""Run LeRobot Robometer parity against upstream Robometer's bundled examples.
+
+Upstream Robometer ships three reference videos with their pre-computed
+progress / success outputs at
+``third_party/robometer/scripts/example_videos/``::
+
+    soar_put_green_stick_in_brown_bowl.mp4
+        + soar_put_green_stick_in_brown_bowl_rewards.npy            (progress)
+        + soar_put_green_stick_in_brown_bowl_rewards_success_probs.npy (success)
+    berkeley_rpt_stack_cup.mp4
+        + berkeley_rpt_stack_cup_rewards.npy
+        + berkeley_rpt_stack_cup_rewards_success_probs.npy
+    jaco_play_pick_up_green_cup.mp4
+        + pick_up_green_cup_rewards.npy
+        + pick_up_green_cup_rewards_success_probs.npy
+
+This script:
+1. Decodes each video at upstream's sampling fps using ``av`` (PyAV), with the
+   same linspace-over-total-frames logic as upstream's ``extract_frames``.
+2. Runs the LeRobot ``RobometerRewardModel`` on those frames + the task from
+   upstream's README.
+3. Compares per-frame progress / success to the pre-saved upstream outputs.
+
+This means you do **not** need to install upstream Robometer to confirm parity.
+
+Run::
+
+    uv run python scripts/parity_robometer_upstream_examples.py \\
+        --lerobot-model lilkm/robometer-4b \\
+        --device cuda \\
+        --decoder decord
+
+The number of frames sampled per video is derived from the length of each
+upstream ``.npy`` reference, so the script does not need a ``--fps`` argument
+(the README documents ``fps=3`` for SOAR / Berkeley, but the Jaco Play
+reference was generated with a different fps).
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+import numpy as np
+import torch
+
+from lerobot.configs.rewards import RewardModelConfig
+from lerobot.rewards.robometer import RobometerConfig, RobometerRewardModel
+from lerobot.rewards.robometer.modeling_robometer import decode_progress_outputs
+from lerobot.rewards.robometer.processor_robometer import RobometerEncoderProcessorStep
+
+try:
+    import decord  # type: ignore
+
+    _HAS_DECORD = True
+except ImportError:
+    decord = None  # type: ignore
+    _HAS_DECORD = False
+
+try:
+    import av
+
+    _HAS_AV = True
+except ImportError:
+    av = None  # type: ignore
+    _HAS_AV = False
+
+EXAMPLES = [
+    {
+        "name": "soar_put_green_stick_in_brown_bowl",
+        "video": "soar_put_green_stick_in_brown_bowl.mp4",
+        "task": "Put green stick in brown bowl",
+        "progress_npy": "soar_put_green_stick_in_brown_bowl_rewards.npy",
+        "success_npy": "soar_put_green_stick_in_brown_bowl_rewards_success_probs.npy",
+    },
+    {
+        "name": "berkeley_rpt_stack_cup",
+        "video": "berkeley_rpt_stack_cup.mp4",
+        "task": "Pick up the yellow cup and stack it on the other cup",
+        "progress_npy": "berkeley_rpt_stack_cup_rewards.npy",
+        "success_npy": "berkeley_rpt_stack_cup_rewards_success_probs.npy",
+    },
+    {
+        "name": "jaco_play_pick_up_green_cup",
+        "video": "jaco_play_pick_up_green_cup.mp4",
+        "task": "Pick up the green cup",
+        "progress_npy": "pick_up_green_cup_rewards.npy",
+        "success_npy": "pick_up_green_cup_rewards_success_probs.npy",
+    },
+]
+
+
+def _extract_frames_decord(video_path: Path, num_frames: int) -> tuple[np.ndarray, str]:
+    """Sample ``num_frames`` indices uniformly from the video using decord.
+
+    Mirrors upstream's ``extract_frames`` indexing
+    (``third_party/robometer/scripts/example_inference.py``): a
+    ``np.linspace(0, total_frames-1, num_frames)`` lookup over decord's
+    ``VideoReader``. We pass ``num_frames`` explicitly (derived from the
+    upstream reference output length) so we don't have to guess what ``fps``
+    upstream actually used when generating each saved ``.npy`` — the file
+    length is the ground truth.
+    """
+    vr = decord.VideoReader(str(video_path), num_threads=1)
+    total_frames = len(vr)
+    if total_frames == 0:
+        raise RuntimeError(f"No decodable frames in {video_path}.")
+    desired_frames = max(1, min(int(num_frames), total_frames))
+    indices = np.linspace(0, total_frames - 1, desired_frames, dtype=int).tolist()
+    frames = vr.get_batch(indices).asnumpy()
+    native_fps = float(vr.get_avg_fps()) or 1.0
+    return frames, f"decord total={total_frames} native_fps={native_fps:.3f}"
+
+
+def _extract_frames_av(video_path: Path, num_frames: int) -> tuple[np.ndarray, str]:
+    """PyAV fallback for environments without decord.
+
+    PyAV and decord can disagree on ``total_frames`` for the same container,
+    so the sampled frame indices can drift. Install ``decord`` for a real
+    parity check; this fallback is for smoke tests only.
+    """
+    container = av.open(str(video_path))
+    stream = container.streams.video[0]
+    native_fps = float(stream.average_rate) if stream.average_rate else float(stream.guessed_rate or 30.0)
+    rgb_frames: list[np.ndarray] = []
+    for frame in container.decode(stream):
+        rgb_frames.append(frame.to_ndarray(format="rgb24"))
+    container.close()
+    total_frames = len(rgb_frames)
+    if total_frames == 0:
+        raise RuntimeError(f"No decodable frames in {video_path}.")
+    desired_frames = max(1, min(int(num_frames), total_frames))
+    indices = np.linspace(0, total_frames - 1, desired_frames, dtype=int)
+    frames = np.stack([rgb_frames[i] for i in indices])
+    return frames, f"av total={total_frames} native_fps={native_fps:.3f}"
+
+
+def _extract_frames(video_path: Path, num_frames: int, prefer: str) -> tuple[np.ndarray, str]:
+    """Decoder dispatch. ``prefer`` is ``"decord"`` | ``"av"`` | ``"auto"``."""
+    if prefer == "decord":
+        if not _HAS_DECORD:
+            raise RuntimeError("decord requested but not installed (`uv pip install decord`).")
+        return _extract_frames_decord(video_path, num_frames)
+    if prefer == "av":
+        if not _HAS_AV:
+            raise RuntimeError("av requested but not installed.")
+        return _extract_frames_av(video_path, num_frames)
+    # auto
+    if _HAS_DECORD:
+        return _extract_frames_decord(video_path, num_frames)
+    if _HAS_AV:
+        return _extract_frames_av(video_path, num_frames)
+    raise RuntimeError("No video decoder available (install `decord` or `av`).")
+
+
+def _pearson(a: np.ndarray, b: np.ndarray) -> float:
+    """Pearson correlation; returns 1.0 for constant inputs (no signal to align)."""
+    a = a.astype(np.float64)
+    b = b.astype(np.float64)
+    if a.size < 2:
+        return 1.0
+    da = a - a.mean()
+    db = b - b.mean()
+    denom = float(np.sqrt((da * da).sum()) * np.sqrt((db * db).sum()))
+    if denom == 0:
+        return 1.0
+    return float((da * db).sum() / denom)
+
+
+def _run_lerobot(
+    model: RobometerRewardModel,
+    encoder: RobometerEncoderProcessorStep,
+    frames: np.ndarray,
+    task: str,
+) -> tuple[np.ndarray, np.ndarray]:
+    batch = encoder.encode_samples([(frames, task)])
+    device = next(model.model.parameters()).device
+    inputs = {key: value.to(device) if hasattr(value, "to") else value for key, value in batch.items()}
+    model.eval()
+    with torch.no_grad():
+        progress_logits, success_logits = model._compute_rbm_logits(inputs)
+    decoded = decode_progress_outputs(
+        progress_logits, success_logits, is_discrete_mode=model.config.use_discrete_progress
+    )
+    progress = np.asarray(decoded["progress_pred"][0], dtype=np.float32)
+    success = (
+        np.asarray(decoded["success_probs"][0], dtype=np.float32)
+        if decoded["success_probs"]
+        else np.array([], dtype=np.float32)
+    )
+    return progress, success
+
+
+def _compare(
+    name: str,
+    lerobot: np.ndarray,
+    upstream: np.ndarray,
+    *,
+    atol: float,
+    pearson_min: float,
+) -> bool:
+    if lerobot.shape != upstream.shape:
+        print(f"  {name:8s}  SHAPE MISMATCH lerobot={lerobot.shape} upstream={upstream.shape}")
+        return False
+    abs_diff = np.abs(lerobot - upstream)
+    pearson = _pearson(lerobot, upstream)
+    abs_ok = bool(abs_diff.max() <= atol)
+    pearson_ok = bool(pearson >= pearson_min)
+    verdict = "PASS" if (abs_ok or pearson_ok) else "FAIL"
+    print(
+        f"  {name:8s}  shape={lerobot.shape}  max|Δ|={abs_diff.max():.3e}  "
+        f"mean|Δ|={abs_diff.mean():.3e}  pearson={pearson:.4f}  "
+        f"(atol={atol:.0e} pearson_min={pearson_min:.3f}) -> {verdict}"
+    )
+    return abs_ok or pearson_ok
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description=__doc__,
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+    )
+    parser.add_argument(
+        "--examples-dir",
+        type=Path,
+        default=Path("third_party/robometer/scripts/example_videos"),
+        help="Directory containing the upstream Robometer example mp4s + .npy outputs.",
+    )
+    parser.add_argument(
+        "--lerobot-model",
+        default="lilkm/robometer-4b",
+        help="LeRobot-format Robometer Hub repo id or local path.",
+    )
+    parser.add_argument(
+        "--device",
+        default="cuda" if torch.cuda.is_available() else "cpu",
+        help="Device for the LeRobot model.",
+    )
+    parser.add_argument(
+        "--decoder",
+        choices=("auto", "decord", "av"),
+        default="auto",
+        help=(
+            "Video decoder. ``auto`` prefers decord (matches upstream) and falls back to av. "
+            "Force ``decord`` for a clean parity check."
+        ),
+    )
+    parser.add_argument(
+        "--progress-atol",
+        type=float,
+        default=1e-2,
+        help="Absolute tolerance for the progress array. Default 1e-2 covers CUDA bf16 noise.",
+    )
+    parser.add_argument(
+        "--success-atol",
+        type=float,
+        default=1e-1,
+        help=(
+            "Absolute tolerance for the success array. Looser than progress because "
+            "``sigmoid`` amplifies logit-space noise near 0.5."
+        ),
+    )
+    parser.add_argument(
+        "--pearson-min",
+        type=float,
+        default=0.99,
+        help="Minimum Pearson correlation for a PASS verdict (per array).",
+    )
+    args = parser.parse_args()
+
+    if args.decoder == "av" or (args.decoder == "auto" and not _HAS_DECORD):
+        print(
+            "WARNING: using PyAV decoder. PyAV's total-frame count can differ from decord's, "
+            "which propagates into different sampled-frame indices. Install `decord` and "
+            "re-run for a clean parity check.",
+            file=sys.stderr,
+        )
+
+    examples_dir = args.examples_dir.resolve()
+    if not examples_dir.is_dir():
+        print(f"ERROR: examples dir {examples_dir} does not exist.", file=sys.stderr)
+        return 2
+
+    # Sanity-check the LeRobot config is a RobometerConfig before loading weights.
+    cfg = RewardModelConfig.from_pretrained(args.lerobot_model)
+    if not isinstance(cfg, RobometerConfig):
+        print(f"ERROR: {args.lerobot_model!r} did not resolve to a RobometerConfig.", file=sys.stderr)
+        return 2
+
+    print(f"Loading LeRobot Robometer from {args.lerobot_model} on {args.device}...")
+    cfg.pretrained_path = args.lerobot_model
+    cfg.device = args.device
+    model = RobometerRewardModel.from_pretrained(args.lerobot_model, config=cfg)
+    encoder = RobometerEncoderProcessorStep(
+        base_model_id=model.config.base_model_id,
+        use_multi_image=model.config.use_multi_image,
+        use_per_frame_progress_token=model.config.use_per_frame_progress_token,
+        max_frames=None,
+    )
+
+    all_ok = True
+    for ex in EXAMPLES:
+        video_path = examples_dir / ex["video"]
+        upstream_progress_path = examples_dir / ex["progress_npy"]
+        upstream_success_path = examples_dir / ex["success_npy"]
+
+        missing = [p for p in (video_path, upstream_progress_path, upstream_success_path) if not p.exists()]
+        if missing:
+            print(f"[skip] {ex['name']}: missing {[str(m) for m in missing]}")
+            all_ok = False
+            continue
+
+        print(f"\n=== {ex['name']} ===")
+        print(f"  task: {ex['task']!r}")
+
+        # Trust the upstream reference array as the source of truth for how
+        # many frames to sample. The README documents fps=3 for SOAR/Berkeley
+        # but Jaco Play was generated with a different fps, so any hardcoded
+        # ``--fps`` mismatches at least one example. The npy length always
+        # tells us what upstream actually used.
+        upstream_progress = np.load(upstream_progress_path).astype(np.float32)
+        upstream_success = np.load(upstream_success_path).astype(np.float32)
+        target_num_frames = int(upstream_progress.shape[0])
+        frames, decoder_info = _extract_frames(video_path, target_num_frames, prefer=args.decoder)
+        print(
+            f"  decoded {frames.shape[0]} frames (matches upstream npy length); "
+            f"shape={frames.shape}  [{decoder_info}]"
+        )
+
+        progress, success = _run_lerobot(model, encoder, frames, ex["task"])
+
+        progress_ok = _compare(
+            "progress",
+            progress,
+            upstream_progress,
+            atol=args.progress_atol,
+            pearson_min=args.pearson_min,
+        )
+        success_ok = _compare(
+            "success",
+            success,
+            upstream_success,
+            atol=args.success_atol,
+            pearson_min=args.pearson_min,
+        )
+        verdict = "PASS" if (progress_ok and success_ok) else "FAIL"
+        print(f"  -> {verdict}")
+        all_ok = all_ok and progress_ok and success_ok
+
+    print()
+    if all_ok:
+        print("All upstream example parity checks passed.")
+        return 0
+    print("Some upstream example parity checks FAILED.")
+    return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/verify_robometer_export.py

@@ -0,0 +1,149 @@
+#!/usr/bin/env python
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+"""Verify that a LeRobot-format Robometer is byte-equivalent to its upstream source.
+
+Run this once after publishing a LeRobot-format Robometer to the Hub, before
+flipping the default `RobometerConfig.pretrained_path` to it. It loads both
+the upstream snapshot and the re-exported copy, compares state dicts, and
+prints a clear pass/fail summary.
+
+Example:
+
+    python scripts/verify_robometer_export.py \\
+        --upstream robometer/Robometer-4B \\
+        --lerobot  lerobot/robometer-4b
+
+    python scripts/verify_robometer_export.py \\
+        --upstream robometer/Robometer-4B \\
+        --lerobot  ./robometer-4b-lerobot   # local folder also works
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+
+from lerobot.configs.rewards import RewardModelConfig
+from lerobot.rewards.robometer import RobometerConfig, RobometerRewardModel
+from lerobot.rewards.robometer._upstream_loader import apply_upstream_checkpoint
+
+
+def _load_upstream(path: str) -> RobometerRewardModel:
+    # Fresh ``RobometerConfig`` (``vlm_config=None``) triggers
+    # ``RobometerRewardModel.__init__``'s upstream-matching path: download
+    # base Qwen, resize for ROBOMETER_SPECIAL_TOKENS. The subsequent
+    # ``apply_upstream_checkpoint`` call resizes again if the checkpoint's
+    # vocab differs (e.g. upstream was trained against an older Qwen).
+    cfg = RobometerConfig(pretrained_path=path, device="cpu")
+    model = RobometerRewardModel(cfg)
+    apply_upstream_checkpoint(model, path)
+    model.eval()
+    return model
+
+
+def _load_lerobot(path: str) -> RobometerRewardModel:
+    cfg = RewardModelConfig.from_pretrained(path)
+    if not isinstance(cfg, RobometerConfig):
+        raise TypeError(f"Expected RobometerConfig in LeRobot export, got {type(cfg)}")
+    cfg.pretrained_path = path
+    cfg.device = "cpu"
+    return RobometerRewardModel.from_pretrained(path, config=cfg)
+
+
+def compare_state_dicts(a: RobometerRewardModel, b: RobometerRewardModel) -> bool:
+    sd_a, sd_b = a.state_dict(), b.state_dict()
+    keys_a, keys_b = set(sd_a), set(sd_b)
+
+    missing = keys_a - keys_b
+    extra = keys_b - keys_a
+    if missing:
+        print(f"❌ {len(missing)} keys missing in LeRobot-format model (sample: {list(missing)[:5]})")
+    if extra:
+        print(f"❌ {len(extra)} extra keys in LeRobot-format model (sample: {list(extra)[:5]})")
+    if missing or extra:
+        return False
+
+    diff_summary: list[tuple[str, float]] = []
+    for key in sorted(keys_a):
+        ta, tb = sd_a[key], sd_b[key]
+        if ta.shape != tb.shape:
+            print(f"❌ shape mismatch at {key}: {tuple(ta.shape)} vs {tuple(tb.shape)}")
+            return False
+        # Compare in float to avoid bfloat16 equality quirks.
+        max_abs = (ta.float() - tb.float()).abs().max().item()
+        if max_abs > 0:
+            diff_summary.append((key, max_abs))
+
+    if not diff_summary:
+        print(f"✅ All {len(keys_a)} parameters identical")
+        return True
+
+    # Some keys differ; show worst offenders.
+    diff_summary.sort(key=lambda kv: kv[1], reverse=True)
+    print(f"⚠️  {len(diff_summary)} keys differ. Top 10 by max abs diff:")
+    for key, value in diff_summary[:10]:
+        print(f"    {key:60s}  max|Δ| = {value:.3e}")
+
+    # Tolerance: bf16 round-trips can introduce ULP-level noise but no real
+    # change. Allow up to 1e-3 absolute difference; anything larger is a real
+    # divergence.
+    worst = diff_summary[0][1]
+    if worst < 1e-3:
+        print(f"✅ Worst diff {worst:.3e} is within bf16 round-trip tolerance")
+        return True
+    print(f"❌ Worst diff {worst:.3e} exceeds tolerance (1e-3)")
+    return False
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(
+        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+    parser.add_argument("--upstream", required=True, help="Upstream Robometer repo id or local path.")
+    parser.add_argument("--lerobot", required=True, help="LeRobot-format Robometer repo id or local path.")
+    args = parser.parse_args()
+
+    print(f"Loading upstream:        {args.upstream}")
+    upstream = _load_upstream(args.upstream)
+    print(f"Loading LeRobot-format:  {args.lerobot}")
+    lerobot = _load_lerobot(args.lerobot)
+
+    print("\n=== Config comparison ===")
+    config_ok = True
+    for field in [
+        "base_model_id",
+        "torch_dtype",
+        "use_multi_image",
+        "use_per_frame_progress_token",
+        "average_temporal_patches",
+        "frame_pooling",
+        "frame_pooling_attn_temperature",
+        "progress_loss_type",
+        "progress_discrete_bins",
+    ]:
+        a, b = getattr(upstream.config, field), getattr(lerobot.config, field)
+        field_ok = a == b
+        config_ok = config_ok and field_ok
+        ok = "✅" if field_ok else "❌"
+        print(f"  {ok} {field}: upstream={a!r}, lerobot={b!r}")
+
+    print("\n=== State-dict comparison ===")
+    state_dict_ok = compare_state_dicts(upstream, lerobot)
+
+    print()
+    if config_ok and state_dict_ok:
+        print("🎉 Verification passed — safe to flip the default.")
+        return 0
+    print("⛔ Verification failed — DO NOT flip the default.")
+    return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

src/lerobot/cameras/opencv/camera_opencv.py

@@ -199,13 +199,12 @@ class OpenCVCamera(Camera):
             DeviceNotConnectedError: If the camera is not connected.
         """
 
+        # Set FOURCC first (if specified) as it can affect available FPS/resolution options
+        if self.config.fourcc is not None:
+            self._validate_fourcc()
         if self.videocapture is None:
             raise DeviceNotConnectedError(f"{self} videocapture is not initialized")
 
-        set_fourcc_after_size_and_fps = platform.system() == "Windows"
-        if self.config.fourcc is not None and not set_fourcc_after_size_and_fps:
-            self._validate_fourcc()
-
         default_width = int(round(self.videocapture.get(cv2.CAP_PROP_FRAME_WIDTH)))
         default_height = int(round(self.videocapture.get(cv2.CAP_PROP_FRAME_HEIGHT)))
 
@@ -223,11 +222,6 @@ class OpenCVCamera(Camera):
         else:
             self._validate_fps()
 
-        if self.config.fourcc is not None and set_fourcc_after_size_and_fps:
-            # On Windows with DSHOW, changing the resolution can silently override the FOURCC setting.
-            # Set FOURCC last to make sure the requested pixel format is actually enforced.
-            self._validate_fourcc()
-
     def _validate_fps(self) -> None:
         """Validates and sets the camera's frames per second (FPS)."""
 

src/lerobot/configs/__init__.py

@@ -24,7 +24,6 @@ Import them directly: ``from lerobot.configs.train import TrainPipelineConfig``
 from .dataset import DatasetRecordConfig
 from .default import DatasetConfig, EvalConfig, PeftConfig, WandBConfig
 from .policies import PreTrainedConfig
-from .recipe import MessageTurn, TrainingRecipe, load_recipe
 from .types import (
     FeatureType,
     NormalizationMode,
@@ -32,12 +31,6 @@ from .types import (
     PolicyFeature,
     RTCAttentionSchedule,
 )
-from .video import (
-    VALID_VIDEO_CODECS,
-    VIDEO_ENCODER_INFO_KEYS,
-    VideoEncoderConfig,
-    camera_encoder_defaults,
-)
 
 __all__ = [
     # Types
@@ -50,16 +43,7 @@ __all__ = [
     "DatasetRecordConfig",
     "DatasetConfig",
     "EvalConfig",
-    "MessageTurn",
     "PeftConfig",
     "PreTrainedConfig",
-    "TrainingRecipe",
     "WandBConfig",
-    "load_recipe",
-    "VideoEncoderConfig",
-    # Defaults
-    "camera_encoder_defaults",
-    # Constants
-    "VALID_VIDEO_CODECS",
-    "VIDEO_ENCODER_INFO_KEYS",
 ]

src/lerobot/configs/dataset.py

@@ -14,12 +14,10 @@
 
 """Shared dataset recording configuration used by both ``lerobot-record`` and ``lerobot-rollout``."""
 
-from dataclasses import dataclass, field
+from dataclasses import dataclass
 from datetime import datetime
 from pathlib import Path
 
-from .video import VideoEncoderConfig, camera_encoder_defaults
-
 
 @dataclass
 class DatasetRecordConfig:
@@ -57,9 +55,10 @@ class DatasetRecordConfig:
     # Number of episodes to record before batch encoding videos
     # Set to 1 for immediate encoding (default behavior), or higher for batched encoding
     video_encoding_batch_size: int = 1
-    # Video encoder settings for camera MP4s (codec, quality, GOP, etc.). Tuned via CLI nested keys,
-    # e.g. ``--dataset.camera_encoder.vcodec=h264`` (see ``VideoEncoderConfig``).
-    camera_encoder: VideoEncoderConfig = field(default_factory=camera_encoder_defaults)
+    # Video codec for encoding videos. Options: 'h264', 'hevc', 'libsvtav1', 'auto',
+    # or hardware-specific: 'h264_videotoolbox', 'h264_nvenc', 'h264_vaapi', 'h264_qsv'.
+    # Use 'auto' to auto-detect the best available hardware encoder.
+    vcodec: str = "libsvtav1"
     # Enable streaming video encoding: encode frames in real-time during capture instead
     # of writing PNG images first. Makes save_episode() near-instant. More info in the documentation: https://huggingface.co/docs/lerobot/streaming_video_encoding
     streaming_encoding: bool = False

src/lerobot/configs/default.py

@@ -17,7 +17,7 @@
 from dataclasses import dataclass, field
 
 from lerobot.transforms import ImageTransformsConfig
-from lerobot.utils.import_utils import get_safe_default_video_backend
+from lerobot.utils.import_utils import get_safe_default_codec
 
 
 @dataclass
@@ -34,7 +34,7 @@ class DatasetConfig:
     image_transforms: ImageTransformsConfig = field(default_factory=ImageTransformsConfig)
     revision: str | None = None
     use_imagenet_stats: bool = True
-    video_backend: str = field(default_factory=get_safe_default_video_backend)
+    video_backend: str = field(default_factory=get_safe_default_codec)
     # When True, video frames are returned as uint8 tensors (0-255) instead of float32 (0.0-1.0).
     # This reduces memory and speeds up DataLoader IPC. The training pipeline handles the conversion.
     return_uint8: bool = False

src/lerobot/configs/eval.py

@@ -18,8 +18,8 @@ from logging import getLogger
 from pathlib import Path
 
 from lerobot import envs, policies  # noqa: F401
+from lerobot.configs import parser
 
-from . import parser
 from .default import EvalConfig
 from .policies import PreTrainedConfig
 

src/lerobot/configs/recipe.py

@@ -1,206 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from __future__ import annotations
-
-import re
-from dataclasses import dataclass
-from pathlib import Path
-from typing import Any, Literal, get_args
-
-MessageRole = Literal["user", "assistant", "system", "tool"]
-MessageStream = Literal["high_level", "low_level"]
-
-DEFAULT_BINDINGS = {
-    "subtask": "active_at(t, style=subtask)",
-    "memory": "active_at(t, style=memory)",
-    "plan": "active_at(t, style=plan)",
-    "speech": "emitted_at(t, role=assistant, tool_name=say)",
-    "interjection": "emitted_at(t, style=interjection)",
-    "vqa": "emitted_at(t, style=vqa, role=assistant)",
-    "vqa_query": "emitted_at(t, style=vqa, role=user)",
-}
-
-PLACEHOLDER_RE = re.compile(r"\$\{([A-Za-z_][A-Za-z0-9_]*)\}")
-"""``${name}`` placeholder pattern used by both recipe binding-reference
-discovery (here) and rendered-message substitution (in ``language_render``)."""
-
-_VALID_ROLES = frozenset(get_args(MessageRole))
-_VALID_STREAMS = frozenset(get_args(MessageStream))
-
-
-@dataclass
-class MessageTurn:
-    """A single chat-style turn in a recipe template.
-
-    ``content`` may be a plain string, a list of HF-style multimodal blocks, or
-    ``None`` when ``tool_calls_from`` supplies tool-call payloads instead.
-    ``stream`` tags the turn for downstream filtering, ``target`` flags it as a
-    training target, and ``if_present`` skips the turn when the named binding
-    resolves to ``None``.
-    """
-
-    role: MessageRole
-    content: str | list[dict[str, Any]] | None = None
-    stream: MessageStream | None = None
-    target: bool = False
-    if_present: str | None = None
-    tool_calls_from: str | None = None
-
-    def __post_init__(self) -> None:
-        """Validate role, stream, and content after dataclass construction."""
-        if self.role not in _VALID_ROLES:
-            raise ValueError(f"Unsupported message role: {self.role!r}")
-        # ``stream`` is typed Optional only so the dataclass can keep its
-        # field ordering, but recipes must always tag every turn with a
-        # stream — the renderer's ``_validate_rendered`` would reject
-        # ``None`` later on. Fail at construction so the bad recipe is
-        # caught at YAML load time rather than at the first sample.
-        if self.stream is None:
-            raise ValueError(
-                f"MessageTurn(role={self.role!r}) is missing a stream — "
-                f"every turn must declare one of {sorted(_VALID_STREAMS)}."
-            )
-        if self.stream not in _VALID_STREAMS:
-            raise ValueError(f"Unsupported message stream: {self.stream!r}")
-        if self.content is None and self.tool_calls_from is None:
-            raise ValueError("MessageTurn.content is required unless tool_calls_from is set.")
-        if self.content is not None and not isinstance(self.content, (str, list)):
-            raise TypeError("MessageTurn.content must be a string, a list of HF-style blocks, or None.")
-        if isinstance(self.content, list):
-            for block in self.content:
-                if not isinstance(block, dict) or "type" not in block:
-                    raise ValueError(
-                        "Multimodal content blocks must be HF-style dictionaries with a type key."
-                    )
-
-    @classmethod
-    def from_dict(cls, data: dict[str, Any]) -> MessageTurn:
-        """Construct a :class:`MessageTurn` from a plain dictionary."""
-        return cls(**data)
-
-
-@dataclass
-class TrainingRecipe:
-    """A recipe describing how to render training samples from language rows.
-
-    A recipe is either a *message recipe* (``messages`` plus optional
-    ``bindings``) or a *blend recipe* (``blend`` mapping names to weighted
-    sub-recipes). ``weight`` is only meaningful inside a blend.
-    """
-
-    messages: list[MessageTurn] | None = None
-    bindings: dict[str, str] | None = None
-    blend: dict[str, TrainingRecipe] | None = None
-    weight: float | None = None
-
-    def __post_init__(self) -> None:
-        """Validate that exactly one of ``messages`` or ``blend`` is set."""
-        if self.messages is not None and self.blend is not None:
-            raise ValueError("TrainingRecipe must set only one of messages or blend.")
-        if self.messages is None and self.blend is None:
-            raise ValueError("TrainingRecipe must set one of messages or blend.")
-
-        if self.messages is not None:
-            self._validate_message_recipe()
-        if self.blend is not None:
-            self._validate_blend_recipe()
-
-    @classmethod
-    def from_dict(cls, data: dict[str, Any]) -> TrainingRecipe:
-        """Construct a :class:`TrainingRecipe` from a nested dictionary."""
-        data = dict(data)
-        if data.get("messages") is not None:
-            data["messages"] = [
-                turn if isinstance(turn, MessageTurn) else MessageTurn.from_dict(turn)
-                for turn in data["messages"]
-            ]
-        if data.get("blend") is not None:
-            data["blend"] = {
-                name: recipe if isinstance(recipe, TrainingRecipe) else cls.from_dict(recipe)
-                for name, recipe in data["blend"].items()
-            }
-        return cls(**data)
-
-    @classmethod
-    def from_yaml(cls, path: str | Path) -> TrainingRecipe:
-        """Load a :class:`TrainingRecipe` from a YAML file at ``path``."""
-        import yaml  # type: ignore[import-untyped]
-
-        with open(path) as f:
-            data = yaml.safe_load(f)
-        if not isinstance(data, dict):
-            raise ValueError(f"Recipe YAML must contain a mapping at the top level: {path}")
-        return cls.from_dict(data)
-
-    def _validate_message_recipe(self) -> None:
-        """Ensure every templated binding is known and at least one turn is a target."""
-        assert self.messages is not None
-        known_bindings = set(DEFAULT_BINDINGS) | set(self.bindings or {}) | {"task"}
-
-        for turn in self.messages:
-            missing = self._referenced_bindings(turn) - known_bindings
-            if missing:
-                raise ValueError(f"MessageTurn references unknown binding(s): {sorted(missing)}")
-
-        if not any(turn.target for turn in self.messages):
-            raise ValueError("Message recipes must contain at least one target turn.")
-
-    def _validate_blend_recipe(self) -> None:
-        """Ensure each blend component is a non-empty, weighted message recipe."""
-        assert self.blend is not None
-        if not self.blend:
-            raise ValueError("Blend recipes must contain at least one component.")
-
-        for name, recipe in self.blend.items():
-            if recipe.blend is not None:
-                raise ValueError(f"Blend component {name!r} cannot itself define a blend.")
-            if recipe.messages is None:
-                raise ValueError(f"Blend component {name!r} must define messages.")
-            if recipe.weight is None:
-                raise ValueError(f"Blend component {name!r} must define weight.")
-            if recipe.weight <= 0:
-                raise ValueError(f"Blend component {name!r} must have a positive weight.")
-
-    def _referenced_bindings(self, turn: MessageTurn) -> set[str]:
-        """Return the binding names that ``turn`` references via placeholders or attributes."""
-        names: set[str] = set()
-        if turn.if_present is not None:
-            names.add(turn.if_present)
-        if turn.tool_calls_from is not None:
-            names.add(turn.tool_calls_from)
-        names.update(_placeholders_in_content(turn.content))
-        return names
-
-
-def _placeholders_in_content(content: str | list[dict[str, Any]] | None) -> set[str]:
-    """Return the set of ``${name}`` placeholders found anywhere in ``content``."""
-    if content is None:
-        return set()
-    if isinstance(content, str):
-        return set(PLACEHOLDER_RE.findall(content))
-
-    names: set[str] = set()
-    for block in content:
-        for value in block.values():
-            if isinstance(value, str):
-                names.update(PLACEHOLDER_RE.findall(value))
-    return names
-
-
-def load_recipe(path: str | Path) -> TrainingRecipe:
-    """Load a :class:`TrainingRecipe` from a YAML file at ``path``."""
-    return TrainingRecipe.from_yaml(path)

src/lerobot/configs/rewards.py

@@ -27,13 +27,12 @@ from huggingface_hub import hf_hub_download
 from huggingface_hub.constants import CONFIG_NAME
 from huggingface_hub.errors import HfHubHTTPError
 
+from lerobot.configs.types import PolicyFeature
 from lerobot.optim.optimizers import OptimizerConfig
 from lerobot.optim.schedulers import LRSchedulerConfig
 from lerobot.utils.device_utils import auto_select_torch_device, is_torch_device_available
 from lerobot.utils.hub import HubMixin
 
-from .types import PolicyFeature
-
 T = TypeVar("T", bound="RewardModelConfig")
 logger = logging.getLogger(__name__)
 
@@ -91,7 +90,14 @@ class RewardModelConfig(draccus.ChoiceRegistry, HubMixin, abc.ABC):
         return None
 
     def get_optimizer_preset(self) -> OptimizerConfig | None:
-        """Default optimizer for this reward model, or ``None`` for zero-shot models."""
+        """Default optimizer for this reward model, or ``None`` for zero-shot models.
+
+        Trainable reward models (e.g. SARM, Classifier) must override this with a
+        concrete optimizer config. Zero-shot reward models (e.g. Robometer) leave
+        the default ``None`` — they error out earlier via the
+        :attr:`~lerobot.rewards.pretrained.PreTrainedRewardModel.is_trainable`
+        check in ``lerobot-train``.
+        """
         return None
 
     def get_scheduler_preset(self) -> LRSchedulerConfig | None:

src/lerobot/configs/train.py

@@ -25,11 +25,11 @@ from huggingface_hub import hf_hub_download
 from huggingface_hub.errors import HfHubHTTPError
 
 from lerobot import envs
+from lerobot.configs import parser
 from lerobot.optim import LRSchedulerConfig, OptimizerConfig
 from lerobot.utils.hub import HubMixin
 from lerobot.utils.sample_weighting import SampleWeightingConfig
 
-from . import parser
 from .default import DatasetConfig, EvalConfig, PeftConfig, WandBConfig
 from .policies import PreTrainedConfig
 from .rewards import RewardModelConfig

src/lerobot/configs/video.py

@@ -1,235 +0,0 @@
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-# Note: We subclass str so that serialization is straightforward
-# https://stackoverflow.com/questions/24481852/serialising-an-enum-member-to-json
-
-"""Video encoder configurations."""
-
-from __future__ import annotations
-
-import logging
-from dataclasses import dataclass, field
-from typing import Any
-
-from lerobot.utils.import_utils import require_package
-
-logger = logging.getLogger(__name__)
-
-# List of hardware encoders to probe for auto-selection. Availability depends on the platform and the chosen video backend.
-# Determines the order of preference for auto-selection when vcodec="auto" is used.
-HW_VIDEO_CODECS = [
-    "h264_videotoolbox",  # macOS
-    "hevc_videotoolbox",  # macOS
-    "h264_nvenc",  # NVIDIA GPU
-    "hevc_nvenc",  # NVIDIA GPU
-    "h264_vaapi",  # Linux Intel/AMD
-    "h264_qsv",  # Intel Quick Sync
-]
-VALID_VIDEO_CODECS: frozenset[str] = frozenset({"h264", "hevc", "libsvtav1", "auto", *HW_VIDEO_CODECS})
-# Aliases for legacy video codec names.
-VIDEO_CODECS_ALIASES: dict[str, str] = {"av1": "libsvtav1"}
-
-
-LIBSVTAV1_DEFAULT_PRESET: int = 12
-
-# Keys persisted under ``features[*]["info"]`` as ``video.<name>`` (from :class:`VideoEncoderConfig`).
-# ``vcodec``` and ``pix_fmt`` are derived from the video stream directly.
-VIDEO_ENCODER_INFO_FIELD_NAMES: frozenset[str] = frozenset(
-    {"g", "crf", "preset", "fast_decode", "extra_options", "video_backend"}
-)
-VIDEO_ENCODER_INFO_KEYS: frozenset[str] = frozenset(
-    f"video.{name}" for name in VIDEO_ENCODER_INFO_FIELD_NAMES
-)
-
-
-@dataclass
-class VideoEncoderConfig:
-    """Video encoder configuration.
-
-    Attributes:
-        vcodec: Video encoder name. ``"auto"`` is resolved during
-            construction (HW encoder if available, else ``libsvtav1``).
-        pix_fmt: Pixel format (e.g. ``"yuv420p"``).
-        g: GOP size (keyframe interval).
-        crf: Quality level — mapped to the native quality parameter of the
-            codec (``crf`` for software, ``qp`` for NVENC/VAAPI,
-            ``q:v`` for VideoToolbox, ``global_quality`` for QSV).
-        preset: Speed/quality preset. Accepted type is per-codec.
-        fast_decode: Fast-decode tuning. For ``libsvtav1`` this is a level (0-2)
-            embedded in ``svtav1-params``. For ``h264`` and ``hevc`` non-zero values
-            set ``tune=fastdecode``. Ignored for other codecs.
-        video_backend: Python to be used for encoding. Only ``"pyav"``
-            is currently supported.
-        extra_options: Free-form dictionary of additional video encoder options
-            (e.g. ``{"tune": "film", "profile:v": "high", "bf": 2}``).
-    """
-
-    vcodec: str = "libsvtav1"  # TODO(CarolinePascal): rename to codec ?
-    pix_fmt: str = "yuv420p"
-    g: int | None = 2
-    crf: int | float | None = 30
-    preset: int | str | None = None
-    fast_decode: int = 0
-    # TODO(CarolinePascal): add torchcodec support + find a way to unify the
-    # two backends (encoding and decoding).
-    video_backend: str = "pyav"
-    extra_options: dict[str, Any] = field(default_factory=dict)
-
-    def __post_init__(self) -> None:
-        self.resolve_vcodec()
-        # Empty-constructor ergonomics: ``VideoEncoderConfig()`` must "just work".
-        if self.preset is None and self.vcodec == "libsvtav1":
-            self.preset = LIBSVTAV1_DEFAULT_PRESET
-        self.validate()
-
-    @classmethod
-    def from_video_info(cls, video_info: dict | None) -> VideoEncoderConfig:
-        """Reconstruct a :class:`VideoEncoderConfig` from a video feature's ``info`` block.
-        Missing or ``None`` values fall back to the class defaults.
-        """
-        video_info = video_info or {}
-        kwargs: dict[str, Any] = {}
-
-        for src_key, dst_field in (("video.codec", "vcodec"), ("video.pix_fmt", "pix_fmt")):
-            value = video_info.get(src_key)
-            if value is not None:
-                kwargs[dst_field] = value
-
-        for field_name in VIDEO_ENCODER_INFO_FIELD_NAMES:
-            value = video_info.get(f"video.{field_name}")
-            if value is None:
-                continue
-            # Persisted as ``{}`` after merges with disagreeing sources — treat as default.
-            if field_name == "extra_options" and not value:
-                continue
-            kwargs[field_name] = value
-
-        return cls(**kwargs)
-
-    def detect_available_encoders(self, encoders: list[str] | str) -> list[str]:
-        """Return the subset of available encoders based on the specified video backend.
-
-        Args:
-            encoders: List of encoder names to detect. If a string, it is converted to a list.
-        Returns:
-            List of available encoder names. If the video backend is not "pyav", returns an empty list.
-        """
-        if self.video_backend == "pyav":
-            require_package("av", extra="dataset")
-            from lerobot.datasets import detect_available_encoders_pyav
-
-            return detect_available_encoders_pyav(encoders)
-        return []
-
-    def validate(self) -> None:
-        """Validate the video encoder configuration."""
-        if self.video_backend == "pyav":
-            require_package("av", extra="dataset")
-            from lerobot.datasets import check_video_encoder_parameters_pyav
-
-            check_video_encoder_parameters_pyav(self.vcodec, self.pix_fmt, self.get_codec_options())
-
-    def resolve_vcodec(self) -> None:
-        """Check ``vcodec`` and, when it is ``"auto"``, pick a concrete encoder.
-
-        For ``"auto"``, the first hardware encoder in the preference list that is available is chosen; if none are available, ``libsvtav1`` is used. If the
-        resolved codec (explicit or after auto-selection) is not available, raises ``ValueError``.
-
-        Stream-derived canonical codec names listed in :data:`VIDEO_CODECS_ALIASES` are
-        rewritten to their corresponding encoder name (e.g. ``"av1"`` → ``"libsvtav1"``).
-        """
-        self.vcodec = VIDEO_CODECS_ALIASES.get(self.vcodec, self.vcodec)
-        if self.vcodec not in VALID_VIDEO_CODECS:
-            raise ValueError(f"Invalid vcodec '{self.vcodec}'. Must be one of: {sorted(VALID_VIDEO_CODECS)}")
-        if self.vcodec == "auto":
-            available = self.detect_available_encoders(HW_VIDEO_CODECS)
-            for encoder in HW_VIDEO_CODECS:
-                if encoder in available:
-                    logger.info(f"Auto-selected video codec: {encoder}")
-                    self.vcodec = encoder
-                    return
-            logger.warning("No hardware encoder available, falling back to software encoder 'libsvtav1'")
-            self.vcodec = "libsvtav1"
-
-        if self.detect_available_encoders(self.vcodec):
-            logger.info(f"Using video codec: {self.vcodec}")
-            return
-        raise ValueError(f"Unsupported video codec: {self.vcodec} with video backend {self.video_backend}")
-
-    def get_codec_options(
-        self, encoder_threads: int | None = None, as_strings: bool = False
-    ) -> dict[str, Any]:
-        """Translate the tuning fields to codec-specific options.
-
-        ``VideoEncoderConfig.extra_options`` are merged last but never override a structured field.
-
-        Args:
-            encoder_threads: Number of encoder threads set globally for all VideoEncoderConfigs.
-                For libsvtav1, this is mapped to ``lp`` via ``svtav1-params``.
-                For h264/hevc, this is mapped to ``threads``.
-                Hardware encoders ignore this parameter.
-            as_strings: If ``True``, casts values to strings.
-        """
-        opts: dict[str, Any] = {}
-
-        def set_if(key: str, value: Any) -> None:
-            if value is not None:
-                opts[key] = value if not as_strings else str(value)
-
-        # GOP size is not a codec-specific option, so it is always set.
-        set_if("g", self.g)
-
-        if self.vcodec == "libsvtav1":
-            set_if("crf", self.crf)
-            set_if("preset", self.preset)
-            svtav1_parts: list[str] = []
-            if self.fast_decode is not None:
-                svtav1_parts.append(f"fast-decode={max(0, min(2, self.fast_decode))}")
-            if encoder_threads is not None:
-                svtav1_parts.append(f"lp={encoder_threads}")
-            if svtav1_parts:
-                opts["svtav1-params"] = ":".join(svtav1_parts)
-        elif self.vcodec in ("h264", "hevc"):
-            set_if("crf", self.crf)
-            set_if("preset", self.preset)
-            if self.fast_decode:
-                opts["tune"] = "fastdecode"
-            set_if("threads", encoder_threads)
-        elif self.vcodec in ("h264_videotoolbox", "hevc_videotoolbox"):
-            if self.crf is not None:
-                opts["q:v"] = max(1, min(100, 100 - self.crf * 2))
-        elif self.vcodec in ("h264_nvenc", "hevc_nvenc"):
-            opts["rc"] = 0
-            set_if("qp", self.crf)
-            set_if("preset", self.preset)
-        elif self.vcodec == "h264_vaapi":
-            set_if("qp", self.crf)
-        elif self.vcodec == "h264_qsv":
-            set_if("global_quality", self.crf)
-            set_if("preset", self.preset)
-        else:
-            set_if("crf", self.crf)
-            set_if("preset", self.preset)
-
-        # Extra options are merged last but never override structured fields (values are kept as given).
-        for k, v in self.extra_options.items():
-            if k not in opts:
-                set_if(k, v)
-
-        return opts
-
-
-def camera_encoder_defaults() -> VideoEncoderConfig:
-    """Return a :class:`VideoEncoderConfig` with RGB-camera defaults."""
-    return VideoEncoderConfig()

src/lerobot/datasets/__init__.py

@@ -31,25 +31,15 @@ from .dataset_tools import (
     modify_features,
     modify_tasks,
     recompute_stats,
-    reencode_dataset,
     remove_feature,
     split_dataset,
 )
 from .factory import make_dataset, resolve_delta_timestamps
 from .image_writer import safe_stop_image_writer
 from .io_utils import load_episodes, write_stats
-from .language import (
-    EVENT_ONLY_STYLES,
-    LANGUAGE_EVENTS,
-    LANGUAGE_PERSISTENT,
-    PERSISTENT_STYLES,
-    STYLE_REGISTRY,
-    column_for_style,
-)
 from .lerobot_dataset import LeRobotDataset
 from .multi_dataset import MultiLeRobotDataset
 from .pipeline_features import aggregate_pipeline_dataset_features, create_initial_features
-from .pyav_utils import check_video_encoder_parameters_pyav, detect_available_encoders_pyav
 from .sampler import EpisodeAwareSampler
 from .streaming_dataset import StreamingLeRobotDataset
 from .utils import DEFAULT_EPISODES_PATH, create_lerobot_dataset_card
@@ -63,19 +53,12 @@ __all__ = [
     "CODEBASE_VERSION",
     "DEFAULT_EPISODES_PATH",
     "DEFAULT_QUANTILES",
-    "EVENT_ONLY_STYLES",
     "EpisodeAwareSampler",
-    "LANGUAGE_EVENTS",
-    "LANGUAGE_PERSISTENT",
     "LeRobotDataset",
     "LeRobotDatasetMetadata",
     "MultiLeRobotDataset",
-    "PERSISTENT_STYLES",
-    "STYLE_REGISTRY",
     "StreamingLeRobotDataset",
     "VideoEncodingManager",
-    "check_video_encoder_parameters_pyav",
-    "detect_available_encoders_pyav",
     "add_features",
     "aggregate_datasets",
     "aggregate_pipeline_dataset_features",
@@ -83,7 +66,6 @@ __all__ = [
     "convert_image_to_video_dataset",
     "create_initial_features",
     "create_lerobot_dataset_card",
-    "column_for_style",
     "delete_episodes",
     "get_feature_stats",
     "load_episodes",
@@ -92,7 +74,6 @@ __all__ = [
     "modify_features",
     "modify_tasks",
     "recompute_stats",
-    "reencode_dataset",
     "remove_feature",
     "resolve_delta_timestamps",
     "safe_stop_image_writer",

src/lerobot/datasets/aggregate.py

@@ -15,7 +15,6 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import copy
 import logging
 import shutil
 from pathlib import Path
@@ -24,11 +23,9 @@ import datasets
 import pandas as pd
 import tqdm
 
-from lerobot.configs import VIDEO_ENCODER_INFO_KEYS
-
 from .compute_stats import aggregate_stats
 from .dataset_metadata import LeRobotDatasetMetadata
-from .feature_utils import features_equal_for_merge, get_hf_features_from_features
+from .feature_utils import get_hf_features_from_features
 from .io_utils import (
     get_file_size_in_mb,
     get_parquet_file_size_in_mb,
@@ -49,54 +46,11 @@ from .utils import (
 from .video_utils import concatenate_video_files, get_video_duration_in_s
 
 
-def merge_video_feature_info_for_aggregate(all_metadata: list[LeRobotDatasetMetadata]) -> dict[str, dict]:
-    """Create a merged video feature info dictionary for aggregation. The video encoder info is merged field-by-field: each key is kept only when every source agrees; otherwise that key is set to ``null`` (or ``{}`` for ``video.extra_options``) and a warning is logged.
-
-    Args:
-        all_metadata: List of LeRobotDatasetMetadata objects to merge.
-
-    Returns:
-        dict: A dictionary of merged video feature info.
-    """
-    merged_info = copy.deepcopy(all_metadata[0].features)
-    video_keys = [k for k in merged_info if merged_info[k].get("dtype") == "video"]
-
-    for vk in video_keys:
-        video_infos = [m.features.get(vk, {}).get("info") or {} for m in all_metadata]
-        base_video_info = video_infos[0]
-
-        merged_encoder_info: dict = {}
-        fallback_keys: list[str] = []
-        for info_key in VIDEO_ENCODER_INFO_KEYS:
-            values = [info.get(info_key, None) for info in video_infos]
-            first_value = values[0]
-            all_match = all(v == first_value for v in values[1:])
-
-            if all_match:
-                merged_encoder_info[info_key] = first_value
-            else:
-                fallback_keys.append(info_key)
-                merged_encoder_info[info_key] = {} if info_key == "video.extra_options" else None
-
-        if fallback_keys:
-            logging.warning(
-                f"Merging heterogeneous or incomplete video encoder metadata for feature {vk}. "
-                f"Setting these keys to null: {fallback_keys}.",
-            )
-
-        merged_info[vk]["info"] = {**base_video_info, **merged_encoder_info}
-        # TODO(CarolinePascal): make this variable once we have support for other video backends.
-        merged_info[vk]["info"]["video.video_backend"] = "pyav"
-
-    return merged_info
-
-
 def validate_all_metadata(all_metadata: list[LeRobotDatasetMetadata]):
     """Validates that all dataset metadata have consistent properties.
 
     Ensures all datasets have the same fps, robot_type, and features to guarantee
     compatibility when aggregating them into a single dataset.
-    Video encoder info is not considered for validation but is merged during aggregation in ``merge_video_feature_info_for_aggregate``.
 
     Args:
         all_metadata: List of LeRobotDatasetMetadata objects to validate.
@@ -120,7 +74,7 @@ def validate_all_metadata(all_metadata: list[LeRobotDatasetMetadata]):
             raise ValueError(
                 f"Same robot_type is expected, but got robot_type={meta.robot_type} instead of {robot_type}."
             )
-        if not features_equal_for_merge(features, meta.features):
+        if features != meta.features:
             raise ValueError(
                 f"Same features is expected, but got features={meta.features} instead of {features}."
             )
@@ -320,8 +274,7 @@ def aggregate_datasets(
             LeRobotDatasetMetadata(repo_id, root=root) for repo_id, root in zip(repo_ids, roots, strict=False)
         ]
     )
-    fps, robot_type, _ = validate_all_metadata(all_metadata)
-    features = merge_video_feature_info_for_aggregate(all_metadata)
+    fps, robot_type, features = validate_all_metadata(all_metadata)
     video_keys = [key for key in features if features[key]["dtype"] == "video"]
 
     dst_meta = LeRobotDatasetMetadata.create(
@@ -379,6 +332,7 @@ def aggregate_videos(src_meta, dst_meta, videos_idx, video_files_size_in_mb, chu
         videos_idx: Dictionary tracking video chunk and file indices.
         video_files_size_in_mb: Maximum size for video files in MB (defaults to DEFAULT_VIDEO_FILE_SIZE_IN_MB)
         chunk_size: Maximum number of files per chunk (defaults to DEFAULT_CHUNK_SIZE)
+
     Returns:
         dict: Updated videos_idx with current chunk and file indices.
     """
@@ -460,11 +414,9 @@ def aggregate_videos(src_meta, dst_meta, videos_idx, video_files_size_in_mb, chu
                 current_dst_duration = dst_file_durations.get(dst_key, 0)
                 videos_idx[key]["src_to_offset"][(src_chunk_idx, src_file_idx)] = current_dst_duration
                 videos_idx[key]["src_to_dst"][(src_chunk_idx, src_file_idx)] = dst_key
-                # TODO(CarolinePascal): Move the check before the loop to avoid failing in the middle + add possibility to re-encode the video if the check fails
                 concatenate_video_files(
                     [dst_path, src_path],
                     dst_path,
-                    compatibility_check=True,
                 )
                 # Update duration of this destination file
                 dst_file_durations[dst_key] = current_dst_duration + src_duration

src/lerobot/datasets/compute_stats.py

@@ -512,7 +512,7 @@ def compute_episode_stats(
 
     ep_stats = {}
     for key, data in episode_data.items():
-        if features[key]["dtype"] in {"string", "language"}:
+        if features[key]["dtype"] == "string":
             continue
 
         if features[key]["dtype"] in ["image", "video"]:

src/lerobot/datasets/dataset_metadata.py

@@ -24,7 +24,6 @@ import pyarrow as pa
 import pyarrow.parquet as pq
 from huggingface_hub import snapshot_download
 
-from lerobot.configs import VideoEncoderConfig
 from lerobot.utils.constants import DEFAULT_FEATURES, HF_LEROBOT_HOME, HF_LEROBOT_HUB_CACHE
 from lerobot.utils.feature_utils import _validate_feature_names
 from lerobot.utils.utils import flatten_dict
@@ -36,12 +35,12 @@ from .io_utils import (
     load_episodes,
     load_info,
     load_stats,
+    load_subtasks,
     load_tasks,
     write_info,
     write_stats,
     write_tasks,
 )
-from .language import DEFAULT_TOOLS, LANGUAGE_COLUMNS
 from .utils import (
     DEFAULT_EPISODES_PATH,
     check_version_compatibility,
@@ -177,6 +176,7 @@ class LeRobotDatasetMetadata:
         self.info = load_info(self.root)
         check_version_compatibility(self.repo_id, self._version, CODEBASE_VERSION)
         self.tasks = load_tasks(self.root)
+        self.subtasks = load_subtasks(self.root)
         self.episodes = load_episodes(self.root)
         self.stats = load_stats(self.root)
 
@@ -342,49 +342,6 @@ class LeRobotDatasetMetadata:
         """Keys to access visual modalities (regardless of their storage method)."""
         return [key for key, ft in self.features.items() if ft["dtype"] in ["video", "image"]]
 
-    @property
-    def has_language_columns(self) -> bool:
-        """Return ``True`` if the dataset declares any language column.
-
-        Used to gate language-aware code paths (collate, render step) so
-        unannotated datasets keep PyTorch's default collate behavior.
-        """
-        return any(col in self.features for col in LANGUAGE_COLUMNS)
-
-    @property
-    def tools(self) -> list[dict]:
-        """OpenAI-style tool schemas declared by this dataset.
-
-        Read from ``meta/info.json["tools"]``. Returns a copy, so callers
-        can mutate the result safely. Falls back to
-        :data:`lerobot.datasets.language.DEFAULT_TOOLS` (the canonical
-        ``say`` schema) when the dataset doesn't declare any — that way
-        unannotated datasets and chat-template consumers
-        (``apply_chat_template(messages, tools=meta.tools)``) keep
-        working out of the box.
-
-        Implementations live under :mod:`lerobot.tools` (one file per
-        tool); see ``docs/source/tools.mdx`` for the authoring guide.
-        """
-        declared = self.info.tools
-        if declared:
-            return [dict(t) for t in declared]
-        return [dict(t) for t in DEFAULT_TOOLS]
-
-    @tools.setter
-    def tools(self, value: list[dict] | None) -> None:
-        """Persist a tool catalog to ``meta/info.json`` and reload metadata.
-
-        Writes ``value`` into the on-disk ``info.json`` (or clears the
-        ``tools`` key when ``value`` is ``None`` or empty), then reloads
-        ``self.info`` so the in-memory metadata matches what's on disk.
-        Saves callers from hand-editing ``info.json`` and re-instantiating
-        the metadata object.
-        """
-        self.info.tools = [dict(t) for t in value] if value else None
-        write_info(self.info, self.root)
-        self.info = load_info(self.root)
-
     @property
     def names(self) -> dict[str, list | dict]:
         """Names of the various dimensions of vector modalities."""
@@ -577,23 +534,10 @@ class LeRobotDatasetMetadata:
         self.stats = aggregate_stats([self.stats, episode_stats]) if self.stats is not None else episode_stats
         write_stats(self.stats, self.root)
 
-    def update_video_info(
-        self,
-        video_key: str | None = None,
-        camera_encoder: VideoEncoderConfig | None = None,
-    ) -> None:
-        """Populate per-feature video info in ``info.json``.
-
+    def update_video_info(self, video_key: str | None = None) -> None:
+        """
         Warning: this function writes info from first episode videos, implicitly assuming that all videos have
         been encoded the same way. Also, this means it assumes the first episode exists.
-
-        Args:
-            video_key: If provided, only update this video key. Otherwise update
-                all video keys in the dataset.
-            camera_encoder: Encoder configuration used to produce the
-                videos. When provided, its fields are recorded as
-                ``video.<field>`` entries alongside the stream-derived
-                ``video.*`` entries (see :func:`get_video_info`).
         """
         if video_key is not None and video_key not in self.video_keys:
             raise ValueError(f"Video key {video_key} not found in dataset")
@@ -602,7 +546,7 @@ class LeRobotDatasetMetadata:
         for key in video_keys:
             if not self.features[key].get("info", None):
                 video_path = self.root / self.video_path.format(video_key=key, chunk_index=0, file_index=0)
-                self.info.features[key]["info"] = get_video_info(video_path, camera_encoder=camera_encoder)
+                self.info.features[key]["info"] = get_video_info(video_path)
 
     def update_chunk_settings(
         self,
@@ -713,6 +657,7 @@ class LeRobotDatasetMetadata:
         _validate_feature_names(features)
 
         obj.tasks = None
+        obj.subtasks = None
         obj.episodes = None
         obj.stats = None
         obj.info = create_empty_dataset_info(

src/lerobot/datasets/dataset_reader.py

@@ -295,4 +295,9 @@ class DatasetReader:
         task_idx = item["task_index"].item()
         item["task"] = self._meta.tasks.iloc[task_idx].name
 
+        # add subtask information if available
+        if "subtask_index" in self._meta.features and self._meta.subtasks is not None:
+            subtask_idx = item["subtask_index"].item()
+            item["subtask"] = self._meta.subtasks.iloc[subtask_idx].name
+
         return item

src/lerobot/datasets/dataset_tools.py

@@ -26,7 +26,7 @@ This module provides utilities for:
 import logging
 import shutil
 from collections.abc import Callable
-from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor, as_completed
+from concurrent.futures import ThreadPoolExecutor, as_completed
 from pathlib import Path
 
 import datasets
@@ -36,7 +36,6 @@ import pyarrow.parquet as pq
 import torch
 from tqdm import tqdm
 
-from lerobot.configs import VideoEncoderConfig, camera_encoder_defaults
 from lerobot.utils.constants import ACTION, HF_LEROBOT_HOME, OBS_IMAGE, OBS_STATE
 from lerobot.utils.utils import flatten_dict
 
@@ -61,14 +60,9 @@ from .utils import (
     DEFAULT_DATA_FILE_SIZE_IN_MB,
     DEFAULT_DATA_PATH,
     DEFAULT_EPISODES_PATH,
-    VIDEO_DIR,
     update_chunk_file_indices,
 )
-from .video_utils import (
-    encode_video_frames,
-    get_video_info,
-    reencode_video,
-)
+from .video_utils import encode_video_frames, get_video_info
 
 
 def _load_episode_with_stats(src_dataset: LeRobotDataset, episode_idx: int) -> dict:
@@ -101,11 +95,6 @@ def delete_episodes(
 ) -> LeRobotDataset:
     """Delete episodes from a LeRobotDataset and create a new dataset.
 
-    Video segments that need re-encoding (because the source file mixes kept and
-    deleted episodes) are re-encoded with the source dataset's existing encoder
-    settings — read back from ``meta/info.json`` — so the output dataset stays
-    consistent with its own metadata.
-
     Args:
         dataset: The source LeRobotDataset.
         episode_indices: List of episode indices to delete.
@@ -168,11 +157,6 @@ def split_dataset(
 ) -> dict[str, LeRobotDataset]:
     """Split a LeRobotDataset into multiple smaller datasets.
 
-    Video segments that need re-encoding (because the source file mixes episodes
-    that fall into different splits) are re-encoded with the source dataset's
-    existing encoder settings — read back from ``meta/info.json`` — so each
-    output split stays consistent with its own metadata.
-
     Args:
         dataset: The source LeRobotDataset to split.
         splits: Either a dict mapping split names to episode indices, or a dict mapping
@@ -594,7 +578,8 @@ def _keep_episodes_from_video_with_av(
     output_path: Path,
     episodes_to_keep: list[tuple[int, int]],
     fps: float,
-    camera_encoder: VideoEncoderConfig,
+    vcodec: str = "libsvtav1",
+    pix_fmt: str = "yuv420p",
 ) -> None:
     """Keep only specified episodes from a video file using PyAV.
 
@@ -608,7 +593,8 @@ def _keep_episodes_from_video_with_av(
             Ranges are half-open intervals: [start_frame, end_frame), where start_frame
             is inclusive and end_frame is exclusive.
         fps: Frame rate of the video.
-        camera_encoder: Video encoder settings used to re-encode the kept frames.
+        vcodec: Video codec to use for encoding.
+        pix_fmt: Pixel format for output video.
     """
     from fractions import Fraction
 
@@ -633,13 +619,12 @@ def _keep_episodes_from_video_with_av(
 
     # Convert fps to Fraction for PyAV compatibility.
     fps_fraction = Fraction(fps).limit_denominator(1000)
-    codec_options = camera_encoder.get_codec_options(as_strings=True)
-    v_out = out.add_stream(camera_encoder.vcodec, rate=fps_fraction, options=codec_options)
+    v_out = out.add_stream(vcodec, rate=fps_fraction)
 
     # PyAV type stubs don't distinguish video streams from audio/subtitle streams.
     v_out.width = v_in.codec_context.width
     v_out.height = v_in.codec_context.height
-    v_out.pix_fmt = camera_encoder.pix_fmt
+    v_out.pix_fmt = pix_fmt
 
     # Set time_base to match the frame rate for proper timestamp handling.
     v_out.time_base = Fraction(1, int(fps))
@@ -702,14 +687,14 @@ def _copy_and_reindex_videos(
     src_dataset: LeRobotDataset,
     dst_meta: LeRobotDatasetMetadata,
     episode_mapping: dict[int, int],
+    vcodec: str = "libsvtav1",
+    pix_fmt: str = "yuv420p",
 ) -> dict[int, dict]:
     """Copy and filter video files, only re-encoding files with deleted episodes.
 
     For video files that only contain kept episodes, we copy them directly.
     For files with mixed kept/deleted episodes, we use PyAV filters to efficiently
-    re-encode only the desired segments. The encoder used for re-encoding is
-    derived per video key from the source dataset's ``meta/info.json`` so the
-    destination metadata keeps describing the videos accurately.
+    re-encode only the desired segments.
 
     Args:
         src_dataset: Source dataset to copy from
@@ -726,9 +711,6 @@ def _copy_and_reindex_videos(
 
     for video_key in src_dataset.meta.video_keys:
         logging.info(f"Processing videos for {video_key}")
-        camera_encoder = VideoEncoderConfig.from_video_info(
-            src_dataset.meta.info.features.get(video_key, {}).get("info")
-        )
 
         if dst_meta.video_path is None:
             raise ValueError("Destination metadata has no video_path defined")
@@ -810,7 +792,8 @@ def _copy_and_reindex_videos(
                     dst_video_path,
                     episodes_to_keep_ranges,
                     src_dataset.meta.fps,
-                    camera_encoder,
+                    vcodec,
+                    pix_fmt,
                 )
 
                 cumulative_ts = 0.0
@@ -1281,7 +1264,11 @@ def _estimate_frame_size_via_calibration(
     episode_indices: list[int],
     temp_dir: Path,
     fps: int,
-    camera_encoder: VideoEncoderConfig,
+    vcodec: str,
+    pix_fmt: str,
+    g: int,
+    crf: int,
+    fast_decode: int,
     num_calibration_frames: int = 30,
 ) -> float:
     """Estimate MB per frame by encoding a small calibration sample.
@@ -1295,7 +1282,11 @@ def _estimate_frame_size_via_calibration(
         episode_indices: List of episode indices being processed.
         temp_dir: Temporary directory for calibration files.
         fps: Frames per second for video encoding.
-        camera_encoder: Video encoder settings used for calibration encoding.
+        vcodec: Video codec (libsvtav1, h264, hevc).
+        pix_fmt: Pixel format (yuv420p, etc.).
+        g: GOP size (group of pictures).
+        crf: Constant Rate Factor (quality).
+        fast_decode: Fast decode tuning parameter.
         num_calibration_frames: Number of frames to use for calibration (default: 30).
 
     Returns:
@@ -1331,7 +1322,11 @@ def _estimate_frame_size_via_calibration(
             imgs_dir=calibration_dir,
             video_path=calibration_video_path,
             fps=fps,
-            camera_encoder=camera_encoder,
+            vcodec=vcodec,
+            pix_fmt=pix_fmt,
+            g=g,
+            crf=crf,
+            fast_decode=fast_decode,
             overwrite=True,
         )
 
@@ -1649,7 +1644,11 @@ def convert_image_to_video_dataset(
     dataset: LeRobotDataset,
     output_dir: Path | None = None,
     repo_id: str | None = None,
-    camera_encoder: VideoEncoderConfig | None = None,
+    vcodec: str = "libsvtav1",
+    pix_fmt: str = "yuv420p",
+    g: int = 2,
+    crf: int = 30,
+    fast_decode: int = 0,
     episode_indices: list[int] | None = None,
     num_workers: int = 4,
     max_episodes_per_batch: int | None = None,
@@ -1664,8 +1663,11 @@ def convert_image_to_video_dataset(
         dataset: The source LeRobot dataset with images
         output_dir: Root directory where the edited dataset will be stored. If not specified, defaults to $HF_LEROBOT_HOME/repo_id. Equivalent to new_root in EditDatasetConfig.
         repo_id: Edited dataset identifier. Equivalent to new_repo_id in EditDatasetConfig.
-        camera_encoder: Video encoder settings
-            (``None`` uses :func:`~lerobot.configs.camera_encoder_defaults`).
+        vcodec: Video codec (default: libsvtav1)
+        pix_fmt: Pixel format (default: yuv420p)
+        g: Group of pictures size (default: 2)
+        crf: Constant rate factor (default: 30)
+        fast_decode: Fast decode tuning (default: 0)
         episode_indices: List of episode indices to convert (None = all episodes)
         num_workers: Number of threads for parallel processing (default: 4)
         max_episodes_per_batch: Maximum episodes per video batch to avoid memory issues (None = no limit)
@@ -1674,9 +1676,6 @@ def convert_image_to_video_dataset(
     Returns:
         New LeRobotDataset with images encoded as videos
     """
-    if camera_encoder is None:
-        camera_encoder = camera_encoder_defaults()
-
     # Check that it's an image dataset
     if len(dataset.meta.video_keys) > 0:
         raise ValueError(
@@ -1700,10 +1699,7 @@ def convert_image_to_video_dataset(
     logging.info(
         f"Converting {len(episode_indices)} episodes with {len(img_keys)} cameras from {dataset.repo_id}"
     )
-    logging.info(
-        f"Video codec: {camera_encoder.vcodec}, pixel format: {camera_encoder.pix_fmt}, "
-        f"GOP: {camera_encoder.g}, CRF: {camera_encoder.crf}"
-    )
+    logging.info(f"Video codec: {vcodec}, pixel format: {pix_fmt}, GOP: {g}, CRF: {crf}")
 
     # Create new features dict, converting image features to video features
     new_features = {}
@@ -1773,7 +1769,11 @@ def convert_image_to_video_dataset(
                 episode_indices=episode_indices,
                 temp_dir=temp_dir,
                 fps=fps,
-                camera_encoder=camera_encoder,
+                vcodec=vcodec,
+                pix_fmt=pix_fmt,
+                g=g,
+                crf=crf,
+                fast_decode=fast_decode,
             )
 
             logging.info(f"Processing camera: {img_key}")
@@ -1815,7 +1815,11 @@ def convert_image_to_video_dataset(
                     imgs_dir=imgs_dir,
                     video_path=video_path,
                     fps=fps,
-                    camera_encoder=camera_encoder,
+                    vcodec=vcodec,
+                    pix_fmt=pix_fmt,
+                    g=g,
+                    crf=crf,
+                    fast_decode=fast_decode,
                     overwrite=True,
                 )
 
@@ -1861,9 +1865,7 @@ def convert_image_to_video_dataset(
                 video_path = new_meta.root / new_meta.video_path.format(
                     video_key=img_key, chunk_index=0, file_index=0
                 )
-                new_meta.info.features[img_key]["info"] = get_video_info(
-                    video_path, camera_encoder=camera_encoder
-                )
+                new_meta.info.features[img_key]["info"] = get_video_info(video_path)
 
         write_info(new_meta.info, new_meta.root)
 
@@ -1886,83 +1888,3 @@ def convert_image_to_video_dataset(
 
     # Return new dataset
     return LeRobotDataset(repo_id=repo_id, root=output_dir)
-
-
-def _reencode_video_worker(args: tuple) -> Path:
-    """Picklable worker for :func:`reencode_dataset`'s process pool."""
-    video_path, camera_encoder, encoder_threads = args
-    reencode_video(
-        input_video_path=video_path,
-        output_video_path=video_path,
-        camera_encoder=camera_encoder,
-        encoder_threads=encoder_threads,
-        overwrite=True,
-    )
-    return video_path
-
-
-def reencode_dataset(
-    dataset: LeRobotDataset,
-    camera_encoder: VideoEncoderConfig,
-    encoder_threads: int | None = None,
-    num_workers: int | None = None,
-) -> LeRobotDataset:
-    """Re-encode every video in a dataset with a new set of encoding parameters.
-
-    Videos are re-encoded in-place and the video information in ``info.json`` is refreshed.
-
-    Args:
-        dataset: An existing :class:`LeRobotDataset` whose videos will be
-            re-encoded.
-        camera_encoder: Target encoder configuration applied to every video
-            file.
-        encoder_threads: Per-encoder thread count forwarded to
-            :func:`reencode_video`. ``None`` lets the codec decide.
-        num_workers: Number of parallel processes. ``None`` or ``0`` means
-            sequential (no multiprocessing); ``1+`` spawns a
-            :class:`~concurrent.futures.ProcessPoolExecutor`.
-
-    Returns:
-        The same :class:`LeRobotDataset` instance with its metadata updated
-        on disk.
-    """
-    meta = dataset.meta
-    video_paths_list = []
-
-    # Only re-encode if the videos are not already encoded with the given video encoding parameters
-    for video_key in meta.video_keys:
-        current_info = meta.info.features[video_key].get("info", {})
-        current_encoder = VideoEncoderConfig.from_video_info(current_info)
-        if current_encoder != camera_encoder:
-            video_paths_list.extend((meta.root / VIDEO_DIR / video_key).rglob("*.mp4"))
-        else:
-            logging.info(f"{video_key} videos are already encoded with {camera_encoder}. Nothing to do.")
-
-    if len(video_paths_list) == 0:
-        logging.warning("Dataset has no videos to re-encode.")
-        return dataset
-    logging.info(f"Re-encoding {len(video_paths_list)} video file(s) with {camera_encoder}")
-
-    worker_args = [(vp, camera_encoder, encoder_threads) for vp in video_paths_list]
-    if num_workers and num_workers > 1:
-        with ProcessPoolExecutor(max_workers=num_workers) as pool:
-            futures = [pool.submit(_reencode_video_worker, args) for args in worker_args]
-            for future in tqdm(
-                as_completed(futures),
-                total=len(futures),
-                desc="Re-encoding videos",
-            ):
-                future.result()
-    else:
-        for args in tqdm(worker_args, desc="Re-encoding videos"):
-            _reencode_video_worker(args)
-
-    # Refresh video info in metadata for every video key.
-    for vid_key in meta.video_keys:
-        video_path = meta.root / meta.get_video_file_path(0, vid_key)
-        meta.info.features[vid_key]["info"] = get_video_info(video_path, camera_encoder=camera_encoder)
-
-    write_info(meta.info, meta.root)
-    logging.info("Dataset metadata updated.")
-
-    return dataset

src/lerobot/datasets/dataset_writer.py

@@ -31,8 +31,6 @@ import PIL.Image
 import pyarrow.parquet as pq
 import torch
 
-from lerobot.configs import VideoEncoderConfig, camera_encoder_defaults
-
 from .compute_stats import compute_episode_stats
 from .dataset_metadata import LeRobotDatasetMetadata
 from .feature_utils import (
@@ -67,19 +65,14 @@ def _encode_video_worker(
     episode_index: int,
     root: Path,
     fps: int,
-    camera_encoder: VideoEncoderConfig | None = None,
+    vcodec: str = "libsvtav1",
     encoder_threads: int | None = None,
 ) -> Path:
     temp_path = Path(tempfile.mkdtemp(dir=root)) / f"{video_key}_{episode_index:03d}.mp4"
     fpath = DEFAULT_IMAGE_PATH.format(image_key=video_key, episode_index=episode_index, frame_index=0)
     img_dir = (root / fpath).parent
     encode_video_frames(
-        img_dir,
-        temp_path,
-        fps,
-        camera_encoder=camera_encoder,
-        encoder_threads=encoder_threads,
-        overwrite=True,
+        img_dir, temp_path, fps, vcodec=vcodec, overwrite=True, encoder_threads=encoder_threads
     )
     shutil.rmtree(img_dir)
     return temp_path
@@ -96,22 +89,20 @@ class DatasetWriter:
         self,
         meta: LeRobotDatasetMetadata,
         root: Path,
-        camera_encoder: VideoEncoderConfig | None,
+        vcodec: str,
         encoder_threads: int | None,
         batch_encoding_size: int,
         streaming_encoder: StreamingVideoEncoder | None = None,
         initial_frames: int = 0,
     ):
-        """Initialize the writer with metadata, codec, and encoder config.
+        """Initialize the writer with metadata, codec, and encoding config.
 
         Args:
             meta: Dataset metadata instance (used for feature schema, chunk
                 settings, and episode persistence).
             root: Local dataset root directory.
-            camera_encoder: Video encoder settings applied to all cameras.
-                ``None`` uses :func:`~lerobot.configs.camera_encoder_defaults`.
-            encoder_threads: Number of encoder threads (global). ``None``
-                lets the codec decide.
+            vcodec: Video codec for encoding (e.g. ``'libsvtav1'``, ``'h264'``).
+            encoder_threads: Threads per encoder instance. ``None`` for auto.
             batch_encoding_size: Number of episodes to accumulate before
                 batch-encoding videos.
             streaming_encoder: Optional pre-built :class:`StreamingVideoEncoder`
@@ -120,7 +111,7 @@ class DatasetWriter:
         """
         self._meta = meta
         self._root = root
-        self._camera_encoder = camera_encoder or camera_encoder_defaults()
+        self._vcodec = vcodec
         self._encoder_threads = encoder_threads
         self._batch_encoding_size = batch_encoding_size
         self._streaming_encoder = streaming_encoder
@@ -293,7 +284,7 @@ class DatasetWriter:
                             episode_index,
                             self._root,
                             self._meta.fps,
-                            self._camera_encoder,
+                            self._vcodec,
                             self._encoder_threads,
                         ): video_key
                         for video_key in self._meta.video_keys
@@ -504,7 +495,7 @@ class DatasetWriter:
 
         # Update video info (only needed when first episode is encoded)
         if episode_index == 0:
-            self._meta.update_video_info(video_key, camera_encoder=self._camera_encoder)
+            self._meta.update_video_info(video_key)
             write_info(self._meta.info, self._meta.root)
 
         metadata = {
@@ -573,12 +564,7 @@ class DatasetWriter:
     def _encode_temporary_episode_video(self, video_key: str, episode_index: int) -> Path:
         """Use ffmpeg to convert frames stored as png into mp4 videos."""
         return _encode_video_worker(
-            video_key,
-            episode_index,
-            self._root,
-            self._meta.fps,
-            self._camera_encoder,
-            self._encoder_threads,
+            video_key, episode_index, self._root, self._meta.fps, self._vcodec, self._encoder_threads
         )
 
     def close_writer(self) -> None:

src/lerobot/datasets/feature_utils.py

@@ -13,23 +13,15 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-import logging
 from pprint import pformat
 
 import datasets
 import numpy as np
 from PIL import Image as PILImage
 
-from lerobot.configs import VIDEO_ENCODER_INFO_KEYS
 from lerobot.utils.constants import DEFAULT_FEATURES
 from lerobot.utils.utils import is_valid_numpy_dtype_string
 
-from .language import (
-    LANGUAGE_PERSISTENT,
-    is_language_column,
-    language_events_column_feature,
-    language_persistent_column_feature,
-)
 from .utils import (
     DEFAULT_CHUNK_SIZE,
     DEFAULT_DATA_FILE_SIZE_IN_MB,
@@ -54,13 +46,7 @@ def get_hf_features_from_features(features: dict) -> datasets.Features:
     """
     hf_features = {}
     for key, ft in features.items():
-        if is_language_column(key):
-            hf_features[key] = (
-                language_persistent_column_feature()
-                if key == LANGUAGE_PERSISTENT
-                else language_events_column_feature()
-            )
-        elif ft["dtype"] == "video":
+        if ft["dtype"] == "video":
             continue
         elif ft["dtype"] == "image":
             hf_features[key] = datasets.Image()
@@ -122,41 +108,6 @@ def create_empty_dataset_info(
     )
 
 
-def features_equal_for_merge(features_a: dict[str, dict], features_b: dict[str, dict]) -> bool:
-    """Return whether two LeRobotDatasetMetadata ``features`` dicts are compatible for aggregation.
-
-    For video features, keys under ``info`` related to video encoding parameters are ignored during
-    comparison as they do not prevent aggregation.
-    """
-
-    def _without_encoder_info_keys(feature: dict) -> dict:
-        filtered = dict(feature)
-        filtered_info = filtered.get("info")
-        if isinstance(filtered_info, dict):
-            filtered["info"] = {
-                info_key: info_value
-                for info_key, info_value in filtered_info.items()
-                if info_key not in VIDEO_ENCODER_INFO_KEYS
-            }
-        return filtered
-
-    if set(features_a) != set(features_b):
-        return False
-    for key in features_a:
-        fa_key = features_a[key]
-        fb_key = features_b[key]
-        if fa_key.get("dtype") != fb_key.get("dtype"):
-            return False
-        if fa_key.get("dtype") != "video":
-            if fa_key != fb_key:
-                return False
-            continue
-
-        if _without_encoder_info_keys(fa_key) != _without_encoder_info_keys(fb_key):
-            return False
-    return True
-
-
 def check_delta_timestamps(
     delta_timestamps: dict[str, list[float]], fps: int, tolerance_s: float, raise_value_error: bool = True
 ) -> bool:
@@ -291,8 +242,6 @@ def validate_feature_dtype_and_shape(
         return validate_feature_image_or_video(name, expected_shape, value)
     elif expected_dtype == "string":
         return validate_feature_string(name, value)
-    elif expected_dtype == "language":
-        return validate_feature_language(name, value)
     else:
         raise NotImplementedError(f"The feature dtype '{expected_dtype}' is not implemented yet.")
 
@@ -372,30 +321,6 @@ def validate_feature_string(name: str, value: str) -> str:
     return ""
 
 
-def validate_feature_language(name: str, value) -> str:
-    """Validate a feature that is expected to hold language annotations.
-
-    Language columns (``language_persistent`` / ``language_events``) are
-    populated after recording by the annotation pipeline, not at record time.
-    Any value supplied here is dropped before the frame is written, so a
-    non-empty value almost certainly signals a mistake. We warn rather than
-    fail to keep recording resilient.
-
-    Args:
-        name (str): The name of the feature.
-        value: The value to validate.
-
-    Returns:
-        str: Always an empty string — language values are non-fatal.
-    """
-    if value is not None:
-        logging.warning(
-            f"The feature '{name}' is a 'language' column populated by the annotation pipeline, "
-            f"not at record time. The provided value will be dropped."
-        )
-    return ""
-
-
 def validate_episode_buffer(episode_buffer: dict, total_episodes: int, features: dict) -> None:
     """Validate the episode buffer before it's written to disk.
 

src/lerobot/datasets/io_utils.py

@@ -31,10 +31,10 @@ from torchvision import transforms
 from lerobot.utils.io_utils import load_json, write_json
 from lerobot.utils.utils import SuppressProgressBars, flatten_dict, unflatten_dict
 
-from .language import LANGUAGE_COLUMNS
 from .utils import (
     DEFAULT_DATA_FILE_SIZE_IN_MB,
     DEFAULT_EPISODES_PATH,
+    DEFAULT_SUBTASKS_PATH,
     DEFAULT_TASKS_PATH,
     EPISODES_DIR,
     INFO_PATH,
@@ -186,6 +186,14 @@ def load_tasks(local_dir: Path) -> pandas.DataFrame:
     return tasks
 
 
+def load_subtasks(local_dir: Path) -> pandas.DataFrame | None:
+    """Load subtasks from subtasks.parquet if it exists."""
+    subtasks_path = local_dir / DEFAULT_SUBTASKS_PATH
+    if subtasks_path.exists():
+        return pd.read_parquet(subtasks_path)
+    return None
+
+
 def write_episodes(episodes: Dataset, local_dir: Path) -> None:
     """Write episode metadata to a parquet file in the LeRobot v3.0 format.
     This function writes episode-level metadata to a single parquet file.
@@ -257,13 +265,11 @@ def hf_transform_to_torch(items_dict: dict[str, list[Any]]) -> dict[str, list[to
         dict: The batch with items converted to torch tensors.
     """
     for key in items_dict:
-        if key in LANGUAGE_COLUMNS:
-            continue
         first_item = items_dict[key][0]
         if isinstance(first_item, PILImage.Image):
             to_tensor = transforms.ToTensor()
             items_dict[key] = [to_tensor(img) for img in items_dict[key]]
-        elif first_item is None or isinstance(first_item, dict):
+        elif first_item is None:
             pass
         else:
             items_dict[key] = [x if isinstance(x, str) else torch.tensor(x) for x in items_dict[key]]
@@ -298,9 +304,8 @@ def item_to_torch(item: dict) -> dict:
     Returns:
         dict: Dictionary with all tensor-like items converted to torch.Tensor.
     """
-    skip_keys = {"task", *LANGUAGE_COLUMNS}
     for key, val in item.items():
-        if isinstance(val, (np.ndarray | list)) and key not in skip_keys:
+        if isinstance(val, (np.ndarray | list)) and key not in ["task"]:
             # Convert numpy arrays and lists to torch tensors
             item[key] = torch.tensor(val)
     return item

src/lerobot/datasets/language.py

@@ -1,242 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from __future__ import annotations
-
-from typing import Literal
-
-import datasets
-import pyarrow as pa
-
-LANGUAGE_PERSISTENT = "language_persistent"
-LANGUAGE_EVENTS = "language_events"
-LANGUAGE_COLUMNS = (LANGUAGE_PERSISTENT, LANGUAGE_EVENTS)
-PERSISTENT_ROW_FIELDS = ("role", "content", "style", "timestamp", "camera", "tool_calls")
-EVENT_ROW_FIELDS = ("role", "content", "style", "camera", "tool_calls")
-
-CORE_STYLES = {
-    "subtask",
-    "plan",
-    "memory",
-    "motion",
-    "interjection",
-    "vqa",
-    "trace",
-    "task_aug",
-}
-# Project-local styles can be registered at import time by appending to
-# ``EXTENDED_STYLES`` before ``column_for_style`` is called. Anything added
-# here is treated as a known style alongside ``CORE_STYLES`` for resolver
-# validation. Empty by default — populate from a downstream module that
-# also extends ``PERSISTENT_STYLES`` or ``EVENT_ONLY_STYLES`` to declare
-# the new style's column.
-EXTENDED_STYLES: set[str] = set()
-STYLE_REGISTRY = CORE_STYLES | EXTENDED_STYLES
-
-PERSISTENT_STYLES = {"subtask", "plan", "memory", "motion", "task_aug"}
-EVENT_ONLY_STYLES = {"interjection", "vqa", "trace"}
-
-# Styles whose ``content`` is grounded in a specific camera view. Rows of these
-# styles MUST carry a non-null ``camera`` referencing an ``observation.images.*``
-# feature key. Rows of every other style MUST have ``camera=None``. ``motion``
-# is intentionally NOT in this set: motion primitives are described in
-# robot-frame (joint / Cartesian) terms, not pixel space, so they are
-# camera-agnostic. ``trace`` is the pixel-trajectory event style and IS
-# view-dependent. The ``camera`` field nevertheless lives on
-# ``PERSISTENT_ROW_FIELDS`` too so the schema, validator, and resolver
-# behave symmetrically across the two columns; persistent rows simply
-# always have ``camera=None`` in practice today.
-VIEW_DEPENDENT_STYLES = {"vqa", "trace"}
-
-LanguageColumn = Literal["language_persistent", "language_events"]
-
-
-def _json_arrow_type() -> pa.DataType:
-    """Return the Arrow JSON type, falling back to ``string`` on older pyarrow."""
-    return pa.json_() if hasattr(pa, "json_") else pa.string()
-
-
-def _json_feature() -> object:
-    """Return the HF ``datasets`` JSON feature, falling back to a string value."""
-    return datasets.Json() if hasattr(datasets, "Json") else datasets.Value("string")
-
-
-def language_persistent_row_arrow_type() -> pa.StructType:
-    """Return the Arrow struct type for a single persistent language row.
-
-    Persistent rows carry their own ``timestamp`` because they represent a state
-    that became active at a specific moment and remains active until superseded.
-    ``timestamp`` is ``float32`` to match the timestamp dtype LeRobotDataset
-    uses for frame data.
-    """
-    return pa.struct(
-        [
-            pa.field("role", pa.string(), nullable=False),
-            pa.field("content", pa.string(), nullable=True),
-            pa.field("style", pa.string(), nullable=True),
-            pa.field("timestamp", pa.float32(), nullable=False),
-            pa.field("camera", pa.string(), nullable=True),
-            pa.field("tool_calls", pa.list_(_json_arrow_type()), nullable=True),
-        ]
-    )
-
-
-def language_event_row_arrow_type() -> pa.StructType:
-    """Return the Arrow struct type for a single event language row.
-
-    Event rows have no ``timestamp`` field: each event is stored on the dataset
-    row whose frame timestamp is the event's firing time.
-    """
-    return pa.struct(
-        [
-            pa.field("role", pa.string(), nullable=False),
-            pa.field("content", pa.string(), nullable=True),
-            pa.field("style", pa.string(), nullable=True),
-            pa.field("camera", pa.string(), nullable=True),
-            pa.field("tool_calls", pa.list_(_json_arrow_type()), nullable=True),
-        ]
-    )
-
-
-def language_persistent_arrow_type() -> pa.ListType:
-    """Return the Arrow list type for the ``language_persistent`` column."""
-    return pa.list_(language_persistent_row_arrow_type())
-
-
-def language_events_arrow_type() -> pa.ListType:
-    """Return the Arrow list type for the ``language_events`` column."""
-    return pa.list_(language_event_row_arrow_type())
-
-
-def language_persistent_row_feature() -> dict[str, object]:
-    """Return the HF ``datasets`` feature mapping for a persistent language row."""
-    return {
-        "role": datasets.Value("string"),
-        "content": datasets.Value("string"),
-        "style": datasets.Value("string"),
-        "timestamp": datasets.Value("float32"),
-        "camera": datasets.Value("string"),
-        "tool_calls": datasets.List(_json_feature()),
-    }
-
-
-def language_event_row_feature() -> dict[str, object]:
-    """Return the HF ``datasets`` feature mapping for an event language row."""
-    return {
-        "role": datasets.Value("string"),
-        "content": datasets.Value("string"),
-        "style": datasets.Value("string"),
-        "camera": datasets.Value("string"),
-        "tool_calls": datasets.List(_json_feature()),
-    }
-
-
-def language_persistent_column_feature() -> datasets.List:
-    """Return the HF ``datasets`` feature for the ``language_persistent`` column."""
-    return datasets.List(language_persistent_row_feature())
-
-
-def language_events_column_feature() -> datasets.List:
-    """Return the HF ``datasets`` feature for the ``language_events`` column."""
-    return datasets.List(language_event_row_feature())
-
-
-def language_feature_info() -> dict[str, dict]:
-    """Return the ``info["features"]`` entries for both language columns."""
-    return {
-        LANGUAGE_PERSISTENT: {"dtype": "language", "shape": (1,), "names": None},
-        LANGUAGE_EVENTS: {"dtype": "language", "shape": (1,), "names": None},
-    }
-
-
-def is_language_column(key: str) -> bool:
-    """Return ``True`` if ``key`` is one of the dataset's language column names."""
-    return key in LANGUAGE_COLUMNS
-
-
-def is_view_dependent_style(style: str | None) -> bool:
-    """Return ``True`` if rows of ``style`` must be tagged with a ``camera`` key."""
-    return style in VIEW_DEPENDENT_STYLES
-
-
-def validate_camera_field(style: str | None, camera: str | None) -> None:
-    """Enforce the ``camera`` invariant: required iff ``style`` is view-dependent.
-
-    Raises ``ValueError`` if a view-dependent style is missing ``camera`` or if
-    a non-view-dependent style carries one. Pipeline writers and the validator
-    should call this on every emitted row.
-    """
-    if is_view_dependent_style(style):
-        if not camera:
-            raise ValueError(
-                f"Rows of view-dependent style {style!r} require a non-empty 'camera' "
-                f"field referencing an 'observation.images.*' feature key."
-            )
-    elif camera is not None:
-        raise ValueError(f"Rows of style {style!r} must have camera=None; got camera={camera!r}.")
-
-
-# --- Tool registry --------------------------------------------------------
-# Tools declared on a dataset live in ``meta/info.json["tools"]`` as a list
-# of OpenAI-style function schemas. The runtime / training stack reads them
-# through :class:`LeRobotDatasetMetadata.tools` (with these constants as
-# fallback when the dataset doesn't declare any). Implementations live
-# under :mod:`lerobot.tools` (one file per tool); see
-# ``docs/source/tools.mdx`` for the authoring guide.
-
-SAY_TOOL_SCHEMA: dict = {
-    "type": "function",
-    "function": {
-        "name": "say",
-        "description": "Speak a short utterance to the user via the TTS executor.",
-        "parameters": {
-            "type": "object",
-            "properties": {
-                "text": {
-                    "type": "string",
-                    "description": "The verbatim text to speak.",
-                }
-            },
-            "required": ["text"],
-        },
-    },
-}
-"""Canonical schema for the ``say`` tool emitted by the steerable
-annotation pipeline (PR 2 Module 2). Single source of truth — PR 2's
-writer, PR 3's runtime tool registry, and the dataset visualizer all
-import this constant rather than duplicating the dict."""
-
-DEFAULT_TOOLS: list[dict] = [SAY_TOOL_SCHEMA]
-"""Fallback tools list. Returned by ``LeRobotDatasetMetadata.tools``
-when ``meta/info.json["tools"]`` is unset, so unannotated datasets and
-chat-template consumers (``apply_chat_template(messages, tools=...)``)
-keep working out of the box."""
-
-
-def column_for_style(style: str | None) -> LanguageColumn:
-    """Map a language style to the column where rows of that style are stored.
-
-    Styles in :data:`PERSISTENT_STYLES` route to :data:`LANGUAGE_PERSISTENT`.
-    Styles in :data:`EVENT_ONLY_STYLES` and the implicit ``None`` style route
-    to :data:`LANGUAGE_EVENTS`.
-    """
-    if style is None:
-        return LANGUAGE_EVENTS
-    if style in PERSISTENT_STYLES:
-        return LANGUAGE_PERSISTENT
-    if style in EVENT_ONLY_STYLES:
-        return LANGUAGE_EVENTS
-    raise ValueError(f"Unknown language style: {style!r}")

src/lerobot/datasets/language_render.py

@@ -1,545 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from __future__ import annotations
-
-import copy
-import hashlib
-import re
-from collections.abc import Sequence
-from typing import Any
-
-from lerobot.configs.recipe import DEFAULT_BINDINGS, PLACEHOLDER_RE, TrainingRecipe
-from lerobot.utils.utils import unwrap_scalar
-
-from .language import LANGUAGE_PERSISTENT, column_for_style
-
-LanguageRow = dict[str, Any]
-RenderedMessages = dict[str, list[Any]]
-
-_RESOLVER_RE = re.compile(r"^(?P<name>[A-Za-z_][A-Za-z0-9_]*)\((?P<args>.*)\)$")
-
-
-def active_at(
-    t: float,
-    *,
-    persistent: Sequence[LanguageRow],
-    style: str | None = None,
-    role: str | None = None,
-    tool_name: str | None = None,
-    camera: str | None = None,
-) -> LanguageRow | None:
-    """Return the persistent row of ``style`` that is active at time ``t``.
-
-    A persistent row is "active" at ``t`` when its own ``timestamp`` is the
-    most recent one ``<= t`` for the given ``style``/``role``/``tool_name``/
-    ``camera`` selector. Only valid for persistent styles.
-    """
-    _validate_persistent_resolver("active_at", style)
-    matches = [
-        row
-        for row in _matching_rows(persistent, style=style, role=role, tool_name=tool_name, camera=camera)
-        if _timestamp(row) <= t
-    ]
-    if not matches:
-        return None
-    latest_ts = max(_timestamp(row) for row in matches)
-    return _select_one(
-        [row for row in matches if _timestamp(row) == latest_ts],
-        style=style,
-        role=role,
-        tool_name=tool_name,
-        camera=camera,
-    )
-
-
-EMITTED_AT_TOLERANCE_S = 0.1
-"""Half-window for matching persistent rows to a frame timestamp in
-``emitted_at``. Persistent timestamps come from parquet (float32) and ``t``
-is also a float32 from parquet, so in the ideal hot path an exact match
-would suffice — but any caller that derives ``t`` arithmetically (e.g.
-``frame_idx / fps``) breaks bit-equality. A 0.1 s tolerance covers
-common arithmetic drift without admitting frames that are visibly far
-apart at typical control rates (30–100 Hz). This does mean two persistent
-rows of the same selector emitted within 0.1 s of each other cannot be
-told apart by ``emitted_at`` — acceptable because persistent annotations
-(subtask / plan / memory transitions) change on a human-action timescale,
-not at the camera frame rate."""
-
-
-def emitted_at(
-    t: float,
-    *,
-    persistent: Sequence[LanguageRow],
-    events: Sequence[LanguageRow],
-    style: str | None = None,
-    role: str | None = None,
-    tool_name: str | None = None,
-    camera: str | None = None,
-) -> LanguageRow | None:
-    """Return the row of ``style`` emitted at exactly time ``t``.
-
-    For persistent styles, this matches persistent rows whose own ``timestamp``
-    is within ``EMITTED_AT_TOLERANCE_S`` of ``t`` (see that constant for why
-    we use a tolerance instead of bit-equality). For event styles, the
-    ``events`` list is assumed to come from the dataset row at frame ``t``
-    (event rows carry no timestamp of their own), so all matching event rows
-    are considered emitted at ``t``. ``camera`` filters by the row's
-    ``camera`` field — required to disambiguate when multiple view-dependent
-    rows share ``(t, role)`` across cameras.
-    """
-    if column_for_style(style) == LANGUAGE_PERSISTENT:
-        matches = [
-            row
-            for row in _matching_rows(persistent, style=style, role=role, tool_name=tool_name, camera=camera)
-            if abs(_timestamp(row) - t) <= EMITTED_AT_TOLERANCE_S
-        ]
-    else:
-        matches = _matching_rows(events, style=style, role=role, tool_name=tool_name, camera=camera)
-    return _select_one(matches, style=style, role=role, tool_name=tool_name, camera=camera)
-
-
-def nth_prev(
-    t: float,
-    *,
-    persistent: Sequence[LanguageRow],
-    style: str | None = None,
-    offset: int = 1,
-    role: str | None = None,
-    tool_name: str | None = None,
-    camera: str | None = None,
-) -> LanguageRow | None:
-    """Return the persistent row that was active ``offset`` steps before ``t``.
-
-    Walks back through chronologically sorted persistent rows of ``style``
-    (filtered by optional ``role``/``tool_name``/``camera``) and returns the
-    one ``offset`` positions before the row active at ``t``. Only valid for
-    persistent styles.
-    """
-    return _nth_relative("nth_prev", t, persistent, style, -offset, role, tool_name, camera)
-
-
-def nth_next(
-    t: float,
-    *,
-    persistent: Sequence[LanguageRow],
-    style: str | None = None,
-    offset: int = 1,
-    role: str | None = None,
-    tool_name: str | None = None,
-    camera: str | None = None,
-) -> LanguageRow | None:
-    """Return the persistent row that becomes active ``offset`` steps after ``t``.
-
-    Walks forward through chronologically sorted persistent rows of ``style``
-    (filtered by optional ``role``/``tool_name``/``camera``) and returns the
-    one ``offset`` positions after the row active at ``t``. Only valid for
-    persistent styles.
-    """
-    return _nth_relative("nth_next", t, persistent, style, offset, role, tool_name, camera)
-
-
-def render_sample(
-    *,
-    recipe: TrainingRecipe,
-    persistent: Sequence[LanguageRow] | None,
-    events: Sequence[LanguageRow] | None,
-    t: float,
-    sample_idx: int,
-    task: str | None = None,
-    dataset_ctx: Any | None = None,
-) -> RenderedMessages | None:
-    """Render the chat-style messages for a single dataset sample.
-
-    Resolves the recipe's bindings against ``persistent`` and ``events`` rows
-    at frame timestamp ``t``, then expands the recipe's message templates.
-    Returns ``None`` if the resolved sample contains no target message.
-    """
-    persistent_rows = _normalize_rows(persistent or [])
-    event_rows = _normalize_rows(events or [])
-    selected_recipe = _select_recipe(recipe, sample_idx)
-    bindings = _resolve_bindings(
-        selected_recipe,
-        persistent=persistent_rows,
-        events=event_rows,
-        t=t,
-        sample_idx=sample_idx,
-        task=task,
-        dataset_ctx=dataset_ctx,
-    )
-    return _render_message_recipe(selected_recipe, bindings)
-
-
-def _select_recipe(recipe: TrainingRecipe, sample_idx: int) -> TrainingRecipe:
-    """Pick a deterministic blend component for ``sample_idx`` (or return ``recipe``)."""
-    if recipe.blend is None:
-        return recipe
-
-    total_weight = sum(component.weight or 0.0 for component in recipe.blend.values())
-    if total_weight <= 0:
-        raise ValueError("Blend weights must sum to a positive value.")
-
-    digest = hashlib.blake2b(str(sample_idx).encode(), digest_size=8).digest()
-    draw = int.from_bytes(digest, "big") / 2**64 * total_weight
-    cumulative = 0.0
-    last_component: TrainingRecipe | None = None
-    for component in recipe.blend.values():
-        last_component = component
-        cumulative += component.weight or 0.0
-        if draw < cumulative:
-            return component
-    assert last_component is not None
-    return last_component
-
-
-def _resolve_bindings(
-    recipe: TrainingRecipe,
-    *,
-    persistent: Sequence[LanguageRow],
-    events: Sequence[LanguageRow],
-    t: float,
-    sample_idx: int,
-    task: str | None,
-    dataset_ctx: Any | None,
-) -> dict[str, LanguageRow | str | None]:
-    """Resolve every binding in ``recipe`` (plus ``task``) at time ``t``."""
-    bindings: dict[str, LanguageRow | str | None] = {
-        "task": _resolve_task(task, dataset_ctx, persistent=persistent, sample_idx=sample_idx),
-    }
-    specs = {**DEFAULT_BINDINGS, **(recipe.bindings or {})}
-    for name, spec in specs.items():
-        bindings[name] = _resolve_spec(spec, persistent=persistent, events=events, t=t)
-    return bindings
-
-
-def _resolve_task(
-    task: str | None,
-    dataset_ctx: Any | None,
-    *,
-    persistent: Sequence[LanguageRow] = (),
-    sample_idx: int = 0,
-) -> str | None:
-    """Return the task string for ``sample_idx``.
-
-    Resolution order:
-
-    1. Explicit ``task`` override (caller-supplied) wins.
-    2. If ``persistent`` contains rows of style ``task_aug`` (role=user),
-       deterministically pick one by ``sample_idx`` so each frame of an
-       episode rotates through the available rephrasings across an epoch.
-       This realizes Xiao 2022 / CAST-style task-prompt diversity without
-       changing ``meta/tasks.parquet`` and without forcing recipes to opt
-       in: ``${task}`` automatically picks a rephrasing when one exists,
-       and falls back to the canonical task otherwise. Recipes that want
-       the literal canonical task can override the binding.
-    3. Otherwise read the canonical task from ``dataset_ctx`` (which is
-       backed by ``meta/tasks.parquet``).
-    """
-    if task is not None:
-        return task
-
-    aug_rows = [r for r in persistent if r.get("style") == "task_aug" and r.get("role") == "user"]
-    if aug_rows:
-        # Deterministic, blake2b-based pick keyed on sample_idx so the
-        # rotation is reproducible across runs (Python's built-in ``hash``
-        # is process-randomized).
-        digest = hashlib.blake2b(f"task_aug:{sample_idx}".encode(), digest_size=8).digest()
-        idx = int.from_bytes(digest, "big") % len(aug_rows)
-        chosen = aug_rows[idx].get("content")
-        if chosen:
-            return str(chosen)
-
-    if dataset_ctx is None:
-        return None
-    if isinstance(dataset_ctx, dict):
-        return dataset_ctx.get("task")
-    return getattr(dataset_ctx, "task", None)
-
-
-def _resolve_spec(
-    spec: str,
-    *,
-    persistent: Sequence[LanguageRow],
-    events: Sequence[LanguageRow],
-    t: float,
-) -> LanguageRow | None:
-    """Parse a single binding's resolver expression and dispatch to its function."""
-    match = _RESOLVER_RE.match(spec.strip())
-    if match is None:
-        raise ValueError(f"Invalid resolver expression: {spec!r}")
-    name = match.group("name")
-    kwargs = _parse_resolver_args(match.group("args"))
-    kwargs.pop("t_arg", None)
-
-    if name == "emitted_at":
-        return emitted_at(t, persistent=persistent, events=events, **kwargs)
-    if name == "active_at":
-        return active_at(t, persistent=persistent, **kwargs)
-    if name == "nth_prev":
-        return nth_prev(t, persistent=persistent, **kwargs)
-    if name == "nth_next":
-        return nth_next(t, persistent=persistent, **kwargs)
-    raise ValueError(f"Unknown language resolver: {name!r}")
-
-
-def _parse_resolver_args(args: str) -> dict[str, Any]:
-    """Parse a comma-separated resolver argument list into a kwargs dict."""
-    kwargs: dict[str, Any] = {}
-    if not args.strip():
-        return kwargs
-
-    parts = [part.strip() for part in args.split(",") if part.strip()]
-    for part in parts:
-        if part == "t":
-            kwargs["t_arg"] = True
-            continue
-        if "=" not in part:
-            raise ValueError(f"Invalid resolver argument: {part!r}")
-        key, value = (item.strip() for item in part.split("=", 1))
-        if key == "offset":
-            kwargs[key] = int(value)
-        else:
-            kwargs[key] = value.strip("\"'")
-    return kwargs
-
-
-def _render_message_recipe(
-    recipe: TrainingRecipe,
-    bindings: dict[str, LanguageRow | str | None],
-) -> RenderedMessages | None:
-    """Expand ``recipe.messages`` into rendered chat messages using ``bindings``."""
-    assert recipe.messages is not None
-    messages: list[dict[str, Any]] = []
-    streams: list[str | None] = []
-    target_indices: list[int] = []
-
-    for turn in recipe.messages:
-        if turn.if_present is not None and bindings.get(turn.if_present) is None:
-            continue
-
-        message = {"role": turn.role}
-        if turn.content is not None:
-            message["content"] = _render_content(turn.content, bindings)
-
-        if turn.tool_calls_from is not None:
-            row = bindings.get(turn.tool_calls_from)
-            tool_calls = row.get("tool_calls") if isinstance(row, dict) else None
-            if tool_calls:
-                message["tool_calls"] = copy.deepcopy(tool_calls)
-
-        message_idx = len(messages)
-        messages.append(message)
-        streams.append(turn.stream)
-        if turn.target:
-            target_indices.append(message_idx)
-
-    if not target_indices:
-        return None
-
-    rendered = {
-        "messages": messages,
-        "message_streams": streams,
-        "target_message_indices": target_indices,
-    }
-    _validate_rendered(rendered)
-    return rendered
-
-
-def _render_content(
-    content: str | list[dict[str, Any]],
-    bindings: dict[str, LanguageRow | str | None],
-) -> str | list[dict[str, Any]]:
-    """Substitute bindings into a string or each string field of multimodal blocks."""
-    if isinstance(content, str):
-        return _substitute(content, bindings)
-
-    rendered_blocks = []
-    for block in content:
-        rendered_block = copy.deepcopy(block)
-        for key, value in rendered_block.items():
-            if isinstance(value, str):
-                rendered_block[key] = _substitute(value, bindings)
-        rendered_blocks.append(rendered_block)
-    return rendered_blocks
-
-
-def _substitute(template: str, bindings: dict[str, LanguageRow | str | None]) -> str:
-    """Replace ``${name}`` placeholders in ``template`` with their bound values."""
-
-    def replace(match: re.Match[str]) -> str:
-        """Resolve a single ``${name}`` match to its bound string value."""
-        name = match.group(1)
-        if name not in bindings:
-            raise ValueError(f"Unknown template binding: {name!r}")
-        value = bindings[name]
-        if value is None:
-            return ""
-        if isinstance(value, dict):
-            content = value.get("content")
-            return "" if content is None else str(content)
-        return str(value)
-
-    return PLACEHOLDER_RE.sub(replace, template)
-
-
-def _validate_rendered(rendered: RenderedMessages) -> None:
-    """Sanity-check the rendered output for stream/target alignment."""
-    messages = rendered["messages"]
-    streams = rendered["message_streams"]
-    target_indices = rendered["target_message_indices"]
-
-    if len(streams) != len(messages):
-        raise ValueError("message_streams must be aligned with messages.")
-    if not target_indices:
-        raise ValueError("Rendered samples must contain at least one target message.")
-    for idx in target_indices:
-        if idx < 0 or idx >= len(messages):
-            raise ValueError(f"Target message index {idx} is out of bounds.")
-    # ``stream`` is enforced non-None at MessageTurn construction time
-    # (see ``MessageTurn.__post_init__``), so a missing stream here would
-    # mean the dataclass invariant was bypassed; no need to re-check.
-
-
-def _nth_relative(
-    name: str,
-    t: float,
-    persistent: Sequence[LanguageRow],
-    style: str | None,
-    offset: int,
-    role: str | None,
-    tool_name: str | None,
-    camera: str | None,
-) -> LanguageRow | None:
-    """Shared body for ``nth_prev`` / ``nth_next`` with signed ``offset``."""
-    _validate_persistent_resolver(name, style)
-    if abs(offset) < 1:
-        raise ValueError(f"{name} offset must be non-zero.")
-
-    rows = sorted(
-        _matching_rows(persistent, style=style, role=role, tool_name=tool_name, camera=camera),
-        key=_row_sort_key,
-    )
-    if not rows:
-        return None
-
-    anchor_idx = None
-    for idx, row in enumerate(rows):
-        if _timestamp(row) <= t:
-            anchor_idx = idx
-        else:
-            break
-
-    target_idx = (offset - 1 if offset > 0 else None) if anchor_idx is None else anchor_idx + offset
-
-    if target_idx is None or target_idx < 0 or target_idx >= len(rows):
-        return None
-    return rows[target_idx]
-
-
-def _validate_persistent_resolver(name: str, style: str | None) -> None:
-    """Reject calls with missing or event-only ``style`` for persistent resolvers."""
-    if style is None:
-        raise ValueError(f"{name} requires a persistent style.")
-    if column_for_style(style) != LANGUAGE_PERSISTENT:
-        raise ValueError(f"{name} cannot be used with event-only style {style!r}.")
-
-
-def _matching_rows(
-    rows: Sequence[LanguageRow],
-    *,
-    style: str | None,
-    role: str | None,
-    tool_name: str | None,
-    camera: str | None,
-) -> list[LanguageRow]:
-    """Return ``rows`` filtered by optional ``style``/``role``/``tool_name``/``camera`` selectors."""
-    return [
-        row
-        for row in rows
-        if (style is None or row.get("style") == style)
-        and (role is None or row.get("role") == role)
-        and (tool_name is None or _row_has_tool_name(row, tool_name))
-        and (camera is None or row.get("camera") == camera)
-    ]
-
-
-def _select_one(
-    rows: Sequence[LanguageRow],
-    *,
-    style: str | None,
-    role: str | None,
-    tool_name: str | None,
-    camera: str | None,
-) -> LanguageRow | None:
-    """Return the single matching row, or raise if the resolver is ambiguous.
-
-    Multiple matches always raise — even when the caller already passed
-    some selectors — because remaining ambiguity means the data has
-    several rows that look identical to the resolver and the caller
-    needs to pin down a specific one (e.g. add ``camera=...`` for VQA
-    rows shared across cameras).
-    """
-    if not rows:
-        return None
-    if len(rows) > 1:
-        raise ValueError(
-            f"Ambiguous resolver for style={style!r} role={role!r} "
-            f"tool_name={tool_name!r} camera={camera!r}: {len(rows)} matching rows. "
-            f"Add a selector that distinguishes them."
-        )
-    return rows[0]
-
-
-def _row_sort_key(row: LanguageRow) -> tuple[float, str, str]:
-    """Stable sort key for both persistent and event rows.
-
-    Event rows lack ``timestamp`` (it is implicit in the frame), so default
-    to ``0.0`` — within a single frame all event rows share the same sort
-    bucket and are tiebroken by ``(style, role)``.
-    """
-    timestamp = row.get("timestamp")
-    ts = float(unwrap_scalar(timestamp)) if timestamp is not None else 0.0
-    return (ts, row.get("style") or "", row.get("role") or "")
-
-
-def _timestamp(row: LanguageRow) -> float:
-    """Extract a row's ``timestamp`` as a Python float (unwrapping numpy scalars)."""
-    return float(unwrap_scalar(row["timestamp"]))
-
-
-def _row_has_tool_name(row: LanguageRow, tool_name: str) -> bool:
-    """Return ``True`` if any of the row's tool calls invokes ``tool_name``."""
-    for tool_call in row.get("tool_calls") or []:
-        if isinstance(tool_call, str):
-            continue
-        function = tool_call.get("function") if isinstance(tool_call, dict) else None
-        if isinstance(function, dict) and function.get("name") == tool_name:
-            return True
-    return False
-
-
-def _normalize_rows(rows: Sequence[Any]) -> list[LanguageRow]:
-    """Convert pyarrow scalars / mappings into a fresh list of plain dict rows."""
-    normalized = []
-    for row in rows:
-        if row is None:
-            continue
-        if hasattr(row, "as_py"):
-            row = row.as_py()
-        if not isinstance(row, dict):
-            raise TypeError(f"Language rows must be dictionaries, got {type(row).__name__}.")
-        normalized.append(dict(row))
-    return normalized

src/lerobot/datasets/lerobot_dataset.py

@@ -24,7 +24,6 @@ import torch.utils
 from huggingface_hub import HfApi, snapshot_download
 from huggingface_hub.errors import RevisionNotFoundError
 
-from lerobot.configs import VideoEncoderConfig
 from lerobot.utils.constants import HF_LEROBOT_HUB_CACHE
 
 from .dataset_metadata import CODEBASE_VERSION, LeRobotDatasetMetadata
@@ -37,7 +36,8 @@ from .utils import (
 )
 from .video_utils import (
     StreamingVideoEncoder,
-    get_safe_default_video_backend,
+    get_safe_default_codec,
+    resolve_vcodec,
 )
 
 logger = logging.getLogger(__name__)
@@ -59,10 +59,10 @@ class LeRobotDataset(torch.utils.data.Dataset):
         video_backend: str | None = None,
         return_uint8: bool = False,
         batch_encoding_size: int = 1,
-        camera_encoder: VideoEncoderConfig | None = None,
-        encoder_threads: int | None = None,
+        vcodec: str = "libsvtav1",
         streaming_encoding: bool = False,
         encoder_queue_maxsize: int = 30,
+        encoder_threads: int | None = None,
     ):
         """
         2 modes are available for instantiating this class, depending on 2 different use cases:
@@ -183,15 +183,16 @@ class LeRobotDataset(torch.utils.data.Dataset):
                 You can also use the 'pyav' decoder used by Torchvision, which used to be the default option, or 'video_reader' which is another decoder of Torchvision.
             batch_encoding_size (int, optional): Number of episodes to accumulate before batch encoding videos.
                 Set to 1 for immediate encoding (default), or higher for batched encoding. Defaults to 1.
-            camera_encoder (VideoEncoderConfig | None, optional): Video encoder settings for cameras
-                (codec, quality, etc.). When ``None``, :func:`~lerobot.configs.video.camera_encoder_defaults`
-                is used by the writer.
-            encoder_threads (int | None, optional): Number of encoder threads (global). ``None`` lets the
-                codec decide.
+            vcodec (str, optional): Video codec for encoding videos during recording. Options: 'h264', 'hevc',
+                'libsvtav1', 'auto', or hardware-specific codecs like 'h264_videotoolbox', 'h264_nvenc'.
+                Defaults to 'libsvtav1'. Use 'auto' to auto-detect the best available hardware encoder.
             streaming_encoding (bool, optional): If True, encode video frames in real-time during capture
                 instead of writing PNG images first. This makes save_episode() near-instant. Defaults to False.
             encoder_queue_maxsize (int, optional): Maximum number of frames to buffer per camera when using
                 streaming encoding. Defaults to 30 (~1s at 30fps).
+            encoder_threads (int | None, optional): Number of threads per encoder instance. None lets the
+                codec auto-detect (default). Lower values reduce CPU usage per encoder. Maps to 'lp' (via svtav1-params) for
+                libsvtav1 and 'threads' for h264/hevc.
 
         Note:
             Write-mode parameters (``streaming_encoding``, ``batch_encoding_size``) passed to
@@ -206,9 +207,10 @@ class LeRobotDataset(torch.utils.data.Dataset):
         self.delta_timestamps = delta_timestamps
         self.tolerance_s = tolerance_s
         self.revision = revision if revision else CODEBASE_VERSION
-        self._video_backend = video_backend if video_backend else get_safe_default_video_backend()
+        self._video_backend = video_backend if video_backend else get_safe_default_codec()
         self._return_uint8 = return_uint8
         self._batch_encoding_size = batch_encoding_size
+        self._vcodec = resolve_vcodec(vcodec)
         self._encoder_threads = encoder_threads
 
         if self._requested_root is not None:
@@ -271,15 +273,12 @@ class LeRobotDataset(torch.utils.data.Dataset):
             streaming_enc = None
             if streaming_encoding and len(self.meta.video_keys) > 0:
                 streaming_enc = self._build_streaming_encoder(
-                    self.meta.fps,
-                    camera_encoder,
-                    encoder_queue_maxsize,
-                    encoder_threads,
+                    self.meta.fps, self._vcodec, encoder_queue_maxsize, encoder_threads
                 )
             self.writer = DatasetWriter(
                 meta=self.meta,
                 root=self.root,
-                camera_encoder=camera_encoder,
+                vcodec=self._vcodec,
                 encoder_threads=encoder_threads,
                 batch_encoding_size=batch_encoding_size,
                 streaming_encoder=streaming_enc,
@@ -321,13 +320,17 @@ class LeRobotDataset(torch.utils.data.Dataset):
     @staticmethod
     def _build_streaming_encoder(
         fps: int,
-        camera_encoder: VideoEncoderConfig | None,
+        vcodec: str,
         encoder_queue_maxsize: int,
         encoder_threads: int | None,
     ) -> StreamingVideoEncoder:
         return StreamingVideoEncoder(
             fps=fps,
-            camera_encoder=camera_encoder,
+            vcodec=vcodec,
+            pix_fmt="yuv420p",
+            g=2,
+            crf=30,
+            preset=None,
             queue_maxsize=encoder_queue_maxsize,
             encoder_threads=encoder_threads,
         )
@@ -644,7 +647,7 @@ class LeRobotDataset(torch.utils.data.Dataset):
         image_writer_threads: int = 0,
         video_backend: str | None = None,
         batch_encoding_size: int = 1,
-        camera_encoder: VideoEncoderConfig | None = None,
+        vcodec: str = "libsvtav1",
         metadata_buffer_size: int = 10,
         streaming_encoding: bool = False,
         encoder_queue_maxsize: int = 30,
@@ -675,20 +678,20 @@ class LeRobotDataset(torch.utils.data.Dataset):
             video_backend: Video decoding backend (used when reading back).
             batch_encoding_size: Number of episodes to accumulate before
                 batch-encoding videos. ``1`` means encode immediately.
-            camera_encoder: Video encoder settings for cameras (codec, quality, etc.).
-                When ``None``, :func:`~lerobot.configs.video.camera_encoder_defaults` is used.
-            encoder_threads: Number of encoder threads (global). ``None``
-                lets the codec decide.
+            vcodec: Video codec for encoding. Options include ``'libsvtav1'``,
+                ``'h264'``, ``'hevc'``, ``'auto'``.
             metadata_buffer_size: Number of episode metadata records to buffer
                 before flushing to parquet.
             streaming_encoding: If ``True``, encode video frames in real-time
                 during capture instead of writing images first.
             encoder_queue_maxsize: Max buffered frames per camera when using
                 streaming encoding.
+            encoder_threads: Threads per encoder instance. ``None`` for auto.
 
         Returns:
             A new :class:`LeRobotDataset` in write mode.
         """
+        vcodec = resolve_vcodec(vcodec)
         obj = cls.__new__(cls)
         obj.meta = LeRobotDatasetMetadata.create(
             repo_id=repo_id,
@@ -709,23 +712,23 @@ class LeRobotDataset(torch.utils.data.Dataset):
         obj.image_transforms = None
         obj.delta_timestamps = None
         obj.episodes = None
-        obj._video_backend = video_backend if video_backend is not None else get_safe_default_video_backend()
+        obj._video_backend = video_backend if video_backend is not None else get_safe_default_codec()
         obj._return_uint8 = False
         obj._batch_encoding_size = batch_encoding_size
+        obj._vcodec = vcodec
         obj._encoder_threads = encoder_threads
 
         # Reader is lazily created on first access (write-only mode)
         obj.reader = None
 
+        # Create writer
         streaming_enc = None
         if streaming_encoding and len(obj.meta.video_keys) > 0:
-            streaming_enc = cls._build_streaming_encoder(
-                fps, camera_encoder, encoder_queue_maxsize, encoder_threads
-            )
+            streaming_enc = cls._build_streaming_encoder(fps, vcodec, encoder_queue_maxsize, encoder_threads)
         obj.writer = DatasetWriter(
             meta=obj.meta,
             root=obj.root,
-            camera_encoder=camera_encoder,
+            vcodec=vcodec,
             encoder_threads=encoder_threads,
             batch_encoding_size=batch_encoding_size,
             streaming_encoder=streaming_enc,
@@ -748,12 +751,12 @@ class LeRobotDataset(torch.utils.data.Dataset):
         force_cache_sync: bool = False,
         video_backend: str | None = None,
         batch_encoding_size: int = 1,
-        camera_encoder: VideoEncoderConfig | None = None,
-        encoder_threads: int | None = None,
+        vcodec: str = "libsvtav1",
         image_writer_processes: int = 0,
         image_writer_threads: int = 0,
         streaming_encoding: bool = False,
         encoder_queue_maxsize: int = 30,
+        encoder_threads: int | None = None,
     ) -> "LeRobotDataset":
         """Resume recording on an existing dataset.
 
@@ -776,15 +779,13 @@ class LeRobotDataset(torch.utils.data.Dataset):
             video_backend: Video decoding backend for reading back data.
             batch_encoding_size: Number of episodes to accumulate before
                 batch-encoding videos.
-            camera_encoder: Video encoder settings for cameras (codec, quality, etc.).
-                When ``None``, :func:`~lerobot.configs.video.camera_encoder_defaults` is used.
-            encoder_threads: Number of encoder threads (global). ``None``
-                lets the codec decide.
+            vcodec: Video codec for encoding.
             image_writer_processes: Subprocesses for async image writing.
             image_writer_threads: Threads for async image writing.
             streaming_encoding: If ``True``, encode video in real-time during
                 capture.
             encoder_queue_maxsize: Max buffered frames per camera for streaming.
+            encoder_threads: Threads per encoder instance. ``None`` for auto.
 
         Returns:
             A :class:`LeRobotDataset` in write mode, ready to append episodes.
@@ -795,6 +796,7 @@ class LeRobotDataset(torch.utils.data.Dataset):
                 "Writing into the revision-safe Hub snapshot cache (used when root=None) would corrupt "
                 "the shared cache. Please provide a local directory path."
             )
+        vcodec = resolve_vcodec(vcodec)
         obj = cls.__new__(cls)
         obj.repo_id = repo_id
         obj._requested_root = Path(root)
@@ -803,9 +805,11 @@ class LeRobotDataset(torch.utils.data.Dataset):
         obj.image_transforms = None
         obj.delta_timestamps = None
         obj.episodes = None
-        obj._video_backend = video_backend if video_backend else get_safe_default_video_backend()
+        obj._video_backend = video_backend if video_backend else get_safe_default_codec()
         obj._return_uint8 = False
         obj._batch_encoding_size = batch_encoding_size
+        obj._vcodec = vcodec
+        obj._encoder_threads = encoder_threads
 
         if obj._requested_root is not None:
             obj._requested_root.mkdir(exist_ok=True, parents=True)
@@ -814,22 +818,21 @@ class LeRobotDataset(torch.utils.data.Dataset):
         obj.meta = LeRobotDatasetMetadata(
             obj.repo_id, obj._requested_root, obj.revision, force_cache_sync=force_cache_sync
         )
-
-        obj._encoder_threads = encoder_threads
         obj.root = obj.meta.root
 
         # Reader is lazily created on first access (write-only mode)
         obj.reader = None
 
+        # Create writer for appending
         streaming_enc = None
         if streaming_encoding and len(obj.meta.video_keys) > 0:
             streaming_enc = cls._build_streaming_encoder(
-                obj.meta.fps, camera_encoder, encoder_queue_maxsize, encoder_threads
+                obj.meta.fps, vcodec, encoder_queue_maxsize, encoder_threads
             )
         obj.writer = DatasetWriter(
             meta=obj.meta,
             root=obj.root,
-            camera_encoder=camera_encoder,
+            vcodec=vcodec,
             encoder_threads=encoder_threads,
             batch_encoding_size=batch_encoding_size,
             streaming_encoder=streaming_enc,

src/lerobot/datasets/pyav_utils.py

@@ -1,174 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-"""PyAV-based compatibility checks for :class:`VideoEncoderConfig`.
-
-Centralises all :mod:`av` introspection of the bundled FFmpeg build.
-Checks degrade to a no-op when the target codec isn't available locally.
-"""
-
-import functools
-import logging
-from typing import Any
-
-import av
-
-logger = logging.getLogger(__name__)
-
-FFMPEG_NUMERIC_OPTION_TYPES = ("INT", "INT64", "UINT64", "FLOAT", "DOUBLE")
-FFMPEG_INTEGER_OPTION_TYPES = ("INT", "INT64", "UINT64")
-
-
-@functools.cache
-def get_codec(vcodec: str) -> av.codec.Codec | None:
-    """PyAV write-mode ``Codec`` for *vcodec*, or ``None`` if unavailable."""
-    try:
-        return av.codec.Codec(vcodec, "w")
-    except Exception:
-        return None
-
-
-@functools.cache
-def _get_codec_options_by_name(vcodec: str) -> dict[str, av.option.Option]:
-    """Private-option name → PyAV ``Option`` for *vcodec* (empty if unavailable)."""
-    codec = get_codec(vcodec)
-    if codec is None:
-        return {}
-    return {opt.name: opt for opt in codec.descriptor.options}
-
-
-@functools.cache
-def _get_codec_video_formats(vcodec: str) -> tuple[str, ...]:
-    """Pixel formats accepted by *vcodec* in PyAV's preferred order (empty if unknown)."""
-    codec = get_codec(vcodec)
-    if codec is None:
-        return ()
-    return tuple(fmt.name for fmt in (codec.video_formats or []))
-
-
-def detect_available_encoders_pyav(encoders: list[str] | str) -> list[str]:
-    """Return the subset of *encoders* available as video encoders in the local FFmpeg build.
-
-    Each name is probed directly via :func:`get_codec`; input order is preserved.
-    """
-    if isinstance(encoders, str):
-        encoders = [encoders]
-
-    available: list[str] = []
-    for name in encoders:
-        codec = get_codec(name)
-        if codec is not None and codec.type == "video":
-            available.append(name)
-        else:
-            logger.debug("encoder '%s' not available as video encoder", name)
-    return available
-
-
-def _check_option_value(vcodec: str, label: str, value: Any, opt: av.option.Option) -> None:
-    """Range-check numeric *value* and choice-check string *value* against *opt*."""
-    type_name = opt.type.name
-    if type_name in FFMPEG_NUMERIC_OPTION_TYPES:
-        if isinstance(value, bool):
-            raise ValueError(
-                f"{label}={value!r} is not numeric; codec {vcodec!r} expects a number for this option."
-            )
-        elif isinstance(value, str):
-            try:
-                num_val = float(value)
-            except ValueError as e:
-                raise ValueError(
-                    f"{label}={value!r} is not numeric; codec {vcodec!r} expects a number for this option."
-                ) from e
-        elif isinstance(value, (float, int)):
-            num_val = value
-        else:
-            raise ValueError(
-                f"{label}={value!r} is not numeric; codec {vcodec!r} expects a number for this option."
-            )
-
-        # Check integer type compatibility
-        if type_name in FFMPEG_INTEGER_OPTION_TYPES and not num_val.is_integer():
-            raise ValueError(
-                f"{label}={num_val!r} must be an integer for codec {vcodec!r} "
-                f"(FFmpeg option {opt.name!r} is {type_name}); float values are not allowed."
-            )
-
-        # Check numeric range compatibility
-        lo, hi = float(opt.min), float(opt.max)
-        if lo < hi and not (lo <= num_val <= hi):
-            raise ValueError(
-                f"{label}={num_val} is out of range for codec {vcodec!r}; must be in [{lo}, {hi}]"
-            )
-
-    elif type_name == "STRING":
-        if isinstance(value, bool):
-            raise ValueError(f"{label}={value!r} is not a valid string value for codec {vcodec!r}.")
-        if isinstance(value, str):
-            str_val = value
-        elif isinstance(value, (int, float)):
-            str_val = str(value)
-        else:
-            raise ValueError(f"{label}={value!r} has unsupported type for STRING option on codec {vcodec!r}")
-
-        # Check string choice compatibility
-        choices = [c.name for c in (opt.choices or [])]
-        if choices and str_val not in choices:
-            raise ValueError(
-                f"{label}={str_val!r} is not a supported choice for codec "
-                f"{vcodec!r}; valid choices: {choices}"
-            )
-    else:
-        return
-
-
-def _check_pixel_format(vcodec: str, pix_fmt: str) -> None:
-    formats = _get_codec_video_formats(vcodec)
-    if formats and pix_fmt not in formats:
-        raise ValueError(
-            f"pix_fmt={pix_fmt!r} is not supported by codec {vcodec!r}; "
-            f"supported pixel formats: {list(formats)}"
-        )
-
-
-def _check_codec_options(vcodec: str, codec_options: dict[str, Any]) -> None:
-    """Validate merged encoder options (typed) against the codec's published AVOptions."""
-    supported_options = _get_codec_options_by_name(vcodec)
-    for key, value in codec_options.items():
-        # GOP size is not a codec-specific option, it has to be validated separately.
-        if key == "g":
-            if isinstance(value, bool) or not isinstance(value, int) or value < 1:
-                raise ValueError(f"g={value!r} must be a positive integer for codec {vcodec!r}")
-            continue
-        if key not in supported_options:
-            continue
-        _check_option_value(vcodec, key, value, supported_options[key])
-
-
-def check_video_encoder_parameters_pyav(vcodec: str, pix_fmt: str, codec_options: dict[str, Any]) -> None:
-    """Verify *config* is compatible with the bundled FFmpeg build.
-
-    Checks pixel format, abstract tuning-field compatibility, and each merged
-    encoder option from :meth:`~lerobot.configs.video.VideoEncoderConfig.get_codec_options`
-    against PyAV (including numeric ``extra_options`` present in that dict).
-    No-op when ``config.vcodec`` isn't in the local FFmpeg build.
-
-    Raises:
-        ValueError: on the first incompatibility encountered.
-    """
-    options = _get_codec_options_by_name(vcodec)
-    if not options:
-        raise ValueError(f"Codec {vcodec!r} is not available in the bundled FFmpeg build")
-    _check_pixel_format(vcodec, pix_fmt)
-    _check_codec_options(vcodec, codec_options)

src/lerobot/datasets/utils.py

@@ -88,6 +88,7 @@ VIDEO_DIR = "videos"
 
 CHUNK_FILE_PATTERN = "chunk-{chunk_index:03d}/file-{file_index:03d}"
 DEFAULT_TASKS_PATH = "meta/tasks.parquet"
+DEFAULT_SUBTASKS_PATH = "meta/subtasks.parquet"
 DEFAULT_EPISODES_PATH = EPISODES_DIR + "/" + CHUNK_FILE_PATTERN + ".parquet"
 DEFAULT_DATA_PATH = DATA_DIR + "/" + CHUNK_FILE_PATTERN + ".parquet"
 DEFAULT_VIDEO_PATH = VIDEO_DIR + "/{video_key}/" + CHUNK_FILE_PATTERN + ".mp4"
@@ -129,9 +130,6 @@ class DatasetInfo:
     # Optional metadata
     robot_type: str | None = None
     splits: dict[str, str] = field(default_factory=dict)
-    # OpenAI-style tool schemas declared by the dataset. ``None`` means the
-    # dataset doesn't declare any — readers fall back to ``DEFAULT_TOOLS``.
-    tools: list[dict] | None = None
 
     def __post_init__(self) -> None:
         # Coerce feature shapes from list to tuple — JSON deserialisation
@@ -153,15 +151,11 @@ class DatasetInfo:
         """Return a JSON-serialisable dict.
 
         Converts tuple shapes back to lists so ``json.dump`` can handle them.
-        Drops ``tools`` when unset so existing datasets keep a clean
-        ``info.json``.
         """
         d = dataclasses.asdict(self)
         for ft in d["features"].values():
             if isinstance(ft.get("shape"), tuple):
                 ft["shape"] = list(ft["shape"])
-        if d.get("tools") is None:
-            d.pop("tools", None)
         return d
 
     @classmethod

src/lerobot/datasets/video_utils.py

@@ -22,7 +22,7 @@ import shutil
 import tempfile
 import threading
 import warnings
-from dataclasses import asdict, dataclass, field
+from dataclasses import dataclass, field
 from fractions import Fraction
 from pathlib import Path
 from threading import Lock
@@ -36,14 +36,86 @@ import torch
 from datasets.features.features import register_feature
 from PIL import Image
 
-from lerobot.configs import (
-    VideoEncoderConfig,
-    camera_encoder_defaults,
-)
-from lerobot.utils.import_utils import get_safe_default_video_backend
+from lerobot.utils.import_utils import get_safe_default_codec
 
 logger = logging.getLogger(__name__)
 
+# List of hardware encoders to probe for auto-selection. Availability depends on the platform and FFmpeg build.
+# Determines the order of preference for auto-selection when vcodec="auto" is used.
+HW_ENCODERS = [
+    "h264_videotoolbox",  # macOS
+    "hevc_videotoolbox",  # macOS
+    "h264_nvenc",  # NVIDIA GPU
+    "hevc_nvenc",  # NVIDIA GPU
+    "h264_vaapi",  # Linux Intel/AMD
+    "h264_qsv",  # Intel Quick Sync
+]
+
+VALID_VIDEO_CODECS = {"h264", "hevc", "libsvtav1", "auto"} | set(HW_ENCODERS)
+
+
+def _get_codec_options(
+    vcodec: str,
+    g: int | None = 2,
+    crf: int | None = 30,
+    preset: int | None = None,
+) -> dict:
+    """Build codec-specific options dict for video encoding."""
+    options = {}
+
+    # GOP size (keyframe interval) - supported by VideoToolbox and software encoders
+    if g is not None and (vcodec in ("h264_videotoolbox", "hevc_videotoolbox") or vcodec not in HW_ENCODERS):
+        options["g"] = str(g)
+
+    # Quality control (codec-specific parameter names)
+    if crf is not None:
+        if vcodec in ("h264", "hevc", "libsvtav1"):
+            options["crf"] = str(crf)
+        elif vcodec in ("h264_videotoolbox", "hevc_videotoolbox"):
+            quality = max(1, min(100, int(100 - crf * 2)))
+            options["q:v"] = str(quality)
+        elif vcodec in ("h264_nvenc", "hevc_nvenc"):
+            options["rc"] = "constqp"
+            options["qp"] = str(crf)
+        elif vcodec in ("h264_vaapi",):
+            options["qp"] = str(crf)
+        elif vcodec in ("h264_qsv",):
+            options["global_quality"] = str(crf)
+
+    # Preset (only for libsvtav1)
+    if vcodec == "libsvtav1":
+        options["preset"] = str(preset) if preset is not None else "12"
+
+    return options
+
+
+def detect_available_hw_encoders() -> list[str]:
+    """Probe PyAV/FFmpeg for available hardware video encoders."""
+    available = []
+    for codec_name in HW_ENCODERS:
+        try:
+            av.codec.Codec(codec_name, "w")
+            available.append(codec_name)
+        except Exception:  # nosec B110
+            logger.debug("HW encoder '%s' not available", codec_name)  # nosec B110
+    return available
+
+
+def resolve_vcodec(vcodec: str) -> str:
+    """Validate vcodec and resolve 'auto' to best available HW encoder, fallback to libsvtav1."""
+    if vcodec not in VALID_VIDEO_CODECS:
+        raise ValueError(f"Invalid vcodec '{vcodec}'. Must be one of: {sorted(VALID_VIDEO_CODECS)}")
+    if vcodec != "auto":
+        logger.info(f"Using video codec: {vcodec}")
+        return vcodec
+    available = detect_available_hw_encoders()
+    for encoder in HW_ENCODERS:
+        if encoder in available:
+            logger.info(f"Auto-selected video codec: {encoder}")
+            return encoder
+    logger.info("No hardware encoder available, falling back to software encoder 'libsvtav1'")
+    return "libsvtav1"
+
 
 def decode_video_frames(
     video_path: Path | str,
@@ -71,7 +143,7 @@ def decode_video_frames(
     Currently supports torchcodec on cpu and pyav.
     """
     if backend is None:
-        backend = get_safe_default_video_backend()
+        backend = get_safe_default_codec()
     if backend == "torchcodec":
         return decode_video_frames_torchcodec(video_path, timestamps, tolerance_s, return_uint8=return_uint8)
     elif backend == "pyav":
@@ -335,17 +407,18 @@ def encode_video_frames(
     imgs_dir: Path | str,
     video_path: Path | str,
     fps: int,
-    camera_encoder: VideoEncoderConfig | None = None,
-    encoder_threads: int | None = None,
-    *,
+    vcodec: str = "libsvtav1",
+    pix_fmt: str = "yuv420p",
+    g: int | None = 2,
+    crf: int | None = 30,
+    fast_decode: int = 0,
     log_level: int | None = av.logging.WARNING,
     overwrite: bool = False,
+    preset: int | None = None,
+    encoder_threads: int | None = None,
 ) -> None:
     """More info on ffmpeg arguments tuning on `benchmark/video/README.md`"""
-    if camera_encoder is None:
-        camera_encoder = camera_encoder_defaults()
-    vcodec = camera_encoder.vcodec
-    pix_fmt = camera_encoder.pix_fmt
+    vcodec = resolve_vcodec(vcodec)
 
     video_path = Path(video_path)
     imgs_dir = Path(imgs_dir)
@@ -356,18 +429,42 @@ def encode_video_frames(
 
     video_path.parent.mkdir(parents=True, exist_ok=True)
 
+    # Encoders/pixel formats incompatibility check
+    if (vcodec == "libsvtav1" or vcodec == "hevc") and pix_fmt == "yuv444p":
+        logger.warning(
+            f"Incompatible pixel format 'yuv444p' for codec {vcodec}, auto-selecting format 'yuv420p'"
+        )
+        pix_fmt = "yuv420p"
+
     # Get input frames
     template = "frame-" + ("[0-9]" * 6) + ".png"
     input_list = sorted(
         glob.glob(str(imgs_dir / template)), key=lambda x: int(x.split("-")[-1].split(".")[0])
     )
 
+    # Define video output frame size (assuming all input frames are the same size)
     if len(input_list) == 0:
         raise FileNotFoundError(f"No images found in {imgs_dir}.")
     with Image.open(input_list[0]) as dummy_image:
         width, height = dummy_image.size
 
-    video_options = camera_encoder.get_codec_options(encoder_threads, as_strings=True)
+    # Define video codec options
+    video_options = _get_codec_options(vcodec, g, crf, preset)
+
+    if fast_decode:
+        key = "svtav1-params" if vcodec == "libsvtav1" else "tune"
+        value = f"fast-decode={fast_decode}" if vcodec == "libsvtav1" else "fastdecode"
+        video_options[key] = value
+
+    if encoder_threads is not None:
+        if vcodec == "libsvtav1":
+            lp_param = f"lp={encoder_threads}"
+            if "svtav1-params" in video_options:
+                video_options["svtav1-params"] += f":{lp_param}"
+            else:
+                video_options["svtav1-params"] = lp_param
+        else:
+            video_options["threads"] = str(encoder_threads)
 
     # Set logging level
     if log_level is not None:
@@ -403,97 +500,8 @@ def encode_video_frames(
         raise OSError(f"Video encoding did not work. File not found: {video_path}.")
 
 
-def reencode_video(
-    input_video_path: Path | str,
-    output_video_path: Path | str,
-    camera_encoder: VideoEncoderConfig | None = None,
-    encoder_threads: int | None = None,
-    log_level: int | None = av.logging.WARNING,
-    overwrite: bool = False,
-) -> None:
-    """Re-encode a video file using the given encoder configuration.
-
-    Args:
-        input_video_path: Existing video file to read.
-        output_video_path: Path for the re-encoded file.
-        camera_encoder: Encoder configuration. Defaults to :func:`camera_encoder_defaults`.
-        encoder_threads: Optional thread count forwarded to :meth:`VideoEncoderConfig.get_codec_options`.
-        log_level: libav log level while encoding, or ``None`` to leave logging unchanged. Defaults to WARNING.
-        overwrite: When ``False`` and ``output_video_path`` already exists, skip and log a warning.
-    """
-
-    camera_encoder = camera_encoder or camera_encoder_defaults()
-
-    output_video_path = Path(output_video_path)
-
-    if output_video_path.exists() and not overwrite:
-        logger.warning(f"Video file already exists: {output_video_path}. Skipping re-encode.")
-        return
-
-    output_video_path.parent.mkdir(parents=True, exist_ok=True)
-
-    video_options = camera_encoder.get_codec_options(encoder_threads, as_strings=True)
-    vcodec = camera_encoder.vcodec
-    pix_fmt = camera_encoder.pix_fmt
-
-    with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as tmp_named_file:
-        tmp_output_video_path = tmp_named_file.name
-
-    if log_level is not None:
-        logging.getLogger("libav").setLevel(log_level)
-
-    try:
-        with av.open(input_video_path, mode="r") as src:
-            try:
-                in_stream = src.streams.video[0]
-            except IndexError as e:
-                raise ValueError(f"No video stream in {input_video_path}") from e
-
-            fps = (
-                in_stream.base_rate
-            )  # We allow fractional fps though LeRobotDataset only supports integer fps
-            width = int(in_stream.width)
-            height = int(in_stream.height)
-
-            with av.open(
-                tmp_output_video_path,
-                mode="w",
-                options={
-                    "movflags": "faststart"
-                },  # faststart is to move the metadata to the beginning of the file to speed up loading
-            ) as dst:
-                out_stream = dst.add_stream(vcodec, fps, options=video_options)
-                out_stream.pix_fmt = pix_fmt
-                out_stream.width = width
-                out_stream.height = height
-
-                for frame in src.decode(in_stream):
-                    frame = frame.reformat(width=width, height=height, format=pix_fmt)
-                    packet = out_stream.encode(frame)
-                    if packet:
-                        dst.mux(packet)
-
-                packet = out_stream.encode()
-                if packet:
-                    dst.mux(packet)
-
-        shutil.move(tmp_output_video_path, output_video_path)
-    except Exception:
-        Path(tmp_output_video_path).unlink(missing_ok=True)
-        raise
-    finally:
-        if log_level is not None:
-            av.logging.restore_default_callback()
-
-    if not output_video_path.exists():
-        raise OSError(f"Video re-encoding did not work. File not found: {output_video_path}.")
-
-
 def concatenate_video_files(
-    input_video_paths: list[Path | str],
-    output_video_path: Path,
-    overwrite: bool = True,
-    compatibility_check: bool = False,
+    input_video_paths: list[Path | str], output_video_path: Path, overwrite: bool = True
 ):
     """
     Concatenate multiple video files into a single video file using pyav.
@@ -506,7 +514,6 @@ def concatenate_video_files(
         input_video_paths: Ordered list of input video file paths to concatenate.
         output_video_path: Path to the output video file.
         overwrite: Whether to overwrite the output video file if it already exists. Default is True.
-        compatibility_check: Whether to check if the input videos are compatible. Default is False.
 
     Note:
         - Creates a temporary directory for intermediate files that is cleaned up after use.
@@ -525,22 +532,6 @@ def concatenate_video_files(
     if len(input_video_paths) == 0:
         raise FileNotFoundError("No input video paths provided.")
 
-    # This check may be skipped at recording time as videos are encoded with the same encoder config.
-    if compatibility_check:
-        reference_video_info = get_video_info(input_video_paths[0])
-        for input_path in input_video_paths[1:]:
-            video_info = get_video_info(input_path)
-            if (
-                video_info["video.height"] != reference_video_info["video.height"]
-                or video_info["video.width"] != reference_video_info["video.width"]
-                or video_info["video.fps"] != reference_video_info["video.fps"]
-                or video_info["video.codec"] != reference_video_info["video.codec"]
-                or video_info["video.pix_fmt"] != reference_video_info["video.pix_fmt"]
-            ):
-                raise ValueError(
-                    f"Input video {input_path} is not compatible with the reference video {input_video_paths[0]}."
-                )
-
     # Create a temporary .ffconcat file to list the input video paths
     with tempfile.NamedTemporaryFile(mode="w", suffix=".ffconcat", delete=False) as tmp_concatenate_file:
         tmp_concatenate_file.write("ffconcat version 1.0\n")
@@ -607,20 +598,26 @@ class _CameraEncoderThread(threading.Thread):
         fps: int,
         vcodec: str,
         pix_fmt: str,
-        codec_options: dict[str, str],
+        g: int | None,
+        crf: int | None,
+        preset: int | None,
         frame_queue: queue.Queue,
         result_queue: queue.Queue,
         stop_event: threading.Event,
+        encoder_threads: int | None = None,
     ):
         super().__init__(daemon=True)
         self.video_path = video_path
         self.fps = fps
         self.vcodec = vcodec
         self.pix_fmt = pix_fmt
-        self.codec_options = codec_options
+        self.g = g
+        self.crf = crf
+        self.preset = preset
         self.frame_queue = frame_queue
         self.result_queue = result_queue
         self.stop_event = stop_event
+        self.encoder_threads = encoder_threads
 
     def run(self) -> None:
         from .compute_stats import RunningQuantileStats, auto_downsample_height_width
@@ -656,9 +653,19 @@ class _CameraEncoderThread(threading.Thread):
                 # Open container on first frame (to get width/height)
                 if container is None:
                     height, width = frame_data.shape[:2]
+                    video_options = _get_codec_options(self.vcodec, self.g, self.crf, self.preset)
+                    if self.encoder_threads is not None:
+                        if self.vcodec == "libsvtav1":
+                            lp_param = f"lp={self.encoder_threads}"
+                            if "svtav1-params" in video_options:
+                                video_options["svtav1-params"] += f":{lp_param}"
+                            else:
+                                video_options["svtav1-params"] = lp_param
+                        else:
+                            video_options["threads"] = str(self.encoder_threads)
                     Path(self.video_path).parent.mkdir(parents=True, exist_ok=True)
                     container = av.open(str(self.video_path), "w")
-                    output_stream = container.add_stream(self.vcodec, self.fps, options=self.codec_options)
+                    output_stream = container.add_stream(self.vcodec, self.fps, options=video_options)
                     output_stream.pix_fmt = self.pix_fmt
                     output_stream.width = width
                     output_stream.height = height
@@ -724,24 +731,22 @@ class StreamingVideoEncoder:
     def __init__(
         self,
         fps: int,
-        camera_encoder: VideoEncoderConfig | None = None,
+        vcodec: str = "libsvtav1",
+        pix_fmt: str = "yuv420p",
+        g: int | None = 2,
+        crf: int | None = 30,
+        preset: int | None = None,
         queue_maxsize: int = 30,
         encoder_threads: int | None = None,
     ):
-        """
-        Args:
-            fps: Frames per second for the output videos.
-            camera_encoder: Video encoder settings applied to all cameras.
-                When ``None``, :func:`camera_encoder_defaults` is used.
-            encoder_threads: Number of encoder threads (global setting).
-                ``None`` lets the codec decide.
-            queue_maxsize: Max frames to buffer per camera before
-                back-pressure drops frames.
-        """
         self.fps = fps
-        self._camera_encoder = camera_encoder or camera_encoder_defaults()
-        self._encoder_threads = encoder_threads
+        self.vcodec = resolve_vcodec(vcodec)
+        self.pix_fmt = pix_fmt
+        self.g = g
+        self.crf = crf
+        self.preset = preset
         self.queue_maxsize = queue_maxsize
+        self.encoder_threads = encoder_threads
 
         self._frame_queues: dict[str, queue.Queue] = {}
         self._result_queues: dict[str, queue.Queue] = {}
@@ -772,17 +777,18 @@ class StreamingVideoEncoder:
             temp_video_dir = Path(tempfile.mkdtemp(dir=temp_dir))
             video_path = temp_video_dir / f"{video_key.replace('/', '_')}_streaming.mp4"
 
-            vcodec = self._camera_encoder.vcodec
-            codec_options = self._camera_encoder.get_codec_options(self._encoder_threads, as_strings=True)
             encoder_thread = _CameraEncoderThread(
                 video_path=video_path,
                 fps=self.fps,
-                vcodec=vcodec,
-                pix_fmt=self._camera_encoder.pix_fmt,
-                codec_options=codec_options,
+                vcodec=self.vcodec,
+                pix_fmt=self.pix_fmt,
+                g=self.g,
+                crf=self.crf,
+                preset=self.preset,
                 frame_queue=frame_queue,
                 result_queue=result_queue,
                 stop_event=stop_event,
+                encoder_threads=self.encoder_threads,
             )
             encoder_thread.start()
 
@@ -987,18 +993,8 @@ def get_audio_info(video_path: Path | str) -> dict:
     return audio_info
 
 
-def get_video_info(
-    video_path: Path | str,
-    camera_encoder: VideoEncoderConfig | None = None,
-) -> dict:
-    """Build the ``video.*`` / ``audio.*`` info dict persisted in ``info.json``.
-
-    Args:
-        video_path: Path to the encoded video file to probe.
-        camera_encoder: If provided, record the exact encoder settings used to encode this
-            video. Stream-derived values take precedence — encoder fields are only written for keys
-            not already populated from the video file itself.
-    """
+def get_video_info(video_path: Path | str) -> dict:
+    # Set logging level
     logging.getLogger("libav").setLevel(av.logging.WARNING)
 
     # Getting video stream information
@@ -1029,14 +1025,6 @@ def get_video_info(
     # Adding audio stream information
     video_info.update(**get_audio_info(video_path))
 
-    # Add additional encoder configuration if provided
-    if camera_encoder is not None:
-        for field_name, field_value in asdict(camera_encoder).items():
-            # vcodec is already populated from the video stream
-            if field_name == "vcodec":
-                continue
-            video_info.setdefault(f"video.{field_name}", field_value)
-
     return video_info
 
 

src/lerobot/policies/eo1/modeling_eo1.py

@@ -28,12 +28,11 @@ import torch.nn.functional as F  # noqa: N812
 import torch.utils.checkpoint
 from torch import Tensor
 
+from lerobot.policies.eo1.configuration_eo1 import EO1Config
+from lerobot.policies.pretrained import PreTrainedPolicy
 from lerobot.utils.constants import ACTION, OBS_STATE
 from lerobot.utils.import_utils import _transformers_available, require_package
 
-from ..pretrained import PreTrainedPolicy
-from .configuration_eo1 import EO1Config
-
 if TYPE_CHECKING or _transformers_available:
     from transformers.activations import ACT2FN
     from transformers.models.qwen2_5_vl import Qwen2_5_VLForConditionalGeneration

src/lerobot/policies/eo1/processor_eo1.py

@@ -22,6 +22,7 @@ from typing import TYPE_CHECKING, Any
 import torch
 
 from lerobot.configs.types import FeatureType, PipelineFeatureType, PolicyFeature
+from lerobot.policies.eo1.configuration_eo1 import EO1Config
 from lerobot.processor import (
     AddBatchDimensionProcessorStep,
     ComplementaryDataProcessorStep,
@@ -43,8 +44,6 @@ from lerobot.utils.constants import (
 )
 from lerobot.utils.import_utils import _transformers_available, require_package
 
-from .configuration_eo1 import EO1Config
-
 if TYPE_CHECKING or _transformers_available:
     from transformers.models.qwen2_5_vl import Qwen2_5_VLProcessor
 else:

src/lerobot/policies/pi05/modeling_pi05.py

@@ -441,13 +441,13 @@ class PaliGemmaWithExpertModel(
         if image.dtype != torch.float32:
             image = image.to(torch.float32)
         image_outputs = self.paligemma.model.get_image_features(image)
-        features = image_outputs.pooler_output
+        features = image_outputs.pooler_output * self.paligemma.config.text_config.hidden_size**0.5
         if features.dtype != out_dtype:
             features = features.to(out_dtype)
         return features
 
     def embed_language_tokens(self, tokens: torch.Tensor):
-        return self.paligemma.model.language_model.get_input_embeddings()(tokens)
+        return self.paligemma.model.language_model.embed_tokens(tokens)
 
     def forward(
         self,
@@ -662,7 +662,8 @@ class PI05Pytorch(nn.Module):  # see openpi `PI0Pytorch`
         # Process language tokens
         def lang_embed_func(tokens):
             lang_emb = self.paligemma_with_expert.embed_language_tokens(tokens)
-            return lang_emb
+            lang_emb_dim = lang_emb.shape[-1]
+            return lang_emb * math.sqrt(lang_emb_dim)
 
         lang_emb = self._apply_checkpoint(lang_embed_func, tokens)
         embs.append(lang_emb)

src/lerobot/processor/__init__.py

@@ -95,13 +95,6 @@ from .relative_action_processor import (
 from .rename_processor import RenameObservationsProcessorStep, rename_stats
 from .tokenizer_processor import ActionTokenizerProcessorStep, TokenizerProcessorStep
 
-# RenderMessagesStep is intentionally NOT re-exported here: it pulls in
-# `lerobot.datasets.language`, which requires the `[dataset]` extra
-# (`datasets`, `pyarrow`). Importing it from the processor package would
-# break every base-install consumer of `lerobot.processor`. Users that
-# need it import directly:
-#   from lerobot.processor.render_messages_processor import RenderMessagesStep
-
 __all__ = [
     "ActionProcessorStep",
     "AddTeleopActionAsComplimentaryDataStep",

src/lerobot/processor/batch_processor.py

@@ -174,24 +174,6 @@ class AddBatchDimensionComplementaryDataStep(ComplementaryDataProcessorStep):
             task_index_value = complementary_data["task_index"]
             if isinstance(task_index_value, Tensor) and task_index_value.dim() == 0:
                 complementary_data["task_index"] = task_index_value.unsqueeze(0)
-
-        complementary_data.pop("language_persistent", None)
-        complementary_data.pop("language_events", None)
-
-        if "messages" in complementary_data:
-            messages = complementary_data["messages"]
-            if isinstance(messages, list) and (not messages or isinstance(messages[0], dict)):
-                complementary_data["messages"] = [messages]
-
-        if "message_streams" in complementary_data:
-            streams = complementary_data["message_streams"]
-            if isinstance(streams, list) and (not streams or isinstance(streams[0], str)):
-                complementary_data["message_streams"] = [streams]
-
-        if "target_message_indices" in complementary_data:
-            indices = complementary_data["target_message_indices"]
-            if isinstance(indices, list) and (not indices or isinstance(indices[0], int)):
-                complementary_data["target_message_indices"] = [indices]
         return complementary_data
 
     def transform_features(

src/lerobot/processor/converters.py

@@ -153,30 +153,26 @@ def from_tensor_to_numpy(x: torch.Tensor | Any) -> np.ndarray | float | int | An
     return x
 
 
-_COMPLEMENTARY_KEYS = (
-    "task",
-    "index",
-    "task_index",
-    "episode_index",
-    "timestamp",
-    "language_persistent",
-    "language_events",
-    "messages",
-    "message_streams",
-    "target_message_indices",
-)
-
-
 def _extract_complementary_data(batch: dict[str, Any]) -> dict[str, Any]:
-    """Extract complementary data from a batch dictionary.
+    """
+    Extract complementary data from a batch dictionary.
 
-    Includes padding flags (any key containing ``_is_pad``) plus the fixed
-    set of metadata / language keys defined in ``_COMPLEMENTARY_KEYS`` —
-    each only when present in ``batch``.
+    This includes padding flags, task description, and indices.
+
+    Args:
+        batch: The batch dictionary.
+
+    Returns:
+        A dictionary with the extracted complementary data.
     """
     pad_keys = {k: v for k, v in batch.items() if "_is_pad" in k}
-    extras = {k: batch[k] for k in _COMPLEMENTARY_KEYS if k in batch}
-    return {**pad_keys, **extras}
+    task_key = {"task": batch["task"]} if "task" in batch else {}
+    subtask_key = {"subtask": batch["subtask"]} if "subtask" in batch else {}
+    index_key = {"index": batch["index"]} if "index" in batch else {}
+    task_index_key = {"task_index": batch["task_index"]} if "task_index" in batch else {}
+    episode_index_key = {"episode_index": batch["episode_index"]} if "episode_index" in batch else {}
+
+    return {**pad_keys, **task_key, **subtask_key, **index_key, **task_index_key, **episode_index_key}
 
 
 def create_transition(

src/lerobot/processor/render_messages_processor.py

@@ -1,84 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from __future__ import annotations
-
-from dataclasses import dataclass
-from typing import Any
-
-from lerobot.configs import PipelineFeatureType, PolicyFeature
-from lerobot.configs.recipe import TrainingRecipe
-from lerobot.datasets.language import LANGUAGE_EVENTS, LANGUAGE_PERSISTENT
-from lerobot.datasets.language_render import render_sample
-from lerobot.types import EnvTransition, TransitionKey
-from lerobot.utils.utils import unwrap_scalar
-
-from .pipeline import ProcessorStep, ProcessorStepRegistry
-
-
-@dataclass
-@ProcessorStepRegistry.register(name="render_messages_processor")
-class RenderMessagesStep(ProcessorStep):
-    """Processor step that turns raw language columns into rendered chat messages.
-
-    Reads ``language_persistent`` and ``language_events`` from the transition's
-    complementary data, renders them through ``recipe`` at the sample timestamp,
-    and replaces the raw columns with the resulting ``messages`` /
-    ``message_streams`` / ``target_message_indices`` keys.
-    """
-
-    recipe: TrainingRecipe
-    dataset_ctx: Any | None = None
-
-    def __call__(self, transition: EnvTransition) -> EnvTransition | None:
-        """Render messages for a single transition; return ``None`` to drop it."""
-        complementary_data = transition.get(TransitionKey.COMPLEMENTARY_DATA) or {}
-        persistent = complementary_data.get(LANGUAGE_PERSISTENT) or []
-        events = complementary_data.get(LANGUAGE_EVENTS) or []
-
-        if not persistent and not events:
-            return transition
-
-        timestamp = complementary_data.get("timestamp")
-        if timestamp is None:
-            raise KeyError("RenderMessagesStep requires sample timestamp in complementary data.")
-
-        sample_idx = complementary_data.get("index", 0)
-        rendered = render_sample(
-            recipe=self.recipe,
-            persistent=persistent,
-            events=events,
-            t=unwrap_scalar(timestamp),
-            sample_idx=int(unwrap_scalar(sample_idx)),
-            task=complementary_data.get("task"),
-            dataset_ctx=self.dataset_ctx,
-        )
-        if rendered is None:
-            return None
-
-        new_transition = transition.copy()
-        new_complementary_data = dict(complementary_data)
-        new_complementary_data.pop(LANGUAGE_PERSISTENT, None)
-        new_complementary_data.pop(LANGUAGE_EVENTS, None)
-        new_complementary_data.update(rendered)
-        new_transition[TransitionKey.COMPLEMENTARY_DATA] = new_complementary_data
-        return new_transition
-
-    def transform_features(
-        self, features: dict[PipelineFeatureType, dict[str, PolicyFeature]]
-    ) -> dict[PipelineFeatureType, dict[str, PolicyFeature]]:
-        """Pass features through unchanged; rendering only touches complementary data."""
-        return features

src/lerobot/rewards/__init__.py

@@ -20,14 +20,14 @@ from .factory import (
     make_reward_pre_post_processors as make_reward_pre_post_processors,
 )
 from .pretrained import PreTrainedRewardModel as PreTrainedRewardModel
+from .robometer.configuration_robometer import RobometerConfig as RobometerConfig
 from .sarm.configuration_sarm import SARMConfig as SARMConfig
-from .topreward.configuration_topreward import TOPRewardConfig as TOPRewardConfig
 
 __all__ = [
     # Configuration classes
     "RewardClassifierConfig",
+    "RobometerConfig",
     "SARMConfig",
-    "TOPRewardConfig",
     # Base class
     "PreTrainedRewardModel",
     # Factory functions

src/lerobot/rewards/classifier/modeling_classifier.py

@@ -17,11 +17,10 @@ import logging
 import torch
 from torch import Tensor, nn
 
+from lerobot.rewards.classifier.configuration_classifier import RewardClassifierConfig
+from lerobot.rewards.pretrained import PreTrainedRewardModel
 from lerobot.utils.constants import OBS_IMAGE, REWARD
 
-from ..pretrained import PreTrainedRewardModel
-from .configuration_classifier import RewardClassifierConfig
-
 
 class ClassifierOutput:
     """Wrapper for classifier outputs with additional metadata."""

src/lerobot/rewards/classifier/processor_classifier.py

@@ -25,8 +25,7 @@ from lerobot.processor import (
     policy_action_to_transition,
     transition_to_policy_action,
 )
-
-from .configuration_classifier import RewardClassifierConfig
+from lerobot.rewards.classifier.configuration_classifier import RewardClassifierConfig
 
 
 def make_classifier_processor(

src/lerobot/rewards/factory.py

@@ -22,11 +22,10 @@ import torch
 
 from lerobot.configs.rewards import RewardModelConfig
 from lerobot.processor import PolicyAction, PolicyProcessorPipeline
-
-from .classifier.configuration_classifier import RewardClassifierConfig
-from .pretrained import PreTrainedRewardModel
-from .sarm.configuration_sarm import SARMConfig
-from .topreward.configuration_topreward import TOPRewardConfig
+from lerobot.rewards.classifier.configuration_classifier import RewardClassifierConfig
+from lerobot.rewards.pretrained import PreTrainedRewardModel
+from lerobot.rewards.robometer.configuration_robometer import RobometerConfig
+from lerobot.rewards.sarm.configuration_sarm import SARMConfig
 
 
 def get_reward_model_class(name: str) -> type[PreTrainedRewardModel]:
@@ -38,7 +37,7 @@ def get_reward_model_class(name: str) -> type[PreTrainedRewardModel]:
 
     Args:
         name: The name of the reward model. Supported names are "reward_classifier",
-              "sarm", "topreward".
+              "sarm", "robometer".
 
     Returns:
         The reward model class corresponding to the given name.
@@ -54,10 +53,10 @@ def get_reward_model_class(name: str) -> type[PreTrainedRewardModel]:
         from lerobot.rewards.sarm.modeling_sarm import SARMRewardModel
 
         return SARMRewardModel
-    elif name == "topreward":
-        from lerobot.rewards.topreward.modeling_topreward import TOPRewardModel
+    elif name == "robometer":
+        from lerobot.rewards.robometer.modeling_robometer import RobometerRewardModel
 
-        return TOPRewardModel
+        return RobometerRewardModel
     else:
         try:
             return _get_reward_model_cls_from_name(name=name)
@@ -74,7 +73,7 @@ def make_reward_model_config(reward_type: str, **kwargs) -> RewardModelConfig:
 
     Args:
         reward_type: The type of the reward model. Supported types include
-                     "reward_classifier", "sarm", "topreward".
+                     "reward_classifier", "sarm", "robometer".
         **kwargs: Keyword arguments to be passed to the configuration class constructor.
 
     Returns:
@@ -87,8 +86,8 @@ def make_reward_model_config(reward_type: str, **kwargs) -> RewardModelConfig:
         return RewardClassifierConfig(**kwargs)
     elif reward_type == "sarm":
         return SARMConfig(**kwargs)
-    elif reward_type == "topreward":
-        return TOPRewardConfig(**kwargs)
+    elif reward_type == "robometer":
+        return RobometerConfig(**kwargs)
     else:
         try:
             config_cls = RewardModelConfig.get_choice_class(reward_type)
@@ -168,11 +167,10 @@ def make_reward_pre_post_processors(
             dataset_stats=kwargs.get("dataset_stats"),
             dataset_meta=kwargs.get("dataset_meta"),
         )
+    elif isinstance(reward_cfg, RobometerConfig):
+        from lerobot.rewards.robometer.processor_robometer import make_robometer_pre_post_processors
 
-    elif isinstance(reward_cfg, TOPRewardConfig):
-        from lerobot.rewards.topreward.processor_topreward import make_topreward_pre_post_processors
-
-        return make_topreward_pre_post_processors(
+        return make_robometer_pre_post_processors(
             config=reward_cfg,
             dataset_stats=kwargs.get("dataset_stats"),
         )

src/lerobot/rewards/robometer/__init__.py

@@ -12,8 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from .configuration_topreward import TOPRewardConfig
-from .modeling_topreward import TOPRewardModel
-from .processor_topreward import make_topreward_pre_post_processors
+from .configuration_robometer import RobometerConfig
+from .modeling_robometer import RobometerRewardModel
+from .processor_robometer import make_robometer_pre_post_processors
 
-__all__ = ["TOPRewardConfig", "TOPRewardModel", "make_topreward_pre_post_processors"]
+__all__ = ["RobometerConfig", "RobometerRewardModel", "make_robometer_pre_post_processors"]

src/lerobot/rewards/robometer/_upstream_loader.py

@@ -0,0 +1,229 @@
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Upstream/legacy Robometer checkpoint loader.
+
+This module is **only** used by the one-time conversion tooling
+(:mod:`lerobot.scripts.lerobot_export_robometer` and
+``scripts/verify_robometer_export.py``). It supports:
+
+- Sharded upstream checkpoints (``model-0000X-of-Y.safetensors`` + index).
+- PEFT/LoRA adapter checkpoints (``adapter_config.json`` + adapter weights).
+- Local snapshot directories or Hugging Face Hub repo ids.
+
+Once :class:`~lerobot.rewards.robometer.RobometerRewardModel` is loaded
+through this module, calling ``save_pretrained`` writes the canonical
+LeRobot-native layout (single ``model.safetensors`` + ``config.json``) that
+the base loader understands.
+
+The runtime path
+(:meth:`~lerobot.rewards.pretrained.PreTrainedRewardModel.from_pretrained`)
+does **not** import this file. It is safe to delete once you no longer need
+the conversion tooling.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+from pathlib import Path
+from typing import Any
+
+from huggingface_hub import snapshot_download
+from safetensors.torch import load_file
+from torch import Tensor, nn
+
+from lerobot.utils.import_utils import require_package
+
+logger = logging.getLogger(__name__)
+
+
+def _download_robometer_snapshot(
+    pretrained_path: str,
+    *,
+    hub_token: str | None = None,
+) -> Path:
+    """Resolve a Robometer snapshot directory.
+
+    - If ``pretrained_path`` is an existing local directory, return it directly.
+    - Otherwise treat ``pretrained_path`` as a Hugging Face repo id (optionally
+      with ``@revision``) and download it via ``snapshot_download``.
+    """
+    local_candidate = Path(pretrained_path)
+    if local_candidate.is_dir():
+        return local_candidate
+
+    if "@" in pretrained_path:
+        repo_id, revision = pretrained_path.split("@", 1)
+    else:
+        repo_id, revision = pretrained_path, None
+
+    return Path(
+        snapshot_download(
+            repo_id=repo_id,
+            revision=revision,
+            token=hub_token,
+            allow_patterns=[
+                "*.json",
+                "*.safetensors",
+                "*.bin",
+                "*.txt",
+                "*.model",
+                "tokenizer*",
+                "special_tokens_map.json",
+            ],
+        )
+    )
+
+
+def _maybe_apply_peft(base_model: Any, snapshot_dir: Path) -> Any:
+    adapter_config = snapshot_dir / "adapter_config.json"
+    if not adapter_config.exists():
+        return base_model
+
+    require_package("peft", extra="peft-dep")
+    from peft import PeftModel
+
+    return PeftModel.from_pretrained(base_model, str(snapshot_dir))
+
+
+def _remap_state_dict_keys(state_dict: dict[str, Tensor], model: nn.Module) -> dict[str, Tensor]:
+    """Try a few common prefix swaps so PEFT-wrapped checkpoints load cleanly."""
+    model_keys = set(model.state_dict().keys())
+    remapped: dict[str, Tensor] = {}
+
+    for key, value in state_dict.items():
+        if key in model_keys:
+            remapped[key] = value
+            continue
+
+        candidates: list[str] = []
+        if key.startswith("model.model."):
+            candidates.append(key.replace("model.model.", "model.base_model.model.model.", 1))
+            candidates.append(key.replace("model.model.", "model.", 1))
+        if key.startswith("model."):
+            candidates.append(f"model.{key}")
+            candidates.append(key.replace("model.", "", 1))
+        else:
+            candidates.append(f"model.{key}")
+        if key.startswith("model.") and not key.startswith("model.base_model."):
+            parts = key.split(".", 1)
+            if len(parts) == 2:
+                candidates.append(f"model.base_model.{parts[1]}")
+
+        for candidate in candidates:
+            if candidate in model_keys:
+                remapped[candidate] = value
+                break
+        else:
+            remapped[key] = value
+
+    return remapped
+
+
+def _resolve_checkpoint_safetensors_files(snapshot_dir: Path) -> list[Path]:
+    """Pick the safetensors files that hold the full model weights.
+
+    When ``model.safetensors.index.json`` is present, only the files it lists are
+    loaded. Otherwise any ``model*.safetensors`` shards are preferred over
+    sidecar files. Falls back to every ``*.safetensors`` in the snapshot.
+    """
+    index_path = snapshot_dir / "model.safetensors.index.json"
+    if index_path.exists():
+        with index_path.open() as f:
+            weight_map = json.load(f).get("weight_map", {})
+        indexed = sorted(
+            {snapshot_dir / name for name in weight_map.values() if (snapshot_dir / name).exists()}
+        )
+        if indexed:
+            return indexed
+
+    model_shards = sorted(snapshot_dir.glob("model*.safetensors"))
+    if model_shards:
+        return model_shards
+
+    return sorted(snapshot_dir.glob("*.safetensors"))
+
+
+def apply_upstream_checkpoint(
+    model: nn.Module,
+    pretrained_path: str,
+    *,
+    hub_token: str | None = None,
+) -> None:
+    """Load an upstream (sharded / PEFT) Robometer checkpoint into ``model``.
+
+    Downloads the snapshot, optionally applies PEFT wrapping, merges sharded
+    ``.safetensors`` files in memory, remaps PEFT-prefixed keys, and loads them
+    into ``model`` non-strictly. ``model`` must already be constructed with the
+    matching Robometer architecture (e.g. via
+    :class:`~lerobot.rewards.robometer.RobometerRewardModel` ``__init__``).
+    """
+    snapshot_dir = _download_robometer_snapshot(pretrained_path, hub_token=hub_token)
+
+    # PEFT adapter checkpoints wrap the base model before weight loading so the
+    # remapper can place adapter tensors at the right prefix.
+    base_model = getattr(model, "model", None)
+    if base_model is not None:
+        wrapped = _maybe_apply_peft(base_model, snapshot_dir)
+        if wrapped is not base_model:
+            model.model = wrapped
+
+    files = _resolve_checkpoint_safetensors_files(snapshot_dir)
+    if not files:
+        logger.warning("No *.safetensors files in %s; using freshly initialised heads", snapshot_dir)
+        return
+
+    merged: dict[str, Tensor] = {}
+    for path in files:
+        merged.update(load_file(str(path)))
+
+    remapped = _remap_state_dict_keys(merged, model)
+
+    # Defensive vocab-match. With the corrected resize logic
+    # (``_resize_embeddings_for_robometer`` uses ``len(tokenizer) + 5``),
+    # a freshly built ``RobometerRewardModel`` should already share the same
+    # vocabulary as the upstream checkpoint (e.g. 151,674 for
+    # ``robometer/Robometer-4B``). This block stays in place as a safety net
+    # in case a future upstream variant uses a different vocab — we never
+    # want ``load_state_dict`` to trip on a silent shape mismatch.
+    base_model = getattr(model, "model", None)
+    if base_model is not None and hasattr(base_model, "get_input_embeddings"):
+        for key in (
+            "model.model.language_model.embed_tokens.weight",
+            "model.language_model.embed_tokens.weight",
+            "model.embed_tokens.weight",
+        ):
+            tensor = remapped.get(key)
+            if tensor is None:
+                continue
+            ckpt_vocab = int(tensor.shape[0])
+            current_vocab = int(base_model.get_input_embeddings().num_embeddings)
+            if ckpt_vocab != current_vocab:
+                logger.info(
+                    "Resizing model embed table %d -> %d to match upstream checkpoint vocab "
+                    "(upstream was trained against a different Qwen revision).",
+                    current_vocab,
+                    ckpt_vocab,
+                )
+                base_model.resize_token_embeddings(ckpt_vocab)
+            break
+
+    missing, unexpected = model.load_state_dict(remapped, strict=False)
+    if missing:
+        logger.debug("Robometer checkpoint missing %d keys (sample: %s)", len(missing), missing[:5])
+    if unexpected:
+        logger.debug(
+            "Robometer checkpoint had %d unexpected keys (sample: %s)", len(unexpected), unexpected[:5]
+        )

src/lerobot/rewards/robometer/configuration_robometer.py

@@ -0,0 +1,162 @@
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+from copy import deepcopy
+from dataclasses import dataclass, field
+from typing import TYPE_CHECKING, Any
+
+from lerobot.configs import FeatureType, NormalizationMode, PolicyFeature
+from lerobot.configs.rewards import RewardModelConfig
+from lerobot.utils.constants import OBS_IMAGES
+from lerobot.utils.import_utils import _transformers_available, require_package
+
+if TYPE_CHECKING or _transformers_available:
+    from transformers import AutoConfig, AutoTokenizer
+else:
+    AutoConfig = None  # type: ignore[assignment]
+    AutoTokenizer = None  # type: ignore[assignment]
+
+
+@RewardModelConfig.register_subclass("robometer")
+@dataclass
+class RobometerConfig(RewardModelConfig):
+    """Configuration for the Robometer reward model."""
+
+    pretrained_path: str | None = "lilkm/Robometer-4B"
+    image_key: str = OBS_IMAGES + ".top"
+    task_key: str = "task"
+    default_task: str | None = None
+
+    max_frames: int | None = 8
+    reward_output: str = "progress"  # "progress" or "success"
+    success_threshold: float = 0.5
+
+    license: str | None = "apache-2.0"
+    tags: list[str] | None = field(
+        default_factory=lambda: ["reward-model", "vision-language", "qwen3-vl", "zero-shot"]
+    )
+
+    base_model_id: str = "Qwen/Qwen3-VL-4B-Instruct"
+    torch_dtype: str = "bfloat16"
+    use_multi_image: bool = True
+    use_per_frame_progress_token: bool = True
+    average_temporal_patches: bool = True
+    frame_pooling: str = "mean"  # "mean" | "boundary" | "attention"
+    frame_pooling_attn_temperature: float = 1.0
+    progress_loss_type: str = "discrete"  # "l1" | "l2" | "discrete"
+    progress_discrete_bins: int = 10
+
+    # Serialised Qwen backbone config (post-resize). Always populated by
+    # ``__post_init__`` from ``base_model_id`` + ``len(tokenizer) + 5``, so it
+    # is never ``None`` after construction (EO-1 style). Saved into
+    # ``config.json`` automatically by the base ``_save_pretrained``.
+    vlm_config: dict[str, Any] | None = None
+
+    input_features: dict[str, PolicyFeature] = field(default_factory=dict)
+    output_features: dict[str, PolicyFeature] = field(default_factory=dict)
+    normalization_mapping: dict[str, NormalizationMode] = field(
+        default_factory=lambda: {
+            "VISUAL": NormalizationMode.IDENTITY,
+            "REWARD": NormalizationMode.IDENTITY,
+        }
+    )
+
+    def __post_init__(self) -> None:
+        super().__post_init__()
+        if self.reward_output not in {"progress", "success"}:
+            raise ValueError(f"reward_output must be 'progress' or 'success', got {self.reward_output!r}")
+        if self.max_frames is not None and self.max_frames < 1:
+            raise ValueError(f"max_frames must be >= 1, got {self.max_frames}")
+        if self.frame_pooling not in {"mean", "boundary", "attention"}:
+            raise ValueError(f"frame_pooling must be mean/boundary/attention; got {self.frame_pooling!r}")
+        if self.frame_pooling_attn_temperature <= 0:
+            raise ValueError("frame_pooling_attn_temperature must be > 0")
+        if self.progress_loss_type not in {"l1", "l2", "discrete"}:
+            raise ValueError(f"progress_loss_type must be l1/l2/discrete; got {self.progress_loss_type!r}")
+        if self.use_per_frame_progress_token and not self.use_multi_image:
+            raise ValueError("use_per_frame_progress_token=True requires use_multi_image=True")
+
+        if self.image_key not in self.input_features:
+            self.input_features[self.image_key] = PolicyFeature(shape=(3, 224, 224), type=FeatureType.VISUAL)
+        self.output_features.setdefault("progress", PolicyFeature(shape=(1,), type=FeatureType.REWARD))
+        self.output_features.setdefault("success", PolicyFeature(shape=(1,), type=FeatureType.REWARD))
+
+        # Deterministically populate ``vlm_config`` so it is never ``None``
+        # after construction (mirrors EO-1's ``__post_init__`` snapshot).
+        # The target vocab matches upstream Robometer's runtime resize
+        # ``base_model.resize_token_embeddings(len(processor.tokenizer))`` —
+        # see ``third_party/robometer/.../setup_utils.py`` —
+        # i.e. ``len(tokenizer) + len(ROBOMETER_SPECIAL_TOKENS)``.
+        #
+        # For ``Qwen/Qwen3-VL-4B-Instruct`` this gives 151,669 + 5 = 151,674,
+        # which is exactly the published ``robometer/Robometer-4B`` checkpoint
+        # vocab. NB: ``text_config.vocab_size`` in the raw Qwen config is the
+        # padded embedding-table size (151,936), not the tokenizer length —
+        # we override it with the tokenizer-driven value to stay consistent
+        # with upstream.
+        if self.vlm_config is None:
+            require_package("transformers", extra="robometer")
+            # Local import avoids a top-level cycle (modeling_robometer imports
+            # this module). ``ROBOMETER_SPECIAL_TOKENS`` is the single source
+            # of truth for the resize delta.
+            from lerobot.rewards.robometer.modeling_robometer import ROBOMETER_SPECIAL_TOKENS
+
+            vlm = AutoConfig.from_pretrained(self.base_model_id).to_dict()
+            tokenizer = AutoTokenizer.from_pretrained(self.base_model_id)
+            text_config = vlm.get("text_config")
+            if not isinstance(text_config, dict):
+                raise ValueError(
+                    f"Backbone config for {self.base_model_id!r} has no nested `text_config`; "
+                    "Robometer expects a Qwen-VL-style config."
+                )
+            text_config["vocab_size"] = len(tokenizer) + len(ROBOMETER_SPECIAL_TOKENS)
+            self.vlm_config = vlm
+
+    @property
+    def use_discrete_progress(self) -> bool:
+        """Whether the progress head outputs distribution logits over bins."""
+        return self.progress_loss_type.lower() == "discrete"
+
+    @property
+    def vlm_backbone_config(self):
+        """Reconstruct the Qwen backbone config from :attr:`vlm_config`.
+
+        ``vlm_config`` is always populated after :meth:`__post_init__`
+        (either fresh, computed from the tokenizer, or loaded from a saved
+        ``config.json`` via draccus).
+        """
+        require_package("transformers", extra="robometer")
+        config_dict = deepcopy(self.vlm_config)
+        model_type = config_dict.pop("model_type", None)
+        if model_type is None:
+            raise ValueError("vlm_config must include `model_type` to reconstruct the backbone config")
+        return AutoConfig.for_model(model_type, **config_dict)
+
+    @property
+    def observation_delta_indices(self) -> list[int] | None:
+        return None
+
+    @property
+    def action_delta_indices(self) -> None:
+        return None
+
+    @property
+    def reward_delta_indices(self) -> None:
+        return None
+
+    def validate_features(self) -> None:
+        if self.image_key not in self.input_features:
+            raise ValueError(f"Robometer requires image input feature {self.image_key!r}")

src/lerobot/rewards/robometer/modeling_robometer.py

@@ -0,0 +1,493 @@
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Robometer reward model.
+
+- Qwen3-VL backbone (default: ``Qwen/Qwen3-VL-4B-Instruct``).
+- Progress + success heads at inference; the preference head is preserved in the
+  state dict but not queried.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import TYPE_CHECKING, Any
+
+import torch
+from torch import Tensor, nn
+
+from lerobot.rewards.pretrained import PreTrainedRewardModel
+from lerobot.rewards.robometer.configuration_robometer import RobometerConfig
+from lerobot.utils.constants import OBS_PREFIX
+from lerobot.utils.import_utils import _transformers_available, require_package
+
+if TYPE_CHECKING or _transformers_available:
+    from transformers import AutoModelForImageTextToText
+else:
+    AutoModelForImageTextToText = None  # type: ignore[assignment]
+
+logger = logging.getLogger(__name__)
+
+# Namespace for Robometer's pre-encoded Qwen-VL observation tensors. The
+# processor writes both Qwen-VL tensors and Robometer-specific token ids /
+# metadata here; the model reads them at inference (no tokenizer needed in
+# the model — EO1-style separation).
+ROBOMETER_FEATURE_PREFIX = f"{OBS_PREFIX}robometer."
+ROBOMETER_QWEN_INPUT_KEYS = (
+    "input_ids",
+    "attention_mask",
+    "pixel_values",
+    "pixel_values_videos",
+    "image_grid_thw",
+    "video_grid_thw",
+    "second_per_grid_ts",
+)
+ROBOMETER_METADATA_KEYS = (
+    "prog_token_id",
+    "vision_start_token_id",
+    "vision_end_token_id",
+    "video_merge_size",
+)
+ROBOMETER_INPUT_KEYS = ROBOMETER_QWEN_INPUT_KEYS + ROBOMETER_METADATA_KEYS
+
+# Order matters: the released checkpoint resized `embed_tokens` after adding
+# these tokens in this order, so changing the set or order would silently
+# misalign the saved embedding rows with their token ids. `<|reward_token|>`
+# and `<|sim_token|>` are vestigial (never read by any head) but still occupy
+# rows the checkpoint expects.
+ROBOMETER_SPECIAL_TOKENS = (
+    "<|split_token|>",
+    "<|reward_token|>",
+    "<|pref_token|>",
+    "<|sim_token|>",
+    "<|prog_token|>",
+)
+
+
+def convert_bins_to_continuous(bin_logits: Tensor) -> Tensor:
+    """Collapse per-bin logits into a single value in ``[0, 1]``.
+
+    The discrete progress head outputs ``num_bins`` logits per frame. Bins are
+    evenly spaced centers in ``[0, 1]``; the continuous prediction is the
+    softmax-weighted mean of those centers.
+    """
+    bin_probs = torch.softmax(bin_logits, dim=-1)
+    num_bins = bin_logits.shape[-1]
+    bin_centers = torch.linspace(0.0, 1.0, num_bins, device=bin_logits.device, dtype=bin_logits.dtype)
+    return (bin_probs * bin_centers).sum(dim=-1)
+
+
+def squeeze_last_safe(x: Tensor) -> Tensor:
+    """Drop a trailing singleton dim only when present.
+
+    Matches the upstream helper of the same name in
+    ``robometer.models.rbm`` (kept module-level and non-underscored to mirror
+    upstream).
+    """
+    return x.squeeze(-1) if x.ndim > 1 and x.shape[-1] == 1 else x
+
+
+def _torch_dtype(name: str) -> torch.dtype:
+    dtype = getattr(torch, name, None)
+    if isinstance(dtype, torch.dtype):
+        return dtype
+    raise ValueError(f"Unknown torch dtype: {name!r}")
+
+
+class RobometerPredictionHead(nn.Sequential):
+    """Small MLP head used for Robometer's progress / success / preference outputs.
+
+    Subclasses ``nn.Sequential`` (not ``nn.Module``) so the ``state_dict`` keys
+    stay flat (``progress_head.0.weight``, ``progress_head.1.weight``, ...) and
+    remain byte-compatible with the published ``lilkm/robometer-4b`` checkpoint.
+    """
+
+    def __init__(self, hidden_dim: int, output_size: int, *, dropout: float, with_sigmoid: bool) -> None:
+        layers: list[nn.Module] = [
+            nn.Linear(hidden_dim, hidden_dim // 2),
+            nn.LayerNorm(hidden_dim // 2),
+            nn.GELU(),
+            nn.Dropout(dropout),
+            nn.Linear(hidden_dim // 2, output_size),
+        ]
+        if with_sigmoid:
+            layers.append(nn.Sigmoid())
+        super().__init__(*layers)
+
+
+def decode_progress_outputs(
+    progress_logits: Tensor | None,
+    success_logits: Tensor | None,
+    *,
+    is_discrete_mode: bool,
+) -> dict[str, list[list[float]]]:
+    """Decode RBM head outputs into per-frame floats.
+
+    Args:
+        progress_logits: ``(B, T)`` (continuous) or ``(B, T, num_bins)`` (discrete).
+        success_logits: ``(B, T)`` raw logits, ``sigmoid``-ed to probabilities.
+        is_discrete_mode: if True the progress logits get a softmax over bins
+            and are projected onto bin centers via :func:`convert_bins_to_continuous`.
+
+    Returns:
+        Dict with ``progress_pred`` and ``success_probs``, each a list of
+        length ``B`` of per-frame float lists.
+    """
+    progress_pred: list[list[float]] = []
+    success_probs: list[list[float]] = []
+
+    if progress_logits is not None:
+        for sample_logits in progress_logits:
+            if is_discrete_mode:
+                continuous = convert_bins_to_continuous(sample_logits.detach().float().cpu())
+                progress_pred.append(continuous.flatten().tolist())
+            else:
+                progress_pred.append(sample_logits.detach().float().cpu().flatten().tolist())
+
+    if success_logits is not None:
+        for sample_logits in success_logits:
+            success_probs.append(torch.sigmoid(sample_logits.detach().float().cpu()).flatten().tolist())
+
+    return {"progress_pred": progress_pred, "success_probs": success_probs}
+
+
+class RobometerRewardModel(PreTrainedRewardModel):
+    """Robometer reward model: Qwen3-VL backbone + progress/success heads."""
+
+    name = "robometer"
+    config_class = RobometerConfig
+
+    def __init__(self, config: RobometerConfig, *, dropout: float = 0.1) -> None:
+        require_package("transformers", extra="robometer")
+        super().__init__(config)
+        self.config = config
+
+        # Two backbone-build paths (EO-1 style, branched on ``pretrained_path``):
+        #
+        #   - Fresh training (``pretrained_path is None``): download the base
+        #     Qwen weights and resize the embed table to match
+        #     ``vlm_config.text_config.vocab_size`` — populated deterministically
+        #     in ``RobometerConfig.__post_init__`` as
+        #     ``len(tokenizer) + len(ROBOMETER_SPECIAL_TOKENS)``, mirroring
+        #     upstream Robometer's ``_add_special_tokens_and_resize`` in
+        #     ``third_party/robometer/.../setup_utils.py``.
+        #
+        #   - Loading a saved checkpoint (``pretrained_path`` is set): rebuild
+        #     the empty architecture from ``vlm_config`` via
+        #     ``AutoModelForImageTextToText.from_config`` so the subsequent
+        #     ``model.safetensors`` load is a direct fill of the right shape —
+        #     no redundant Qwen weight download.
+        torch_dtype = _torch_dtype(config.torch_dtype)
+        if config.pretrained_path is None:
+            self.model = AutoModelForImageTextToText.from_pretrained(
+                config.base_model_id,
+                dtype=torch_dtype,
+                trust_remote_code=True,
+            )
+            target_vocab = config.vlm_config["text_config"]["vocab_size"]
+            self.model.resize_token_embeddings(target_vocab)
+        else:
+            self.model = AutoModelForImageTextToText.from_config(
+                config.vlm_backbone_config,
+                dtype=torch_dtype,
+                trust_remote_code=True,
+            )
+
+        # All Qwen-VL backbones Robometer supports expose `text_config.hidden_size`.
+        # Falls back to the top-level `hidden_size` so future non-multimodal
+        # variants would still resolve.
+        backbone_config = self.model.config
+        text_config = getattr(backbone_config, "text_config", None)
+        hidden_size = getattr(text_config, "hidden_size", None) if text_config is not None else None
+        if hidden_size is None:
+            hidden_size = getattr(backbone_config, "hidden_size", None)
+        if hidden_size is None:
+            raise AttributeError(
+                f"Could not infer hidden_size from backbone config of {config.base_model_id}"
+            )
+        hidden_dim = int(hidden_size)
+
+        # Robometer's three prediction heads + frame-pool attention. The
+        # preference head is preserved to match the published state-dict layout
+        # even though only progress + success are consumed at inference, and
+        # `frame_pool_attn` is always allocated so checkpoints trained with
+        # `frame_pooling="attention"` load without remapping.
+        progress_output = config.progress_discrete_bins if config.use_discrete_progress else 1
+        self.progress_head = RobometerPredictionHead(
+            hidden_dim,
+            progress_output,
+            dropout=dropout,
+            with_sigmoid=not config.use_discrete_progress,
+        )
+        self.preference_head = RobometerPredictionHead(hidden_dim, 1, dropout=dropout, with_sigmoid=False)
+        self.success_head = RobometerPredictionHead(hidden_dim, 1, dropout=dropout, with_sigmoid=False)
+        self.frame_pool_attn = nn.Linear(hidden_dim, 1, bias=False)
+
+        # Match the dtype of the loaded base model so weight loading is a no-op cast.
+        model_dtype = next(self.model.parameters()).dtype
+        self.progress_head.to(dtype=model_dtype)
+        self.preference_head.to(dtype=model_dtype)
+        self.success_head.to(dtype=model_dtype)
+        self.frame_pool_attn.to(dtype=model_dtype)
+
+    def compute_reward(self, batch: dict[str, Tensor]) -> Tensor:
+        inputs = {
+            key: batch[f"{ROBOMETER_FEATURE_PREFIX}{key}"]
+            for key in ROBOMETER_INPUT_KEYS
+            if f"{ROBOMETER_FEATURE_PREFIX}{key}" in batch
+        }
+        if "input_ids" not in inputs:
+            raise KeyError(
+                f"Robometer batch missing pre-encoded inputs (expected "
+                f"`{ROBOMETER_FEATURE_PREFIX}input_ids`). Make sure the "
+                "RobometerEncoderProcessorStep ran before `compute_reward`."
+            )
+
+        device = next(self.model.parameters()).device
+        inputs = {key: value.to(device) if hasattr(value, "to") else value for key, value in inputs.items()}
+
+        self.eval()
+        with torch.no_grad():
+            progress_logits, success_logits = self._compute_rbm_logits(inputs)
+
+        decoded = decode_progress_outputs(
+            progress_logits,
+            success_logits,
+            is_discrete_mode=self.config.use_discrete_progress,
+        )
+        values = (
+            decoded["success_probs"] if self.config.reward_output == "success" else decoded["progress_pred"]
+        )
+
+        rewards = torch.stack([torch.as_tensor(seq, dtype=torch.float32)[-1] for seq in values])
+        if self.config.reward_output == "success":
+            rewards = (rewards > self.config.success_threshold).float()
+        return rewards.to(self.config.device or "cpu")
+
+    def _compute_rbm_logits(
+        self,
+        inputs: dict[str, Any],
+    ) -> tuple[Tensor, Tensor]:
+        """Run the Qwen3-VL backbone and apply Robometer's heads.
+
+        ``inputs`` is the encoded batch produced by
+        :class:`RobometerEncoderProcessorStep`. It carries Qwen tensors as well
+        as Robometer-specific metadata (``prog_token_id``,
+        ``vision_start_token_id``, ``vision_end_token_id``, ``video_merge_size``)
+        — the metadata is popped here so the rest can be forwarded straight to
+        the Qwen model.
+
+        Returns ``(progress_logits, success_logits)``. Shapes:
+
+        - ``progress_logits``: ``(B, T)`` (continuous) or ``(B, T, num_bins)`` (discrete).
+        - ``success_logits``: ``(B, T)`` raw logits (sigmoid happens at decode time).
+        """
+        prog_token_id = inputs.pop("prog_token_id", None)
+        vision_start_token_id = inputs.pop("vision_start_token_id", None)
+        vision_end_token_id = inputs.pop("vision_end_token_id", None)
+        video_merge_size = inputs.pop("video_merge_size", 14)
+
+        # Qwen3-VL doesn't reliably populate `last_hidden_state`; ask for the
+        # full hidden-state tuple and take the last layer. This matches the
+        # `is_qwen3` path in upstream Robometer's `RBM.forward_qwen` (main).
+        outputs = self.model(**inputs, output_hidden_states=True, return_dict=True)
+        hidden_state = (
+            outputs.hidden_states[-1]
+            if getattr(outputs, "hidden_states", None)
+            else outputs.last_hidden_state
+        )
+
+        input_ids = inputs["input_ids"]
+        if self.config.use_per_frame_progress_token:
+            if prog_token_id is None:
+                raise KeyError("`prog_token_id` missing in batch (run RobometerEncoderProcessorStep first)")
+            return self._process_token_extraction(hidden_state, input_ids, prog_token_id=prog_token_id)
+        if self.config.use_multi_image:
+            if vision_start_token_id is None or vision_end_token_id is None:
+                raise KeyError(
+                    "`vision_start_token_id` / `vision_end_token_id` missing in batch "
+                    "(run RobometerEncoderProcessorStep first)"
+                )
+            return self._process_multi_image_frames(
+                hidden_state,
+                input_ids,
+                start_id=vision_start_token_id,
+                end_id=vision_end_token_id,
+            )
+        video_grid_thw = inputs.get("video_grid_thw")
+        if video_grid_thw is None:
+            raise ValueError("video_grid_thw is required for video-mode Robometer inference")
+        if vision_start_token_id is None:
+            raise KeyError("`vision_start_token_id` missing in batch")
+        return self._process_video_frames(
+            hidden_state,
+            input_ids,
+            video_grid_thw,
+            start_id=vision_start_token_id,
+            merge_size=video_merge_size,
+        )
+
+    def _apply_heads_to_hidden_states(self, frame_embeddings: Tensor) -> tuple[Tensor, Tensor]:
+        """Apply progress + success heads to a tensor of frame embeddings.
+
+        Mirrors upstream ``RBM._apply_heads_to_hidden_states``.
+        """
+        progress_out = self.progress_head(frame_embeddings)
+        progress = progress_out if self.config.use_discrete_progress else squeeze_last_safe(progress_out)
+        success = squeeze_last_safe(self.success_head(frame_embeddings))
+        return progress, success
+
+    def _process_token_extraction(
+        self,
+        hidden_state: Tensor,
+        input_ids: Tensor,
+        *,
+        prog_token_id: int,
+    ) -> tuple[Tensor, Tensor]:
+        """Per-frame progress/success from ``<|prog_token|>`` positions.
+
+        Mirrors the progress-sample branch of upstream
+        ``RBM._process_token_extraction``.
+        """
+        token_mask = input_ids == prog_token_id
+        batch_indices, positions = token_mask.nonzero(as_tuple=True)
+        if positions.numel() == 0:
+            raise ValueError("`<|prog_token|>` not found in any sequence")
+
+        per_sample_hidden = [
+            hidden_state[i, positions[batch_indices == i]] for i in range(input_ids.shape[0])
+        ]
+        progress_list, success_list = [], []
+        for embeddings in per_sample_hidden:
+            if embeddings.shape[0] == 0:
+                raise ValueError("`<|prog_token|>` missing in a sequence")
+            progress, success = self._apply_heads_to_hidden_states(embeddings)
+            progress_list.append(progress)
+            success_list.append(success)
+
+        return torch.stack(progress_list), torch.stack(success_list)
+
+    def _process_multi_image_frames(
+        self,
+        hidden_state: Tensor,
+        input_ids: Tensor,
+        *,
+        start_id: int,
+        end_id: int,
+    ) -> tuple[Tensor, Tensor]:
+        """Per-frame progress/success in multi-image mode (Qwen-VL).
+
+        Mirrors upstream ``RBM._process_multi_image_frames`` (progress-sample
+        branch only — we don't run preference at inference).
+        """
+        progress_list, success_list = [], []
+        for batch_idx in range(input_ids.shape[0]):
+            seq_ids = input_ids[batch_idx]
+            seq_hidden = hidden_state[batch_idx]
+            frame_embeddings = self._extract_hidden_states_from_token_pairs(
+                seq_hidden, seq_ids, start_id, end_id
+            )
+            progress, success = self._apply_heads_to_hidden_states(frame_embeddings)
+            progress_list.append(progress)
+            success_list.append(success)
+
+        return torch.stack(progress_list), torch.stack(success_list)
+
+    def _extract_hidden_states_from_token_pairs(
+        self,
+        hidden_state: Tensor,
+        input_ids: Tensor,
+        start_id: int,
+        end_id: int,
+    ) -> Tensor:
+        start_positions = (input_ids == start_id).nonzero(as_tuple=True)[0]
+        end_positions = (input_ids == end_id).nonzero(as_tuple=True)[0]
+        if start_positions.numel() == 0:
+            raise ValueError("`<|vision_start|>` not found in sequence")
+        if start_positions.numel() != end_positions.numel():
+            raise ValueError(
+                f"Mismatched vision token counts: {start_positions.numel()} start vs "
+                f"{end_positions.numel()} end"
+            )
+
+        frames: list[Tensor] = []
+        for start, end in zip(start_positions.tolist(), end_positions.tolist(), strict=True):
+            if start >= end:
+                raise ValueError(f"Invalid vision token pair: start={start} end={end}")
+            patch_tokens = hidden_state[start + 1 : end]
+            if patch_tokens.shape[0] == 0:
+                frames.append((hidden_state[start] + hidden_state[end]) / 2.0)
+                continue
+
+            pooling = self.config.frame_pooling
+            if pooling == "mean":
+                frames.append(patch_tokens.mean(dim=0))
+            elif pooling == "boundary":
+                frames.append(patch_tokens[-1])
+            else:  # attention
+                scores = (
+                    self.frame_pool_attn(patch_tokens).squeeze(-1)
+                    / self.config.frame_pooling_attn_temperature
+                )
+                weights = torch.softmax(scores, dim=0).unsqueeze(-1)
+                frames.append((weights * patch_tokens).sum(dim=0))
+
+        return torch.stack(frames)
+
+    def _process_video_frames(
+        self,
+        hidden_state: Tensor,
+        input_ids: Tensor,
+        video_grid_thw: Tensor,
+        *,
+        start_id: int,
+        merge_size: int,
+    ) -> tuple[Tensor, Tensor]:
+        """Per-frame progress/success in video mode (Qwen-VL).
+
+        Mirrors upstream ``RBM._process_video_frames`` /
+        ``RBM._extract_progress_from_trajectory`` (progress-sample branch
+        only — preference is not run at inference). In particular,
+        ``average_temporal_patches=False`` reads the *boundary* token at
+        ``cursor + tokens_per_frame`` to match upstream byte-for-byte.
+        """
+        progress_list, success_list = [], []
+        for batch_idx in range(input_ids.shape[0]):
+            seq_ids = input_ids[batch_idx]
+            seq_hidden = hidden_state[batch_idx]
+            start_positions = (seq_ids == start_id).nonzero(as_tuple=True)[0]
+            if start_positions.numel() == 0:
+                raise ValueError("`<|vision_start|>` not found in sequence")
+            t_dim, h_dim, w_dim = (int(x) for x in video_grid_thw[batch_idx].tolist())
+            tokens_per_frame = (h_dim * w_dim) // (merge_size**2)
+
+            cursor = start_positions[0].item()
+            frame_embeddings: list[Tensor] = []
+            for _ in range(t_dim):
+                if self.config.average_temporal_patches:
+                    patch = seq_hidden[cursor : cursor + tokens_per_frame]
+                    frame_embeddings.append(patch.mean(dim=0))
+                else:
+                    # Upstream takes the position *one past* the patch span as
+                    # the per-frame boundary; see
+                    # `RBM._extract_progress_from_trajectory`.
+                    frame_embeddings.append(seq_hidden[cursor + tokens_per_frame])
+                cursor += tokens_per_frame
+
+            stacked = torch.stack(frame_embeddings)
+            progress, success = self._apply_heads_to_hidden_states(stacked)
+            progress_list.append(progress)
+            success_list.append(success)
+
+        return torch.stack(progress_list), torch.stack(success_list)

src/lerobot/rewards/robometer/processor_robometer.py

@@ -0,0 +1,348 @@
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Robometer pre/post processing pipelines."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import TYPE_CHECKING, Any
+
+import numpy as np
+import torch
+from PIL import Image
+from torch import Tensor
+
+from lerobot.configs import PipelineFeatureType, PolicyFeature
+from lerobot.processor import (
+    AddBatchDimensionProcessorStep,
+    DeviceProcessorStep,
+    PolicyAction,
+    PolicyProcessorPipeline,
+    ProcessorStep,
+    ProcessorStepRegistry,
+    policy_action_to_transition,
+)
+from lerobot.rewards.robometer.configuration_robometer import RobometerConfig
+from lerobot.rewards.robometer.modeling_robometer import (
+    ROBOMETER_FEATURE_PREFIX,
+    ROBOMETER_SPECIAL_TOKENS,
+)
+from lerobot.types import EnvTransition, TransitionKey
+from lerobot.utils.constants import (
+    OBS_IMAGES,
+    POLICY_POSTPROCESSOR_DEFAULT_NAME,
+    POLICY_PREPROCESSOR_DEFAULT_NAME,
+)
+from lerobot.utils.import_utils import _transformers_available, require_package
+
+if TYPE_CHECKING or _transformers_available:
+    from transformers import AutoProcessor
+else:
+    AutoProcessor = None
+
+PROGRESS_PROMPT = (
+    "The task for the robot is '{task}'. Given the trajectory video, predict "
+    "the task progress at each frame, how far along the robot is towards "
+    "completing the task, a float between 0 and 1, where 0 is the starting "
+    "state and 1 is when the task is completed. If the robot is not "
+    "performing the same task, predict 0 progress."
+)
+
+
+def _frames_to_pil(frames: np.ndarray) -> list[Image.Image]:
+    """Convert ``(T, H, W, C)`` uint8 frames to a list of PIL images."""
+    if frames.ndim != 4:
+        raise ValueError(f"Expected (T,H,W,C) frames; got shape {frames.shape}")
+    if frames.dtype != np.uint8:
+        frames = np.clip(frames, 0, 255).astype(np.uint8)
+    return [Image.fromarray(frames[i]) for i in range(frames.shape[0])]
+
+
+def _video_to_numpy(video: Tensor, *, max_frames: int | None) -> np.ndarray:
+    """Convert one trajectory tensor to a ``(T, H, W, C) uint8`` numpy array."""
+    if max_frames is not None:
+        video = video[-max_frames:]
+    if video.shape[1] in (1, 3):
+        video = video.permute(0, 2, 3, 1)
+    elif video.shape[-1] not in (1, 3):
+        raise ValueError(f"Expected channel dim of size 1 or 3, got shape {tuple(video.shape)}")
+
+    array = video.detach().cpu().numpy()
+    if np.issubdtype(array.dtype, np.floating) and array.size > 0 and array.max() <= 1.0:
+        array = array * 255.0
+    return np.clip(array, 0, 255).astype(np.uint8)
+
+
+def _expand_tasks(task: Any, *, batch_size: int, default: str | None) -> list[str]:
+    if task is None:
+        task = default
+    if task is None:
+        raise KeyError("Robometer expected a task description in complementary data")
+    if isinstance(task, str):
+        return [task] * batch_size
+    if isinstance(task, tuple):
+        task = list(task)
+    if not (isinstance(task, list) and all(isinstance(item, str) for item in task)):
+        raise TypeError(f"Robometer task must be a string or list of strings, got {type(task)}")
+    if len(task) == 1 and batch_size > 1:
+        return task * batch_size
+    if len(task) != batch_size:
+        raise ValueError(f"Expected {batch_size} tasks, got {len(task)}")
+    return task
+
+
+@dataclass
+@ProcessorStepRegistry.register(name="robometer_encoder")
+class RobometerEncoderProcessorStep(ProcessorStep):
+    """Encode raw frames + task into Qwen-VL tensors for the Robometer model.
+
+    Loads a :class:`~transformers.AutoProcessor` matching ``base_model_id`` and
+    registers Robometer's special tokens on the tokenizer. The matching
+    embedding resize happens model-side in
+    :meth:`RobometerRewardModel.__init__`. This step owns the tokenizer — the
+    model itself never needs one — and is the EO1-style boundary between
+    pre-processing and modeling.
+
+    At call time the step reads:
+
+    - ``observation[image_key]``: ``(B, T, C, H, W)`` or ``(B, C, H, W)`` frames.
+    - ``complementary_data[task_key]``: a string or list of strings.
+
+    and writes ``observation[f"{ROBOMETER_FEATURE_PREFIX}<name>"]`` for:
+
+    - the Qwen-VL processor outputs: ``input_ids``, ``attention_mask``,
+      ``pixel_values``, ``image_grid_thw``, ``video_grid_thw``, ...
+    - Robometer-specific token ids consumed by the model heads:
+      ``prog_token_id``, ``vision_start_token_id``, ``vision_end_token_id``,
+      ``video_merge_size``.
+    """
+
+    base_model_id: str = "Qwen/Qwen3-VL-4B-Instruct"
+    image_key: str = OBS_IMAGES + ".top"
+    task_key: str = "task"
+    default_task: str | None = None
+    max_frames: int | None = 8
+    use_multi_image: bool = True
+    use_per_frame_progress_token: bool = True
+    max_length: int = 1024
+
+    _processor: Any = field(default=None, init=False, repr=False)
+
+    def __post_init__(self) -> None:
+        require_package("transformers", extra="robometer")
+        require_package("qwen-vl-utils", extra="robometer", import_name="qwen_vl_utils")
+
+        self._processor = AutoProcessor.from_pretrained(
+            self.base_model_id,
+            trust_remote_code=True,
+            do_sample_frames=False,
+            padding_side="right",
+        )
+
+        # Register Robometer's special tokens on the tokenizer. The matching
+        # embedding resize happens model-side in `RobometerRewardModel.__init__`.
+        tokenizer = self._processor.tokenizer
+        # Qwen tokenizers may not define a pad token, but batched prompts/videos
+        # require padding, so reuse EOS as the padding token.
+        if tokenizer.pad_token is None:
+            tokenizer.pad_token = tokenizer.eos_token
+        for token in ROBOMETER_SPECIAL_TOKENS:
+            if token not in tokenizer.get_vocab():
+                tokenizer.add_special_tokens({"additional_special_tokens": [token]})
+
+    def __call__(self, transition: EnvTransition) -> EnvTransition:
+        observation = transition.get(TransitionKey.OBSERVATION)
+        complementary = transition.get(TransitionKey.COMPLEMENTARY_DATA) or {}
+        if not isinstance(observation, dict):
+            raise ValueError("RobometerEncoderProcessorStep requires an observation dict")
+
+        if self.image_key not in observation:
+            raise KeyError(f"Robometer expected image key {self.image_key!r} in observation")
+
+        frames = observation[self.image_key]
+        tensor = frames.detach().cpu() if isinstance(frames, Tensor) else torch.as_tensor(frames)
+        if tensor.ndim == 4:
+            tensor = tensor.unsqueeze(1)
+        elif tensor.ndim != 5:
+            raise ValueError(
+                f"Expected Robometer frames with shape (B,C,H,W) or (B,T,C,H,W); got {tuple(tensor.shape)}"
+            )
+
+        batch_size = tensor.shape[0]
+        tasks = _expand_tasks(
+            complementary.get(self.task_key, self.default_task),
+            batch_size=batch_size,
+            default=self.default_task,
+        )
+
+        samples = [
+            (_video_to_numpy(tensor[i], max_frames=self.max_frames), tasks[i]) for i in range(batch_size)
+        ]
+        encoded = self.encode_samples(samples)
+
+        new_observation = dict(observation)
+        for key, value in encoded.items():
+            new_observation[f"{ROBOMETER_FEATURE_PREFIX}{key}"] = value
+
+        new_transition = transition.copy()
+        new_transition[TransitionKey.OBSERVATION] = new_observation
+        return new_transition
+
+    def encode_samples(self, samples: list[tuple[np.ndarray, str]]) -> dict[str, Tensor]:
+        """Run the Qwen-VL processor on a list of ``(frames, task)`` samples.
+
+        Used internally by ``__call__`` and exposed for callers that want to
+        run the encoder on a single trajectory without building an
+        :class:`EnvTransition` (see ``examples/dataset/create_robometer_progress_videos.py``).
+        """
+        from qwen_vl_utils import process_vision_info
+
+        conversations = [self._build_conversation(frames, task) for frames, task in samples]
+
+        texts = [
+            self._processor.apply_chat_template(
+                msg,
+                tokenize=False,
+                add_generation_prompt=False,
+                add_vision_id=True,
+                enable_thinking=False,
+                fps=1,
+            )
+            for msg in conversations
+        ]
+
+        process_kwargs: dict[str, Any] = {
+            "return_video_kwargs": True,
+            "return_video_metadata": True,
+        }
+        image_processor = getattr(self._processor, "image_processor", None)
+        if image_processor is not None and hasattr(image_processor, "patch_size"):
+            process_kwargs["image_patch_size"] = image_processor.patch_size
+
+        image_inputs, video_inputs, video_kwargs = process_vision_info(conversations, **process_kwargs)
+
+        videos: list[Any] | None = None
+        video_metadatas: list[Any] | None = None
+        if video_inputs:
+            if isinstance(video_inputs[0], tuple) and len(video_inputs[0]) == 2:
+                videos_seq, metadatas_seq = zip(*video_inputs, strict=False)
+                videos = list(videos_seq)
+                video_metadatas = list(metadatas_seq)
+            else:
+                videos = list(video_inputs)
+
+        processor_kwargs: dict[str, Any] = {
+            "text": texts,
+            "images": image_inputs,
+            "padding": True,
+            "truncation": False,
+            "max_length": self.max_length,
+            "return_tensors": "pt",
+            "do_resize": False,
+        }
+        if videos is not None:
+            processor_kwargs["videos"] = videos
+        if video_metadatas is not None:
+            processor_kwargs["video_metadata"] = video_metadatas
+        if video_kwargs:
+            processor_kwargs.update(video_kwargs)
+
+        encoded = self._processor(**processor_kwargs)
+
+        # Write Robometer-specific token ids and the video patch merge size into
+        # the encoded batch so `RobometerRewardModel` doesn't need its own
+        # tokenizer at inference (EO1-style separation: the processor owns the
+        # tokenizer, the model owns the backbone and heads).
+        tokenizer = self._processor.tokenizer
+        encoded["prog_token_id"] = tokenizer.convert_tokens_to_ids("<|prog_token|>")
+        encoded["vision_start_token_id"] = tokenizer.convert_tokens_to_ids("<|vision_start|>")
+        encoded["vision_end_token_id"] = tokenizer.convert_tokens_to_ids("<|vision_end|>")
+        video_processor = getattr(self._processor, "video_processor", None)
+        encoded["video_merge_size"] = int(getattr(video_processor, "merge_size", 14))
+        return encoded
+
+    def _build_conversation(self, frames: np.ndarray, task: str) -> list[dict[str, Any]]:
+        pil_frames = _frames_to_pil(frames)
+        prompt = PROGRESS_PROMPT.format(task=task)
+        content: list[dict[str, Any]] = [{"type": "text", "text": prompt}]
+
+        if self.use_multi_image:
+            for image in pil_frames:
+                content.append({"type": "image", "image": image})
+                if self.use_per_frame_progress_token:
+                    content.append({"type": "text", "text": "<|prog_token|>"})
+        else:
+            content.append({"type": "video", "video": pil_frames, "sample_fps": 1.0})
+
+        return [{"role": "user", "content": content}]
+
+    def transform_features(
+        self, features: dict[PipelineFeatureType, dict[str, PolicyFeature]]
+    ) -> dict[PipelineFeatureType, dict[str, PolicyFeature]]:
+        # The Qwen-VL processor produces variable-length sequence tensors that
+        # don't fit the static `PolicyFeature(shape=...)` mould; we deliberately
+        # do not advertise the new observation keys here.
+        return features
+
+    def get_config(self) -> dict[str, Any]:
+        return {
+            "base_model_id": self.base_model_id,
+            "image_key": self.image_key,
+            "task_key": self.task_key,
+            "default_task": self.default_task,
+            "max_frames": self.max_frames,
+            "use_multi_image": self.use_multi_image,
+            "use_per_frame_progress_token": self.use_per_frame_progress_token,
+            "max_length": self.max_length,
+        }
+
+
+def make_robometer_pre_post_processors(
+    config: RobometerConfig,
+    dataset_stats: dict[str, dict[str, Any]] | None = None,
+) -> tuple[
+    PolicyProcessorPipeline[dict[str, Any], dict[str, Any]],
+    PolicyProcessorPipeline[PolicyAction, PolicyAction],
+]:
+    """Pipeline that pre-encodes frames + task into Qwen-VL tensors.
+
+    The preprocessor adds a batch dimension if needed, runs Robometer's
+    encoder, and moves everything to the configured device. The
+    postprocessor is the identity since Robometer outputs a single reward
+    tensor (no action to un-normalise).
+    """
+    del dataset_stats  # Robometer has its own normalisation inside the Qwen-VL processor.
+
+    preprocessor = PolicyProcessorPipeline[dict[str, Any], dict[str, Any]](
+        steps=[
+            AddBatchDimensionProcessorStep(),
+            RobometerEncoderProcessorStep(
+                base_model_id=config.base_model_id,
+                image_key=config.image_key,
+                task_key=config.task_key,
+                default_task=config.default_task,
+                max_frames=config.max_frames,
+                use_multi_image=config.use_multi_image,
+                use_per_frame_progress_token=config.use_per_frame_progress_token,
+            ),
+            DeviceProcessorStep(device=config.device or "cpu"),
+        ],
+        name=POLICY_PREPROCESSOR_DEFAULT_NAME,
+    )
+    postprocessor = PolicyProcessorPipeline(
+        name=POLICY_POSTPROCESSOR_DEFAULT_NAME,
+        to_transition=policy_action_to_transition,
+    )
+    return preprocessor, postprocessor

src/lerobot/rewards/sarm/compute_rabc_weights.py

@@ -58,10 +58,9 @@ import torch
 from tqdm import tqdm
 
 from lerobot.datasets import LeRobotDataset
-
-from .modeling_sarm import SARMRewardModel
-from .processor_sarm import make_sarm_pre_post_processors
-from .sarm_utils import normalize_stage_tau
+from lerobot.rewards.sarm.modeling_sarm import SARMRewardModel
+from lerobot.rewards.sarm.processor_sarm import make_sarm_pre_post_processors
+from lerobot.rewards.sarm.sarm_utils import normalize_stage_tau
 
 
 def get_reward_model_path_from_parquet(parquet_path: Path) -> str | None:

src/lerobot/rewards/sarm/modeling_sarm.py

@@ -32,14 +32,13 @@ import torch.nn as nn
 import torch.nn.functional as F  # noqa: N812
 from torch import Tensor
 
-from lerobot.utils.constants import OBS_STR
-
-from ..pretrained import PreTrainedRewardModel
-from .configuration_sarm import SARMConfig
-from .sarm_utils import (
+from lerobot.rewards.pretrained import PreTrainedRewardModel
+from lerobot.rewards.sarm.configuration_sarm import SARMConfig
+from lerobot.rewards.sarm.sarm_utils import (
     normalize_stage_tau,
     pad_state_to_max_dim,
 )
+from lerobot.utils.constants import OBS_STR
 
 
 class StageTransformer(nn.Module):

src/lerobot/rewards/sarm/processor_sarm.py

@@ -58,16 +58,15 @@ from lerobot.processor import (
     policy_action_to_transition,
     transition_to_policy_action,
 )
-from lerobot.types import EnvTransition, PolicyAction, TransitionKey
-from lerobot.utils.constants import POLICY_POSTPROCESSOR_DEFAULT_NAME, POLICY_PREPROCESSOR_DEFAULT_NAME
-
-from .configuration_sarm import SARMConfig
-from .sarm_utils import (
+from lerobot.rewards.sarm.configuration_sarm import SARMConfig
+from lerobot.rewards.sarm.sarm_utils import (
     apply_rewind_augmentation,
     compute_absolute_indices,
     find_stage_and_tau,
     pad_state_to_max_dim,
 )
+from lerobot.types import EnvTransition, PolicyAction, TransitionKey
+from lerobot.utils.constants import POLICY_POSTPROCESSOR_DEFAULT_NAME, POLICY_PREPROCESSOR_DEFAULT_NAME
 
 
 class SARMEncodingProcessorStep(ProcessorStep):

src/lerobot/rewards/topreward/compute_rabc_weights.py

@@ -1,353 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Compute per-frame TOPReward progress curves for a LeRobot dataset.
-
-For each episode, scores trajectory prefixes of increasing length using
-the TOPReward reward model, min-max normalises the raw log-prob rewards per episode,
-and writes a parquet file with one row per frame.
-
-The parquet uses the same schema as SARM's :mod:`lerobot.rewards.sarm.compute_rabc_weights`.
-
-Usage:
-    # Sparse-dense mode (15 anchors per episode, matches upstream)
-    python -m lerobot.rewards.topreward.compute_rabc_weights \\
-        --dataset-repo-id lerobot/libero_10_image \\
-        --num-samples 15
-
-    # Use a different VLM backbone
-    python -m lerobot.rewards.topreward.compute_rabc_weights \\
-        --dataset-repo-id lerobot/libero_10_image \\
-        --vlm-name Qwen/Qwen3-VL-4B-Instruct
-"""
-
-from __future__ import annotations
-
-import argparse
-import logging
-from pathlib import Path
-from typing import Any
-
-import numpy as np
-import pyarrow as pa
-import pyarrow.parquet as pq
-import torch
-from tqdm import tqdm
-
-from lerobot.datasets import LeRobotDataset
-from lerobot.rewards.topreward.configuration_topreward import TOPRewardConfig
-from lerobot.rewards.topreward.modeling_topreward import TOPRewardModel
-from lerobot.rewards.topreward.processor_topreward import TOPRewardEncoderProcessorStep
-from lerobot.types import TransitionKey
-
-DEFAULT_OUTPUT_FILENAME = "topreward_progress.parquet"
-
-
-def get_reward_model_path_from_parquet(parquet_path: Path) -> str | None:
-    """Read ``reward_model_path`` from parquet metadata if available."""
-    if not parquet_path.exists():
-        return None
-    try:
-        metadata = pq.read_metadata(parquet_path).schema.to_arrow_schema().metadata
-        if metadata and b"reward_model_path" in metadata:
-            return metadata[b"reward_model_path"].decode()
-    except Exception:  # nosec B110
-        return None
-    return None
-
-
-def _resolve_task(sample: dict[str, Any], default: str) -> str:
-    """Best-effort task extraction from a dataset sample."""
-    task = sample.get("task")
-    if isinstance(task, str) and task:
-        return task
-    return default
-
-
-def normalize_rewards(rewards: list[float] | np.ndarray) -> np.ndarray:
-    """Min-max normalise raw log-prob rewards into ``[0, 1]``."""
-    rewards_arr = np.asarray(rewards, dtype=np.float64)
-    if rewards_arr.size == 0:
-        return rewards_arr.astype(np.float32)
-    if rewards_arr.size == 1:
-        return np.array([1.0], dtype=np.float32)
-    r_min, r_max = rewards_arr.min(), rewards_arr.max()
-    if r_max == r_min:
-        return np.ones_like(rewards_arr, dtype=np.float32)
-    return ((rewards_arr - r_min) / (r_max - r_min)).astype(np.float32)
-
-
-def compute_instruction_rewards_for_prefixes(
-    model: TOPRewardModel,
-    encoder: TOPRewardEncoderProcessorStep,
-    dataset: LeRobotDataset,
-    ep_start: int,
-    num_frames: int,
-    task: str,
-    image_key: str,
-    num_samples: int | None,
-    device: str,
-) -> np.ndarray:
-    """Score an episode via prefix sweep and return a per-frame normalised curve."""
-    if num_samples is None or num_samples >= num_frames:
-        prefix_lengths = np.arange(1, num_frames + 1, dtype=np.int64)
-    else:
-        prefix_lengths = np.unique(np.linspace(1, num_frames, num_samples).round().astype(np.int64))
-
-    episode_frames = torch.stack([dataset[ep_start + i][image_key] for i in range(num_frames)])
-    rewards: list[float] = []
-    for length in prefix_lengths:
-        frames = episode_frames[: int(length)].unsqueeze(0)  # (1, T, C, H, W)
-
-        transition = {
-            TransitionKey.OBSERVATION: {image_key: frames},
-            TransitionKey.COMPLEMENTARY_DATA: {"task": task},
-        }
-        encoded = encoder(transition)
-        obs = encoded[TransitionKey.OBSERVATION]
-        batch = {
-            key: value.to(device) if isinstance(value, torch.Tensor) else value for key, value in obs.items()
-        }
-
-        with torch.no_grad():
-            reward = model.compute_reward(batch)
-        rewards.append(float(reward.item()))
-
-    normalized_rewards = normalize_rewards(rewards)
-
-    if prefix_lengths.shape[0] == num_frames:
-        return normalized_rewards
-
-    return np.interp(
-        np.arange(1, num_frames + 1, dtype=np.float64),
-        prefix_lengths.astype(np.float64),
-        normalized_rewards.astype(np.float64),
-    ).astype(np.float32)
-
-
-def compute_topreward_progress(
-    dataset_repo_id: str,
-    reward_model_path: str | None = None,
-    vlm_name: str | None = None,
-    output_path: str | None = None,
-    device: str = "cuda",
-    num_samples: int | None = None,
-    fps: float | None = None,
-    episodes: list[int] | None = None,
-) -> Path:
-    """Run TOPReward over a dataset and write per-frame progress."""
-    if reward_model_path is not None:
-        logging.info(f"Loading TOPReward config from: {reward_model_path}")
-        model = TOPRewardModel.from_pretrained(reward_model_path)
-        config = model.config
-        config.device = device
-        if vlm_name is not None and vlm_name != config.vlm_name:
-            logging.info(f"Overriding vlm_name from config: {config.vlm_name} -> {vlm_name}")
-            config.vlm_name = vlm_name
-            model = TOPRewardModel(config)
-    else:
-        config_kwargs: dict[str, Any] = {"device": device}
-        if vlm_name is not None:
-            config_kwargs["vlm_name"] = vlm_name
-        if fps is not None:
-            config_kwargs["fps"] = fps
-        config = TOPRewardConfig(**config_kwargs)
-        logging.info(f"Constructing TOPReward with VLM: {config.vlm_name}")
-        model = TOPRewardModel(config)
-
-    model.to(device).eval()
-
-    encoder = TOPRewardEncoderProcessorStep(
-        vlm_name=config.vlm_name,
-        image_key=config.image_key,
-        task_key=config.task_key,
-        default_task=config.default_task,
-        max_frames=None,  # no tail-crop: we control prefix length explicitly
-        fps=config.fps,
-        prompt_prefix=config.prompt_prefix,
-        prompt_suffix_template=config.prompt_suffix_template,
-        add_chat_template=config.add_chat_template,
-        max_length=config.max_input_length,
-    )
-
-    image_key = config.image_key
-
-    logging.info(f"Loading dataset: {dataset_repo_id}")
-    dataset = LeRobotDataset(dataset_repo_id, download_videos=True)
-    logging.info(f"Dataset: {dataset.num_episodes} episodes, {dataset.num_frames} frames")
-
-    episode_indices = list(range(dataset.num_episodes)) if episodes is None else episodes
-    logging.info(f"Processing {len(episode_indices)} episode(s)")
-
-    all_index: list[int] = []
-    all_episode: list[int] = []
-    all_frame: list[int] = []
-    all_progress: list[float] = []
-
-    for episode_idx in tqdm(episode_indices, desc="Episodes"):
-        ep = dataset.meta.episodes[episode_idx]
-        ep_start = int(ep["dataset_from_index"])
-        ep_end = int(ep["dataset_to_index"])
-        num_frames = ep_end - ep_start
-        if num_frames <= 0:
-            continue
-
-        first_sample = dataset[ep_start]
-        task = _resolve_task(first_sample, default=config.default_task or "perform the task")
-
-        per_frame = compute_instruction_rewards_for_prefixes(
-            model=model,
-            encoder=encoder,
-            dataset=dataset,
-            ep_start=ep_start,
-            num_frames=num_frames,
-            task=task,
-            image_key=image_key,
-            num_samples=num_samples,
-            device=device,
-        )
-
-        for local in range(num_frames):
-            all_index.append(ep_start + local)
-            all_episode.append(episode_idx)
-            all_frame.append(local)
-            all_progress.append(float(per_frame[local]))
-
-        if device.startswith("cuda"):
-            torch.cuda.empty_cache()
-
-    table = pa.table(
-        {
-            "index": np.asarray(all_index, dtype=np.int64),
-            "episode_index": np.asarray(all_episode, dtype=np.int64),
-            "frame_index": np.asarray(all_frame, dtype=np.int64),
-            "progress_sparse": np.asarray(all_progress, dtype=np.float32),
-        }
-    )
-
-    schema_metadata: dict[bytes, bytes] = {b"vlm_name": config.vlm_name.encode()}
-    if reward_model_path is not None:
-        schema_metadata[b"reward_model_path"] = reward_model_path.encode()
-    table = table.replace_schema_metadata(schema_metadata)
-
-    out = Path(dataset.root) / DEFAULT_OUTPUT_FILENAME if output_path is None else Path(output_path)
-    out.parent.mkdir(parents=True, exist_ok=True)
-    pq.write_table(table, out)
-    logging.info(f"Saved {len(table)} frame values to {out}")
-
-    progress_arr = np.asarray(all_progress, dtype=np.float32)
-    if progress_arr.size:
-        logging.info(
-            f"Progress: mean={float(progress_arr.mean()):.4f}, "
-            f"std={float(progress_arr.std()):.4f}, "
-            f"min={float(progress_arr.min()):.4f}, "
-            f"max={float(progress_arr.max()):.4f}"
-        )
-    return out
-
-
-def main():
-    parser = argparse.ArgumentParser(
-        description="Compute per-frame TOPReward progress curves for RA-BC weighting.",
-        formatter_class=argparse.RawDescriptionHelpFormatter,
-        epilog="""
-Examples:
-    # Sparse-dense mode (matches upstream TOPReward num_samples=15)
-    python -m lerobot.rewards.topreward.compute_rabc_weights \\
-        --dataset-repo-id lerobot/libero_10_image \\
-        --num-samples 15
-
-    # Use a smaller VLM
-    python -m lerobot.rewards.topreward.compute_rabc_weights \\
-        --dataset-repo-id lerobot/libero_10_image \\
-        --vlm-name Qwen/Qwen3-VL-4B-Instruct
-        """,
-    )
-    parser.add_argument(
-        "--dataset-repo-id", type=str, required=True, help="HuggingFace dataset repo id or local path."
-    )
-    parser.add_argument(
-        "--reward-model-path", type=str, default=None, help="Optional TOPReward LeRobot config."
-    )
-    parser.add_argument("--vlm-name", type=str, default=None, help="Override the VLM backbone (HF Hub id).")
-    parser.add_argument("--output-path", type=str, default=None, help="Output parquet path.")
-    parser.add_argument("--device", type=str, default="cuda", help="Device to use (default: cuda).")
-    parser.add_argument(
-        "--num-samples",
-        type=int,
-        default=None,
-        help="Anchor prefix samples per episode. None = dense. 15 matches upstream.",
-    )
-    parser.add_argument(
-        "--episodes",
-        type=int,
-        nargs="+",
-        default=None,
-        help="Process only these episode indices (e.g. --episodes 0 or --episodes 0 5 10).",
-    )
-    parser.add_argument("--fps", type=float, default=None, help="Override TOPRewardConfig.fps.")
-    parser.add_argument(
-        "--push-to-hub", action="store_true", help="Upload to the dataset repo on HuggingFace Hub."
-    )
-
-    args = parser.parse_args()
-
-    logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
-
-    output_path = compute_topreward_progress(
-        dataset_repo_id=args.dataset_repo_id,
-        reward_model_path=args.reward_model_path,
-        vlm_name=args.vlm_name,
-        output_path=args.output_path,
-        device=args.device,
-        num_samples=args.num_samples,
-        fps=args.fps,
-        episodes=args.episodes,
-    )
-
-    print(f"\nTOPReward progress saved to: {output_path}")
-
-    if args.push_to_hub:
-        from huggingface_hub import HfApi
-
-        api = HfApi()
-        hub_path = DEFAULT_OUTPUT_FILENAME
-
-        print(f"\nUploading to Hub: {args.dataset_repo_id}/{hub_path}")
-        api.upload_file(
-            path_or_fileobj=str(output_path),
-            path_in_repo=hub_path,
-            repo_id=args.dataset_repo_id,
-            repo_type="dataset",
-        )
-        print(
-            "Successfully uploaded to: "
-            f"https://huggingface.co/datasets/{args.dataset_repo_id}/blob/main/{hub_path}"
-        )
-
-        print("\nTo use in training, add to your config:")
-        print("  use_rabc: true")
-        print(f"  rabc_progress_path: hf://datasets/{args.dataset_repo_id}/{hub_path}")
-        print("  rabc_head_mode: sparse")
-    else:
-        print("\nTo use in training, add to your config:")
-        print("  use_rabc: true")
-        print(f"  rabc_progress_path: {output_path}")
-        print("  rabc_head_mode: sparse")
-
-
-if __name__ == "__main__":
-    main()

src/lerobot/rewards/topreward/configuration_topreward.py

@@ -1,146 +0,0 @@
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from __future__ import annotations
-
-from dataclasses import dataclass, field
-
-from lerobot.configs import FeatureType, NormalizationMode, PolicyFeature
-from lerobot.configs.rewards import RewardModelConfig
-from lerobot.utils.constants import OBS_IMAGES
-
-# Default prompt scaffolding from the upstream TOPReward paper / reference
-# implementation (``QwenClient.compute_instruction_reward``). The prompt
-# scores the terminal ``True`` token in ``f"{instruction} ... True"``
-# given the video.
-DEFAULT_PROMPT_PREFIX = (
-    "The above video shows a robot manipulation trajectory that completes the following task: "
-)
-DEFAULT_PROMPT_SUFFIX_TEMPLATE = (
-    "{instruction} Decide whether the above statement is True or not. The answer is: True"
-)
-
-
-@RewardModelConfig.register_subclass("topreward")
-@dataclass
-class TOPRewardConfig(RewardModelConfig):
-    """Configuration for the TOPReward zero-shot reward model.
-
-    TOPReward is **zero-shot**: it has no learnable parameters of its own.
-    The "model" is a generic vision-language model (default
-    ``Qwen/Qwen3-VL-8B-Instruct``) used with a fixed prompt to extract
-    token log-probabilities as a reward signal. There is therefore no
-    fine-tuned checkpoint to host: ``pretrained_path`` is unused at
-    runtime — the model identity is :attr:`vlm_name` (an HF Hub id).
-
-    Args:
-        vlm_name: Hugging Face Hub id of the underlying VLM. Must be a
-            Qwen3-VL family model (the only client implemented in this
-            LeRobot port).
-        torch_dtype: Torch dtype name passed to the VLM loader
-            (``"auto"``, ``"bfloat16"``, ``"float16"``, ...).
-        attn_implementation: ``transformers`` attention implementation
-            (e.g. ``"flash_attention_2"``, ``"sdpa"``). Defaults to
-            ``None`` so the upstream picks the best available.
-        image_key: Observation key that holds the trajectory frames.
-        task_key: Complementary-data key that holds the task instruction.
-        default_task: Fallback instruction when ``task_key`` is absent.
-        max_frames: Cap on the number of frames fed to the VLM per
-            sample. ``None`` = use all frames.
-        fps: Frames-per-second metadata for the Qwen video processor.
-        prompt_prefix: Text shown to the VLM right after the video and
-            before the suffix template.
-        prompt_suffix_template: Suffix appended after ``prompt_prefix``.
-            Must contain ``{instruction}``; the VLM scores the
-            log-likelihood of the tokens that follow the prefix.
-        add_chat_template: If ``True``, wrap the full prompt with the
-            tokenizer's chat template before tokenisation (matches
-            upstream ``add_chat_template=True``).
-        success_threshold: Optional log-prob threshold. If finite,
-            :meth:`TOPRewardModel.compute_reward` returns
-            ``(reward > success_threshold).float()`` instead of the raw
-            log-prob.
-        max_input_length: Hard limit on the total tokenized input length;
-            samples that exceed it raise a ``ValueError``.
-    """
-
-    # Path to a local LeRobot dir or HF repo that holds a ``config.json``
-    # snapshot of this TOPRewardConfig. The VLM weights themselves are
-    # always identified by ``vlm_name``.
-    pretrained_path: str | None = None
-
-    vlm_name: str = "Qwen/Qwen3-VL-8B-Instruct"
-    torch_dtype: str = "auto"
-    attn_implementation: str | None = None
-
-    image_key: str = OBS_IMAGES + ".top"
-    task_key: str = "task"
-    default_task: str | None = None
-    max_frames: int | None = 16
-    fps: float = 2.0
-
-    prompt_prefix: str = DEFAULT_PROMPT_PREFIX
-    prompt_suffix_template: str = DEFAULT_PROMPT_SUFFIX_TEMPLATE
-    add_chat_template: bool = False
-
-    success_threshold: float = float("-inf")
-    max_input_length: int = 32768
-
-    license: str | None = "mit"  # matches upstream TOPReward
-    tags: list[str] | None = field(
-        default_factory=lambda: ["reward-model", "vision-language", "qwen3-vl", "zero-shot"]
-    )
-
-    input_features: dict[str, PolicyFeature] = field(default_factory=dict)
-    output_features: dict[str, PolicyFeature] = field(default_factory=dict)
-    normalization_mapping: dict[str, NormalizationMode] = field(
-        default_factory=lambda: {
-            "VISUAL": NormalizationMode.IDENTITY,
-            "REWARD": NormalizationMode.IDENTITY,
-        }
-    )
-
-    def __post_init__(self) -> None:
-        super().__post_init__()
-        if self.max_frames is not None and self.max_frames < 1:
-            raise ValueError(f"max_frames must be >= 1, got {self.max_frames}")
-        if self.fps <= 0:
-            raise ValueError(f"fps must be > 0, got {self.fps}")
-        if "{instruction}" not in self.prompt_suffix_template:
-            raise ValueError(
-                "prompt_suffix_template must contain `{instruction}` so the model "
-                "scores the log-likelihood of the task suffix."
-            )
-        if self.max_input_length <= 0:
-            raise ValueError(f"max_input_length must be > 0, got {self.max_input_length}")
-
-        if self.image_key not in self.input_features:
-            self.input_features[self.image_key] = PolicyFeature(shape=(3, 224, 224), type=FeatureType.VISUAL)
-        self.output_features.setdefault("reward", PolicyFeature(shape=(1,), type=FeatureType.REWARD))
-
-    @property
-    def observation_delta_indices(self) -> list[int] | None:
-        return None
-
-    @property
-    def action_delta_indices(self) -> None:
-        return None
-
-    @property
-    def reward_delta_indices(self) -> None:
-        return None
-
-    def validate_features(self) -> None:
-        if self.image_key not in self.input_features:
-            raise ValueError(f"TOPReward requires image input feature {self.image_key!r}")

src/lerobot/rewards/topreward/modeling_topreward.py

@@ -1,238 +0,0 @@
-# Copyright 2026 Shirui Chen, Cole Harrison, Ying-Chun Lee, Angela Jin Yang,
-# Zhongzheng Ren, Lillian J. Ratliff, Jiafei Duan, Dieter Fox, Ranjay Krishna
-# and The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics.
-
-Paper:         https://arxiv.org/abs/2602.19313
-Project:       https://topreward.github.io/webpage/
-Original code: https://github.com/TOPReward/TOPReward
-Backbone:      https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct  (default)
-
-TOPReward is a **zero-shot** reward model: it has no fine-tuned weights of
-its own. Given a video trajectory and a task instruction, it asks an
-off-the-shelf VLM how likely the instruction is, conditioned on the video,
-and returns that log-likelihood as the reward signal.
-
-Inference recipe:
-
-1. The processor builds a chat-style prompt, tokenises it, and emits
-   ``input_ids``, ``attention_mask``, vision tensors, and ``labels``.
-   The processor label-masks everything except the terminal answer token with
-   ``-100``.
-2. Forward the full token sequence through the VLM.
-3. Read the terminal answer token log-probability from the logits as the
-   scalar reward.
-
-With the default ``prompt_suffix_template``, the only unmasked token is the
-literal ``"True"`` at the end — the reward is
-``log P("True" | video + prompt + instruction)``.
-
-This LeRobot port is **inference-only and not trainable** — :meth:`forward`
-is intentionally inherited from :class:`PreTrainedRewardModel` and raises
-``NotImplementedError``, making :attr:`PreTrainedRewardModel.is_trainable`
-return ``False``.
-
-Because the VLM weights live on the Hugging Face Hub under their canonical
-id (``Qwen/Qwen3-VL-8B-Instruct`` etc.) and TOPReward never modifies them,
-:meth:`_save_pretrained` and :meth:`from_pretrained` are overridden so a
-TOPReward LeRobot "checkpoint" is a single ``config.json`` (the VLM is
-re-fetched from the Hub at load time).
-"""
-
-from __future__ import annotations
-
-import builtins
-import logging
-import os
-from pathlib import Path
-from tempfile import TemporaryDirectory
-from typing import TYPE_CHECKING, Any, TypeVar
-
-import numpy as np
-import torch
-from huggingface_hub import HfApi, hf_hub_download
-from huggingface_hub.constants import CONFIG_NAME
-from huggingface_hub.errors import HfHubHTTPError
-from torch import Tensor
-from torch.nn.functional import cross_entropy
-
-from lerobot.configs.rewards import RewardModelConfig
-from lerobot.rewards.pretrained import PreTrainedRewardModel
-from lerobot.rewards.topreward.configuration_topreward import TOPRewardConfig
-from lerobot.rewards.topreward.processor_topreward import TOPREWARD_FEATURE_PREFIX, TOPREWARD_INPUT_KEYS
-from lerobot.utils.import_utils import _transformers_available, require_package
-
-if TYPE_CHECKING:
-    from lerobot.configs.train import TrainPipelineConfig
-
-if TYPE_CHECKING or _transformers_available:
-    from transformers import Qwen3VLForConditionalGeneration
-else:
-    Qwen3VLForConditionalGeneration = None  # type: ignore[assignment]
-
-logger = logging.getLogger(__name__)
-
-T = TypeVar("T", bound="TOPRewardModel")
-
-
-def _torch_dtype(name: str) -> torch.dtype | str:
-    """Resolve a torch dtype name; ``"auto"`` is passed through verbatim."""
-    if name == "auto":
-        return "auto"
-    dtype = getattr(torch, name, None)
-    if isinstance(dtype, torch.dtype):
-        return dtype
-    raise ValueError(f"Unknown torch dtype: {name!r}")
-
-
-class TOPRewardModel(PreTrainedRewardModel):
-    """TOPReward zero-shot reward model."""
-
-    name = "topreward"
-    config_class = TOPRewardConfig
-
-    def __init__(self, config: TOPRewardConfig) -> None:
-        require_package("transformers", extra="topreward")
-        super().__init__(config)
-        self.config = config
-
-        torch_dtype = _torch_dtype(config.torch_dtype)
-        model_kwargs: dict[str, Any] = {"dtype": torch_dtype, "trust_remote_code": True}
-        if config.attn_implementation is not None:
-            model_kwargs["attn_implementation"] = config.attn_implementation
-
-        self.model = Qwen3VLForConditionalGeneration.from_pretrained(config.vlm_name, **model_kwargs)
-
-    def compute_reward(self, batch: dict[str, Any]) -> Tensor:
-        """Return one log-prob reward per sample in the batch."""
-        inputs: dict[str, Any] = {}
-        for key in TOPREWARD_INPUT_KEYS:
-            batch_key = f"{TOPREWARD_FEATURE_PREFIX}{key}"
-            if batch_key not in batch:
-                raise KeyError(
-                    f"TOPReward batch missing `{batch_key}`. Make sure the "
-                    "TOPRewardEncoderProcessorStep ran before `compute_reward`."
-                )
-            inputs[key] = batch[batch_key]
-
-        device = next(self.model.parameters()).device
-        inputs = {key: value.to(device) if hasattr(value, "to") else value for key, value in inputs.items()}
-        labels = inputs.pop("labels")
-        inputs["logits_to_keep"] = 2
-
-        self.eval()
-        with torch.no_grad():
-            outputs = self.model(**inputs)
-        logits = outputs.logits
-        rewards = -cross_entropy(logits[:, -2, :].float(), labels[:, -1], reduction="none")
-        if np.isfinite(self.config.success_threshold):
-            rewards = (rewards > self.config.success_threshold).float()
-        return rewards.to(self.config.device or "cpu")
-
-    def _save_pretrained(self, save_directory: Path) -> None:
-        """Save ``config.json`` only."""
-        self.config._save_pretrained(save_directory)
-
-    @classmethod
-    def from_pretrained(
-        cls: builtins.type[T],
-        pretrained_name_or_path: str | Path,
-        *,
-        config: RewardModelConfig | None = None,
-        force_download: bool = False,
-        resume_download: bool | None = None,
-        proxies: dict | None = None,
-        token: str | bool | None = None,
-        cache_dir: str | Path | None = None,
-        local_files_only: bool = False,
-        revision: str | None = None,
-        strict: bool = False,  # noqa: ARG003 — accepted for API parity; unused (no safetensors to load)
-        **kwargs: Any,
-    ) -> T:
-        """Load a TOPReward configuration and instantiate the wrapped VLM."""
-        if config is None:
-            config = RewardModelConfig.from_pretrained(
-                pretrained_name_or_path=pretrained_name_or_path,
-                force_download=force_download,
-                resume_download=resume_download,
-                proxies=proxies,
-                token=token,
-                cache_dir=cache_dir,
-                local_files_only=local_files_only,
-                revision=revision,
-                **kwargs,
-            )
-        if not isinstance(config, TOPRewardConfig):
-            raise TypeError(
-                f"Expected a TOPRewardConfig, got {type(config).__name__}. Make sure "
-                f"`pretrained_name_or_path={pretrained_name_or_path!r}` points at a "
-                "TOPReward checkpoint."
-            )
-
-        model_id = str(pretrained_name_or_path)
-        if not os.path.isdir(model_id):
-            try:
-                hf_hub_download(
-                    repo_id=model_id,
-                    filename=CONFIG_NAME,
-                    revision=revision,
-                    cache_dir=cache_dir,
-                    force_download=force_download,
-                    proxies=proxies,
-                    resume_download=resume_download,
-                    token=token,
-                    local_files_only=local_files_only,
-                )
-            except HfHubHTTPError as e:
-                raise FileNotFoundError(
-                    f"{CONFIG_NAME} not found on the HuggingFace Hub in {model_id}"
-                ) from e
-
-        instance = cls(config, **kwargs)
-        instance.to(config.device)
-        instance.eval()
-        return instance
-
-    def push_model_to_hub(self, cfg: TrainPipelineConfig):
-        """Push the TOPReward ``config.json`` + model card to the Hub."""
-        api = HfApi()
-        repo_id = api.create_repo(
-            repo_id=self.config.repo_id, private=self.config.private, exist_ok=True
-        ).repo_id
-
-        with TemporaryDirectory(ignore_cleanup_errors=True) as tmp:
-            saved_path = Path(tmp) / repo_id
-            saved_path.mkdir(parents=True, exist_ok=True)
-
-            self.config._save_pretrained(saved_path)
-
-            card = self.generate_model_card(
-                cfg.dataset.repo_id, self.config.type, self.config.license, self.config.tags
-            )
-            card.save(str(saved_path / "README.md"))
-
-            cfg.save_pretrained(saved_path)
-
-            commit_info = api.upload_folder(
-                repo_id=repo_id,
-                repo_type="model",
-                folder_path=saved_path,
-                commit_message="Upload TOPReward config and readme",
-                allow_patterns=["*.json", "*.yaml", "*.md"],
-                ignore_patterns=["*.tmp", "*.log", "*.safetensors"],
-            )
-
-            logger.info(f"Model pushed to {commit_info.repo_url.url}")

src/lerobot/rewards/topreward/processor_topreward.py

@@ -1,305 +0,0 @@
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""TOPReward pre/post processing pipeline."""
-
-from __future__ import annotations
-
-from dataclasses import dataclass, field
-from typing import TYPE_CHECKING, Any
-
-import torch
-from torch import Tensor
-
-from lerobot.configs import PipelineFeatureType, PolicyFeature
-from lerobot.processor import (
-    AddBatchDimensionProcessorStep,
-    DeviceProcessorStep,
-    PolicyAction,
-    PolicyProcessorPipeline,
-    ProcessorStep,
-    ProcessorStepRegistry,
-    policy_action_to_transition,
-)
-from lerobot.rewards.topreward.configuration_topreward import (
-    DEFAULT_PROMPT_PREFIX,
-    DEFAULT_PROMPT_SUFFIX_TEMPLATE,
-    TOPRewardConfig,
-)
-from lerobot.types import EnvTransition, TransitionKey
-from lerobot.utils.constants import (
-    OBS_IMAGES,
-    OBS_PREFIX,
-    POLICY_POSTPROCESSOR_DEFAULT_NAME,
-    POLICY_PREPROCESSOR_DEFAULT_NAME,
-)
-from lerobot.utils.import_utils import _transformers_available, require_package
-
-if TYPE_CHECKING or _transformers_available:
-    from transformers import AutoProcessor
-else:
-    AutoProcessor = None
-
-TOPREWARD_FEATURE_PREFIX = f"{OBS_PREFIX}topreward."
-
-_TRUE_ANSWER = "True"
-
-TOPREWARD_VLM_INPUT_KEYS = (
-    "input_ids",
-    "attention_mask",
-    "pixel_values_videos",
-    "video_grid_thw",
-    "mm_token_type_ids",
-)
-TOPREWARD_INPUT_KEYS = TOPREWARD_VLM_INPUT_KEYS + ("labels",)
-
-
-def _prepare_video_batch(video: Tensor, *, max_frames: int | None) -> Tensor:
-    """Return videos as ``(B, T, C, H, W)`` uint8 tensors for Qwen3-VL."""
-    if video.ndim == 4:
-        video = video.unsqueeze(1)
-    elif video.ndim != 5:
-        raise ValueError(
-            f"Expected TOPReward frames with shape (B,C,H,W) or (B,T,C,H,W); got {tuple(video.shape)}"
-        )
-
-    if max_frames is not None:
-        video = video[:, -max_frames:]
-    if video.shape[-1] in (1, 3):
-        video = video.permute(0, 1, 4, 2, 3)
-    elif video.shape[2] not in (1, 3):
-        raise ValueError(f"Expected channel dim of size 1 or 3, got shape {tuple(video.shape)}")
-
-    if video.is_floating_point():
-        video = video * 255.0
-
-    return video.clamp(0, 255).to(torch.uint8).contiguous()
-
-
-def _expand_tasks(task: Any, *, batch_size: int, default: str | None) -> list[str]:
-    if task is None:
-        task = default
-    if task is None:
-        raise KeyError("TOPReward expected a task description in complementary data")
-    if isinstance(task, str):
-        return [task] * batch_size
-    if isinstance(task, tuple):
-        task = list(task)
-    if not (isinstance(task, list) and all(isinstance(item, str) for item in task)):
-        raise TypeError(f"TOPReward task must be a string or list of strings, got {type(task)}")
-    if len(task) == 1 and batch_size > 1:
-        return task * batch_size
-    if len(task) != batch_size:
-        raise ValueError(f"Expected {batch_size} tasks, got {len(task)}")
-    return task
-
-
-@dataclass
-@ProcessorStepRegistry.register(name="topreward_encoder")
-class TOPRewardEncoderProcessorStep(ProcessorStep):
-    """Encode raw frames + task into Qwen-VL tensors for the TOPReward model.
-
-    Loads a :class:`~transformers.AutoProcessor` matching ``vlm_name`` and
-    builds the full chat prompt including the instruction suffix. The
-    resulting ``input_ids``, ``attention_mask``, vision tensors, and
-    ``labels`` are written under the ``observation.topreward.*`` namespace
-    so the model can score without re-tokenising.
-
-    At call time the step reads:
-
-    - ``observation[image_key]``: ``(B, T, C, H, W)`` or ``(B, C, H, W)`` frames.
-    - ``complementary_data[task_key]``: a string or list of strings.
-
-    and writes ``observation[f"{TOPREWARD_FEATURE_PREFIX}<name>"]`` for the
-    Qwen-VL tensors plus ``labels``.
-    """
-
-    vlm_name: str = "Qwen/Qwen3-VL-8B-Instruct"
-    image_key: str = OBS_IMAGES + ".top"
-    task_key: str = "task"
-    default_task: str | None = None
-    max_frames: int | None = 16
-    fps: float = 2.0
-    prompt_prefix: str = DEFAULT_PROMPT_PREFIX
-    prompt_suffix_template: str = DEFAULT_PROMPT_SUFFIX_TEMPLATE
-    add_chat_template: bool = False
-    max_length: int = 32768
-
-    _processor: Any = field(default=None, init=False, repr=False)
-
-    def __post_init__(self) -> None:
-        require_package("transformers", extra="topreward")
-        self._processor = AutoProcessor.from_pretrained(self.vlm_name, trust_remote_code=True)
-
-    def __call__(self, transition: EnvTransition) -> EnvTransition:
-        observation = transition.get(TransitionKey.OBSERVATION)
-        complementary = transition.get(TransitionKey.COMPLEMENTARY_DATA) or {}
-        if self.image_key not in observation:
-            raise KeyError(f"TOPReward expected image key {self.image_key!r} in observation")
-
-        frames = observation[self.image_key]
-        videos = frames.detach().cpu() if isinstance(frames, Tensor) else torch.as_tensor(frames)
-        videos = _prepare_video_batch(videos, max_frames=self.max_frames)
-
-        batch_size = videos.shape[0]
-        tasks = _expand_tasks(
-            complementary.get(self.task_key, self.default_task),
-            batch_size=batch_size,
-            default=self.default_task,
-        )
-
-        encoded = self._encode_batch(videos, tasks, batch_size)
-
-        new_observation = dict(observation)
-        for key, value in encoded.items():
-            new_observation[f"{TOPREWARD_FEATURE_PREFIX}{key}"] = value
-
-        new_transition = transition.copy()
-        new_transition[TransitionKey.OBSERVATION] = new_observation
-        return new_transition
-
-    def _encode_batch(self, videos: Tensor, tasks: list[str], batch_size) -> dict[str, Any]:
-        """Tokenise a batch of (frames, task) pairs into Qwen-VL tensors.
-
-        The loop only builds per-sample chat strings. Tokenisation, padding,
-        video preprocessing, and label construction are batched.
-        """
-
-        texts: list[str] = []
-        video_metadata = [
-            {
-                "total_num_frames": int(videos.shape[1]),
-                "fps": float(self.fps),
-                "frames_indices": list(range(int(videos.shape[1]))),
-            }
-            for _ in range(batch_size)
-        ]
-        eos_token = self._processor.tokenizer.eos_token
-
-        for i in range(batch_size):
-            instruction_suffix = self.prompt_suffix_template.format(instruction=tasks[i])
-            if self.add_chat_template:
-                suffix_for_template = instruction_suffix.removesuffix(_TRUE_ANSWER).rstrip()
-                templated_messages = [
-                    {
-                        "role": "user",
-                        "content": [
-                            {"type": "video", "video": videos[i], "fps": self.fps},
-                            {"type": "text", "text": f"{self.prompt_prefix}{suffix_for_template}"},
-                        ],
-                    }
-                ]
-                prompt_chat = self._processor.apply_chat_template(
-                    templated_messages, tokenize=False, add_generation_prompt=True
-                )
-                full_text = f"{prompt_chat}{_TRUE_ANSWER}"
-            else:
-                user_messages = [
-                    {
-                        "role": "user",
-                        "content": [
-                            {"type": "video", "video": videos[i], "fps": self.fps},
-                            {"type": "text", "text": self.prompt_prefix},
-                        ],
-                    }
-                ]
-                prompt_chat = self._processor.apply_chat_template(
-                    user_messages, tokenize=False, add_generation_prompt=False
-                )
-                if eos_token is not None:
-                    prompt_chat = prompt_chat.split(eos_token)[0]
-                full_text = f"{prompt_chat}{instruction_suffix}"
-
-            texts.append(full_text)
-
-        result = self._processor(
-            text=texts,
-            videos=videos,
-            video_metadata=video_metadata,
-            do_sample_frames=False,
-            padding=True,
-            padding_side="left",
-            return_tensors="pt",
-        )
-        input_ids = result["input_ids"]
-
-        if input_ids.shape[-1] > self.max_length:
-            raise ValueError(
-                f"TOPReward input length {input_ids.shape[-1]} exceeds max_length "
-                f"{self.max_length}; lower `max_frames` or raise `max_length`."
-            )
-
-        labels = torch.full_like(input_ids, -100)
-        labels[:, -1] = input_ids[:, -1]
-        result["labels"] = labels
-        return result
-
-    def transform_features(
-        self, features: dict[PipelineFeatureType, dict[str, PolicyFeature]]
-    ) -> dict[PipelineFeatureType, dict[str, PolicyFeature]]:
-        return features
-
-    def get_config(self) -> dict[str, Any]:
-        return {
-            "vlm_name": self.vlm_name,
-            "image_key": self.image_key,
-            "task_key": self.task_key,
-            "default_task": self.default_task,
-            "max_frames": self.max_frames,
-            "fps": self.fps,
-            "prompt_prefix": self.prompt_prefix,
-            "prompt_suffix_template": self.prompt_suffix_template,
-            "add_chat_template": self.add_chat_template,
-            "max_length": self.max_length,
-        }
-
-
-def make_topreward_pre_post_processors(
-    config: TOPRewardConfig,
-    dataset_stats: dict[str, dict[str, Any]] | None = None,
-) -> tuple[
-    PolicyProcessorPipeline[dict[str, Any], dict[str, Any]],
-    PolicyProcessorPipeline[PolicyAction, PolicyAction],
-]:
-    """Pipeline that pre-encodes frames + task into Qwen-VL tensors.
-
-    The preprocessor adds a batch dimension if needed, runs TOPReward's
-    encoder (which tokenises the full prompt and emits ``labels``), and
-    moves everything to the configured device. The postprocessor is
-    the identity since TOPReward outputs a single reward tensor.
-    """
-    preprocessor = PolicyProcessorPipeline[dict[str, Any], dict[str, Any]](
-        steps=[
-            AddBatchDimensionProcessorStep(),
-            TOPRewardEncoderProcessorStep(
-                vlm_name=config.vlm_name,
-                image_key=config.image_key,
-                task_key=config.task_key,
-                default_task=config.default_task,
-                max_frames=config.max_frames,
-                fps=config.fps,
-                prompt_prefix=config.prompt_prefix,
-                prompt_suffix_template=config.prompt_suffix_template,
-                add_chat_template=config.add_chat_template,
-                max_length=config.max_input_length,
-            ),
-            DeviceProcessorStep(device=config.device or "cpu"),
-        ],
-        name=POLICY_PREPROCESSOR_DEFAULT_NAME,
-    )
-    postprocessor = PolicyProcessorPipeline(
-        name=POLICY_POSTPROCESSOR_DEFAULT_NAME,
-        to_transition=policy_action_to_transition,
-    )
-    return preprocessor, postprocessor

src/lerobot/robots/bi_rebot_b601_follower/__init__.py

@@ -1,20 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from .bi_rebot_b601_follower import BiRebotB601Follower
-from .config_bi_rebot_b601_follower import BiRebotB601FollowerConfig
-
-__all__ = ["BiRebotB601Follower", "BiRebotB601FollowerConfig"]

src/lerobot/robots/bi_rebot_b601_follower/bi_rebot_b601_follower.py

@@ -1,150 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import logging
-from functools import cached_property
-
-from lerobot.types import RobotAction, RobotObservation
-from lerobot.utils.decorators import check_if_already_connected, check_if_not_connected
-
-from ..rebot_b601_follower import RebotB601Follower, RebotB601FollowerRobotConfig
-from ..robot import Robot
-from .config_bi_rebot_b601_follower import BiRebotB601FollowerConfig
-
-logger = logging.getLogger(__name__)
-
-
-class BiRebotB601Follower(Robot):
-    """Bimanual Seeed Studio reBot B601-DM follower.
-
-    Composes two single-arm :class:`RebotB601Follower` instances. Observation and
-    action keys of each arm are namespaced with a ``left_`` / ``right_`` prefix.
-    """
-
-    config_class = BiRebotB601FollowerConfig
-    name = "bi_rebot_b601_follower"
-
-    def __init__(self, config: BiRebotB601FollowerConfig):
-        super().__init__(config)
-        self.config = config
-
-        left_arm_config = RebotB601FollowerRobotConfig(
-            id=f"{config.id}_left" if config.id else None,
-            calibration_dir=config.calibration_dir,
-            port=config.left_arm_config.port,
-            can_adapter=config.left_arm_config.can_adapter,
-            dm_serial_baud=config.left_arm_config.dm_serial_baud,
-            disable_torque_on_disconnect=config.left_arm_config.disable_torque_on_disconnect,
-            max_relative_target=config.left_arm_config.max_relative_target,
-            cameras=config.left_arm_config.cameras,
-            motor_can_ids=config.left_arm_config.motor_can_ids,
-            pos_vel_velocity=config.left_arm_config.pos_vel_velocity,
-            gripper_torque_ratio=config.left_arm_config.gripper_torque_ratio,
-            joint_limits=config.left_arm_config.joint_limits,
-        )
-
-        right_arm_config = RebotB601FollowerRobotConfig(
-            id=f"{config.id}_right" if config.id else None,
-            calibration_dir=config.calibration_dir,
-            port=config.right_arm_config.port,
-            can_adapter=config.right_arm_config.can_adapter,
-            dm_serial_baud=config.right_arm_config.dm_serial_baud,
-            disable_torque_on_disconnect=config.right_arm_config.disable_torque_on_disconnect,
-            max_relative_target=config.right_arm_config.max_relative_target,
-            cameras=config.right_arm_config.cameras,
-            motor_can_ids=config.right_arm_config.motor_can_ids,
-            pos_vel_velocity=config.right_arm_config.pos_vel_velocity,
-            gripper_torque_ratio=config.right_arm_config.gripper_torque_ratio,
-            joint_limits=config.right_arm_config.joint_limits,
-        )
-
-        self.left_arm = RebotB601Follower(left_arm_config)
-        self.right_arm = RebotB601Follower(right_arm_config)
-
-        # Only for compatibility with parts of the codebase that expect `robot.cameras`.
-        self.cameras = {**self.left_arm.cameras, **self.right_arm.cameras}
-
-    @property
-    def _motors_ft(self) -> dict[str, type]:
-        return {
-            **{f"left_{k}": v for k, v in self.left_arm._motors_ft.items()},
-            **{f"right_{k}": v for k, v in self.right_arm._motors_ft.items()},
-        }
-
-    @property
-    def _cameras_ft(self) -> dict[str, tuple]:
-        return {
-            **{f"left_{k}": v for k, v in self.left_arm._cameras_ft.items()},
-            **{f"right_{k}": v for k, v in self.right_arm._cameras_ft.items()},
-        }
-
-    @cached_property
-    def observation_features(self) -> dict[str, type | tuple]:
-        return {**self._motors_ft, **self._cameras_ft}
-
-    @cached_property
-    def action_features(self) -> dict[str, type]:
-        return self._motors_ft
-
-    @property
-    def is_connected(self) -> bool:
-        return self.left_arm.is_connected and self.right_arm.is_connected
-
-    @check_if_already_connected
-    def connect(self, calibrate: bool = True) -> None:
-        self.left_arm.connect(calibrate)
-        self.right_arm.connect(calibrate)
-
-    @property
-    def is_calibrated(self) -> bool:
-        return self.left_arm.is_calibrated and self.right_arm.is_calibrated
-
-    def calibrate(self) -> None:
-        self.left_arm.calibrate()
-        self.right_arm.calibrate()
-
-    def configure(self) -> None:
-        self.left_arm.configure()
-        self.right_arm.configure()
-
-    @check_if_not_connected
-    def get_observation(self) -> RobotObservation:
-        obs_dict = {}
-        obs_dict.update({f"left_{k}": v for k, v in self.left_arm.get_observation().items()})
-        obs_dict.update({f"right_{k}": v for k, v in self.right_arm.get_observation().items()})
-        return obs_dict
-
-    @check_if_not_connected
-    def send_action(self, action: RobotAction) -> RobotAction:
-        left_action = {
-            key.removeprefix("left_"): value for key, value in action.items() if key.startswith("left_")
-        }
-        right_action = {
-            key.removeprefix("right_"): value for key, value in action.items() if key.startswith("right_")
-        }
-
-        sent_action_left = self.left_arm.send_action(left_action)
-        sent_action_right = self.right_arm.send_action(right_action)
-
-        return {
-            **{f"left_{k}": v for k, v in sent_action_left.items()},
-            **{f"right_{k}": v for k, v in sent_action_right.items()},
-        }
-
-    @check_if_not_connected
-    def disconnect(self) -> None:
-        self.left_arm.disconnect()
-        self.right_arm.disconnect()

src/lerobot/robots/bi_rebot_b601_follower/config_bi_rebot_b601_follower.py

@@ -1,29 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from dataclasses import dataclass
-
-from ..config import RobotConfig
-from ..rebot_b601_follower import RebotB601FollowerConfig
-
-
-@RobotConfig.register_subclass("bi_rebot_b601_follower")
-@dataclass
-class BiRebotB601FollowerConfig(RobotConfig):
-    """Configuration class for the bimanual reBot B601-DM follower robot."""
-
-    left_arm_config: RebotB601FollowerConfig
-    right_arm_config: RebotB601FollowerConfig

src/lerobot/robots/rebot_b601_follower/__init__.py

@@ -1,20 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from .config_rebot_b601_follower import RebotB601FollowerConfig, RebotB601FollowerRobotConfig
-from .rebot_b601_follower import RebotB601Follower
-
-__all__ = ["RebotB601Follower", "RebotB601FollowerConfig", "RebotB601FollowerRobotConfig"]

src/lerobot/robots/rebot_b601_follower/config_rebot_b601_follower.py

@@ -1,94 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from dataclasses import dataclass, field
-
-from lerobot.cameras import CameraConfig
-
-from ..config import RobotConfig
-
-
-@dataclass
-class RebotB601FollowerConfig:
-    """Base configuration class for the Seeed Studio reBot B601-DM follower arm.
-
-    The B601-DM is a 6-DOF arm plus gripper driven by Damiao CAN motors. Motor
-    communication goes through the ``motorbridge`` package.
-    """
-
-    # Communication port. For ``can_adapter="damiao"`` this is the Damiao serial
-    # bridge device (e.g. "/dev/ttyACM0"); for ``can_adapter="socketcan"`` it is
-    # the CAN channel name (e.g. "can0").
-    port: str
-
-    # CAN adapter type:
-    #   "damiao"    - Damiao dedicated serial bridge (default)
-    #   "socketcan" - SocketCAN based adapters (PCAN, slcan, embedded controllers, ...)
-    can_adapter: str = "damiao"
-
-    # Baud rate for the Damiao serial bridge (only used when can_adapter="damiao").
-    dm_serial_baud: int = 921600
-
-    disable_torque_on_disconnect: bool = True
-
-    # `max_relative_target` limits the magnitude of the relative positional target
-    # vector for safety purposes (in degrees). Set to a positive scalar to apply the
-    # same value to all motors, or to a dict mapping motor names to per-motor values.
-    max_relative_target: float | dict[str, float] | None = None
-
-    # cameras
-    cameras: dict[str, CameraConfig] = field(default_factory=dict)
-
-    # Maps motor names to their (send_can_id, recv_can_id) pair.
-    motor_can_ids: dict[str, tuple[int, int]] = field(
-        default_factory=lambda: {
-            "shoulder_pan": (0x01, 0x11),
-            "shoulder_lift": (0x02, 0x12),
-            "elbow_flex": (0x03, 0x13),
-            "wrist_flex": (0x04, 0x14),
-            "wrist_yaw": (0x05, 0x15),
-            "wrist_roll": (0x06, 0x16),
-            "gripper": (0x07, 0x17),
-        }
-    )
-
-    # Target velocity for joints running in POS_VEL mode, in degrees/s. A scalar is
-    # applied to every joint; a list provides one value per joint (in motor order).
-    pos_vel_velocity: float | list[float] = field(default_factory=lambda: [150.0] * 7)
-
-    # Torque/current ratio for the gripper's FORCE_POS mode, in range [0, 1].
-    gripper_torque_ratio: float = 0.1
-
-    # Soft joint limits (degrees). These are clipped against on every action.
-    joint_limits: dict[str, tuple[float, float]] = field(
-        default_factory=lambda: {
-            "shoulder_pan": (-145.0, 145.0),
-            "shoulder_lift": (-170.0, 1.0),
-            "elbow_flex": (-200.0, 1.0),
-            "wrist_flex": (-80.0, 90.0),
-            "wrist_yaw": (-90.0, 90.0),
-            "wrist_roll": (-90.0, 90.0),
-            "gripper": (-270.0, 0.0),
-        }
-    )
-
-
-@RobotConfig.register_subclass("rebot_b601_follower")
-@dataclass
-class RebotB601FollowerRobotConfig(RobotConfig, RebotB601FollowerConfig):
-    """Registered configuration for the reBot B601-DM follower robot."""
-
-    pass

src/lerobot/robots/rebot_b601_follower/rebot_b601_follower.py

@@ -1,289 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import logging
-import math
-import time
-from functools import cached_property
-from typing import TYPE_CHECKING
-
-from lerobot.cameras import make_cameras_from_configs
-from lerobot.motors import MotorCalibration
-from lerobot.types import RobotAction, RobotObservation
-from lerobot.utils.decorators import check_if_already_connected, check_if_not_connected
-from lerobot.utils.import_utils import _motorbridge_available, require_package
-
-from ..robot import Robot
-from ..utils import ensure_safe_goal_position
-from .config_rebot_b601_follower import RebotB601FollowerRobotConfig
-
-if TYPE_CHECKING or _motorbridge_available:
-    from motorbridge import Controller as MotorBridgeController, Mode as MotorBridgeMode
-else:
-    MotorBridgeController = None
-    MotorBridgeMode = None
-
-logger = logging.getLogger(__name__)
-
-# Joint controlled in FORCE_POS mode; every other joint runs in POS_VEL mode.
-GRIPPER_MOTOR = "gripper"
-# Per-joint Damiao motor models for the B601-DM (passed to motorbridge).
-MOTOR_MODELS = {
-    "shoulder_pan": "4340P",
-    "shoulder_lift": "4340P",
-    "elbow_flex": "4340P",
-    "wrist_flex": "4310",
-    "wrist_yaw": "4310",
-    "wrist_roll": "4310",
-    "gripper": "4310",
-}
-_ENSURE_MODE_RETRIES = 9
-_SETTLE_SEC = 0.01
-_ZERO_SETTLE_SEC = 0.1
-
-
-class RebotB601Follower(Robot):
-    """Seeed Studio reBot B601-DM follower arm (6-DOF + gripper, Damiao CAN motors).
-
-    Motor communication is handled by the ``motorbridge`` package over a CAN bus,
-    reached either through a Damiao serial bridge or a SocketCAN adapter.
-    """
-
-    config_class = RebotB601FollowerRobotConfig
-    name = "rebot_b601_follower"
-
-    def __init__(self, config: RebotB601FollowerRobotConfig):
-        require_package("motorbridge", extra="rebot")
-        super().__init__(config)
-        self.config = config
-        self.bus: MotorBridgeController | None = None
-        self.motors: dict = {}
-        self.motor_names = list(config.motor_can_ids.keys())
-        self.cameras = make_cameras_from_configs(config.cameras)
-
-    @property
-    def _motors_ft(self) -> dict[str, type]:
-        return {f"{motor}.pos": float for motor in self.motor_names}
-
-    @property
-    def _cameras_ft(self) -> dict[str, tuple]:
-        return {
-            cam: (self.config.cameras[cam].height, self.config.cameras[cam].width, 3) for cam in self.cameras
-        }
-
-    @cached_property
-    def observation_features(self) -> dict[str, type | tuple]:
-        return {**self._motors_ft, **self._cameras_ft}
-
-    @cached_property
-    def action_features(self) -> dict[str, type]:
-        return self._motors_ft
-
-    @property
-    def is_connected(self) -> bool:
-        return self.bus is not None and all(cam.is_connected for cam in self.cameras.values())
-
-    @check_if_already_connected
-    def connect(self, calibrate: bool = True) -> None:
-        logger.info(f"Connecting {self} on {self.config.port} (adapter={self.config.can_adapter})...")
-        if self.config.can_adapter == "damiao":
-            self.bus = MotorBridgeController.from_dm_serial(
-                serial_port=self.config.port,
-                baud=self.config.dm_serial_baud,
-            )
-        elif self.config.can_adapter == "socketcan":
-            self.bus = MotorBridgeController(channel=self.config.port)
-        else:
-            raise ValueError(
-                f"Unsupported can_adapter '{self.config.can_adapter}'. Use 'damiao' or 'socketcan'."
-            )
-
-        for motor_name, (send_id, recv_id) in self.config.motor_can_ids.items():
-            self.motors[motor_name] = self.bus.add_damiao_motor(send_id, recv_id, MOTOR_MODELS[motor_name])
-
-        if not self.is_calibrated and calibrate:
-            logger.info(
-                "Mismatch between calibration values in the motor and the calibration file or no calibration file found"
-            )
-            self.calibrate()
-
-        for cam in self.cameras.values():
-            cam.connect()
-
-        self.configure()
-        logger.info(f"{self} connected.")
-
-    @property
-    def is_calibrated(self) -> bool:
-        return bool(self.calibration)
-
-    def calibrate(self) -> None:
-        if self.calibration:
-            user_input = input(
-                f"Press ENTER to use provided calibration file associated with the id {self.id}, "
-                "or type 'c' and press ENTER to run calibration: "
-            )
-            if user_input.strip().lower() != "c":
-                logger.info(f"Using calibration file associated with the id {self.id}")
-                return
-
-        logger.info(f"\nRunning calibration of {self}")
-        self.bus.disable_all()
-        print(
-            "\nCalibration: set zero position.\n"
-            "Manually move the reBot B601 to its ZERO POSITION and close the gripper.\n"
-            "See the B601 manual for the zero pose (the default sit-down position).\n"
-        )
-        input("Press ENTER when ready...")
-
-        for motor in self.motors.values():
-            motor.set_zero_position()
-            time.sleep(_ZERO_SETTLE_SEC)
-        logger.info("Arm zero position set.")
-
-        self.calibration = {}
-        for motor_name, (send_id, _recv_id) in self.config.motor_can_ids.items():
-            range_min, range_max = self.config.joint_limits[motor_name]
-            self.calibration[motor_name] = MotorCalibration(
-                id=send_id,
-                drive_mode=0,
-                homing_offset=0,
-                range_min=int(range_min),
-                range_max=int(range_max),
-            )
-
-        self._save_calibration()
-        print(f"Calibration saved to {self.calibration_fpath}")
-
-    def configure(self) -> None:
-        self.bus.enable_all()
-        for motor_name, motor in self.motors.items():
-            target_mode = (
-                MotorBridgeMode.FORCE_POS if motor_name == GRIPPER_MOTOR else MotorBridgeMode.POS_VEL
-            )
-            for attempt in range(_ENSURE_MODE_RETRIES + 1):
-                try:
-                    motor.ensure_mode(target_mode)
-                    break
-                except Exception:
-                    if attempt == _ENSURE_MODE_RETRIES:
-                        raise
-                    time.sleep(_SETTLE_SEC)
-            logger.debug(f"{motor_name} mode set to {target_mode}")
-
-    @check_if_not_connected
-    def disable_torque(self) -> None:
-        """Disable motor torque so the arm can be moved by hand (read-only debugging)."""
-        self.bus.disable_all()
-        logger.info(f"{self} torque disabled.")
-
-    def _present_pos(self) -> dict[str, float]:
-        """Read present joint positions in degrees."""
-        for motor in self.motors.values():
-            motor.request_feedback()
-        try:
-            self.bus.poll_feedback_once()
-        except Exception:
-            logger.warning("CAN bus poll feedback failed.")
-
-        present_pos = {}
-        for motor_name, motor in self.motors.items():
-            state = motor.get_state()
-            present_pos[motor_name] = math.degrees(state.pos) if state is not None else 0.0
-        return present_pos
-
-    @check_if_not_connected
-    def get_observation(self) -> RobotObservation:
-        start = time.perf_counter()
-        obs_dict = {f"{motor}.pos": pos for motor, pos in self._present_pos().items()}
-        dt_ms = (time.perf_counter() - start) * 1e3
-        logger.debug(f"{self} read state: {dt_ms:.1f}ms")
-
-        for cam_key, cam in self.cameras.items():
-            start = time.perf_counter()
-            obs_dict[cam_key] = cam.read_latest()
-            dt_ms = (time.perf_counter() - start) * 1e3
-            logger.debug(f"{self} read {cam_key}: {dt_ms:.1f}ms")
-
-        return obs_dict
-
-    @check_if_not_connected
-    def send_action(self, action: RobotAction) -> RobotAction:
-        """Command the arm to a target joint configuration.
-
-        Positions are expressed in degrees. The relative action magnitude may be
-        clipped depending on `max_relative_target`, so the action actually sent is
-        always returned.
-        """
-        goal_pos = {key.removesuffix(".pos"): val for key, val in action.items() if key.endswith(".pos")}
-
-        # Clip against soft joint limits.
-        for motor_name in list(goal_pos):
-            if motor_name in self.config.joint_limits:
-                min_limit, max_limit = self.config.joint_limits[motor_name]
-                clipped = max(min_limit, min(max_limit, goal_pos[motor_name]))
-                if clipped != goal_pos[motor_name]:
-                    logger.debug(f"Clipped {motor_name} from {goal_pos[motor_name]:.2f} to {clipped:.2f}")
-                goal_pos[motor_name] = clipped
-
-        # Tolerate 6-DOF leaders that have no wrist_yaw joint by holding it at zero.
-        # This is intentional: it lets a 6-DOF leader such as the SO-100 / SO-101
-        # (so100_leader / so101_leader) teleoperate this 7-DOF follower — the missing
-        # wrist_yaw command is simply treated as 0.0 instead of raising.
-        if "wrist_yaw" not in goal_pos:
-            goal_pos["wrist_yaw"] = 0.0
-
-        # Cap relative target when too far from the present position.
-        if self.config.max_relative_target is not None:
-            present_pos = self._present_pos()
-            goal_present_pos = {key: (g, present_pos.get(key, g)) for key, g in goal_pos.items()}
-            goal_pos = ensure_safe_goal_position(goal_present_pos, self.config.max_relative_target)
-
-        for motor_name, position_deg in goal_pos.items():
-            motor = self.motors.get(motor_name)
-            if motor is None:
-                continue
-            idx = self.motor_names.index(motor_name)
-            vel_deg_s = (
-                self.config.pos_vel_velocity[idx]
-                if isinstance(self.config.pos_vel_velocity, list)
-                else self.config.pos_vel_velocity
-            )
-            pos_rad = math.radians(position_deg)
-            vel_rad = math.radians(vel_deg_s)
-            if motor_name == GRIPPER_MOTOR:
-                motor.send_force_pos(pos_rad, vel_rad, self.config.gripper_torque_ratio)
-            else:
-                motor.send_pos_vel(pos_rad, vel_rad)
-
-        return {f"{motor}.pos": val for motor, val in goal_pos.items()}
-
-    @check_if_not_connected
-    def disconnect(self) -> None:
-        for motor in self.motors.values():
-            if self.config.disable_torque_on_disconnect:
-                motor.disable()
-            motor.clear_error()
-            motor.close()
-
-        self.bus.close()
-        self.bus = None
-        self.motors = {}
-
-        for cam in self.cameras.values():
-            cam.disconnect()
-
-        logger.info(f"{self} disconnected.")

src/lerobot/robots/utils.py

@@ -68,14 +68,6 @@ def make_robot_from_config(config: RobotConfig) -> Robot:
         from .bi_openarm_follower import BiOpenArmFollower
 
         return BiOpenArmFollower(config)
-    elif config.type == "rebot_b601_follower":
-        from .rebot_b601_follower import RebotB601Follower
-
-        return RebotB601Follower(config)
-    elif config.type == "bi_rebot_b601_follower":
-        from .bi_rebot_b601_follower import BiRebotB601Follower
-
-        return BiRebotB601Follower(config)
     elif config.type == "mock_robot":
         from tests.mocks.mock_robot import MockRobot
 

src/lerobot/rollout/context.py

@@ -332,7 +332,7 @@ def build_rollout_context(
                 cfg.dataset.repo_id,
                 root=cfg.dataset.root,
                 batch_encoding_size=cfg.dataset.video_encoding_batch_size,
-                camera_encoder=cfg.dataset.camera_encoder,
+                vcodec=cfg.dataset.vcodec,
                 streaming_encoding=cfg.dataset.streaming_encoding,
                 encoder_queue_maxsize=cfg.dataset.encoder_queue_maxsize,
                 encoder_threads=cfg.dataset.encoder_threads,
@@ -367,7 +367,7 @@ def build_rollout_context(
                 image_writer_threads=cfg.dataset.num_image_writer_threads_per_camera
                 * len(robot.cameras if hasattr(robot, "cameras") else []),
                 batch_encoding_size=cfg.dataset.video_encoding_batch_size,
-                camera_encoder=cfg.dataset.camera_encoder,
+                vcodec=cfg.dataset.vcodec,
                 streaming_encoding=cfg.dataset.streaming_encoding,
                 encoder_queue_maxsize=cfg.dataset.encoder_queue_maxsize,
                 encoder_threads=cfg.dataset.encoder_threads,

src/lerobot/scripts/lerobot_calibrate.py

@@ -39,7 +39,6 @@ from lerobot.robots import (  # noqa: F401
     Robot,
     RobotConfig,
     bi_openarm_follower,
-    bi_rebot_b601_follower,
     bi_so_follower,
     hope_jr,
     koch_follower,
@@ -47,14 +46,12 @@ from lerobot.robots import (  # noqa: F401
     make_robot_from_config,
     omx_follower,
     openarm_follower,
-    rebot_b601_follower,
     so_follower,
 )
 from lerobot.teleoperators import (  # noqa: F401
     Teleoperator,
     TeleoperatorConfig,
     bi_openarm_leader,
-    bi_rebot_102_leader,
     bi_so_leader,
     homunculus,
     koch_leader,
@@ -62,7 +59,6 @@ from lerobot.teleoperators import (  # noqa: F401
     omx_leader,
     openarm_leader,
     openarm_mini,
-    rebot_102_leader,
     so_leader,
     unitree_g1,
 )

src/lerobot/scripts/lerobot_edit_dataset.py

@@ -178,31 +178,6 @@ Recompute stats for relative actions and push to hub:
         --operation.num_workers 4 \
         --push_to_hub true
 
-Re-encode all videos in a dataset (saves to lerobot/pusht_reencoded by default):
-    lerobot-edit-dataset \
-        --repo_id lerobot/pusht \
-        --operation.type reencode_videos \
-        --operation.camera_encoder.vcodec h264 \
-        --operation.camera_encoder.pix_fmt yuv420p \
-        --operation.camera_encoder.crf 23
-
-Re-encode videos into a new dataset using 4 parallel processes:
-    lerobot-edit-dataset \
-        --repo_id lerobot/pusht \
-        --new_repo_id lerobot/pusht_h264 \
-        --operation.type reencode_videos \
-        --operation.camera_encoder.vcodec h264 \
-        --operation.camera_encoder.crf 23 \
-        --operation.num_workers 4
-
-Re-encode videos in-place (overwrites original dataset):
-    lerobot-edit-dataset \
-        --repo_id lerobot/pusht \
-        --new_repo_id lerobot/pusht \
-        --operation.type reencode_videos \
-        --operation.camera_encoder.vcodec h264 \
-        --operation.overwrite true
-
 Using JSON config file:
     lerobot-edit-dataset \
         --config_path path/to/edit_config.json
@@ -212,12 +187,12 @@ import abc
 import logging
 import shutil
 import sys
-from dataclasses import dataclass, field
+from dataclasses import dataclass
 from pathlib import Path
 
 import draccus
 
-from lerobot.configs import VideoEncoderConfig, camera_encoder_defaults, parser
+from lerobot.configs import parser
 from lerobot.datasets import (
     LeRobotDataset,
     convert_image_to_video_dataset,
@@ -225,7 +200,6 @@ from lerobot.datasets import (
     merge_datasets,
     modify_tasks,
     recompute_stats,
-    reencode_dataset,
     remove_feature,
     split_dataset,
 )
@@ -276,7 +250,11 @@ class ModifyTasksConfig(OperationConfig):
 @dataclass
 class ConvertImageToVideoConfig(OperationConfig):
     output_dir: str | None = None
-    camera_encoder: VideoEncoderConfig = field(default_factory=camera_encoder_defaults)
+    vcodec: str = "libsvtav1"
+    pix_fmt: str = "yuv420p"
+    g: int = 2
+    crf: int = 30
+    fast_decode: int = 0
     episode_indices: list[int] | None = None
     num_workers: int = 4
     max_episodes_per_batch: int | None = None
@@ -294,15 +272,6 @@ class RecomputeStatsConfig(OperationConfig):
     overwrite: bool = False
 
 
-@OperationConfig.register_subclass("reencode_videos")
-@dataclass
-class ReencodeVideosConfig(OperationConfig):
-    camera_encoder: VideoEncoderConfig = field(default_factory=camera_encoder_defaults)
-    num_workers: int = 0
-    encoder_threads: int | None = None
-    overwrite: bool = False
-
-
 @OperationConfig.register_subclass("info")
 @dataclass
 class InfoConfig(OperationConfig):
@@ -588,7 +557,11 @@ def handle_convert_image_to_video(cfg: EditDatasetConfig) -> None:
         dataset=dataset,
         output_dir=output_dir,
         repo_id=output_repo_id,
-        camera_encoder=getattr(cfg.operation, "camera_encoder", None) or camera_encoder_defaults(),
+        vcodec=getattr(cfg.operation, "vcodec", "libsvtav1"),
+        pix_fmt=getattr(cfg.operation, "pix_fmt", "yuv420p"),
+        g=getattr(cfg.operation, "g", 2),
+        crf=getattr(cfg.operation, "crf", 30),
+        fast_decode=getattr(cfg.operation, "fast_decode", 0),
         episode_indices=getattr(cfg.operation, "episode_indices", None),
         num_workers=getattr(cfg.operation, "num_workers", 4),
         max_episodes_per_batch=getattr(cfg.operation, "max_episodes_per_batch", None),
@@ -669,58 +642,6 @@ def handle_recompute_stats(cfg: EditDatasetConfig) -> None:
         dataset.push_to_hub()
 
 
-def handle_reencode_videos(cfg: EditDatasetConfig) -> None:
-    if not isinstance(cfg.operation, ReencodeVideosConfig):
-        raise ValueError("Operation config must be ReencodeVideosConfig")
-
-    output_repo_id, input_root, output_root = _resolve_io_paths(
-        cfg.repo_id,
-        cfg.new_repo_id,
-        cfg.root,
-        cfg.new_root,
-        default_new_repo_id=f"{cfg.repo_id}_reencoded",
-    )
-    in_place = output_root == input_root
-
-    if in_place and not cfg.operation.overwrite:
-        raise ValueError(
-            f"reencode_videos would overwrite the dataset in-place at {input_root}. "
-            "Pass --operation.overwrite true to allow in-place modification, "
-            "or use --new_repo_id / --new_root to write to a different location. "
-            f"Default output repo_id when neither is set: '{cfg.repo_id}_reencoded'."
-        )
-
-    if in_place:
-        logging.warning(
-            f"Overwriting dataset videos in-place at {input_root}. The original videos will be lost."
-        )
-        dataset = LeRobotDataset(cfg.repo_id, root=input_root)
-    else:
-        logging.info(f"Copying dataset from {input_root} to {output_root}")
-        if output_root.exists():
-            backup_path = output_root.with_name(output_root.name + "_old")
-            logging.warning(f"Output directory {output_root} already exists. Moving to {backup_path}")
-            if backup_path.exists():
-                shutil.rmtree(backup_path)
-            shutil.move(output_root, backup_path)
-        shutil.copytree(input_root, output_root)
-        dataset = LeRobotDataset(output_repo_id, root=output_root)
-
-    logging.info(f"Re-encoding videos in {output_repo_id} with {cfg.operation.camera_encoder}")
-    reencode_dataset(
-        dataset,
-        camera_encoder=cfg.operation.camera_encoder,
-        encoder_threads=cfg.operation.encoder_threads,
-        num_workers=cfg.operation.num_workers,
-    )
-
-    logging.info(f"All videos re-encoded at {dataset.root}")
-
-    if cfg.push_to_hub:
-        logging.info(f"Pushing to hub as {output_repo_id}...")
-        dataset.push_to_hub()
-
-
 def _get_dataset_size(repo_path):
     import os
 
@@ -794,8 +715,6 @@ def edit_dataset(cfg: EditDatasetConfig) -> None:
         handle_convert_image_to_video(cfg)
     elif operation_type == "recompute_stats":
         handle_recompute_stats(cfg)
-    elif operation_type == "reencode_videos":
-        handle_reencode_videos(cfg)
     elif operation_type == "info":
         handle_info(cfg)
     else:

src/lerobot/scripts/lerobot_export_robometer.py

@@ -0,0 +1,151 @@
+#!/usr/bin/env python
+# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Re-save a Robometer checkpoint in LeRobot HF format.
+
+LeRobot's reward model format is ``config.json`` (a draccus-encoded
+:class:`~lerobot.rewards.robometer.RobometerConfig`) plus a single
+``model.safetensors`` containing the merged base + heads weights. The
+released checkpoint at ``lilkm/robometer-4b`` already follows this layout;
+this script is for converting other Robometer variants (e.g. a future
+upstream release or a local training run) into the same format.
+
+Example:
+
+.. code-block:: shell
+
+   lerobot-export-robometer \\
+       --src robometer/Robometer-4B \\
+       --dst ./robometer-4b-lerobot
+"""
+
+from __future__ import annotations
+
+import argparse
+import logging
+from pathlib import Path
+
+from lerobot.rewards.robometer import RobometerConfig, RobometerRewardModel
+from lerobot.rewards.robometer._upstream_loader import apply_upstream_checkpoint
+from lerobot.utils.utils import init_logging
+
+
+def export_robometer_to_lerobot(
+    src: str,
+    dst: str | Path,
+    *,
+    device: str = "cpu",
+    dataset_repo_id: str = "",
+    write_model_card: bool = True,
+) -> Path:
+    """Load Robometer from ``src`` and re-save it under ``dst`` in LeRobot HF format.
+
+    Produces ``config.json``, ``model.safetensors``, and (optionally) ``README.md``.
+
+    Args:
+        src: Upstream source. Hugging Face repo id (``"robometer/Robometer-4B"``,
+            optionally ``"...@revision"``) or a local snapshot directory.
+        dst: Output directory. ``config.json`` and ``model.safetensors`` are
+            written here.
+        device: Where to place the model during loading. Defaults to CPU; use
+            ``"cuda"`` if you want to verify on GPU before saving.
+        dataset_repo_id: Hugging Face dataset id the model was trained on
+            (e.g. ``"robometer/RBM-1M"``). Written into the model card's
+            ``datasets:`` metadata. Leave empty if not applicable.
+        write_model_card: Generate a ``README.md`` using LeRobot's reward
+            model card template. Disable if you want to write the README
+            yourself.
+
+    Returns:
+        The resolved output directory.
+    """
+    # A fresh ``RobometerConfig`` has ``vlm_config=None``, which routes
+    # ``__init__`` through the upstream-matching path: download base Qwen,
+    # resize embeddings per ``ROBOMETER_SPECIAL_TOKENS``. ``apply_upstream_checkpoint``
+    # then resizes again (if needed) to match the upstream checkpoint's vocab
+    # and overlays its weights. ``_save_pretrained`` snapshots the resulting
+    # post-resize architecture into ``vlm_config`` for fast future loads.
+    cfg = RobometerConfig(pretrained_path=src, device=device)
+    model = RobometerRewardModel(cfg)
+    apply_upstream_checkpoint(model, src)
+    model.to(device)
+    model.eval()
+
+    dst = Path(dst)
+    dst.mkdir(parents=True, exist_ok=True)
+    model.save_pretrained(str(dst))
+
+    if write_model_card:
+        card = model.generate_model_card(
+            dataset_repo_id=dataset_repo_id,
+            model_type=model.config.type,
+            license=model.config.license,
+            tags=model.config.tags,
+        )
+        card.save(str(dst / "README.md"))
+
+    return dst
+
+
+def _parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+    parser.add_argument(
+        "--src",
+        default="robometer/Robometer-4B",
+        help="Upstream Robometer source (HF repo id or local directory).",
+    )
+    parser.add_argument(
+        "--dst",
+        required=True,
+        help="Output directory for the LeRobot-format checkpoint.",
+    )
+    parser.add_argument(
+        "--device",
+        default="cpu",
+        help="Torch device to load the model on (default: cpu). Conversion only "
+        "needs CPU; use cuda if you also want to smoke-test inference.",
+    )
+    parser.add_argument(
+        "--dataset",
+        default="",
+        help="Optional Hugging Face dataset id used for training "
+        "(e.g. `robometer/RBM-1M`). Written into the auto-generated model card's "
+        "`datasets:` metadata.",
+    )
+    parser.add_argument(
+        "--no-readme",
+        action="store_true",
+        help="Skip writing README.md. Use if you want to author the model card by hand.",
+    )
+    return parser.parse_args()
+
+
+def main() -> None:
+    init_logging()
+    args = _parse_args()
+    out = export_robometer_to_lerobot(
+        src=args.src,
+        dst=args.dst,
+        device=args.device,
+        dataset_repo_id=args.dataset,
+        write_model_card=not args.no_readme,
+    )
+    logging.info("Saved LeRobot-format Robometer checkpoint to %s", out)
+
+
+if __name__ == "__main__":
+    main()

src/lerobot/scripts/lerobot_find_joint_limits.py

@@ -45,19 +45,16 @@ from lerobot.model import RobotKinematics
 from lerobot.robots import (  # noqa: F401
     RobotConfig,
     bi_openarm_follower,
-    bi_rebot_b601_follower,
     bi_so_follower,
     koch_follower,
     make_robot_from_config,
     omx_follower,
     openarm_follower,
-    rebot_b601_follower,
     so_follower,
 )
 from lerobot.teleoperators import (  # noqa: F401
     TeleoperatorConfig,
     bi_openarm_leader,
-    bi_rebot_102_leader,
     bi_so_leader,
     gamepad,
     koch_leader,
@@ -65,7 +62,6 @@ from lerobot.teleoperators import (  # noqa: F401
     omx_leader,
     openarm_leader,
     openarm_mini,
-    rebot_102_leader,
     so_leader,
 )
 from lerobot.utils.robot_utils import precise_sleep

src/lerobot/scripts/lerobot_record.py

@@ -63,27 +63,6 @@ lerobot-record \\
   --dataset.streaming_encoding=true \\
   --dataset.encoder_threads=2
 ```
-
-Example recording with custom video encoding parameters:
-```shell
-lerobot-record \\
-    --robot.type=so100_follower \\
-    --robot.port=/dev/tty.usbmodem58760431541 \\
-    --robot.cameras="{laptop: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \\
-    --robot.id=black \\
-    --teleop.type=so100_leader \\
-    --teleop.port=/dev/tty.usbmodem58760431551 \\
-    --teleop.id=blue \\
-    --dataset.repo_id=<my_username>/<my_dataset_name> \\
-    --dataset.num_episodes=2 \\
-    --dataset.single_task="Grab the cube" \\
-    --dataset.streaming_encoding=true \\
-    --dataset.encoder_threads=2 \\
-    --dataset.camera_encoder.vcodec=h264 \\
-    --dataset.camera_encoder.preset=fast \\
-    --dataset.camera_encoder.extra_options={"tune": "film", "profile:v": "high", "bf": 2} \\
-    --display_data=true
-```
 """
 
 import logging
@@ -120,7 +99,6 @@ from lerobot.robots import (  # noqa: F401
     Robot,
     RobotConfig,
     bi_openarm_follower,
-    bi_rebot_b601_follower,
     bi_so_follower,
     earthrover_mini_plus,
     hope_jr,
@@ -129,7 +107,6 @@ from lerobot.robots import (  # noqa: F401
     omx_follower,
     openarm_follower,
     reachy2,
-    rebot_b601_follower,
     so_follower,
     unitree_g1 as unitree_g1_robot,
 )
@@ -137,7 +114,6 @@ from lerobot.teleoperators import (  # noqa: F401
     Teleoperator,
     TeleoperatorConfig,
     bi_openarm_leader,
-    bi_rebot_102_leader,
     bi_so_leader,
     homunculus,
     koch_leader,
@@ -146,7 +122,6 @@ from lerobot.teleoperators import (  # noqa: F401
     openarm_leader,
     openarm_mini,
     reachy2_teleoperator,
-    rebot_102_leader,
     so_leader,
     unitree_g1,
 )
@@ -402,10 +377,10 @@ def record(
                 cfg.dataset.repo_id,
                 root=cfg.dataset.root,
                 batch_encoding_size=cfg.dataset.video_encoding_batch_size,
-                camera_encoder=cfg.dataset.camera_encoder,
-                encoder_threads=cfg.dataset.encoder_threads,
+                vcodec=cfg.dataset.vcodec,
                 streaming_encoding=cfg.dataset.streaming_encoding,
                 encoder_queue_maxsize=cfg.dataset.encoder_queue_maxsize,
+                encoder_threads=cfg.dataset.encoder_threads,
                 image_writer_processes=cfg.dataset.num_image_writer_processes if num_cameras > 0 else 0,
                 image_writer_threads=cfg.dataset.num_image_writer_threads_per_camera * num_cameras
                 if num_cameras > 0
@@ -431,10 +406,10 @@ def record(
                 image_writer_processes=cfg.dataset.num_image_writer_processes,
                 image_writer_threads=cfg.dataset.num_image_writer_threads_per_camera * len(robot.cameras),
                 batch_encoding_size=cfg.dataset.video_encoding_batch_size,
-                camera_encoder=cfg.dataset.camera_encoder,
-                encoder_threads=cfg.dataset.encoder_threads,
+                vcodec=cfg.dataset.vcodec,
                 streaming_encoding=cfg.dataset.streaming_encoding,
                 encoder_queue_maxsize=cfg.dataset.encoder_queue_maxsize,
+                encoder_threads=cfg.dataset.encoder_threads,
             )
 
         robot.connect()
@@ -445,7 +420,7 @@ def record(
 
         if not cfg.dataset.streaming_encoding:
             logging.info(
-                "Streaming encoding is disabled. If you have capable hardware, consider enabling it for way faster episode saving. --dataset.streaming_encoding=true --dataset.encoder_threads=2 # --dataset.camera_encoder.vcodec=auto. More info in the documentation: https://huggingface.co/docs/lerobot/streaming_video_encoding"
+                "Streaming encoding is disabled. If you have capable hardware, consider enabling it for way faster episode saving. --dataset.streaming_encoding=true --dataset.encoder_threads=2 # --dataset.vcodec=auto. More info in the documentation: https://huggingface.co/docs/lerobot/streaming_video_encoding"
             )
 
         with VideoEncodingManager(dataset):

src/lerobot/scripts/lerobot_replay.py

@@ -56,7 +56,6 @@ from lerobot.robots import (  # noqa: F401
     Robot,
     RobotConfig,
     bi_openarm_follower,
-    bi_rebot_b601_follower,
     bi_so_follower,
     earthrover_mini_plus,
     hope_jr,
@@ -65,7 +64,6 @@ from lerobot.robots import (  # noqa: F401
     omx_follower,
     openarm_follower,
     reachy2,
-    rebot_b601_follower,
     so_follower,
     unitree_g1,
 )

src/lerobot/scripts/lerobot_rollout.py

@@ -120,18 +120,6 @@ Usage examples
         --dataset.repo_id=user/rollout_sentry_data \\
         --dataset.single_task="patrol" \\
         --resume=true
-
-    # Rollout with custom video encoding parameters
-    lerobot-rollout \\
-        --strategy.type=base \\
-        --policy.path=lerobot/act_koch_real \\
-        --robot.type=koch_follower \\
-        --robot.port=/dev/ttyACM0 \\
-        --task="pick up cube" --duration=60 \\
-        --display_data=true \\
-        --dataset.camera_encoder.vcodec=h264 \\
-        --dataset.camera_encoder.preset=fast \\
-        --dataset.camera_encoder.extra_options={"tune": "film", "profile:v": "high", "bf": 2}
 """
 
 import logging
@@ -144,7 +132,6 @@ from lerobot.robots import (  # noqa: F401
     Robot,
     RobotConfig,
     bi_openarm_follower,
-    bi_rebot_b601_follower,
     bi_so_follower,
     earthrover_mini_plus,
     hope_jr,
@@ -152,7 +139,6 @@ from lerobot.robots import (  # noqa: F401
     omx_follower,
     openarm_follower,
     reachy2,
-    rebot_b601_follower,
     so_follower,
     unitree_g1 as unitree_g1_robot,
 )
@@ -161,7 +147,6 @@ from lerobot.teleoperators import (  # noqa: F401
     Teleoperator,
     TeleoperatorConfig,
     bi_openarm_leader,
-    bi_rebot_102_leader,
     bi_so_leader,
     homunculus,
     koch_leader,
@@ -169,7 +154,6 @@ from lerobot.teleoperators import (  # noqa: F401
     openarm_leader,
     openarm_mini,
     reachy2_teleoperator,
-    rebot_102_leader,
     so_leader,
     unitree_g1,
 )

src/lerobot/scripts/lerobot_setup_motors.py

@@ -30,24 +30,20 @@ import draccus
 
 from lerobot.robots import (  # noqa: F401
     RobotConfig,
-    bi_rebot_b601_follower,
     bi_so_follower,
     koch_follower,
     lekiwi,
     make_robot_from_config,
     omx_follower,
-    rebot_b601_follower,
     so_follower,
 )
 from lerobot.teleoperators import (  # noqa: F401
     TeleoperatorConfig,
-    bi_rebot_102_leader,
     bi_so_leader,
     koch_leader,
     make_teleoperator_from_config,
     omx_leader,
     openarm_mini,
-    rebot_102_leader,
     so_leader,
 )
 

src/lerobot/scripts/lerobot_teleoperate.py

@@ -72,7 +72,6 @@ from lerobot.robots import (  # noqa: F401
     Robot,
     RobotConfig,
     bi_openarm_follower,
-    bi_rebot_b601_follower,
     bi_so_follower,
     earthrover_mini_plus,
     hope_jr,
@@ -81,7 +80,6 @@ from lerobot.robots import (  # noqa: F401
     omx_follower,
     openarm_follower,
     reachy2,
-    rebot_b601_follower,
     so_follower,
     unitree_g1 as unitree_g1_robot,
 )
@@ -89,7 +87,6 @@ from lerobot.teleoperators import (  # noqa: F401
     Teleoperator,
     TeleoperatorConfig,
     bi_openarm_leader,
-    bi_rebot_102_leader,
     bi_so_leader,
     gamepad,
     homunculus,
@@ -100,7 +97,6 @@ from lerobot.teleoperators import (  # noqa: F401
     openarm_leader,
     openarm_mini,
     reachy2_teleoperator,
-    rebot_102_leader,
     so_leader,
     unitree_g1,
 )

src/lerobot/scripts/lerobot_train.py

@@ -48,7 +48,6 @@ from lerobot.envs import close_envs, make_env, make_env_pre_post_processors
 from lerobot.optim.factory import make_optimizer_and_scheduler
 from lerobot.policies import PreTrainedPolicy, make_policy, make_pre_post_processors
 from lerobot.rewards import make_reward_pre_post_processors
-from lerobot.utils.collate import lerobot_collate_fn
 from lerobot.utils.import_utils import register_third_party_plugins
 from lerobot.utils.logging_utils import AverageMeter, MetricsTracker
 from lerobot.utils.random_utils import set_seed
@@ -402,10 +401,6 @@ def train(cfg: TrainPipelineConfig, accelerator: "Accelerator | None" = None):
         shuffle = True
         sampler = None
 
-    # Only swap in the language-aware collate when the dataset actually
-    # declares language columns; otherwise stay on PyTorch's default
-    # collate so non-language training runs are unaffected.
-    collate_fn = lerobot_collate_fn if dataset.meta.has_language_columns else None
     dataloader = torch.utils.data.DataLoader(
         dataset,
         num_workers=cfg.num_workers,
@@ -414,7 +409,6 @@ def train(cfg: TrainPipelineConfig, accelerator: "Accelerator | None" = None):
         sampler=sampler,
         pin_memory=device.type == "cuda",
         drop_last=False,
-        collate_fn=collate_fn,
         prefetch_factor=cfg.prefetch_factor if cfg.num_workers > 0 else None,
         persistent_workers=cfg.persistent_workers and cfg.num_workers > 0,
     )

src/lerobot/teleoperators/bi_rebot_102_leader/__init__.py

@@ -1,20 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from .bi_rebot_102_leader import BiRebotArm102Leader
-from .config_bi_rebot_102_leader import BiRebotArm102LeaderConfig
-
-__all__ = ["BiRebotArm102Leader", "BiRebotArm102LeaderConfig"]

src/lerobot/teleoperators/bi_rebot_102_leader/bi_rebot_102_leader.py

@@ -1,113 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import logging
-from functools import cached_property
-
-from lerobot.types import RobotAction
-from lerobot.utils.decorators import check_if_already_connected, check_if_not_connected
-
-from ..rebot_102_leader import RebotArm102Leader, RebotArm102LeaderTeleopConfig
-from ..teleoperator import Teleoperator
-from .config_bi_rebot_102_leader import BiRebotArm102LeaderConfig
-
-logger = logging.getLogger(__name__)
-
-
-class BiRebotArm102Leader(Teleoperator):
-    """Bimanual Seeed Studio StarArm102 / reBot Arm 102 leader.
-
-    Composes two single-arm :class:`RebotArm102Leader` instances. Action keys of
-    each arm are namespaced with a ``left_`` / ``right_`` prefix, so a bimanual
-    leader can teleoperate a bimanual reBot B601 follower.
-    """
-
-    config_class = BiRebotArm102LeaderConfig
-    name = "bi_rebot_102_leader"
-
-    def __init__(self, config: BiRebotArm102LeaderConfig):
-        super().__init__(config)
-        self.config = config
-
-        left_arm_config = RebotArm102LeaderTeleopConfig(
-            id=f"{config.id}_left" if config.id else None,
-            calibration_dir=config.calibration_dir,
-            port=config.left_arm_config.port,
-            baudrate=config.left_arm_config.baudrate,
-            joint_ids=config.left_arm_config.joint_ids,
-            joint_directions=config.left_arm_config.joint_directions,
-            joint_ranges=config.left_arm_config.joint_ranges,
-        )
-
-        right_arm_config = RebotArm102LeaderTeleopConfig(
-            id=f"{config.id}_right" if config.id else None,
-            calibration_dir=config.calibration_dir,
-            port=config.right_arm_config.port,
-            baudrate=config.right_arm_config.baudrate,
-            joint_ids=config.right_arm_config.joint_ids,
-            joint_directions=config.right_arm_config.joint_directions,
-            joint_ranges=config.right_arm_config.joint_ranges,
-        )
-
-        self.left_arm = RebotArm102Leader(left_arm_config)
-        self.right_arm = RebotArm102Leader(right_arm_config)
-
-    @cached_property
-    def action_features(self) -> dict[str, type]:
-        return {
-            **{f"left_{k}": v for k, v in self.left_arm.action_features.items()},
-            **{f"right_{k}": v for k, v in self.right_arm.action_features.items()},
-        }
-
-    @cached_property
-    def feedback_features(self) -> dict[str, type]:
-        return {}
-
-    @property
-    def is_connected(self) -> bool:
-        return self.left_arm.is_connected and self.right_arm.is_connected
-
-    @check_if_already_connected
-    def connect(self, calibrate: bool = True) -> None:
-        self.left_arm.connect(calibrate)
-        self.right_arm.connect(calibrate)
-
-    @property
-    def is_calibrated(self) -> bool:
-        return self.left_arm.is_calibrated and self.right_arm.is_calibrated
-
-    def calibrate(self) -> None:
-        self.left_arm.calibrate()
-        self.right_arm.calibrate()
-
-    def configure(self) -> None:
-        self.left_arm.configure()
-        self.right_arm.configure()
-
-    @check_if_not_connected
-    def get_action(self) -> RobotAction:
-        action_dict = {}
-        action_dict.update({f"left_{k}": v for k, v in self.left_arm.get_action().items()})
-        action_dict.update({f"right_{k}": v for k, v in self.right_arm.get_action().items()})
-        return action_dict
-
-    def send_feedback(self, feedback: dict[str, float]) -> None:
-        raise NotImplementedError("Feedback is not implemented for the reBot Arm 102 leader.")
-
-    @check_if_not_connected
-    def disconnect(self) -> None:
-        self.left_arm.disconnect()
-        self.right_arm.disconnect()

src/lerobot/teleoperators/bi_rebot_102_leader/config_bi_rebot_102_leader.py

@@ -1,29 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from dataclasses import dataclass
-
-from ..config import TeleoperatorConfig
-from ..rebot_102_leader import RebotArm102LeaderConfig
-
-
-@TeleoperatorConfig.register_subclass("bi_rebot_102_leader")
-@dataclass
-class BiRebotArm102LeaderConfig(TeleoperatorConfig):
-    """Configuration class for the bimanual reBot Arm 102 leader teleoperator."""
-
-    left_arm_config: RebotArm102LeaderConfig
-    right_arm_config: RebotArm102LeaderConfig

src/lerobot/teleoperators/rebot_102_leader/__init__.py

@@ -1,20 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from .config_rebot_102_leader import RebotArm102LeaderConfig, RebotArm102LeaderTeleopConfig
-from .rebot_102_leader import RebotArm102Leader
-
-__all__ = ["RebotArm102Leader", "RebotArm102LeaderConfig", "RebotArm102LeaderTeleopConfig"]

src/lerobot/teleoperators/rebot_102_leader/config_rebot_102_leader.py

@@ -1,83 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from dataclasses import dataclass, field
-
-from ..config import TeleoperatorConfig
-
-
-@dataclass
-class RebotArm102LeaderConfig:
-    """Base configuration class for the Seeed Studio StarArm102 / reBot Arm 102 leader.
-
-    The reBot Arm 102 is a 7-joint (incl. gripper) leader arm driven by FashionStar
-    UART smart servos. Servo communication goes through ``motorbridge-smart-servo``.
-    """
-
-    # USB-to-UART device the leader arm is connected to (e.g. "/dev/ttyUSB0").
-    port: str
-
-    baudrate: int = 1_000_000
-
-    # Servo id of each joint on the UART bus.
-    joint_ids: dict[str, int] = field(
-        default_factory=lambda: {
-            "shoulder_pan": 0,
-            "shoulder_lift": 1,
-            "elbow_flex": 2,
-            "wrist_flex": 3,
-            "wrist_yaw": 4,
-            "wrist_roll": 5,
-            "gripper": 6,
-        }
-    )
-
-    # Per-joint sign applied to raw servo angles so the leader matches the follower
-    # convention. The gripper additionally carries a scale (e.g. -6) to widen its
-    # range to the reBot B601 follower's gripper travel.
-    joint_directions: dict[str, int] = field(
-        default_factory=lambda: {
-            "shoulder_pan": -1,
-            "shoulder_lift": -1,
-            "elbow_flex": 1,
-            "wrist_flex": 1,
-            "wrist_yaw": 1,
-            "wrist_roll": -1,
-            "gripper": -6,
-        }
-    )
-
-    # Per-joint [min, max] output range in degrees. Matches the reBot B601 follower
-    # joint limits so leader actions can drive the follower key-for-key.
-    joint_ranges: dict[str, list[int]] = field(
-        default_factory=lambda: {
-            "shoulder_pan": [-150, 150],
-            "shoulder_lift": [-170, 1],
-            "elbow_flex": [-200, 1],
-            "wrist_flex": [-80, 90],
-            "wrist_yaw": [-90, 90],
-            "wrist_roll": [-90, 90],
-            "gripper": [-270, 0],
-        }
-    )
-
-
-@TeleoperatorConfig.register_subclass("rebot_102_leader")
-@dataclass
-class RebotArm102LeaderTeleopConfig(TeleoperatorConfig, RebotArm102LeaderConfig):
-    """Registered configuration for the reBot Arm 102 leader teleoperator."""
-
-    pass

src/lerobot/teleoperators/rebot_102_leader/rebot_102_leader.py

@@ -1,207 +0,0 @@
-#!/usr/bin/env python
-
-# Copyright 2026 The HuggingFace Inc. team. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import logging
-import time
-from typing import TYPE_CHECKING
-
-from lerobot.motors import MotorCalibration
-from lerobot.types import RobotAction
-from lerobot.utils.decorators import check_if_already_connected, check_if_not_connected
-from lerobot.utils.import_utils import _motorbridge_smart_servo_available, require_package
-
-from ..teleoperator import Teleoperator
-from .config_rebot_102_leader import RebotArm102LeaderTeleopConfig
-
-if TYPE_CHECKING or _motorbridge_smart_servo_available:
-    from motorbridge_smart_servo import FashionStarServo, ServoMonitor
-else:
-    FashionStarServo = None
-    ServoMonitor = None
-
-logger = logging.getLogger(__name__)
-
-_SETTLE_SEC = 0.01
-
-
-class RebotArm102Leader(Teleoperator):
-    """Seeed Studio StarArm102 / reBot Arm 102 leader arm.
-
-    A 7-joint (incl. gripper) leader built on FashionStar UART smart servos. Servo
-    communication is handled by the ``motorbridge-smart-servo`` package; this class
-    only reads joint angles, so it produces actions but accepts no feedback.
-    """
-
-    config_class = RebotArm102LeaderTeleopConfig
-    name = "rebot_102_leader"
-
-    def __init__(self, config: RebotArm102LeaderTeleopConfig):
-        require_package("motorbridge-smart-servo", extra="rebot", import_name="motorbridge_smart_servo")
-        super().__init__(config)
-        self.config = config
-        self.bus: FashionStarServo | None = None
-        self.motor_names = list(config.joint_ids.keys())
-        self._last_raw_positions: dict[str, float] = {}
-
-    @property
-    def action_features(self) -> dict[str, type]:
-        return {f"{motor}.pos": float for motor in self.motor_names}
-
-    @property
-    def feedback_features(self) -> dict[str, type]:
-        return {}
-
-    @property
-    def is_connected(self) -> bool:
-        return self.bus is not None
-
-    @check_if_already_connected
-    def connect(self, calibrate: bool = True) -> None:
-        logger.info(f"Connecting {self} on {self.config.port}...")
-        bus = FashionStarServo(self.config.port, baudrate=self.config.baudrate)
-        try:
-            for motor_name, motor_id in self.config.joint_ids.items():
-                if not bus.ping(motor_id):
-                    raise RuntimeError(f"Servo not found for {motor_name} (id={motor_id}).")
-                self._last_raw_positions[motor_name] = 0.0
-            self.bus = bus
-
-            if not self.is_calibrated and calibrate:
-                logger.info(
-                    "Mismatch between calibration values in the motor and the calibration file or no calibration file found"
-                )
-                self.calibrate()
-
-            self.configure()
-        except Exception:
-            bus.close()
-            self.bus = None
-            raise
-
-        logger.info(f"{self} connected.")
-
-    @property
-    def is_calibrated(self) -> bool:
-        return bool(self.calibration) and set(self.calibration) == set(self.motor_names)
-
-    def calibrate(self) -> None:
-        if self.calibration:
-            user_input = input(
-                f"Press ENTER to use provided calibration file associated with the id {self.id}, "
-                "or type 'c' and press ENTER to run calibration: "
-            )
-            if user_input.strip().lower() != "c":
-                logger.info(f"Using calibration file associated with the id {self.id}")
-                return
-
-        logger.info(f"\nRunning calibration of {self}")
-        input(
-            "\nCalibration: set zero position.\n"
-            "Manually move the reBot Arm 102 to its zero pose and close the gripper.\n"
-            "Press ENTER when ready..."
-        )
-
-        self.calibration = {}
-        for motor_name, motor_id in self.config.joint_ids.items():
-            self.bus.unlock(motor_id)
-            time.sleep(_SETTLE_SEC)
-            self.bus.set_origin_point(motor_id)
-            range_min, range_max = self.config.joint_ranges[motor_name]
-            self.calibration[motor_name] = MotorCalibration(
-                id=motor_id,
-                drive_mode=0,
-                homing_offset=0,
-                range_min=int(range_min),
-                range_max=int(range_max),
-            )
-
-        self._save_calibration()
-        logger.info(f"Calibration saved to {self.calibration_fpath}")
-
-    def configure(self) -> None:
-        for motor_id in self.config.joint_ids.values():
-            self.bus.unlock(motor_id)
-            time.sleep(_SETTLE_SEC)
-        # Reset the multi-turn counter of each servo individually.
-        for motor_id in self.config.joint_ids.values():
-            self.bus.reset_multi_turn(motor_id)
-
-    def _read_raw_positions(self) -> dict[str, float]:
-        result: dict[int, ServoMonitor | None] = self.bus.sync_monitor(list(self.config.joint_ids.values()))
-        id_to_name = {v: k for k, v in self.config.joint_ids.items()}
-        raw_positions: dict[str, float] = {}
-        for motor_id, monitor in result.items():
-            motor_name = id_to_name[motor_id]
-            if monitor is None:
-                raise RuntimeError(f"Servo {motor_name} (id={motor_id}) has never responded.")
-            raw_positions[motor_name] = monitor.angle_deg
-        return raw_positions
-
-    @staticmethod
-    def _round_to_valid_range(value: float, min_value: float, max_value: float) -> tuple[float, int]:
-        """Unwrap a multi-turn angle into the ±180° window centred on (min+max)/2.
-
-        The servo may report an angle that has accumulated extra full rotations
-        (value = true_angle + N*360). Subtract the nearest whole number of turns
-        to bring it back into [center-180, center+180]. Returns the unwrapped
-        angle and the number of turns removed.
-        """
-        center = (min_value + max_value) / 2.0
-        turns = round((value - center) / 360.0)
-        return value - turns * 360.0, abs(turns)
-
-    @check_if_not_connected
-    def get_action(self) -> RobotAction:
-        start = time.perf_counter()
-        try:
-            raw_positions = self._read_raw_positions()
-            self._last_raw_positions = raw_positions
-        except Exception as e:
-            logger.error(f"Failed to read raw positions: {e}")
-            logger.warning("[EMERGENCY STOP] Hold the follower arm and cut off the main power to the arms.")
-            logger.warning(
-                "[EMERGENCY STOP] Break the teleoperation session and check the leader USB connection or power."
-            )
-            raw_positions = self._last_raw_positions
-
-        action_dict: dict[str, float] = {}
-        for motor_name in self.motor_names:
-            range_min, range_max = self.config.joint_ranges[motor_name]
-            direction = self.config.joint_directions[motor_name]
-            sign = 1.0 if direction >= 0 else -1.0
-            unwrapped, k = self._round_to_valid_range(
-                raw_positions[motor_name], range_min * sign, range_max * sign
-            )
-            position = unwrapped * direction
-            if k > 0:
-                logger.debug(
-                    f"Servo {motor_name} (id={self.config.joint_ids[motor_name]}) wrapped {k} * 360°. "
-                    f"Unwrapped pos: {unwrapped:.1f}° (raw: {raw_positions[motor_name]:.1f}°)"
-                )
-            action_dict[f"{motor_name}.pos"] = max(float(range_min), min(float(range_max), position))
-
-        dt_ms = (time.perf_counter() - start) * 1e3
-        logger.debug(f"{self} read action: {dt_ms:.1f}ms")
-        return action_dict
-
-    def send_feedback(self, feedback: dict[str, float]) -> None:
-        raise NotImplementedError("Feedback is not implemented for the reBot Arm 102 leader.")
-
-    @check_if_not_connected
-    def disconnect(self) -> None:
-        self.bus.close()
-        self.bus = None
-        logger.info(f"{self} disconnected.")

src/lerobot/teleoperators/utils.py

@@ -99,14 +99,6 @@ def make_teleoperator_from_config(config: TeleoperatorConfig) -> "Teleoperator":
         from .openarm_mini import OpenArmMini
 
         return OpenArmMini(config)
-    elif config.type == "rebot_102_leader":
-        from .rebot_102_leader import RebotArm102Leader
-
-        return RebotArm102Leader(config)
-    elif config.type == "bi_rebot_102_leader":
-        from .bi_rebot_102_leader import BiRebotArm102Leader
-
-        return BiRebotArm102Leader(config)
     else:
         try:
             return cast("Teleoperator", make_device_from_device_class(config))

src/lerobot/templates/lerobot_rewardmodel_modelcard_template.md

@@ -13,8 +13,8 @@
 A reward classifier is a lightweight neural network that scores observations or trajectories for task success, providing a learned reward signal or offline evaluation when explicit rewards are unavailable.
 {% elif model_name == "sarm" %}
 A Success-Aware Reward Model (SARM) predicts a dense reward signal from observations, typically used downstream for reinforcement learning or human-in-the-loop fine-tuning when task success is not directly observable.
-{% elif model_name == "topreward" %}
-TOPReward is a **zero-shot** reward model that extracts token log-probabilities from an off-the-shelf vision-language model (default Qwen3-VL) as a reward signal. Given a video trajectory and a task instruction, it returns the VLM's log-likelihood of the instruction being true, with no fine-tuning required.
+{% elif model_name == "robometer" %}
+Robometer is a zero-shot general-purpose robotic reward model built on a fine-tuned Qwen3-VL backbone with progress, preference, and success heads. Given a video and a task description it outputs a per-frame progress signal in [0, 1] and a per-frame success probability — suitable for offline reward labelling and for low-frequency reward signals during RL fine-tuning of robot policies.
 {% else %}
 _Reward model type not recognized — please update this template._
 {% endif %}

src/lerobot/transport/utils.py

@@ -25,10 +25,9 @@ from typing import Any
 
 import torch
 
+from lerobot.transport import services_pb2
 from lerobot.utils.transition import Transition
 
-from . import services_pb2
-
 # FIX for protobuf: Assign the enum to a variable and ignore the type error once
 TransferState = services_pb2.TransferState  # type: ignore[attr-defined]