* fix(record): pass rename_map to make_policy in lerobot-record
Fixes#3181. The rename_map from dataset config was used for preprocessor
construction but not passed to make_policy(), causing feature mismatch
errors when camera key names differ between dataset and model config.
make_policy() already accepts a rename_map parameter and uses it to skip
visual feature consistency validation when remapping is active, but
lerobot_record.py was not passing it through.
* style: fix ruff format for ternary expression
---------
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
* docs(benchmarks): add benchmark integration guide and standardize benchmark docs
Add a comprehensive guide for adding new benchmarks to LeRobot, and
refactor the existing LIBERO and Meta-World docs to follow the new
standardized template.
Made-with: Cursor
* refactor(envs): move dispatch logic from factory into EnvConfig subclasses
Replace hardcoded if/elif chains in factory.py with create_envs() and
get_env_processors() methods on EnvConfig. New benchmarks now only need
to register a config subclass — no factory.py edits required.
Net -23 lines: factory.py shrinks from ~200 to ~70 lines of logic.
Made-with: Cursor
* docs(benchmarks): clean up adding-benchmarks guide for clarity
Rewrite for simpler language, better structure, and easier navigation.
Move quick-reference table to the top, fold eval explanation into
architecture section, condense the doc template to a bulleted outline.
Made-with: Cursor
* fix link
* fix task count
* fix: enable SmolVLA eval on LIBERO with custom camera mappings
- Thread camera_name_mapping from LiberoEnv config through to gym envs
- Sync features_map with camera_name_mapping in LiberoEnv.__post_init__
- Fix render() to use first available camera instead of hardcoded "image"
- Handle non-dict final_info in rollout by falling back to info["is_success"]
- Add use_peft legacy field to SmolVLAConfig for checkpoint compat
- Add defaults to GR00TN15Config init=False fields for transformers 5.3
Made-with: Cursor
* fix: use direct AutoresetMode import for gymnasium compat
Made-with: Cursor
* fix: handle gymnasium < 1.0 without AutoresetMode
Made-with: Cursor
* refactor: revert policy changes, keep env-only camera mapping fixes
- Revert GR00T N1.5 default_factory/default changes (transformers compat)
- Revert SmolVLA use_peft legacy field
- Apply ruff formatting fixes
- camera_name_mapping stays entirely in env/eval layer (no policy changes)
Made-with: Cursor
* Update docs/source/env_processor.mdx
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* feat(envs): lazy env init + AsyncVectorEnv as default for n_envs > 1
LiberoEnv and MetaworldEnv previously allocated GPU resources (EGL context,
OpenGL framebuffer) in __init__, before AsyncVectorEnv's fork(). Worker
processes inherited stale GPU handles, causing EGL_BAD_CONTEXT crashes on
first render.
Fix: defer OffScreenRenderEnv / MT1 construction to _ensure_env(), called on
first reset() or step() inside the worker subprocess. Each worker creates its
own clean context after fork().
Also fixes lerobot_eval.py:170 (add_envs_task TODO): replace with
env.call("task") which works with both SyncVectorEnv and AsyncVectorEnv.
AsyncVectorEnv is now the default for n_envs > 1; auto-downgraded to
SyncVectorEnv when n_envs=1 (no benefit, less overhead).
Expected speedup: ~15-20x for LIBERO Spatial with batch_size=50.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: close envs between tasks to prevent worker process accumulation
eval_policy_all never closed environments after each task completed,
causing AsyncVectorEnv worker processes to accumulate (N_tasks × n_envs).
This led to OOM, BrokenPipeError and EOFError on multi-task benchmarks.
Also fixes:
- AsyncVectorEnv compat in envs/utils.py (use get_attr/call instead of .envs)
- Tuple task handling in tokenizer_processor and lerobot_eval
- _LazyAsyncVectorEnv for deferred worker spawning in LIBERO
Made-with: Cursor
* fix(eval): use task_description instead of task for language conditioning
env.call("task") returns the LIBERO task name with underscores
(e.g. "pick_up_the_black_bowl_...") instead of the natural language
description ("pick up the black bowl ..."). The VLM tokenizes these
completely differently, causing 0.0 reward across all episodes.
Made-with: Cursor
* docs: update adding_benchmarks for async env changes
- Replace add_envs_task reference with env.call("task_description")
- Update use_async_envs default to True
- Add note about lazy GPU init for AsyncVectorEnv compatibility
Made-with: Cursor
* feat(eval): batch_size=auto + faster env loading
- batch_size=0 (default) auto-tunes based on CPU cores, capped by
n_episodes and 64. Removes the need for users to guess the right
value. The old batch_size > n_episodes error is replaced by silently
clamping to n_episodes.
- _LazyAsyncVectorEnv accepts pre-computed spaces so only one temp env
is created per suite (not per task). For libero_spatial (10 tasks)
this avoids 9 redundant LiberoEnv instantiations during env setup.
Made-with: Cursor
* docs: add evaluation guide and update benchmarks doc
- New docs/source/evaluation.mdx covering lerobot-eval usage, batch_size
auto-tuning, AsyncVectorEnv performance, tuning tips, output format,
multi-task evaluation, and programmatic usage.
- Add evaluation page to _toctree.yml under Benchmarks section.
- Update adding_benchmarks.mdx to reference batch_size auto default and
link to the evaluation guide.
Made-with: Cursor
* docs(evaluation): remove benchmark table, rename section header
Made-with: Cursor
* perf(eval): shared memory, observation passthrough, task prefetch
- AsyncVectorEnv now uses shared_memory=True for zero-copy observation transfer
- LiberoEnvConfig.gym_kwargs passes observation_height/width to the env
- eval_policy_all prefetches next task's workers while current task runs
Made-with: Cursor
* style: ruff format
Made-with: Cursor
* chore: revert env_processor.mdx changes (not part of this PR)
Made-with: Cursor
* ci(benchmarks): add isolated integration tests for libero and metaworld
Each benchmark gets its own Docker image (lerobot[libero] / lerobot[metaworld]
only) so incompatible dep trees cannot collide. A 1-episode smoke eval runs
per benchmark on GPU runners.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci(benchmarks): pin action hashes and use uv sync --locked
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci(benchmarks): trigger only on envs/ or lerobot_eval.py changes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): set LIBERO_DATA_FOLDER to bypass interactive stdin prompt
libero/__init__.py calls input() to ask about a custom dataset path,
which raises EOFError when stdin is closed inside Docker. Setting
LIBERO_DATA_FOLDER skips the prompt entirely.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(benchmarks): add CI smoke test step to adding_benchmarks guide
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): pre-create libero config in Dockerfile to bypass stdin prompt
libero/__init__.py calls input() when ~/.libero/config.yaml is missing.
We write the config at image build time (without importing libero) so
the prompt never fires at runtime. Also trigger CI on pyproject.toml changes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): use shell to create libero config instead of multiline python -c
The multiline RUN python -c "..." was being parsed as Dockerfile
instructions. Use printf to write ~/.libero/config.yaml directly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): point libero config to bundled package init_files
The config was pointing to /tmp/libero_init which doesn't exist.
Use importlib.util.find_spec to locate the hf-libero package directory
and write paths to the actual bundled bddl_files/init_files/assets.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): add smolvla extra to benchmark Dockerfiles
num2words (required by SmolVLM processor) is declared in lerobot[smolvla],
not lerobot[libero/metaworld]. Install both extras together.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(eval): render_frame covers _LazyAsyncVectorEnv
isinstance(env, AsyncVectorEnv) silently skipped _LazyAsyncVectorEnv,
causing video rendering to produce no frames on the default async path.
Switch to hasattr(env, "call") so any async-compatible env (including
_LazyAsyncVectorEnv) hits the call("render") branch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(envs): remove unused _get_sub_env_attr helper
_get_sub_env_attr was defined but never called anywhere in the codebase.
_sub_env_has_attr (its sibling) is kept — it is actively used in utils.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: apply prettier formatting to docs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(env_processor): remove deprecated add_envs_task from pipeline example
add_envs_task is replaced by env.call("task_description") in this PR.
Remove it from the pipeline walkthrough and renumber the steps (8→7).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(envs): remove __del__ from _LazyAsyncVectorEnv
__del__ is unreliable as a cleanup mechanism. close() is already called
explicitly in the eval loop's finally block, so the finalizer is redundant.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(eval): prefetch next task's workers after close to avoid GPU memory overlap
Previously, next task's AsyncVectorEnv workers were spawned while the
current task was still running, causing both tasks' GPU contexts to coexist.
Moving the prefetch start into the finally block (after env.close()) ensures
workers for task N+1 only spin up once task N has released GPU memory.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(envs): move _LazyAsyncVectorEnv to utils and apply to metaworld
_LazyAsyncVectorEnv lived in libero.py but metaworld had the same OOM
problem: all tasks' AsyncVectorEnv workers were spawned eagerly, wasting
GPU memory for tasks not yet running.
Move the class to envs/utils.py so both environments share it, then apply
the same is_async + lazy wrapping pattern in create_metaworld_envs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: remove out-of-scope benchmark/CI/docs files from PR
Benchmark CI workflow, Dockerfiles, benchmark docs, evaluation smoke-test
doc, and dispatch tests belong in a separate PR. Scope this PR to the
async env init changes only.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: restore adding_benchmarks + test_dispatch, drop env_processor changes
- Restore docs/source/adding_benchmarks.mdx (belongs in this PR)
- Restore tests/envs/test_dispatch.py (belongs in this PR)
- Revert docs/source/env_processor.mdx to main (out of scope for this PR)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(adding_benchmarks): remove CI smoke test step (coming in separate PR)
Step 7 (Dockerfile + benchmark_tests.yml CI job) and its table rows are
out of scope for this PR. The CI infrastructure will be added on top in a
follow-up PR.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(envs): remove unused add_envs_task
Replaced by env.call("task_description") in lerobot_eval.py. No callers
remain in the codebase.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: fix prettier formatting in env_processor.mdx
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(eval): catch AttributeError and NotImplementedError explicitly for task description
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(envs): use forkserver context and close envs in test to prevent deadlock
AsyncVectorEnv with default fork context leaks worker processes between
test_policy parametrized cases; subsequent env creation deadlocks because
new forked workers inherit stale pipe FDs from previous test's leaked workers.
- configs.py: pass context="forkserver" to AsyncVectorEnv (matches _LazyAsyncVectorEnv)
- test_policies.py: call close_envs(envs) at end of test_policy to clean up workers
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(envs): default use_async_envs=False in create_envs and make_env
Tests that call make_env(n_envs=2) without passing use_async_envs were
getting AsyncVectorEnv, whose forked workers can't resolve gym namespaces
registered at runtime. Default to False (sync) so existing tests pass.
lerobot_eval.py explicitly passes cfg.eval.use_async_envs, so the CLI
async behaviour (controlled by EvalConfig.use_async_envs) is unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(benchmarks): add benchmark integration guide and standardize benchmark docs
Add a comprehensive guide for adding new benchmarks to LeRobot, and
refactor the existing LIBERO and Meta-World docs to follow the new
standardized template.
* refactor(envs): move dispatch logic from factory into EnvConfig subclasses
Replace hardcoded if/elif chains in factory.py with create_envs() and
get_env_processors() methods on EnvConfig. New benchmarks now only need
to register a config subclass — no factory.py edits required.
Net -23 lines: factory.py shrinks from ~200 to ~70 lines of logic.
* docs(benchmarks): clean up adding-benchmarks guide for clarity
Rewrite for simpler language, better structure, and easier navigation.
Move quick-reference table to the top, fold eval explanation into
architecture section, condense the doc template to a bulleted outline.
* fix link
* fix task count
* fix(tests): fix 3 failing dispatch tests
- test_registry_all_types: skip non-EnvConfig stubs (e.g. TestPluginConfig)
- test_processors_delegation: use None instead of abstract PreTrainedConfig
- test_custom_get_env_processors_override: use DataProcessorPipeline for isinstance check (PolicyProcessorPipeline is a subscripted generic)
* fix: enable SmolVLA eval on LIBERO with custom camera mappings
- Thread camera_name_mapping from LiberoEnv config through to gym envs
- Sync features_map with camera_name_mapping in LiberoEnv.__post_init__
- Fix render() to use first available camera instead of hardcoded "image"
- Handle non-dict final_info in rollout by falling back to info["is_success"]
- Add use_peft legacy field to SmolVLAConfig for checkpoint compat
- Add defaults to GR00TN15Config init=False fields for transformers 5.3
Made-with: Cursor
* fix: use direct AutoresetMode import for gymnasium compat
Made-with: Cursor
* fix: handle gymnasium < 1.0 without AutoresetMode
Made-with: Cursor
* refactor: revert policy changes, keep env-only camera mapping fixes
- Revert GR00T N1.5 default_factory/default changes (transformers compat)
- Revert SmolVLA use_peft legacy field
- Apply ruff formatting fixes
- camera_name_mapping stays entirely in env/eval layer (no policy changes)
Made-with: Cursor
* Update docs/source/env_processor.mdx
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* Update docs/source/env_processor.mdx
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* Update docs/source/env_processor.mdx
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* fix(eval): raise RuntimeError for unsupported final_info format (Gymnasium < 1.0)
Made-with: Cursor
* style: fix markdown code fences in env_processor.mdx
Made-with: Cursor
* docs: remove duplicate code blocks in env_processor.mdx
Made-with: Cursor
* style: revert quadruple backticks to triple (prettier compat)
* docs(env_processor): add EnvConfig subclass step and policy_cfg examples
- Add missing '### 2. Update Your EnvConfig Subclass' section with
get_env_processors() snippet
- Update factory usage example to show policy_cfg parameter and
keyword-argument style for both SmolVLA and ACT cases
* docs(env_processor): rename step 2 and fix policy_cfg examples
- Rename '### 2. Update the Factory' → '### 2. Update Your EnvConfig Subclass'
- Update factory usage examples to use keyword-argument style with
policy_cfg parameter for both SmolVLA and ACT cases
---------
Signed-off-by: Pepijn <138571049+pkooij@users.noreply.github.com>
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
* feat: HIL data collection, RTC interpolator, and action queue improvements
- Add Human-in-the-Loop (HIL) data collection examples (sync + RTC)
- Add HIL data collection documentation
- Add ActionInterpolator for smoother policy control at higher rates
- Integrate interpolator into lerobot-record and eval_with_real_robot
- Add action queue clear() and get_processed_left_over() methods
- Add rtc/__init__.py for cleaner imports
* docs: expand Related Work section with paper summaries
* fix: only record dataset frames at original fps, not at interpolated rate
The interpolator speeds up robot control (e.g. 2x) but dataset frames
should still be recorded at the original fps. Interpolated-only
iterations now only send actions to the robot without writing to the
dataset.
* refactor: merge HIL sync and RTC scripts into single file with --rtc.enabled toggle
Combines hil_data_collection.py and hil_data_collection_rtc.py into one
script. RTC is toggled via --rtc.enabled=true (defaults to off for sync
inference). Deletes the separate hil_data_collection_rtc.py and updates
docs to reflect the single-script usage.
* test: add ActionInterpolator test suite (29 tests)
Covers constructor validation, passthrough (multiplier=1), 2x and 3x
interpolation with exact value checks, reset/episode boundaries,
control interval calculation, multi-dim actions, and simulated
control loop integration.
* test: add ActionQueue + ActionInterpolator integration tests
Verifies the interpolator doesn't interfere with RTC's leftover chunk
tracking: queue consumption rate matches base fps regardless of
multiplier, get_left_over/get_processed_left_over only change on
queue.get(), merge preserves smooth interpolation across chunks,
and interpolator reset is independent of queue state.
* feat: register SO follower/leader configs in HIL script
Adds SOFollowerRobotConfig and SOLeaderTeleopConfig imports so
SO100/SO101 robots can be used via --robot.type=so_follower
and --teleop.type=so_leader. Updates docs accordingly.
Made-with: Cursor
* docs: remove em dashes from HIL documentation
Made-with: Cursor
* refactor: rename examples/rac to examples/hil
Updates directory name and all references in docs and script docstrings.
Made-with: Cursor
* fix: encorperate pr feedback comments
* refactor(tests): enhance ActionInterpolator test structure and add detailed docstrings
* feedback pr and test fix
* fix(test): pass correct real_delay in interpolator delay test
The test was passing real_delay=0 and relying on _check_delays to
silently override it with the index-based diff. Now passes real_delay=3
to match the 3 actions consumed during the simulated inference period.
* fix pr feedback
* ordering
* update hil script
* fix
* default name
* fix(bi_openarm): use kw_only=True to fix dataclass field ordering
BiOpenArmFollowerConfig overrides `id` with a default, making it
positional in the child — non-default `left_arm_config` then follows a
default field, which Python dataclasses forbid. Adding kw_only=True
(matching the parent RobotConfig) removes positional constraints.
Made-with: Cursor
* style: format long line in hil_data_collection.py
Made-with: Cursor
* pr feedback
---------
Co-authored-by: Khalil Meftah <khalil.meftah@huggingface.co>
* Add option for pi family models to train with relative actions (relative to state)
* formatting
* add recomputation of stats and option to compute delta stats
* normalzie after delta conversion
* only recompute state for stats
* calulate chunk based stats
* sample 100k
* load from parquet
* sample 1m
* stats per chunck
* fix
* use quantiles
* stats for entire dataset
* fix
* max 1m frames
* compute before dist
* fix multi gpu processor bug
* Fix RTC with delta actions and OpenArms motor_type wiring
* feat: align pi0_fast delta actions with pi0/pi05 and add RTC integration tests
- Add delta_exclude_joints and action_feature_names to PI0FastConfig
- Move to_absolute_actions from modeling to processor pipeline for pi0_fast
- Add delta action detection and logging to eval_with_real_robot.py
- Add delta actions documentation to pi0 and pi05 READMEs
- Fix ruff lint issues in test_delta_actions.py
- Add test_rtc_delta_actions.py (24 tests) covering:
- ActionQueue with delta vs absolute actions
- RTC denoise step with delta leftovers
- Full pipeline roundtrip (delta → RTC → absolute)
- State rebasing approximation bounds
- Non-delta policy compatibility
- Multi-chunk consistency
* chore: clean up test comments, add OpenPI attribution, remove debug logging
- Replace decorative comment separators in test files with plain section headers
- Add attribution comments for 1e-6 epsilon in normalize_processor.py (from OpenPI)
- Remove debug logging blocks from lerobot_train.py
* refactor: extract compute_delta_action_stats into compute_stats.py
Move the ~70-line inline delta action stats block from lerobot_train.py
into a dedicated function in compute_stats.py, where all other stats
computation already lives. The training script now calls it in 6 lines.
* refactor: remove unused get_processed_left_over from ActionQueue
This method was never called outside of tests. Leftover actions for RTC
guidance are always retrieved via get_left_over() (delta/original space).
* revert: remove logging-only changes from eval_with_real_robot.py
The delta actions detection helper and log message added no functional
value — the script already handles delta policies correctly via the
processor pipeline.
* refactor: use ACTION/OBS_STATE constants instead of hardcoded strings
Replace hardcoded "action" and "observation.state" with ACTION and
OBS_STATE from utils.constants in compute_stats.py, dataset_tools.py,
and lerobot_train.py.
* style: remove stray blank lines in training loop
* refactor: move delta action stats to preprocessing step, remove on-the-fly computation
- Remove on-the-fly compute_delta_action_stats from lerobot_train.py
- Rewrite recompute_stats to delegate action stats to compute_delta_action_stats
(chunk-based sampling matching what the model sees during training)
- Add chunk_size parameter to recompute_stats for delta action computation
- Add delta actions documentation to pi0.mdx and pi05.mdx
* feat: add recompute_stats CLI operation to lerobot-edit-dataset
* fix(tests): relax quantile normalization test tolerance for 1e-6 epsilon
* chore: remove agents_memory/pr_details.md from repo
* refactor: rename delta actions to relative actions throughout
What OpenPI calls "DeltaActions" is actually UMI's "relative trajectory"
representation: each action in the chunk is an offset from the current
state, not from the previous action. This avoids error accumulation.
Renamed across all source, tests, docs, and CLI:
- DeltaActionsProcessorStep → RelativeActionsProcessorStep
- to_delta_actions → to_relative_actions
- use_delta_actions → use_relative_actions
- delta_exclude_joints → relative_exclude_joints
- compute_delta_action_stats → compute_relative_action_stats
- delta_action_processor.py → relative_action_processor.py
- test_delta_actions.py → test_relative_actions.py
Kept as-is: AbsoluteActionsProcessorStep (converts TO absolute),
registry ID "delta_actions_processor" (backward compat), and unrelated
delta references (IK pipeline, Robosuite, RA-BC metrics, gym envs).
* docs: add Action Representations guide
Dedicated page explaining absolute, relative, and delta actions with
numerical examples, joint vs EE space, and how to use kinematics
pipelines and the relative action processor. References UMI paper
(Chi et al., 2024) for the terminology.
* docs: remove redundant OpenPI naming note from action representations
* docs: remove opinionated OpenPI reference from delta actions section
* docs: replace ASCII diagram with UMI paper figure
* docs: remove OpenPI reference from action representations
* docs: use HF-hosted image instead of local asset
* docs: clarify figure attribution
* revert: restore original normalization epsilon behavior
The 1e-6 unconditional epsilon change perturbed all normalized values,
breaking backward compatibility tests. The original approach (1e-8 eps
for MEAN_STD, conditional torch.where for QUANTILES) already handles
division by zero correctly without affecting non-degenerate cases.
* fix: restore delta_action_processor.py used by phone/RL teleop
The rename commit incorrectly deleted delta_action_processor.py and
duplicated its classes into relative_action_processor.py. Restore the
original file and import from it instead.
* fix(processor): address PR #2970 review comments
- Remove shebang from relative_action_processor.py (library module, not script)
- Add device alignment in to_relative_actions/to_absolute_actions so _last_state
on CPU doesn't cause cross-device errors when actions are on CUDA
- Rename delta_step → relative_step in AbsoluteActionsProcessorStep for naming
consistency; update factory.py, all processor files, and tests
- Expand _reconnect_relative_absolute_steps docstring to explain why post-hoc
rewiring is needed after deserialization
- Fix off-by-one in compute_stats.py: sample_upper_bound = total_frames - chunk_size + 1
so last valid start index is included and total_frames == chunk_size is not rejected
- Remove redundant NOTE comment in processor_pi05.py (duplicated two lines below)
- Fix pi0_fast processor ordering: move relative_step before NormalizerProcessorStep
so normalizer sees delta actions (matching pi0/pi05); flip postprocessor to
unnormalize → absolute accordingly. Relative stats are now required for all pi models
- Revert use_relative_joint_actions_aloha → use_delta_joint_actions_aloha in
configuration_smolvla.py (preserve existing public API)
- Update action_representations.mdx: add missing joint to 6-DOF example, fix
'based on a figure', clarify pi family ordering, add RTC compatibility section
* update rtc link
* feat: compute relative action stats over full dataset with optional parallelism
Remove the 100k sample cap from compute_relative_action_stats and process
all valid chunks. Vectorize with numpy (pre-load actions/states, fancy
indexing + broadcasting) for a large speedup over the per-index HF dataset
loop. Add num_workers param for thread-based parallelism (numpy releases
the GIL). Update docs to show --push_to_hub for recompute_stats.
* style: apply ruff formatting to compute_stats.py
* testing on real robot
* style: fix ruff format and remove redundant .keys() calls
* refactor(dataset): split reader and writer
* chore(dataset): remove proxys
* refactor(dataset): better reader & writer encapsulation
* refactor(datasets): clean API + reduce leaky implementations
* refactor(dataset): API cleaning for writer, reader and meta
* refactor(dataset): expose writer & reader + other minor improvements
* refactor(dataset): improve teardown routine
* refactor(dataset): add hf_dataset property at the facade level
* chore(dataset): add init for datasset module
* docs(dataset): add docstrings for public API of the dataset classes
* tests(dataset): add tests for new classes
* fix(dataset): remove circular dependecy
Add a `cudnn_deterministic` flag to `TrainPipelineConfig` (default: False)
that sets `torch.backends.cudnn.deterministic = True` and disables benchmark
mode, eliminating CUDA floating-point non-determinism at the cost of ~10-20%
training speed. When False (default) the existing benchmark=True behaviour
is preserved.
* fix(root): adding proper support for the root and new_root arguments
* feat(roots): adding a roots agrument for the merge operation
* chore(clean): cleaning up code
* chore(doctrings): updating doctrings with new features
* fix(repo_id): setting repo_id to None when not needed
* fix(roots/repo_ids): making mypy happy by using repo_ids and roots for merge operation
* fix(path): fixing path related issues
* fix(repo_id): fixing issues related to repo_id
* chore(doctrings): updating docstrings + fix typo
* chore(clean): cleaning code
* fix(split new_repo_id): reverting new_repo_id addition for split operation
* docs(dosctrings): completing docstrings
* fix(repo_ids/roots): improving checks for repo_ids/roots lengths
* fix(repo_ids): making repo_ids optional in MergeConfig but raise if not given
* fix(docstrings): fixing docstrings for split operation
* fix(hints): updating get_output_path hints to accept paths as strings too
* fix(y/N prompts): removing y/N prompts in lerobot_edit_dataset
* fix(merge repo_id): fixing merge operation to use new_repo_id instead of repo_id
* fix(typo): fixing typo in doctrings
* fix(frame_index): making rerun's "frame_index" timeline compatible with behaviour1k datasets
* fix(segfault risk): removing segfault risk by calling batch["index"] in the dataloader loop
* add OpenArm Mini config and module init
* add OpenArm Mini teleoperator implementation
* add OpenArm Mini into factory and setup motors
---------
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* feat(motors): add initial implementation of robstride
Co-authored-by: Virgile <virgilebatto@gmail.com>
* chore(motors): solve some linter
* remove kp/kd attribute
* code uniformisation between damiao and robstride
* remove normalization warning
* remove non valid baudrates and small docstring update
* remove all useless files. Only keeping robstride.py and table.py
* typing for mypy
* reduce NameOrId usage
* align signature with damiao
* put the same helper than in the damiao implementation
* bug correction : expect a response after each bus.send
---------
Co-authored-by: Virgile <virgilebatto@gmail.com>
* Add New featrue to lerobot_edit_datset.py that show dataset information.
* Fix to draccus error when happen give only --operation.type=info
* Updating test and documents regarding lerobot-edit-dataset info function.
* Updating documents regarding lerobot-edit-dataset extract function. option name in document is mistake.
* feat(datasets): Update to align formatting with pre-commit.(#2917)
Update to align formatting by pre-commit.
---------
Co-authored-by: Caroline Pascal <caroline8.pascal@gmail.com>
* feat(datasets): add modify_tasks function for in-place task editing
Add a new utility function to modify tasks in LeRobotDataset in-place.
This allows users to:
- Set a single task for all episodes
- Set specific tasks for individual episodes
- Combine a default task with per-episode overrides
* feat(edit-dataset): add CLI support for modify_tasks operation
Integrate the modify_tasks function into lerobot_edit_dataset CLI.
Users can now modify dataset tasks via command line:
Supports setting a default task, per-episode tasks, or both combined.
* test(datasets): add tests for modify_tasks function
Add comprehensive test coverage for the modify_tasks utility:
- Single task for all episodes
- Episode-specific task assignment
- Default task with per-episode overrides
- Error handling for missing/invalid arguments
- Verification of task_index correctness
- In-place modification behavior
- Metadata preservation
* respond to copilot review
* feat(cameras): add new read_latest() method
* fix(cameras): fix threading bug + clear state
* refactor(cameras): multiple improvements
* feat(camera): add context manager to camera base class
* chore(camera): slight modifications to opencv
* test(cameras): update opencv tests according to the changes
* refactor(cameras): reflect desing changes to realsense + deal with depth
* test(cameras): fix realsense tests accordingly to new changes
* refactor(cameras): update reachymini and zmq accordingly
* chore: wrap resource sensitive examples into a try/finally
* test(cameras): add test for new read_latest
* test(cameras): fix problem with image artifact in opencv tests
* test(cameras): fix test_read_latest_high_frequency expectations
* Apply suggestions from code review 1
Co-authored-by: Caroline Pascal <caroline8.pascal@gmail.com>
Signed-off-by: Steven Palma <imstevenpmwork@ieee.org>
* chore(cameras): address feedback
* feat(cameras): add max_age_ms check in read_latest
* test(cameras): fix read_latest tests
* chore(redundancies): removing redundancies in Reachy 2 camera class
* fix(warmup): replacing the arbitrary time.sleep in by an actual warmup in the RealSense camera class
* chore(format): formatting latest changes
* chore(warning): adding a "to be implemented" warning for read_latest() in Camera base class
* chore(warning): making read_latest() warning message shorter and clearer
---------
Signed-off-by: Steven Palma <imstevenpmwork@ieee.org>
Co-authored-by: Caroline Pascal <caroline8.pascal@gmail.com>
* fix(motors): cleanup imports + fix signatures
* feat(motors): add damiao canbus + multiple fixes
* fix(motors): address comments -> last_state + different gains + sleep
* refactor(motors): reduce duplicated code + adressed some comments in the PR
* chore(motors): better timeouts
* tests(motors): damiao test and imports
* chore(deps): fix space
* feat(robot): add openarm leader
Co-authored-by: Pepijn <pepijn@huggingface.co>
* feat(robot): add openarm follower
Co-authored-by: Pepijn <pepijn@huggingface.co>
* refactor(robot): remove mechanical compensations and double arm assumption + rename
* chore(robots): remove left arm references
* refactor(teleop): multiple improvements to leader
* refactor(teleop): multiple improvements to leader
* feat(robots): add open arm to util CLI
* chore(robot): add alias openarm
* Apply suggestions from code review
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
Signed-off-by: Steven Palma <imstevenpmwork@ieee.org>
* chore(motors): remove normalization tables damiao
* fix(motors): imports and signatures
* feat(motors): add motor_type_str + recv_id to motor class and _get_motor_recv_id raises if no motor_obj.recv_id
* chore(motors): remove normalize from base motor class and damaio
* tests(motors): remove bad tests (to be replaced)
* chore(motors): updated import check
* fix(robots): open arm mirrored config for joint limits
* chore(motors): update position_kd gain values
* chore(robots): set to 0 if openarm is calibrated at connect time
* chore(robots): remove macos in open arm as can doesn't support it
* chore(robots): update for motor_type_str in Motor class
* chore(robots): no default value for can port in open arms
* feat(robots): add bi manual openarm follower and leader
* use constant for kp and kd range and check responses in mit_control_batch()
* Add docs on setting up canbus and use damiao otor bus, also add lerobot_setup_can.py and log if there is not response from a write command
* precommit format
* supress bandit as these are intentional cli commands
* fix setup-can
* add test
* skip test in ci
* nit precommit
* update doc example
* dont import can for tests
* remove comment
* Add openarms docs
* format
* update purchase link
* can to none if nit availabl;e
* add canfd option in bus
* make handshake logic similar to lerobot-can
* type hint
* type check
* add temp teleop test
* remove script
* mock class
* mock class
* ignore linter
* pre-commit
* Add command for bimanual openarm
* fix import
* fix import leader
* fix import draccus
---------
Signed-off-by: Steven Palma <imstevenpmwork@ieee.org>
Co-authored-by: Pepijn <pepijn@huggingface.co>
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* fix(motors): cleanup imports + fix signatures
* feat(motors): add damiao canbus + multiple fixes
* fix(motors): address comments -> last_state + different gains + sleep
* refactor(motors): reduce duplicated code + adressed some comments in the PR
* chore(motors): better timeouts
* tests(motors): damiao test and imports
* chore(deps): fix space
* feat(robot): add openarm leader
Co-authored-by: Pepijn <pepijn@huggingface.co>
* feat(robot): add openarm follower
Co-authored-by: Pepijn <pepijn@huggingface.co>
* refactor(robot): remove mechanical compensations and double arm assumption + rename
* chore(robots): remove left arm references
* refactor(teleop): multiple improvements to leader
* refactor(teleop): multiple improvements to leader
* feat(robots): add open arm to util CLI
* chore(robot): add alias openarm
* Apply suggestions from code review
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
Signed-off-by: Steven Palma <imstevenpmwork@ieee.org>
* chore(motors): remove normalization tables damiao
* fix(motors): imports and signatures
* feat(motors): add motor_type_str + recv_id to motor class and _get_motor_recv_id raises if no motor_obj.recv_id
* chore(motors): remove normalize from base motor class and damaio
* tests(motors): remove bad tests (to be replaced)
* chore(motors): updated import check
* fix(robots): open arm mirrored config for joint limits
* chore(motors): update position_kd gain values
* chore(robots): set to 0 if openarm is calibrated at connect time
* chore(robots): remove macos in open arm as can doesn't support it
* chore(robots): update for motor_type_str in Motor class
* chore(robots): no default value for can port in open arms
* use constant for kp and kd range and check responses in mit_control_batch()
* Add docs on setting up canbus and use damiao otor bus, also add lerobot_setup_can.py and log if there is not response from a write command
* precommit format
* supress bandit as these are intentional cli commands
* fix setup-can
* add test
* skip test in ci
* nit precommit
* update doc example
* dont import can for tests
* remove comment
* Add openarms docs
* format
* update purchase link
* can to none if nit availabl;e
* add canfd option in bus
* make handshake logic similar to lerobot-can
* type hint
* type check
* add temp teleop test
* remove script
* mock class
* ignore linter
---------
Signed-off-by: Steven Palma <imstevenpmwork@ieee.org>
Co-authored-by: Pepijn <pepijn@huggingface.co>
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
* fix(motors): cleanup imports + fix signatures
* feat(motors): add damiao canbus + multiple fixes
* fix(motors): address comments -> last_state + different gains + sleep
* refactor(motors): reduce duplicated code + adressed some comments in the PR
* chore(motors): better timeouts
* tests(motors): damiao test and imports
* chore(deps): fix space
* Apply suggestions from code review
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
Signed-off-by: Steven Palma <imstevenpmwork@ieee.org>
* chore(motors): remove normalization tables damiao
* fix(motors): imports and signatures
* feat(motors): add motor_type_str + recv_id to motor class and _get_motor_recv_id raises if no motor_obj.recv_id
* chore(motors): remove normalize from base motor class and damaio
* tests(motors): remove bad tests (to be replaced)
* chore(motors): updated import check
* use constant for kp and kd range and check responses in mit_control_batch()
* Add docs on setting up canbus and use damiao otor bus, also add lerobot_setup_can.py and log if there is not response from a write command
* precommit format
* supress bandit as these are intentional cli commands
* fix setup-can
* add test
* skip test in ci
* nit precommit
* update doc example
* dont import can for tests
---------
Signed-off-by: Steven Palma <imstevenpmwork@ieee.org>
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
Co-authored-by: Pepijn <pepijn@huggingface.co>
* improve image2video
* add episodes video encoding
* fix mypy failing
* iterate on review
* nit
* remove max, and let it be optional
* iterate more
* update docs
* fix test
---------
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
* move peft config from `lerobot_train` to policy level
* Update src/lerobot/scripts/lerobot_train.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Michel Aractingi <michel.aractingi@huggingface.co>
* copilot response
* Change the polciy function to return targets rather than peft config.`_get_default_peft_targets()` override in PI0, PI0.5, SmolVLA
* remove none check when building config dict
---------
Signed-off-by: Michel Aractingi <michel.aractingi@huggingface.co>
This PR extends the integration of Unitree g1 with the LeRobot codebase. By converting robot state to a flat dict we can now record and replay episodes (example groot/holosoma scripts need to be adjusted as well). We also improve the simulation integration by calling .step @ _subscribe_motor_state instead of it running in a separate thread. We also add ZMQ camera to lerobot, streaming base64 images over json
* feat(robots): consolidates bi SO setups
* fix(robots): solve circular dependecy
* fix(robots): teleop & record working
* feat(robots): only one SO
* fix(utils): rename bi so
* fix(scripts): bi so import
* fix(rl): remove imports
