Commit Graph

3 Commits

Author SHA1 Message Date
Pepijn Kooijmans
a9e355bd03 Lazy env creation + smart sharding to fix container OOM 2026-03-23 23:15:23 +01:00
Pepijn Kooijmans
e52e7e644a fix(docker): add libero_plus install workaround to generic Dockerfile.benchmark
The generic Dockerfile.benchmark was using a plain `uv pip install ".[libero_plus]"`
which silently fails to make `libero` importable due to an upstream LIBERO-plus
packaging bug. Port the dedicated clone + .pth workaround from
Dockerfile.eval-libero-plus so `docker build --build-arg BENCHMARK=libero_plus`
produces working containers.

Also fix eval worker using nonexistent `parser.parse()` — use `draccus.parse()`.

Made-with: Cursor
2026-03-23 18:31:57 +01:00
Pepijn
3d5d8fa88a feat(eval): implement docker runtime with HTTP policy inference server
Add docker_runtime.py (host-side) and lerobot_eval_worker.py (container-side)
for --eval.runtime=docker. Policy loads once on the host GPU; Docker containers
run env-only workers that call back via HTTP for action chunks, maximising GPU
utilisation across parallel benchmark tasks.

- _InferenceServer: HTTP server wrapping predict_action_chunk with a single lock
- run_eval_in_docker: spawns instance_count containers, collects + merges per-task
  JSON, writes eval_info.json compatible with _aggregate_eval_from_per_task
- lerobot-eval-worker CLI: make_env → shard tasks → run episodes → write JSON
- EvalDockerConfig: add port field (default 50051)
- pyproject.toml: add lerobot-eval-worker entry point

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 22:35:59 -07:00