diff --git a/README.md b/README.md index 9c39b910a..a654676e6 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,95 @@ +--- + +🤗 LeRobot aims to provide models, datasets, and tools for real-world robotics in PyTorch. The goal is to lower the barrier to entry to robotics so that everyone can contribute and benefit from sharing datasets and pretrained models. + +🤗 LeRobot contains state-of-the-art approaches that have been shown to transfer to the real-world with a focus on imitation learning and reinforcement learning. + +🤗 LeRobot already provides a set of pretrained models, datasets with human collected demonstrations, and simulation environments to get started without assembling a robot. In the coming weeks, the plan is to add more and more support for real-world robotics on the most affordable and capable robots out there. + +🤗 LeRobot hosts pretrained models and datasets on this Hugging Face community page: [huggingface.co/lerobot](https://huggingface.co/lerobot) + +#### Examples of pretrained models on simulation environments + + + + + + + + + + + + +
ACT policy on ALOHA envTDMPC policy on SimXArm envDiffusion policy on PushT env
ACT policy on ALOHA envTDMPC policy on SimXArm envDiffusion policy on PushT env
+ +## Installation + +Download our source code: + +```bash +git clone https://github.com/huggingface/lerobot.git +cd lerobot +``` + +LeRobot works with Python 3.10+ and PyTorch 2.2+. +Create a virtual environment with Python 3.10 and activate it, e.g. with [`miniconda`](https://docs.anaconda.com/free/miniconda/index.html): + +```bash +conda create -y -n lerobot python=3.10 +conda activate lerobot +``` + +When using `miniconda`, install `ffmpeg` in your environment: + +```bash +conda install ffmpeg -c conda-forge +``` + +> **NOTE:** This usually installs `ffmpeg 7.X` for your platform compiled with the `libsvtav1` encoder. If `libsvtav1` is not supported (check supported encoders with `ffmpeg -encoders`), you can: +> +> - _[On any platform]_ Explicitly install `ffmpeg 7.X` using: +> +> ```bash +> conda install ffmpeg=7.1.1 -c conda-forge +> ``` +> +> - _[On Linux only]_ Install [ffmpeg build dependencies](https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu#GettheDependencies) and [compile ffmpeg from source with libsvtav1](https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu#libsvtav1), and make sure you use the corresponding ffmpeg binary to your install with `which ffmpeg`. + +Install 🤗 LeRobot: + +```bash +pip install lerobot +``` + +> **NOTE:** If you encounter build errors, you may need to install additional dependencies (`cmake`, `build-essential`, and `ffmpeg libs`). On Linux, run: +> `sudo apt-get install cmake build-essential python3-dev pkg-config libavformat-dev libavcodec-dev libavdevice-dev libavutil-dev libswscale-dev libswresample-dev libavfilter-dev`. For other systems, see: [Compiling PyAV](https://pyav.org/docs/develop/overview/installation.html#bring-your-own-ffmpeg) + +For simulations, 🤗 LeRobot comes with gymnasium environments that can be installed as extras: + +- [aloha](https://github.com/huggingface/gym-aloha) +- [xarm](https://github.com/huggingface/gym-xarm) +- [pusht](https://github.com/huggingface/gym-pusht) + +For instance, to install 🤗 LeRobot with aloha and pusht, use: + +```bash +pip install -e ".[aloha, pusht]" +``` + +To use [Weights and Biases](https://docs.wandb.ai/quickstart) for experiment tracking, log in with + +```bash +wandb login +``` + +(note: you will also need to enable WandB in the configuration. See below.) + +### Build your own robot +

Build Your Own HopeJR Robot!

@@ -85,92 +174,6 @@

LeRobot: State-of-the-art AI for real-world robotics

---- - -🤗 LeRobot aims to provide models, datasets, and tools for real-world robotics in PyTorch. The goal is to lower the barrier to entry to robotics so that everyone can contribute and benefit from sharing datasets and pretrained models. - -🤗 LeRobot contains state-of-the-art approaches that have been shown to transfer to the real-world with a focus on imitation learning and reinforcement learning. - -🤗 LeRobot already provides a set of pretrained models, datasets with human collected demonstrations, and simulation environments to get started without assembling a robot. In the coming weeks, the plan is to add more and more support for real-world robotics on the most affordable and capable robots out there. - -🤗 LeRobot hosts pretrained models and datasets on this Hugging Face community page: [huggingface.co/lerobot](https://huggingface.co/lerobot) - -#### Examples of pretrained models on simulation environments - - - - - - - - - - - - -
ACT policy on ALOHA envTDMPC policy on SimXArm envDiffusion policy on PushT env
ACT policy on ALOHA envTDMPC policy on SimXArm envDiffusion policy on PushT env
- -## Installation - -Download our source code: - -```bash -git clone https://github.com/huggingface/lerobot.git -cd lerobot -``` - -Create a virtual environment with Python 3.10 and activate it, e.g. with [`miniconda`](https://docs.anaconda.com/free/miniconda/index.html): - -```bash -conda create -y -n lerobot python=3.10 -conda activate lerobot -``` - -When using `miniconda`, install `ffmpeg` in your environment: - -```bash -conda install ffmpeg -c conda-forge -``` - -> **NOTE:** This usually installs `ffmpeg 7.X` for your platform compiled with the `libsvtav1` encoder. If `libsvtav1` is not supported (check supported encoders with `ffmpeg -encoders`), you can: -> -> - _[On any platform]_ Explicitly install `ffmpeg 7.X` using: -> -> ```bash -> conda install ffmpeg=7.1.1 -c conda-forge -> ``` -> -> - _[On Linux only]_ Install [ffmpeg build dependencies](https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu#GettheDependencies) and [compile ffmpeg from source with libsvtav1](https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu#libsvtav1), and make sure you use the corresponding ffmpeg binary to your install with `which ffmpeg`. - -Install 🤗 LeRobot: - -```bash -pip install -e . -``` - -> **NOTE:** If you encounter build errors, you may need to install additional dependencies (`cmake`, `build-essential`, and `ffmpeg libs`). On Linux, run: -> `sudo apt-get install cmake build-essential python3-dev pkg-config libavformat-dev libavcodec-dev libavdevice-dev libavutil-dev libswscale-dev libswresample-dev libavfilter-dev`. For other systems, see: [Compiling PyAV](https://pyav.org/docs/develop/overview/installation.html#bring-your-own-ffmpeg) - -For simulations, 🤗 LeRobot comes with gymnasium environments that can be installed as extras: - -- [aloha](https://github.com/huggingface/gym-aloha) -- [xarm](https://github.com/huggingface/gym-xarm) -- [pusht](https://github.com/huggingface/gym-pusht) - -For instance, to install 🤗 LeRobot with aloha and pusht, use: - -```bash -pip install -e ".[aloha, pusht]" -``` - -To use [Weights and Biases](https://docs.wandb.ai/quickstart) for experiment tracking, log in with - -```bash -wandb login -``` - -(note: you will also need to enable WandB in the configuration. See below.) - ### Visualize datasets Check out [example 1](./examples/1_load_lerobot_dataset.py) that illustrates how to use our dataset class which automatically downloads data from the Hugging Face hub.