nemo
7fd8b4c773
Implement loading of PEFT adapters
...
Loading a PEFT adapter is currently done by initializing a policy with default config
and then applying the adapter on the resulting model. This has the obvious drawback
that any configurations done during training are not applied in the adapted model.
Currently the `use_peft` attribute of `PreTrainedConfig` is only set during loading
to signal the following code that it has to deal with a PEFT adapter. However
we could imagine a scenario where this is already set at training time and stored
alongside the adapter.
2025-06-22 19:10:10 +02:00
nemo
98856662c1
Add basic support for PEFT adapter methods
...
This changes adds support for training policies with much less parameters
by applying adapter methods such as LoRA on specific parts of the policies
and therefore possibly higher learning rates / batch sizes.
To make this as accessible as possible I thought it useful to provide
defaults for `target_modules` and `modules_to_save`. Currently only SmolVLA
has such defaults but when we agree that this change is useful I will set
out to generate more such defaults. While the user can override these
settings, they are expected to only change the peft_method, rank and init_type
parameters.
2025-06-22 13:45:07 +02:00
Steven Palma
c940676bdd
fix(benchmarks): remove .numpy() from frame in benchmark script ( #1354 )
2025-06-19 17:07:13 +02:00
Steven Palma
2b71789e15
docs: fix imitation learning robots docs command ( #1308 )
2025-06-15 11:47:48 +02:00
Francesco Capuano
7c8be7fb9b
bump pi0 and hil transformers version ( #1298 )
2025-06-15 08:57:08 +02:00
koenvanwijk
b8637c09ec
Update lekiwi.mdx ( #1229 )
2025-06-14 23:41:45 +02:00
David
1688fa3a88
(chore): incorrect resume parameter in recording documentation ( #1301 )
2025-06-14 23:38:10 +02:00
Michel Aractingi
b852d15774
gym_manipulator.py Remove None value action_intervention of BaseLeaderTeleoperator (#1299 )
2025-06-14 20:53:40 +02:00
Francesco Capuano
ce6a26deeb
Fixing PI0 Policy ( #1297 )
2025-06-14 19:25:50 +02:00
Michel Aractingi
697c76f75e
learner.py import so101_leader instead of so100 (#1295 )
...
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
2025-06-14 15:30:19 +02:00
Steven Palma
8d7969e7cb
fix(record): no teleop arg in reset environment ( #1294 )
2025-06-14 14:23:07 +02:00
tidely
dcc0c234dd
Improve type hints ( #1293 )
2025-06-14 14:06:22 +02:00
Michel Aractingi
6007a221f0
Add keyboard teleop device to control the end effector robot ( #1289 )
2025-06-14 09:10:09 +02:00
Simon Alibert
35e67585bf
Fixes on robot integration tutorial ( #1290 )
2025-06-14 01:47:22 +02:00
Pepijn
438334d58e
Add sim tutorial, fix lekiwi motor config, add notebook links ( #1275 )
...
Co-authored-by: AdilZouitine <adilzouitinegm@gmail.com >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Michel Aractingi <michel.aractingi@gmail.com >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
2025-06-13 18:48:39 +02:00
Steven Palma
69e8946480
fix(docs): update send_feedback docstrings
2025-06-13 18:29:19 +02:00
Simon Alibert
96fa48b5ec
Robot integration tutorial ( #1285 )
2025-06-13 18:23:07 +02:00
Adil Zouitine
8fc18be065
chore(dependencies): add gamepad support with pygame and hidapi ( #1287 )
2025-06-13 17:07:11 +02:00
Steven Palma
5350a02dc1
chore(teleop): print calibration path saved ( #1286 )
2025-06-13 15:29:10 +02:00
Dana Aubakirova
58afa2fbb0
fix(docs): SmolVLA fine-tuning getting started ( #1201 )
...
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Francesco Capuano <francesco_capuano@aol.com >
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
2025-06-13 14:17:59 +02:00
Adil Zouitine
d8079587a2
Port HIL SERL ( #644 )
...
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co >
Co-authored-by: Eugene Mironov <helper2424@gmail.com >
Co-authored-by: s1lent4gnt <kmeftah.khalil@gmail.com >
Co-authored-by: Ke Wang <superwk1017@gmail.com >
Co-authored-by: Yoel Chornton <yoel.chornton@gmail.com >
Co-authored-by: imstevenpmwork <steven.palma@huggingface.co >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
2025-06-13 13:15:47 +02:00
Steven Palma
f976935ba1
fix(record): no teleop needed when running with policy ( #1284 )
2025-06-13 12:41:30 +02:00
Simon Alibert
5c87365cc1
Skip normalization parameters in load_smolvla ( #1274 )
2025-06-13 11:06:45 +02:00
Quentin Gallouédec
edfebd522c
Use HF Papers ( #1120 )
2025-06-12 09:58:59 +02:00
Steven Palma
2de93a8000
fix(docs): update realsense documentation ( #1268 )
2025-06-11 23:16:37 +02:00
Dana Aubakirova
d0521189b1
fix issues: checkpoints keys mismatch and 'task' tokenisation in smolvla ( #1256 )
...
Co-authored-by: danaaubakirova <d.aubakirova@alumni.edu.kz >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
2025-06-11 16:56:55 +02:00
Pepijn
10b7b35325
Match motor names with ids lekiwi ( #1261 )
2025-06-11 14:21:30 +02:00
Yushun Xiang
459c95197b
fix: update pi0 dependency version constraint ( #1247 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-06-10 18:46:41 +02:00
koenvanwijk
37748c83ca
Proposal for fix for enter_pressed on Windows ( #1230 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
2025-06-10 18:36:02 +02:00
pre-commit-ci[bot]
3fb04efec1
[pre-commit.ci] pre-commit autoupdate ( #1185 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com >
2025-06-10 18:04:09 +02:00
Sarunas Kalade
2889f3a06a
update KochFollower.get_observation() so it returns same observation structure as SO101 ( #1248 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-06-10 12:42:54 +02:00
Daisuke Sato
f5335fe696
Update tutorial link ( #1250 )
2025-06-10 11:05:08 +02:00
Ben Zhang
f0a903c98f
Fix unable to set camera width/height to non-default ( #1225 )
2025-06-10 10:23:33 +02:00
mshukor
0e7caae714
Update SmolVLA README.md ( #1228 )
2025-06-08 23:15:26 +02:00
Caroline Pascal
1ee2ca5c26
fix(pyserial): adding pyserial dependency to global ones ( #1219 )
2025-06-06 14:38:33 +02:00
Simon Alibert
4e4eec92dc
Fix smolVLA dependencies ( #1218 )
2025-06-06 11:28:47 +02:00
Simon Alibert
95df341b4f
Fix LeKiwi example ( #1217 )
2025-06-06 10:08:03 +02:00
Simon Alibert
9e6f49f507
Fix test_teleoperate ( #1216 )
2025-06-06 09:38:37 +02:00
Dhruva
a28f02ecb3
replaced OBS_ROBOT with OBS_STATE constant ( #1211 )
2025-06-06 09:25:51 +02:00
Steven Palma
09343acce7
fix(smolvla): update record.py, fix populate_queues and remove unused dependencies ( #1208 )
2025-06-06 09:17:02 +02:00
Simon Alibert
e23b41e79a
Hardware API redesign ( #777 )
...
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co >
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com >
Co-authored-by: Pepijn <pepijn@huggingface.co >
2025-06-05 17:48:43 +02:00
Ben Zhang
b536f47e3f
Fix SmolVLA loss not sent to wandb ( #1198 )
2025-06-05 11:13:03 +02:00
mshukor
bfd26eef5a
Add SmolVLA ( #1175 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: fracapuano <francesco.capuano@huggingface.co >
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org >
Co-authored-by: Dana Aubakirova <118912928+danaaubakirova@users.noreply.github.com >
Co-authored-by: Remi <remi.cadene@huggingface.co >
2025-06-03 17:11:50 +02:00
pre-commit-ci[bot]
1537d0ab90
[pre-commit.ci] pre-commit autoupdate ( #1048 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Simon Alibert <simon.alibert@huggingface.co >
2025-06-02 19:30:39 +02:00
Adil Zouitine
2be7f3a3ff
(hotfix): nightly CI by clipping pymunk version below 7.0.0 ( #1182 )
2025-06-02 13:18:02 +02:00
Adil Zouitine
0cf864870c
[Fix] Unpin torch beyond 2.6.0 & torchcodec beyond 0.2.1 ( #1127 )
2025-05-28 16:54:20 +02:00
mshukor
1786916a16
Update README.md ( #1163 )
2025-05-27 11:50:43 +02:00
mshukor
0507ad4f68
Update README.md ( #1160 )
2025-05-27 11:45:07 +02:00
Ragnar
bed90e3a41
fix: typos and grammar ( #1148 )
2025-05-25 17:20:45 +02:00
Francesco Capuano
6163daaaa4
Fix: emptying action queue between resets ( #1117 )
2025-05-22 21:37:21 +02:00
