- Updated imports in various files to include RobotAction and PolicyAction directly from the processor module, improving clarity and consistency.
- Removed redundant imports from core, streamlining the codebase and enhancing maintainability.
- Adjusted type annotations and references in the RobotProcessorPipeline and related components to align with the new import structure, ensuring better type safety and readability.
* refactor(processor): enhance type annotations for processors in record, replay, teleoperate, and control utils
- Updated type annotations for preprocessor and postprocessor parameters in record_loop and predict_action functions to specify the expected dictionary types.
- Adjusted robot_action_processor type in ReplayConfig and TeleoperateConfig to improve clarity and maintainability.
- Ensured consistency in type definitions across multiple files, enhancing overall code readability.
* refactor(processor): enhance type annotations for RobotProcessorPipeline in various files
- Updated type annotations for RobotProcessorPipeline instances in evaluate.py, record.py, replay.py, teleoperate.py, and other related files to specify input and output types more clearly.
- Introduced new type conversions for PolicyAction and EnvTransition to improve type safety and maintainability across the processing pipelines.
- Ensured consistency in type definitions, enhancing overall code readability and reducing potential runtime errors.
* refactor(processor): update transition handling in processors to use transition_to_batch
- Replaced direct transition handling with transition_to_batch in various processor tests and implementations to ensure consistent batching of input data.
- Updated assertions in tests to reflect changes in data structure, enhancing clarity and maintainability.
- Improved overall code readability by standardizing the way transitions are processed across different processor types.
* refactor(tests): standardize transition key usage in processor tests
- Updated assertions in processor test files to utilize the TransitionKey for action references, enhancing consistency across tests.
- Replaced direct string references with TransitionKey constants for improved readability and maintainability.
- Ensured that all relevant tests reflect these changes, contributing to a more uniform approach in handling transitions.
* refactor(eval): integrate preprocessor and postprocessor into rollout and eval_policy functions
- Updated the `rollout` and `eval_policy` functions to accept preprocessor and postprocessor parameters, enhancing the flexibility of the evaluation pipeline.
- Adjusted the implementation to apply preprocessing and postprocessing steps during policy evaluation, improving the overall data handling and processing flow.
* refactor(eval): remove redundant observation device conversion in rollout function
- Eliminated unnecessary device conversion for the observation dictionary within the `rollout` function, streamlining the code and enhancing readability.
- This change simplifies the observation handling process, aligning with the preference for clearer solutions.
* debug
* refactor(utils): enhance task handling in add_envs_task function
- Improved the `add_envs_task` function to validate the output of `task_description` and `task` calls, ensuring they return lists of strings.
- Removed the use of `else` statement for environments without language instructions, simplifying the logic and enhancing readability.
- Streamlined the observation dictionary handling by ensuring consistent data types for task attributes.
- Updated constant names from PREPROCESSOR_DEFAULT_NAME and POSTPROCESSOR_DEFAULT_NAME to POLICY_PREPROCESSOR_DEFAULT_NAME and POLICY_POSTPROCESSOR_DEFAULT_NAME for better context.
- Adjusted references across multiple files to use the new constant names, ensuring consistency in the codebase.
* refactor(processor): rename MapDeltaActionToRobotAction and MapTensorToDeltaActionDict for consistency
* refactor(processor): rename DeviceProcessor to DeviceProcessorStep for consistency across modules
* refactor(processor): rename Torch2NumpyActionProcessor to Torch2NumpyActionProcessorStep for consistency
* refactor(processor): rename Numpy2TorchActionProcessor to Numpy2TorchActionProcessorStep for consistency
* refactor(processor): rename AddTeleopActionAsComplimentaryData to AddTeleopActionAsComplimentaryDataStep for consistency
* refactor(processor): rename ImageCropResizeProcessor and AddTeleopEventsAsInfo for consistency
* refactor(processor): rename TimeLimitProcessor to TimeLimitProcessorStep for consistency
* refactor(processor): rename GripperPenaltyProcessor to GripperPenaltyProcessorStep for consistency
* refactor(processor): rename InterventionActionProcessor to InterventionActionProcessorStep for consistency
* refactor(processor): rename RewardClassifierProcessor to RewardClassifierProcessorStep for consistency
* refactor(processor): rename JointVelocityProcessor to JointVelocityProcessorStep for consistency
* refactor(processor): rename MotorCurrentProcessor to MotorCurrentProcessorStep for consistency
* refactor(processor): rename NormalizerProcessor and UnnormalizerProcessor to NormalizerProcessorStep and UnnormalizerProcessorStep for consistency
* refactor(processor): rename VanillaObservationProcessor to VanillaObservationProcessorStep for consistency
* refactor(processor): rename RenameProcessor to RenameProcessorStep for consistency
* refactor(processor): rename TokenizerProcessor to TokenizerProcessorStep for consistency
* refactor(processor): rename ToBatchProcessor to AddBatchDimensionProcessorStep for consistency
* refactor(processor): update config file name in test for RenameProcessorStep consistency
* refactor(processors): enhance transform_features method across multiple processors
- Updated the transform_features method in various processors to utilize a copy of the features dictionary, ensuring immutability of the original features.
- Added handling for new feature keys and removed obsolete ones in the MapTensorToDeltaActionDict, JointVelocityProcessor, and others.
- Improved readability and maintainability by following consistent patterns in feature transformation.
* refactor(processors): standardize action and observation keys in delta_action_processor and joint_observations_processor
- Updated action and observation keys to use constants for improved readability and maintainability.
- Refactored the transform_features method in multiple processors to ensure consistent handling of feature keys.
- Enhanced error handling by raising exceptions for missing required components in action and observation processing.
- Removed obsolete code and improved overall structure for better clarity.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* refactor(processors): remove unused import in joint_observations_processor
* refactor(processors): simplify transform_features method in delta_action_processor
* refactor(processors): streamline transform_features method in ImageCropResizeProcessor
* refactor(processors): improve error handling and streamline transform_features method in phone_processor
- Raised a ValueError for missing position and rotation in action to enhance error handling.
* refactor(processors): enhance error handling in JointVelocityProcessor
- Added a ValueError raise for missing current joint positions in the observation method to improve error handling and ensure the integrity of the transform_features method.
* refactor(processors): simplify transform_features method in robot kinematic processors
* refactor(processors): standardize action keys in phone_processor
* fix(processor): RKP feature obs -> act
---------
Signed-off-by: Adil Zouitine <adilzouitinegm@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co>
* refactor(converters): move batch transition functions to converters module
- Moved `_default_batch_to_transition` and `_default_transition_to_batch` functions from `pipeline.py` to `converters.py` for better organization and separation of concerns.
- Updated references in `RobotProcessor` to use the new location of these functions.
- Added tests to ensure correct functionality of the transition functions, including handling of index and task_index fields.
- Removed redundant tests from `pipeline.py` to streamline the test suite.
* refactor(processor): reorganize EnvTransition and TransitionKey definitions
- Moved `EnvTransition` and `TransitionKey` classes from `pipeline.py` to a new `core.py` module for better structure and maintainability.
- Updated import statements across relevant modules to reflect the new location of these definitions, ensuring consistent access throughout the codebase.
* refactor(converters): rename and update dataset frame conversion functions
- Replaced `to_dataset_frame` with `transition_to_dataset_frame` for clarity and consistency in naming.
- Updated references in `record.py`, `pipeline.py`, and tests to use the new function name.
- Introduced `merge_transitions` to streamline the merging of transitions, enhancing readability and maintainability.
- Adjusted related tests to ensure correct functionality with the new naming conventions.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(processor): solve conflict artefacts
* refactor(converters): remove unused identity function and update type hints for merge_transitions
* refactor(processor): remove unused identity import and clean up gym_manipulator.py
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Steven Palma <steven.palma@huggingface.co>
* [Port codebase pipeline] General fixes for RL and scripts (#1748)
* Refactor dataset configuration in documentation and codebase
- Updated dataset configuration keys from `dataset_root` to `root` and `num_episodes` to `num_episodes_to_record` for consistency.
- Adjusted replay episode handling by renaming `episode` to `replay_episode`.
- Enhanced documentation
- added specific processor to transform from policy actions to delta actions
* Added Robot action to tensor processor
Added new processor script for dealing with gym specific action processing
* removed RobotAction2Tensor processor; imrpoved choosing observations in actor
* nit in delta action
* added missing reset functions to kinematics
* Adapt teleoperate and replay to pipeline similar to record
* refactor(processors): move to inheritance (#1750)
* fix(teleoperator): improvements phone implementation (#1752)
* fix(teleoperator): protect shared state in phone implementation
* refactor(teleop): separate classes in phone
* fix: solve breaking changes (#1753)
* refactor(policies): multiple improvements (#1754)
* refactor(processor): simpler logic in device processor (#1755)
* refactor(processor): euclidean distance in delta action processor (#1757)
* refactor(processor): improvements to joint observations processor migration (#1758)
* refactor(processor): improvements to tokenizer migration (#1759)
* refactor(processor): improvements to tokenizer migration
* fix(tests): tokenizer tests regression from #1750
* fix(processors): fix float comparison and config in hil processors (#1760)
* chore(teleop): remove unnecessary callbacks in KeyboardEndEffectorTeleop (#1761)
* refactor(processor): improvements normalize pipeline migration (#1756)
* refactor(processor): several improvements normalize processor step
* refactor(processor): more improvements normalize processor
* refactor(processor): more changes to normalizer
* refactor(processor): take a different approach to DRY
* refactor(processor): final design
* chore(record): revert comment and continue deleted (#1764)
* refactor(examples): pipeline phone examples (#1769)
* refactor(examples): phone teleop + teleop script
* refactor(examples): phone replay + replay
* chore(examples): rename phone example files & folders
* feat(processor): fix improvements to the pipeline porting (#1796)
* refactor(processor): enhance tensor device handling in normalization process (#1795)
* refactor(tests): remove unsupported device detection test for complementary data (#1797)
* chore(tests): update ToBatchProcessor test (#1798)
* refactor(tests): remove in-place mutation tests for actions and complementary data in batch processor
* test(tests): add tests for action and task processing in batch processor
* add names for android and ios phone (#1799)
* use _tensor_stats in normalize processor (#1800)
* fix(normalize_processor): correct device reference for tensor epsilon handling (#1801)
* add point 5 add missing feature contracts (#1806)
* Fix PR comments 1452 (#1807)
* use key to determine image
* Address rest of PR comments
* use PolicyFeatures in transform_features
---------
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
---------
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
Co-authored-by: Adil Zouitine <adilzouitinegm@gmail.com>
Co-authored-by: Pepijn <138571049+pkooij@users.noreply.github.com>
- Introduced `PREPROCESSOR_DEFAULT_NAME` and `POSTPROCESSOR_DEFAULT_NAME` constants for consistent naming across various processor implementations.
- Updated processor creation in multiple policy files to utilize these constants, enhancing code readability and maintainability.
- Modified the training script to load and save the preprocessor and postprocessor using the new constants.
* Migrate gym_manipulator to use the pipeline
Added get_teleop_events function to capture relevant events from teleop devices unrelated to actions
* Added the capability to record a dataset
* Added the replay functionality with the pipeline
* Refactored `actor.py` to use the pipeline
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* RL works at this commit - fixed actor.py and bugs in gym_manipulator
* change folder structure to reduce the size of gym_manip
* Refactored hilserl config
* Remove dataset and mode from HilSerlEnvConfig to a GymManipulatorConfig to reduce verbose of configs during training
* format docs
* removed get_teleop_events from abc
* Refactor environment configuration and processing pipeline for GymHIL support. Removed device attribute from HILSerlRobotEnvConfig, added DummyTeleopDevice for simulation, and updated processor creation to accommodate GymHIL environments.
* Improved typing for HILRobotEnv config and GymManipulator config
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Migrated `gym_manipulator` to use a more modular structure similar to phone teleop
* Refactor gripper handling and transition processing in HIL and robot kinematic processors
- Updated gripper position handling to use a consistent key format across processors
- Improved the EEReferenceAndDelta class to handle reference joint positions.
- Added support for discrete gripper actions in the GripperVelocityToJoint processor.
- Refactored the gym manipulator to improve modularity and clarity in processing steps.
* Added delta_action_processor mapping wrapper
* Added missing file delta_action_processor and improved imports in `gym_manipulator`
* nit
* Added missing file joint_observation_processor
* Enhance processing architecture with new teleoperation processors
- Introduced `AddTeleopActionAsComplimentaryData` and `AddTeleopEventsAsInfo` for integrating teleoperator actions and events into transitions.
- Added `Torch2NumpyActionProcessor` and `Numpy2TorchActionProcessor` for seamless conversion between PyTorch tensors and NumPy arrays.
- Updated `__init__.py` to include new processors in module exports, improving modularity and clarity in the processing pipeline.
- GymHIL is now fully supported with HIL using the pipeline
* Refactor configuration structure for gym_hil integration
- Renamed sections for better readability, such as changing "Gym Wrappers Configuration" to "Processor Configuration."
- Enhanced documentation with clear examples for dataset collection and policy evaluation configurations.
* Enhance reset configuration and teleoperation event handling
- Added `terminate_on_success` parameter to `ResetConfig` and `InterventionActionProcessor` for controlling episode termination behavior upon success detection.
- Updated documentation to clarify the impact of `terminate_on_success` on data collection for reward classifier training.
- Refactored teleoperation event handling to use `TeleopEvents` constants for improved readability and maintainability across various modules.
* fix(keyboard teleop), delta action keys
* Added transform features and feature contract
* Added transform features for image crop
* Enum for TeleopEvents
* Update tranform_features delta action proc
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- Added support for RobotProcessor in the record_loop function to enhance data processing capabilities.
- Updated the logic to reset both policy and processor when provided, ensuring proper state management.
- Modified action prediction to utilize the processor, improving the overall functionality of the recording process.
- Adjusted the save_checkpoint function to include preprocessor state saving, enhancing checkpointing capabilities.
- Simplified the retrieval of observation features by removing the normalization step from the `get_cached_image_features` method calls.
- This change enhances clarity and aligns with the recent updates to policy processors.
* Merge together proto files and refactor Async inference
* Fixup for Async inference
* Drop not reuqired changes
* Fix tests
* Drop old async files
* Drop chunk_size param
* Fix versions
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix wrong fix
Co-authored-by: Ben Zhang <ben.zhang@uwaterloo.ca>
* Fixup
---------
Co-authored-by: Michel Aractingi <michel.aractingi@huggingface.co>
Co-authored-by: Ben Zhang <ben.zhang@uwaterloo.ca>
Co-authored-by: Francesco Capuano <74058581+fracapuano@users.noreply.github.com>
