Skip to content

Improves recorder performance and add additional recording capability#3302

Merged
kellyguo11 merged 18 commits intoisaac-sim:mainfrom
peterd-NV:peterd/lw_patches
Sep 5, 2025
Merged

Improves recorder performance and add additional recording capability#3302
kellyguo11 merged 18 commits intoisaac-sim:mainfrom
peterd-NV:peterd/lw_patches

Conversation

@peterd-NV
Copy link
Copy Markdown
Contributor

@peterd-NV peterd-NV commented Aug 28, 2025

Description

This PR adds fixes from LightWheel Labs and additional functionality to the IsaacLab recorder.

Fixes # (issue)

  • Fixes performance issue when recording long episode data by replacing the use of torch.cat at every timestep with list append.
  • Fixes configclass validation when key is not a string

Adds Functionality

  • Adds optional episode meta data to HDF5 recorder
  • Adds option to record data pre-physics step
  • Adds joint target data to episode data. Joint target data can be optionally recorded by users and replayed to bypass action term controllers and improve replay determinism.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@peterd-NV peterd-NV changed the title Peterd/lw patches Improve recorder performance and add additional recording capability Aug 28, 2025
@peterd-NV peterd-NV requested a review from kellyguo11 August 28, 2025 19:50
self.scene.write_data_to_sim()
# simulate
self.sim.step(render=False)
self.recorder_manager.record_pre_physics_step()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this happening now after the physics step?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The recording right now records the states after a physics step. Here, it's adding functionality to record the states prior to every physics step in the decimation loop which provides more fine grain recording.

Copy link
Copy Markdown
Contributor Author

@peterd-NV peterd-NV Aug 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The motivation here is that full determinism during replay is not possible when running parallel envs as the physX buffers cannot be flushed on every IsaacLab reset. This can cause significant divergence when replaying data collected with parallel envs by just using env.step as it has to run through the entire action space which may contain controllers that are sensitive to this nondeterminism. Recording at the decimation level every sim dt can help alleviate this by giving users the option of more fine grained control.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that recording pre physics steps is fully optional and will not enabled by our existing default recorder workflows to limit dataset size. Users can take advantage of it via their own recorder terms if needed (like how Lightwheel is currently doing).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but should this be called record_post_physics_step() or record_post_physics_decimated_step because it is after ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I misunderstood Mayank's original comment. Yes I agree the naming here is confusing. I have updated to be called "record_post_physics_decimation_step."

@peterd-NV peterd-NV requested a review from Mayankm96 September 2, 2025 15:26
@kellyguo11 kellyguo11 changed the title Improve recorder performance and add additional recording capability Improves recorder performance and add additional recording capability Sep 4, 2025
Comment thread source/isaaclab/isaaclab/managers/recorder_manager.py Outdated
Comment thread source/isaaclab/isaaclab/managers/recorder_manager.py Outdated
Signed-off-by: Kelly Guo <kellyg@nvidia.com>
Signed-off-by: Kelly Guo <kellyg@nvidia.com>
@kellyguo11 kellyguo11 merged commit fb270ab into isaac-sim:main Sep 5, 2025
8 checks passed
george-nehma pushed a commit to george-nehma/DreamLander-IsaacLab that referenced this pull request Oct 24, 2025
…isaac-sim#3302)

# Description

<!--
Thank you for your interest in sending a pull request. Please make sure
to check the contribution guidelines.

Link:
https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html
-->

This PR adds fixes from LightWheel Labs and additional functionality to
the IsaacLab recorder.

Fixes # (issue)

- Fixes performance issue when recording long episode data by replacing
the use of torch.cat at every timestep with list append.
- Fixes configclass validation when key is not a string

Adds Functionality

- Adds optional episode meta data to HDF5 recorder
- Adds option to record data pre-physics step
- Adds joint target data to episode data. Joint target data can be
optionally recorded by users and replayed to bypass action term
controllers and improve replay determinism.


## Type of change

<!-- As you go through the list, delete the ones that are not
applicable. -->

- Bug fix (non-breaking change which fixes an issue)
- New feature (non-breaking change which adds functionality)

## Screenshots

Please attach before and after screenshots of the change if applicable.

<!--
Example:

| Before | After |
| ------ | ----- |
| _gif/png before_ | _gif/png after_ |

To upload images to a PR -- simply drag and drop an image while in edit
mode and it should upload the image directly. You can then paste that
source into the above before/after sections.
-->

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->

---------

Signed-off-by: Kelly Guo <kellyg@nvidia.com>
Co-authored-by: Kelly Guo <kellyg@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants