Skip to content

fix(one shot load): adding metadata loading when reading from a dataset after writing #3305

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
CarolinePascal wants to merge 1 commit into
base: main
Choose a base branch
from
+1 −0
Open

fix(one shot load): adding metadata loading when reading from a dataset after writing #3305

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/lerobot/datasets/dataset_reader.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ def try_load(self) -> bool:

def load_and_activate(self) -> None:
"""Load HF dataset from disk and build index mapping. Call after data is on disk."""
self._meta._load_metadata()
Comment on lines 99 to +101
Copy link

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

load_and_activate() now relies on LeRobotDatasetMetadata._load_metadata(), which is a private method on another class. To avoid coupling to a private API (and potential future breakage), consider adding/using a public metadata refresh method (e.g., reload() / load_from_disk()) on LeRobotDatasetMetadata and calling that here instead.

Suggested change
def load_and_activate(self) -> None:
"""Load HF dataset from disk and build index mapping. Call after data is on disk."""
self._meta._load_metadata()
def _refresh_metadata(self) -> None:
"""Refresh dataset metadata using a public API exposed by the metadata object."""
for method_name in ("reload", "load_from_disk"):
refresh = getattr(self._meta, method_name, None)
if callable(refresh):
refresh()
return
raise AttributeError(
"LeRobotDatasetMetadata must expose a public metadata refresh method "
"(for example, 'reload()' or 'load_from_disk()') before "
"DatasetReader.load_and_activate() can refresh metadata."
)
def load_and_activate(self) -> None:
"""Load HF dataset from disk and build index mapping. Call after data is on disk."""
self._refresh_metadata()

Copilot uses AI. Check for mistakes.
self.hf_dataset = self._load_hf_dataset()
self._build_index_mapping()
Comment on lines 99 to 103
Copy link

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change fixes a read-after-write path (metadata reloaded before building the index), but there’s no regression test covering the failing scenario described in the PR (create dataset with video features, write episodes, finalize, then access dataset[0] without re-instantiating). Adding a test in the existing DatasetReader/LeRobotDataset test suite would prevent this from regressing again.

Copilot uses AI. Check for mistakes.

Expand Down
Loading