Back to Details

rerun-lerobot

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Rerun LeRobot ingestion

Rerun LeRobot 数据导入

Rerun has a built-in LeRobot importer: point 
log_file_from_path
(or the viewer, or 
rerun <dir>
on the CLI) at the dataset directory and it ingests episodes, camera videos, and state/action tables with no conversion code. There is no chunk-level 
LeRobotReader
; the chunk-processing route is to import first, then reprocess the resulting RRD with 
RrdReader
.
The download step needs 
huggingface_hub
.
Rerun 内置了LeRobot 导入器：只需将 
log_file_from_path
（或查看器，或CLI命令 
rerun <dir>
）指向数据集目录，即可无需转换代码直接导入剧集、摄像头视频以及状态/动作表。 目前没有针对块级别的 
LeRobotReader
；如需进行块处理，需先导入数据集，再使用 
RrdReader
重新处理生成的RRD文件。
下载步骤需要依赖 
huggingface_hub
。

Step 1: dataset -> one combined RRD

步骤1：将数据集转换为单个合并RRD文件

python
from huggingface_hub import snapshot_download
import rerun as rr

dataset_dir = snapshot_download(repo_id="rerun/so101-pick-and-place", repo_type="dataset", local_dir=dest)

with rr.RecordingStream("rerun_example_lerobot") as rec:
    rec.save(str(combined_rrd))
    rec.log_file_from_path(str(dataset_dir))  # the built-in importer
The importer emits one recording per episode (recording ids like 
episode_1
), plus a metadata-only root recording, all into the single RRD.
rr.RecordingStream
+ 
log_file_from_path
here is the importer bootstrap — the one place 
RecordingStream
is correct in an ingestion pipeline (it drives the built-in importer, not per-message logging). Do not generalize it to 
rr.log
-per-message loops; for everything after import, reprocess the RRD with 
RrdReader
+ lenses (see 
rerun-chunk-processing
: Chunk API vs logging API).
python
from huggingface_hub import snapshot_download
import rerun as rr

dataset_dir = snapshot_download(repo_id="rerun/so101-pick-and-place", repo_type="dataset", local_dir=dest)

with rr.RecordingStream("rerun_example_lerobot") as rec:
    rec.save(str(combined_rrd))
    rec.log_file_from_path(str(dataset_dir))  # 内置导入器
该导入器会为每个剧集生成一个录制文件（录制ID类似 
episode_1
），再加上一个仅包含元数据的根录制文件，所有内容都存入单个RRD文件中。
此处的 
rr.RecordingStream
+ 
log_file_from_path
是导入器启动流程——这是在导入流水线中使用 
RecordingStream
的正确场景（它驱动内置导入器，而非逐消息日志记录）。不要将其推广到逐消息调用 
rr.log
的循环中；导入完成后的所有操作，都需使用 
RrdReader
+ 透镜重新处理RRD文件（详见 
rerun-chunk-processing
：块API vs 日志API）。

Step 2: split into per-episode RRDs

步骤2：分割为单剧集RRD文件

Catalog segments are one-recording-per-file, and 
recording_id
becomes the segment id on registration. Split with 
RrdReader
:
python
reader = rr.experimental.RrdReader(str(combined_rrd))
for entry in reader.recordings():
    store = reader.store(store=entry)
    if not store.schema().entity_paths():  # skip the metadata-only root recording
        continue
    episode_id = zero_pad(entry.recording_id)  # episode_1 -> episode_00001
    with rr.RecordingStream("rerun_example_lerobot", recording_id=episode_id, send_properties=False) as rec:
        rec.save(str(rrd_dir / f"{episode_id}.rrd"))
        rec.send_chunks(store)
Two non-obvious moves:
  • Zero-pad the episode id. 
    episode_10
    sorts before 
    episode_2
    lexicographically; segment tables and viewers sort lexicographically. Pad to a fixed width when re-assigning 
    recording_id
    .
  • send_properties=False
     on the new stream, so the copy doesn't inject fresh recording properties on top of the copied chunks.
send_chunks
does not preserve the source store's identity; the new stream's 
recording_id
wins, which is exactly what makes the rename work.
If episodes need cleanup (drop topics, fix data, add derived components), run the store through lenses between read and write: 
reader.stream(store=entry).drop(...).lenses(...)
then 
collect().write_rrd(..., recording_id=episode_id)
(see 
rerun-chunk-processing
).
Computed layers and per-episode properties then follow the standard patterns in 
rerun-data-model
(layer 
recording_id
must equal the episode segment id).
目录中的片段是“单录制文件对应单个片段”，
recording_id
会在注册时成为片段ID。 使用 
RrdReader
进行分割：
python
reader = rr.experimental.RrdReader(str(combined_rrd))
for entry in reader.recordings():
    store = reader.store(store=entry)
    if not store.schema().entity_paths():  # 跳过仅含元数据的根录制文件
        continue
    episode_id = zero_pad(entry.recording_id)  # episode_1 -> episode_00001
    with rr.RecordingStream("rerun_example_lerobot", recording_id=episode_id, send_properties=False) as rec:
        rec.save(str(rrd_dir / f"{episode_id}.rrd"))
        rec.send_chunks(store)
有两个容易忽略的细节：
  • 为剧集ID补零。按字典序排序时，
    episode_10
    会排在 
    episode_2
    之前；片段表和查看器均按字典序排序。重新分配 
    recording_id
    时，需将其补零至固定长度。
  • 新流设置 
    send_properties=False
    ，这样在复制块时不会在已复制的块上注入新的录制属性。
send_chunks
不会保留源存储的标识；新流的 
recording_id
会生效，这正是实现重命名的关键。
如果需要清理剧集（删除主题、修复数据、添加派生组件），可在读取和写入之间通过透镜处理存储：
reader.stream(store=entry).drop(...).lenses(...)
然后调用 
collect().write_rrd(..., recording_id=episode_id)
（详见 
rerun-chunk-processing
）。
计算层和单剧集属性需遵循 
rerun-data-model
中的标准模式（层的 
recording_id
必须与剧集片段ID一致）。

Gotchas

注意事项

  1. log_file_from_path
    must target the dataset root directory, not a file inside it.
  2. Unpadded episode ids sort incorrectly downstream; pad before registering.
  3. The combined RRD contains a metadata-only root recording; skip stores with no entity paths or you register an empty segment.
  1. log_file_from_path
    必须指向数据集根目录，而非目录内的单个文件。
  2. 未补零的剧集ID在后续流程中排序会出错；注册前需补零。
  3. 合并后的RRD文件包含一个仅含元数据的根录制文件；需跳过没有实体路径的存储，否则会注册一个空片段。

References

参考资料

  • https://github.com/rerun-io/rerun/tree/main/examples/python/dataloader
    prepare_dataset.py
    (download → import → split → register, complete and runnable) and 
    train.py
    (training-side consumption via 
    rerun.experimental.dataloader
    )
  • https://github.com/rerun-io/rerun/tree/main/examples/python/dataloader
    中的 
    prepare_dataset.py
    （完整可运行的下载→导入→分割→注册流程）和 
    train.py
    （通过 
    rerun.experimental.dataloader
    在训练侧消费数据） ",