robot-data-diff 0.1.0__tar.gz

robot_data_diff-0.1.0/PKG-INFO

@@ -0,0 +1,127 @@
+Metadata-Version: 2.4
+Name: robot-data-diff
+Version: 0.1.0
+Summary: Episode-aware diff for LeRobot robot-training datasets: what changed in the data since the last good run?
+License-Expression: Apache-2.0
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: pyarrow>=12
+Requires-Dist: pandas>=2
+Requires-Dist: numpy>=1.24
+Requires-Dist: huggingface_hub>=0.20
+
+# robot-data-diff
+
+**Episode-aware diff for LeRobot robot-training datasets.**
+
+You retrained the policy and it got worse. Was it the code — or the data?
+`robot-data-diff` answers *"what changed in the data since the last good run?"*
+by diffing two revisions of a dataset, episode by episode.
+
+Free, CPU-only, pure Python. No GPU, no uploads, no accounts needed.
+
+## Install
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e .
+```
+
+Dependencies (all free): `pyarrow`, `pandas`, `numpy`, `huggingface_hub`.
+
+> Note: this sandbox's egress proxy breaks `httpx`'s `no_proxy` parsing
+> (IPv6 entries). The tool strips `no_proxy`/`NO_PROXY` for its own
+> Hugging Face calls; your shell is untouched.
+
+## Usage
+
+```bash
+# two local dataset directories
+robot-data-diff diff ./my_dataset_v1 ./my_dataset_v2
+
+# Hugging Face ids (videos are never downloaded — meta + data parquet only)
+robot-data-diff diff lerobot/pusht ./my_local_copy
+
+# same repo, two revisions
+robot-data-diff diff lerobot/pusht@main lerobot/pusht@<commit-sha>
+
+# link the diff to a policy checkpoint, and write an HTML report
+robot-data-diff diff v1 v2 --checkpoint ckpt-42 --html report.html
+
+# metadata-only mode (skips per-episode quality stats; fastest)
+robot-data-diff diff v1 v2 --no-quality
+```
+
+Each side can be a local path or a Hugging Face dataset id
+(`org/repo`, optionally `org/repo@revision` or `hf://org/repo@revision`).
+
+## What the report shows
+
+- **Episode inventory:** total counts, episodes added / removed / changed
+  (same index, different data fingerprint)
+- **Per-episode deltas** for changed episodes: frame count, action
+  min/max/mean per dimension, idle-frame-ratio delta, action-spike delta
+- **Task inventory:** episodes per task, old → new
+- **Schema drift:** features added / removed / changed
+- **Quality summary:** mean idle-frame ratio and total action spikes,
+  old → new (lightweight CPU heuristics, not a full validator)
+
+## Supported formats
+
+- **LeRobot v3.0** — verified against the real `lerobot/pusht` dataset
+  (parquet episode metadata + data chunks + `meta/info.json`).
+- **LeRobot v2.1** — implemented against the documented v2.1 layout
+  (`meta/episodes.jsonl`, `meta/episodes_stats.jsonl`, per-episode data
+  files). Tested with a synthetic v2 fixture (`tests/make_v2_fixture.py`);
+  no public v2 dataset was reachable at build time, so treat v2 support
+  as beta and report issues.
+
+## Example
+
+`demo/diff-demo-output.txt` is the real output of diffing `lerobot/pusht`
+(206 episodes) against a modified copy: episodes 200–205 removed, episodes
+0–4 had their actions scaled ×1.5, episode 206 added. The tool reports
+exactly that:
+
+```
+Episodes: 206 -> 201 (+1 added, -6 removed, ~5 changed, =195 unchanged)
+  added:   [206]
+  removed: [200, 201, 202, 203, 204, 205]
+
+CHANGED EPISODES (5):
+  [ep 0] length 161 -> 161 (+0 frames)
+           d(action_min)  [+46.5000, +35.5000]
+           d(action_max)  [+187.5000, +224.5000]
+           ...
+```
+
+See `demo/diff-demo-report.html` for the HTML version.
+
+## Reproduce the demo
+
+```bash
+source .venv/bin/activate
+python tests/make_modified_fixture.py      # builds testdata/pusht_v2
+robot-data-diff diff testdata/pusht testdata/pusht_v2 \
+    --checkpoint ckpt-42 --html demo/diff-demo-report.html
+```
+
+## Honest limits
+
+- This is a **diff**, not a validator: it tells you *what changed* between
+  revisions, not whether your data is good. (The validator lane already has
+  6+ tools — that's why this exists.)
+- "Changed" is detected via a stats fingerprint (length, action
+  min/max/mean, idle ratio, spike count). Bit-identical data with reordered
+  episodes shows as added+removed, not moved — episode identity is the index.
+- Remote datasets download `meta/**` + `data/**`; very large datasets will
+  take a while on first run (cached afterwards via `--cache-dir`).
+- RLDS / Zarr / HDF5 / rosbag support is out of scope for v0.1 (LeRobot only).
+
+## Related work
+
+- [openbot-data](https://github.com/openbotai/openbot-data) (MIT) — a broader robot-data toolkit whose `openbot-data diff` compares two *snapshot files* (`snapshot` first, then `diff`, with `--fail-on breaking` for CI-style breaking-change detection). robot-data-diff is narrower on purpose: it diffs two dataset revisions *directly* — local dirs or Hugging Face ids, no snapshot step — episode by episode, with per-episode stat deltas.
+
+## License
+
+Apache-2.0

robot_data_diff-0.1.0/README.md

@@ -0,0 +1,115 @@
+# robot-data-diff
+
+**Episode-aware diff for LeRobot robot-training datasets.**
+
+You retrained the policy and it got worse. Was it the code — or the data?
+`robot-data-diff` answers *"what changed in the data since the last good run?"*
+by diffing two revisions of a dataset, episode by episode.
+
+Free, CPU-only, pure Python. No GPU, no uploads, no accounts needed.
+
+## Install
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e .
+```
+
+Dependencies (all free): `pyarrow`, `pandas`, `numpy`, `huggingface_hub`.
+
+> Note: this sandbox's egress proxy breaks `httpx`'s `no_proxy` parsing
+> (IPv6 entries). The tool strips `no_proxy`/`NO_PROXY` for its own
+> Hugging Face calls; your shell is untouched.
+
+## Usage
+
+```bash
+# two local dataset directories
+robot-data-diff diff ./my_dataset_v1 ./my_dataset_v2
+
+# Hugging Face ids (videos are never downloaded — meta + data parquet only)
+robot-data-diff diff lerobot/pusht ./my_local_copy
+
+# same repo, two revisions
+robot-data-diff diff lerobot/pusht@main lerobot/pusht@<commit-sha>
+
+# link the diff to a policy checkpoint, and write an HTML report
+robot-data-diff diff v1 v2 --checkpoint ckpt-42 --html report.html
+
+# metadata-only mode (skips per-episode quality stats; fastest)
+robot-data-diff diff v1 v2 --no-quality
+```
+
+Each side can be a local path or a Hugging Face dataset id
+(`org/repo`, optionally `org/repo@revision` or `hf://org/repo@revision`).
+
+## What the report shows
+
+- **Episode inventory:** total counts, episodes added / removed / changed
+  (same index, different data fingerprint)
+- **Per-episode deltas** for changed episodes: frame count, action
+  min/max/mean per dimension, idle-frame-ratio delta, action-spike delta
+- **Task inventory:** episodes per task, old → new
+- **Schema drift:** features added / removed / changed
+- **Quality summary:** mean idle-frame ratio and total action spikes,
+  old → new (lightweight CPU heuristics, not a full validator)
+
+## Supported formats
+
+- **LeRobot v3.0** — verified against the real `lerobot/pusht` dataset
+  (parquet episode metadata + data chunks + `meta/info.json`).
+- **LeRobot v2.1** — implemented against the documented v2.1 layout
+  (`meta/episodes.jsonl`, `meta/episodes_stats.jsonl`, per-episode data
+  files). Tested with a synthetic v2 fixture (`tests/make_v2_fixture.py`);
+  no public v2 dataset was reachable at build time, so treat v2 support
+  as beta and report issues.
+
+## Example
+
+`demo/diff-demo-output.txt` is the real output of diffing `lerobot/pusht`
+(206 episodes) against a modified copy: episodes 200–205 removed, episodes
+0–4 had their actions scaled ×1.5, episode 206 added. The tool reports
+exactly that:
+
+```
+Episodes: 206 -> 201 (+1 added, -6 removed, ~5 changed, =195 unchanged)
+  added:   [206]
+  removed: [200, 201, 202, 203, 204, 205]
+
+CHANGED EPISODES (5):
+  [ep 0] length 161 -> 161 (+0 frames)
+           d(action_min)  [+46.5000, +35.5000]
+           d(action_max)  [+187.5000, +224.5000]
+           ...
+```
+
+See `demo/diff-demo-report.html` for the HTML version.
+
+## Reproduce the demo
+
+```bash
+source .venv/bin/activate
+python tests/make_modified_fixture.py      # builds testdata/pusht_v2
+robot-data-diff diff testdata/pusht testdata/pusht_v2 \
+    --checkpoint ckpt-42 --html demo/diff-demo-report.html
+```
+
+## Honest limits
+
+- This is a **diff**, not a validator: it tells you *what changed* between
+  revisions, not whether your data is good. (The validator lane already has
+  6+ tools — that's why this exists.)
+- "Changed" is detected via a stats fingerprint (length, action
+  min/max/mean, idle ratio, spike count). Bit-identical data with reordered
+  episodes shows as added+removed, not moved — episode identity is the index.
+- Remote datasets download `meta/**` + `data/**`; very large datasets will
+  take a while on first run (cached afterwards via `--cache-dir`).
+- RLDS / Zarr / HDF5 / rosbag support is out of scope for v0.1 (LeRobot only).
+
+## Related work
+
+- [openbot-data](https://github.com/openbotai/openbot-data) (MIT) — a broader robot-data toolkit whose `openbot-data diff` compares two *snapshot files* (`snapshot` first, then `diff`, with `--fail-on breaking` for CI-style breaking-change detection). robot-data-diff is narrower on purpose: it diffs two dataset revisions *directly* — local dirs or Hugging Face ids, no snapshot step — episode by episode, with per-episode stat deltas.
+
+## License
+
+Apache-2.0

robot_data_diff-0.1.0/pyproject.toml

@@ -0,0 +1,24 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "robot-data-diff"
+version = "0.1.0"
+description = "Episode-aware diff for LeRobot robot-training datasets: what changed in the data since the last good run?"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "Apache-2.0"
+dependencies = [
+    "pyarrow>=12",
+    "pandas>=2",
+    "numpy>=1.24",
+    "huggingface_hub>=0.20",
+]
+
+[project.scripts]
+robot-data-diff = "robot_data_diff.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["robot_data_diff*"]

robot_data_diff-0.1.0/robot_data_diff/__init__.py

@@ -0,0 +1,6 @@
+"""robot-data-diff: episode-aware diffs for LeRobot robot-training datasets.
+
+Answers: "we retrained the policy and it got worse — what changed in the data?"
+"""
+
+__version__ = "0.1.0"

robot_data_diff-0.1.0/robot_data_diff/__main__.py

@@ -0,0 +1,5 @@
+"""Allow `python -m robot_data_diff`."""
+from .cli import main
+
+if __name__ == "__main__":
+    raise SystemExit(main())

robot_data_diff-0.1.0/robot_data_diff/cli.py

@@ -0,0 +1,62 @@
+"""CLI entry point: robot-data-diff diff <A> <B> [--html OUT] [--checkpoint LBL]."""
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+from .differ import diff_datasets
+from .loader import load_dataset
+from .report import render_html, render_text
+
+
+def build_parser() -> argparse.ArgumentParser:
+    p = argparse.ArgumentParser(
+        prog="robot-data-diff",
+        description="Episode-aware diff for LeRobot robot-training datasets. "
+                    "Answers: what changed in the data since the last good run?",
+    )
+    sub = p.add_subparsers(dest="cmd", required=True)
+    d = sub.add_parser("diff", help="diff two dataset revisions")
+    d.add_argument("dataset_a", help="local path or HF id (org/repo[@rev])")
+    d.add_argument("dataset_b", help="local path or HF id (org/repo[@rev])")
+    d.add_argument("--html", metavar="OUT",
+                   help="also write an HTML report to OUT")
+    d.add_argument("--checkpoint", metavar="LABEL",
+                   help="policy checkpoint name to link this diff to")
+    d.add_argument("--no-quality", action="store_true",
+                   help="skip per-episode quality stats (metadata-only, faster)")
+    d.add_argument("--cache-dir", metavar="DIR", default=None,
+                   help="where to cache Hugging Face downloads")
+    return p
+
+
+def cmd_diff(args) -> int:
+    print("loading A ...", file=sys.stderr)
+    a = load_dataset(args.dataset_a, cache_dir=args.cache_dir,
+                     with_quality=not args.no_quality)
+    print(f"  {a.version}, {len(a.episodes)} episodes", file=sys.stderr)
+    print("loading B ...", file=sys.stderr)
+    b = load_dataset(args.dataset_b, cache_dir=args.cache_dir,
+                     with_quality=not args.no_quality)
+    print(f"  {b.version}, {len(b.episodes)} episodes", file=sys.stderr)
+
+    result = diff_datasets(a, b, checkpoint=args.checkpoint)
+    print(render_text(result))
+    if args.html:
+        out = Path(args.html)
+        out.parent.mkdir(parents=True, exist_ok=True)
+        out.write_text(render_html(result))
+        print(f"\nHTML report written to {args.html}", file=sys.stderr)
+    return 0
+
+
+def main(argv=None) -> int:
+    args = build_parser().parse_args(argv)
+    if args.cmd == "diff":
+        return cmd_diff(args)
+    return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

robot_data_diff-0.1.0/robot_data_diff/differ.py

@@ -0,0 +1,124 @@
+"""Episode-aware diff between two DatasetViews."""
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+
+from .loader import DatasetView
+from .stats import fingerprint
+
+
+@dataclass
+class EpisodeDelta:
+    index: int
+    old_length: int
+    new_length: int
+    d_length: int
+    tasks_changed: bool
+    old_tasks: list = field(default_factory=list)
+    new_tasks: list = field(default_factory=list)
+    d_action_min: list | None = None   # per-dim new - old
+    d_action_max: list | None = None
+    d_action_mean: list | None = None
+    d_idle_ratio: float | None = None
+    d_spike_count: int | None = None
+
+
+@dataclass
+class DiffResult:
+    a_name: str
+    b_name: str
+    a_version: str
+    b_version: str
+    a_fps: float | None
+    b_fps: float | None
+    a_episodes: int
+    b_episodes: int
+    added: list[int] = field(default_factory=list)
+    removed: list[int] = field(default_factory=list)
+    changed: list[EpisodeDelta] = field(default_factory=list)
+    unchanged: int = 0
+    task_counts: dict = field(default_factory=dict)  # task -> (count_a, count_b)
+    features_added: list = field(default_factory=list)
+    features_removed: list = field(default_factory=list)
+    features_changed: list = field(default_factory=list)
+    mean_idle_a: float | None = None
+    mean_idle_b: float | None = None
+    spikes_a: int | None = None
+    spikes_b: int | None = None
+    checkpoint: str | None = None
+
+
+def _sub(a, b):
+    if a is None or b is None or len(a) != len(b):
+        return None
+    return [round(float(x) - float(y), 6) for x, y in zip(b, a)]
+
+
+def diff_datasets(a: DatasetView, b: DatasetView,
+                  checkpoint: str | None = None) -> DiffResult:
+    res = DiffResult(
+        a_name=a.name, b_name=b.name,
+        a_version=a.version, b_version=b.version,
+        a_fps=a.fps, b_fps=b.fps,
+        a_episodes=len(a.episodes), b_episodes=len(b.episodes),
+        checkpoint=checkpoint,
+    )
+    ai, bi = set(a.episodes), set(b.episodes)
+    res.added = sorted(bi - ai)
+    res.removed = sorted(ai - bi)
+
+    for idx in sorted(ai & bi):
+        ea, eb = a.episodes[idx], b.episodes[idx]
+        if fingerprint(ea) == fingerprint(eb):
+            res.unchanged += 1
+            continue
+        res.changed.append(EpisodeDelta(
+            index=idx,
+            old_length=ea.length, new_length=eb.length,
+            d_length=eb.length - ea.length,
+            tasks_changed=sorted(ea.tasks) != sorted(eb.tasks),
+            old_tasks=ea.tasks, new_tasks=eb.tasks,
+            d_action_min=_sub(ea.action_min, eb.action_min),
+            d_action_max=_sub(ea.action_max, eb.action_max),
+            d_action_mean=_sub(ea.action_mean, eb.action_mean),
+            d_idle_ratio=(round(eb.idle_ratio - ea.idle_ratio, 4)
+                          if ea.idle_ratio is not None and eb.idle_ratio is not None
+                          else None),
+            d_spike_count=(eb.spike_count - ea.spike_count
+                           if ea.spike_count is not None and eb.spike_count is not None
+                           else None),
+        ))
+
+    # task inventory: task string -> (count in A, count in B)
+    def task_hist(ds: DatasetView) -> dict:
+        h: dict[str, int] = {}
+        for ep in ds.episodes.values():
+            for t in ep.tasks or ["<no task>"]:
+                h[t] = h.get(t, 0) + 1
+        return h
+
+    ha, hb = task_hist(a), task_hist(b)
+    for t in sorted(set(ha) | set(hb)):
+        res.task_counts[t] = (ha.get(t, 0), hb.get(t, 0))
+
+    # schema drift
+    fa, fb = set(a.features), set(b.features)
+    res.features_added = sorted(fb - fa)
+    res.features_removed = sorted(fa - fb)
+    for f in sorted(fa & fb):
+        if a.features[f] != b.features[f]:
+            res.features_changed.append(
+                f"{f}: {a.features[f]} -> {b.features[f]}")
+
+    # quality summary
+    def qmean(ds: DatasetView):
+        vals = [e.idle_ratio for e in ds.episodes.values()
+                if e.idle_ratio is not None]
+        sp = [e.spike_count for e in ds.episodes.values()
+              if e.spike_count is not None]
+        return (round(sum(vals) / len(vals), 4) if vals else None,
+                sum(sp) if sp else None)
+
+    res.mean_idle_a, res.spikes_a = qmean(a)
+    res.mean_idle_b, res.spikes_b = qmean(b)
+    return res

robot_data_diff-0.1.0/robot_data_diff/loader.py

@@ -0,0 +1,258 @@
+"""Load LeRobot datasets (v2.1 and v3.0) from a local path or a Hugging Face repo id.
+
+v3.0 layout (current):
+    meta/info.json                  codebase_version "v3.0", fps, features, totals
+    meta/episodes/chunk-*/file-*.parquet   one row per episode; includes
+        episode_index, tasks, length, and per-episode stats/action/min|max|mean
+    meta/tasks.parquet              task_index -> task string (in the index)
+    data/chunk-*/file-*.parquet     frames: episode_index, frame_index,
+        timestamp, action, observation.state, ...
+
+v2.1 layout (legacy, still common on the Hub):
+    meta/info.json                  codebase_version "v2.1", fps, features
+    meta/episodes.jsonl             episode_index, tasks, length (one JSON/line)
+    meta/episodes_stats.jsonl       per-episode min/max/mean/std per feature
+    meta/tasks.jsonl                task_index -> task
+    data/chunk-*/episode_*.parquet  one parquet per episode with the frames
+
+Videos are never read — everything here is metadata + data parquet only.
+"""
+from __future__ import annotations
+
+import json
+import os
+from dataclasses import dataclass, field
+from pathlib import Path
+
+import numpy as np
+import pyarrow.parquet as pq
+
+from .stats import action_quality
+
+
+@dataclass
+class EpisodeInfo:
+    index: int
+    length: int
+    tasks: list = field(default_factory=list)
+    action_min: list | None = None
+    action_max: list | None = None
+    action_mean: list | None = None
+    state_min: list | None = None
+    state_max: list | None = None
+    idle_ratio: float | None = None
+    spike_count: int | None = None
+
+
+@dataclass
+class DatasetView:
+    name: str
+    version: str
+    fps: float | None
+    features: dict  # feature name -> {"dtype": str, "shape": list}
+    episodes: dict  # episode_index -> EpisodeInfo
+    root: Path
+
+
+# ---------------------------------------------------------------------------
+# reference resolution: local path or Hugging Face id
+# ---------------------------------------------------------------------------
+
+def _strip_no_proxy():
+    # The sandbox proxy env breaks httpx2's no_proxy parsing (IPv6 entries);
+    # the egress proxy handles everything, so drop no_proxy for HF calls.
+    for k in ("no_proxy", "NO_PROXY"):
+        os.environ.pop(k, None)
+
+
+def resolve_ref(ref: str, cache_dir: str | Path | None = None) -> Path:
+    """Turn a dataset reference into a local directory Path.
+
+    Accepts a local path, or a Hugging Face dataset id like
+    "lerobot/pusht" or "lerobot/pusht@<revision>".  HF downloads fetch
+    meta/** and data/** only — videos are skipped.
+    """
+    p = Path(ref)
+    if p.exists():
+        return p
+    _strip_no_proxy()
+    from huggingface_hub import snapshot_download
+
+    repo_id, _, revision = ref.removeprefix("hf://").partition("@")
+    if "/" not in repo_id:
+        raise ValueError(
+            f"'{ref}' is not a local path and not a Hugging Face id "
+            "(expected like 'lerobot/pusht' or 'lerobot/pusht@<rev>')."
+        )
+    local = snapshot_download(
+        repo_id,
+        repo_type="dataset",
+        revision=revision or None,
+        allow_patterns=["meta/**", "data/**"],
+        local_dir=cache_dir,
+    )
+    return Path(local)
+
+
+def _as_list(v):
+    if v is None:
+        return None
+    return [float(x) for x in np.asarray(v).ravel()]
+
+
+# ---------------------------------------------------------------------------
+# v3 loader
+# ---------------------------------------------------------------------------
+
+def _load_v3(root: Path, name: str) -> DatasetView:
+    info = json.loads((root / "meta" / "info.json").read_text())
+    features = {
+        k: {"dtype": str(v.get("dtype")), "shape": list(v.get("shape") or [])}
+        for k, v in (info.get("features") or {}).items()
+    }
+    fps = info.get("fps")
+
+    ep_files = sorted((root / "meta" / "episodes").rglob("*.parquet"))
+    if not ep_files:
+        raise ValueError(f"no episode metadata found under {root}/meta/episodes")
+    episodes: dict[int, EpisodeInfo] = {}
+    for f in ep_files:
+        table = pq.read_table(f).to_pandas()
+        for _, row in table.iterrows():
+            idx = int(row["episode_index"])
+            tasks = [str(t) for t in (row.get("tasks") or [])]
+            episodes[idx] = EpisodeInfo(
+                index=idx,
+                length=int(row["length"]),
+                tasks=tasks,
+                action_min=_as_list(row.get("stats/action/min")),
+                action_max=_as_list(row.get("stats/action/max")),
+                action_mean=_as_list(row.get("stats/action/mean")),
+                state_min=_as_list(row.get("stats/observation.state/min")),
+                state_max=_as_list(row.get("stats/observation.state/max")),
+            )
+    ds = DatasetView(name=name, version="v3.0", fps=fps, features=features,
+                     episodes=episodes, root=root)
+    _add_quality_v3(ds)
+    return ds
+
+
+def _add_quality_v3(ds: DatasetView):
+    """Compute idle-ratio / spike-count per episode from v3 data chunks."""
+    data_files = sorted((ds.root / "data").rglob("*.parquet"))
+    if not data_files:
+        return
+    for f in data_files:
+        try:
+            table = pq.read_table(f, columns=["episode_index", "action"])
+        except Exception:
+            continue  # chunk without action column; skip
+        df = table.to_pandas()
+        for ep_idx, grp in df.groupby("episode_index"):
+            ep = ds.episodes.get(int(ep_idx))
+            if ep is None:
+                continue
+            acts = np.stack([np.asarray(a, dtype=np.float64).ravel()
+                             for a in grp["action"].to_numpy()])
+            ep.idle_ratio, ep.spike_count = action_quality(acts)
+
+
+# ---------------------------------------------------------------------------
+# v2 loader (documented v2.1 layout)
+# ---------------------------------------------------------------------------
+
+def _load_v2(root: Path, name: str) -> DatasetView:
+    info = json.loads((root / "meta" / "info.json").read_text())
+    features = {
+        k: {"dtype": str(v.get("dtype")), "shape": list(v.get("shape") or [])}
+        for k, v in (info.get("features") or {}).items()
+    }
+    fps = info.get("fps")
+
+    episodes: dict[int, EpisodeInfo] = {}
+    ep_path = root / "meta" / "episodes.jsonl"
+    for line in ep_path.read_text().splitlines():
+        line = line.strip()
+        if not line:
+            continue
+        rec = json.loads(line)
+        idx = int(rec["episode_index"])
+        episodes[idx] = EpisodeInfo(
+            index=idx,
+            length=int(rec.get("length", 0)),
+            tasks=[str(t) for t in (rec.get("tasks") or [])],
+        )
+
+    stats_path = root / "meta" / "episodes_stats.jsonl"
+    if stats_path.exists():
+        for line in stats_path.read_text().splitlines():
+            line = line.strip()
+            if not line:
+                continue
+            rec = json.loads(line)
+            ep = episodes.get(int(rec["episode_index"]))
+            if ep is None:
+                continue
+            feats = rec.get("stats") or rec.get("features") or rec
+            for feat_key, target, attr in (
+                ("action", "action", None),
+                ("observation.state", "state", None),
+                ("observation_state", "state", None),
+            ):
+                f = feats.get(feat_key)
+                if isinstance(f, dict):
+                    if target == "action":
+                        ep.action_min = _as_list(f.get("min"))
+                        ep.action_max = _as_list(f.get("max"))
+                        ep.action_mean = _as_list(f.get("mean"))
+                    else:
+                        ep.state_min = _as_list(f.get("min"))
+                        ep.state_max = _as_list(f.get("max"))
+
+    ds = DatasetView(name=name, version="v2.1", fps=fps, features=features,
+                     episodes=episodes, root=root)
+    _add_quality_v2(ds)
+    return ds
+
+
+def _add_quality_v2(ds: DatasetView):
+    """Per-episode quality from v2 per-episode data files."""
+    for f in sorted((ds.root / "data").rglob("episode_*.parquet")):
+        try:
+            table = pq.read_table(f, columns=["action"])
+        except Exception:
+            continue
+        acts = np.stack([np.asarray(a, dtype=np.float64).ravel()
+                         for a in table.column("action").to_pylist()])
+        # episode index from filename episode_000123.parquet
+        try:
+            ep_idx = int(f.stem.split("_")[-1])
+        except ValueError:
+            continue
+        ep = ds.episodes.get(ep_idx)
+        if ep is not None:
+            ep.idle_ratio, ep.spike_count = action_quality(acts)
+
+
+# ---------------------------------------------------------------------------
+
+def load_dataset(ref: str, cache_dir: str | Path | None = None,
+                 with_quality: bool = True) -> DatasetView:
+    """Load a dataset reference into a DatasetView (v2.1 or v3.0)."""
+    root = resolve_ref(ref, cache_dir)
+    info_path = root / "meta" / "info.json"
+    if not info_path.exists():
+        raise ValueError(f"{root} does not look like a LeRobot dataset "
+                         "(missing meta/info.json).")
+    version = str(json.loads(info_path.read_text()).get("codebase_version", ""))
+    name = ref if isinstance(ref, str) else str(ref)
+    if version.startswith("v3"):
+        ds = _load_v3(root, name)
+    elif version.startswith("v2"):
+        ds = _load_v2(root, name)
+    else:
+        raise ValueError(f"unsupported LeRobot codebase_version: {version!r}")
+    if not with_quality:
+        for ep in ds.episodes.values():
+            ep.idle_ratio, ep.spike_count = None, None
+    return ds

robot_data_diff-0.1.0/robot_data_diff/report.py

@@ -0,0 +1,154 @@
+"""Human-readable text and HTML reports for a DiffResult."""
+from __future__ import annotations
+
+import html
+
+from .differ import DiffResult, EpisodeDelta
+
+
+def _fmt_vec(v, nd=4):
+    if v is None:
+        return "n/a"
+    return "[" + ", ".join(f"{x:+.{nd}f}" for x in v) + "]"
+
+
+def render_text(r: DiffResult) -> str:
+    L: list[str] = []
+    L.append("=" * 72)
+    L.append("robot-data-diff  —  episode-aware dataset diff")
+    L.append("=" * 72)
+    L.append(f"A: {r.a_name}   ({r.a_version}, {r.a_episodes} episodes"
+             + (f", {r.a_fps} fps" if r.a_fps else "") + ")")
+    L.append(f"B: {r.b_name}   ({r.b_version}, {r.b_episodes} episodes"
+             + (f", {r.b_fps} fps" if r.b_fps else "") + ")")
+    if r.checkpoint:
+        L.append(f"policy checkpoint: {r.checkpoint}")
+    L.append("")
+
+    L.append(f"Episodes: {r.a_episodes} -> {r.b_episodes} "
+             f"(+{len(r.added)} added, -{len(r.removed)} removed, "
+             f"~{len(r.changed)} changed, ={r.unchanged} unchanged)")
+    if r.added:
+        L.append(f"  added:   {r.added}")
+    if r.removed:
+        L.append(f"  removed: {r.removed}")
+    L.append("")
+
+    if r.changed:
+        L.append(f"CHANGED EPISODES ({len(r.changed)}):")
+        for d in r.changed:
+            L.append(f"  [ep {d.index}] length {d.old_length} -> {d.new_length} "
+                     f"({d.d_length:+d} frames)")
+            if d.tasks_changed:
+                L.append(f"           tasks: {d.old_tasks} -> {d.new_tasks}")
+            if d.d_action_min is not None:
+                L.append(f"           d(action_min)  {_fmt_vec(d.d_action_min)}")
+            if d.d_action_max is not None:
+                L.append(f"           d(action_max)  {_fmt_vec(d.d_action_max)}")
+            if d.d_action_mean is not None:
+                L.append(f"           d(action_mean) {_fmt_vec(d.d_action_mean)}")
+            if d.d_idle_ratio is not None:
+                L.append(f"           d(idle_ratio)  {d.d_idle_ratio:+.4f}")
+            if d.d_spike_count is not None:
+                L.append(f"           d(spikes)      {d.d_spike_count:+d}")
+        L.append("")
+    else:
+        L.append("No changed episodes (same index + identical stats).")
+        L.append("")
+
+    if r.task_counts:
+        L.append("TASK INVENTORY (episodes per task: A -> B):")
+        for t, (ca, cb) in r.task_counts.items():
+            mark = "  *" if ca != cb else "   "
+            L.append(f"{mark} {t[:60]:62s} {ca:5d} -> {cb:5d}")
+        L.append("")
+
+    if r.features_added or r.features_removed or r.features_changed:
+        L.append("SCHEMA DRIFT:")
+        for f in r.features_added:
+            L.append(f"  + {f}")
+        for f in r.features_removed:
+            L.append(f"  - {f}")
+        for f in r.features_changed:
+            L.append(f"  ~ {f}")
+        L.append("")
+    else:
+        L.append("Schema: identical feature sets.")
+        L.append("")
+
+    if r.mean_idle_a is not None:
+        L.append("QUALITY SUMMARY (heuristic, CPU-only):")
+        L.append(f"  mean idle-frame ratio: {r.mean_idle_a:.4f} -> {r.mean_idle_b:.4f} "
+                 f"({r.mean_idle_b - r.mean_idle_a:+.4f})")
+        L.append(f"  total action spikes:   {r.spikes_a} -> {r.spikes_b} "
+                 f"({r.spikes_b - r.spikes_a:+d})")
+        L.append("")
+
+    L.append("=" * 72)
+    return "\n".join(L)
+
+
+def render_html(r: DiffResult) -> str:
+    e = html.escape
+
+    def changed_rows():
+        rows = []
+        for d in r.changed:
+            rows.append(
+                f"<tr><td>{d.index}</td><td>{d.old_length} &rarr; {d.new_length} "
+                f"({d.d_length:+d})</td>"
+                f"<td>{_fmt_vec(d.d_action_min)}</td>"
+                f"<td>{_fmt_vec(d.d_action_max)}</td>"
+                f"<td>{d.d_idle_ratio if d.d_idle_ratio is not None else 'n/a'}</td>"
+                f"<td>{d.d_spike_count if d.d_spike_count is not None else 'n/a'}</td></tr>"
+            )
+        return "\n".join(rows) or '<tr><td colspan="6">none</td></tr>'
+
+    def task_rows():
+        rows = []
+        for t, (ca, cb) in r.task_counts.items():
+            cls = ' class="hl"' if ca != cb else ""
+            rows.append(f'<tr{cls}><td>{e(t[:80])}</td><td>{ca}</td><td>{cb}</td></tr>')
+        return "\n".join(rows)
+
+    ckpt = f"<p><b>Policy checkpoint:</b> {e(r.checkpoint)}</p>" if r.checkpoint else ""
+    qual = ""
+    if r.mean_idle_a is not None:
+        qual = (f"<p>Mean idle-frame ratio: {r.mean_idle_a:.4f} &rarr; "
+                f"{r.mean_idle_b:.4f} ({r.mean_idle_b - r.mean_idle_a:+.4f})<br>"
+                f"Total action spikes: {r.spikes_a} &rarr; {r.spikes_b} "
+                f"({r.spikes_b - r.spikes_a:+d})</p>")
+
+    return f"""<!DOCTYPE html>
+<html><head><meta charset="utf-8"><title>robot-data-diff report</title>
+<style>
+body {{ font-family: system-ui, sans-serif; max-width: 1000px; margin: 2em auto; padding: 0 1em; }}
+table {{ border-collapse: collapse; width: 100%; margin: 1em 0; }}
+th, td {{ border: 1px solid #ccc; padding: 6px 10px; text-align: left; font-size: 14px; }}
+th {{ background: #f4f4f4; }} .hl {{ background: #fff8e1; }}
+code {{ background: #f4f4f4; padding: 1px 5px; border-radius: 3px; }}
+h1 {{ font-size: 1.4em; }} h2 {{ font-size: 1.1em; margin-top: 1.6em; }}
+</style></head><body>
+<h1>robot-data-diff report</h1>
+<p><b>A:</b> {e(r.a_name)} <code>{e(r.a_version)}</code> ({r.a_episodes} episodes)<br>
+<b>B:</b> {e(r.b_name)} <code>{e(r.b_version)}</code> ({r.b_episodes} episodes)</p>
+{ckpt}
+<h2>Episode inventory</h2>
+<p>{r.a_episodes} &rarr; {r.b_episodes} episodes &mdash;
++{len(r.added)} added, &minus;{len(r.removed)} removed,
+~{len(r.changed)} changed, ={r.unchanged} unchanged</p>
+<p>Added: <code>{e(str(r.added))}</code><br>Removed: <code>{e(str(r.removed))}</code></p>
+<h2>Changed episodes</h2>
+<table><tr><th>ep</th><th>length</th><th>&Delta;action_min</th>
+<th>&Delta;action_max</th><th>&Delta;idle</th><th>&Delta;spikes</th></tr>
+{changed_rows()}</table>
+<h2>Task inventory</h2>
+<table><tr><th>task</th><th>A</th><th>B</th></tr>{task_rows()}</table>
+<h2>Schema drift</h2>
+<p>Added: <code>{e(str(r.features_added))}</code><br>
+Removed: <code>{e(str(r.features_removed))}</code><br>
+Changed: <code>{e(str(r.features_changed))}</code></p>
+<h2>Quality summary</h2>
+{qual}
+</body></html>
+"""

robot_data_diff-0.1.0/robot_data_diff/stats.py

@@ -0,0 +1,53 @@
+"""Per-episode statistics and lightweight quality heuristics (CPU-only, numpy)."""
+from __future__ import annotations
+
+import numpy as np
+
+
+def action_quality(action: np.ndarray) -> tuple[float, int]:
+    """Idle-frame ratio and action-spike count for one episode's actions.
+
+    action: (T, D) float array of the episode's action vectors.
+    Returns (idle_ratio, spike_count).
+    - idle frame: max |delta| across dims below eps (eps scales with the
+      episode's own action range, so it works for any robot/action units).
+    - spike: a frame-to-frame jump far above the episode's typical motion.
+    """
+    action = np.asarray(action, dtype=np.float64)
+    if action.ndim == 1:
+        action = action[:, None]
+    if len(action) < 2:
+        return 0.0, 0
+    deltas = np.abs(np.diff(action, axis=0))
+    maxd = deltas.max(axis=1)
+    overall_range = float(np.ptp(action))
+    eps = max(1e-3 * overall_range, 1e-9)
+    idle_ratio = float((maxd < eps).mean())
+    med = float(np.median(maxd))
+    spike_thresh = max(10.0 * med, 5.0 * eps)
+    spike_count = int((maxd > spike_thresh).sum())
+    return idle_ratio, spike_count
+
+
+def _r6(x) -> float:
+    return round(float(x), 6)
+
+
+def fingerprint(ep) -> tuple:
+    """Hashable summary of an episode used to detect 'changed' episodes.
+
+    Two episodes with the same index but different fingerprints changed
+    between dataset revisions.
+    """
+    def rl(vals):
+        return tuple(_r6(v) for v in vals) if vals is not None else None
+
+    return (
+        ep.length,
+        tuple(sorted(ep.tasks)),
+        rl(ep.action_min),
+        rl(ep.action_max),
+        rl(ep.action_mean),
+        round(ep.idle_ratio, 4) if ep.idle_ratio is not None else None,
+        ep.spike_count,
+    )

robot_data_diff-0.1.0/robot_data_diff.egg-info/PKG-INFO

@@ -0,0 +1,127 @@
+Metadata-Version: 2.4
+Name: robot-data-diff
+Version: 0.1.0
+Summary: Episode-aware diff for LeRobot robot-training datasets: what changed in the data since the last good run?
+License-Expression: Apache-2.0
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: pyarrow>=12
+Requires-Dist: pandas>=2
+Requires-Dist: numpy>=1.24
+Requires-Dist: huggingface_hub>=0.20
+
+# robot-data-diff
+
+**Episode-aware diff for LeRobot robot-training datasets.**
+
+You retrained the policy and it got worse. Was it the code — or the data?
+`robot-data-diff` answers *"what changed in the data since the last good run?"*
+by diffing two revisions of a dataset, episode by episode.
+
+Free, CPU-only, pure Python. No GPU, no uploads, no accounts needed.
+
+## Install
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e .
+```
+
+Dependencies (all free): `pyarrow`, `pandas`, `numpy`, `huggingface_hub`.
+
+> Note: this sandbox's egress proxy breaks `httpx`'s `no_proxy` parsing
+> (IPv6 entries). The tool strips `no_proxy`/`NO_PROXY` for its own
+> Hugging Face calls; your shell is untouched.
+
+## Usage
+
+```bash
+# two local dataset directories
+robot-data-diff diff ./my_dataset_v1 ./my_dataset_v2
+
+# Hugging Face ids (videos are never downloaded — meta + data parquet only)
+robot-data-diff diff lerobot/pusht ./my_local_copy
+
+# same repo, two revisions
+robot-data-diff diff lerobot/pusht@main lerobot/pusht@<commit-sha>
+
+# link the diff to a policy checkpoint, and write an HTML report
+robot-data-diff diff v1 v2 --checkpoint ckpt-42 --html report.html
+
+# metadata-only mode (skips per-episode quality stats; fastest)
+robot-data-diff diff v1 v2 --no-quality
+```
+
+Each side can be a local path or a Hugging Face dataset id
+(`org/repo`, optionally `org/repo@revision` or `hf://org/repo@revision`).
+
+## What the report shows
+
+- **Episode inventory:** total counts, episodes added / removed / changed
+  (same index, different data fingerprint)
+- **Per-episode deltas** for changed episodes: frame count, action
+  min/max/mean per dimension, idle-frame-ratio delta, action-spike delta
+- **Task inventory:** episodes per task, old → new
+- **Schema drift:** features added / removed / changed
+- **Quality summary:** mean idle-frame ratio and total action spikes,
+  old → new (lightweight CPU heuristics, not a full validator)
+
+## Supported formats
+
+- **LeRobot v3.0** — verified against the real `lerobot/pusht` dataset
+  (parquet episode metadata + data chunks + `meta/info.json`).
+- **LeRobot v2.1** — implemented against the documented v2.1 layout
+  (`meta/episodes.jsonl`, `meta/episodes_stats.jsonl`, per-episode data
+  files). Tested with a synthetic v2 fixture (`tests/make_v2_fixture.py`);
+  no public v2 dataset was reachable at build time, so treat v2 support
+  as beta and report issues.
+
+## Example
+
+`demo/diff-demo-output.txt` is the real output of diffing `lerobot/pusht`
+(206 episodes) against a modified copy: episodes 200–205 removed, episodes
+0–4 had their actions scaled ×1.5, episode 206 added. The tool reports
+exactly that:
+
+```
+Episodes: 206 -> 201 (+1 added, -6 removed, ~5 changed, =195 unchanged)
+  added:   [206]
+  removed: [200, 201, 202, 203, 204, 205]
+
+CHANGED EPISODES (5):
+  [ep 0] length 161 -> 161 (+0 frames)
+           d(action_min)  [+46.5000, +35.5000]
+           d(action_max)  [+187.5000, +224.5000]
+           ...
+```
+
+See `demo/diff-demo-report.html` for the HTML version.
+
+## Reproduce the demo
+
+```bash
+source .venv/bin/activate
+python tests/make_modified_fixture.py      # builds testdata/pusht_v2
+robot-data-diff diff testdata/pusht testdata/pusht_v2 \
+    --checkpoint ckpt-42 --html demo/diff-demo-report.html
+```
+
+## Honest limits
+
+- This is a **diff**, not a validator: it tells you *what changed* between
+  revisions, not whether your data is good. (The validator lane already has
+  6+ tools — that's why this exists.)
+- "Changed" is detected via a stats fingerprint (length, action
+  min/max/mean, idle ratio, spike count). Bit-identical data with reordered
+  episodes shows as added+removed, not moved — episode identity is the index.
+- Remote datasets download `meta/**` + `data/**`; very large datasets will
+  take a while on first run (cached afterwards via `--cache-dir`).
+- RLDS / Zarr / HDF5 / rosbag support is out of scope for v0.1 (LeRobot only).
+
+## Related work
+
+- [openbot-data](https://github.com/openbotai/openbot-data) (MIT) — a broader robot-data toolkit whose `openbot-data diff` compares two *snapshot files* (`snapshot` first, then `diff`, with `--fail-on breaking` for CI-style breaking-change detection). robot-data-diff is narrower on purpose: it diffs two dataset revisions *directly* — local dirs or Hugging Face ids, no snapshot step — episode by episode, with per-episode stat deltas.
+
+## License
+
+Apache-2.0

robot_data_diff-0.1.0/robot_data_diff.egg-info/SOURCES.txt

@@ -0,0 +1,15 @@
+README.md
+pyproject.toml
+robot_data_diff/__init__.py
+robot_data_diff/__main__.py
+robot_data_diff/cli.py
+robot_data_diff/differ.py
+robot_data_diff/loader.py
+robot_data_diff/report.py
+robot_data_diff/stats.py
+robot_data_diff.egg-info/PKG-INFO
+robot_data_diff.egg-info/SOURCES.txt
+robot_data_diff.egg-info/dependency_links.txt
+robot_data_diff.egg-info/entry_points.txt
+robot_data_diff.egg-info/requires.txt
+robot_data_diff.egg-info/top_level.txt

robot_data_diff-0.1.0/robot_data_diff.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

robot_data_diff-0.1.0/robot_data_diff.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+robot-data-diff = robot_data_diff.cli:main

robot_data_diff-0.1.0/robot_data_diff.egg-info/requires.txt

@@ -0,0 +1,4 @@
+pyarrow>=12
+pandas>=2
+numpy>=1.24
+huggingface_hub>=0.20

robot_data_diff-0.1.0/robot_data_diff.egg-info/top_level.txt

@@ -0,0 +1 @@
+robot_data_diff

robot_data_diff-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+