phys-ai-dataconverter 0.1.0__tar.gz

phys_ai_dataconverter-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Anand Krishniyer
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

phys_ai_dataconverter-0.1.0/PKG-INFO

@@ -0,0 +1,138 @@
+Metadata-Version: 2.4
+Name: phys-ai-dataconverter
+Version: 0.1.0
+Summary: Convert robot recordings (ROS 2 MCAP) into training datasets (LeRobot v3): cameras, joint state and actions aligned by capture time
+Author: Anand Krishniyer
+Author-email: akrishniyer@gmail.com
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/akrishni/phys-ai-dataconverter
+Project-URL: Documentation, https://github.com/akrishni/phys-ai-dataconverter#readme
+Project-URL: Issues, https://github.com/akrishni/phys-ai-dataconverter/issues
+Keywords: robotics,dataset,mcap,ros2,rosbag,lerobot,imitation-learning,conversion
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: phys-ai-dataprofiler>=0.4.0
+Requires-Dist: mcap>=1.0
+Requires-Dist: mcap-ros2-support>=0.5
+Requires-Dist: av>=12
+Requires-Dist: numpy
+Requires-Dist: pyarrow>=14
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Dynamic: license-file
+
+# phys-ai-dataconverter
+
+Convert robot recordings into datasets you can train on. Version 0.1 turns ROS 2 teleoperation recordings
+(MCAP) into [LeRobot](https://github.com/huggingface/lerobot) v3 datasets. It pairs camera frames, joint
+state and the operator's commands by capture time, and refuses to fill gaps with stale values.
+
+```bash
+pip install phys-ai-dataconverter
+phys-ai-dataconverter recordings/ -o ./pick_bottles --task "pick up the bottles" --check preset:teleop-imitation
+```
+
+Each `.mcap` file becomes one episode. `--check` then profiles the result's data table and videos with
+[phys-ai-dataprofiler](https://github.com/akrishni/phys-ai-dataprofiler) and checks it against a
+requirements file or profiler preset, saving the report in `meta/check.md`. The output works unchanged with LeRobot's `LeRobotDataset` and `lerobot-train`.
+
+## What goes in
+
+The camera's frames are the timeline, one row per frame:
+
+| Column | From | Default |
+|---|---|---|
+| `observation.images.<camera>` | `sensor_msgs/CompressedImage` (JPEG, PNG) or raw `sensor_msgs/Image`, encoded to AV1 | the highest-resolution camera; `--camera` |
+| `observation.state` | `sensor_msgs/JointState` positions | the joints the action commands; `--state-joints` |
+| `action` | `trajectory_msgs/JointTrajectory` commands (the teleop leader), concatenated | every such topic; `--action-topic`, `--no-action` |
+
+The camera name comes from the topic: `/zed/zed_node/left/image_rect_color/compressed` becomes
+`observation.images.zed_left`, or set it with `--camera-key`. The topics are chosen from the first file and
+printed, and every later file must have the same ones.
+
+## Alignment
+
+Each frame takes the joint reading and the command nearest to it in time, within half a frame interval
+(`--tolerance-ms`). Leading or trailing frames with no match yet are dropped. A frame in the middle with no
+match stops the conversion with the time and size of the gap. That's a gap in the recording, so the frame
+isn't filled with a stale value. Raise `--tolerance-ms` to accept it.
+
+**Clocks.** A ROS message has two times: when the sensor captured it (`header.stamp`) and when the
+recorder wrote it (MCAP `log_time`). On real robots a camera frame reaches the recorder about 25 ms after
+capture and a joint reading about 1 ms after, so pairing on write time matches each frame with the arm's
+position a frame later. The converter pairs on capture time. Teleop commands usually carry no capture
+time, so they're placed on the capture timeline using the joint topic's recorder delay. `--clock log`
+pairs on write time; `--clock header` requires a capture time on every topic.
+
+## What comes out
+
+LeRobot v3.0, as written by lerobot 0.4 and later:
+
+```
+data/chunk-000/file-000.parquet            observation.state, action, timestamp, frame_index, episode_index, index, task_index
+videos/<camera>/chunk-000/file-000.mp4     all episodes back to back, AV1, a keyframe every 2 frames
+meta/info.json  stats.json  tasks.parquet  episodes/chunk-000/file-000.parquet
+meta/conversion.json                       sources, topics, clock, worst match per episode, notes
+```
+
+`meta/conversion.json` also records what the converter noticed: dropped camera frames, a camera rate
+that differs from the dataset fps, or a robot clock that isn't synchronised with the recorder (two of
+the public test recordings below are 73 days off). The output is built in `<output>.partial` and renamed
+when complete, so a failed run leaves nothing behind. `--overwrite` only ever replaces a folder that is
+already a LeRobot dataset.
+
+## Tested on
+
+`ola31/readme_test_rosbag_1` and `Dongkkka/physical_ai_tools_test_rosbag` on the Hugging Face Hub, a
+two-arm teleop robot: 3 recordings, 1,777 frames at 30 fps, 19 joints.
+- **Values:** every state and action value equals the source message.
+- **Timing:** each frame's joint reading is within 5 ms of the frame's capture time.
+- **Video:** video frames are in source order.
+- **LeRobot:** `LeRobotDataset` loads the result, and `lerobot-train` trains ACT on it.
+
+## Limits
+
+Version 0.1 handles one camera per dataset, ROS 2 MCAP input, and joint state and commands from
+`JointState` and `JointTrajectory`. Planned next: several cameras, ROS 1 `.bag` and ROS 2 `.db3`.
+
+## Development
+
+```bash
+git clone https://github.com/akrishni/phys-ai-dataconverter
+cd phys-ai-dataconverter
+pip install -e ".[dev]"
+python tests/run_all.py            # offline: MCAP files with known answers are written by the tests
+python tests/run_all.py --hub      # also convert a public recording from the Hub
+```
+
+```
+phys_ai_dataconverter/
+  model.py           Stream, image and topic types shared by everything below
+  readers/           one module per input format (mcap_ros2)
+  episode.py         one recording to one aligned episode: topic choice, clocks, alignment
+  align.py           nearest-timestamp matching with a tolerance
+  writers/           one module per output format (lerobot_v3)
+  convert.py         the pipeline: read, align, write to <output>.partial, rename
+  verify.py          --check: profile the output with phys-ai-dataprofiler and evaluate requirements
+  cli.py             the phys-ai-dataconverter command
+```
+
+A new input format is a new module in `readers/` that produces `Stream`s. A new output format is a new
+module in `writers/`. Nothing else changes.
+
+## License
+
+MIT License, copyright (c) 2026 Anand Krishniyer. See [LICENSE](LICENSE).

phys_ai_dataconverter-0.1.0/README.md

@@ -0,0 +1,102 @@
+# phys-ai-dataconverter
+
+Convert robot recordings into datasets you can train on. Version 0.1 turns ROS 2 teleoperation recordings
+(MCAP) into [LeRobot](https://github.com/huggingface/lerobot) v3 datasets. It pairs camera frames, joint
+state and the operator's commands by capture time, and refuses to fill gaps with stale values.
+
+```bash
+pip install phys-ai-dataconverter
+phys-ai-dataconverter recordings/ -o ./pick_bottles --task "pick up the bottles" --check preset:teleop-imitation
+```
+
+Each `.mcap` file becomes one episode. `--check` then profiles the result's data table and videos with
+[phys-ai-dataprofiler](https://github.com/akrishni/phys-ai-dataprofiler) and checks it against a
+requirements file or profiler preset, saving the report in `meta/check.md`. The output works unchanged with LeRobot's `LeRobotDataset` and `lerobot-train`.
+
+## What goes in
+
+The camera's frames are the timeline, one row per frame:
+
+| Column | From | Default |
+|---|---|---|
+| `observation.images.<camera>` | `sensor_msgs/CompressedImage` (JPEG, PNG) or raw `sensor_msgs/Image`, encoded to AV1 | the highest-resolution camera; `--camera` |
+| `observation.state` | `sensor_msgs/JointState` positions | the joints the action commands; `--state-joints` |
+| `action` | `trajectory_msgs/JointTrajectory` commands (the teleop leader), concatenated | every such topic; `--action-topic`, `--no-action` |
+
+The camera name comes from the topic: `/zed/zed_node/left/image_rect_color/compressed` becomes
+`observation.images.zed_left`, or set it with `--camera-key`. The topics are chosen from the first file and
+printed, and every later file must have the same ones.
+
+## Alignment
+
+Each frame takes the joint reading and the command nearest to it in time, within half a frame interval
+(`--tolerance-ms`). Leading or trailing frames with no match yet are dropped. A frame in the middle with no
+match stops the conversion with the time and size of the gap. That's a gap in the recording, so the frame
+isn't filled with a stale value. Raise `--tolerance-ms` to accept it.
+
+**Clocks.** A ROS message has two times: when the sensor captured it (`header.stamp`) and when the
+recorder wrote it (MCAP `log_time`). On real robots a camera frame reaches the recorder about 25 ms after
+capture and a joint reading about 1 ms after, so pairing on write time matches each frame with the arm's
+position a frame later. The converter pairs on capture time. Teleop commands usually carry no capture
+time, so they're placed on the capture timeline using the joint topic's recorder delay. `--clock log`
+pairs on write time; `--clock header` requires a capture time on every topic.
+
+## What comes out
+
+LeRobot v3.0, as written by lerobot 0.4 and later:
+
+```
+data/chunk-000/file-000.parquet            observation.state, action, timestamp, frame_index, episode_index, index, task_index
+videos/<camera>/chunk-000/file-000.mp4     all episodes back to back, AV1, a keyframe every 2 frames
+meta/info.json  stats.json  tasks.parquet  episodes/chunk-000/file-000.parquet
+meta/conversion.json                       sources, topics, clock, worst match per episode, notes
+```
+
+`meta/conversion.json` also records what the converter noticed: dropped camera frames, a camera rate
+that differs from the dataset fps, or a robot clock that isn't synchronised with the recorder (two of
+the public test recordings below are 73 days off). The output is built in `<output>.partial` and renamed
+when complete, so a failed run leaves nothing behind. `--overwrite` only ever replaces a folder that is
+already a LeRobot dataset.
+
+## Tested on
+
+`ola31/readme_test_rosbag_1` and `Dongkkka/physical_ai_tools_test_rosbag` on the Hugging Face Hub, a
+two-arm teleop robot: 3 recordings, 1,777 frames at 30 fps, 19 joints.
+- **Values:** every state and action value equals the source message.
+- **Timing:** each frame's joint reading is within 5 ms of the frame's capture time.
+- **Video:** video frames are in source order.
+- **LeRobot:** `LeRobotDataset` loads the result, and `lerobot-train` trains ACT on it.
+
+## Limits
+
+Version 0.1 handles one camera per dataset, ROS 2 MCAP input, and joint state and commands from
+`JointState` and `JointTrajectory`. Planned next: several cameras, ROS 1 `.bag` and ROS 2 `.db3`.
+
+## Development
+
+```bash
+git clone https://github.com/akrishni/phys-ai-dataconverter
+cd phys-ai-dataconverter
+pip install -e ".[dev]"
+python tests/run_all.py            # offline: MCAP files with known answers are written by the tests
+python tests/run_all.py --hub      # also convert a public recording from the Hub
+```
+
+```
+phys_ai_dataconverter/
+  model.py           Stream, image and topic types shared by everything below
+  readers/           one module per input format (mcap_ros2)
+  episode.py         one recording to one aligned episode: topic choice, clocks, alignment
+  align.py           nearest-timestamp matching with a tolerance
+  writers/           one module per output format (lerobot_v3)
+  convert.py         the pipeline: read, align, write to <output>.partial, rename
+  verify.py          --check: profile the output with phys-ai-dataprofiler and evaluate requirements
+  cli.py             the phys-ai-dataconverter command
+```
+
+A new input format is a new module in `readers/` that produces `Stream`s. A new output format is a new
+module in `writers/`. Nothing else changes.
+
+## License
+
+MIT License, copyright (c) 2026 Anand Krishniyer. See [LICENSE](LICENSE).

phys_ai_dataconverter-0.1.0/phys_ai_dataconverter/__init__.py

@@ -0,0 +1,8 @@
+"""phys_ai_dataconverter: convert robot recordings (ROS 2 MCAP) into training datasets (LeRobot v3).
+
+Copyright (c) 2026 Anand Krishniyer. MIT License."""
+
+__version__ = "0.1.0"
+
+from phys_ai_dataconverter.model import ConversionError  # noqa: E402,F401
+from phys_ai_dataconverter.convert import ConversionReport, ConvertOptions, convert  # noqa: E402,F401

phys_ai_dataconverter-0.1.0/phys_ai_dataconverter/__main__.py

@@ -0,0 +1,5 @@
+import sys
+
+from phys_ai_dataconverter.cli import main
+
+sys.exit(main())

phys_ai_dataconverter-0.1.0/phys_ai_dataconverter/align.py

@@ -0,0 +1,45 @@
+"""
+Nearest-timestamp alignment of one stream onto the ticks of another.
+
+Each reference tick (a camera frame) takes the source sample (a joint reading) closest to it in time.
+The distance is kept, and a tick whose nearest sample is further than the tolerance gets no value
+rather than a stale one: the caller decides whether to drop the frame or fail.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+import numpy as np
+
+
+@dataclass
+class Alignment:
+    index: np.ndarray       # per tick: index of the nearest sample, -1 when outside the tolerance
+    age_ns: np.ndarray      # per tick: |tick - sample time| in ns (of the nearest sample, even when rejected)
+    matched: np.ndarray     # per tick: True when within the tolerance
+
+    @property
+    def n_unmatched(self) -> int:
+        return int((~self.matched).sum())
+
+
+def nearest(ticks_ns: np.ndarray, sample_ns: np.ndarray, tolerance_ns: int) -> Alignment:
+    """Match every tick to the nearest sample. Both arrays must be sorted ascending; ties go to the
+    earlier sample, so a value is never taken from after it could have been known when both are
+    equally close."""
+    ticks_ns = np.asarray(ticks_ns, dtype=np.int64)
+    sample_ns = np.asarray(sample_ns, dtype=np.int64)
+    if len(sample_ns) == 0:
+        n = len(ticks_ns)
+        return Alignment(np.full(n, -1), np.full(n, np.iinfo(np.int64).max), np.zeros(n, dtype=bool))
+    if np.any(np.diff(sample_ns) < 0) or np.any(np.diff(ticks_ns) < 0):
+        raise ValueError("timestamps must be sorted")
+    right = np.searchsorted(sample_ns, ticks_ns, side="left").clip(0, len(sample_ns) - 1)
+    left = (right - 1).clip(0, len(sample_ns) - 1)
+    d_right = np.abs(sample_ns[right] - ticks_ns)
+    d_left = np.abs(ticks_ns - sample_ns[left])
+    idx = np.where(d_left <= d_right, left, right)
+    age = np.minimum(d_left, d_right)
+    ok = age <= tolerance_ns
+    return Alignment(np.where(ok, idx, -1), age, ok)

phys_ai_dataconverter-0.1.0/phys_ai_dataconverter/cli.py

@@ -0,0 +1,117 @@
+"""
+phys-ai-dataconverter: convert robot recordings into training datasets.
+
+    phys-ai-dataconverter recordings/ -o ./pick_bottles --task "pick up the bottles"
+    phys-ai-dataconverter rec1.mcap rec2.mcap -o ./ds --check preset:teleop-imitation
+
+Exit codes: 0 converted (and, with --check, the check passed), 1 the --check failed,
+2 the conversion was refused (the message says why), 4 the --check was unknown.
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+from phys_ai_dataconverter import __version__
+
+PROG = "phys-ai-dataconverter"
+
+
+def build_parser() -> argparse.ArgumentParser:
+    p = argparse.ArgumentParser(
+        prog=PROG,
+        description="Convert robot recordings (one episode per file) into a LeRobot v3 dataset: one camera, "
+                    "observation.state from a joint state topic and action from joint command topics, aligned "
+                    "to the camera frames by nearest capture time. Input: ROS 2 MCAP.",
+        epilog="Example: phys-ai-dataconverter recordings/ -o ./pick_bottles --task \"pick up the bottles\" "
+               "--camera /zed/zed_node/left/image_rect_color/compressed --check preset:teleop-imitation",
+    )
+    p.add_argument("--version", action="version", version=f"{PROG} {__version__}")
+    p.add_argument("inputs", nargs="+", help="Recording files, or folders searched for them (one episode per file)")
+    p.add_argument("--output", "-o", required=True, help="Folder for the new dataset")
+    p.add_argument("--task", default="teleoperation", help="Task description stored with every episode")
+    s = p.add_argument_group("streams (defaults are picked from the first file and printed)")
+    s.add_argument("--camera", help="Image topic (default: the highest-resolution one)")
+    s.add_argument("--camera-key", help="Feature name (default: observation.images.<from the topic>)")
+    s.add_argument("--state-topic", help="Joint state topic (default: the busiest one)")
+    s.add_argument("--state-joints", help="Comma-separated joint names for observation.state "
+                                          "(default: the joints the action topics command)")
+    s.add_argument("--action-topic", action="append", dest="action_topics",
+                   help="Joint command topic; repeat for several, concatenated in order (default: all, sorted)")
+    s.add_argument("--no-action", action="store_true", help="Write no action column")
+    t = p.add_argument_group("timing")
+    t.add_argument("--fps", type=int, help="Dataset frame rate (default: the camera's measured rate, rounded)")
+    t.add_argument("--tolerance-ms", type=float,
+                   help="Largest allowed distance from a frame to its matched state/command (default: half a frame)")
+    t.add_argument("--clock", choices=["auto", "header", "log"], default="auto",
+                   help="auto (default): pair camera and state on capture time (header.stamp), placing unstamped "
+                        "commands via the state topic's recorder delay; header: capture time for every topic; "
+                        "log: recorder write time for every topic")
+    v = p.add_argument_group("output")
+    v.add_argument("--format", dest="output_format", default="lerobot-v3", choices=["lerobot-v3"])
+    v.add_argument("--vcodec", default="libsvtav1", help="libsvtav1 (LeRobot's default) or h264 (faster)")
+    v.add_argument("--crf", type=int, default=30, help="Video quality; lower is better and larger (default 30)")
+    v.add_argument("--robot-type", help="Stored in info.json")
+    v.add_argument("--overwrite", action="store_true", help="Replace an existing dataset at --output")
+    p.add_argument("--check", metavar="FILE|preset:NAME",
+                   help="After converting, profile the output with phys-ai-dataprofiler and check it against "
+                        "these requirements (a YAML file or a profiler preset); the report goes in meta/")
+    p.add_argument("--quiet", action="store_true")
+    return p
+
+
+def _inputs(args: list[str]) -> list[Path]:
+    from phys_ai_dataconverter.readers import input_extensions
+    exts = input_extensions()
+    files: list[Path] = []
+    for i in args:
+        q = Path(i)
+        if q.is_dir():
+            found = sorted(f for f in q.rglob("*") if f.is_file() and f.suffix.lower() in exts)
+            if not found:
+                raise SystemExit(f"Error: no {' '.join(exts)} files under {q}")
+            files += found
+        else:
+            files.append(q)
+    return files
+
+
+def main(argv: list[str] | None = None) -> int:
+    a = build_parser().parse_args(argv)
+    log = (lambda m: None) if a.quiet else (lambda m: print(m, file=sys.stderr))
+    from phys_ai_dataconverter.convert import ConvertOptions, convert, print_report
+    from phys_ai_dataconverter.model import ConversionError
+    try:
+        files = _inputs(a.inputs)
+        report = convert(files, a.output, ConvertOptions(
+            camera=a.camera, camera_key=a.camera_key, state_topic=a.state_topic,
+            state_joints=[j.strip() for j in a.state_joints.split(",")] if a.state_joints else None,
+            action_topics=a.action_topics, no_action=a.no_action, task=a.task, fps=a.fps,
+            tolerance_ms=a.tolerance_ms, clock=a.clock, output_format=a.output_format, vcodec=a.vcodec,
+            crf=a.crf, robot_type=a.robot_type, overwrite=a.overwrite), log=log)
+    except ConversionError as exc:
+        print(f"Error: {exc}", file=sys.stderr)
+        return 2
+    except SystemExit as exc:
+        if isinstance(exc.code, str):
+            print(exc.code, file=sys.stderr)
+            return 2
+        raise
+    print_report(report, log)
+    if not a.check:
+        log(f"Check it: {PROG} ... --check preset:teleop-imitation (or a requirements file)")
+        return 0
+    from phys_ai_dataconverter.verify import check
+    log("")
+    log(f"Checking {a.output} against {a.check} with phys-ai-dataprofiler")
+    try:
+        return check(a.output, a.check, log if not a.quiet else (lambda m: None))
+    except ConversionError as exc:
+        print(f"Error: {exc}", file=sys.stderr)
+        return 2
+
+
+if __name__ == "__main__":
+    sys.exit(main())