inspect-robots-so101 0.3.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- inspect_robots_so101/CLAUDE.md +36 -0
- inspect_robots_so101/__init__.py +38 -0
- inspect_robots_so101/config.py +180 -0
- inspect_robots_so101/embodiment.py +236 -0
- inspect_robots_so101/operator.py +51 -0
- inspect_robots_so101/packing.py +86 -0
- inspect_robots_so101/policy.py +224 -0
- inspect_robots_so101/preflight.py +95 -0
- inspect_robots_so101/py.typed +0 -0
- inspect_robots_so101-0.3.0.dist-info/METADATA +209 -0
- inspect_robots_so101-0.3.0.dist-info/RECORD +14 -0
- inspect_robots_so101-0.3.0.dist-info/WHEEL +4 -0
- inspect_robots_so101-0.3.0.dist-info/entry_points.txt +8 -0
- inspect_robots_so101-0.3.0.dist-info/licenses/LICENSE +21 -0
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# `inspect_robots_so101` package — module map
|
|
2
|
+
|
|
3
|
+
Two Inspect Robots components + the glue to make them an honest, testable, safe pair.
|
|
4
|
+
The package is `mypy --strict` clean, ships `py.typed`, and is 100%-covered.
|
|
5
|
+
|
|
6
|
+
## Modules
|
|
7
|
+
|
|
8
|
+
| Module | Responsibility |
|
|
9
|
+
|--------|----------------|
|
|
10
|
+
| `packing.py` | **Pure** 6-D SO-ARM packing — the single source of truth for the motor order (`shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper`) and how the flat vector maps to LeRobot's `"<motor>.pos"` dicts. `to_action_dict`/`from_obs_dict`/`validate_dim`, `STATE_KEY`, `STATE_SPEC`, `MOTORS`. No optional deps. |
|
|
11
|
+
| `config.py` | `SOArmConfig` / `LeRobotPolicyConfig` (frozen, `from_kwargs` for CLI scalars) + shared `action_box()` / `observation_space()` / `ACTION_SEMANTICS` so both components declare an **identical** contract. |
|
|
12
|
+
| `operator.py` | `OperatorIO` (injectable stdin/stdout) for readiness + success prompts; `default_poll_end` (real TTY poll, `# pragma: no cover`). |
|
|
13
|
+
| `policy.py` | `LeRobotPolicy` — wraps a LeRobot checkpoint. `act()` builds the RAW robot payload (`"<motor>.pos"` floats, frames keyed by camera name, `task`), runs the injectable `predict_fn`, truncates to `chunk_size`, returns an `ActionChunk`. Real in-process inference is `_default_predict` (lazy torch + lerobot; obs prep via lerobot's own `raw_observation_to_observation`) — NOT pragma'd: it is covered by `sys.modules`-fake tests, and its real import paths are validated by the `lerobot-seam` CI job. |
|
|
14
|
+
| `embodiment.py` | `SOArmEmbodiment` — LeRobot SO follower driver. Clamp backstop, optional delta→abs, `SELF_PACED` pacing, operator-keypress success, context manager (`with emb:` guarantees `close()`; `close()` clears the handle even if disconnect raises). The driver's `get_observation` yields motor positions **and** cameras. Hardware seam (`_default_driver_factory`) is injected/pragma'd; it connects with `calibrate=False` and fails fast via the tested `_check_calibrated` (never lerobot's blocking interactive calibration). |
|
|
15
|
+
| `preflight.py` | `build` / `run_preflight` + the `inspect-robots-so101-preflight` CLI: run the compat check, print, exit non-zero on errors. |
|
|
16
|
+
| `__init__.py` | Public API fenced by `__all__` (guarded by `tests/test_api_snapshot.py`). |
|
|
17
|
+
|
|
18
|
+
## Key invariants
|
|
19
|
+
|
|
20
|
+
- **Contract symmetry:** policy and embodiment build their `action_space` /
|
|
21
|
+
`observation_space` from the *same* `config.py` helpers. If you change the dim,
|
|
22
|
+
semantics, camera names, or state key, change them there once — not in two
|
|
23
|
+
places — or compat breaks.
|
|
24
|
+
- **Construction is inert:** `__init__` touches no hardware/model/stdin (only
|
|
25
|
+
`.info`). The driver connects, and the model loads, lazily on first use. This is
|
|
26
|
+
what lets the registry (`factories[name]()`) and preflight construct components
|
|
27
|
+
freely — and keeps `import inspect_robots_so101` free of torch.
|
|
28
|
+
- **Coverage discipline:** the only uncoverable code is hardware/TTY I/O,
|
|
29
|
+
isolated in `# pragma: no cover` seams (`_default_driver_factory`,
|
|
30
|
+
`default_poll_end`, the `_require_driver` pre-reset guard, `__main__`).
|
|
31
|
+
`_default_predict` is NOT pragma'd — it is exercised with `sys.modules` fakes
|
|
32
|
+
(see `tests/test_policy.py`), and the `lerobot-seam` CI job imports the real
|
|
33
|
+
lerobot symbols it relies on. Prefer that pattern (fake the modules, assert the
|
|
34
|
+
wiring, guard the imports in CI) over new pragmas.
|
|
35
|
+
- **Safety lives in `step()`**, not in an optional Approver — see the root
|
|
36
|
+
`CLAUDE.md`.
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
"""inspect-robots-so101 — Inspect Robots adapters for LeRobot SO-ARM followers + LeRobot policies.
|
|
2
|
+
|
|
3
|
+
Registers two Inspect Robots components via entry points:
|
|
4
|
+
|
|
5
|
+
* embodiment ``so_arm`` — :class:`~inspect_robots_so101.embodiment.SOArmEmbodiment`
|
|
6
|
+
* policy ``lerobot`` — :class:`~inspect_robots_so101.policy.LeRobotPolicy`
|
|
7
|
+
|
|
8
|
+
so ``inspect-robots run --task cubepick-reach --policy lerobot --embodiment so_arm``
|
|
9
|
+
works once both packages are installed. Use
|
|
10
|
+
:func:`~inspect_robots_so101.preflight.run_preflight` (or the ``inspect-robots-so101-preflight``
|
|
11
|
+
CLI) to verify compatibility before any motion.
|
|
12
|
+
"""
|
|
13
|
+
|
|
14
|
+
from __future__ import annotations
|
|
15
|
+
|
|
16
|
+
from inspect_robots_so101.config import LeRobotPolicyConfig, SOArmConfig
|
|
17
|
+
from inspect_robots_so101.embodiment import SOArmEmbodiment
|
|
18
|
+
from inspect_robots_so101.operator import OperatorIO
|
|
19
|
+
from inspect_robots_so101.packing import MOTORS, STATE_KEY, TOTAL_DIM, from_obs_dict, to_action_dict
|
|
20
|
+
from inspect_robots_so101.policy import LeRobotPolicy
|
|
21
|
+
from inspect_robots_so101.preflight import build, run_preflight
|
|
22
|
+
|
|
23
|
+
__version__ = "0.3.0"
|
|
24
|
+
|
|
25
|
+
__all__ = [
|
|
26
|
+
"MOTORS",
|
|
27
|
+
"STATE_KEY",
|
|
28
|
+
"TOTAL_DIM",
|
|
29
|
+
"LeRobotPolicy",
|
|
30
|
+
"LeRobotPolicyConfig",
|
|
31
|
+
"OperatorIO",
|
|
32
|
+
"SOArmConfig",
|
|
33
|
+
"SOArmEmbodiment",
|
|
34
|
+
"build",
|
|
35
|
+
"from_obs_dict",
|
|
36
|
+
"run_preflight",
|
|
37
|
+
"to_action_dict",
|
|
38
|
+
]
|
|
@@ -0,0 +1,180 @@
|
|
|
1
|
+
"""Configuration for the SO-ARM embodiment and the LeRobot policy client.
|
|
2
|
+
|
|
3
|
+
Both configs are frozen dataclasses with defaults that match a stock LeRobot SO
|
|
4
|
+
follower (SO-100 / SO-101) and a SmolVLA checkpoint, so zero-arg construction
|
|
5
|
+
"just works" for `.info` / preflight. Each exposes :meth:`from_kwargs` so the
|
|
6
|
+
adapters accept flat scalar keyword arguments — this is what lets
|
|
7
|
+
``inspect-robots run -P pretrained_path=... -E port=...`` configure them, since the
|
|
8
|
+
Inspect Robots CLI only forwards scalar ``key=value`` pairs.
|
|
9
|
+
"""
|
|
10
|
+
|
|
11
|
+
from __future__ import annotations
|
|
12
|
+
|
|
13
|
+
import dataclasses
|
|
14
|
+
from dataclasses import dataclass
|
|
15
|
+
from typing import Any, TypeVar
|
|
16
|
+
|
|
17
|
+
import numpy as np
|
|
18
|
+
import numpy.typing as npt
|
|
19
|
+
from inspect_robots.spaces import (
|
|
20
|
+
ActionSemantics,
|
|
21
|
+
Box,
|
|
22
|
+
CameraSpec,
|
|
23
|
+
ObservationSpace,
|
|
24
|
+
)
|
|
25
|
+
|
|
26
|
+
from inspect_robots_so101.packing import NUM_JOINTS, STATE_KEY, STATE_SPEC, TOTAL_DIM
|
|
27
|
+
|
|
28
|
+
_T = TypeVar("_T", bound="_FromKwargs")
|
|
29
|
+
|
|
30
|
+
# Conservative default action limits: the five revolute joints in [-180, 180]
|
|
31
|
+
# degrees, the gripper in [0, 100]. These are SAFETY limits — override with your
|
|
32
|
+
# real, calibrated SO-ARM joint limits before trusting them on hardware.
|
|
33
|
+
_DEFAULT_LOW: tuple[float, ...] = (-180.0,) * NUM_JOINTS + (0.0,)
|
|
34
|
+
_DEFAULT_HIGH: tuple[float, ...] = (180.0,) * NUM_JOINTS + (100.0,)
|
|
35
|
+
|
|
36
|
+
DEFAULT_CAMERAS: tuple[str, ...] = ("front",)
|
|
37
|
+
|
|
38
|
+
# The SO follower variants lerobot ships. At lerobot v0.5.x both names alias the
|
|
39
|
+
# SAME driver + config classes (SO100Follower = SO101Follower = SOFollower), so
|
|
40
|
+
# `robot_type` is a documented, validated label — it does not change runtime
|
|
41
|
+
# behavior. It exists so configs stay self-describing and so a future lerobot
|
|
42
|
+
# that splits the classes has an obvious wiring point.
|
|
43
|
+
VALID_ROBOT_TYPES: tuple[str, ...] = ("so101_follower", "so100_follower")
|
|
44
|
+
|
|
45
|
+
|
|
46
|
+
class _FromKwargs:
|
|
47
|
+
"""Mixin: build a frozen dataclass from flat scalar kwargs (CLI-friendly)."""
|
|
48
|
+
|
|
49
|
+
@classmethod
|
|
50
|
+
def from_kwargs(cls: type[_T], **flat: Any) -> _T:
|
|
51
|
+
names = {f.name for f in dataclasses.fields(cls)} # type: ignore[arg-type]
|
|
52
|
+
unknown = set(flat) - names
|
|
53
|
+
if unknown:
|
|
54
|
+
raise TypeError(f"{cls.__name__} got unexpected config keys: {sorted(unknown)}")
|
|
55
|
+
return cls(**flat)
|
|
56
|
+
|
|
57
|
+
|
|
58
|
+
@dataclass(frozen=True)
|
|
59
|
+
class SOArmConfig(_FromKwargs):
|
|
60
|
+
"""Static configuration for a single SO-ARM follower embodiment."""
|
|
61
|
+
|
|
62
|
+
port: str = "/dev/ttyACM0"
|
|
63
|
+
robot_type: str = "so101_follower" # or "so100_follower" (see VALID_ROBOT_TYPES)
|
|
64
|
+
# The lerobot robot id: selects the calibration file
|
|
65
|
+
# (<calibration_dir>/<robot_id>.json, written by `lerobot-calibrate`). Without
|
|
66
|
+
# it lerobot would look for "None.json" — set it to the id you calibrated with.
|
|
67
|
+
robot_id: str | None = None
|
|
68
|
+
# Where lerobot stores calibration files. None -> lerobot's default
|
|
69
|
+
# (~/.cache/huggingface/lerobot/calibration/robots/<robot_type>/).
|
|
70
|
+
calibration_dir: str | None = None
|
|
71
|
+
cameras: tuple[str, ...] = DEFAULT_CAMERAS
|
|
72
|
+
control_hz: float = 30.0
|
|
73
|
+
cam_height: int = 480
|
|
74
|
+
cam_width: int = 640
|
|
75
|
+
joint_low: tuple[float, ...] = _DEFAULT_LOW
|
|
76
|
+
joint_high: tuple[float, ...] = _DEFAULT_HIGH
|
|
77
|
+
home_pose: tuple[float, ...] | None = None
|
|
78
|
+
joints_are_delta: bool = False
|
|
79
|
+
use_degrees: bool = True
|
|
80
|
+
max_relative_target: float | None = None
|
|
81
|
+
disable_torque_on_disconnect: bool = True
|
|
82
|
+
# LeRobot CameraConfig objects keyed by camera name, used only by the default
|
|
83
|
+
# (hardware) driver factory. Opaque to this package; not CLI-settable. Their
|
|
84
|
+
# keys should match ``cameras``. ``None`` means "build the robot with no
|
|
85
|
+
# cameras" — fine for preflight, but a real run needs camera streams.
|
|
86
|
+
camera_configs: Any = None
|
|
87
|
+
|
|
88
|
+
def __post_init__(self) -> None:
|
|
89
|
+
for name in ("joint_low", "joint_high"):
|
|
90
|
+
if len(getattr(self, name)) != TOTAL_DIM:
|
|
91
|
+
raise ValueError(f"{name} must have {TOTAL_DIM} entries")
|
|
92
|
+
if self.home_pose is not None and len(self.home_pose) != TOTAL_DIM:
|
|
93
|
+
raise ValueError(f"home_pose must have {TOTAL_DIM} entries")
|
|
94
|
+
if self.robot_type not in VALID_ROBOT_TYPES:
|
|
95
|
+
raise ValueError(
|
|
96
|
+
f"robot_type must be one of {VALID_ROBOT_TYPES}, got {self.robot_type!r}"
|
|
97
|
+
)
|
|
98
|
+
if not self.use_degrees:
|
|
99
|
+
# Supporting normalized (+/-100) joints means deriving STATE_SPEC units
|
|
100
|
+
# and the clamp bounds from the config throughout packing/policy/
|
|
101
|
+
# embodiment; not implemented yet. Tracked as an issue.
|
|
102
|
+
raise ValueError(
|
|
103
|
+
"use_degrees=False is not supported: this package's state spec and "
|
|
104
|
+
"default joint limits assume degrees (lerobot's SO follower default). "
|
|
105
|
+
"Leave use_degrees=True, or file/upvote the issue for normalized units."
|
|
106
|
+
)
|
|
107
|
+
if self.home_pose is not None and self.max_relative_target is None:
|
|
108
|
+
# Homing sends ONE absolute command; without lerobot's
|
|
109
|
+
# max_relative_target slew limit the arm would slam to home at full
|
|
110
|
+
# speed from wherever it is. Interpolated homing is tracked as an issue.
|
|
111
|
+
raise ValueError(
|
|
112
|
+
"home_pose without max_relative_target would command a full-speed "
|
|
113
|
+
"jump to the home pose; set SOArmConfig.max_relative_target (degrees "
|
|
114
|
+
"per step) to slew-limit it, or unset home_pose"
|
|
115
|
+
)
|
|
116
|
+
|
|
117
|
+
@property
|
|
118
|
+
def low(self) -> npt.NDArray[np.float64]:
|
|
119
|
+
return np.asarray(self.joint_low, dtype=np.float64)
|
|
120
|
+
|
|
121
|
+
@property
|
|
122
|
+
def high(self) -> npt.NDArray[np.float64]:
|
|
123
|
+
return np.asarray(self.joint_high, dtype=np.float64)
|
|
124
|
+
|
|
125
|
+
|
|
126
|
+
@dataclass(frozen=True)
|
|
127
|
+
class LeRobotPolicyConfig(_FromKwargs):
|
|
128
|
+
"""Static configuration for a LeRobot policy loaded from a checkpoint."""
|
|
129
|
+
|
|
130
|
+
pretrained_path: str = "lerobot/smolvla_base"
|
|
131
|
+
policy_type: str = "smolvla" # act, smolvla, pi0, pi05, diffusion, ...
|
|
132
|
+
device: str = "cuda"
|
|
133
|
+
cameras: tuple[str, ...] = DEFAULT_CAMERAS
|
|
134
|
+
state_key: str = STATE_KEY
|
|
135
|
+
# Max actions consumed per inference: ``act()`` truncates the model's chunk to
|
|
136
|
+
# its first ``chunk_size`` actions (mirrors the async policy server's
|
|
137
|
+
# ``actions_per_chunk``) and advertises it as ``PolicyConfig.action_horizon``.
|
|
138
|
+
# Distinct from the framework-side ``DefaultController.replan_interval``, which
|
|
139
|
+
# caps how many actions of an already-returned chunk get executed per replan.
|
|
140
|
+
chunk_size: int = 50
|
|
141
|
+
cam_height: int = 480
|
|
142
|
+
cam_width: int = 640
|
|
143
|
+
|
|
144
|
+
def __post_init__(self) -> None:
|
|
145
|
+
if self.chunk_size < 1:
|
|
146
|
+
raise ValueError(f"chunk_size must be >= 1, got {self.chunk_size}")
|
|
147
|
+
|
|
148
|
+
|
|
149
|
+
# The action *semantics* both the policy and the embodiment declare. Compatibility
|
|
150
|
+
# checking compares control_mode + rotation_repr (errors) and gripper + frame
|
|
151
|
+
# (warnings); declaring this single constant on both sides guarantees a clean check.
|
|
152
|
+
ACTION_SEMANTICS = ActionSemantics(
|
|
153
|
+
control_mode="joint_pos",
|
|
154
|
+
rotation_repr="none",
|
|
155
|
+
gripper="continuous",
|
|
156
|
+
frame="base",
|
|
157
|
+
)
|
|
158
|
+
|
|
159
|
+
|
|
160
|
+
def camera_specs(height: int, width: int, names: tuple[str, ...]) -> tuple[CameraSpec, ...]:
|
|
161
|
+
"""Build CameraSpecs for the given names at one resolution (single source of truth)."""
|
|
162
|
+
return tuple(CameraSpec(name=n, height=height, width=width, channels=3) for n in names)
|
|
163
|
+
|
|
164
|
+
|
|
165
|
+
def action_box(
|
|
166
|
+
low: npt.NDArray[np.float64] | None = None,
|
|
167
|
+
high: npt.NDArray[np.float64] | None = None,
|
|
168
|
+
) -> Box:
|
|
169
|
+
"""The shared 6-D joint-position action space. ``low``/``high`` are optional
|
|
170
|
+
safety limits (the embodiment supplies them; the policy leaves them unset)."""
|
|
171
|
+
return Box(shape=(TOTAL_DIM,), low=low, high=high, semantics=ACTION_SEMANTICS)
|
|
172
|
+
|
|
173
|
+
|
|
174
|
+
def observation_space(height: int, width: int, names: tuple[str, ...]) -> ObservationSpace:
|
|
175
|
+
"""The shared observation space: the configured cameras + packed 6-D ``joint_pos``."""
|
|
176
|
+
return ObservationSpace(
|
|
177
|
+
cameras=camera_specs(height, width, names),
|
|
178
|
+
state_keys=frozenset({STATE_KEY}),
|
|
179
|
+
state=STATE_SPEC,
|
|
180
|
+
)
|
|
@@ -0,0 +1,236 @@
|
|
|
1
|
+
"""``SOArmEmbodiment`` — Inspect Robots embodiment for a LeRobot SO-ARM follower.
|
|
2
|
+
|
|
3
|
+
Wraps the LeRobot SO follower driver (SO-100 / SO-101). Designed for real-robot
|
|
4
|
+
reality:
|
|
5
|
+
|
|
6
|
+
* **Safety backstop** — every command is clamped to the configured joint limits
|
|
7
|
+
inside :meth:`step`, *independently* of any Inspect Robots ``Approver`` (so unclamped
|
|
8
|
+
model outputs can never reach the motors). This is layered on top of LeRobot's
|
|
9
|
+
own ``max_relative_target`` slew limit, which the driver applies.
|
|
10
|
+
* **Operator-in-the-loop success** — there is no privileged oracle; when the
|
|
11
|
+
operator signals end-of-episode the embodiment returns
|
|
12
|
+
``StepResult(terminated=True, termination_reason="success"|"failure")``, which is
|
|
13
|
+
the only path that reaches the scorer.
|
|
14
|
+
* **Self-paced** — declares ``SELF_PACED`` and sleeps to the control rate inside
|
|
15
|
+
:meth:`step` (the framework does not pace for us).
|
|
16
|
+
* **No interactive calibration** — the default driver connects with
|
|
17
|
+
``calibrate=False`` and fails fast (:func:`_check_calibrated`) when the arm
|
|
18
|
+
isn't calibrated for ``SOArmConfig.robot_id``, instead of dropping into
|
|
19
|
+
lerobot's blocking, arm-moving calibration prompt mid-:meth:`reset`.
|
|
20
|
+
|
|
21
|
+
The driver is injected (``driver_factory``) and so are the clock / sleep / operator
|
|
22
|
+
seams, so the whole embodiment runs in tests with no serial port, no motors, no
|
|
23
|
+
cameras, and no stdin. The real driver — a connected ``lerobot`` SO follower whose
|
|
24
|
+
``get_observation`` already returns both motor positions *and* camera frames — is
|
|
25
|
+
built in a pragma'd default that only executes on hardware.
|
|
26
|
+
"""
|
|
27
|
+
|
|
28
|
+
from __future__ import annotations
|
|
29
|
+
|
|
30
|
+
import time
|
|
31
|
+
from collections.abc import Callable, Mapping
|
|
32
|
+
from typing import TYPE_CHECKING, Any, Protocol, cast, runtime_checkable
|
|
33
|
+
|
|
34
|
+
import numpy as np
|
|
35
|
+
import numpy.typing as npt
|
|
36
|
+
from inspect_robots.embodiment import SELF_PACED, EmbodimentInfo
|
|
37
|
+
from inspect_robots.scene import Scene
|
|
38
|
+
from inspect_robots.types import Action, Observation, StepResult
|
|
39
|
+
|
|
40
|
+
from inspect_robots_so101 import packing
|
|
41
|
+
from inspect_robots_so101.config import SOArmConfig, action_box, observation_space
|
|
42
|
+
from inspect_robots_so101.operator import OperatorIO, default_poll_end
|
|
43
|
+
|
|
44
|
+
if TYPE_CHECKING:
|
|
45
|
+
from types import TracebackType
|
|
46
|
+
|
|
47
|
+
ImageMap = Mapping[str, npt.NDArray[np.uint8]]
|
|
48
|
+
Vec = npt.NDArray[np.float64]
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
@runtime_checkable
|
|
52
|
+
class SOArmDriver(Protocol):
|
|
53
|
+
"""The minimal LeRobot-robot surface the embodiment needs.
|
|
54
|
+
|
|
55
|
+
Satisfied directly by ``lerobot.robots.so_follower.SOFollower`` (and any other
|
|
56
|
+
LeRobot ``Robot``): observations are dicts of ``"<motor>.pos"`` floats plus
|
|
57
|
+
camera frames keyed by camera name; actions are dicts of ``"<motor>.pos"``.
|
|
58
|
+
"""
|
|
59
|
+
|
|
60
|
+
def get_observation(self) -> Mapping[str, Any]: ...
|
|
61
|
+
|
|
62
|
+
def send_action(self, action: Mapping[str, float]) -> Mapping[str, Any]: ...
|
|
63
|
+
|
|
64
|
+
def disconnect(self) -> None: ...
|
|
65
|
+
|
|
66
|
+
|
|
67
|
+
DriverFactory = Callable[[SOArmConfig], SOArmDriver]
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
def _check_calibrated(robot: Any, cfg: SOArmConfig) -> None:
|
|
71
|
+
"""Refuse to run with an uncalibrated arm, loudly and before any motion.
|
|
72
|
+
|
|
73
|
+
``connect(calibrate=False)`` does **not** raise on missing/mismatched
|
|
74
|
+
calibration — without this guard the failure would surface mid-``reset()`` as
|
|
75
|
+
an opaque error at the first normalized read/write. This also catches a
|
|
76
|
+
calibration file that exists but no longer matches the motors' EEPROM.
|
|
77
|
+
"""
|
|
78
|
+
if not robot.is_calibrated:
|
|
79
|
+
raise RuntimeError(
|
|
80
|
+
f"SO-ARM follower (robot_id={cfg.robot_id!r}) is not calibrated: no calibration "
|
|
81
|
+
f"matching the motors at {getattr(robot, 'calibration_fpath', '<unknown>')}. "
|
|
82
|
+
f"Calibrate first with `lerobot-calibrate --robot.type={cfg.robot_type} "
|
|
83
|
+
f"--robot.port={cfg.port} --robot.id=<your-id>`, then set "
|
|
84
|
+
"SOArmConfig.robot_id (and calibration_dir if you used a custom one) to match."
|
|
85
|
+
)
|
|
86
|
+
|
|
87
|
+
|
|
88
|
+
def _default_driver_factory(cfg: SOArmConfig) -> SOArmDriver: # pragma: no cover - real hardware
|
|
89
|
+
from pathlib import Path
|
|
90
|
+
|
|
91
|
+
from lerobot.robots.so_follower import SOFollower
|
|
92
|
+
from lerobot.robots.so_follower.config_so_follower import SOFollowerRobotConfig
|
|
93
|
+
|
|
94
|
+
robot = SOFollower(
|
|
95
|
+
SOFollowerRobotConfig(
|
|
96
|
+
id=cfg.robot_id,
|
|
97
|
+
calibration_dir=Path(cfg.calibration_dir) if cfg.calibration_dir else None,
|
|
98
|
+
port=cfg.port,
|
|
99
|
+
cameras=dict(cfg.camera_configs or {}),
|
|
100
|
+
max_relative_target=cfg.max_relative_target,
|
|
101
|
+
use_degrees=cfg.use_degrees,
|
|
102
|
+
disable_torque_on_disconnect=cfg.disable_torque_on_disconnect,
|
|
103
|
+
)
|
|
104
|
+
)
|
|
105
|
+
# NEVER auto-calibrate: lerobot's default connect(calibrate=True) drops into a
|
|
106
|
+
# BLOCKING interactive calibration that moves the arm — mid-reset(), unprompted.
|
|
107
|
+
robot.connect(calibrate=False)
|
|
108
|
+
_check_calibrated(robot, cfg)
|
|
109
|
+
return cast(SOArmDriver, robot)
|
|
110
|
+
|
|
111
|
+
|
|
112
|
+
class SOArmEmbodiment:
|
|
113
|
+
"""Inspect Robots embodiment for a single SO-ARM follower (joint-position control)."""
|
|
114
|
+
|
|
115
|
+
def __init__(
|
|
116
|
+
self,
|
|
117
|
+
config: SOArmConfig | None = None,
|
|
118
|
+
*,
|
|
119
|
+
driver_factory: DriverFactory | None = None,
|
|
120
|
+
operator: OperatorIO | None = None,
|
|
121
|
+
poll_end: Callable[[], bool] | None = None,
|
|
122
|
+
sleep_fn: Callable[[float], None] | None = None,
|
|
123
|
+
clock: Callable[[], float] | None = None,
|
|
124
|
+
**flat: Any,
|
|
125
|
+
) -> None:
|
|
126
|
+
self._cfg = config if config is not None else SOArmConfig.from_kwargs(**flat)
|
|
127
|
+
self._driver_factory: DriverFactory = driver_factory or _default_driver_factory
|
|
128
|
+
self._operator = operator if operator is not None else OperatorIO()
|
|
129
|
+
self._poll_end: Callable[[], bool] = poll_end or default_poll_end
|
|
130
|
+
self._sleep: Callable[[float], None] = sleep_fn or time.sleep
|
|
131
|
+
self._clock: Callable[[], float] = clock or time.perf_counter
|
|
132
|
+
|
|
133
|
+
self._driver: SOArmDriver | None = None
|
|
134
|
+
self._instruction: str | None = None
|
|
135
|
+
self._t_last = 0.0
|
|
136
|
+
self.num_steps = 0
|
|
137
|
+
|
|
138
|
+
self.info = EmbodimentInfo(
|
|
139
|
+
name="so_arm",
|
|
140
|
+
action_space=action_box(low=self._cfg.low, high=self._cfg.high),
|
|
141
|
+
observation_space=observation_space(
|
|
142
|
+
self._cfg.cam_height, self._cfg.cam_width, self._cfg.cameras
|
|
143
|
+
),
|
|
144
|
+
control_hz=self._cfg.control_hz,
|
|
145
|
+
is_simulated=False,
|
|
146
|
+
capabilities=frozenset({SELF_PACED}),
|
|
147
|
+
)
|
|
148
|
+
|
|
149
|
+
# -- lifecycle ---------------------------------------------------------
|
|
150
|
+
|
|
151
|
+
def reset(self, scene: Scene, *, seed: int | None = None) -> Observation:
|
|
152
|
+
"""Connect (if needed), drive to home, and block on operator readiness."""
|
|
153
|
+
if self._driver is None:
|
|
154
|
+
self._driver = self._driver_factory(self._cfg)
|
|
155
|
+
if self._cfg.home_pose is not None:
|
|
156
|
+
self._send(np.asarray(self._cfg.home_pose, dtype=np.float64))
|
|
157
|
+
self._operator.wait_ready()
|
|
158
|
+
self._instruction = scene.instruction
|
|
159
|
+
self.num_steps = 0
|
|
160
|
+
self._t_last = self._clock()
|
|
161
|
+
return self._observe(scene.instruction)
|
|
162
|
+
|
|
163
|
+
def step(self, action: Action) -> StepResult:
|
|
164
|
+
"""Clamp + command one action, pace to the control rate, then maybe end."""
|
|
165
|
+
driver = self._require_driver()
|
|
166
|
+
self.num_steps += 1
|
|
167
|
+
cmd = packing.validate_dim(action.data)
|
|
168
|
+
if self._cfg.joints_are_delta:
|
|
169
|
+
cmd = packing.from_obs_dict(driver.get_observation()) + cmd
|
|
170
|
+
self._send(cmd)
|
|
171
|
+
self._pace()
|
|
172
|
+
|
|
173
|
+
obs = self._observe(self._instruction)
|
|
174
|
+
if self._poll_end():
|
|
175
|
+
success = self._operator.confirm_success()
|
|
176
|
+
return StepResult(
|
|
177
|
+
observation=obs,
|
|
178
|
+
terminated=True,
|
|
179
|
+
termination_reason="success" if success else "failure",
|
|
180
|
+
info={"operator_confirmed": success},
|
|
181
|
+
)
|
|
182
|
+
return StepResult(observation=obs, terminated=False)
|
|
183
|
+
|
|
184
|
+
def close(self) -> None:
|
|
185
|
+
"""Disconnect and release the driver (no-op if never connected).
|
|
186
|
+
|
|
187
|
+
The handle is cleared even if ``disconnect()`` raises, so a failed
|
|
188
|
+
shutdown can't leave a half-dead driver behind (``close()`` stays
|
|
189
|
+
idempotent either way).
|
|
190
|
+
"""
|
|
191
|
+
if self._driver is not None:
|
|
192
|
+
try:
|
|
193
|
+
self._driver.disconnect()
|
|
194
|
+
finally:
|
|
195
|
+
self._driver = None
|
|
196
|
+
|
|
197
|
+
def __enter__(self) -> SOArmEmbodiment:
|
|
198
|
+
"""Support ``with SOArmEmbodiment(...) as emb:`` for guaranteed shutdown."""
|
|
199
|
+
return self
|
|
200
|
+
|
|
201
|
+
def __exit__(
|
|
202
|
+
self,
|
|
203
|
+
exc_type: type[BaseException] | None,
|
|
204
|
+
exc: BaseException | None,
|
|
205
|
+
tb: TracebackType | None,
|
|
206
|
+
) -> None:
|
|
207
|
+
self.close()
|
|
208
|
+
|
|
209
|
+
# -- internals ---------------------------------------------------------
|
|
210
|
+
|
|
211
|
+
def _require_driver(self) -> SOArmDriver:
|
|
212
|
+
if self._driver is None: # pragma: no cover - reset() always connects first
|
|
213
|
+
raise RuntimeError("step() called before reset()")
|
|
214
|
+
return self._driver
|
|
215
|
+
|
|
216
|
+
def _send(self, cmd: Vec) -> None:
|
|
217
|
+
"""Clamp to joint limits (safety backstop) and command the motors."""
|
|
218
|
+
clamped = np.clip(cmd, self._cfg.low, self._cfg.high)
|
|
219
|
+
self._require_driver().send_action(packing.to_action_dict(clamped))
|
|
220
|
+
|
|
221
|
+
def _pace(self) -> None:
|
|
222
|
+
hz = self._cfg.control_hz
|
|
223
|
+
if hz and hz > 0:
|
|
224
|
+
elapsed = self._clock() - self._t_last
|
|
225
|
+
self._sleep(max(0.0, 1.0 / hz - elapsed))
|
|
226
|
+
self._t_last = self._clock()
|
|
227
|
+
|
|
228
|
+
def _observe(self, instruction: str | None) -> Observation:
|
|
229
|
+
raw = self._require_driver().get_observation()
|
|
230
|
+
state = packing.from_obs_dict(raw)
|
|
231
|
+
images = {cam: np.asarray(raw[cam], dtype=np.uint8) for cam in self._cfg.cameras}
|
|
232
|
+
return Observation(
|
|
233
|
+
images=images,
|
|
234
|
+
state={packing.STATE_KEY: state},
|
|
235
|
+
instruction=instruction,
|
|
236
|
+
)
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
"""Operator-in-the-loop confirmation for real hardware runs.
|
|
2
|
+
|
|
3
|
+
A real tabletop has no privileged success oracle, so the human operator decides.
|
|
4
|
+
All stdin/stdout goes through injectable ``input_fn`` / ``output_fn`` so tests
|
|
5
|
+
drive these paths without a real terminal. The one genuinely TTY-bound piece —
|
|
6
|
+
the non-blocking "operator pressed end" poll — is isolated in
|
|
7
|
+
:func:`default_poll_end`, which is excluded from coverage.
|
|
8
|
+
"""
|
|
9
|
+
|
|
10
|
+
from __future__ import annotations
|
|
11
|
+
|
|
12
|
+
from collections.abc import Callable
|
|
13
|
+
from dataclasses import dataclass
|
|
14
|
+
|
|
15
|
+
# Affirmative answers (case-insensitive) for the end-of-episode success prompt.
|
|
16
|
+
_AFFIRMATIVE = frozenset({"y", "yes", "1", "true", "success", "pass"})
|
|
17
|
+
|
|
18
|
+
|
|
19
|
+
@dataclass
|
|
20
|
+
class OperatorIO:
|
|
21
|
+
"""Console I/O for operator prompts, with injectable functions for testing."""
|
|
22
|
+
|
|
23
|
+
input_fn: Callable[[str], str] = input
|
|
24
|
+
output_fn: Callable[[str], None] = print
|
|
25
|
+
|
|
26
|
+
def wait_ready(self, prompt: str = "Position the scene, then press Enter to start...") -> None:
|
|
27
|
+
"""Block until the operator confirms the scene is set up."""
|
|
28
|
+
self.input_fn(prompt)
|
|
29
|
+
|
|
30
|
+
def confirm_success(self, prompt: str = "Did the robot succeed? [y/N]: ") -> bool:
|
|
31
|
+
"""Return the operator's success verdict (affirmative answers → True)."""
|
|
32
|
+
answer = self.input_fn(prompt)
|
|
33
|
+
return answer.strip().lower() in _AFFIRMATIVE
|
|
34
|
+
|
|
35
|
+
|
|
36
|
+
def default_poll_end() -> bool: # pragma: no cover - requires a real TTY
|
|
37
|
+
"""Real non-blocking check for an operator "end episode" keypress.
|
|
38
|
+
|
|
39
|
+
Platform/TTY-specific; replaced by a scripted callable in tests. The default
|
|
40
|
+
returns ``False`` so an unattended run simply runs to ``max_steps``.
|
|
41
|
+
"""
|
|
42
|
+
import select
|
|
43
|
+
import sys
|
|
44
|
+
|
|
45
|
+
if not sys.stdin.isatty():
|
|
46
|
+
return False
|
|
47
|
+
ready, _, _ = select.select([sys.stdin], [], [], 0)
|
|
48
|
+
if not ready:
|
|
49
|
+
return False
|
|
50
|
+
sys.stdin.readline()
|
|
51
|
+
return True
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
"""Canonical 6-D joint packing for an SO-ARM follower (SO-100 / SO-101).
|
|
2
|
+
|
|
3
|
+
A LeRobot SO follower is a single 6-motor arm. LeRobot names the motors and keys
|
|
4
|
+
its observations / actions by ``"<motor>.pos"``; Inspect Robots, like the rest of this
|
|
5
|
+
package, works in a flat **6-D** vector. This module is the *one* place that
|
|
6
|
+
defines how those 6 numbers map to the named motors, so the policy (a LeRobot
|
|
7
|
+
model) and the embodiment (the LeRobot driver) can never disagree.
|
|
8
|
+
|
|
9
|
+
Convention (6-D): ``[shoulder_pan, shoulder_lift, elbow_flex, wrist_flex,
|
|
10
|
+
wrist_roll, gripper]`` — the five revolute joints in order, gripper last. This is
|
|
11
|
+
exactly the motor order of :class:`lerobot.robots.so_follower.SOFollower`.
|
|
12
|
+
|
|
13
|
+
This module is pure NumPy with no optional/hardware dependencies (no torch, no
|
|
14
|
+
``lerobot``), so it imports and tests anywhere.
|
|
15
|
+
"""
|
|
16
|
+
|
|
17
|
+
from __future__ import annotations
|
|
18
|
+
|
|
19
|
+
from collections.abc import Mapping
|
|
20
|
+
from typing import Any
|
|
21
|
+
|
|
22
|
+
import numpy as np
|
|
23
|
+
import numpy.typing as npt
|
|
24
|
+
from inspect_robots.spaces import StateField, StateSpec
|
|
25
|
+
|
|
26
|
+
# The SO follower motor names, in LeRobot's canonical order (see
|
|
27
|
+
# lerobot/robots/so_follower/so_follower.py). The five arm joints then the gripper.
|
|
28
|
+
MOTORS: tuple[str, ...] = (
|
|
29
|
+
"shoulder_pan",
|
|
30
|
+
"shoulder_lift",
|
|
31
|
+
"elbow_flex",
|
|
32
|
+
"wrist_flex",
|
|
33
|
+
"wrist_roll",
|
|
34
|
+
"gripper",
|
|
35
|
+
)
|
|
36
|
+
NUM_JOINTS = 5 # revolute joints
|
|
37
|
+
GRIPPER_DOF = 1 # one gripper
|
|
38
|
+
TOTAL_DIM = NUM_JOINTS + GRIPPER_DOF # 6-D
|
|
39
|
+
GRIPPER_IDX = TOTAL_DIM - 1 # index 5
|
|
40
|
+
|
|
41
|
+
# LeRobot keys motor positions as "<motor>.pos" in observations and actions.
|
|
42
|
+
POS_SUFFIX = ".pos"
|
|
43
|
+
|
|
44
|
+
# The canonical proprioception key. Joints are degrees (the SO follower default,
|
|
45
|
+
# ``use_degrees=True``) and the trailing gripper is a 0..100 position; we model the
|
|
46
|
+
# vector as a single field so ``StateSpec.keys == {"joint_pos"}`` stays consistent
|
|
47
|
+
# with the ``state_keys`` both components declare for compatibility.
|
|
48
|
+
STATE_KEY = "joint_pos"
|
|
49
|
+
STATE_SPEC = StateSpec(
|
|
50
|
+
fields=(StateField(key=STATE_KEY, shape=(TOTAL_DIM,), unit="deg+normalized"),)
|
|
51
|
+
)
|
|
52
|
+
|
|
53
|
+
Vec = npt.NDArray[np.float64]
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
def motor_keys() -> tuple[str, ...]:
|
|
57
|
+
"""The LeRobot ``"<motor>.pos"`` keys, in canonical order."""
|
|
58
|
+
return tuple(f"{m}{POS_SUFFIX}" for m in MOTORS)
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
def validate_dim(vec: npt.ArrayLike, n: int = TOTAL_DIM) -> Vec:
|
|
62
|
+
"""Return ``vec`` as a 1-D float array, raising ``ValueError`` if not length ``n``."""
|
|
63
|
+
arr = np.asarray(vec, dtype=np.float64).reshape(-1)
|
|
64
|
+
if arr.shape[0] != n:
|
|
65
|
+
raise ValueError(
|
|
66
|
+
f"expected a {n}-D vector, got shape {np.shape(vec)} ({arr.shape[0]} elems)"
|
|
67
|
+
)
|
|
68
|
+
return arr
|
|
69
|
+
|
|
70
|
+
|
|
71
|
+
def to_action_dict(vec: npt.ArrayLike) -> dict[str, float]:
|
|
72
|
+
"""Turn a flat 6-D vector into LeRobot's ``{"<motor>.pos": value}`` action dict."""
|
|
73
|
+
arr = validate_dim(vec)
|
|
74
|
+
return {f"{m}{POS_SUFFIX}": float(arr[i]) for i, m in enumerate(MOTORS)}
|
|
75
|
+
|
|
76
|
+
|
|
77
|
+
def from_obs_dict(obs: Mapping[str, Any]) -> Vec:
|
|
78
|
+
"""Extract the flat 6-D joint vector from a LeRobot observation/action dict.
|
|
79
|
+
|
|
80
|
+
Reads ``"<motor>.pos"`` for each motor in canonical order. Raises ``KeyError``
|
|
81
|
+
if any motor position is missing (a misconfigured driver).
|
|
82
|
+
"""
|
|
83
|
+
try:
|
|
84
|
+
return np.asarray([float(obs[f"{m}{POS_SUFFIX}"]) for m in MOTORS], dtype=np.float64)
|
|
85
|
+
except KeyError as exc:
|
|
86
|
+
raise KeyError(f"observation missing motor position {exc} (need {motor_keys()})") from exc
|
|
@@ -0,0 +1,224 @@
|
|
|
1
|
+
"""``LeRobotPolicy`` — a Inspect Robots policy backed by a LeRobot checkpoint.
|
|
2
|
+
|
|
3
|
+
LeRobot policies (ACT, SmolVLA, π0, diffusion, …) are ordinary ``nn.Module``\\ s
|
|
4
|
+
loaded from the Hugging Face Hub and run **in process** on the GPU. Unlike the
|
|
5
|
+
YAM/MolmoAct2 stack (where the model owns its own HTTP server), LeRobot models are
|
|
6
|
+
a library you import — so the heavy, GPU-bound dependencies (``torch`` +
|
|
7
|
+
``lerobot`` + the checkpoint) live behind a single injectable seam: a
|
|
8
|
+
``predict_fn`` that maps a **raw robot-style observation dict** (``"<motor>.pos"``
|
|
9
|
+
floats, camera frames keyed by camera name, and ``"task"``) to an action chunk.
|
|
10
|
+
|
|
11
|
+
The default ``predict_fn`` (``_default_predict``) lazily builds the policy and its
|
|
12
|
+
pre/post-processors from a pretrained path and prepares each observation with
|
|
13
|
+
lerobot's own ``raw_observation_to_observation`` helper — the same path
|
|
14
|
+
``lerobot.async_inference.policy_server`` uses — so torch conversion, HWC→CHW,
|
|
15
|
+
0..1 scaling, resizing, and batching stay lerobot's job, not ours. Tests exercise
|
|
16
|
+
this wiring with ``sys.modules`` fakes (no torch, no lerobot, no network), and the
|
|
17
|
+
``lerobot-seam`` CI job re-imports the real symbols on Python 3.12 to catch
|
|
18
|
+
upstream drift. ``import inspect_robots_so101`` never imports torch.
|
|
19
|
+
|
|
20
|
+
The action chunk this policy returns is already in the robot's **native motor
|
|
21
|
+
units** (degrees for the joints, 0..100 for the gripper): LeRobot's postprocessor
|
|
22
|
+
unnormalizes for us, so the embodiment commands the values verbatim (after its
|
|
23
|
+
safety clamp). There is therefore no gripper renormalization here.
|
|
24
|
+
"""
|
|
25
|
+
|
|
26
|
+
from __future__ import annotations
|
|
27
|
+
|
|
28
|
+
import time
|
|
29
|
+
from collections.abc import Callable, Mapping
|
|
30
|
+
from typing import Any
|
|
31
|
+
|
|
32
|
+
import numpy as np
|
|
33
|
+
import numpy.typing as npt
|
|
34
|
+
from inspect_robots.policy import PolicyConfig, PolicyInfo
|
|
35
|
+
from inspect_robots.scene import Scene
|
|
36
|
+
from inspect_robots.types import Action, ActionChunk, Observation
|
|
37
|
+
|
|
38
|
+
from inspect_robots_so101 import packing
|
|
39
|
+
from inspect_robots_so101.config import LeRobotPolicyConfig, action_box, observation_space
|
|
40
|
+
|
|
41
|
+
# A RAW robot-style observation dict -> an (N, action_dim) action-chunk array.
|
|
42
|
+
# Keys: "<motor>.pos" floats (packing.motor_keys()), camera frames keyed by the
|
|
43
|
+
# camera name, and "task" (the instruction) — exactly what a lerobot Robot's
|
|
44
|
+
# get_observation() returns, plus the task string. The default predict_fn feeds
|
|
45
|
+
# this to lerobot's raw_observation_to_observation, which packs it into
|
|
46
|
+
# "observation.state" / "observation.images.<cam>" tensors itself.
|
|
47
|
+
LeRobotObs = Mapping[str, Any]
|
|
48
|
+
PredictFn = Callable[[LeRobotObs], npt.NDArray[np.floating[Any]]]
|
|
49
|
+
|
|
50
|
+
# LeRobot key conventions (see lerobot/utils/constants.py). Hardcoded so this
|
|
51
|
+
# module needs no lerobot import.
|
|
52
|
+
OBS_STR = "observation"
|
|
53
|
+
OBS_IMAGE_PREFIX = "observation.images."
|
|
54
|
+
TASK_KEY = "task"
|
|
55
|
+
|
|
56
|
+
|
|
57
|
+
def _import_hw_to_dataset_features() -> Callable[..., dict[str, Any]]:
|
|
58
|
+
"""Import lerobot's ``hw_to_dataset_features`` across supported versions.
|
|
59
|
+
|
|
60
|
+
lerobot moved it from ``lerobot.datasets.utils`` (v0.5.0) to
|
|
61
|
+
``lerobot.datasets.feature_utils`` (v0.5.1); the ``lerobot-seam`` CI job
|
|
62
|
+
guards this against further drift.
|
|
63
|
+
"""
|
|
64
|
+
try:
|
|
65
|
+
from lerobot.datasets.feature_utils import hw_to_dataset_features
|
|
66
|
+
except ImportError: # lerobot == 0.5.0
|
|
67
|
+
from lerobot.datasets.utils import hw_to_dataset_features
|
|
68
|
+
return hw_to_dataset_features # type: ignore[no-any-return]
|
|
69
|
+
|
|
70
|
+
|
|
71
|
+
def _hw_observation_features(cfg: LeRobotPolicyConfig) -> dict[str, Any]:
|
|
72
|
+
"""Reconstruct the SO follower's ``observation_features`` mapping.
|
|
73
|
+
|
|
74
|
+
The policy never holds a lerobot ``Robot`` object (the embodiment owns the
|
|
75
|
+
driver), so we synthesize the same mapping the driver would report —
|
|
76
|
+
``"<motor>.pos" -> float`` for each motor in canonical order plus
|
|
77
|
+
``"<camera>" -> (height, width, 3)`` for each configured camera — for
|
|
78
|
+
``hw_to_dataset_features`` to turn into lerobot dataset features.
|
|
79
|
+
"""
|
|
80
|
+
features: dict[str, Any] = dict.fromkeys(packing.motor_keys(), float)
|
|
81
|
+
for cam in cfg.cameras:
|
|
82
|
+
features[cam] = (cfg.cam_height, cfg.cam_width, 3)
|
|
83
|
+
return features
|
|
84
|
+
|
|
85
|
+
|
|
86
|
+
def _default_predict(cfg: LeRobotPolicyConfig) -> PredictFn:
|
|
87
|
+
"""Build an in-process LeRobot inference closure from a pretrained checkpoint.
|
|
88
|
+
|
|
89
|
+
Mirrors ``lerobot.async_inference.policy_server``: load the policy class from
|
|
90
|
+
the factory, build the matching pre/post-processor pipelines, then per call
|
|
91
|
+
``raw_observation_to_observation`` → preprocess → ``predict_action_chunk`` →
|
|
92
|
+
postprocess back to native motor units.
|
|
93
|
+
"""
|
|
94
|
+
import torch
|
|
95
|
+
from lerobot.async_inference.helpers import raw_observation_to_observation
|
|
96
|
+
from lerobot.policies.factory import get_policy_class, make_pre_post_processors
|
|
97
|
+
|
|
98
|
+
hw_to_dataset_features = _import_hw_to_dataset_features()
|
|
99
|
+
|
|
100
|
+
policy = get_policy_class(cfg.policy_type).from_pretrained(cfg.pretrained_path)
|
|
101
|
+
policy.to(cfg.device)
|
|
102
|
+
policy.eval() # torch nn.Module inference mode (not Python's eval builtin)
|
|
103
|
+
preprocessor, postprocessor = make_pre_post_processors(
|
|
104
|
+
policy.config,
|
|
105
|
+
pretrained_path=cfg.pretrained_path,
|
|
106
|
+
preprocessor_overrides={"device_processor": {"device": cfg.device}},
|
|
107
|
+
postprocessor_overrides={"device_processor": {"device": cfg.device}},
|
|
108
|
+
)
|
|
109
|
+
|
|
110
|
+
# The dataset-feature spec raw_observation_to_observation uses to pack raw
|
|
111
|
+
# "<motor>.pos" keys into "observation.state" and camera frames into
|
|
112
|
+
# "observation.images.<cam>" (use_video=False: these are live frames).
|
|
113
|
+
lerobot_features = hw_to_dataset_features(
|
|
114
|
+
_hw_observation_features(cfg), OBS_STR, use_video=False
|
|
115
|
+
)
|
|
116
|
+
policy_image_features = dict(policy.config.image_features)
|
|
117
|
+
missing = [
|
|
118
|
+
cam for cam in cfg.cameras if f"{OBS_IMAGE_PREFIX}{cam}" not in policy_image_features
|
|
119
|
+
]
|
|
120
|
+
if missing:
|
|
121
|
+
raise ValueError(
|
|
122
|
+
f"checkpoint {cfg.pretrained_path!r} was trained with image features "
|
|
123
|
+
f"{sorted(policy_image_features)} which do not cover configured camera(s) "
|
|
124
|
+
f"{missing} (expected {[OBS_IMAGE_PREFIX + cam for cam in missing]}); "
|
|
125
|
+
"set LeRobotPolicyConfig.cameras to the camera names the checkpoint expects"
|
|
126
|
+
)
|
|
127
|
+
|
|
128
|
+
def _predict(obs: LeRobotObs) -> npt.NDArray[np.floating[Any]]:
|
|
129
|
+
observation = raw_observation_to_observation(
|
|
130
|
+
dict(obs), lerobot_features, policy_image_features
|
|
131
|
+
)
|
|
132
|
+
batch = preprocessor(observation)
|
|
133
|
+
with torch.no_grad():
|
|
134
|
+
chunk = policy.predict_action_chunk(batch)
|
|
135
|
+
if chunk.ndim == 2:
|
|
136
|
+
chunk = chunk.unsqueeze(0)
|
|
137
|
+
out = torch.stack(
|
|
138
|
+
[postprocessor(chunk[:, i, :]) for i in range(chunk.shape[1])], dim=1
|
|
139
|
+
).squeeze(0)
|
|
140
|
+
return out.detach().cpu().numpy() # type: ignore[no-any-return]
|
|
141
|
+
|
|
142
|
+
return _predict
|
|
143
|
+
|
|
144
|
+
|
|
145
|
+
class LeRobotPolicy:
|
|
146
|
+
"""Inspect Robots policy wrapping a LeRobot checkpoint for the SO-ARM action space."""
|
|
147
|
+
|
|
148
|
+
def __init__(
|
|
149
|
+
self,
|
|
150
|
+
config: LeRobotPolicyConfig | None = None,
|
|
151
|
+
*,
|
|
152
|
+
predict_fn: PredictFn | None = None,
|
|
153
|
+
**flat: Any,
|
|
154
|
+
) -> None:
|
|
155
|
+
self._cfg = config if config is not None else LeRobotPolicyConfig.from_kwargs(**flat)
|
|
156
|
+
self._predict_fn = predict_fn
|
|
157
|
+
self._instruction: str | None = None
|
|
158
|
+
self.num_inferences = 0
|
|
159
|
+
self.info = PolicyInfo(
|
|
160
|
+
name="lerobot",
|
|
161
|
+
action_space=action_box(), # semantics only; the embodiment owns limits
|
|
162
|
+
observation_space=observation_space(
|
|
163
|
+
self._cfg.cam_height, self._cfg.cam_width, self._cfg.cameras
|
|
164
|
+
),
|
|
165
|
+
# Intentionally None: advertising a rate would trip a (harmless) compat
|
|
166
|
+
# control_rate warning. The embodiment paces the rollout.
|
|
167
|
+
control_hz=None,
|
|
168
|
+
)
|
|
169
|
+
self.config = PolicyConfig(action_horizon=self._cfg.chunk_size)
|
|
170
|
+
|
|
171
|
+
def _predict(self) -> PredictFn:
|
|
172
|
+
"""Lazily build the real inference closure on first use."""
|
|
173
|
+
if self._predict_fn is None:
|
|
174
|
+
self._predict_fn = _default_predict(self._cfg)
|
|
175
|
+
return self._predict_fn
|
|
176
|
+
|
|
177
|
+
def reset(self, scene: Scene) -> None:
|
|
178
|
+
"""Stash the scene's instruction (fed to the VLA verbatim)."""
|
|
179
|
+
self._instruction = scene.instruction
|
|
180
|
+
self.num_inferences = 0
|
|
181
|
+
|
|
182
|
+
def act(self, observation: Observation) -> ActionChunk:
|
|
183
|
+
"""Build the raw robot-style observation, run inference, return the chunk.
|
|
184
|
+
|
|
185
|
+
The chunk is truncated to the first ``chunk_size`` actions (see
|
|
186
|
+
``LeRobotPolicyConfig.chunk_size``).
|
|
187
|
+
"""
|
|
188
|
+
cfg = self._cfg
|
|
189
|
+
try:
|
|
190
|
+
images = {cam: observation.images[cam] for cam in cfg.cameras}
|
|
191
|
+
except KeyError as exc:
|
|
192
|
+
raise ValueError(f"observation missing camera {exc} for lerobot policy") from exc
|
|
193
|
+
if cfg.state_key not in observation.state:
|
|
194
|
+
raise ValueError(f"observation missing state key {cfg.state_key!r}")
|
|
195
|
+
state = packing.validate_dim(observation.state[cfg.state_key])
|
|
196
|
+
|
|
197
|
+
# The raw robot format lerobot's raw_observation_to_observation expects:
|
|
198
|
+
# "<motor>.pos" floats, camera frames keyed by camera name, and "task".
|
|
199
|
+
lerobot_obs: dict[str, Any] = {
|
|
200
|
+
**packing.to_action_dict(state),
|
|
201
|
+
TASK_KEY: self._instruction or "",
|
|
202
|
+
**images,
|
|
203
|
+
}
|
|
204
|
+
|
|
205
|
+
t0 = time.perf_counter()
|
|
206
|
+
raw = self._predict()(lerobot_obs)
|
|
207
|
+
elapsed = time.perf_counter() - t0
|
|
208
|
+
|
|
209
|
+
actions = np.asarray(raw, dtype=np.float64)
|
|
210
|
+
if actions.ndim != 2 or actions.shape[1] != packing.TOTAL_DIM:
|
|
211
|
+
raise ValueError(
|
|
212
|
+
f"policy returned actions of shape {actions.shape}; "
|
|
213
|
+
f"expected (N, {packing.TOTAL_DIM})"
|
|
214
|
+
)
|
|
215
|
+
if actions.shape[0] == 0:
|
|
216
|
+
raise ValueError("policy returned an empty action chunk")
|
|
217
|
+
actions = actions[: cfg.chunk_size]
|
|
218
|
+
|
|
219
|
+
self.num_inferences += 1
|
|
220
|
+
return ActionChunk(
|
|
221
|
+
actions=[Action(data=row.copy()) for row in actions],
|
|
222
|
+
control_hz=None,
|
|
223
|
+
inference_latency_s=elapsed,
|
|
224
|
+
)
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
"""Preflight: prove an SO-ARM + LeRobot pairing is compatible *before any motion*.
|
|
2
|
+
|
|
3
|
+
``inspect-robots-so101-preflight`` constructs the policy and embodiment (no serial
|
|
4
|
+
connection, no model load — only their declared ``.info``), runs Inspect Robots's
|
|
5
|
+
compatibility check, prints the report, and exits non-zero on any error. Pass
|
|
6
|
+
``--task cubepick-reach`` (or any registered task) to also verify every scene is
|
|
7
|
+
realizable.
|
|
8
|
+
|
|
9
|
+
This is the "is it safe to move?" one-liner: a green preflight means dims, control
|
|
10
|
+
mode, cameras, and state keys all line up. (Absolute-vs-delta joint behavior is
|
|
11
|
+
*not* checkable here — see the README; use ``--dry-run`` plus a single jog to
|
|
12
|
+
confirm on hardware.)
|
|
13
|
+
"""
|
|
14
|
+
|
|
15
|
+
from __future__ import annotations
|
|
16
|
+
|
|
17
|
+
import argparse
|
|
18
|
+
import json
|
|
19
|
+
from collections.abc import Callable
|
|
20
|
+
|
|
21
|
+
from inspect_robots.compat import CompatibilityReport, check_compatibility
|
|
22
|
+
from inspect_robots.registry import resolve
|
|
23
|
+
from inspect_robots.task import Task
|
|
24
|
+
|
|
25
|
+
from inspect_robots_so101.config import LeRobotPolicyConfig, SOArmConfig
|
|
26
|
+
from inspect_robots_so101.embodiment import SOArmEmbodiment
|
|
27
|
+
from inspect_robots_so101.policy import LeRobotPolicy
|
|
28
|
+
|
|
29
|
+
CheckFn = Callable[..., CompatibilityReport]
|
|
30
|
+
|
|
31
|
+
|
|
32
|
+
def build(
|
|
33
|
+
soarm_cfg: SOArmConfig | None = None,
|
|
34
|
+
policy_cfg: LeRobotPolicyConfig | None = None,
|
|
35
|
+
) -> tuple[LeRobotPolicy, SOArmEmbodiment]:
|
|
36
|
+
"""Construct the (policy, embodiment) pair without connecting to anything."""
|
|
37
|
+
return LeRobotPolicy(policy_cfg), SOArmEmbodiment(soarm_cfg)
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
def run_preflight(
|
|
41
|
+
task_name: str | None = None,
|
|
42
|
+
*,
|
|
43
|
+
policy: LeRobotPolicy | None = None,
|
|
44
|
+
embodiment: SOArmEmbodiment | None = None,
|
|
45
|
+
check: CheckFn = check_compatibility,
|
|
46
|
+
) -> CompatibilityReport:
|
|
47
|
+
"""Return the compatibility report for the SO-ARM + LeRobot pairing."""
|
|
48
|
+
pol = policy if policy is not None else LeRobotPolicy()
|
|
49
|
+
emb = embodiment if embodiment is not None else SOArmEmbodiment()
|
|
50
|
+
task: Task | None = resolve("task", task_name) if task_name else None
|
|
51
|
+
return check(pol, emb, task)
|
|
52
|
+
|
|
53
|
+
|
|
54
|
+
def _format_human(report: CompatibilityReport, *, dry_run: bool) -> str:
|
|
55
|
+
lines = []
|
|
56
|
+
if report.ok:
|
|
57
|
+
lines.append("OK: policy and embodiment are compatible.")
|
|
58
|
+
else:
|
|
59
|
+
lines.append("INCOMPATIBLE:")
|
|
60
|
+
for issue in report.errors:
|
|
61
|
+
lines.append(f" ERROR [{issue.code}] {issue.message}")
|
|
62
|
+
for issue in report.warnings:
|
|
63
|
+
lines.append(f" WARNING [{issue.code}] {issue.message}")
|
|
64
|
+
if dry_run:
|
|
65
|
+
lines.append("(dry-run) No motion will be commanded.")
|
|
66
|
+
return "\n".join(lines)
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
def main(argv: list[str] | None = None, *, run: CheckFn | None = None) -> int:
|
|
70
|
+
"""CLI entry point. Returns a process exit code (non-zero on compat errors)."""
|
|
71
|
+
parser = argparse.ArgumentParser(prog="inspect-robots-so101-preflight")
|
|
72
|
+
parser.add_argument(
|
|
73
|
+
"--task", default=None, help="optional task name to check scene realizability"
|
|
74
|
+
)
|
|
75
|
+
parser.add_argument("--json", action="store_true", help="emit the report as JSON")
|
|
76
|
+
parser.add_argument("--dry-run", action="store_true", help="affirm no motion will be commanded")
|
|
77
|
+
args = parser.parse_args(argv)
|
|
78
|
+
|
|
79
|
+
run_fn: Callable[..., CompatibilityReport] = run if run is not None else run_preflight
|
|
80
|
+
report = run_fn(args.task)
|
|
81
|
+
|
|
82
|
+
if args.json:
|
|
83
|
+
payload = {
|
|
84
|
+
"ok": report.ok,
|
|
85
|
+
"errors": [{"code": i.code, "message": i.message} for i in report.errors],
|
|
86
|
+
"warnings": [{"code": i.code, "message": i.message} for i in report.warnings],
|
|
87
|
+
}
|
|
88
|
+
print(json.dumps(payload, indent=2))
|
|
89
|
+
else:
|
|
90
|
+
print(_format_human(report, dry_run=args.dry_run))
|
|
91
|
+
return 1 if report.errors else 0
|
|
92
|
+
|
|
93
|
+
|
|
94
|
+
if __name__ == "__main__": # pragma: no cover
|
|
95
|
+
raise SystemExit(main())
|
|
File without changes
|
|
@@ -0,0 +1,209 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: inspect-robots-so101
|
|
3
|
+
Version: 0.3.0
|
|
4
|
+
Summary: Inspect Robots adapters for LeRobot SO-ARM followers (SO-100/SO-101) driven by LeRobot policies.
|
|
5
|
+
Project-URL: Homepage, https://github.com/robocurve/inspect-robots-so101
|
|
6
|
+
Project-URL: Framework, https://github.com/robocurve/inspect-robots
|
|
7
|
+
Project-URL: LeRobot, https://github.com/huggingface/lerobot
|
|
8
|
+
Author: RoboCurve
|
|
9
|
+
License-Expression: MIT
|
|
10
|
+
License-File: LICENSE
|
|
11
|
+
Keywords: inspect-robots,lerobot,robotics,so-arm,so100,so101,vla
|
|
12
|
+
Classifier: Development Status :: 3 - Alpha
|
|
13
|
+
Classifier: Intended Audience :: Science/Research
|
|
14
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
17
|
+
Classifier: Typing :: Typed
|
|
18
|
+
Requires-Python: >=3.10
|
|
19
|
+
Requires-Dist: inspect-robots>=0.3
|
|
20
|
+
Requires-Dist: numpy>=1.24
|
|
21
|
+
Provides-Extra: dev
|
|
22
|
+
Requires-Dist: mypy>=1.11; extra == 'dev'
|
|
23
|
+
Requires-Dist: numpy<2.5; extra == 'dev'
|
|
24
|
+
Requires-Dist: pre-commit>=3.5; extra == 'dev'
|
|
25
|
+
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
|
|
26
|
+
Requires-Dist: pytest>=8.0; extra == 'dev'
|
|
27
|
+
Requires-Dist: ruff>=0.6; extra == 'dev'
|
|
28
|
+
Provides-Extra: lerobot
|
|
29
|
+
Requires-Dist: lerobot[feetech]<0.6,>=0.5; (python_version >= '3.12') and extra == 'lerobot'
|
|
30
|
+
Description-Content-Type: text/markdown
|
|
31
|
+
|
|
32
|
+
<div align="center">
|
|
33
|
+
|
|
34
|
+
# 🦾 inspect-robots-so101
|
|
35
|
+
|
|
36
|
+
**Run [Inspect Robots](https://github.com/robocurve/inspect-robots) evals on real
|
|
37
|
+
[SO-ARM](https://github.com/TheRobotStudio/SO-ARM100) followers (SO-100 / SO-101)
|
|
38
|
+
driven by [LeRobot](https://github.com/huggingface/lerobot) policies.**
|
|
39
|
+
|
|
40
|
+
[](https://github.com/robocurve/inspect-robots-so101/actions/workflows/ci.yml)
|
|
41
|
+
[](LICENSE)
|
|
42
|
+
[](https://github.com/robocurve/inspect-robots-so101/actions/workflows/ci.yml)
|
|
43
|
+
[](https://github.com/robocurve/inspect-robots)
|
|
44
|
+
|
|
45
|
+
</div>
|
|
46
|
+
|
|
47
|
+
Inspect Robots has **two** swappable inputs: a `Policy` (the VLA brain) and an
|
|
48
|
+
`Embodiment` (the robot body + world). This package provides both for the
|
|
49
|
+
SO-ARM + LeRobot stack, so any embodiment-agnostic Inspect Robots task runs on a real
|
|
50
|
+
arm:
|
|
51
|
+
|
|
52
|
+
- **`lerobot` policy** — wraps a LeRobot checkpoint (ACT, SmolVLA, π0, diffusion…)
|
|
53
|
+
and runs it **in process** on the GPU, returning an action chunk per inference.
|
|
54
|
+
- **`so_arm` embodiment** — the LeRobot SO follower driver (Feetech bus), with a
|
|
55
|
+
hard safety clamp, operator-in-the-loop success, and self-paced control.
|
|
56
|
+
|
|
57
|
+
Both declare the **same 6-D joint-position contract** (`shoulder_pan`,
|
|
58
|
+
`shoulder_lift`, `elbow_flex`, `wrist_flex`, `wrist_roll`, `gripper`; the cameras
|
|
59
|
+
you configure; packed `joint_pos` state), so Inspect Robots's compatibility check passes
|
|
60
|
+
with **zero errors and zero warnings** — verifiable before any motion.
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
inspect-robots run --task cubepick-reach --policy lerobot --embodiment so_arm
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
> This is the SO-ARM/LeRobot sibling of
|
|
67
|
+
> [inspect-robots-yam](https://github.com/robocurve/inspect-robots-yam) (bimanual I2RT YAM +
|
|
68
|
+
> MolmoAct2). Same Inspect Robots contract, different body and brain.
|
|
69
|
+
|
|
70
|
+
## Install (on the robot/GPU machine)
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
# Inspect Robots isn't on PyPI yet; uv resolves it from git. The `lerobot` extra pulls
|
|
74
|
+
# torch + lerobot + the Feetech motor bus the SO follower uses.
|
|
75
|
+
uv pip install "inspect-robots-so101[lerobot] @ git+https://github.com/robocurve/inspect-robots-so101"
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
- `lerobot` → `lerobot[feetech]` (torch, the policy, and the SO-ARM driver).
|
|
79
|
+
- **The `lerobot` extra needs Python ≥ 3.12** (lerobot ≥ 0.5's floor). On
|
|
80
|
+
3.10/3.11 the extra silently resolves to *nothing*: the core package still
|
|
81
|
+
imports, but no torch/lerobot is installed and hardware runs will fail.
|
|
82
|
+
|
|
83
|
+
Then pick a checkpoint. Any LeRobot policy trained on your SO-ARM works — e.g. the
|
|
84
|
+
public `lerobot/smolvla_base`, or your own ACT/π0 checkpoint on the Hub or a path.
|
|
85
|
+
|
|
86
|
+
## Preflight — *prove compatibility before any motion*
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
inspect-robots-so101-preflight # dims/semantics/cameras/state
|
|
90
|
+
inspect-robots-so101-preflight --task cubepick-reach # + scene realizability
|
|
91
|
+
inspect-robots-so101-preflight --dry-run # affirm no motion
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
A green preflight means action dim (6), control mode (`joint_pos`), cameras, and
|
|
95
|
+
state keys all line up. **It does not prove the joint values are interpreted the
|
|
96
|
+
same way** — see *Safety* below.
|
|
97
|
+
|
|
98
|
+
## Calibrate first (once, with lerobot)
|
|
99
|
+
|
|
100
|
+
The embodiment **never** runs lerobot's interactive calibration — connecting with
|
|
101
|
+
an uncalibrated arm would otherwise drop into a *blocking* prompt that moves the
|
|
102
|
+
arm mid-eval. Calibrate once with lerobot's own tool, then tell the config which
|
|
103
|
+
identity you used:
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
lerobot-calibrate --robot.type=so101_follower --robot.port=/dev/ttyACM0 --robot.id=my_follower
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
`SOArmConfig(robot_id="my_follower")` selects that calibration file
|
|
110
|
+
(`<calibration_dir>/<robot_id>.json`; leave `calibration_dir=None` for lerobot's
|
|
111
|
+
default location). If the arm isn't calibrated — or the file no longer matches
|
|
112
|
+
the motors — `reset()` fails fast with an actionable error instead of prompting.
|
|
113
|
+
|
|
114
|
+
## Run on hardware
|
|
115
|
+
|
|
116
|
+
You must point the embodiment at your serial port, calibration id, and camera
|
|
117
|
+
config, and the policy at a checkpoint:
|
|
118
|
+
|
|
119
|
+
```python
|
|
120
|
+
from inspect_robots import eval
|
|
121
|
+
from inspect_robots.approver import ClampApprover
|
|
122
|
+
from inspect_robots_so101 import LeRobotPolicy, SOArmEmbodiment, SOArmConfig, LeRobotPolicyConfig
|
|
123
|
+
from lerobot.cameras.opencv import OpenCVCameraConfig # your camera backend
|
|
124
|
+
|
|
125
|
+
emb = SOArmEmbodiment(SOArmConfig(
|
|
126
|
+
port="/dev/ttyACM0",
|
|
127
|
+
robot_type="so101_follower",
|
|
128
|
+
robot_id="my_follower", # the id you ran `lerobot-calibrate` with
|
|
129
|
+
max_relative_target=10.0, # lerobot's slew limit (deg/step); required for home_pose
|
|
130
|
+
cameras=("front",),
|
|
131
|
+
camera_configs={"front": OpenCVCameraConfig(index_or_path=0, width=640, height=480, fps=30)},
|
|
132
|
+
))
|
|
133
|
+
pol = LeRobotPolicy(LeRobotPolicyConfig(
|
|
134
|
+
pretrained_path="lerobot/smolvla_base", policy_type="smolvla", device="cuda",
|
|
135
|
+
))
|
|
136
|
+
|
|
137
|
+
with emb: # guarantees disconnect (and torque-off) even if the eval raises
|
|
138
|
+
(log,) = eval("cubepick-reach", pol, emb,
|
|
139
|
+
approver=ClampApprover(emb.info.action_space)) # defense in depth
|
|
140
|
+
print(log.status, log.results.metrics)
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
(Equivalently, wrap the `eval(...)` in `try: ... finally: emb.close()`.)
|
|
144
|
+
|
|
145
|
+
At each episode end the embodiment asks the operator (y/N); a `yes` records
|
|
146
|
+
`termination_reason="success"`, which the task's `success_at_end` scorer reads.
|
|
147
|
+
Unattended runs simply run to `max_steps` and score as failures.
|
|
148
|
+
|
|
149
|
+
## Safety
|
|
150
|
+
|
|
151
|
+
- **Hard clamp backstop.** Every command is clipped to `SOArmConfig.joint_low/high`
|
|
152
|
+
*inside* `step()`, independent of any Inspect Robots `Approver` and on top of LeRobot's
|
|
153
|
+
own `max_relative_target` slew limit — unclamped model outputs can never reach
|
|
154
|
+
the motors. **Set these to your real, calibrated SO-ARM joint limits** (the
|
|
155
|
+
defaults are conservative placeholders: joints ±180°, gripper 0–100).
|
|
156
|
+
- **Use `ClampApprover`** on hardware for a second layer.
|
|
157
|
+
- **Native units, no renormalization.** LeRobot's postprocessor unnormalizes the
|
|
158
|
+
policy output to the robot's native motor units (degrees for joints, 0–100 for
|
|
159
|
+
the gripper), so the embodiment commands them verbatim after the clamp. Train
|
|
160
|
+
and run the policy in the *same* units. `use_degrees=False` (lerobot's
|
|
161
|
+
normalized ±100 mode) is **rejected** — the state spec and default limits here
|
|
162
|
+
assume degrees. Also note the units are *degrees* while Inspect Robots's canonical
|
|
163
|
+
`joint_pos` convention is radians: the compat check compares state **keys**
|
|
164
|
+
only, so pairing either component with a third-party counterpart will *not*
|
|
165
|
+
flag a unit mismatch — verify units yourself when mixing stacks.
|
|
166
|
+
- **Homing is slew-limited or refused.** `home_pose` sends a single absolute
|
|
167
|
+
command, so the config requires `max_relative_target` (LeRobot's per-step slew
|
|
168
|
+
limit) whenever `home_pose` is set — otherwise the arm would slam to home at
|
|
169
|
+
full speed from wherever it happens to be.
|
|
170
|
+
- **Absolute vs. delta joints — verify first.** Actions are treated as **absolute**
|
|
171
|
+
joint targets by default. If your checkpoint emits deltas, set
|
|
172
|
+
`SOArmConfig(joints_are_delta=True)` (the embodiment converts to absolute
|
|
173
|
+
internally so the declared `joint_pos` stays honest). The compat check *cannot*
|
|
174
|
+
tell these apart — confirm with `--dry-run` and a single slow jog before a task.
|
|
175
|
+
|
|
176
|
+
## Configuration
|
|
177
|
+
|
|
178
|
+
`SOArmConfig`: `port`, `robot_type`, `robot_id`, `calibration_dir`, `cameras`,
|
|
179
|
+
`camera_configs`, `control_hz`, `cam_height/width`, `joint_low/high`,
|
|
180
|
+
`home_pose` (requires `max_relative_target`), `joints_are_delta`, `use_degrees`
|
|
181
|
+
(must stay `True`), `max_relative_target`, `disable_torque_on_disconnect`.
|
|
182
|
+
`robot_type` is validated (`so101_follower` / `so100_follower`) but is a label:
|
|
183
|
+
at lerobot v0.5.x both names alias the same driver class, so it changes no
|
|
184
|
+
runtime behavior.
|
|
185
|
+
`LeRobotPolicyConfig`: `pretrained_path`, `policy_type`, `device`, `cameras`,
|
|
186
|
+
`state_key`, `chunk_size`, `cam_height/width`.
|
|
187
|
+
|
|
188
|
+
Scalar knobs are settable from the CLI:
|
|
189
|
+
`inspect-robots run -P pretrained_path=lerobot/smolvla_base -E port=/dev/ttyACM0 ...`.
|
|
190
|
+
|
|
191
|
+
## Development
|
|
192
|
+
|
|
193
|
+
```bash
|
|
194
|
+
uv venv && uv pip install -e ".[dev]" # inspect-robots from a git tag
|
|
195
|
+
uv run pre-commit install
|
|
196
|
+
uv run pytest --cov # 100% coverage required
|
|
197
|
+
uv run ruff check . && uv run mypy
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
The whole suite runs with **no hardware, no GPU, no torch, no lerobot, and no
|
|
201
|
+
stdin** — the SO-ARM driver, the policy inference, the clock, and operator I/O are
|
|
202
|
+
all injected. The real model seam (`_default_predict`) is covered via
|
|
203
|
+
`sys.modules` fakes, and a dedicated `lerobot-seam` CI job (py3.12) imports the
|
|
204
|
+
real lerobot symbols it uses; only direct hardware/TTY I/O keeps
|
|
205
|
+
`# pragma: no cover`.
|
|
206
|
+
|
|
207
|
+
## License
|
|
208
|
+
|
|
209
|
+
[MIT](LICENSE)
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
inspect_robots_so101/CLAUDE.md,sha256=HYIsb-FdBSSp7mvyjPhBc7fUs7H-rsMxIzO2dv268lM,3490
|
|
2
|
+
inspect_robots_so101/__init__.py,sha256=FN4mkP5UDb_DWeyjhW-r-26v8-ZFgtoo5_OSnUeH--I,1316
|
|
3
|
+
inspect_robots_so101/config.py,sha256=lZbPnXDtJ8Z2wohTk25xgALwygajVsFtijVXp17ynCc,8029
|
|
4
|
+
inspect_robots_so101/embodiment.py,sha256=tvUSivIfqRAm4I5VXKjiOkqLrCmQEOkDaMmkL2xccg4,9821
|
|
5
|
+
inspect_robots_so101/operator.py,sha256=yUix11a3-VFZTg3vOsYxoM-nxsnk0S_yfufPzVQbRuI,1890
|
|
6
|
+
inspect_robots_so101/packing.py,sha256=acL7bd_8LNZSW79YOblxNRo9rTqflD39u3F5q6aKlcY,3358
|
|
7
|
+
inspect_robots_so101/policy.py,sha256=C1JqHZVwCV76MvQe8z8EN_NRyDlnBbDPUYlLYWgKt9Y,10085
|
|
8
|
+
inspect_robots_so101/preflight.py,sha256=iWu5XPL8eJ32_huWcnqVf476m8274Ft65aueuXvywLo,3730
|
|
9
|
+
inspect_robots_so101/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
10
|
+
inspect_robots_so101-0.3.0.dist-info/METADATA,sha256=daDDnCgAvcitlQx73RStWlgNrUyqUOlZsWt-xGF3YFA,10097
|
|
11
|
+
inspect_robots_so101-0.3.0.dist-info/WHEEL,sha256=mffPy8wBnZQn2VnJUU5jE99KsxaSfiyMHV9Yt0aLVxs,87
|
|
12
|
+
inspect_robots_so101-0.3.0.dist-info/entry_points.txt,sha256=t7Zi78FShR_reiRZ1Nwu4c5mlAu67o8iyrk2nwTBx2g,253
|
|
13
|
+
inspect_robots_so101-0.3.0.dist-info/licenses/LICENSE,sha256=Dv_L__TNAedADjrqdg67n216Joum0pHNDiY5XgiWREQ,1066
|
|
14
|
+
inspect_robots_so101-0.3.0.dist-info/RECORD,,
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 RoboCurve
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|