realbench-robotics 0.2.1__tar.gz

realbench_robotics-0.2.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Raghavan Madabushi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

realbench_robotics-0.2.1/PKG-INFO

@@ -0,0 +1,146 @@
+Metadata-Version: 2.4
+Name: realbench-robotics
+Version: 0.2.1
+Summary: Fail-closed robot readiness checks for integration-test reset cycles
+Author-email: Raghavan Madabushi <raghavanmadabushi10@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://pypi.org/project/realbench-robotics/
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: yourdfpy>=0.0.56
+Requires-Dist: numpy>=1.24
+Requires-Dist: rich>=13
+Requires-Dist: typer>=0.12
+Requires-Dist: fastapi>=0.110
+Requires-Dist: uvicorn[standard]>=0.27
+Requires-Dist: python-multipart>=0.0.9
+Requires-Dist: defusedxml>=0.7.1
+Requires-Dist: mujoco>=3.0
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: httpx; extra == "dev"
+Provides-Extra: ros2
+Dynamic: license-file
+
+# RealBench
+
+Fail-closed readiness checks that answer: is this robot verifiably ready for the next run?
+
+## Install
+
+```bash
+pip install realbench-robotics
+```
+
+The import package and CLI remain `realbench`. `pip install realbench-robotics[ros2]` does not install `rclpy`; that comes from your ROS2 distro.
+
+## ROS2 warning
+
+A positions-only ROS2 connection **cannot reach READY**. Faults and actuator state stay unverifiable, and the run quarantines by design.
+
+This is the command that can reach READY:
+
+```bash
+realbench check --urdf robot.urdf --source ros2 \
+  --ros2-topic /joint_states \
+  --diagnostics-topic /diagnostics \
+  --controller-manager /controller_manager
+```
+
+Run `realbench doctor --urdf robot.urdf --source ros2` first if anything is missing.
+
+`realbench check --source ros2` with no sensors or nodes declared exits 3
+(usage error) and does not print a readiness verdict. Open the Components
+tab and save, or run `realbench capture-components`.
+
+## Quickstart
+
+```bash
+realbench analyze --urdf robot.urdf
+realbench doctor --urdf robot.urdf --source ros2
+realbench capture-home --urdf robot.urdf --source ros2 --repo .
+realbench check --urdf robot.urdf --source ros2 \
+  --diagnostics-topic /diagnostics \
+  --controller-manager /controller_manager
+```
+
+`--urdf` accepts a URDF or MJCF file. `analyze` needs only the model. `doctor` diagnoses setup and never produces a readiness verdict. `capture-home` writes `<repo>/.realbench/home.json`. `check` and `cycle` look for that file when `--home` is omitted.
+
+## Serve
+
+```bash
+realbench serve --source ros2 --host 0.0.0.0 --no-browser
+```
+
+No `--urdf`. The model is read from the latched `*/robot_description` topic
+(TRANSIENT_LOCAL `std_msgs/String`). Joint states, diagnostics, and
+controller manager are inferred from the graph, including namespaced
+paths such as `/JK3/platform/joint_states`. Continuous-joint robots skip
+home pose.
+
+Until at least one sensor or node is declared, the stored verdict is
+**SETUP INCOMPLETE**, not QUARANTINE. The Components tab pre-ticks
+discoverable sensors and nodes so save is one click. Saving a LaserScan
+or Imu also writes a default `value_field` and plausible range so those
+checks evaluate (PASS or FAIL), not NOT_APPLICABLE.
+
+A failing diagnostic, sensor, or node can be marked a known issue with a
+required note. Acknowledged rows stay recorded and visible; they do not
+drive `classify()` until the reason string changes.
+
+Host wall-clock is stored next to robot time. If they disagree by more
+than 5 seconds, `check_clock_sane` fails at HEALTH and the UI shows both
+clocks.
+
+`check_state_stream_fresh` uses 3× the observed `joint_states` period and
+3× the recorder sample interval, floored at 2.0s.
+
+## Export
+
+Do not copy `.realbench/` while serve is running. The recording database
+is WAL-mode; a folder copy drops the `-wal` file.
+
+History lives in `<repo>/.realbench/records.db`. There is no separate
+`runs.db` on the serve path.
+
+```bash
+realbench export
+realbench export --out capture.db --repo .
+```
+
+SQLite's backup API writes one consistent file, prints path and size, and
+only reports success if `integrity_check` passes. Safe while serve is up.
+
+## Exit codes
+
+| Code | Meaning |
+|------|---------|
+| 0 | READY |
+| 1 | NOT_READY_RECOVERABLE (pose can be reset) |
+| 2 | QUARANTINE (liveness/health failed or unverifiable) |
+| 3 | Could not run (missing URDF, home, source, or undeclared ROS2 components) |
+
+`doctor` uses 0 (no blocking setup problems), 1 (problems that would block a useful verdict), and 3 (could not run).
+
+## CI
+
+```bash
+realbench check --urdf robot.urdf --source ros2 --diagnostics-topic /diagnostics --controller-manager /controller_manager || { echo "robot not ready, aborting run"; exit 1; }
+```
+
+## Read-only
+
+Read-only. RealBench never commands motion, never writes firmware, and never modifies your robot.
+
+## Known limits
+
+- Reset execution is stubbed.
+- Sensor and firmware checks are not implemented.
+- The ROS2 path is validated against a Dockerised Jazzy node, not physical hardware.

realbench_robotics-0.2.1/README.md

@@ -0,0 +1,115 @@
+# RealBench
+
+Fail-closed readiness checks that answer: is this robot verifiably ready for the next run?
+
+## Install
+
+```bash
+pip install realbench-robotics
+```
+
+The import package and CLI remain `realbench`. `pip install realbench-robotics[ros2]` does not install `rclpy`; that comes from your ROS2 distro.
+
+## ROS2 warning
+
+A positions-only ROS2 connection **cannot reach READY**. Faults and actuator state stay unverifiable, and the run quarantines by design.
+
+This is the command that can reach READY:
+
+```bash
+realbench check --urdf robot.urdf --source ros2 \
+  --ros2-topic /joint_states \
+  --diagnostics-topic /diagnostics \
+  --controller-manager /controller_manager
+```
+
+Run `realbench doctor --urdf robot.urdf --source ros2` first if anything is missing.
+
+`realbench check --source ros2` with no sensors or nodes declared exits 3
+(usage error) and does not print a readiness verdict. Open the Components
+tab and save, or run `realbench capture-components`.
+
+## Quickstart
+
+```bash
+realbench analyze --urdf robot.urdf
+realbench doctor --urdf robot.urdf --source ros2
+realbench capture-home --urdf robot.urdf --source ros2 --repo .
+realbench check --urdf robot.urdf --source ros2 \
+  --diagnostics-topic /diagnostics \
+  --controller-manager /controller_manager
+```
+
+`--urdf` accepts a URDF or MJCF file. `analyze` needs only the model. `doctor` diagnoses setup and never produces a readiness verdict. `capture-home` writes `<repo>/.realbench/home.json`. `check` and `cycle` look for that file when `--home` is omitted.
+
+## Serve
+
+```bash
+realbench serve --source ros2 --host 0.0.0.0 --no-browser
+```
+
+No `--urdf`. The model is read from the latched `*/robot_description` topic
+(TRANSIENT_LOCAL `std_msgs/String`). Joint states, diagnostics, and
+controller manager are inferred from the graph, including namespaced
+paths such as `/JK3/platform/joint_states`. Continuous-joint robots skip
+home pose.
+
+Until at least one sensor or node is declared, the stored verdict is
+**SETUP INCOMPLETE**, not QUARANTINE. The Components tab pre-ticks
+discoverable sensors and nodes so save is one click. Saving a LaserScan
+or Imu also writes a default `value_field` and plausible range so those
+checks evaluate (PASS or FAIL), not NOT_APPLICABLE.
+
+A failing diagnostic, sensor, or node can be marked a known issue with a
+required note. Acknowledged rows stay recorded and visible; they do not
+drive `classify()` until the reason string changes.
+
+Host wall-clock is stored next to robot time. If they disagree by more
+than 5 seconds, `check_clock_sane` fails at HEALTH and the UI shows both
+clocks.
+
+`check_state_stream_fresh` uses 3× the observed `joint_states` period and
+3× the recorder sample interval, floored at 2.0s.
+
+## Export
+
+Do not copy `.realbench/` while serve is running. The recording database
+is WAL-mode; a folder copy drops the `-wal` file.
+
+History lives in `<repo>/.realbench/records.db`. There is no separate
+`runs.db` on the serve path.
+
+```bash
+realbench export
+realbench export --out capture.db --repo .
+```
+
+SQLite's backup API writes one consistent file, prints path and size, and
+only reports success if `integrity_check` passes. Safe while serve is up.
+
+## Exit codes
+
+| Code | Meaning |
+|------|---------|
+| 0 | READY |
+| 1 | NOT_READY_RECOVERABLE (pose can be reset) |
+| 2 | QUARANTINE (liveness/health failed or unverifiable) |
+| 3 | Could not run (missing URDF, home, source, or undeclared ROS2 components) |
+
+`doctor` uses 0 (no blocking setup problems), 1 (problems that would block a useful verdict), and 3 (could not run).
+
+## CI
+
+```bash
+realbench check --urdf robot.urdf --source ros2 --diagnostics-topic /diagnostics --controller-manager /controller_manager || { echo "robot not ready, aborting run"; exit 1; }
+```
+
+## Read-only
+
+Read-only. RealBench never commands motion, never writes firmware, and never modifies your robot.
+
+## Known limits
+
+- Reset execution is stubbed.
+- Sensor and firmware checks are not implemented.
+- The ROS2 path is validated against a Dockerised Jazzy node, not physical hardware.

realbench_robotics-0.2.1/pyproject.toml

@@ -0,0 +1,54 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "realbench-robotics"
+version = "0.2.1"
+description = "Fail-closed robot readiness checks for integration-test reset cycles"
+readme = "README.md"
+license = "MIT"
+authors = [
+    { name = "Raghavan Madabushi", email = "raghavanmadabushi10@gmail.com" },
+]
+requires-python = ">=3.10"
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Topic :: Scientific/Engineering",
+]
+dependencies = [
+    "yourdfpy>=0.0.56",
+    "numpy>=1.24",
+    "rich>=13",
+    "typer>=0.12",
+    "fastapi>=0.110",
+    "uvicorn[standard]>=0.27",
+    "python-multipart>=0.0.9",
+    "defusedxml>=0.7.1",
+    "mujoco>=3.0",
+]
+
+[project.urls]
+Homepage = "https://pypi.org/project/realbench-robotics/"
+
+[project.optional-dependencies]
+dev = ["pytest", "httpx"]
+# rclpy is provided by the ROS2 distro, not pip; this extra is a marker only.
+ros2 = []
+
+[project.scripts]
+realbench = "realbench.cli:app"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+realbench = ["web/static/*", "web/static/assets/*"]
+
+[tool.pytest.ini_options]
+pythonpath = ["src"]
+testpaths = ["tests"]

realbench_robotics-0.2.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

realbench_robotics-0.2.1/src/realbench/__init__.py

@@ -0,0 +1,78 @@
+"""RealBench: fail-closed readiness checks for robot integration-test reset cycles."""
+
+from realbench.orchestrator import CycleReport, evaluate_readiness, run_cycle
+from realbench.report import exit_code_for, render_terminal, to_json
+from realbench.reset import MockResetExecutor, ResetExecutor, ResetOutcome
+from realbench.verdict import ReadinessReport, Verdict, classify
+from realbench.checks import (
+    ALL_CHECKS,
+    CheckConfig,
+    CheckResult,
+    CheckStatus,
+    CheckTier,
+    run_all_checks,
+)
+from realbench.gazebo_source import (
+    GazeboStateSource,
+    GzLiveStateSource,
+    parse_gz_joint_state,
+)
+from realbench.ros2_source import Ros2StateSource, Ros2Unavailable
+from realbench.state import RobotState
+from realbench.state_source import (
+    MockStateSource,
+    SourceCapabilities,
+    StateLoadError,
+    StateSource,
+    coverage_warning,
+    empty_robot_state,
+)
+from realbench.urdf_model import (
+    HomePoseError,
+    JointLimit,
+    ModelProvenance,
+    UrdfLoadError,
+    UrdfModel,
+    load_urdf_model,
+)
+from realbench.robot_model import load_robot_model
+
+__all__ = [
+    "ALL_CHECKS",
+    "CheckConfig",
+    "CheckResult",
+    "CheckStatus",
+    "CheckTier",
+    "CycleReport",
+    "GazeboStateSource",
+    "GzLiveStateSource",
+    "HomePoseError",
+    "JointLimit",
+    "ModelProvenance",
+    "MockResetExecutor",
+    "MockStateSource",
+    "ReadinessReport",
+    "ResetExecutor",
+    "Ros2StateSource",
+    "Ros2Unavailable",
+    "ResetOutcome",
+    "RobotState",
+    "SourceCapabilities",
+    "StateLoadError",
+    "StateSource",
+    "coverage_warning",
+    "empty_robot_state",
+    "UrdfLoadError",
+    "UrdfModel",
+    "Verdict",
+    "classify",
+    "evaluate_readiness",
+    "exit_code_for",
+    "load_robot_model",
+    "load_urdf_model",
+    "parse_gz_joint_state",
+    "render_terminal",
+    "run_all_checks",
+    "run_cycle",
+    "to_json",
+]

realbench_robotics-0.2.1/src/realbench/acknowledgements.py

@@ -0,0 +1,216 @@
+"""Known-issue acknowledgements. classify() is unchanged; copies are filtered here."""
+
+from __future__ import annotations
+
+from collections.abc import Mapping, Sequence
+from dataclasses import replace
+
+from realbench.checks import CheckResult, CheckStatus
+from realbench.components import (
+    Acknowledgement,
+    ComponentsDeclaration,
+    rewrite_acknowledgements,
+)
+from realbench.state import RobotState
+
+_BLOCKING = {CheckStatus.FAIL, CheckStatus.UNVERIFIABLE}
+_MEMBER_CHECKS = frozenset(
+    {
+        "check_sensors_reporting",
+        "check_sensors_plausible",
+        "check_sensors_changing",
+        "check_nodes_alive",
+    }
+)
+
+
+def split_fault(fault: str) -> tuple[str, str]:
+    """Split ``\"name: message\"`` from diagnostics_to_faults."""
+
+    if ": " in fault:
+        name, message = fault.split(": ", 1)
+        return name, message
+    return fault, ""
+
+
+def current_failing_reasons(
+    state: RobotState,
+    checks: Sequence[CheckResult],
+) -> dict[str, set[str]]:
+    """Map component id → current failing reason strings.
+
+    One component can fail in more than one check (reporting vs changing).
+    Keeping a set is what lets a written ack be read back instead of
+    being overwritten by the later check.
+    """
+
+    reasons: dict[str, set[str]] = {}
+    for fault in state.faults or ():
+        name, _message = split_fault(fault)
+        reasons.setdefault(name, set()).add(fault)
+    for check in checks:
+        for component_id, reason in _failing_members(check):
+            reasons.setdefault(component_id, set()).add(reason)
+    return reasons
+
+
+def fault_is_acknowledged(
+    fault: str,
+    acknowledgements: Mapping[str, Acknowledgement],
+) -> bool:
+    name, message = split_fault(fault)
+    for component_id, ack in acknowledgements.items():
+        if ack.reason == fault:
+            return True
+        if component_id == name and ack.reason in (fault, message):
+            return True
+    return False
+
+
+def member_is_acknowledged(
+    component_id: str,
+    reason: str,
+    acknowledgements: Mapping[str, Acknowledgement],
+) -> bool:
+    del reason
+    return component_id in acknowledgements
+
+
+def acknowledgement_is_stale(
+    ack: Acknowledgement,
+    current_reasons: Mapping[str, set[str]],
+    faults: Sequence[str],
+) -> bool:
+    """True when this id is currently failing with a different reason."""
+
+    currents = current_reasons.get(ack.component_id)
+    if currents:
+        return ack.reason not in currents
+    related = [
+        fault
+        for fault in faults
+        if split_fault(fault)[0] == ack.component_id or ack.reason == fault
+    ]
+    if not related:
+        return False
+    return not any(
+        ack.reason == fault or ack.reason == split_fault(fault)[1] for fault in related
+    )
+
+
+def prune_stale_acknowledgements(
+    declaration: ComponentsDeclaration | None,
+    state: RobotState,
+    checks: Sequence[CheckResult],
+) -> ComponentsDeclaration | None:
+    """Drop acks whose stored reason no longer matches. Writes the file."""
+
+    if declaration is None or not declaration.acknowledgements:
+        return declaration
+    reasons = current_failing_reasons(state, checks)
+    faults = list(state.faults or ())
+    kept: dict[str, Acknowledgement] = {}
+    for component_id, ack in declaration.acknowledgements.items():
+        if acknowledgement_is_stale(ack, reasons, faults):
+            continue
+        kept[component_id] = ack
+    if kept == dict(declaration.acknowledgements):
+        return declaration
+    updated = replace(declaration, acknowledgements=kept)
+    if declaration.path is not None:
+        try:
+            rewrite_acknowledgements(declaration.path, kept)
+        except OSError:
+            pass
+    return updated
+
+
+def active_acknowledgements(
+    declaration: ComponentsDeclaration | None,
+    state: RobotState,
+    checks: Sequence[CheckResult],
+) -> dict[str, Acknowledgement]:
+    """Acks that currently match a failing reason and suppress classify input."""
+
+    if declaration is None:
+        return {}
+    reasons = current_failing_reasons(state, checks)
+    faults = list(state.faults or ())
+    active: dict[str, Acknowledgement] = {}
+    for component_id, ack in declaration.acknowledgements.items():
+        if acknowledgement_is_stale(ack, reasons, faults):
+            continue
+        currents = reasons.get(component_id)
+        if currents and ack.reason in currents:
+            active[component_id] = ack
+            continue
+        if any(fault_is_acknowledged(fault, {component_id: ack}) for fault in faults):
+            active[component_id] = ack
+    return active
+
+
+def known_issues_acknowledged(
+    declaration: ComponentsDeclaration | None,
+    state: RobotState,
+    checks: Sequence[CheckResult],
+) -> int:
+    return len(active_acknowledgements(declaration, state, checks))
+
+
+def checks_for_verdict(
+    checks: Sequence[CheckResult],
+    state: RobotState,
+    acknowledgements: Mapping[str, Acknowledgement] | None,
+) -> list[CheckResult]:
+    """Copies for classify(). Original check objects are not mutated."""
+
+    acks = acknowledgements or {}
+    out: list[CheckResult] = []
+    for check in checks:
+        if check.status not in _BLOCKING:
+            out.append(check)
+            continue
+        if check.name == "check_no_latched_faults":
+            faults = state.faults
+            if (
+                isinstance(faults, list)
+                and len(faults) > 0
+                and all(fault_is_acknowledged(item, acks) for item in faults)
+            ):
+                out.append(replace(check, status=CheckStatus.PASS))
+            else:
+                out.append(check)
+            continue
+        if check.name in _MEMBER_CHECKS:
+            members = _failing_members(check)
+            if members and all(
+                member_is_acknowledged(component_id, reason, acks)
+                for component_id, reason in members
+            ):
+                out.append(replace(check, status=CheckStatus.PASS))
+            else:
+                out.append(check)
+            continue
+        out.append(check)
+    return out
+
+
+def _failing_members(check: CheckResult) -> list[tuple[str, str]]:
+    observed = check.observed
+    if not isinstance(observed, dict):
+        return []
+    components = observed.get("components")
+    if not isinstance(components, dict):
+        return []
+    members: list[tuple[str, str]] = []
+    for component_id, spec in components.items():
+        if not isinstance(spec, dict):
+            continue
+        if spec.get("required") is False:
+            continue
+        status = spec.get("status")
+        if status in ("FAIL", "UNVERIFIABLE", CheckStatus.FAIL, CheckStatus.UNVERIFIABLE):
+            members.append(
+                (str(component_id), str(spec.get("reason") or check.reason))
+            )
+    return members