react-manip 0.1.0__tar.gz

react_manip-0.1.0/LICENSE

@@ -0,0 +1,131 @@
+# PolyForm Noncommercial License 1.0.0
+
+<https://polyformproject.org/licenses/noncommercial/1.0.0>
+
+## Acceptance
+
+In order to get any license under these terms, you must agree
+to them as both strict obligations and conditions to all
+your licenses.
+
+## Copyright License
+
+The licensor grants you a copyright license for the
+software to do everything you might do with the software
+that would otherwise infringe the licensor's copyright
+in it for any permitted purpose.  However, you may
+only distribute the software according to [Distribution
+License](#distribution-license) and make changes or new works
+based on the software according to [Changes and New Works
+License](#changes-and-new-works-license).
+
+## Distribution License
+
+The licensor grants you an additional copyright license
+to distribute copies of the software.  Your license
+to distribute covers distributing the software with
+changes and new works permitted by [Changes and New Works
+License](#changes-and-new-works-license).
+
+## Notices
+
+You must ensure that anyone who gets a copy of any part of
+the software from you also gets a copy of these terms or the
+URL for them above, as well as copies of any plain-text lines
+beginning with `Required Notice:` that the licensor provided
+with the software.  For example:
+
+> Required Notice: Copyright Yoyodyne, Inc. (http://example.com)
+
+## Changes and New Works License
+
+The licensor grants you an additional copyright license to
+make changes and new works based on the software for any
+permitted purpose.
+
+## Patent License
+
+The licensor grants you a patent license for the software that
+covers patent claims the licensor can license, or becomes able
+to license, that you would infringe by using the software.
+
+## Noncommercial Purposes
+
+Any noncommercial purpose is a permitted purpose.
+
+## Personal Uses
+
+Personal use for research, experiment, and testing for
+the benefit of public knowledge, personal study, private
+entertainment, hobby projects, amateur pursuits, or religious
+observance, without any anticipated commercial application,
+is use for a permitted purpose.
+
+## Noncommercial Organizations
+
+Use by any charitable organization, educational institution,
+public research organization, public safety or health
+organization, environmental protection organization,
+or government institution is use for a permitted purpose
+regardless of the source of funding or obligations resulting
+from the funding.
+
+## Fair Use
+
+You may have "fair use" rights for the software under the
+law. These terms do not limit them.
+
+## No Other Rights
+
+These terms do not allow you to sublicense or transfer any of
+your licenses to anyone else, or prevent the licensor from
+granting licenses to anyone else.  These terms do not imply
+any other licenses.
+
+## Patent Defense
+
+If you make any written claim that the software infringes or
+contributes to infringement of any patent, your patent license
+for the software granted under these terms ends immediately. If
+your company makes such a claim, your patent license ends
+immediately for work on behalf of your company.
+
+## Violations
+
+The first time you are notified in writing that you have
+violated any of these terms, or done anything with the software
+not covered by your licenses, your licenses can nonetheless
+continue if you come into full compliance with these terms,
+and take practical steps to correct past violations, within
+32 days of receiving notice.  Otherwise, all your licenses
+end immediately.
+
+## No Liability
+
+***As far as the law allows, the software comes as is, without
+any warranty or condition, and the licensor will not be liable
+to you for any damages arising out of these terms or the use
+or nature of the software, under any kind of legal claim.***
+
+## Definitions
+
+The **licensor** is the individual or entity offering these
+terms, and the **software** is the software the licensor makes
+available under these terms.
+
+**You** refers to the individual or entity agreeing to these
+terms.
+
+**Your company** is any legal entity, sole proprietorship,
+or other kind of organization that you work for, plus all
+organizations that have control over, are under the control of,
+or are under common control with that organization.  **Control**
+means ownership of substantially all the assets of an entity,
+or the power to direct its management and policies by vote,
+contract, or otherwise.  Control can be direct or indirect.
+
+**Your licenses** are all the licenses granted to you for the
+software under these terms.
+
+**Use** means anything you do with the software requiring one
+of your licenses.

react_manip-0.1.0/PKG-INFO

@@ -0,0 +1,446 @@
+Metadata-Version: 2.4
+Name: react-manip
+Version: 0.1.0
+Summary: Development API for sampled manipulation instances and MuJoCo simulation
+License-Expression: PolyForm-Noncommercial-1.0.0
+Requires-Python: <3.11,>=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy==1.26.4
+Requires-Dist: mujoco==3.3.1
+Requires-Dist: robosuite==1.5.2
+Requires-Dist: scipy==1.15.3
+Requires-Dist: numba==0.67.0
+Provides-Extra: assets
+Requires-Dist: trimesh==5.1.1; extra == "assets"
+Requires-Dist: pycollada==0.9.3; extra == "assets"
+Requires-Dist: lxml==6.1.3; extra == "assets"
+Requires-Dist: python-dateutil==2.9.0.post0; extra == "assets"
+Requires-Dist: six==1.17.0; extra == "assets"
+Requires-Dist: coacd==1.0.7; extra == "assets"
+Requires-Dist: rtree==1.4.1; extra == "assets"
+Requires-Dist: networkx==3.4.2; extra == "assets"
+Dynamic: license-file
+
+# ReAct-Manip 0.3
+
+> **Robot asset update:** The active embodiments now use official PiPER-X arms:
+> `piperx`, `dual_piperx`, and `mobile_dual_piperx` (custom Tracer assembly).
+> Build with `python scripts/build_agilex_assets.py`; pinned upstream revisions,
+> URLs, and SHA-256 digests are recorded in [`scripts/piperx_assets.lock.json`](scripts/piperx_assets.lock.json). The PiPER-X candidate bank
+> contains regenerated initial states and main/wrist camera observations. Old ALOHA
+> selections remain incompatible; draw from the new bank below.
+
+> Optional HarnessVLA and four RPent native-embodiment integrations live under
+> `integrations/harness_vla/` and `integrations/rpent_embodiments/`. They do not
+> change the task defaults or the maintained PiPER-X evaluation bank.
+
+[简体中文](README.zh-CN.md)
+
+A robotics evaluation runtime for external policies, with 16 tasks, a fixed candidate bank, and one evaluation entry point. Optional features include a policy-view window, video recording, and keyboard control through Pygame. Training and reference solving policies are outside this release. Robot asset construction and rebuilding existing resolved instance banks are supported by scripts.
+
+> **Full evaluation scope:** A complete ReAct-Manip protocol run must run all 16 tasks in the main bank, all three Task Instances for every task, and all 15 Episodes for every instance (10 practice + 5 exam): 48 evaluation groups and 720 Episodes in total. A single task, task subset, single instance, shortened run, smoke check, aborted run, or otherwise incomplete run is for debugging or partial analysis only and must not be reported as a full protocol result. This pilot release does not yet produce an `official_score`.
+
+## License
+
+Project-authored code, benchmark records, documentation, and original assets are released under the [PolyForm Noncommercial License 1.0.0](LICENSE). This permits use, modification, and redistribution for permitted noncommercial purposes; it is a source-available license, not an Open Source Initiative approved open-source license. Third-party dependencies and assets retain their own licenses, including bundled third-party textures and robot assets obtained through the asset builder. See the notices shipped with those assets and [`scripts/piperx_assets.lock.json`](scripts/piperx_assets.lock.json).
+
+## Background: why we built this Benchmark
+
+Physical manipulation is sensitive to changes in the robot, workspace, object geometry, mass, friction, contact dynamics, and visual conditions. A policy that succeeds in one setting may fail after a small change. Reliable generalization therefore involves more than recognizing a task or producing an initial action: the robot must observe the consequences of its actions and respond to the environment.
+
+ReAct-Manip studies this process of acting, observing, learning, and adapting. Zero-shot performance is a useful baseline, but contact-rich tasks also require feedback, repeated attempts, and recovery. We build tasks that make these capabilities measurable through reproducible physical interactions, including cases where an initially plausible strategy does not work.
+
+The practice/exam protocol gives this motivation a concrete experimental setting. A policy receives 10 practice episodes followed by 5 exam episodes on the same Task Instance. It can retain experience across those episodes and continue adapting during exams; a fresh policy is created for the next Task Instance. The primary score measures exam performance. Practice trajectories, failures, and recovery behavior help explain how that performance was reached. The runtime provides the evaluation setting; the external policy determines whether and how to learn.
+
+Across those 15 Episodes, all four closed-loop parameter-adaptation tasks continuously change task-critical conditions. The policy does not face one fixed geometry and dynamics configuration: it must infer the current condition from action consequences while carrying useful probing and correction methods forward from practice. A complete evaluation runs every candidate and every saved parameter sequence; selecting only favorable Episodes cannot produce a complete score.
+
+## Use cases
+
+- **In-context adaptation:** use task language and interaction history to infer properties of the current instance and improve subsequent actions.
+- **Continual adaptation within a Task Instance:** compare performance across practice and exam episodes while retaining policy state. The public protocol resets policy state between Task Instances, so it does not by itself measure lifelong learning across tasks.
+- **Generalization across scenes and tasks:** study sensitivity to the geometry, physical parameters, object choices, and initial conditions that each task actually samples. Randomization is task-specific, not a promise that every parameter varies in every task.
+- **Embodiment research:** develop policies for the single-arm, fixed dual-arm, and mobile dual-arm PiPER-X configurations. Each released task declares its supported robot; changing that embodiment requires compatible assets, task validation, and regenerated instances rather than a bank-mode CLI override.
+- **Closed-loop behavior analysis:** inspect actions, observations, contacts, task events, errors, and recovery using the feedback available for that task, alongside final scores.
+- **Geometry diagnostics:** inspect simulator geometry truth during local debugging. These Features are never inputs to a benchmark policy.
+- **Simulation-to-real preparation:** investigate collision, reachability, and contact sensitivity before hardware experiments. Simulator performance alone does not establish hardware calibration or real-world transfer.
+
+## Benchmark design principles
+
+1. **Evaluate adaptation through interaction.** An unsuccessful attempt is useful evidence. Retries, recovery, and the time available for them follow the task's rules; success is not assumed for an Oracle or a reference strategy.
+2. **Use physical execution as the source of truth.** Runtime actions must flow through the task's controller, actuators, and MuJoCo contact dynamics. Initialization and explicit replay may restore saved state; writing poses or velocities during an attempt to manufacture success is not a valid solution.
+3. **Separate rules, instances, and episodes.** TaskSpec and the instance contract define the task; Task Instances resolve the parameters they own; Episode Instances specify their permitted initial-state or instruction variation. Profiles describe sampling distributions. Ownership is declared per task so that randomization remains controlled and reproducible.
+4. **Separate public observations from privileged information.** Proprioception, policy images, language, and declared feedback form the public interface. Simulator geometry, hidden assignments, and object locations belong to a separate internal channel. Comparisons must state which information a policy received.
+5. **Keep time and scoring explicit.** Physics advances synchronously with control steps. Task-defined completion, timeout, and score rules govern each episode; exam scores are aggregated by the protocol. Diagnostics and wall-clock inference latency help interpret results but do not replace the declared score.
+6. **Make comparisons reproducible.** Preserve task and episode identities, resolved parameters, source/model/controller digests, dependency versions, and the selected policy track. Changed task semantics or physics require compatible instances; old and new results should not be silently mixed.
+7. **Keep extensions declarative and evidence scoped.** Register tasks and geometry adapters through the existing interfaces. Distinguish successful loading, exact replay, a physical target-state witness, and a complete robot solution: none automatically proves the next.
+
+## Cross-Episode parameter changes in closed-loop tasks
+
+The main bank stores explicit Episode-level variations for all four closed-loop parameter-adaptation tasks: three Task Instances per task and 15 Episodes per instance, for 180 parameter-varying Episodes. Parameters are resolved before an Episode starts and remain fixed within it. Resetting the same Episode replays the exact saved condition instead of resampling it.
+
+| Task | Condition changed between Episodes | Domain |
+| --- | --- | --- |
+| `leveling_board1` | 3D dimensions of all six shims; identity and thin-axis orientation of the three thin shims | Ordinary dimensions 6–14 cm; thin dimensions 3, 4, and 5 cm |
+| `conveyor_grasp` | Conveyor speed and the size of every target object | 0.08–0.20 m/s; target scale 0.80–1.15× |
+| `box_insertion` | Sliding friction of the left/right contact regions and total block mass | Friction 0.6–1.4× baseline, clipped to [0.01, 0.6]; mass 0.7–1.3× baseline |
+| `car_drive2` | Vehicle nominal-speed gain associated with throttle displacement | 0.90–1.15 m/s |
+
+Continuous variables use five-stratum coverage. Practice Episodes 1–5, practice Episodes 6–10, and exam Episodes 1–5 each cover the same full domain in an independently shuffled order. Practice and exam therefore share a parameter domain but differ in exact values and ordering. The saved instance records, including sampled parameters, are published in this repository. The policy API does not pass those records directly; which values appear in observations or feedback depends on each task's declared visibility contract. A policy must adapt through the permitted interface. A result should disclose any use of instance records or task-specific tuning, since those conditions differ from the policy-interface-only setting.
+
+The variations are applied before MuJoCo model compilation and initial-state capture, then stored with the resolved runtime, model identity, and initial integration state in each Episode Instance. `phone_lean` also changes phone/support initial positions between Episodes, but support identities, sizes, masses, friction, and gravity belong to the Task Instance and stay fixed across its 15 Episodes; it is therefore reported separately from these four parameter-adaptation tasks.
+
+## Installation
+
+The source checkout now builds a minimal wheel for the `box_insertion` development API. The distribution has **not been published to PyPI**. Installation from this checkout does not include the 16-task candidate bank or the complete evaluation CLI.
+
+The candidate bank requires **Python 3.10, NumPy 1.26.4, MuJoCo 3.3.1, and robosuite 1.5.2**. Use the same virtual-environment setup on macOS or Linux. From the repository root:
+
+```bash
+python3.10 -m venv .venv
+source .venv/bin/activate
+python -m pip install -r requirements.txt
+python scripts/build_agilex_assets.py
+```
+
+For the minimal installed API, use Python 3.10 and install from this checkout:
+
+```bash
+python3.10 -m venv .venv
+source .venv/bin/activate
+python -m pip install '.[assets]'
+react-manip-assets prepare
+```
+
+`react-manip-assets prepare` builds the checksum-locked PiPER-X assets in the checkout when installed from source, or in the user cache when installed from a wheel. The first build needs network access. To try a wheel without publishing it, build one with `python -m pip wheel . --no-deps` and install that wheel with its `[assets]` extra.
+
+The installed development API generates one validated Task/Episode pair, replays it, and exposes the task's current score. It is for debugging and integration work; it does not produce an official benchmark result:
+
+```python
+import numpy as np
+from react_manip import create_instance, open_instance, task_score
+
+bundle = create_instance('box_insertion', seed=7, episode_index=0)
+bundle.save('example_instance')
+with open_instance(bundle, cameras=False) as env:
+    obs, reset_info = env.reset()
+    obs, reward, terminated, truncated, info = env.step(np.zeros(env.action_dim))
+    print(task_score(info, 'box_insertion'))
+```
+
+Robot assets are not committed to Git. The builder downloads official source models and meshes from pinned upstream commits and generates `assets/robots/agilex/`. The first build requires network access; cached sources are reused. Generated assets are gitignored and do not need to be downloaded again for evaluation. Small scene assets are included in the repository. Do not substitute other object assets: the bank validates the compiled model identity. The single `requirements.txt` also includes asset-builder, visualization, Pygame, and HarnessVLA API dependencies.
+
+The runtime defaults to `NUMBA_DISABLE_JIT=0` before importing simulation dependencies, matching the recorded bank. Do not switch JIT modes within one process. Optional IK or private-configuration warnings from robosuite do not imply missing dependencies for the controllers used here.
+
+Visual evaluation requires a working OpenGL context. macOS uses its default backend. Headless Linux typically uses `MUJOCO_GL=egl` with a working EGL driver, or `MUJOCO_GL=osmesa` with OSMesa installed. `--state-only-smoke` checks the pipeline without images and produces no protocol score.
+
+## Evaluation
+
+The main-bank command below performs a full protocol run. It does not use `--draw-seed` to choose only one candidate per task: it runs all 16 tasks, all 48 Task Instances, and all 720 Episodes, including all 180 parameter-varying Episodes from the four closed-loop tasks. The aggregate `protocol_score` is a full-run pilot result only when every group finishes validly; if any group is missing or invalid, the score is `null`. `--task`, task-specific manifests, and `--state-only-smoke` are not full protocol runs.
+
+When neither `--manifest` nor `--task` is specified, evaluation defaults to the shipped `react_manip/suites/piperx_candidates_v1/bank.json`, resolved relative to the repository rather than the current working directory. `--draw-seed` remains required. Use `--candidate-bank <path>` to select another bank; explicit manifest and task modes retain their existing behavior.
+
+For a live evaluation window or video recording, see [First-person cameras and visualization](#first-person-cameras-and-visualization). Add `--visualize` to bank or manifest evaluation; add `--video-dir` to save videos.
+
+```bash
+python scripts/evaluate_all_tasks.py \
+  --draw-seed 42 \
+  --policy my_policy:build \
+  --out-dir outputs/model_seed42_full
+```
+
+`evaluate_all_tasks.py` is the complete-evaluation entry point. It invokes the lower-level `evaluate_policy.py` and additionally verifies the task count, 48 evaluation groups, 720 completed Episodes, and the 10/5 protocol shape before writing results, logs, and `summary.json` under the output directory. Place `my_policy.py` in the repository root or on the Python module search path. Each task has three candidate Task Instances, and candidate-bank evaluation runs all three. The draw seed is retained for reproducible selection provenance but no longer drops any instance; do not pass `candidates.json` directly as an evaluation manifest. Omitting `--policy` selects the random policy for pipeline checks; it is not a model result.
+
+Evaluations also have wall-clock safeguards: by default, each episode is limited to 30 minutes. The aggregate limit for all episodes of one task is 2 hours for `conveyor_grasp`, 4 hours for `find_object_in_drawer2`, and 1 hour for other tasks. Completed results are retained, but a timed-out task/evaluation is marked invalid. Override these limits with `--episode-wall-timeout-sec` and `--task-wall-timeout-sec`. The limits run in the shared evaluator layer, so they also apply when calling `BenchmarkRunner` or the manifest suite directly.
+
+By default, the draw is saved beside the result as `<result_stem>_selection/`, containing a replayable manifest and all three instances and their episode records for every task. The same bank and `--draw-seed` produce the same manifest contents. Result files, selection directories, and trace directories must use new paths.
+Selection sidecars generated by the old one-instance-per-task protocol are rejected; draw a fresh selection after this change.
+
+```bash
+# Replay the exact selected instances.
+python scripts/evaluate_policy.py \
+  --manifest outputs/model_seed42_full/all_tasks_selection/manifest.json \
+  --policy my_policy:build --out outputs/model_seed42_repeat.json
+
+# State-only pipeline check, using the default random policy; no protocol_score.
+python scripts/evaluate_all_tasks.py \
+  --draw-seed 0 --state-only-smoke --out-dir outputs/smoke_full
+```
+
+Use `--trace-dir outputs/new_trace` to record public observations, actions, and transitions. In bank/manifest mode, the records own the robot, scene, horizon, control frequency, and episode counts; CLI overrides are rejected. For task-default debugging, use `--task TASK --policy my_policy:build`. Its protocol follows the task declaration and is separate from fixed-bank evaluation.
+
+## External policy interface
+
+This minimal zero-action adapter implements the interface; it is not a solving policy:
+
+```python
+# my_policy.py
+import numpy as np
+
+class ModelPolicy:
+    policy_id = "my-model"
+
+    def __init__(self):
+        self.history = []
+
+    def set_action_dim(self, action_dim):
+        self.action_dim = action_dim
+
+    def reset(self, *, seed=None, task_spec=None):
+        # Independent --task episodes: initialize your model state here.
+        pass
+
+    def begin_episode(self, *, phase, episode_index, seed, task_spec):
+        # phase is practice or exam; episode_index starts at zero per phase.
+        # Clear episode-local caches while retaining cross-episode learning.
+        self.phase = phase
+
+    def act(self, observation, info):
+        # observation["images"]: {camera_id: RGB uint8 H×W×3 array}
+        # info.get("language_prompt"): current task instruction
+        # Map model outputs to the robot's action layout.
+        return np.zeros(self.action_dim, dtype=np.float32)
+
+    def end_episode(self, **feedback):
+        self.history.append(feedback)
+
+def build():
+    return ModelPolicy()
+```
+
+The factory must return a fresh policy. Fixed-bank evaluation creates one policy per Task Instance and reuses it for all 15 episodes. Learning state persists between practice and exam; online adaptation is allowed during exams. Each new task receives a fresh policy. Avoid leaking learned state across tasks through module globals.
+
+Actions must be finite vectors of length `action_dim`, with all values in `[-1, 1]`. Invalid actions invalidate the group; they are not silently clipped. Each Task Instance selects its declared PiPER-X embodiment and recorded controller layout; a seven-dimensional single-arm policy cannot be applied unchanged to a dual-arm instance. Actual observation dictionaries and TaskSpecs define the image, instruction, and proprioceptive inputs. Instance files are not passed to the policy. Observation filtering follows task declarations; some tasks still expose task state or physical parameters. The public track rejects policies with `requires_geometry_truth=True`.
+
+An instance/group score is the mean of its five exam scores; the candidate-bank protocol score is the mean across all 48 groups (equivalently, each task's 15 exam episodes receive equal weight). If any group is invalid, the protocol score is null and the CLI exits with a nonzero status. `official_score` remains null: this pilot release has not established an official comparison track or certified solvability for every task. Report `protocol_score` as a pilot protocol result, together with the observation track, policy setup, and run provenance. Simulation advances synchronously; slow inference does not consume extra control steps.
+
+## Reported pilot baselines
+
+The following results were reported from tests using the [RPent framework](https://github.com/RLinf/RPent) on the 48-group, 720-Episode protocol. Each completed row uses 480 practice and 240 exam Episodes; the exam mean is on a 0–100 scale. Pass counts and mean scores are reported values, not independently reproduced by this repository. The exact RPent configuration, policy implementation, prompts, run logs, runtime revision, and any timeout overrides are not included with these figures here. Comparisons should disclose those settings and any access to the published instance records.
+
+| Model / configuration | Practice passes | Exam passes | Exam mean / 100 | Groups passing all five exams |
+| --- | ---: | ---: | ---: | ---: |
+| GPT-6 Astra | 177/480 (36.88%) | 130/240 (54.17%) | 52.62 | 22/48 |
+| GPT-6.1 Sol | 134/480 (27.92%) | 95/240 (39.58%) | 38.30 | 16/48 |
+| GPT-6.1 Terra | 96/480 (20.00%) | 94/240 (39.17%) | 33.73 | 17/48 |
+| GPT-5.6 Sol | 42/480 (8.75%) | 31/240 (12.92%) | 17.93 | 4/48 |
+| GPT-6 Astra + PI - 0.5 | Pending | Pending | Pending | Pending |
+
+## First-person cameras and visualization
+
+PiPER-X evaluation supplies all public views through `observation["images"]` as 640×480 RGB uint8 arrays:
+
+| Embodiment | Camera IDs in order |
+| --- | --- |
+| `piperx` | `robot0_head_camera`, `robot0_wrist_right` |
+| `dual_piperx` / `mobile_dual_piperx` | `robot0_head_camera`, `robot0_wrist_right`, `robot0_wrist_left` |
+
+The single-arm main camera is fixed above and behind the arm base. Dual models retain their main mounts. Wrist cameras now use the local pose `(-0.1, 0, -0.08)` and 65° field of view on the rigid gripper frame; their optical axis targets `(0, 0, 0.06)` and local `-X` is image-up. This first fit trial lifts the sensor 10 cm in the local image-up direction to reduce gripper-root occlusion while preserving the TCP aiming point. They follow the wrist, not finger opening, and never track targets. All views render the same simulation state without advancing physics. These are simulation sensor configurations, not hardware calibration. Pygame and evaluation use the same camera poses and image orientation; the display transpose only adapts array width/height to pygame and is not a 90° image rotation. Wrist images still roll with the wrist during wrist roll, while the neutral view is upright; the gripper can occlude part of the initial view, so use the main view alongside it.
+
+Visualization dependencies are included in `requirements.txt`; no second install command is needed.
+
+```bash
+# Evaluate the main bank with a live window and video recording.
+python scripts/evaluate_all_tasks.py \
+  --draw-seed 42 \
+  --policy my_policy:build \
+  --visualize \
+  --video-dir outputs/visual_eval_42_videos \
+  --out-dir outputs/visual_eval_42
+```
+
+Run from the repository root in your configured Python environment. Replace `my_policy:build` with your policy entry point. Use new result, selection and video paths. `--visualize` requires a desktop display and a working OpenGL context; do not set `MUJOCO_GL=disable`. On a headless machine, omit `--visualize` and use `--video-dir` with a working offscreen rendering backend.
+
+```bash
+# Task-default debugging.
+python scripts/evaluate_policy.py \
+  --task box_insertion --policy my_policy:build \
+  --visualize --video-dir outputs/new_camera_videos \
+  --out outputs/new_camera_result.json
+```
+
+The task selects its configured robot. Display and recording work independently in task mode and with regenerated manifests/banks. Use a new video directory. Omitting `--policy` uses random actions for pipeline checks.
+
+- The viewer uses the actual policy images: main view on the left, wrist views on the right, instructions and feedback in the sidebar.
+- Each episode produces `EPISODE.mp4` (composite), `EPISODE.CAMERA_ID.mp4` for each full-resolution view, and JSON metadata. Videos share frame counts/order and control frequency; none contain UI/debug overlays.
+- Up/Down scroll instructions, Space pauses, Tab shows a debug view excluded from policy input, and Esc aborts evaluation. Pauses neither advance physics nor duplicate recorded frames.
+- `--state-only-smoke` disables images for task/instance checks and labels results accordingly; these are not RGB evaluation results. It cannot display or record. `--render` remains the separate native debugging viewer.
+
+Camera contracts are now version 3.0. Old single-camera instances need regenerated runtime records and compatible instances, not merely a fresh draw. Source, TaskSpec and compiled-model identities remain checked. Existing PiPER-X mesh assets do not need downloading again: the common model assembly installs the new main sensor.
+
+## Tasks
+
+| task_id | Task |
+| --- | --- |
+| curling_push | Push a curling stone |
+| button_indicator_matching | Match buttons and indicators |
+| phone_lean | Lean a phone against a support |
+| leveling_board1 | Level a board with shims |
+| rail_ball_to_cup | Roll a ball along a rail into a cup |
+| pitch_pot | Toss into a pot |
+| conveyor_grasp | Sort conveyor objects into bins |
+| turntable_gamble1 | Push a turntable through contact |
+| ring_toss1 | Toss rings |
+| box_insertion | Insert a block into a slot |
+| car_drive2 | Drive using two physical joysticks |
+| rope_hang | Rebalance a suspended chain |
+| pan_flip | Flip food in a pan |
+| laptop_into_drawer | Place a laptop in a drawer |
+| find_object_in_drawer2 | Find an object in drawers |
+| tilte_maze | Control a tilting maze |
+
+Button episodes clamp the final score to [0, 100], including truncation. Running totals and signed ±20 rewards remain unchanged; `raw_score` retains the signed sum. Exam averages use individually clamped episode scores.
+
+The phone table is 1.6 × 1.4 m, covering the arm base and mounting plate. Small supports span about 3.0–4.6 cm at their largest dimension.
+
+Each `phone_lean` Task selects exactly two large and two small supports and varies only their initial positions between Episodes. Its three fixed candidate seeds each contain at least one bowl or bottle. No candidate or Episode seed is replaced to guarantee solvability; all four selected supports may score, and the phone angle must remain within 45–75 degrees for ten seconds.
+
+Phone instances are maintained only in the main bank, `react_manip/suites/piperx_candidates_v1/bank.json`. For phone debugging, copy all three existing candidates and all 45 Episodes using the command below, then evaluate the resulting manifest. Selection copies existing records without resampling. `--draw-seed` remains in provenance, but this task-specific command does not use it to discard candidates. Retired banks and their old generator have been removed; historical versions remain available in Git history. Evaluation selection snapshots remain under `outputs/` for reproducibility.
+
+```bash
+python scripts/select_task_instances.py --task phone_lean --draw-seed 42 \
+  --out outputs/phone_selection
+python scripts/evaluate_policy.py --manifest outputs/phone_selection/manifest.json \
+  --policy my_policy:build --out outputs/phone_results.json
+```
+
+The phone has a black screen, a light-blue back, and a circular Home button on the lower half of the front. Keep the Home-button end down, back against the support, screen outward, and rounded bottom on the table. Upside-down, sideways, load-bearing screen contact, and angles above 75 degrees are rejected. After release with grasp assist disabled, angle, speed, and drift limits must hold continuously for ten seconds. Language rules come exclusively from the environment runtime interface; JSON contains no language copy.
+
+For all 16 tasks, the versioned `task_spec.json`, `instance_contract.json`, and scoring implementation are authoritative for goals, scoring, failure/termination conditions, timing thresholds, and randomization ownership. The English instructions seen by policies come from `env.py` at reset and after every step.
+
+The four closed-loop tasks above use the same parameter sampler in saved-bank evaluation and task-default 10/5 evaluation; bare single-scene debugging uses baseline Task parameters. Preserve every Episode's variation and runtime identity in result analysis so adaptation curves can be stratified by physical condition.
+
+## Optional manual Pygame control
+
+```bash
+# Pygame dependencies are included in requirements.txt.
+python scripts/pygame_demo.py --task phone_lean
+```
+
+This tool only replays saved Task Instances and Episodes from the main bank; it does not resample or produce a protocol score. Use `--instance`/`--episode` to select exact records. N/P moves between Episodes and I/O between Task Instances. Arrow keys and PageUp/PageDown control translation; A/D, W/S, and Q/E control rotation. Space toggles the active gripper, Tab switches arms, and 1–9 or square brackets switch cameras. Backspace resets; Esc exits. Headless evaluation does not require Pygame.
+
+For a temporary task-default scene selected by seed, a resolved Scenario, or embodiment debugging outside the fixed bank, use the separate development entry point:
+
+```bash
+python scripts/pygame_task_debug.py --task phone_lean --seed 0
+```
+
+`pygame_task_debug.py` accepts `--robot` and `--scenario`. It constructs an unscored debugging environment rather than replaying saved Task/Episode Instances; use `pygame_demo.py` when inspecting release-bank records or reproducibility.
+
+Manual controls default to a target translation speed of 0.06 m/s and rotation speed of 0.3 rad/s (simulation time). Hold Shift for fast mode, or press F to toggle it: initially 1.0 m/s and 1.2 rad/s, useful for pushing or throwing. Use + / - to adjust the fast translation target by 0.05 m/s (0.05–2.0 m/s), or set it at launch with `--fast-speed`. The sidebar shows the active mode and target speed. For curling, approach in FINE mode, then use FAST for the push; adjust speed and release timing to the visible target. Actual speed depends on controller tracking and the task action limits. Releasing a movement key stops advancing that target axis and holds its release position; physical braking is not instantaneous. Losing window focus or switching arms clears movement input and holds the current poses. This tuning applies only to Pygame, not policy evaluation. On MacBook, PageUp/PageDown are Fn+Up/Fn+Down.
+
+Run `scripts/verify_curling_reachability.py` to reproduce curling action-interface reachability checks. These use offline-calibrated diagnostic action replays, not learned-policy benchmark scores.
+
+
+## Layout and reproducibility
+
+The release keeps only `README.md` and `README.zh-CN.md` as Markdown documentation. Versioned JSON contracts and implementation code are the source of truth for task rules, interfaces, and reproducibility constraints. Diagnostics are exposed directly through `scripts/verify_*.py` and `scripts/audit_task_solvability.py`, so released instructions do not depend on excluded auxiliary Markdown files.
+
+- `scripts/evaluate_all_tasks.py`: complete 16-task evaluation entry point. `evaluate_policy.py`: lower-level bank/manifest/task evaluator. `pygame_demo.py`: fixed-bank manual replay. `pygame_task_debug.py`: temporary task-default scene debugging. `build_agilex_assets.py` and its helpers build robot assets.
+- `react_manip/`: runner, protocols, observation filtering, cameras, robot integration, instance replay, and candidate selection.
+- `tasks/<task>/`: environments, scoring/physics helpers, task declarations, and required assets.
+- `configs/`: default profiles needed for task construction and parameter validation; not CLI overrides for the bank.
+- `react_manip/suites/piperx_candidates_v1/`: the only maintained candidate bank, containing task instances and Episodes.
+- `assets/`: scene textures and locally generated robot assets. Generated `provenance.json` records robot source versions and hashes.
+
+Fixed instances validate source code, TaskSpecs, parameter contracts, compiled models, controllers, and dependency versions. Changing runtime Python/XML invalidates the recorded identity by design. Do not manually bypass validation by replacing hashes. Verified release migrations are recorded in `generation.json`. Write run outputs to new paths under `outputs/`; development tools and historical outputs are not distributed.
+
+`rope_hang` starts with the chain draped over the bar. The task is to catch the slipping chain, redistribute its hanging lengths, release it, and maintain stable suspension. Its three candidates and 45 episodes were regenerated for this scene (TaskSpec 3.0 / contract 2.0). Old table-start selections must be redrawn; scores from the old and new scenes are not directly comparable.
+
+Fixed dual PiPER-X `conveyor_grasp` uses two side tables. Bin centers are `(1.25,0.50,0.76)`, `(2.75,0.50,0.76)`, `(1.25,0.90,0.76)`, and `(2.75,0.90,0.76)` meters, using the first two, three, or four bins according to category count. This layout avoids initial arm/base interference. Its three groups and 45 episodes were rebuilt (TaskSpec 3.0 / contract 2.0); do not reuse selections from the previous layout.
+
+Python imports now use `react_manip.*` instead of `benchmark.*`, and the bank lives under `react_manip/suites/piperx_candidates_v1/`. There is no compatibility alias for the old package. Update external-policy imports and draw a new selection. The PiPER-X migration reconstructs initial simulation states with each task’s configured robot and camera contract 3.0. pip packaging remains future work.
+
+### Rebuild saved instances
+
+The single active bank has 48 Task Instances and 720 Episodes, including three candidates and 45 Episodes for `phone_lean`; the four closed-loop parameter-adaptation tasks contribute 180 Episodes with explicit variations. Rebuilding compiles each Episode's resolved model, captures a fresh initial state, verifies its model/controller/source identity, restores the state exactly, and advances one zero-action step. This verifies replay compatibility, not robot task solvability. IDs and round plans are retained. Existing variations are preserved on subsequent rebuilds; old records without variations are migrated by the sampler. Generation reports record language samples and task-specific physical evidence.
+
+```bash
+python scripts/regenerate_instance_bank.py \
+  --source react_manip/suites/piperx_candidates_v1/bank.json \
+  --out outputs/rebuilt_piperx_bank
+```
+
+Use a new output directory. The script rebuilds task/episode records and the candidate manifest with linked digests. Evaluate its `bank.json` to draw a fresh 16-task manifest; do not reuse old selected manifests.
+
+Language is owned by each task’s `env.py` through `get_episode_language_prompt()`. The adapter refreshes it after reset and every step and supplies `info["language_prompt"]` and `info["language_prompt_structured"]` to the policy. Structured fields separate `task_instruction` from `round_instruction` and `round_index` (zero-based, or null when no round is active). Missing runtime interfaces fail explicitly; task JSON is not a language fallback.
+
+The drawer search task (`find_object_in_drawer2`) accepts either arm or both arms together. Both pose commands and both grippers are active. Its fixed rules, current-round instructions, constraints and end statuses are English-only; all five ordered placements still share 600 seconds and earn 20 points each.
+
+## Geometry ground-truth Feature interface (debugging only)
+
+Geometry Features are an internal, read-only channel used only for local debugging and diagnostics. They are obtained through [`GeometryFeatureAdapter`](react_manip/core/geometry.py), separately from public policy observations. Do not expose these privileged Features to a benchmark policy or use them for benchmark evaluation or scoring.
+
+Adapters read the compiled MuJoCo model, simulation data, and task-owned read-only state. Extraction must not advance physics, change `qpos`/`qvel`, or alter scoring and dwell timers. Body poses use world coordinates, meters, and MuJoCo `(w, x, y, z)` quaternions. Velocity fields use SI units; matrices, local coordinates, and fields explicitly suffixed `_deg` retain their task-defined conventions. Check field definitions instead of treating every numeric array as a pose.
+
+The stable top-level schema is:
+
+```text
+{
+  "schema_version": "geometry.v1",
+  "entities": {"<entity name>": ...},
+  "relations": {"<relation name>": ...},
+  "task_state": {...},
+  "validity_mask": {...}
+}
+```
+
+Entity contents and array sizes vary by task; some entries are lists of objects. The validity mask describes the adapter's reported entities, not a universal sensor-quality certificate. Geometry diagnostic fields are not the scoring authority: use the task's declared final `info[score_key]` and terminal feedback for evaluation.
+
+### Debugging example
+
+Keep the returned environment wrapper for `reset()` and `step()`, and unwrap it only to select the task's geometry adapter. This minimal internal debugging example uses the button task and one zero-action transition; it is not a solving policy or a scored evaluation:
+
+```python
+import numpy as np
+from react_manip.builtin import discover_task_definitions
+from react_manip.core import GeometryFeatureAdapter, normalize_reset, normalize_step
+from react_manip.core.compatibility import resolve_task_robot
+
+spec, factory = next(
+    (spec, factory) for spec, factory in discover_task_definitions()
+    if spec.task_id == "button_indicator_matching"
+)
+robot_id = resolve_task_robot(spec)
+env = factory(seed=0, robot_id=robot_id,
+              has_offscreen_renderer=False, use_camera_obs=False)
+try:
+    observation, info = normalize_reset(env.reset())
+    task_env = getattr(env, "env", env)
+    adapter = GeometryFeatureAdapter.for_task(task_env)
+    before = adapter.extract()
+    transition = normalize_step(env.step(np.zeros(env.action_dim)))
+    after = adapter.extract()
+    record = {
+        "task_id": spec.task_id,
+        "robot_id": robot_id,
+        "scenario_id": getattr(getattr(env, "scenario", None), "scenario_id", None),
+        "simulation_time_s": float(task_env.sim.data.time),
+        "features": after,
+        "reward": transition.reward,
+        "terminated": transition.terminated,
+        "truncated": transition.truncated,
+    }
+finally:
+    env.close()
+```
+
+Extract a fresh snapshot after each step and align it with that transition's action, public observation, language, feedback, and simulation timestamp. For reproducible debugging, also record Task/Episode IDs, scenario and runtime digests, phase, step index, schema version, and camera IDs when images are enabled. NumPy arrays require an array-capable storage format or explicit conversion before JSON serialization. The example disables RGB; enable the documented camera interface on the same environment when debugging with images.
+
+Keep privileged Features in a separate debugging record. Do not use robosuite's concatenated `object-state` as a cross-task schema, and do not pass an unfiltered raw observation directly to a public policy. The runner applies [`policy_observation` and `policy_info`](react_manip/core/observation.py); declared legacy privileged keys are filtered, and geometry truth requires an explicit internal opt-in that must be used only for debugging. Public candidate-bank evaluation rejects `requires_geometry_truth=True`. `--trace-dir` records the public evaluation stream; use this interface only for separate local diagnostics. The current contracts remain task-specific, so debugging code must check which additional state fields are public.
+
+### Available adapter entry points
+
+These are the adapters currently implemented in this release, not a claim that all 16 tasks expose geometry Features. Most live in [`react_manip/core/geometry.py`](react_manip/core/geometry.py); the drawer-search adapter registers from its task module when that environment is imported.
+
+| Task | Adapter | Feature contents |
+| --- | --- | --- |
+| `box_insertion` | `BoxInsertionGeometry` | Block and end-effector geometry, slot bounds, clearance, contact force, and insertion state |
+| `laptop_into_drawer` | `LaptopIntoDrawerGeometry` | Laptop, drawer, handle, and end-effector geometry; containment bounds and relative distances |
+| `find_object_in_drawer2` | `FindObjectInDrawer2Geometry` | Object poses and drawer membership, drawer slides, destination-box bounds, current target, and placement state |
+| `conveyor_grasp` | `ConveyorGraspGeometry` | Object poses, velocities, categories, bins, examples, assignments, and sorting state |
+| `phone_lean` | `PhoneLeanGeometry` | Phone, selected supports, and end-effector poses; relative positions and diagnostic distance/height fields |
+| `pitch_pot` | `PitchPotGeometry` | Projectile pose/velocity, pot and tray geometry, relative position, and hit/attempt state |
+| `button_indicator_matching` | `ButtonIndicatorMatchingGeometry` | Buttons, indicator labels and states, hidden wiring, current target, raw score, and events |
+
+For a new task, subclass `GeometryFeatureAdapter`, register its task ID with `@GeometryFeatureAdapter.register("my_task")`, implement `extract()` using `_result(...)`, and declare the field semantics and visibility in TaskSpec. Add reset/step checks for shapes, validity, and read-only behavior. Debugging code must report a missing adapter explicitly rather than inventing a generic geometry view. Existing registration does not require a task-specific branch in the runner or Pygame frontend.