SparkRT 0.1.0rc1__tar.gz

sparkrt-0.1.0rc1/LICENSE

@@ -0,0 +1,164 @@
+Apache License
+Version 2.0, January 2004
+https://www.apache.org/licenses/
+
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1. Definitions.
+
+"License" shall mean the terms and conditions for use, reproduction, and
+distribution as defined by Sections 1 through 9 of this document.
+
+"Licensor" shall mean the copyright owner or entity authorized by the
+copyright owner that is granting the License.
+
+"Legal Entity" shall mean the union of the acting entity and all other
+entities that control, are controlled by, or are under common control with
+that entity. For the purposes of this definition, "control" means (i) the
+power, direct or indirect, to cause the direction or management of such
+entity, whether by contract or otherwise, or (ii) ownership of fifty percent
+(50%) or more of the outstanding shares, or (iii) beneficial ownership of
+such entity.
+
+"You" (or "Your") shall mean an individual or Legal Entity exercising
+permissions granted by this License.
+
+"Source" form shall mean the preferred form for making modifications,
+including but not limited to software source code, documentation source, and
+configuration files.
+
+"Object" form shall mean any form resulting from mechanical transformation
+or translation of a Source form, including but not limited to compiled object
+code, generated documentation, and conversions to other media types.
+
+"Work" shall mean the work of authorship, whether in Source or Object form,
+made available under the License, as indicated by a copyright notice that is
+included in or attached to the work.
+
+"Derivative Works" shall mean any work, whether in Source or Object form,
+that is based on (or derived from) the Work and for which the editorial
+revisions, annotations, elaborations, or other modifications represent, as a
+whole, an original work of authorship. For the purposes of this License,
+Derivative Works shall not include works that remain separable from, or
+merely link (or bind by name) to the interfaces of, the Work and Derivative
+Works thereof.
+
+"Contribution" shall mean any work of authorship, including the original
+version of the Work and any modifications or additions to that Work or
+Derivative Works thereof, that is intentionally submitted to Licensor for
+inclusion in the Work by the copyright owner or by an individual or Legal
+Entity authorized to submit on behalf of the copyright owner. For the
+purposes of this definition, "submitted" means any form of electronic,
+verbal, or written communication sent to the Licensor or its representatives,
+including but not limited to communication on electronic mailing lists,
+source code control systems, and issue tracking systems that are managed by,
+or on behalf of, the Licensor for the purpose of discussing and improving the
+Work, but excluding communication that is conspicuously marked or otherwise
+designated in writing by the copyright owner as "Not a Contribution."
+
+"Contributor" shall mean Licensor and any individual or Legal Entity on
+behalf of whom a Contribution has been received by Licensor and subsequently
+incorporated within the Work.
+
+2. Grant of Copyright License. Subject to the terms and conditions of this
+License, each Contributor hereby grants to You a perpetual, worldwide,
+non-exclusive, no-charge, royalty-free, irrevocable copyright license to
+reproduce, prepare Derivative Works of, publicly display, publicly perform,
+sublicense, and distribute the Work and such Derivative Works in Source or
+Object form.
+
+3. Grant of Patent License. Subject to the terms and conditions of this
+License, each Contributor hereby grants to You a perpetual, worldwide,
+non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this
+section) patent license to make, have made, use, offer to sell, sell, import,
+and otherwise transfer the Work, where such license applies only to those
+patent claims licensable by such Contributor that are necessarily infringed
+by their Contribution(s) alone or by combination of their Contribution(s)
+with the Work to which such Contribution(s) was submitted. If You institute
+patent litigation against any entity (including a cross-claim or counterclaim
+in a lawsuit) alleging that the Work or a Contribution incorporated within
+the Work constitutes direct or contributory patent infringement, then any
+patent licenses granted to You under this License for that Work shall
+terminate as of the date such litigation is filed.
+
+4. Redistribution. You may reproduce and distribute copies of the Work or
+Derivative Works thereof in any medium, with or without modifications, and in
+Source or Object form, provided that You meet the following conditions:
+
+(a) You must give any other recipients of the Work or Derivative Works a copy
+of this License; and
+
+(b) You must cause any modified files to carry prominent notices stating that
+You changed the files; and
+
+(c) You must retain, in the Source form of any Derivative Works that You
+distribute, all copyright, patent, trademark, and attribution notices from the
+Source form of the Work, excluding those notices that do not pertain to any
+part of the Derivative Works; and
+
+(d) If the Work includes a "NOTICE" text file as part of its distribution,
+then any Derivative Works that You distribute must include a readable copy of
+the attribution notices contained within such NOTICE file, excluding those
+notices that do not pertain to any part of the Derivative Works, in at least
+one of the following places: within a NOTICE text file distributed as part of
+the Derivative Works; within the Source form or documentation, if provided
+along with the Derivative Works; or within a display generated by the
+Derivative Works, if and wherever such third-party notices normally appear.
+The contents of the NOTICE file are for informational purposes only and do
+not modify the License. You may add Your own attribution notices within
+Derivative Works that You distribute, alongside or as an addendum to the
+NOTICE text from the Work, provided that such additional attribution notices
+cannot be construed as modifying the License.
+
+You may add Your own copyright statement to Your modifications and may
+provide additional or different license terms and conditions for use,
+reproduction, or distribution of Your modifications, or for any such
+Derivative Works as a whole, provided Your use, reproduction, and
+distribution of the Work otherwise complies with the conditions stated in
+this License.
+
+5. Submission of Contributions. Unless You explicitly state otherwise, any
+Contribution intentionally submitted for inclusion in the Work by You to the
+Licensor shall be under the terms and conditions of this License, without any
+additional terms or conditions. Notwithstanding the above, nothing herein
+shall supersede or modify the terms of any separate license agreement you may
+have executed with Licensor regarding such Contributions.
+
+6. Trademarks. This License does not grant permission to use the trade names,
+trademarks, service marks, or product names of the Licensor, except as
+required for reasonable and customary use in describing the origin of the Work
+and reproducing the content of the NOTICE file.
+
+7. Disclaimer of Warranty. Unless required by applicable law or agreed to in
+writing, Licensor provides the Work on an "AS IS" BASIS, WITHOUT WARRANTIES
+OR CONDITIONS OF ANY KIND, either express or implied, including, without
+limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT,
+MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely
+responsible for determining the appropriateness of using or redistributing
+the Work and assume any risks associated with Your exercise of permissions
+under this License.
+
+8. Limitation of Liability. In no event and under no legal theory, whether in
+tort (including negligence), contract, or otherwise, unless required by
+applicable law (such as deliberate and grossly negligent acts) or agreed to in
+writing, shall any Contributor be liable to You for damages, including any
+direct, indirect, special, incidental, or consequential damages of any
+character arising as a result of this License or out of the use or inability
+to use the Work (including but not limited to damages for loss of goodwill,
+work stoppage, computer failure or malfunction, or any and all other
+commercial damages or losses), even if such Contributor has been advised of
+the possibility of such damages.
+
+9. Accepting Warranty or Additional Liability. While redistributing the Work
+or Derivative Works thereof, You may choose to offer, and charge a fee for,
+acceptance of support, warranty, indemnity, or other liability obligations
+and/or rights consistent with this License. However, in accepting such
+obligations, You may act only on Your own behalf and on Your sole
+responsibility, not on behalf of any other Contributor, and only if You agree
+to indemnify, defend, and hold each Contributor harmless for any liability
+incurred by, or claims asserted against, such Contributor by reason of your
+accepting any such warranty or additional liability.
+
+END OF TERMS AND CONDITIONS
+
+Copyright 2026 Synria Robotics

sparkrt-0.1.0rc1/PKG-INFO

@@ -0,0 +1,334 @@
+Metadata-Version: 2.4
+Name: SparkRT
+Version: 0.1.0rc1
+Summary: SparkRT - edge-side low-latency inference runtime for SparkMind2 robot/VLA/world models
+Author: Synria Robotics
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/Synria-Robotics/SparkRT
+Project-URL: Repository, https://github.com/Synria-Robotics/SparkRT.git
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.23
+Provides-Extra: presets
+Requires-Dist: omegaconf>=2.3; extra == "presets"
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Dynamic: license-file
+
+# SparkRT
+
+**SparkRT is a low-latency inference SDK for SparkMind2 robot policies.** Load a
+trained checkpoint and run it from your control loop in a few lines - no
+knowledge of SparkMind2 internals, tensor layouts, or preprocessor key names
+required:
+
+```python
+import sparkrt
+from sparkrt import Observation
+
+policy = sparkrt.load_policy("/path/to/checkpoints/020000/pretrained_model")
+
+obs = Observation(
+    images={"image": rgb_frame, "image2": wrist_frame},  # numpy HWC RGB uint8
+    state=robot_state,                                    # 1-D float array
+    instruction="pick up the object and place it in the basket",
+)
+action = policy.step(obs)   # -> numpy action for your robot
+```
+
+Under the hood SparkRT wraps trained SparkMind2 agents and optimizes the
+action-inference hot path without changing SparkMind2 model definitions,
+checkpoint formats, or pre/post-processing logic:
+
+- SparkMind2 owns training, checkpoints, model modules, normalization,
+  tokenizers, environment logic, and evaluation metrics.
+- SparkRT owns the ergonomic `Policy` SDK, inference sessions, action queues,
+  runtime configuration, and pluggable execution backends.
+
+Detailed benchmark results and experiment history are kept in
+[docs/EXPERIMENTS.md](docs/EXPERIMENTS.md). Backend integration guidance is in
+[docs/INTEGRATION.md](docs/INTEGRATION.md).
+
+## Status
+
+SparkRT currently supports:
+
+- ACT policies (single-shot action chunking, optional temporal ensemble).
+- Pi0.5 policies (vision-language-action, flow-matching denoising).
+- Eager PyTorch execution.
+- CUDA Graph execution for fixed-shape hot regions.
+- TorchCompile execution for fused PyTorch/Inductor kernels.
+- Preset-based runtime configuration.
+- A SparkMind2 LIBERO eval wrapper for development certification.
+
+The public SDK is Python-first. Native C++/CUDA or TensorRT backends can be added
+later behind the same `ExecutionBackend` contract.
+
+## Installation
+
+Install inside the SparkMind2 environment:
+
+```bash
+source /path/to/SparkMind2/.venv/bin/activate
+pip install -e /path/to/SparkRT
+```
+
+SparkMind2 is a local dependency and should be installed or available on
+`PYTHONPATH` separately. `torch` is intentionally not pinned by SparkRT so the
+runtime can use the CUDA/Torch build provided by the target machine.
+
+YAML presets require the optional preset extra:
+
+```bash
+pip install -e '/path/to/SparkRT[presets]'
+```
+
+## Loading a Policy
+
+Load a SparkMind2 `pretrained_model` checkpoint directly:
+
+```python
+import sparkrt
+from sparkrt import RuntimeConfig
+
+policy = sparkrt.load_policy(
+    "/path/to/checkpoints/020000/pretrained_model",
+    config=RuntimeConfig.from_preset("latency"),  # optional; no config defaults to eager
+)
+```
+
+Or wrap an agent that SparkMind2 has already constructed:
+
+```python
+import sparkrt
+
+policy = sparkrt.from_sparkmind_agent(agent, config=RuntimeConfig.from_preset("quality"))
+```
+
+Both return a `Policy`. Inspect what the policy expects before building inputs:
+
+```python
+policy.camera_names          # e.g. ("image", "image2")  or  ("top",)
+policy.state_dim             # length of the state vector (0 if unused)
+policy.requires_instruction  # True for Pi0.5, False for ACT
+policy.action_dim            # environment action dimensionality
+```
+
+## Building an Observation
+
+`policy.step(obs)` takes an `Observation` describing one timestep. You supply raw
+camera frames and the robot state with **friendly camera names**; SparkRT handles
+the channel-order, dtype, normalization, batching, and tokenization internally.
+
+| Field | Type | Notes |
+|-------|------|-------|
+| `images` | `dict[str, ndarray]` | Keyed by `policy.camera_names`. Each frame: NumPy (or torch) **HWC RGB**, `uint8` `[0,255]` or float `[0,1]`. CHW is also accepted. A single-camera policy may take a bare array. |
+| `state` | 1-D array | Length `policy.state_dim`. Any range - SparkRT applies the checkpoint's normalization. |
+| `instruction` | `str` or `None` | Required when `policy.requires_instruction` is `True` (Pi0.5). Ignored by ACT. |
+
+Image frames are RGB with no batch dimension; SparkRT converts `HWC -> CHW`,
+scales `uint8 -> [0,1]`, and lets the checkpoint preprocessor handle batching.
+
+### ACT example (images + state, no language)
+
+ACT is a deterministic single-camera policy with no instruction:
+
+```python
+import numpy as np
+import sparkrt
+from sparkrt import Observation
+
+policy = sparkrt.load_policy("/path/to/act/checkpoints/200000/pretrained_model")
+# policy.camera_names == ("top",); policy.state_dim == 14; requires_instruction == False
+
+policy.reset()  # at the start of each episode
+for _ in range(horizon):
+    obs = Observation(
+        images={"top": camera_rgb},          # (480, 640, 3) uint8 HWC RGB
+        state=joint_positions,               # shape (14,) float
+    )
+    action = policy.step(obs)                # numpy action, shape (action_dim,)
+    robot.apply(action)
+```
+
+### Pi0.5 example (multi-camera + state + instruction)
+
+Pi0.5 is a vision-language-action policy with two cameras and a task prompt:
+
+```python
+import numpy as np
+import sparkrt
+from sparkrt import Observation, RuntimeConfig
+
+policy = sparkrt.load_policy(
+    "/path/to/pi05/checkpoints/020000/pretrained_model",
+    config=RuntimeConfig.from_preset("latency"),
+)
+# policy.camera_names == ("image", "image2"); policy.state_dim == 8; requires_instruction == True
+
+policy.reset()
+for _ in range(horizon):
+    obs = Observation(
+        images={
+            "image": base_rgb,    # (256, 256, 3) uint8 HWC RGB
+            "image2": wrist_rgb,  # (256, 256, 3) uint8 HWC RGB
+        },
+        state=robot_state,        # shape (8,) float
+        instruction="pick up the object and place it in the basket",
+    )
+    action = policy.step(obs)
+    robot.apply(action)
+```
+
+For advanced use (parity checks, raw chunk prediction, custom loops) the
+underlying session is available as `policy.session`.
+
+## Runtime Presets
+
+Built-in presets live under [sparkrt/config/presets](sparkrt/config/presets).
+They are intended as named deployment profiles rather than one-off benchmark
+commands. Calling `sparkrt.load_policy(..., config=None)` uses the eager backend;
+`RuntimeConfig()` has the same dependency-light eager default. Presets are
+explicit opt-in profiles and require `sparkrt[presets]` / `omegaconf`.
+
+| Preset | Purpose |
+|--------|---------|
+| `default` / `safe` | Conservative CUDA Graph path with the checkpoint's original Pi0.5 schedule. Use when bit-exact behavior versus eager SparkMind2 is required. |
+| `quality` | Conservative TorchCompile path with the full Pi0.5 denoise schedule. Use when you want compile acceleration while avoiding reduced-step solver risk. |
+| `latency` | Aggressive TorchCompile path with fewer Pi0.5 denoise steps. Use when lowest single-stream latency is more important than strict task-level stability. |
+| `memory` | Development profile for the INT8-artifact memory path. Artifact loading is currently handled by the quantization scripts, not the public `load_policy` API. |
+
+Override individual fields when needed:
+
+```python
+from sparkrt.config import RuntimeConfig
+
+cfg = RuntimeConfig.from_preset("quality", kind="torchcompile")
+cfg = RuntimeConfig.from_preset("latency", num_steps=8)
+```
+
+Programmatic configuration is also available for deployments that should not
+load YAML:
+
+```python
+from sparkrt.config import RuntimeConfig, BackendConfig, Pi05RuntimeConfig
+
+cfg = RuntimeConfig(
+    backend=BackendConfig(kind="torchcompile"),
+    pi05=Pi05RuntimeConfig(num_steps=10, schedule="uniform"),
+    device="cuda:0",
+)
+```
+
+## Backend Integration
+
+Recommended service shape:
+
+1. Load one SparkRT `Policy` per worker process and robot/environment stream.
+2. Warm the policy during worker startup, especially for TorchCompile.
+3. Call `policy.reset()` when the task changes or a new episode starts.
+4. Call `policy.step(obs)` from the control loop.
+5. Do not share a single stateful policy across concurrent tasks.
+
+For SparkMind2 eval flows that already run the policy pre/post-processors, use
+`sparkrt.eval.PreprocessedRuntimePolicy` rather than passing a `Policy` /
+`InferenceSession` directly. This avoids applying normalization and action
+post-processing twice.
+
+See [docs/INTEGRATION.md](docs/INTEGRATION.md) for the full integration plan,
+lifecycle recommendations, concurrency model, and failure handling.
+
+## Architecture
+
+```text
+Observation (friendly camera names)
+  -> Policy (SDK surface: step / reset / discovery)
+  -> InferenceSession (stateful action queue / reset semantics)
+  -> SparkMind processor (normalize / tokenize)
+  -> model adapter (model orchestration, including Pi0.5 denoising)
+  -> backend-compiled Regions (eager / CUDA Graph / TorchCompile)
+  -> SparkMind processor (action postprocess)
+  -> action
+```
+
+| Layer | Package | Responsibility |
+|-------|---------|----------------|
+| Processors | `sparkrt.processors` | Convert observations to model-ready batches and convert model actions back to environment actions. |
+| Adapters | `sparkrt.adapters` | Describe model computation as named fixed-shape `Region`s and own model-specific orchestration such as the Pi0.5 denoise loop. |
+| Backends | `sparkrt.backends` | Execute regions with eager PyTorch, CUDA Graph replay, or TorchCompile. |
+| Session | `sparkrt.session` | Own runtime state, action-chunk queues, reset semantics, and the model-agnostic action loop. |
+| Config | `sparkrt.config` | Provide frozen dataclass configuration plus optional YAML presets. |
+
+The core contracts in `sparkrt.core` are intentionally lightweight and import
+without SparkMind2 or torch.
+
+## Development
+
+Run dependency-light tests:
+
+```bash
+python -m pytest tests/ -q
+```
+
+Run the main Pi0.5 runtime benchmark:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 MUJOCO_GL=egl PYOPENGL_PLATFORM=egl \
+PYTHONPATH=/path/to/SparkRT:/path/to/SparkMind2 \
+    python scripts/bench/bench_pi05_runtime.py \
+    --checkpoint /path/to/pi05/pretrained_model \
+    --backend torchcompile --num-steps 10 --schedule uniform
+```
+
+Run the development LIBERO success-rate runner:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 MUJOCO_GL=egl PYOPENGL_PLATFORM=egl \
+PYTHONPATH=/path/to/SparkMind2:/path/to/SparkRT \
+    python scripts/eval/run_libero_eval_sparkrt.py \
+    --checkpoint /path/to/pretrained_model \
+    --preset quality \
+    --n-episodes 5 \
+    --seed 1000 \
+    --output-dir runs/my_eval
+```
+
+TorchCompile has a large cold-start cost. Use a persistent Inductor cache for
+long-running workers:
+
+```bash
+export TORCHINDUCTOR_CACHE_DIR=$HOME/.cache/sparkrt/inductor
+```
+
+## Documentation
+
+- [docs/INTEGRATION.md](docs/INTEGRATION.md): service/backend integration plan.
+- [docs/EXPERIMENTS.md](docs/EXPERIMENTS.md): benchmark and accuracy results.
+- [scripts/README.md](scripts/README.md): developer script reference.
+
+## Repository Layout
+
+```text
+sparkrt/
+  api.py          public load_policy / from_sparkmind_agent entry points
+  policy.py       Policy SDK facade (step / reset / discovery)
+  observation.py  Observation input type + camera-name conversion
+  config/         RuntimeConfig dataclasses and YAML presets
+  core/           backend, adapter, region, and shape contracts
+  processors/     SparkMind processor reuse layer
+  adapters/       ACT and Pi0.5 model adapters
+  backends/       eager, cudagraph, and torchcompile executors
+  session/        stateful inference session
+  io/             SparkMind2 checkpoint loading helpers
+  eval/           SparkMind2 eval-compatible policy wrapper
+
+scripts/
+  bench/          latency, profiling, and parity scripts
+  eval/           LIBERO dev eval runner
+  quant/          INT8 artifact export and benchmark scripts
+
+docs/
+  EXPERIMENTS.md
+  INTEGRATION.md
+```