simrig 0.3.0__tar.gz → 0.4.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (61) hide show
  1. {simrig-0.3.0/simrig.egg-info → simrig-0.4.0}/PKG-INFO +175 -17
  2. simrig-0.4.0/README.md +394 -0
  3. {simrig-0.3.0 → simrig-0.4.0}/pyproject.toml +2 -2
  4. {simrig-0.3.0 → simrig-0.4.0}/simrig/_version.py +1 -1
  5. {simrig-0.3.0 → simrig-0.4.0}/simrig/browser_render.py +19 -0
  6. simrig-0.4.0/simrig/browser_shell.py +333 -0
  7. {simrig-0.3.0 → simrig-0.4.0}/simrig/cli.py +67 -3
  8. {simrig-0.3.0 → simrig-0.4.0}/simrig/core.py +2 -1
  9. simrig-0.4.0/simrig/custom_env.py +248 -0
  10. {simrig-0.3.0 → simrig-0.4.0}/simrig/huggingface.py +7 -5
  11. {simrig-0.3.0 → simrig-0.4.0}/simrig/lambda_cloud.py +32 -2
  12. {simrig-0.3.0 → simrig-0.4.0}/simrig/model_view.py +135 -5
  13. simrig-0.4.0/simrig/networks.py +80 -0
  14. {simrig-0.3.0 → simrig-0.4.0}/simrig/playground_backend.py +380 -52
  15. simrig-0.4.0/simrig/presets.py +176 -0
  16. {simrig-0.3.0 → simrig-0.4.0}/simrig/preview.py +529 -57
  17. {simrig-0.3.0 → simrig-0.4.0}/simrig/rendering.py +11 -0
  18. {simrig-0.3.0 → simrig-0.4.0}/simrig/runtime.py +94 -0
  19. {simrig-0.3.0 → simrig-0.4.0}/simrig/three_scene.py +25 -0
  20. simrig-0.4.0/simrig/validate_env.py +495 -0
  21. {simrig-0.3.0 → simrig-0.4.0/simrig.egg-info}/PKG-INFO +175 -17
  22. {simrig-0.3.0 → simrig-0.4.0}/simrig.egg-info/SOURCES.txt +3 -0
  23. {simrig-0.3.0 → simrig-0.4.0}/simrig.egg-info/requires.txt +1 -1
  24. {simrig-0.3.0 → simrig-0.4.0}/tests/test_cli.py +45 -1
  25. {simrig-0.3.0 → simrig-0.4.0}/tests/test_custom_env.py +93 -1
  26. simrig-0.4.0/tests/test_huggingface.py +73 -0
  27. {simrig-0.3.0 → simrig-0.4.0}/tests/test_lambda_cloud.py +32 -3
  28. {simrig-0.3.0 → simrig-0.4.0}/tests/test_model_view.py +58 -0
  29. simrig-0.4.0/tests/test_networks.py +71 -0
  30. simrig-0.4.0/tests/test_playground_backend.py +259 -0
  31. simrig-0.4.0/tests/test_presets.py +87 -0
  32. simrig-0.4.0/tests/test_preview.py +108 -0
  33. {simrig-0.3.0 → simrig-0.4.0}/tests/test_rendering.py +35 -1
  34. {simrig-0.3.0 → simrig-0.4.0}/tests/test_runtime.py +30 -1
  35. simrig-0.4.0/tests/test_scaffold.py +165 -0
  36. simrig-0.3.0/README.md +0 -236
  37. simrig-0.3.0/simrig/browser_shell.py +0 -130
  38. simrig-0.3.0/simrig/custom_env.py +0 -109
  39. simrig-0.3.0/simrig/presets.py +0 -93
  40. simrig-0.3.0/simrig/validate_env.py +0 -211
  41. simrig-0.3.0/tests/test_huggingface.py +0 -36
  42. simrig-0.3.0/tests/test_playground_backend.py +0 -65
  43. simrig-0.3.0/tests/test_presets.py +0 -34
  44. simrig-0.3.0/tests/test_scaffold.py +0 -67
  45. {simrig-0.3.0 → simrig-0.4.0}/LICENSE +0 -0
  46. {simrig-0.3.0 → simrig-0.4.0}/setup.cfg +0 -0
  47. {simrig-0.3.0 → simrig-0.4.0}/simrig/__init__.py +0 -0
  48. {simrig-0.3.0 → simrig-0.4.0}/simrig/io.py +0 -0
  49. {simrig-0.3.0 → simrig-0.4.0}/simrig/live_view.py +0 -0
  50. {simrig-0.3.0 → simrig-0.4.0}/simrig/mujoco_backend.py +0 -0
  51. {simrig-0.3.0 → simrig-0.4.0}/simrig/paths.py +0 -0
  52. {simrig-0.3.0 → simrig-0.4.0}/simrig/scaffold.py +0 -0
  53. {simrig-0.3.0 → simrig-0.4.0}/simrig.egg-info/dependency_links.txt +0 -0
  54. {simrig-0.3.0 → simrig-0.4.0}/simrig.egg-info/entry_points.txt +0 -0
  55. {simrig-0.3.0 → simrig-0.4.0}/simrig.egg-info/top_level.txt +0 -0
  56. {simrig-0.3.0 → simrig-0.4.0}/tests/test_core.py +0 -0
  57. {simrig-0.3.0 → simrig-0.4.0}/tests/test_demo_reach.py +0 -0
  58. {simrig-0.3.0 → simrig-0.4.0}/tests/test_jax_compat.py +0 -0
  59. {simrig-0.3.0 → simrig-0.4.0}/tests/test_live_view.py +0 -0
  60. {simrig-0.3.0 → simrig-0.4.0}/tests/test_mujoco_integration.py +0 -0
  61. {simrig-0.3.0 → simrig-0.4.0}/tests/test_paths.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: simrig
3
- Version: 0.3.0
3
+ Version: 0.4.0
4
4
  Summary: Agent- and human-friendly workflows for MuJoCo Playground training and evaluation.
5
5
  Author: SimRig contributors
6
6
  License-Expression: MIT
@@ -8,7 +8,7 @@ Project-URL: Homepage, https://github.com/Su1eym4n/simrig
8
8
  Project-URL: Repository, https://github.com/Su1eym4n/simrig
9
9
  Project-URL: Issues, https://github.com/Su1eym4n/simrig/issues
10
10
  Project-URL: Changelog, https://github.com/Su1eym4n/simrig/blob/main/CHANGELOG.md
11
- Requires-Python: >=3.10
11
+ Requires-Python: >=3.11
12
12
  Description-Content-Type: text/markdown
13
13
  License-File: LICENSE
14
14
  Provides-Extra: mujoco
@@ -22,7 +22,7 @@ Requires-Dist: mujoco-mjx==3.10.0; extra == "playground"
22
22
  Requires-Dist: jax==0.10.2; extra == "playground"
23
23
  Requires-Dist: brax==0.14.2; extra == "playground"
24
24
  Requires-Dist: playground==0.2.0; extra == "playground"
25
- Requires-Dist: warp-lang==1.15.0; extra == "playground"
25
+ Requires-Dist: warp-lang==1.13.0; extra == "playground"
26
26
  Provides-Extra: dev
27
27
  Requires-Dist: build; extra == "dev"
28
28
  Requires-Dist: numpy; extra == "dev"
@@ -32,6 +32,16 @@ Dynamic: license-file
32
32
 
33
33
  # SimRig
34
34
 
35
+ <div align="center">
36
+
37
+ <img src="assets/g1-greeting.gif" alt="Unitree G1 waving hello in SimRig's interactive browser preview" width="100%">
38
+
39
+ <br>
40
+
41
+ <em>A pretrained Unitree G1 policy preview with a scripted greeting.</em>
42
+
43
+ </div>
44
+
35
45
  Turn MuJoCo robots into trained policies with agent-guided task design, PPO
36
46
  training, evaluation, and interactive previews.
37
47
 
@@ -46,6 +56,63 @@ Raw robot XML is not automatically a training task. SimRig helps the agent move
46
56
  from a model and a requested behavior to explicit observations, actions,
47
57
  rewards, resets, termination conditions, validation, training, and evaluation.
48
58
 
59
+ ## From prompt to simulation
60
+
61
+ ### Train Go1 through a smoke-gated cloud workflow
62
+
63
+ SimRig prepares the existing Go1 locomotion environment and runs local smoke
64
+ tests before requesting the user's Lambda Cloud details. The full cloud run
65
+ starts only after that handoff. The recorded result below shows the trained
66
+ checkpoint downloaded and running in SimRig's interactive browser preview.
67
+
68
+ <details>
69
+ <summary><strong>Prompt</strong></summary>
70
+
71
+ > Prepare the full Go1 robot training setup and run local smoke tests to verify
72
+ > everything works. Once the tests pass, ask me for my Lambda Cloud details
73
+ > before starting the full training run.
74
+
75
+ </details>
76
+
77
+ <img src="assets/go1-training.gif" alt="Codex running a smoke-gated Go1 training workflow and previewing the trained policy in SimRig" width="100%">
78
+
79
+ ### Trace a five-pointed star with Franka Panda
80
+
81
+ This example uses a directly scripted Cartesian trajectory and inverse
82
+ kinematics—no reinforcement learning is needed. The controller keeps the
83
+ end-effector orientation fixed, moves smoothly between star vertices, and
84
+ publishes the running simulation and visible trace through SimRig's browser
85
+ viewer.
86
+
87
+ <details>
88
+ <summary><strong>Prompt</strong></summary>
89
+
90
+ > Can you configure a Franka Panda robotic arm in simulation so that its end
91
+ > effector traces a five-pointed star trajectory?
92
+ >
93
+ > Please:
94
+ >
95
+ > - Use the Franka Panda arm and gripper.
96
+ > - Move the end effector along a clear five-pointed star path in Cartesian
97
+ > space.
98
+ > - Keep the end-effector orientation fixed and stable throughout the motion.
99
+ > - Use inverse kinematics, trajectory planning, or a direct scripted controller
100
+ > rather than reinforcement learning unless training is genuinely necessary.
101
+ > - Add a visible trajectory trace, marker, or drawing surface so the completed
102
+ > star can be verified.
103
+ > - Make the motion smooth, with controlled velocity and acceleration between
104
+ > each star vertex.
105
+ > - Provide the complete runnable script and all required launch commands.
106
+ > - Explain how to modify the star size, star position, drawing plane,
107
+ > end-effector height, motion speed, and number of repetitions.
108
+ > - State clearly whether the solution is directly scripted or trained, and
109
+ > explain why that method is appropriate.
110
+ > - Prefer the simplest reliable direct-control implementation.
111
+
112
+ </details>
113
+
114
+ <img src="assets/franka-panda-star.gif" alt="Codex configuring a Franka Panda arm to trace a five-pointed star in SimRig's browser viewer" width="100%">
115
+
49
116
  ## What SimRig can do
50
117
 
51
118
  | Goal | SimRig workflow |
@@ -61,7 +128,7 @@ supported backend.
61
128
 
62
129
  ## Installation
63
130
 
64
- SimRig requires Python 3.10 or newer. Install the Playground training stack
131
+ SimRig requires Python 3.11 or newer. Install the Playground training stack
65
132
  from PyPI:
66
133
 
67
134
  ```bash
@@ -125,10 +192,20 @@ activate automatically when the request matches its description.
125
192
  simrig list-envs --backend mujoco-playground
126
193
  simrig inspect-env Go1JoystickFlatTerrain
127
194
  simrig smoke Go1JoystickFlatTerrain --steps 10
128
- simrig train Go1JoystickFlatTerrain --preset smoke
195
+ simrig train Go1JoystickFlatTerrain --preset smoke --impl auto --seed 0
129
196
  ```
130
197
 
131
198
  Use the `smoke` preset before a longer `local` or `cloud` configuration.
199
+ For registered Playground environments, SimRig starts from the environment's
200
+ tuned Brax PPO/network configuration and declared domain randomizer, then
201
+ bounds the expensive dimensions for `smoke` or `local`. The `cloud` preset uses
202
+ the full upstream task configuration unless you pass explicit overrides.
203
+
204
+ `--impl auto` uses the environment's default implementation when supported. It
205
+ selects MuJoCo Warp on a JAX-visible GPU when the environment defaults to Warp,
206
+ and falls back to JAX on CPU-only hosts. Use `--impl jax` or `--impl warp` to
207
+ make the choice explicit. Disable an available randomizer only for a deliberate
208
+ baseline with `--no-domain-randomization`.
132
209
 
133
210
  ### Custom robot or scene
134
211
 
@@ -142,9 +219,16 @@ simrig view-model path/to/robot.xml --port 8766
142
219
  Open `http://127.0.0.1:8766/` to inspect the compiled model, orbit/zoom/pan the
143
220
  camera, and adjust named joints. The default `threejs` renderer sends MuJoCo's
144
221
  visual meshes and primitives to a GPU-accelerated WebGL scene, so camera motion
145
- stays smooth without streaming image frames from Python. If the MJCF defines a
146
- keyframe, the viewer starts from its first authored pose and **Reset Joints**
147
- restores it.
222
+ stays smooth. When the MJCF contains named cameras, the page also shows a
223
+ **Robot View** inset. Its default **Emulated** mode uses a second Three.js camera
224
+ with the compiled MuJoCo camera's live world pose and vertical field of view,
225
+ giving it the same visual design as the orbit view. Switch to **Sensor** for the
226
+ native MuJoCo offscreen image, use the camera dropdown to switch cameras, or
227
+ choose the initial camera with `--camera NAME`. The inset follows joint-slider
228
+ changes while the main Three.js camera remains freely movable. The emulation is
229
+ for human inspection; vision policies train on MJX/Warp pixels, not this WebGL
230
+ render. If the MJCF defines a keyframe, the viewer starts from its first
231
+ authored pose and **Reset Joints** restores it.
148
232
 
149
233
  The Three.js modules are pinned and loaded from jsDelivr, so the default viewer
150
234
  needs an internet connection when the page first loads. For an entirely local
@@ -189,13 +273,74 @@ simrig new-env my_task --model path/to/scene.xml --template mjx
189
273
  simrig validate-env envs/my_task.py
190
274
  simrig validate-env envs/my_task.py --runtime
191
275
  simrig smoke envs/my_task.py --steps 10
192
- simrig train envs/my_task.py --preset smoke
276
+ simrig train envs/my_task.py --preset smoke --seed 0
193
277
  ```
194
278
 
195
279
  The generated environment is a starter, not an invented task definition. The
196
280
  reward, observations, actions, resets, and termination logic remain explicit
197
281
  and editable in Python.
198
282
 
283
+ ### Train from rendered pixels
284
+
285
+ Custom environments can declare a Brax vision CNN instead of the legacy MLP.
286
+ The included cartpole example renders real 64x64 MuJoCo frames, stacks three
287
+ grayscale frames, feeds pixels plus the previous action to the actor, and gives
288
+ the critic additional simulator state:
289
+
290
+ ```bash
291
+ # CPU-safe metadata check
292
+ simrig validate-env examples/vision_cartpole.py --vision
293
+
294
+ # These require a JAX-visible CUDA GPU and MuJoCo Warp.
295
+ simrig validate-env examples/vision_cartpole.py --runtime --vision
296
+ simrig smoke examples/vision_cartpole.py --steps 5
297
+ simrig train examples/vision_cartpole.py --preset smoke \
298
+ --output runs/vision-cartpole-smoke
299
+ ```
300
+
301
+ A vision module declares literal `NETWORK_SPEC`, `VISION_SPEC`, and
302
+ `DEFAULT_CONFIG` mappings for import-free static validation. Runtime hooks
303
+ `network_spec()`, `vision_spec()`, and optionally `training_config()` may enrich
304
+ those declarations after dependencies are installed. SimRig persists the
305
+ selected `network_type` and complete network factory in `config.json`, then
306
+ reconstructs the same CNN for `eval`, `demo`, and `preview`. Checkpoints created
307
+ before vision support remain MLP by default.
308
+
309
+ #### Run the pretrained vision reference
310
+
311
+ The published reference policy was trained for 5,079,040 PPO steps with a
312
+ 1,000-step episode horizon. It completed all 1,000 requested steps without
313
+ termination for evaluation seeds 0 through 4. Install both optional extras so
314
+ SimRig can run the Playground environment and resolve the Hub artifact:
315
+
316
+ ```bash
317
+ python3.11 -m venv .venv
318
+ .venv/bin/python -m pip install -e ".[playground,hf]"
319
+
320
+ .venv/bin/simrig eval \
321
+ hf://ssuleiman/simrig-vision-cartpole/policy.params \
322
+ --env examples/vision_cartpole.py \
323
+ --hf-revision v1 \
324
+ --steps 1000 \
325
+ --seed 0
326
+
327
+ .venv/bin/simrig preview \
328
+ hf://ssuleiman/simrig-vision-cartpole/policy.params \
329
+ --env examples/vision_cartpole.py \
330
+ --hf-revision v1 \
331
+ --auto-reset \
332
+ --port 8765
333
+ ```
334
+
335
+ Both commands require a JAX-visible CUDA GPU and MuJoCo Warp. Hub resolution
336
+ downloads `policy.params` with its sibling `config.json` so SimRig can rebuild
337
+ the recorded vision CNN. Exact evaluation should use the recorded Python and
338
+ package versions. `--allow-runtime-mismatch` is only for an explicitly
339
+ qualitative preview on a different compatible runtime. The checkpoint,
340
+ training configuration, metrics, environment snapshot, and five-seed report
341
+ are published at
342
+ [ssuleiman/simrig-vision-cartpole](https://huggingface.co/ssuleiman/simrig-vision-cartpole).
343
+
199
344
  ### Evaluate and preview
200
345
 
201
346
  ```bash
@@ -208,15 +353,23 @@ simrig eval runs/<run-dir>/policy.params \
208
353
  simrig preview runs/<run-dir>/policy.params \
209
354
  --env Go1JoystickFlatTerrain \
210
355
  --command 0.5 0.0 0.0 \
356
+ --auto-reset \
211
357
  --port 8765
212
358
  ```
213
359
 
214
- Open `http://127.0.0.1:8765/` to orbit, zoom, pan, change commands, pause, and
215
- inspect the live rollout. Preview uses the Three.js renderer by default: the
360
+ Open `http://127.0.0.1:8765/` to orbit, zoom, pan, change supported commands,
361
+ toggle automatic episode reset, pause, and inspect the live rollout. Preview
362
+ uses the Three.js renderer by default: the
216
363
  policy advances on a server-side rollout clock while lightweight MuJoCo geom
217
364
  transforms update the browser scene. The camera follows the robot without
218
- streaming rendered image frames. Use `--render-mode mujoco` for the older local
219
- image-stream preview or `--render-mode topdown` for the schematic fallback.
365
+ coupling policy stepping to display rendering. Environments with named MuJoCo
366
+ cameras also get a **Robot View** inset, selectable without moving the human
367
+ orbit camera. **Emulated** renders the live authored camera pose and FOV in the
368
+ same Three.js scene; **Sensor** shows the native MuJoCo offscreen image for
369
+ comparison. Neither changes policy input: training and rollout inference keep
370
+ using the environment's configured observation pipeline. Use
371
+ `--render-mode mujoco` for the older full-page local image stream or
372
+ `--render-mode topdown` for the schematic fallback.
220
373
 
221
374
  ### Train on a Lambda Cloud GPU
222
375
 
@@ -231,7 +384,9 @@ simrig cloud lambda smoke INSTANCE_IP Go1JoystickFlatTerrain \
231
384
  --identity ~/Downloads/lambda-key.pem
232
385
  simrig cloud lambda train INSTANCE_IP Go1JoystickFlatTerrain \
233
386
  --identity ~/Downloads/lambda-key.pem \
234
- --preset smoke
387
+ --preset smoke \
388
+ --impl auto \
389
+ --seed 0
235
390
  ```
236
391
 
237
392
  Only after the environment and PPO smoke gates pass, start a detached large
@@ -241,9 +396,12 @@ complete [Lambda Cloud GPU guide](docs/lambda-cloud.md), including persistent
241
396
  storage, monitoring, artifact download, and shutdown reminders.
242
397
 
243
398
  Lambda preparation requires Python 3.11+ and installs a pinned Playground
244
- training stack. Every run records its Python and package versions; checkpoint
245
- eval, demo, and preview reject a different recorded runtime unless
246
- `--allow-runtime-mismatch` is explicitly selected for qualitative review.
399
+ training stack. Every run records its resolved PPO/network configuration,
400
+ implementation, seed, randomizer, source hashes, Git state, JAX devices,
401
+ precision-related environment, and package versions. Checkpoint eval, demo, and
402
+ preview reconstruct the recorded implementation and network, and reject a
403
+ different runtime unless `--allow-runtime-mismatch` is explicitly selected for
404
+ qualitative review.
247
405
 
248
406
  ## Documentation
249
407
 
simrig-0.4.0/README.md ADDED
@@ -0,0 +1,394 @@
1
+ # SimRig
2
+
3
+ <div align="center">
4
+
5
+ <img src="assets/g1-greeting.gif" alt="Unitree G1 waving hello in SimRig's interactive browser preview" width="100%">
6
+
7
+ <br>
8
+
9
+ <em>A pretrained Unitree G1 policy preview with a scripted greeting.</em>
10
+
11
+ </div>
12
+
13
+ Turn MuJoCo robots into trained policies with agent-guided task design, PPO
14
+ training, evaluation, and interactive previews.
15
+
16
+ SimRig combines:
17
+
18
+ - a Python CLI that inspects models, runs MuJoCo Playground environments,
19
+ trains Brax PPO policies, evaluates checkpoints, and serves previews;
20
+ - an agent skill that teaches Codex, Claude Code, and Cursor how to use that
21
+ pipeline safely.
22
+
23
+ Raw robot XML is not automatically a training task. SimRig helps the agent move
24
+ from a model and a requested behavior to explicit observations, actions,
25
+ rewards, resets, termination conditions, validation, training, and evaluation.
26
+
27
+ ## From prompt to simulation
28
+
29
+ ### Train Go1 through a smoke-gated cloud workflow
30
+
31
+ SimRig prepares the existing Go1 locomotion environment and runs local smoke
32
+ tests before requesting the user's Lambda Cloud details. The full cloud run
33
+ starts only after that handoff. The recorded result below shows the trained
34
+ checkpoint downloaded and running in SimRig's interactive browser preview.
35
+
36
+ <details>
37
+ <summary><strong>Prompt</strong></summary>
38
+
39
+ > Prepare the full Go1 robot training setup and run local smoke tests to verify
40
+ > everything works. Once the tests pass, ask me for my Lambda Cloud details
41
+ > before starting the full training run.
42
+
43
+ </details>
44
+
45
+ <img src="assets/go1-training.gif" alt="Codex running a smoke-gated Go1 training workflow and previewing the trained policy in SimRig" width="100%">
46
+
47
+ ### Trace a five-pointed star with Franka Panda
48
+
49
+ This example uses a directly scripted Cartesian trajectory and inverse
50
+ kinematics—no reinforcement learning is needed. The controller keeps the
51
+ end-effector orientation fixed, moves smoothly between star vertices, and
52
+ publishes the running simulation and visible trace through SimRig's browser
53
+ viewer.
54
+
55
+ <details>
56
+ <summary><strong>Prompt</strong></summary>
57
+
58
+ > Can you configure a Franka Panda robotic arm in simulation so that its end
59
+ > effector traces a five-pointed star trajectory?
60
+ >
61
+ > Please:
62
+ >
63
+ > - Use the Franka Panda arm and gripper.
64
+ > - Move the end effector along a clear five-pointed star path in Cartesian
65
+ > space.
66
+ > - Keep the end-effector orientation fixed and stable throughout the motion.
67
+ > - Use inverse kinematics, trajectory planning, or a direct scripted controller
68
+ > rather than reinforcement learning unless training is genuinely necessary.
69
+ > - Add a visible trajectory trace, marker, or drawing surface so the completed
70
+ > star can be verified.
71
+ > - Make the motion smooth, with controlled velocity and acceleration between
72
+ > each star vertex.
73
+ > - Provide the complete runnable script and all required launch commands.
74
+ > - Explain how to modify the star size, star position, drawing plane,
75
+ > end-effector height, motion speed, and number of repetitions.
76
+ > - State clearly whether the solution is directly scripted or trained, and
77
+ > explain why that method is appropriate.
78
+ > - Prefer the simplest reliable direct-control implementation.
79
+
80
+ </details>
81
+
82
+ <img src="assets/franka-panda-star.gif" alt="Codex configuring a Franka Panda arm to trace a five-pointed star in SimRig's browser viewer" width="100%">
83
+
84
+ ## What SimRig can do
85
+
86
+ | Goal | SimRig workflow |
87
+ |---|---|
88
+ | Train a known Playground robot | Inspect the environment, smoke-test it, train, evaluate, and preview |
89
+ | Use a custom MJCF/XML robot | Inspect the model, define the task, create an editable environment, then validate and train |
90
+ | Build locomotion or posture behaviors | Design command tracking, contacts, rewards, failures, and evaluation scenarios |
91
+ | Build custom scenes | Add terrain, props, targets, sensors, cameras, or contact rules in ordinary MJCF and Python |
92
+ | Evaluate an existing policy | Run reproducible headless rollouts and open a browser or native MuJoCo preview |
93
+
94
+ SimRig v0 uses MuJoCo and MuJoCo Playground. Isaac Lab is not currently a
95
+ supported backend.
96
+
97
+ ## Installation
98
+
99
+ SimRig requires Python 3.11 or newer. Install the Playground training stack
100
+ from PyPI:
101
+
102
+ ```bash
103
+ python3.12 -m venv .venv
104
+ source .venv/bin/activate
105
+ python -m pip install --upgrade pip
106
+ python -m pip install "simrig[playground]"
107
+
108
+ simrig --version
109
+ ```
110
+
111
+ For an editable source installation, clone the repository and install from its
112
+ root instead:
113
+
114
+ ```bash
115
+ git clone https://github.com/Su1eym4n/simrig.git
116
+ cd simrig
117
+ python3.12 -m venv .venv
118
+ source .venv/bin/activate
119
+ python -m pip install --upgrade pip
120
+ python -m pip install -e ".[playground]"
121
+ ```
122
+
123
+ Development dependencies, tests, and contribution checks are documented in
124
+ [CONTRIBUTING.md](CONTRIBUTING.md).
125
+
126
+ ## Install the agent skill
127
+
128
+ Inside this repository, Codex discovers the SimRig skill automatically through
129
+ `.agents/skills/simrig`.
130
+
131
+ To make the skill available from any project, install it globally with the
132
+ Skills CLI:
133
+
134
+ ```bash
135
+ npx skills add Su1eym4n/simrig --skill simrig --global
136
+ ```
137
+
138
+ The installer supports Codex, Claude Code, Cursor, and other agents. Restart the
139
+ agent after installation. See
140
+ [Agent skill installation](docs/skill-installation.md) for provider-specific
141
+ commands, manual installation, and troubleshooting.
142
+
143
+ ## Use SimRig
144
+
145
+ Open a project containing a MuJoCo robot or scene and ask the agent naturally:
146
+
147
+ > Train this MuJoCo robot to walk forward.
148
+
149
+ > Create a crouching task for this robot, smoke-test it, and start a small
150
+ > training run.
151
+
152
+ > Evaluate this checkpoint across five seeds and preview the policy.
153
+
154
+ You can invoke the workflow explicitly with `$simrig`, but the skill can also
155
+ activate automatically when the request matches its description.
156
+
157
+ ### Existing Playground environment
158
+
159
+ ```bash
160
+ simrig list-envs --backend mujoco-playground
161
+ simrig inspect-env Go1JoystickFlatTerrain
162
+ simrig smoke Go1JoystickFlatTerrain --steps 10
163
+ simrig train Go1JoystickFlatTerrain --preset smoke --impl auto --seed 0
164
+ ```
165
+
166
+ Use the `smoke` preset before a longer `local` or `cloud` configuration.
167
+ For registered Playground environments, SimRig starts from the environment's
168
+ tuned Brax PPO/network configuration and declared domain randomizer, then
169
+ bounds the expensive dimensions for `smoke` or `local`. The `cloud` preset uses
170
+ the full upstream task configuration unless you pass explicit overrides.
171
+
172
+ `--impl auto` uses the environment's default implementation when supported. It
173
+ selects MuJoCo Warp on a JAX-visible GPU when the environment defaults to Warp,
174
+ and falls back to JAX on CPU-only hosts. Use `--impl jax` or `--impl warp` to
175
+ make the choice explicit. Disable an available randomizer only for a deliberate
176
+ baseline with `--no-domain-randomization`.
177
+
178
+ ### Custom robot or scene
179
+
180
+ Inspect the model before designing a task:
181
+
182
+ ```bash
183
+ simrig inspect-model path/to/robot.xml --save-report
184
+ simrig view-model path/to/robot.xml --port 8766
185
+ ```
186
+
187
+ Open `http://127.0.0.1:8766/` to inspect the compiled model, orbit/zoom/pan the
188
+ camera, and adjust named joints. The default `threejs` renderer sends MuJoCo's
189
+ visual meshes and primitives to a GPU-accelerated WebGL scene, so camera motion
190
+ stays smooth. When the MJCF contains named cameras, the page also shows a
191
+ **Robot View** inset. Its default **Emulated** mode uses a second Three.js camera
192
+ with the compiled MuJoCo camera's live world pose and vertical field of view,
193
+ giving it the same visual design as the orbit view. Switch to **Sensor** for the
194
+ native MuJoCo offscreen image, use the camera dropdown to switch cameras, or
195
+ choose the initial camera with `--camera NAME`. The inset follows joint-slider
196
+ changes while the main Three.js camera remains freely movable. The emulation is
197
+ for human inspection; vision policies train on MJX/Warp pixels, not this WebGL
198
+ render. If the MJCF defines a keyframe, the viewer starts from its first
199
+ authored pose and **Reset Joints** restores it.
200
+
201
+ The Three.js modules are pinned and loaded from jsDelivr, so the default viewer
202
+ needs an internet connection when the page first loads. For an entirely local
203
+ MuJoCo-rendered image stream, use:
204
+
205
+ ```bash
206
+ simrig view-model path/to/robot.xml --render-mode mujoco --port 8766
207
+ ```
208
+
209
+ Use `--render-mode topdown` only for the schematic debugging fallback.
210
+
211
+ ### View a running MuJoCo script
212
+
213
+ Standalone controllers can publish the `MjModel` and `MjData` they already own
214
+ to the same Three.js viewer. SimRig does not step, pause, or replay the script:
215
+
216
+ ```python
217
+ from simrig import LiveWebViewer
218
+
219
+ with LiveWebViewer(
220
+ model,
221
+ data,
222
+ name="my controller",
223
+ tracking_body="end_effector",
224
+ ) as web:
225
+ while running:
226
+ with web.lock:
227
+ data.ctrl[:] = controller(data)
228
+ mujoco.mj_step(model, data)
229
+ web.sync(phase="moving")
230
+ ```
231
+
232
+ Open the printed `http://127.0.0.1:8767/` URL. The page receives lightweight
233
+ geom transforms while the Python script retains full control of simulation
234
+ timing and state. A named `tracking_body` also draws its live path. Use
235
+ `wait_for_client()` when motion should begin only after the page is ready.
236
+
237
+ After defining the task, scaffold and validate an editable environment:
238
+
239
+ ```bash
240
+ simrig new-env my_task --model path/to/scene.xml --template mjx
241
+ simrig validate-env envs/my_task.py
242
+ simrig validate-env envs/my_task.py --runtime
243
+ simrig smoke envs/my_task.py --steps 10
244
+ simrig train envs/my_task.py --preset smoke --seed 0
245
+ ```
246
+
247
+ The generated environment is a starter, not an invented task definition. The
248
+ reward, observations, actions, resets, and termination logic remain explicit
249
+ and editable in Python.
250
+
251
+ ### Train from rendered pixels
252
+
253
+ Custom environments can declare a Brax vision CNN instead of the legacy MLP.
254
+ The included cartpole example renders real 64x64 MuJoCo frames, stacks three
255
+ grayscale frames, feeds pixels plus the previous action to the actor, and gives
256
+ the critic additional simulator state:
257
+
258
+ ```bash
259
+ # CPU-safe metadata check
260
+ simrig validate-env examples/vision_cartpole.py --vision
261
+
262
+ # These require a JAX-visible CUDA GPU and MuJoCo Warp.
263
+ simrig validate-env examples/vision_cartpole.py --runtime --vision
264
+ simrig smoke examples/vision_cartpole.py --steps 5
265
+ simrig train examples/vision_cartpole.py --preset smoke \
266
+ --output runs/vision-cartpole-smoke
267
+ ```
268
+
269
+ A vision module declares literal `NETWORK_SPEC`, `VISION_SPEC`, and
270
+ `DEFAULT_CONFIG` mappings for import-free static validation. Runtime hooks
271
+ `network_spec()`, `vision_spec()`, and optionally `training_config()` may enrich
272
+ those declarations after dependencies are installed. SimRig persists the
273
+ selected `network_type` and complete network factory in `config.json`, then
274
+ reconstructs the same CNN for `eval`, `demo`, and `preview`. Checkpoints created
275
+ before vision support remain MLP by default.
276
+
277
+ #### Run the pretrained vision reference
278
+
279
+ The published reference policy was trained for 5,079,040 PPO steps with a
280
+ 1,000-step episode horizon. It completed all 1,000 requested steps without
281
+ termination for evaluation seeds 0 through 4. Install both optional extras so
282
+ SimRig can run the Playground environment and resolve the Hub artifact:
283
+
284
+ ```bash
285
+ python3.11 -m venv .venv
286
+ .venv/bin/python -m pip install -e ".[playground,hf]"
287
+
288
+ .venv/bin/simrig eval \
289
+ hf://ssuleiman/simrig-vision-cartpole/policy.params \
290
+ --env examples/vision_cartpole.py \
291
+ --hf-revision v1 \
292
+ --steps 1000 \
293
+ --seed 0
294
+
295
+ .venv/bin/simrig preview \
296
+ hf://ssuleiman/simrig-vision-cartpole/policy.params \
297
+ --env examples/vision_cartpole.py \
298
+ --hf-revision v1 \
299
+ --auto-reset \
300
+ --port 8765
301
+ ```
302
+
303
+ Both commands require a JAX-visible CUDA GPU and MuJoCo Warp. Hub resolution
304
+ downloads `policy.params` with its sibling `config.json` so SimRig can rebuild
305
+ the recorded vision CNN. Exact evaluation should use the recorded Python and
306
+ package versions. `--allow-runtime-mismatch` is only for an explicitly
307
+ qualitative preview on a different compatible runtime. The checkpoint,
308
+ training configuration, metrics, environment snapshot, and five-seed report
309
+ are published at
310
+ [ssuleiman/simrig-vision-cartpole](https://huggingface.co/ssuleiman/simrig-vision-cartpole).
311
+
312
+ ### Evaluate and preview
313
+
314
+ ```bash
315
+ simrig eval runs/<run-dir>/policy.params \
316
+ --env Go1JoystickFlatTerrain \
317
+ --steps 500 \
318
+ --seed 0 \
319
+ --command 0.5 0.0 0.0
320
+
321
+ simrig preview runs/<run-dir>/policy.params \
322
+ --env Go1JoystickFlatTerrain \
323
+ --command 0.5 0.0 0.0 \
324
+ --auto-reset \
325
+ --port 8765
326
+ ```
327
+
328
+ Open `http://127.0.0.1:8765/` to orbit, zoom, pan, change supported commands,
329
+ toggle automatic episode reset, pause, and inspect the live rollout. Preview
330
+ uses the Three.js renderer by default: the
331
+ policy advances on a server-side rollout clock while lightweight MuJoCo geom
332
+ transforms update the browser scene. The camera follows the robot without
333
+ coupling policy stepping to display rendering. Environments with named MuJoCo
334
+ cameras also get a **Robot View** inset, selectable without moving the human
335
+ orbit camera. **Emulated** renders the live authored camera pose and FOV in the
336
+ same Three.js scene; **Sensor** shows the native MuJoCo offscreen image for
337
+ comparison. Neither changes policy input: training and rollout inference keep
338
+ using the environment's configured observation pipeline. Use
339
+ `--render-mode mujoco` for the older full-page local image stream or
340
+ `--render-mode topdown` for the schematic fallback.
341
+
342
+ ### Train on a Lambda Cloud GPU
343
+
344
+ After launching a Lambda On-Demand instance with an SSH key, SimRig can connect,
345
+ sync this checkout, verify JAX GPU visibility, train, monitor a detached run,
346
+ and download its artifacts:
347
+
348
+ ```bash
349
+ simrig cloud lambda connect INSTANCE_IP --identity ~/Downloads/lambda-key.pem
350
+ simrig cloud lambda prepare INSTANCE_IP --identity ~/Downloads/lambda-key.pem
351
+ simrig cloud lambda smoke INSTANCE_IP Go1JoystickFlatTerrain \
352
+ --identity ~/Downloads/lambda-key.pem
353
+ simrig cloud lambda train INSTANCE_IP Go1JoystickFlatTerrain \
354
+ --identity ~/Downloads/lambda-key.pem \
355
+ --preset smoke \
356
+ --impl auto \
357
+ --seed 0
358
+ ```
359
+
360
+ Only after the environment and PPO smoke gates pass, start a detached large
361
+ run with `--preset cloud --detach`. SimRig operates on an instance you already
362
+ provisioned; it never launches or terminates billable Lambda resources. See the
363
+ complete [Lambda Cloud GPU guide](docs/lambda-cloud.md), including persistent
364
+ storage, monitoring, artifact download, and shutdown reminders.
365
+
366
+ Lambda preparation requires Python 3.11+ and installs a pinned Playground
367
+ training stack. Every run records its resolved PPO/network configuration,
368
+ implementation, seed, randomizer, source hashes, Git state, JAX devices,
369
+ precision-related environment, and package versions. Checkpoint eval, demo, and
370
+ preview reconstruct the recorded implementation and network, and reject a
371
+ different runtime unless `--allow-runtime-mismatch` is explicitly selected for
372
+ qualitative review.
373
+
374
+ ## Documentation
375
+
376
+ - [Examples](examples/README.md)
377
+ - [Agent workflow](docs/agent_workflow.md)
378
+ - [Lambda Cloud GPU training](docs/lambda-cloud.md)
379
+ - [Agent skill installation](docs/skill-installation.md)
380
+ - [Contributing](CONTRIBUTING.md)
381
+ - [SimRig skill source](skills/simrig/SKILL.md)
382
+
383
+ ## Outputs
384
+
385
+ SimRig writes project-local artifacts:
386
+
387
+ - `reports/` — model, environment, and evaluation reports
388
+ - `runs/` — training configuration, metrics, checkpoints, and policy parameters
389
+ - `envs/` — editable custom environment modules
390
+ - `artifacts/` and `configs/` — user-managed outputs and configuration
391
+
392
+ ## License
393
+
394
+ [MIT](LICENSE)