@iwsdk/create 0.4.1 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (107) hide show
  1. package/README.md +103 -41
  2. package/dist/cli.js +1003 -3418
  3. package/dist/guidance/agents/.agents/skills/iwsdk-debug/SKILL.md +96 -0
  4. package/dist/guidance/agents/.agents/skills/iwsdk-depth-occlusion/SKILL.md +135 -0
  5. package/dist/guidance/agents/.agents/skills/iwsdk-grab/SKILL.md +129 -0
  6. package/dist/guidance/agents/.agents/skills/iwsdk-physics/SKILL.md +92 -0
  7. package/dist/guidance/agents/.agents/skills/iwsdk-physics/references/component-reference.md +154 -0
  8. package/dist/guidance/agents/.agents/skills/iwsdk-physics/references/tuning-and-config.md +88 -0
  9. package/dist/guidance/agents/.agents/skills/iwsdk-physics/references/workflows.md +303 -0
  10. package/dist/guidance/agents/.agents/skills/iwsdk-planner/SKILL.md +364 -0
  11. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/api-reference.md +1489 -0
  12. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/build-milestones.md +241 -0
  13. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/design-deck.md +101 -0
  14. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/grounding.md +176 -0
  15. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/ideation.md +156 -0
  16. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/verification.md +199 -0
  17. package/dist/guidance/agents/.agents/skills/iwsdk-ray/SKILL.md +159 -0
  18. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/SKILL.md +255 -0
  19. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/agents/openai.yaml +5 -0
  20. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/asset-authoring.md +295 -0
  21. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/composition-patterns.md +243 -0
  22. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/image-intake.md +191 -0
  23. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/review-and-stop.md +99 -0
  24. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/scene-format.md +458 -0
  25. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/text-intake.md +82 -0
  26. package/dist/guidance/agents/.agents/skills/iwsdk-ui/SKILL.md +208 -0
  27. package/dist/guidance/claude/.claude/agents/iwsdk-project-code-reviewer.md +450 -0
  28. package/dist/guidance/claude/.claude/rules/assets-and-manifest.md +55 -0
  29. package/dist/guidance/claude/.claude/rules/ecs-api.md +102 -0
  30. package/dist/guidance/claude/.claude/rules/scene-json.md +64 -0
  31. package/dist/guidance/claude/.claude/rules/uikitml.md +60 -0
  32. package/dist/guidance/claude/.claude/settings.json +19 -0
  33. package/dist/guidance/claude/.claude/skills/iwsdk-debug/SKILL.md +96 -0
  34. package/dist/guidance/claude/.claude/skills/iwsdk-depth-occlusion/SKILL.md +135 -0
  35. package/dist/guidance/claude/.claude/skills/iwsdk-grab/SKILL.md +129 -0
  36. package/dist/guidance/claude/.claude/skills/iwsdk-physics/SKILL.md +92 -0
  37. package/dist/guidance/claude/.claude/skills/iwsdk-physics/references/component-reference.md +154 -0
  38. package/dist/guidance/claude/.claude/skills/iwsdk-physics/references/tuning-and-config.md +88 -0
  39. package/dist/guidance/claude/.claude/skills/iwsdk-physics/references/workflows.md +303 -0
  40. package/dist/guidance/claude/.claude/skills/iwsdk-planner/SKILL.md +364 -0
  41. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/api-reference.md +1489 -0
  42. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/build-milestones.md +241 -0
  43. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/design-deck.md +101 -0
  44. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/grounding.md +176 -0
  45. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/ideation.md +156 -0
  46. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/verification.md +199 -0
  47. package/dist/guidance/claude/.claude/skills/iwsdk-ray/SKILL.md +159 -0
  48. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/SKILL.md +255 -0
  49. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/agents/openai.yaml +5 -0
  50. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/asset-authoring.md +295 -0
  51. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/composition-patterns.md +243 -0
  52. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/image-intake.md +191 -0
  53. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/review-and-stop.md +99 -0
  54. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/scene-format.md +458 -0
  55. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/text-intake.md +82 -0
  56. package/dist/guidance/claude/.claude/skills/iwsdk-ui/SKILL.md +208 -0
  57. package/dist/guidance/claude/CLAUDE.md +142 -0
  58. package/dist/guidance/codex/.codex/config.toml +2 -0
  59. package/dist/guidance/common/AGENTS.md +151 -0
  60. package/dist/guidance/copilot/.github/instructions/assets-and-manifest.instructions.md +52 -0
  61. package/dist/guidance/copilot/.github/instructions/ecs-api.instructions.md +100 -0
  62. package/dist/guidance/copilot/.github/instructions/scene-json.instructions.md +63 -0
  63. package/dist/guidance/copilot/.github/instructions/uikitml.instructions.md +59 -0
  64. package/dist/guidance/cursor/.cursor/rules/assets-and-manifest.mdc +57 -0
  65. package/dist/guidance/cursor/.cursor/rules/ecs-api.mdc +104 -0
  66. package/dist/guidance/cursor/.cursor/rules/scene-json.mdc +66 -0
  67. package/dist/guidance/cursor/.cursor/rules/uikitml.mdc +62 -0
  68. package/dist/guidance/scoped-agents/public/scenes/AGENTS.md +63 -0
  69. package/dist/guidance/scoped-agents/public/ui/AGENTS.md +59 -0
  70. package/dist/guidance/scoped-agents/src/AGENTS.md +149 -0
  71. package/dist/template/common/README.md +13 -0
  72. package/dist/template/common/gitignore.template +45 -0
  73. package/dist/template/common/index.html +16 -0
  74. package/dist/template/common/nvmrc.template +1 -0
  75. package/dist/template/common/public/audio/chime.mp3 +0 -0
  76. package/dist/template/common/public/gltf/webxr-banner/banner.gltf +68 -0
  77. package/dist/template/common/public/textures/webxr.png +0 -0
  78. package/dist/template/common/public/ui/welcome.uikitml +98 -0
  79. package/dist/template/common/src/assets.ts +54 -0
  80. package/dist/template/common/src/components.ts +11 -0
  81. package/dist/template/common/src/index.ts +19 -0
  82. package/dist/template/common/src/panel.ts +38 -0
  83. package/dist/template/common/src/robot-component.ts +10 -0
  84. package/dist/template/common/src/robot.ts +38 -0
  85. package/dist/template/common/src/vite-env.d.ts +2 -0
  86. package/dist/template/common/tsconfig.json +15 -0
  87. package/dist/template/common/vite.config.ts +27 -0
  88. package/dist/template/common-js/README.md +13 -0
  89. package/dist/template/common-js/gitignore.template +45 -0
  90. package/dist/template/common-js/index.html +16 -0
  91. package/dist/template/common-js/nvmrc.template +1 -0
  92. package/dist/template/common-js/public/audio/chime.mp3 +0 -0
  93. package/dist/template/common-js/public/gltf/webxr-banner/banner.gltf +68 -0
  94. package/dist/template/common-js/public/textures/webxr.png +0 -0
  95. package/dist/template/common-js/public/ui/welcome.uikitml +98 -0
  96. package/dist/template/common-js/src/assets.js +45 -0
  97. package/dist/template/common-js/src/components.js +9 -0
  98. package/dist/template/common-js/src/index.js +14 -0
  99. package/dist/template/common-js/src/panel.js +31 -0
  100. package/dist/template/common-js/src/robot-component.js +8 -0
  101. package/dist/template/common-js/src/robot.js +37 -0
  102. package/dist/template/common-js/vite.config.js +25 -0
  103. package/dist/template/scenes/ar.iwsdk.scene.json +96 -0
  104. package/dist/template/scenes/immersive.iwsdk.scene.json +109 -0
  105. package/package.json +23 -17
  106. package/scripts/verify-release-ready.mjs +76 -0
  107. package/LICENSE +0 -21
@@ -0,0 +1,102 @@
1
+ ---
2
+ paths:
3
+ - "src/**/*.ts"
4
+ - "src/**/*.js"
5
+ ---
6
+
7
+ # ECS API reference
8
+
9
+ ## Field types
10
+
11
+ ```
12
+ Types.Float32 Float64 Int8 Int16 Int32 Boolean String FilePath
13
+ Types.Vec2 Vec3 Vec4 Color(RGBA) Entity Enum
14
+ Types.Object // avoid — not optimised
15
+ ```
16
+
17
+ ## Component
18
+
19
+ Declare in a system-free module, then list it in `src/components.ts` via
20
+ `defineComponents()` or the editor cannot author it.
21
+
22
+ ```ts
23
+ export const MyComponent = createComponent('MyComponent', {
24
+ speed: { type: Types.Float32, default: 1 },
25
+ tint: { type: Types.Color, default: [1, 1, 1, 1] }, // RGBA
26
+ });
27
+ ```
28
+
29
+ Optional metadata per field — `label`, `step`, `min`, `max`, `enum`, `help`,
30
+ `widget` — drives the editor inspector.
31
+
32
+ ## System
33
+
34
+ ```ts
35
+ export class MySystem extends createSystem(
36
+ {
37
+ items: { required: [MyComponent] },
38
+ active: { required: [MyComponent], excluded: [Grabbed] },
39
+ },
40
+ { speed: { type: Types.Float32, default: 1 } }, // optional config
41
+ ) {
42
+ private temp!: Vector3;
43
+
44
+ init() {
45
+ this.temp = new Vector3();
46
+ this.queries.items.subscribe('qualify', (entity) => { /* matched */ });
47
+ this.cleanupFuncs.push(this.config.speed.subscribe((v) => { /* … */ }));
48
+ }
49
+
50
+ update(delta: number, time: number) {
51
+ for (const entity of this.queries.active.entities) { /* no allocations */ }
52
+ }
53
+ }
54
+ ```
55
+
56
+ Queries take `required`, `excluded` and `where` (value predicates via `eq`, `ne`,
57
+ `lt`, `le`, `gt`, `ge`, `isin`, `nin`). `excluded` is the clean way to skip
58
+ entities in a transient state — e.g. excluding `Grabbed` so a held object is left
59
+ alone.
60
+
61
+ Register explicitly in `src/index.ts` with `world.registerSystem(MySystem)`.
62
+
63
+ Read fields with `entity.getValue(Component, 'field')`. For vector fields use
64
+ `entity.getVectorView(Component, 'field')` — it returns a `Float32Array` view with
65
+ no allocation, and `setValue` **throws** on vector fields in elics 3.4.x.
66
+
67
+ ## Interaction components
68
+
69
+ `RayInteractable` plus `Hovered`/`Pressed` covers both mouse/touch canvas input
70
+ and XR rays. Grab components: `OneHandGrabbable` and `TwoHandsGrabbable` are
71
+ proximity-based and respond to **squeeze**, while `DistanceGrabbable` is
72
+ ray-based and responds to **trigger**. `Grabbed` is a transient tag managed by
73
+ `GrabSystem` — read it, never add or remove it. `GrabSystem.useHandPinchForGrab`
74
+ defaults to **false**, so hand pinch does not grab unless you enable it.
75
+
76
+ `DistanceGrabbable` fields: `movementMode` (`MoveTowardsTarget`, `MoveFromTarget`,
77
+ `MoveAtSource`, `RotateAtSource`), `returnToOrigin`, `moveSpeedFactor`,
78
+ `detachOnGrab`, `translate`, `rotate`, `scale`, `targetPositionOffset`,
79
+ `targetQuaternionOffset`. `returnToOrigin: true` gives spring-back on release with
80
+ no custom code.
81
+
82
+ For a held object use `GrabSystem`'s public methods — `forceRelease(entity)`,
83
+ `getHolderHand(entity)` — never deep-import `Handle`.
84
+
85
+ ## XR input
86
+
87
+ ```ts
88
+ const pad = this.input.xr.gamepads.right;
89
+ pad?.getButtonDown(InputComponent.Trigger); // just pressed
90
+ pad?.getAxesValues(InputComponent.Thumbstick);
91
+ this.player.head; this.player.raySpaces.left; this.player.gripSpaces.right;
92
+ ```
93
+
94
+ Prefer `world.input.actions` for reusable intent (`locomotion.move`) over raw
95
+ buttons. Browser controls live at `world.input.keyboard` and
96
+ `world.input.browserGamepads`.
97
+
98
+ ## Lifecycle
99
+
100
+ `world.visibilityState` reports `NonImmersive`, `Visible`, `VisibleBlurred` —
101
+ pause simulation on blur. `world.camera.position` is local to `world.player`; use
102
+ `getWorldPosition()` when logic needs the true viewer position.
@@ -0,0 +1,64 @@
1
+ ---
2
+ paths:
3
+ - "public/scenes/**"
4
+ ---
5
+
6
+ # Scene JSON — things that fail silently
7
+
8
+ Format reference lives in
9
+ `.claude/skills/iwsdk-scene-composer/references/scene-format.md`: the document
10
+ boundary, content kinds, prefabs, patterns, light fields, authoring views. Read
11
+ it before composing. This file is only the short list of behaviours that produce
12
+ no error when you get them wrong.
13
+
14
+ Only `iwsdk.scene.v1` exists, units are always meters, and structural objects are
15
+ closed — an unknown field fails validation.
16
+
17
+ ## Imports stop at the authoring boundary
18
+
19
+ Authoring tools can validate and render a root with `imports`, so modular scratch
20
+ scenes remain useful for composition. The editable scene session and application
21
+ runtime deliberately reject imports. Run `scene_flatten_file` or
22
+ `npx iwsdk scene flatten` once, verify the source and output runtime hashes match,
23
+ then make the flat output the sole editable source of truth. Never re-flatten over
24
+ later human editor changes.
25
+
26
+ ## The player materialises at the scene origin
27
+
28
+ `0,0,0` unless the document authors `player.transform`. Nothing warns you. A
29
+ scene can pass every authored camera view and still spawn the player inside the
30
+ geometry, because review cameras are not the viewpoint that ships. Keep that
31
+ volume clear or place the player deliberately, and author a diagnostic view at
32
+ player eye height to prove it.
33
+
34
+ ## `visibleNodeIds` is the silent-failure detector
35
+
36
+ `renderStats.visibleNodeIds` lists what actually rendered. A node missing from it
37
+ did not appear — even when the image looks plausible and `valid` is `true`. Check
38
+ it against what you expect rather than trusting the picture. Single-sided
39
+ surfaces facing away from the camera are the usual cause, and they report nothing.
40
+
41
+ ## Component keys come in two forms
42
+
43
+ Root-level components use the namespaced form
44
+ (`com.iwsdk.components.DomeGradient`). Node-level components accept the bare name
45
+ (`RayInteractable`, `LocomotionEnvironment`). **The editor writes the namespaced
46
+ form on nodes**, so a file touched by both hands contains both. Match whatever the
47
+ node already uses instead of normalising.
48
+
49
+ `LevelRoot`, `LevelTag` and `Transform` are runtime-owned; never author them.
50
+
51
+ ## The scene owns the runtime camera
52
+
53
+ The hero authoring view supersedes `world.render.camera` from
54
+ `iwsdk.config.json` once the level initialises. Changing the config camera to fix
55
+ runtime framing will appear to do nothing — change the hero view instead.
56
+
57
+ ## Validation
58
+
59
+ `scene_render_file` validates, composes, lowers and renders without touching the
60
+ open document. A result is usable only when `valid: true` **and** a PNG comes
61
+ back. Render each changed module before the root.
62
+
63
+ For composing a scene rather than editing one, use the `iwsdk-scene-composer`
64
+ skill — it owns the intake, review and stop-rule procedure this file does not.
@@ -0,0 +1,60 @@
1
+ ---
2
+ paths:
3
+ - "public/ui/**"
4
+ ---
5
+
6
+ # UIKitML panels
7
+
8
+ HTML/CSS-shaped, not a browser engine. Validate rather than assume support.
9
+
10
+ **Numeric sizes are UIKit units (centimetres), not pixels.** `width: 95` is a
11
+ 0.95 m panel. Font sizes follow the same scale: `font-size: 3` is 3 cm tall.
12
+
13
+ **Panels are single-sided and face +Z.** To aim one at a camera, yaw by
14
+ `atan2(dx, dz)` toward it — there is no 180° term. A back-facing panel renders
15
+ nothing, reports no error, and still returns `valid: true` from
16
+ `scene_render_file`; the only signal is its absence from
17
+ `renderStats.visibleNodeIds`.
18
+
19
+ Verified working: `display: flex`, `flex-direction`, `align-items`,
20
+ `justify-content: space-between`, `gap`, `margin-*`, `padding-*`, `width`,
21
+ `height`, `background-color`, `border-color`, `border-width`, `border-radius`,
22
+ `font-size`, `font-weight`, `color`, `text-align`, `cursor`. Tags `div`, `span`,
23
+ `button` all work.
24
+
25
+ Expand shorthands (`padding: 12 18` → four properties), use unitless
26
+ line-height multipliers, and prefer uniform `border-color`/`border-width` over
27
+ directional variants. Parser errors include source locations.
28
+
29
+ ## Runtime API
30
+
31
+ Resolve the placed panel by **scene node id**, then elements by their `id`
32
+ attribute:
33
+
34
+ ```ts
35
+ const panel = world.getSceneObject<UIKitMLAsset>('config-panel');
36
+ const chip = panel?.getElementById<UIKit.Text>('finish-gold');
37
+ chip?.addEventListener('click', onClick);
38
+ chip?.setProperties({ backgroundColor: '#d8e6f5', color: '#0e1116' });
39
+ ```
40
+
41
+ `setProperties` updates both text (`{ text }`) and style
42
+ (`backgroundColor`, `color`, `borderColor`, `display`) — style changes do apply,
43
+ including over class-based styles from the `<style>` block. `UIKit.Text` is the
44
+ public element type; no deep import from `@pmndrs/uikit` is needed.
45
+
46
+ Three distinct identities, don't conflate them: the **manifest ID** names the
47
+ reusable asset, the **scene node ID** names the placed instance, and the element
48
+ `id` names a field inside it. Never depend on a transient entity index.
49
+
50
+ Give the panel node `RayInteractable` so clicks land.
51
+
52
+ The editor render does not run application systems, so `scene_screenshot` can
53
+ never prove a panel *behaves* — only `browser_screenshot`, against the runtime,
54
+ can.
55
+
56
+ ---
57
+
58
+ For the authoring procedure — isolated preview, placement, editing, wiring
59
+ runtime behavior, and the completion checklist — use the `iwsdk-ui` skill rather
60
+ than improvising. This file is the format reference it relies on.
@@ -0,0 +1,19 @@
1
+ {
2
+ "$schema": "https://json.schemastore.org/claude-code-settings.json",
3
+ "permissions": {
4
+ "allow": [
5
+ "mcp__iwsdk-runtime__*",
6
+ "mcp__iwsdk-reference__*",
7
+ "mcp__metavr__*",
8
+ "Skill(iwsdk-debug)",
9
+ "Skill(iwsdk-depth-occlusion)",
10
+ "Skill(iwsdk-grab)",
11
+ "Skill(iwsdk-physics)",
12
+ "Skill(iwsdk-scene-composer)",
13
+ "Skill(iwsdk-planner)",
14
+ "Skill(iwsdk-ray)",
15
+ "Skill(iwsdk-ui)"
16
+ ]
17
+ },
18
+ "enabledMcpjsonServers": ["iwsdk-runtime", "iwsdk-reference", "metavr"]
19
+ }
@@ -0,0 +1,96 @@
1
+ ---
2
+ name: iwsdk-debug
3
+ description: Debug continuous behavior in WebXR scenes — physics, animations, collisions, game loops, or any real-time interaction that happens too fast for an agent to observe. Uses ECS pause/step/snapshot/diff to freeze time and inspect state frame by frame.
4
+ argument-hint: '[description of behavior to debug]'
5
+ ---
6
+
7
+ # Debug Continuous Behavior
8
+
9
+ Real-time behavior (physics, animations, collisions, game loops) happens too fast for an agent to observe directly. By the time you take a screenshot, the action is over. This skill uses ECS time-control tools to freeze, step, and diff state frame by frame.
10
+
11
+ User request is in `$ARGUMENTS`.
12
+
13
+ ## Core Workflow
14
+
15
+ Every debugging session follows this pattern:
16
+
17
+ 1. **Set up** the scenario (position objects, aim controllers, etc.)
18
+ 2. **`ecs_pause`** — freeze ECS updates right before the interesting moment
19
+ 3. **`ecs_snapshot({ "label": "before" })`** — capture state before the action
20
+ 4. **Trigger** the action (release grip, apply force, start animation, etc.)
21
+ 5. **`ecs_step(count, delta)`** — advance a few frames at fixed timestep
22
+ 6. **`browser_screenshot`** — visually verify what happened
23
+ 7. **`ecs_snapshot({ "label": "after" })`** — capture state after stepping
24
+ 8. **`ecs_diff({ "from": "before", "to": "after" })`** — see exactly what changed
25
+ 9. **Repeat** steps 5-8, stepping further until the behavior completes
26
+ 10. **`ecs_resume`** — return to normal execution when done
27
+
28
+ The key insight: **pause BEFORE triggering the action**, not after. If you pause after, you've already missed the first frames.
29
+
30
+ ## Tool Reference
31
+
32
+ | Tool | Purpose |
33
+ | ------------------------------------------------- | ------------------------------------------------------------------------------------ |
34
+ | `ecs_pause` | Freeze all ECS system updates. Render loop continues — screenshots still work. |
35
+ | `ecs_step({"count":N,"delta":SECONDS})` | Advance N frames with fixed timestep (seconds). Must pause first. |
36
+ | `ecs_resume` | Resume normal execution. First frame uses capped delta to avoid physics explosions. |
37
+ | `ecs_snapshot({"label":"..."})` | Capture full ECS state. Stores up to 2 snapshots. |
38
+ | `ecs_diff({"from":"...","to":"..."})` | Compare two snapshots. Shows added/removed entities and field-level value changes. |
39
+ | `ecs_toggle_system({"name":"...","paused":true})` | Pause/resume a single system. Use `ecs_list_systems` to discover names. |
40
+ | `browser_screenshot` | Runtime visual verification — works while paused and auto-switches from editor view. |
41
+
42
+ ## Stepping Guidelines
43
+
44
+ - **`delta`** is in seconds. Common values: `0.016` (60fps), `0.0139` (72fps/Quest refresh rate).
45
+ - **Start small** — step 1-3 frames first to catch the initial moment, then step more.
46
+ - **Don't overshoot** — stepping 100 frames at once defeats the purpose. Step in batches of 5-20.
47
+
48
+ ## Patterns
49
+
50
+ Short domain-specific tips. Apply the core workflow above, plus these hints.
51
+
52
+ ### Physics (falling, bouncing, collisions)
53
+
54
+ - Pause BEFORE releasing the object or applying force.
55
+ - Step 1-3 frames at `delta: 0.016` to catch initial acceleration.
56
+ - In diffs, check `PhysicsBody._linearVelocity` and `PhysicsBody._angularVelocity` to see motion direction and speed.
57
+ - Check `Transform.position` to track movement.
58
+ - If an object falls through a surface: verify the surface entity has both `PhysicsBody` (Static) and `PhysicsShape` (TriMesh for complex geometry).
59
+ - Use `ecs_query_entity` to inspect `PhysicsShape` and `PhysicsBody` on both the falling object and the surface.
60
+
61
+ ### Grab and Throw
62
+
63
+ - Pause while the object is still held (trigger/grip engaged).
64
+ - Release the input (set button value to 0) while paused.
65
+ - Step frame by frame to observe the release velocity.
66
+ - In diffs, `PhysicsBody._linearVelocity` shows the throw direction and speed.
67
+ - If the object doesn't move after release: check that it has `PhysicsBody` with `state: Dynamic`.
68
+
69
+ ### Animations and Transitions
70
+
71
+ - Step 1 frame at a time with `delta` matching your target framerate.
72
+ - Compare `Transform.position`, `Transform.orientation`, and `Transform.scale` across snapshots to track interpolation.
73
+ - Use screenshots between steps to observe visual progression.
74
+
75
+ ### Collision Detection
76
+
77
+ - Pause just before two objects meet.
78
+ - Step 1 frame at a time.
79
+ - Watch for `PhysicsBody._linearVelocity` sign changes (indicates bounce/impact).
80
+ - Watch for `PhysicsBody._angularVelocity` spikes (indicates tumbling from impact).
81
+ - If objects pass through each other: check `PhysicsShape` exists on both entities, and verify shape types are appropriate (use `TriMesh` for complex static geometry).
82
+
83
+ ### System Isolation
84
+
85
+ - Use `ecs_list_systems` to see all systems and their priorities.
86
+ - Use `ecs_toggle_system({ "name": "SystemName", "paused": true })` to pause a suspect system while others run.
87
+ - Step forward and observe — if the bug disappears, that system is the cause.
88
+ - Remember to unpause the system when done.
89
+
90
+ ## Notes
91
+
92
+ - **Snapshots overwrite** — only 2 are stored. Label them clearly ("before"/"after") and diff before taking new ones.
93
+ - **Resume is safe** — the first frame after resume uses a capped delta to prevent physics explosions from accumulated time.
94
+ - **Render loop continues while paused** — screenshots always work, and the XR session stays alive.
95
+ - **Stepping requires pause** — `ecs_step` will fail if you haven't called `ecs_pause` first.
96
+
@@ -0,0 +1,135 @@
1
+ ---
2
+ name: iwsdk-depth-occlusion
3
+ description: Guide for implementing depth sensing and occlusion in IWSDK projects. Use when adding depth-based occlusion to hide virtual objects behind real-world surfaces, configuring DepthSensingSystem, choosing occlusion modes, or troubleshooting objects that disappear or fail to occlude.
4
+ argument-hint: '[description of depth occlusion task]'
5
+ ---
6
+
7
+ # Depth Occlusion
8
+
9
+ Hide virtual objects behind real-world surfaces using WebXR depth sensing. The system samples a per-pixel depth texture from the XR device and compares it against each virtual fragment's depth — if the real surface is closer, the fragment is faded out.
10
+
11
+ ## Setup
12
+
13
+ Three things are required: XR session depth config, the system, and the component.
14
+
15
+ ### 1. Enable depth sensing on the XR session
16
+
17
+ ```jsonc
18
+ // iwsdk.config.json
19
+ {
20
+ "world": {
21
+ "xr": {
22
+ "mode": "ar",
23
+ "referenceSpace": "unbounded",
24
+ "features": {
25
+ "depthSensing": {
26
+ "required": true,
27
+ "usage": "gpu-optimized",
28
+ "format": "float32"
29
+ },
30
+ "hitTest": { "required": true },
31
+ "anchors": { "required": true },
32
+ "unbounded": { "required": true }
33
+ }
34
+ }
35
+ }
36
+ }
37
+ ```
38
+
39
+ ### 2. Register `DepthSensingSystem` and `DepthOccludable`
40
+
41
+ ```typescript
42
+ import { DepthSensingSystem, DepthOccludable } from '@iwsdk/core';
43
+
44
+ world
45
+ .registerSystem(DepthSensingSystem, {
46
+ configData: {
47
+ enableDepthTexture: true,
48
+ enableOcclusion: true,
49
+ useFloat32: true,
50
+ blurRadius: 20.0,
51
+ },
52
+ });
53
+ ```
54
+
55
+ ### 3. Add `DepthOccludable` to entities
56
+
57
+ ```typescript
58
+ import { DepthOccludable, OcclusionShadersMode } from '@iwsdk/core';
59
+
60
+ // Soft occlusion (default) — smooth edges via 13-tap blur
61
+ entity.addComponent(DepthOccludable);
62
+
63
+ // Hard occlusion — sharp edges, single depth sample
64
+ entity.addComponent(DepthOccludable, {
65
+ mode: OcclusionShadersMode.HardOcclusion,
66
+ });
67
+
68
+ // MinMax occlusion — best quality, extra preprocessing pass
69
+ entity.addComponent(DepthOccludable, {
70
+ mode: OcclusionShadersMode.MinMaxSoftOcclusion,
71
+ });
72
+ ```
73
+
74
+ The material must have `transparent: true`. The system sets this automatically, but verify it on custom materials.
75
+
76
+ ## Occlusion Modes
77
+
78
+ | Mode | Quality | Cost | Best For |
79
+ | --------------------- | ------- | ------ | ----------------------------------------------------------------- |
80
+ | `SoftOcclusion` | Good | Low | Most objects — smooth edges, hides depth aliasing |
81
+ | `HardOcclusion` | Basic | Lowest | Small objects or when sharp edges are acceptable |
82
+ | `MinMaxSoftOcclusion` | Best | Medium | Large objects with complex silhouettes against varied backgrounds |
83
+
84
+ ## DepthSensingSystem Config
85
+
86
+ | Property | Type | Default | Description |
87
+ | -------------------- | ------- | ------- | ----------------------------------------- |
88
+ | `enableOcclusion` | Boolean | `true` | Master switch for all occlusion |
89
+ | `enableDepthTexture` | Boolean | `true` | Create GPU textures from depth data |
90
+ | `useFloat32` | Boolean | `true` | Float32 depth textures (higher precision) |
91
+ | `blurRadius` | Float32 | `20.0` | Blur radius for soft occlusion (pixels) |
92
+
93
+ ## Depth Sensing Modes
94
+
95
+ | Mode | When to use |
96
+ | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
97
+ | `cpu-optimized` | Simpler, works everywhere. Depth as linear meters in a DataArrayTexture. |
98
+ | `gpu-optimized` | **Recommended.** Matches Quest hardware format. Depth as reverse-Z inverse depth in an ExternalTexture. Required for production parity with on-device behavior. Note that Quest devices only support this mode. |
99
+
100
+ ## AR Session Requirements
101
+
102
+ Depth occlusion only works in AR mode. The scene background must be `null` for passthrough:
103
+
104
+ ```typescript
105
+ scene.background = null;
106
+ ```
107
+
108
+ ## Troubleshooting
109
+
110
+ **Objects never occlude (always visible on top)**
111
+
112
+ - Verify the entity has `DepthOccludable` component
113
+ - Verify `DepthSensingSystem` is registered with `enableOcclusion: true`
114
+ - Check that `depthSensing` is in the XR features config
115
+
116
+ **Objects always invisible in IWER**
117
+
118
+ - In the IWER emulator, the SEM must have loaded environment geometry. If no room is loaded, no depth data is produced.
119
+ - Check the console for "Warning: depth-sensing feature not enabled"
120
+
121
+ **Flickering or noisy occlusion edges**
122
+
123
+ - Increase `blurRadius` (try 30-40)
124
+ - Switch from `HardOcclusion` to `SoftOcclusion`
125
+ - Use `MinMaxSoftOcclusion` for best edge quality
126
+
127
+ ## Notes
128
+
129
+ - **Only works in AR mode** — set `world.xr.mode` to `"ar"` and configure
130
+ `world.xr.features.depthSensing` in `iwsdk.config.json`.
131
+ - **`DepthOccludable` may be incompatible with custom shaders** that override `diffuse` or `fog_vertex` includes, since the occlusion code is injected at those shader hook points.
132
+ - **Non-occludable objects** are simply entities without `DepthOccludable` — they render normally on top of everything.
133
+ - **`DepthOccludable` is built into IWSDK** and is already registered — it appears
134
+ in `scene_get_capabilities` under `registeredComponents`. Do not call
135
+ `world.registerComponent()` for it; only the system needs registering.
@@ -0,0 +1,129 @@
1
+ ---
2
+ name: iwsdk-grab
3
+ description: Grab an object in the WebXR scene using emulated controllers. Use when the user wants to pick up, move, or test grabbing an object. Supports OneHandGrabbable and TwoHandsGrabbable components which use proximity-based grip (squeeze button), not trigger.
4
+ argument-hint: <object-name> [destination]
5
+ ---
6
+
7
+ # Grab Object
8
+
9
+ Grab an object in the XR scene using the IWER emulated controllers. The workflow has a **required core** (steps 1-5) that must always execute, and **optional extensions** (steps 6-9) that depend on the user's intent.
10
+
11
+ User request is in `$ARGUMENTS`.
12
+
13
+ ## Required Core
14
+
15
+ These steps always execute in order. A grab cannot succeed without them.
16
+
17
+ ### Step 1: Enter XR
18
+
19
+ Check session status. If not in an active XR session, accept and enter.
20
+
21
+ ```
22
+ xr_get_session_status → if not sessionActive → xr_accept_session
23
+ ```
24
+
25
+ ### Step 2: Locate the target
26
+
27
+ Find the live entity by name or grabbable component, then query its component data.
28
+
29
+ ```
30
+ ecs_find_entities({namePattern: target}) → ecs_query_entity(entityIndex)
31
+ ```
32
+
33
+ If the object is not found, search `OneHandGrabbable` and `TwoHandsGrabbable`
34
+ entities, report the available matches, and stop.
35
+
36
+ ### Step 3: Get its transform
37
+
38
+ Read the object's Transform position from the queried entity.
39
+
40
+ ```
41
+ ecs_query_entity(entityIndex) → Transform.position
42
+ ```
43
+
44
+ ### Step 4: Animate controller to target
45
+
46
+ Animate the controller to the object's position. Default to `"controller-right"` unless the user specified left.
47
+
48
+ ```
49
+ xr_animate_to({
50
+ device: "controller-right",
51
+ position: { x, y, z },
52
+ duration: 0.5,
53
+ })
54
+ ```
55
+
56
+ ### Step 5: Engage grip
57
+
58
+ OneHandGrabbable and TwoHandsGrabbable are proximity-based and use the **squeeze/grip button (index 1)**, not the trigger.
59
+
60
+ ```
61
+ xr_set_gamepad_state({
62
+ device: "controller-right",
63
+ buttons: [{ index: 1, value: 1 }],
64
+ })
65
+ ```
66
+
67
+ The object is now grabbed. If the user only asked to grab (not move), stop here.
68
+
69
+ ## Optional Extensions
70
+
71
+ Apply these based on the user's request.
72
+
73
+ ### Step 6: Move to destination
74
+
75
+ If the user specified a destination position, animate the controller there. If no position was given but the user asked to "move" the object, animate it to in front of the headset.
76
+
77
+ To find "in front of headset": `xr_get_transform({ "device": "headset" })` → place at `(head.x, head.y - 0.2, head.z - 0.5)` adjusted for head orientation.
78
+
79
+ ```
80
+ xr_animate_to({
81
+ device: "controller-right",
82
+ position: { x, y, z },
83
+ duration: 0.5,
84
+ })
85
+ ```
86
+
87
+ ### Step 7: Release grip
88
+
89
+ Release the squeeze button to drop the object.
90
+
91
+ ```
92
+ xr_set_gamepad_state({
93
+ device: "controller-right",
94
+ buttons: [{ index: 1, value: 0 }],
95
+ })
96
+ ```
97
+
98
+ ### Step 8: Return controller
99
+
100
+ Animate the controller back to its resting position so it's not overlapping the dropped object.
101
+
102
+ ```
103
+ xr_animate_to({
104
+ device: "controller-right",
105
+ position: { x: 0.2, y: 1.4, z: -0.3 },
106
+ duration: 0.5,
107
+ })
108
+ ```
109
+
110
+ Default resting positions: right `(0.2, 1.4, -0.3)`, left `(-0.2, 1.4, -0.3)`.
111
+
112
+ ### Step 9: Verify
113
+
114
+ Take a screenshot to confirm the result.
115
+
116
+ ```
117
+ browser_screenshot
118
+ ```
119
+
120
+ The screenshot command is runtime-only. If the editor is visible, the managed
121
+ workspace switches to runtime before capture.
122
+
123
+ ## Notes
124
+
125
+ - **Never use `xr_set_device_state` to move controllers** — it teleports instead of animating, which can break grab state.
126
+ - **Never use `xr_select` or trigger (button index 0) for grabs** — OneHandGrabbable/TwoHandsGrabbable respond to squeeze (button index 1).
127
+ - **DistanceGrabbable is different** — it uses ray-based selection, not proximity. This skill does not cover DistanceGrabbable.
128
+ - If the object lacks a name in the hierarchy, suggest adding `mesh.name = "MyObject"` in code before `createTransformEntity`.
129
+