@iwsdk/create 0.4.2 → 0.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (106) hide show
  1. package/README.md +103 -41
  2. package/dist/cli.js +1003 -3418
  3. package/dist/guidance/agents/.agents/skills/iwsdk-debug/SKILL.md +96 -0
  4. package/dist/guidance/agents/.agents/skills/iwsdk-depth-occlusion/SKILL.md +135 -0
  5. package/dist/guidance/agents/.agents/skills/iwsdk-grab/SKILL.md +129 -0
  6. package/dist/guidance/agents/.agents/skills/iwsdk-physics/SKILL.md +92 -0
  7. package/dist/guidance/agents/.agents/skills/iwsdk-physics/references/component-reference.md +154 -0
  8. package/dist/guidance/agents/.agents/skills/iwsdk-physics/references/tuning-and-config.md +88 -0
  9. package/dist/guidance/agents/.agents/skills/iwsdk-physics/references/workflows.md +303 -0
  10. package/dist/guidance/agents/.agents/skills/iwsdk-planner/SKILL.md +364 -0
  11. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/api-reference.md +1489 -0
  12. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/build-milestones.md +241 -0
  13. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/design-deck.md +101 -0
  14. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/grounding.md +176 -0
  15. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/ideation.md +156 -0
  16. package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/verification.md +199 -0
  17. package/dist/guidance/agents/.agents/skills/iwsdk-ray/SKILL.md +159 -0
  18. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/SKILL.md +255 -0
  19. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/agents/openai.yaml +5 -0
  20. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/asset-authoring.md +295 -0
  21. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/composition-patterns.md +243 -0
  22. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/image-intake.md +191 -0
  23. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/review-and-stop.md +99 -0
  24. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/scene-format.md +458 -0
  25. package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/text-intake.md +82 -0
  26. package/dist/guidance/agents/.agents/skills/iwsdk-ui/SKILL.md +208 -0
  27. package/dist/guidance/claude/.claude/agents/iwsdk-project-code-reviewer.md +450 -0
  28. package/dist/guidance/claude/.claude/rules/assets-and-manifest.md +55 -0
  29. package/dist/guidance/claude/.claude/rules/ecs-api.md +102 -0
  30. package/dist/guidance/claude/.claude/rules/scene-json.md +64 -0
  31. package/dist/guidance/claude/.claude/rules/uikitml.md +60 -0
  32. package/dist/guidance/claude/.claude/settings.json +19 -0
  33. package/dist/guidance/claude/.claude/skills/iwsdk-debug/SKILL.md +96 -0
  34. package/dist/guidance/claude/.claude/skills/iwsdk-depth-occlusion/SKILL.md +135 -0
  35. package/dist/guidance/claude/.claude/skills/iwsdk-grab/SKILL.md +129 -0
  36. package/dist/guidance/claude/.claude/skills/iwsdk-physics/SKILL.md +92 -0
  37. package/dist/guidance/claude/.claude/skills/iwsdk-physics/references/component-reference.md +154 -0
  38. package/dist/guidance/claude/.claude/skills/iwsdk-physics/references/tuning-and-config.md +88 -0
  39. package/dist/guidance/claude/.claude/skills/iwsdk-physics/references/workflows.md +303 -0
  40. package/dist/guidance/claude/.claude/skills/iwsdk-planner/SKILL.md +364 -0
  41. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/api-reference.md +1489 -0
  42. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/build-milestones.md +241 -0
  43. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/design-deck.md +101 -0
  44. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/grounding.md +176 -0
  45. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/ideation.md +156 -0
  46. package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/verification.md +199 -0
  47. package/dist/guidance/claude/.claude/skills/iwsdk-ray/SKILL.md +159 -0
  48. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/SKILL.md +255 -0
  49. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/agents/openai.yaml +5 -0
  50. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/asset-authoring.md +295 -0
  51. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/composition-patterns.md +243 -0
  52. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/image-intake.md +191 -0
  53. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/review-and-stop.md +99 -0
  54. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/scene-format.md +458 -0
  55. package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/text-intake.md +82 -0
  56. package/dist/guidance/claude/.claude/skills/iwsdk-ui/SKILL.md +208 -0
  57. package/dist/guidance/claude/CLAUDE.md +142 -0
  58. package/dist/guidance/codex/.codex/config.toml +2 -0
  59. package/dist/guidance/common/AGENTS.md +151 -0
  60. package/dist/guidance/copilot/.github/instructions/assets-and-manifest.instructions.md +52 -0
  61. package/dist/guidance/copilot/.github/instructions/ecs-api.instructions.md +100 -0
  62. package/dist/guidance/copilot/.github/instructions/scene-json.instructions.md +63 -0
  63. package/dist/guidance/copilot/.github/instructions/uikitml.instructions.md +59 -0
  64. package/dist/guidance/cursor/.cursor/rules/assets-and-manifest.mdc +57 -0
  65. package/dist/guidance/cursor/.cursor/rules/ecs-api.mdc +104 -0
  66. package/dist/guidance/cursor/.cursor/rules/scene-json.mdc +66 -0
  67. package/dist/guidance/cursor/.cursor/rules/uikitml.mdc +62 -0
  68. package/dist/guidance/scoped-agents/public/scenes/AGENTS.md +63 -0
  69. package/dist/guidance/scoped-agents/public/ui/AGENTS.md +59 -0
  70. package/dist/guidance/scoped-agents/src/AGENTS.md +149 -0
  71. package/dist/template/common/README.md +13 -0
  72. package/dist/template/common/gitignore.template +45 -0
  73. package/dist/template/common/index.html +16 -0
  74. package/dist/template/common/nvmrc.template +1 -0
  75. package/dist/template/common/public/audio/chime.mp3 +0 -0
  76. package/dist/template/common/public/gltf/webxr-banner/banner.gltf +68 -0
  77. package/dist/template/common/public/textures/webxr.png +0 -0
  78. package/dist/template/common/public/ui/welcome.uikitml +98 -0
  79. package/dist/template/common/src/assets.ts +54 -0
  80. package/dist/template/common/src/components.ts +11 -0
  81. package/dist/template/common/src/index.ts +19 -0
  82. package/dist/template/common/src/panel.ts +38 -0
  83. package/dist/template/common/src/robot-component.ts +10 -0
  84. package/dist/template/common/src/robot.ts +38 -0
  85. package/dist/template/common/src/vite-env.d.ts +2 -0
  86. package/dist/template/common/tsconfig.json +15 -0
  87. package/dist/template/common/vite.config.ts +27 -0
  88. package/dist/template/common-js/README.md +13 -0
  89. package/dist/template/common-js/gitignore.template +45 -0
  90. package/dist/template/common-js/index.html +16 -0
  91. package/dist/template/common-js/nvmrc.template +1 -0
  92. package/dist/template/common-js/public/audio/chime.mp3 +0 -0
  93. package/dist/template/common-js/public/gltf/webxr-banner/banner.gltf +68 -0
  94. package/dist/template/common-js/public/textures/webxr.png +0 -0
  95. package/dist/template/common-js/public/ui/welcome.uikitml +98 -0
  96. package/dist/template/common-js/src/assets.js +45 -0
  97. package/dist/template/common-js/src/components.js +9 -0
  98. package/dist/template/common-js/src/index.js +14 -0
  99. package/dist/template/common-js/src/panel.js +31 -0
  100. package/dist/template/common-js/src/robot-component.js +8 -0
  101. package/dist/template/common-js/src/robot.js +37 -0
  102. package/dist/template/common-js/vite.config.js +25 -0
  103. package/dist/template/scenes/ar.iwsdk.scene.json +96 -0
  104. package/dist/template/scenes/immersive.iwsdk.scene.json +109 -0
  105. package/package.json +17 -12
  106. package/scripts/verify-release-ready.mjs +76 -0
@@ -0,0 +1,199 @@
1
+ # Phases 5–7 Playbook — Runtime Verification
2
+
3
+ Everything here uses the `iwsdk` CLI (works in every harness). MCP
4
+ equivalents exist for every command but several names differ: `xr enter` =
5
+ `xr_accept_session`, `xr status` = `xr_get_session_status`, `ecs find` =
6
+ `ecs_find_entities`, `ecs query` = `ecs_query_entity`, `browser logs` =
7
+ `browser_get_console_logs`, `browser reload` = `browser_reload_page`,
8
+ `scene state` = `scene_get_state`, and `scene render-file` =
9
+ `scene_render_file`; the rest map 1:1
10
+ (`xr_select` → `xr select`, `ecs_pause` → `ecs pause`, …). Use MCP tools only
11
+ if your harness clearly has them connected. The sibling `iwsdk-grab` /
12
+ `iwsdk-ray` / `iwsdk-debug` / `iwsdk-ui` skills cover the same runtime
13
+ operations as focused per-interaction recipes — consistent with this file.
14
+
15
+ **Ground rules.** Run every `npx iwsdk …` from **inside the app directory**
16
+ (the app must depend on `@iwsdk/cli`; a bare `npx iwsdk` elsewhere resolves
17
+ to an unrelated npm package). One CLI call per shell command — no `&&`
18
+ chains between CLI calls; use separate `sleep N` calls for waits. Every
19
+ command prints a JSON envelope `{ok, data|error}` on stdout — parse it and
20
+ check the assertion **before** the next command. Add `--timeout 20000` to
21
+ slow ops (reload, xr enter, animate-to, screenshot).
22
+
23
+ ## The Standard Loop
24
+
25
+ ```bash
26
+ npx tsc --noEmit # 0. types clean first — always
27
+ npx iwsdk dev status # 1. already running? if state.running AND
28
+ # state.browserCommandReady are true,
29
+ # REUSE it — never start a second server
30
+ npx iwsdk dev up --timeout 60000 # 2. only if not running: detached daemon;
31
+ # returns once the browser is command-ready.
32
+ # (`npm run dev` = `iwsdk dev up --open
33
+ # --foreground` — never returns; use it only
34
+ # where your harness manages background
35
+ # processes. NEVER a bare '&'.)
36
+ npx iwsdk ecs systems # 3. connectivity: returns system list
37
+ npx iwsdk browser reload --timeout 20000 # 4. fresh page state
38
+ sleep 3
39
+ npx iwsdk browser screenshot --output-file design/verify/base.png --timeout 20000
40
+ # 5. runtime renders? (not black/empty).
41
+ # Browser screenshots are runtime-only and
42
+ # auto-switch away from editor view. Without
43
+ # --output-file the PNG lands in the system
44
+ # temp dir (path in the JSON envelope)
45
+ npx iwsdk xr enter --timeout 20000 # 6. enter emulated XR session
46
+ sleep 2
47
+ npx iwsdk browser logs --input-json '{"count":30}'
48
+ # 7. console clean — scan ALL levels (a
49
+ # level filter can miss important errors);
50
+ # fail the gate on any error-looking line
51
+ ```
52
+
53
+ If `dev up` times out, poll yourself: run `npx iwsdk dev status`, `sleep 5`,
54
+ repeat up to ~60 s (a shell `until` loop around the _single_ CLI call is fine
55
+ — the no-chaining rule is about chaining different CLI calls), then inspect
56
+ `npx iwsdk dev logs --tail 100`.
57
+
58
+ Then per-scenario: discover → simulate → assert. Finish with
59
+ `npx iwsdk dev down` when done for the session.
60
+
61
+ The dev server owns one managed headed Playwright Chromium window, with IWER enabled
62
+ for XR starters. It starts with the dev server; do not launch a second browser or turn
63
+ Vite's independent `server.open` back on. Use `scene screenshot` / `scene render-file`
64
+ for editor evidence and `browser screenshot` for the application runtime.
65
+
66
+ ## Discover (after every reload — entity indices are NOT stable)
67
+
68
+ ```bash
69
+ npx iwsdk ecs find --input-json '{"withComponents":["OneHandGrabbable"]}'
70
+ npx iwsdk ecs query --input-json '{"entityIndex":3}' # full entity dump
71
+ npx iwsdk scene state --raw # active scene/runtime state
72
+ npx iwsdk ecs components # component registry
73
+ npx iwsdk ecs systems # system registry
74
+ ```
75
+
76
+ ## Simulate input (XR emulation cheat sheet)
77
+
78
+ Devices: `headset`, `controller-left`, `controller-right`, `hand-left`,
79
+ `hand-right`.
80
+
81
+ ```bash
82
+ # Aim/move devices
83
+ npx iwsdk xr set-transform --input-json '{"device":"headset","position":{"x":0,"y":1.6,"z":0}}'
84
+ npx iwsdk xr look-at --input-json '{"device":"controller-right","target":{"x":0,"y":1,"z":-2}}'
85
+ npx iwsdk xr animate-to --input-json '{"device":"controller-right","position":{"x":0.2,"y":1.1,"z":-0.4},"duration":0.5}' --timeout 20000
86
+
87
+ # Buttons — WRONG BUTTON FAILS SILENTLY, know which mechanic uses which:
88
+ npx iwsdk xr set-select-value --input-json '{"device":"controller-right","value":1}' # press TRIGGER — UI pointerdown or distance-grab engage
89
+ npx iwsdk xr set-select-value --input-json '{"device":"controller-right","value":0}' # release TRIGGER — UI pointerup or distance-grab release
90
+ npx iwsdk xr set-gamepad-state --input-json '{"device":"controller-right","buttons":[{"index":1,"value":1}]}' # SQUEEZE — near grab (one/two-hand)
91
+ npx iwsdk xr set-gamepad-state --input-json '{"device":"controller-right","buttons":[{"index":3,"value":1}]}' # A/X button (jump default)
92
+
93
+ # Thumbsticks (axes index 0=X, 1=Y):
94
+ npx iwsdk xr set-gamepad-state --input-json '{"device":"controller-left","axes":[{"index":1,"value":-1}]}' # slide forward (0 to stop)
95
+ npx iwsdk xr set-gamepad-state --input-json '{"device":"controller-right","axes":[{"index":0,"value":1}]}' # snap turn — EDGE-TRIGGERED: return to 0 before turning again
96
+ npx iwsdk xr set-gamepad-state --input-json '{"device":"controller-right","axes":[{"index":1,"value":1}]}' # hold then release to 0 = teleport
97
+ ```
98
+
99
+ Synthetic gamepad indices for `set-gamepad-state`: 0=trigger, 1=squeeze,
100
+ 2=thumbstick-press, 3=A/X, 4=B/Y, 5=thumbrest. (App code uses
101
+ `InputComponent` enums; these raw indices are only for the emulator CLI.)
102
+
103
+ ## Assert
104
+
105
+ Pick the right instrument per behavior:
106
+
107
+ | Behavior | Assertion |
108
+ | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
109
+ | hover/click state | `ecs query` for `Hovered`/`Pressed` on the entity |
110
+ | object moved/scaled/spawned | `ecs snapshot` (label before) → act → `ecs snapshot` (after) → `ecs diff` |
111
+ | physics settled/fell/collided | snapshot/diff on Transform position, or pause+step deterministically |
112
+ | **locomotion** (player moved) | **screenshots before/after** — the XR origin moves, headset transform stays constant; never assert on headset transform |
113
+ | **near-field grab** | snapshot/diff — `Hovered`/`Pressed` are NOT applied on near grabs |
114
+ | UI reacted | `browser logs` with a pattern: instrument handlers with distinctive `console.log('[GAME] score=…')` and assert `{"pattern":"\\[GAME\\]"}` |
115
+ | game logic over time | `ecs pause` → `ecs step --input-json '{"count":N}'` → query → `ecs resume` |
116
+ | looks right | `browser screenshot` and actually look at the image |
117
+
118
+ ```bash
119
+ npx iwsdk ecs snapshot --input-json '{"label":"before"}'
120
+ # …act…
121
+ npx iwsdk ecs snapshot --input-json '{"label":"after"}'
122
+ npx iwsdk ecs diff --input-json '{"from":"before","to":"after"}'
123
+ ```
124
+
125
+ Only the two most recent snapshot labels are retained.
126
+
127
+ ## Known traps
128
+
129
+ - Entity indices change on every reload — re-run `ecs find`; never reuse.
130
+ - JSON values must be real types: `true` not `"true"`, numbers not strings
131
+ (`ecs set-component` silently no-ops on wrong types).
132
+ - Trigger vs squeeze mix-ups produce _no error and no effect_ — check the
133
+ mechanic's grab type first (`DistanceGrabbable` → trigger/select-value;
134
+ `OneHandGrabbable`/`TwoHandsGrabbable` → squeeze button 1).
135
+ - Teleport is blocked while the ray hovers an `Interactable`.
136
+ - Snap turn re-fires only after the stick returns to 0.
137
+ - `--input-json` takes one single-quoted JSON argument; malformed JSON errors
138
+ out — build it with a heredoc if it contains quotes.
139
+ - UIKitML files live under `public/ui`, are registered in the asset manifest with
140
+ `AssetType.UIKitML`, and are parsed directly at runtime. Scene JSON references the
141
+ manifest asset ID; application code uses the placed scene node ID and element IDs.
142
+ - After deploying: verify the live HTML references the newly built hashed
143
+ assets (stale-dist deploys are the classic ship failure).
144
+
145
+ ## Managed-runtime hardening
146
+
147
+ - Prefer `xr set-select-value` 1→0 when a test needs observable press and release
148
+ phases. Use `xr set-gamepad-state` for explicit raw gamepad buttons. Application UI
149
+ should use its normal `click` handlers unless the product itself needs lower-level
150
+ pointer phases.
151
+ - **Physics steps on the raw frame delta.** Reloads, screenshots, and
152
+ `ecs snapshot` (full-state serialize) stall the tab for hundreds of ms to
153
+ seconds; a single huge step tunnels thin/falling bodies and pops stacked
154
+ contacts. Don't run heavy CLI commands mid-interaction; snapshot between
155
+ scenarios, not during them. Build apps defensively (see
156
+ build-milestones.md runtime-hitch hardening defaults).
157
+ - **Parse the right payload path**: `browser logs` returns the entries under
158
+ `data.result` (a LIST of {level, message}); `browser screenshot` returns
159
+ `data.screenshotPath` directly under `data`. Shapes vary per command —
160
+ check one real response before scripting assertions.
161
+ - **Log-window noise**: `[IWSDK-MCP-TRACE]` lines flood small `count`
162
+ windows; use `count` 60–100 or a `pattern` filter for game logs.
163
+ - **Queries and screenshots disagreeing** (state changes you can't explain)
164
+ usually means **two browser tabs** are connected — the relay is
165
+ first-response-wins. Check `dev status` → `connectedClientCount`; if >1,
166
+ `npx iwsdk dev restart` to converge on one tab.
167
+
168
+ ## Recovery ladder (runtime unresponsive / weird)
169
+
170
+ 1. `npx iwsdk xr status` and `npx iwsdk dev status` — what state is it in?
171
+ 2. `npx iwsdk browser logs` — crashed app code shows here.
172
+ 3. `npx iwsdk browser reload --timeout 20000` — clears bad page state.
173
+ 4. `npx iwsdk dev restart` — bad transport/server state.
174
+ 5. Re-enter XR (`xr enter`) and re-discover entities after any of the above.
175
+ 6. Dev server won't start: check `npx iwsdk dev logs --tail 100`; commonest
176
+ causes are a missing `dev:runtime` script, port conflict from an orphaned
177
+ server (`iwsdk dev down` in that app dir), or Playwright browser revision
178
+ mismatch (`npx playwright@<pinned> install chromium`).
179
+
180
+ ## VERIFICATION.md format (Phase 6 output)
181
+
182
+ ```markdown
183
+ # <Title> — Verification Report (<date>, commit <sha>)
184
+
185
+ | # | Success criterion | Result | Evidence |
186
+ | --- | ----------------- | ------ | ------------------------------------------ |
187
+ | S1 | throw knocks pins | PASS | diff: pin-3 rot Δ87°; design/verify/s1.png |
188
+
189
+ ## Scenario transcripts
190
+
191
+ ### S1
192
+
193
+ <the exact commands run + the assertion output snippets>
194
+
195
+ ## Deferred / known gaps
196
+ ```
197
+
198
+ Keep screenshots in `design/verify/`. A criterion without evidence is not a
199
+ PASS.
@@ -0,0 +1,159 @@
1
+ ---
2
+ name: iwsdk-ray
3
+ description: Ray-based interactions in the WebXR scene — click objects, press UI buttons, or distance-grab with DistanceGrabbable. Use when the user wants to point at and interact with something at a distance, click a UI button, or test ray-based selection.
4
+ argument-hint: <target> [action]
5
+ ---
6
+
7
+ # Ray Interaction
8
+
9
+ Point at and interact with objects or UI elements in the XR scene using the controller ray. The workflow has a **required core** (steps 1-4) and **optional extensions** that depend on whether the target is an object or UI element, and what kind of interaction is needed.
10
+
11
+ User request is in `$ARGUMENTS`.
12
+
13
+ ## Required Core
14
+
15
+ These steps always execute in order.
16
+
17
+ ### Step 1: Enter XR
18
+
19
+ Check session status. If not in an active XR session, accept and enter.
20
+
21
+ ```
22
+ xr_get_session_status → if not sessionActive → xr_accept_session
23
+ ```
24
+
25
+ ### Step 2: Locate the target
26
+
27
+ **For scene objects:** Find a live entity by name or interaction component, then
28
+ query its component data.
29
+
30
+ ```
31
+ ecs_find_entities({namePattern: target}) → ecs_query_entity(entityIndex)
32
+ ```
33
+
34
+ **For UI elements:** start from the stable UIKitML identities instead of trying to
35
+ discover an internal panel component:
36
+
37
+ 1. Read the UIKITML source file to find the element's `id` (e.g., `<button id="xr-button">`)
38
+ 2. Find the manifest asset ID and the placed scene node whose `content.asset` uses it
39
+ 3. In application code, resolve the placed `UIKitMLAsset` with
40
+ `world.requireSceneObject(sceneNodeId)` and resolve the element with
41
+ `requireElementById(elementId)`
42
+ 4. Add an application-level test hook exposing the element's world target when the
43
+ runtime tools need a precise point to aim at
44
+
45
+ If the element already has a name in the hierarchy, skip straight to getting its transform.
46
+
47
+ If the object is not found, report the available named objects and stop.
48
+
49
+ ### Step 3: Get its transform
50
+
51
+ Read the target's Transform position from the queried entity or application test
52
+ hook from step 2.
53
+
54
+ ```
55
+ ecs_query_entity(entityIndex) → Transform.position
56
+ ```
57
+
58
+ ### Step 4: Aim the controller
59
+
60
+ Point the controller at the target. Default to `"controller-right"` unless the user specified left. Do NOT move the controller — only rotate it.
61
+
62
+ ```
63
+ xr_look_at({ device: "controller-right", target: { x, y, z } })
64
+ ```
65
+
66
+ The controller ray is now pointing at the target. What happens next depends on the interaction type.
67
+
68
+ ## Interaction Branches
69
+
70
+ Based on the user's intent and the target's components, choose ONE of the following.
71
+
72
+ ### Branch A: Click / Select (objects with RayInteractable, or UI buttons)
73
+
74
+ For simple clicks, send an explicit trigger press and release. Use for UI buttons and objects that respond to the Pressed component.
75
+
76
+ ```
77
+ xr_set_select_value({ device: "controller-right", value: 1 })
78
+ xr_set_select_value({ device: "controller-right", value: 0 })
79
+ ```
80
+
81
+ This explicit press/release sequence is deterministic and makes the held state visible
82
+ between the two commands.
83
+
84
+ ### Branch B: Distance Grab (objects with DistanceGrabbable)
85
+
86
+ DistanceGrabbable requires **press and hold** on the trigger (button index 0), not a quick select.
87
+
88
+ #### B1: Engage trigger
89
+
90
+ ```
91
+ xr_set_gamepad_state({
92
+ device: "controller-right",
93
+ buttons: [{ index: 0, value: 1 }],
94
+ })
95
+ ```
96
+
97
+ The object is now distance-grabbed. Behavior depends on the `movementMode`:
98
+
99
+ - **MoveFromTarget / MoveAtSource / RotateAtSource** — object stays remote, moves relative to controller movement
100
+ - **MoveTowardsTarget** — object flies into the controller's hand, then behaves like a proximity grab
101
+
102
+ #### B2: Move to destination (optional)
103
+
104
+ If the user wants to move the object somewhere, animate the controller to the destination.
105
+
106
+ ```
107
+ xr_animate_to({
108
+ device: "controller-right",
109
+ position: { x, y, z },
110
+ duration: 0.5,
111
+ })
112
+ ```
113
+
114
+ If no destination specified but user asked to "move" or "bring" the object, animate to in front of the headset: `xr_get_transform({ "device": "headset" })` → place at `(head.x, head.y - 0.2, head.z - 0.5)`.
115
+
116
+ #### B3: Release trigger
117
+
118
+ ```
119
+ xr_set_gamepad_state({
120
+ device: "controller-right",
121
+ buttons: [{ index: 0, value: 0 }],
122
+ })
123
+ ```
124
+
125
+ #### B4: Return controller
126
+
127
+ Animate back to resting position.
128
+
129
+ ```
130
+ xr_animate_to({
131
+ device: "controller-right",
132
+ position: { x: 0.2, y: 1.4, z: -0.3 },
133
+ duration: 0.5,
134
+ })
135
+ ```
136
+
137
+ Default resting positions: right `(0.2, 1.4, -0.3)`, left `(-0.2, 1.4, -0.3)`.
138
+
139
+ ### Step 5: Verify (optional)
140
+
141
+ Take a screenshot to confirm the result.
142
+
143
+ ```
144
+ browser_screenshot
145
+ ```
146
+
147
+ `browser_screenshot` is runtime-only. If the editor is visible, the managed workspace
148
+ switches to runtime before capture.
149
+
150
+ ## Notes
151
+
152
+ - **Click vs hold:** use `xr_set_select_value` 1→0 for quick clicks. Hold it at 1 while moving a DistanceGrabbable, then release with 0.
153
+ - **Trigger vs squeeze:** Ray interactions use the **trigger (button index 0)**. Proximity grabs (OneHandGrabbable/TwoHandsGrabbable) use **squeeze (button index 1)**. Don't mix them up.
154
+ - **UI element discovery:** use manifest asset ID → stable scene node ID → UIKitML
155
+ element ID. Expose a test hook for the resolved element when a world-space aim target
156
+ is needed. Guessing from panel offsets or transient entity indices is fragile.
157
+ - **DistanceGrabbable movement modes:** Check the entity's DistanceGrabbable component to see which `movementMode` is set. Use `ecs_query_entity` if unsure.
158
+ - **Don't move the controller to the target** — ray interactions work at a distance. Only rotate via `xr_look_at`, don't translate.
159
+
@@ -0,0 +1,255 @@
1
+ ---
2
+ name: iwsdk-scene-composer
3
+ description: Compose editable static IWSDK scenes from text, images, or hybrid references using a shared application asset manifest, v1 scene JSON, modular scene files, and the managed editor's validation and rendering tools. Use for 3D environments, props, architecture, staged scenes, procedural Three.js assets, custom PBR or shader materials, glTF assets, prefabs, patterns, lighting, camera matching, or visual review.
4
+ ---
5
+
6
+ # IWSDK Scene Composer
7
+
8
+ Author native IWSDK assets and scene files. Application code owns geometry and
9
+ materials; scene JSON owns composition; the managed editor provides visual feedback
10
+ and human transform/component adjustment.
11
+
12
+ Read these references when relevant:
13
+
14
+ - [scene-format.md](references/scene-format.md) before editing scene JSON;
15
+ - [asset-authoring.md](references/asset-authoring.md) before creating or changing
16
+ glTF or procedural assets;
17
+ - [text-intake.md](references/text-intake.md) for text-only requests;
18
+ - [image-intake.md](references/image-intake.md) for image or hybrid requests;
19
+ - [composition-patterns.md](references/composition-patterns.md) for decomposition
20
+ and repetition strategies;
21
+ - [review-and-stop.md](references/review-and-stop.md) before final review.
22
+
23
+ ## Fixed Boundaries
24
+
25
+ - Use only `iwsdk.scene.v1`. There is no compatibility schema.
26
+ - Scene files are the composition source of truth. Create and edit them with normal
27
+ filesystem tools under `public/scenes/`.
28
+ - Scene JSON has one renderable content kind: `asset`. It does not define models,
29
+ primitive geometry, material resources, or material overrides.
30
+ - The default export of the configured application asset manifest is the asset source
31
+ of truth. It may contain URL-backed glTF and UIKitML entries plus parentless
32
+ `Object3D` prototypes with arbitrary Three.js geometry and materials.
33
+ - The application runtime and editor import the same manifest module independently.
34
+ Never depend on shared object identity, iframe messaging, DOM state, or a live
35
+ runtime world when defining assets.
36
+ - Humans use the editor for selection, hierarchy, transforms, components, root
37
+ lighting, and preview visibility. They do not edit geometry or materials there.
38
+ - Agents may edit asset TypeScript and scene JSON, then use the editor to validate and
39
+ render the result.
40
+
41
+ The public scene MCP surface is intentionally small:
42
+
43
+ ```text
44
+ scene_open
45
+ scene_render_file
46
+ scene_flatten_file
47
+ scene_get_state
48
+ scene_get_capabilities
49
+ scene_screenshot
50
+ scene_select
51
+ scene_set_camera
52
+ scene_set_preview_visibility
53
+ scene_measure_image_regions
54
+ ```
55
+
56
+ Document creation and mutation happen through direct file edits. Do not look for MCP
57
+ create/add/update/remove/patch/save/compose/review/publish tools.
58
+
59
+ When MCP is unavailable, use the CLI equivalents:
60
+
61
+ ```bash
62
+ npx iwsdk dev status
63
+ npx iwsdk dev up
64
+ npx iwsdk scene capabilities --raw
65
+ npx iwsdk scene render-file \
66
+ --input-json '{"path":"public/scenes/room.iwsdk.scene.json","viewId":"hero"}' \
67
+ --output-file artifacts/room.png
68
+ npx iwsdk scene flatten \
69
+ --input-json '{"path":"public/scenes/room.composition.iwsdk.scene.json","outputPath":"public/scenes/room.iwsdk.scene.json"}' --raw
70
+ npx iwsdk scene open \
71
+ --input-json '{"path":"public/scenes/room.iwsdk.scene.json"}' --raw
72
+ npx iwsdk scene state --raw
73
+ ```
74
+
75
+ `iwsdk dev up` starts the server in the background, launches the configured
76
+ managed editor browser, and waits for the command bridge. Do not edit
77
+ `vite.config.ts` to change browser mode as an ad hoc startup workaround.
78
+
79
+ `scene_render_file` renders a file without replacing the editor's active document,
80
+ but it still uses the managed editor browser for manifest evaluation and WebGL.
81
+ If startup reports `dev_browser_not_ready`, inspect `iwsdk dev status` and
82
+ `iwsdk dev logs --tail 100`. Retry only when the diagnostics indicate a transient
83
+ startup failure. Do not invent a custom CPU or Playwright renderer and present it as
84
+ authoritative editor evidence. Preserve the structured failure, continue
85
+ type/schema/build checks that remain meaningful, and report the visual-verification
86
+ gate as blocked.
87
+
88
+ Camera parameters are intentionally distinct: `view` accepts only the built-in
89
+ presets (`current`, `top`, `front`, `back`, `left`, `right`, `quarter`, `orbit`),
90
+ while `viewId` selects an exact camera declared in `authoring.views`. Once a level
91
+ loads, its saved hero view owns runtime framing and supersedes the initial
92
+ `World.create({ render: { camera } })` pose.
93
+
94
+ ## Workflow
95
+
96
+ ### 1. Specify
97
+
98
+ Turn the request into a compact implementation brief:
99
+
100
+ - required and optional features;
101
+ - source evidence regions for image input;
102
+ - silhouette, proportions, parts, negative space, contacts, and material response;
103
+ - hero and diagnostic views;
104
+ - measurable acceptance criteria;
105
+ - assumptions, uncertainty, and fidelity ceiling.
106
+
107
+ A single image proves visible composition, not hidden geometry. Do not silently invent
108
+ occluded detail or lower requested fidelity.
109
+
110
+ ### 2. Plan Assets And Modules
111
+
112
+ Call `scene_get_capabilities` once. Inspect `src/assets.ts` and existing asset modules.
113
+
114
+ Choose an external asset source deliberately:
115
+
116
+ - Use the configured MetaVR asset search, or
117
+ `npx @meta-quest/metavr --json asset search "<query>"`, for ready-made static
118
+ props and background dressing. Results provide previews plus GLB/FBX downloads,
119
+ but do not promise semantic subparts, rigging, articulation, or independently
120
+ editable pieces. Inspect the downloaded hierarchy, and copy selected files into
121
+ project-owned storage instead of persisting a returned CDN URL.
122
+ - Use `npx @drawcall/market skill` and `npx @drawcall/market types`, then search a
123
+ concrete need with `npx @drawcall/market search "<query>" --type <type> --limit 3`,
124
+ when an installable reusable asset, template, or provider-generated result is a
125
+ better starting point. Preview finalists and install the exact printed
126
+ `name@version`; trust the install output for consumer paths.
127
+ - Drawcall Market is a marketplace/install/generation CLI, not a universal
128
+ procedural-geometry engine. When the request depends on controllable parts,
129
+ parametric dimensions, articulation, or code-driven variation and no suitable
130
+ code-backed template exists, author a deterministic Three.js `Object3D` prototype.
131
+
132
+ For every visible form, choose one of:
133
+
134
+ 1. reuse an existing manifest asset;
135
+ 2. add a glTF entry to the manifest;
136
+ 3. register a UIKitML file with `AssetType.UIKitML`;
137
+ 4. create a deterministic parentless `Object3D` prototype in code and register it;
138
+ 5. assemble existing assets with a scene prefab or module.
139
+
140
+ Create custom geometry and materials in asset code, not JSON. Prefer separate
141
+ `*.scene-asset.ts` modules for substantial procedural assets and import their
142
+ prototypes into `src/assets.ts`.
143
+
144
+ For initial construction, plan independent semantic groups as standalone scratch
145
+ scene modules. Give each module a local origin, size envelope, attachment points,
146
+ required views, and asset IDs. Asset and component IDs are application-global;
147
+ imported node and prefab IDs are namespaced. Imports are an authoring-only assembly
148
+ mechanism, never a runtime or editable-project format.
149
+
150
+ ### 3. Build
151
+
152
+ Author assets first, then scene JSON. Build in dependency order:
153
+
154
+ 1. support/stage and representative lighting;
155
+ 2. large composition masses;
156
+ 3. identity-critical groups;
157
+ 4. repeated secondary detail;
158
+ 5. hero camera and final environment.
159
+
160
+ Use meters, stable descriptive IDs, deterministic ordering, and explicit transforms.
161
+ Groups supply hierarchy, never visible mass. Use `castShadow` and `receiveShadow` on
162
+ asset nodes only when needed. Use prefabs and patterns for repetition; keep repeated
163
+ asset prototypes resource-sharing friendly.
164
+
165
+ ### 4. Validate And Materialize
166
+
167
+ With the managed editor command-ready, call `scene_render_file` on every changed
168
+ scratch module, then the composition root. It resolves imports for authoring preview,
169
+ validates schema and manifest references, lowers the scene, and returns a PNG plus
170
+ diagnostics without changing the active document. Fix failures in the owning asset or
171
+ scratch file.
172
+
173
+ After the composition root passes, run `scene_flatten_file` / `iwsdk scene flatten`
174
+ once to materialize an import-free final scene. The command preserves import wrapper
175
+ groups, validates the output, and refuses to write if its runtime hash differs from
176
+ the composed source. This is a one-way publication boundary: the flat file becomes
177
+ the sole source of truth, and later scratch-module changes must not be re-flattened
178
+ over human edits.
179
+
180
+ Call `scene_open` only on the flattened file for live collaboration. Import-bearing
181
+ files remain renderable composition previews but are never opened as editable scenes
182
+ and never load in the application runtime.
183
+
184
+ Use `scene_get_state` for selection, hashes, diagnostics, dirty/conflict state, runtime
185
+ readiness, and render statistics. Use camera, screenshot, selection, and preview
186
+ visibility tools only when their live-editor context is useful.
187
+
188
+ ### 5. Review And Refine
189
+
190
+ Review in three passes:
191
+
192
+ 1. **Layout**: hierarchy, scale, support contacts, and arrangement.
193
+ 2. **Geometry**: silhouette, proportions, parts, negative space, and alternate views.
194
+ 3. **Final**: material response, color, lighting, environment, and hero framing.
195
+
196
+ Keep review orchestration and evidence outside the editor. The editor supplies
197
+ authoritative screenshots, hashes, camera state, diagnostics, and render measurements.
198
+ Derive comparisons, defect lists, lineage, and stop decisions in ordinary task files.
199
+
200
+ Fix the highest-impact defect in its owning asset or scene module, rerender that file,
201
+ then rerender the root. Default to two focused correction rounds. Stop earlier on a
202
+ repeated defect, oscillation, plateau, missing input/asset, or representation ceiling.
203
+
204
+ ### 6. Finish
205
+
206
+ Finish only when:
207
+
208
+ - every scratch module and the composition root validates and renders;
209
+ - the final editable scene is flattened and contains no `imports`;
210
+ - the active editor state is clean and conflict-free;
211
+ - required views are nonblank and correctly framed;
212
+ - required features pass measurable and visual checks;
213
+ - manifest asset IDs resolve in both editor and application runtime;
214
+ - the application build and selected scene load without blocking errors.
215
+
216
+ If a required gate is unavailable, finish with an explicit blocked or
217
+ accepted-with-gaps result. Passing a local schema check, production build, or custom
218
+ diagnostic image does not substitute for authoritative editor renders and state.
219
+
220
+ ## Modular Composition
221
+
222
+ ```json
223
+ {
224
+ "version": "iwsdk.scene.v1",
225
+ "units": "meters",
226
+ "imports": [
227
+ {
228
+ "id": "reading-nook",
229
+ "src": "./modules/reading-nook.iwsdk.scene.json",
230
+ "transform": { "position": [1.8, 0, -0.6] }
231
+ }
232
+ ],
233
+ "resources": {},
234
+ "nodes": []
235
+ }
236
+ ```
237
+
238
+ Each scratch module must be valid by itself. Imports resolve recursively in
239
+ declaration order. The import entry becomes a transform group. The composition root
240
+ owns global components, environment, metadata, and authoring settings. Cycles, unsafe
241
+ IDs, missing files, duplicate namespaced IDs, and invalid modules fail composition.
242
+
243
+ For parallel initial construction, assign one scratch module file per worker. Never
244
+ let two workers edit one file. Render modules independently, import only passing
245
+ modules, correct cross-module scale, contact, occlusion, lighting, and framing at the
246
+ composition root, then flatten exactly once. Parallel module iteration ends at that
247
+ boundary; continue all later edits in the flat file.
248
+
249
+ ## Regeneration And Provenance
250
+
251
+ Preserve stable IDs when revising the flat file. Never overwrite unrelated
252
+ human-authored files or re-flatten over editor changes. Record the skill/runtime
253
+ versions, input hashes, composition/final/module paths, capability hash,
254
+ source/composed/runtime hashes, assumptions, and fidelity ceiling in authoring
255
+ metadata or adjacent task evidence.
@@ -0,0 +1,5 @@
1
+ interface:
2
+ display_name: 'IWSDK Scene Composer'
3
+ short_description: 'Compose editable IWSDK scenes from text or images'
4
+ default_prompt: 'Use $iwsdk-scene-composer to compose this request as an editable, validated IWSDK scene.'
5
+