@iwsdk/create 0.5.3 → 1.0.0-rc.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli.js +7224 -216
- package/dist/guidance/agents/.agents/skills/iwsdk-build-model/SKILL.md +311 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-build-model/agents/openai.yaml +4 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-build-model/assets/hardsurface.ts.template +599 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-compose-scene/SKILL.md +262 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-compose-scene/agents/openai.yaml +4 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-compose-scene/references/extended-workflow.md +166 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-compose-scene/references/scene-composition.md +80 -0
- package/dist/guidance/{claude/.claude/skills/iwsdk-scene-composer → agents/.agents/skills/iwsdk-compose-scene}/references/scene-format.md +3 -3
- package/dist/guidance/agents/.agents/skills/iwsdk-compose-scene/references/scene-review.md +66 -0
- package/dist/guidance/agents/.agents/skills/{iwsdk-scene-composer → iwsdk-compose-scene}/references/text-intake.md +3 -5
- package/dist/guidance/agents/.agents/skills/iwsdk-debug/SKILL.md +69 -91
- package/dist/guidance/agents/.agents/skills/iwsdk-depth-occlusion/SKILL.md +81 -129
- package/dist/guidance/agents/.agents/skills/iwsdk-dev/SKILL.md +45 -0
- package/dist/guidance/{claude/.claude/skills/iwsdk-planner → agents/.agents/skills/iwsdk-dev}/references/api-reference.md +53 -45
- package/dist/guidance/agents/.agents/skills/iwsdk-dev/references/iterate.md +239 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-dev/references/planner.md +303 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-grab/SKILL.md +37 -114
- package/dist/guidance/agents/.agents/skills/iwsdk-hosting/SKILL.md +177 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-hosting/references/existing-hosts.md +181 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-hosting/references/vercel.md +114 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-native-xr-test/SKILL.md +246 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-physics/SKILL.md +92 -78
- package/dist/guidance/agents/.agents/skills/iwsdk-physics/references/tuning-and-config.md +15 -1
- package/dist/guidance/agents/.agents/skills/iwsdk-pwa-packaging/SKILL.md +218 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-pwa-packaging/references/app-modes.md +104 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-pwa-packaging/references/bubblewrap.md +219 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-pwa-packaging/references/manifest-and-icons.md +149 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-pwa-packaging/references/store-upload.md +104 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-pwa-packaging/references/troubleshooting.md +178 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-ray/SKILL.md +45 -136
- package/dist/guidance/agents/.agents/skills/iwsdk-ui/SKILL.md +251 -157
- package/dist/guidance/claude/.claude/agents/iwsdk-project-code-reviewer.md +9 -6
- package/dist/guidance/claude/.claude/rules/scene-json.md +6 -4
- package/dist/guidance/claude/.claude/settings.json +6 -2
- package/dist/guidance/claude/.claude/skills/iwsdk-build-model/SKILL.md +311 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-build-model/agents/openai.yaml +4 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-build-model/assets/hardsurface.ts.template +599 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-compose-scene/SKILL.md +262 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-compose-scene/agents/openai.yaml +4 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-compose-scene/references/extended-workflow.md +166 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-compose-scene/references/scene-composition.md +80 -0
- package/dist/guidance/{agents/.agents/skills/iwsdk-scene-composer → claude/.claude/skills/iwsdk-compose-scene}/references/scene-format.md +3 -3
- package/dist/guidance/claude/.claude/skills/iwsdk-compose-scene/references/scene-review.md +66 -0
- package/dist/guidance/claude/.claude/skills/{iwsdk-scene-composer → iwsdk-compose-scene}/references/text-intake.md +3 -5
- package/dist/guidance/claude/.claude/skills/iwsdk-debug/SKILL.md +69 -91
- package/dist/guidance/claude/.claude/skills/iwsdk-depth-occlusion/SKILL.md +81 -129
- package/dist/guidance/claude/.claude/skills/iwsdk-dev/SKILL.md +45 -0
- package/dist/guidance/{agents/.agents/skills/iwsdk-planner → claude/.claude/skills/iwsdk-dev}/references/api-reference.md +53 -45
- package/dist/guidance/claude/.claude/skills/iwsdk-dev/references/iterate.md +239 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-dev/references/planner.md +303 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-grab/SKILL.md +37 -114
- package/dist/guidance/claude/.claude/skills/iwsdk-hosting/SKILL.md +177 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-hosting/references/existing-hosts.md +181 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-hosting/references/vercel.md +114 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-native-xr-test/SKILL.md +246 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-physics/SKILL.md +92 -78
- package/dist/guidance/claude/.claude/skills/iwsdk-physics/references/tuning-and-config.md +15 -1
- package/dist/guidance/claude/.claude/skills/iwsdk-pwa-packaging/SKILL.md +218 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-pwa-packaging/references/app-modes.md +104 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-pwa-packaging/references/bubblewrap.md +219 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-pwa-packaging/references/manifest-and-icons.md +149 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-pwa-packaging/references/store-upload.md +104 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-pwa-packaging/references/troubleshooting.md +178 -0
- package/dist/guidance/claude/.claude/skills/iwsdk-ray/SKILL.md +45 -136
- package/dist/guidance/claude/.claude/skills/iwsdk-ui/SKILL.md +251 -157
- package/dist/guidance/claude/CLAUDE.md +2 -141
- package/dist/guidance/codex/.codex/config.toml +1 -1
- package/dist/guidance/common/AGENTS.md +96 -37
- package/dist/guidance/copilot/.github/instructions/scene-json.instructions.md +6 -4
- package/dist/guidance/cursor/.cursor/rules/scene-json.mdc +6 -4
- package/dist/guidance/scoped-agents/public/scenes/AGENTS.md +6 -4
- package/dist/template/common/README.md +13 -0
- package/dist/template/common/public/ui/welcome.uikitml +0 -19
- package/dist/template/common/src/robot.ts +5 -0
- package/dist/template/common/vite.config.ts +18 -0
- package/dist/template/common-js/README.md +13 -0
- package/dist/template/common-js/public/ui/welcome.uikitml +0 -19
- package/dist/template/common-js/src/robot.js +5 -0
- package/dist/template/common-js/vite.config.js +18 -0
- package/package.json +8 -7
- package/scripts/verify-release-ready.mjs +25 -0
- package/dist/guidance/agents/.agents/skills/iwsdk-planner/SKILL.md +0 -364
- package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/build-milestones.md +0 -243
- package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/design-deck.md +0 -101
- package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/grounding.md +0 -176
- package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/ideation.md +0 -156
- package/dist/guidance/agents/.agents/skills/iwsdk-planner/references/verification.md +0 -199
- package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/SKILL.md +0 -257
- package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/agents/openai.yaml +0 -4
- package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/asset-authoring.md +0 -295
- package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/composition-patterns.md +0 -243
- package/dist/guidance/agents/.agents/skills/iwsdk-scene-composer/references/review-and-stop.md +0 -99
- package/dist/guidance/claude/.claude/skills/iwsdk-planner/SKILL.md +0 -364
- package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/build-milestones.md +0 -243
- package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/design-deck.md +0 -101
- package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/grounding.md +0 -176
- package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/ideation.md +0 -156
- package/dist/guidance/claude/.claude/skills/iwsdk-planner/references/verification.md +0 -199
- package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/SKILL.md +0 -257
- package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/agents/openai.yaml +0 -4
- package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/asset-authoring.md +0 -295
- package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/composition-patterns.md +0 -243
- package/dist/guidance/claude/.claude/skills/iwsdk-scene-composer/references/review-and-stop.md +0 -99
- /package/dist/guidance/agents/.agents/skills/{iwsdk-scene-composer → iwsdk-compose-scene}/references/image-intake.md +0 -0
- /package/dist/guidance/claude/.claude/skills/{iwsdk-scene-composer → iwsdk-compose-scene}/references/image-intake.md +0 -0
|
@@ -1,158 +1,67 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: iwsdk-ray
|
|
3
|
-
description:
|
|
4
|
-
argument-hint:
|
|
3
|
+
description: Verify ray clicks, UIKit selections, and DistanceGrabbable interactions with emulated XR controllers. Use for pointing, selecting, pressing spatial UI, or moving an object through a distance grab.
|
|
4
|
+
argument-hint: '(use the current request)'
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
# Ray Interaction
|
|
7
|
+
# IWSDK Ray Interaction
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Choose one interaction branch from measured target components. Ray clicks and
|
|
10
|
+
distance grabs share aiming but not input duration or movement behavior.
|
|
10
11
|
|
|
11
|
-
|
|
12
|
+
## Locate and aim once
|
|
12
13
|
|
|
13
|
-
|
|
14
|
+
1. Check XR session state and enter only when inactive.
|
|
15
|
+
2. Find the named entity or runtime-named UIKit control, then query its
|
|
16
|
+
components and transform. For UIKit, prefer the live name assigned through
|
|
17
|
+
`requireElementById`; use a project test hook only when no addressable runtime
|
|
18
|
+
target exists.
|
|
19
|
+
3. Aim the requested controller with `xr look-at`. Do not translate the
|
|
20
|
+
controller toward a ray target.
|
|
14
21
|
|
|
15
|
-
|
|
22
|
+
## Select exactly one branch
|
|
16
23
|
|
|
17
|
-
###
|
|
24
|
+
### Click or UIKit button
|
|
18
25
|
|
|
19
|
-
|
|
26
|
+
For `RayInteractable`, `Pressed`, or a UIKit control, use one normal select:
|
|
20
27
|
|
|
21
|
-
```
|
|
22
|
-
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
### Step 2: Locate the target
|
|
26
|
-
|
|
27
|
-
**For scene objects:** Find a live entity by name or interaction component, then
|
|
28
|
-
query its component data.
|
|
29
|
-
|
|
30
|
-
```
|
|
31
|
-
ecs_find_entities({namePattern: target}) → ecs_query_entity(entityIndex)
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
**For UI elements:** start from the stable UIKitML identities instead of trying to
|
|
35
|
-
discover an internal panel component:
|
|
36
|
-
|
|
37
|
-
1. Read the UIKITML source file to find the element's `id` (e.g., `<button id="xr-button">`)
|
|
38
|
-
2. Find the manifest asset ID and the placed scene node whose `content.asset` uses it
|
|
39
|
-
3. In application code, resolve the placed `UIKitMLAsset` with
|
|
40
|
-
`world.requireSceneObject(sceneNodeId)` and resolve the element with
|
|
41
|
-
`requireElementById(elementId)`
|
|
42
|
-
4. Add an application-level test hook exposing the element's world target when the
|
|
43
|
-
runtime tools need a precise point to aim at
|
|
44
|
-
|
|
45
|
-
If the element already has a name in the hierarchy, skip straight to getting its transform.
|
|
46
|
-
|
|
47
|
-
If the object is not found, report the available named objects and stop.
|
|
48
|
-
|
|
49
|
-
### Step 3: Get its transform
|
|
50
|
-
|
|
51
|
-
Read the target's Transform position from the queried entity or application test
|
|
52
|
-
hook from step 2.
|
|
53
|
-
|
|
54
|
-
```
|
|
55
|
-
ecs_query_entity(entityIndex) → Transform.position
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
### Step 4: Aim the controller
|
|
59
|
-
|
|
60
|
-
Point the controller at the target. Default to `"controller-right"` unless the user specified left. Do NOT move the controller — only rotate it.
|
|
61
|
-
|
|
62
|
-
```
|
|
63
|
-
xr_look_at({ device: "controller-right", target: { x, y, z } })
|
|
28
|
+
```bash
|
|
29
|
+
npx @iwsdk/cli xr select --input-json '{"device":"controller-right","duration":0.2}'
|
|
64
30
|
```
|
|
65
31
|
|
|
66
|
-
|
|
32
|
+
Use an explicit select-value press/release only when the task needs to inspect
|
|
33
|
+
the held interval. Query the named application state or control effect after the
|
|
34
|
+
click; a screenshot alone is not interaction proof.
|
|
67
35
|
|
|
68
|
-
|
|
36
|
+
### Distance grab
|
|
69
37
|
|
|
70
|
-
|
|
38
|
+
For `DistanceGrabbable`, hold trigger button index `0`, optionally move the
|
|
39
|
+
controller, then release index `0`:
|
|
71
40
|
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
For simple clicks, send an explicit trigger press and release. Use for UI buttons and objects that respond to the Pressed component.
|
|
75
|
-
|
|
76
|
-
```
|
|
77
|
-
xr_set_select_value({ device: "controller-right", value: 1 })
|
|
78
|
-
xr_set_select_value({ device: "controller-right", value: 0 })
|
|
41
|
+
```json
|
|
42
|
+
{"device":"controller-right","buttons":[{"index":0,"value":1}]}
|
|
79
43
|
```
|
|
80
44
|
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
### Branch B: Distance Grab (objects with DistanceGrabbable)
|
|
85
|
-
|
|
86
|
-
DistanceGrabbable requires **press and hold** on the trigger (button index 0), not a quick select.
|
|
87
|
-
|
|
88
|
-
#### B1: Engage trigger
|
|
45
|
+
Check `movementMode` before deciding whether the object should remain remote or
|
|
46
|
+
move toward the hand. Query `Grabbed` or transform state while held and after
|
|
47
|
+
release. Do not substitute squeeze index `1`; that is the proximity-grab path.
|
|
89
48
|
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
The object is now distance-grabbed. Behavior depends on the `movementMode`:
|
|
98
|
-
|
|
99
|
-
- **MoveFromTarget / MoveAtSource / RotateAtSource** — object stays remote, moves relative to controller movement
|
|
100
|
-
- **MoveTowardsTarget** — object flies into the controller's hand, then behaves like a proximity grab
|
|
101
|
-
|
|
102
|
-
#### B2: Move to destination (optional)
|
|
103
|
-
|
|
104
|
-
If the user wants to move the object somewhere, animate the controller to the destination.
|
|
105
|
-
|
|
106
|
-
```
|
|
107
|
-
xr_animate_to({
|
|
108
|
-
device: "controller-right",
|
|
109
|
-
position: { x, y, z },
|
|
110
|
-
duration: 0.5,
|
|
111
|
-
})
|
|
112
|
-
```
|
|
113
|
-
|
|
114
|
-
If no destination specified but user asked to "move" or "bring" the object, animate to in front of the headset: `xr_get_transform({ "device": "headset" })` → place at `(head.x, head.y - 0.2, head.z - 0.5)`.
|
|
115
|
-
|
|
116
|
-
#### B3: Release trigger
|
|
117
|
-
|
|
118
|
-
```
|
|
119
|
-
xr_set_gamepad_state({
|
|
120
|
-
device: "controller-right",
|
|
121
|
-
buttons: [{ index: 0, value: 0 }],
|
|
122
|
-
})
|
|
123
|
-
```
|
|
124
|
-
|
|
125
|
-
#### B4: Return controller
|
|
126
|
-
|
|
127
|
-
Animate back to resting position.
|
|
128
|
-
|
|
129
|
-
```
|
|
130
|
-
xr_animate_to({
|
|
131
|
-
device: "controller-right",
|
|
132
|
-
position: { x: 0.2, y: 1.4, z: -0.3 },
|
|
133
|
-
duration: 0.5,
|
|
134
|
-
})
|
|
135
|
-
```
|
|
136
|
-
|
|
137
|
-
Default resting positions: right `(0.2, 1.4, -0.3)`, left `(-0.2, 1.4, -0.3)`.
|
|
138
|
-
|
|
139
|
-
### Step 5: Verify (optional)
|
|
140
|
-
|
|
141
|
-
Take a screenshot to confirm the result.
|
|
142
|
-
|
|
143
|
-
```
|
|
144
|
-
browser_screenshot
|
|
145
|
-
```
|
|
49
|
+
For `MoveFromTarget`, do not inspect implementation source. Before pressing,
|
|
50
|
+
record controller position `P` and object position `S`, and compute the grab
|
|
51
|
+
distance `d = |S - P|`. While trigger is held, call `xr look-at` with the
|
|
52
|
+
requested object destination as `target` and `moveToDistance: d`; this preserves
|
|
53
|
+
the ray distance while moving its endpoint to the destination. Query the object
|
|
54
|
+
before release to confirm the move, then release and query again.
|
|
146
55
|
|
|
147
|
-
|
|
148
|
-
switches to runtime before capture.
|
|
56
|
+
## Evidence and stopping
|
|
149
57
|
|
|
150
|
-
|
|
58
|
+
For a task with both a click and a distance grab, locate/aim/click/query the first
|
|
59
|
+
target, then locate/aim/hold-move-release/query the second. Do not rediscover the
|
|
60
|
+
CLI family, inspect pointer internals, or guess panel offsets from screenshots.
|
|
151
61
|
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
- **Don't move the controller to the target** — ray interactions work at a distance. Only rotate via `xr_look_at`, don't translate.
|
|
62
|
+
Capture one runtime image only when visual placement is part of the request.
|
|
63
|
+
When the request names an artifact path, save it directly with
|
|
64
|
+
`npx @iwsdk/cli browser screenshot --output-file <path>`; do not capture into model
|
|
65
|
+
context and then add a copy step.
|
|
66
|
+
Stop when the real input and resulting named state are proven, leaving all
|
|
67
|
+
buttons released.
|