@relu-interactives/spatial-ecs 0.4.3 → 0.4.5

package/dist/docs/guide/mixed-reality.md

@@ -7,9 +7,34 @@ With passthrough on, the experience composites over the user's real room instead
 of a virtual background. Turn it on in the inspector (XR Rig → Mixed Reality), or
 from a script by authoring the rig.
 
-Mixed reality is a **Quest and Android XR** capability. Safari on visionOS grants
-`immersive-ar` to no web page at all, so a passthrough project falls back to an
-ordinary VR session there rather than failing to start.
+Mixed reality passthrough is a **Quest and Android XR** capability.
+
+## Apple Vision Pro
+
+Safari on visionOS — checked through visionOS 27 — supports WebXR
+`immersive-vr` only. It grants `immersive-ar` to no web page, and exposes no
+plane detection, mesh detection or depth sensing. This is a platform policy,
+not a missing feature on our side; every web XR tool works within it (the
+industry-standard answer is a VR fallback mode).
+
+A mixed-reality project still works there, three ways:
+
+1. **Automatic VR fallback.** The session probe falls back to `immersive-vr`,
+   the authored background applies, and hands/pinch interaction works. The
+   entry page says so honestly: "Passthrough isn't available in this browser —
+   this experience will run in VR."
+2. **A synthetic floor, no scan needed.** With `environmentPhysics` on and no
+   scene understanding granted at all, the engine synthesizes one floor
+   collider whose top face sits at y = 0 — which `local-floor` makes the
+   user's REAL floor. Dropped objects land at floor height instead of falling
+   forever. (This only happens when the detection features are entirely
+   absent; a Quest that merely has not scanned yet keeps its room-capture
+   prompt instead.)
+3. **"View in your room" via AR Quick Look.** On Apple devices the entry page
+   offers a native placement preview: the scene's content is exported to
+   `.usdz` on the fly and handed to AR Quick Look, which places it in the real
+   room with passthrough — Apple's sanctioned web-AR path (it also works on
+   iPhone and iPad). This is a static snapshot: no scripts, no physics.
 
 ## What the rig controls
 
@@ -41,6 +66,87 @@ Each detected surface becomes an ordinary entity, so it goes through the same
 physics as everything else: a dynamic rigidbody dropped in a mixed-reality scene
 lands on the real floor with no extra code.
 
+### The room has to have been scanned first
+
+This is the single most common reason a mixed-reality scene comes up empty, and
+it is not a bug in the experience.
+
+**Plane and mesh detection do not scan anything.** They replay a scan the user
+captured earlier through the headset's own room setup — on Quest, *Settings >
+Guardian > Mixed Reality* (Space Setup). If the user has never run it,
+`detectedPlanes` and `detectedMeshes` simply stay empty, with no prompt and no
+error. If they ran it in a **different room**, you get that room's furniture
+overlaid on this one, which looks like the tracking is broken.
+
+`XREnvironmentSystem` handles this: when `environmentPhysics` is on and nothing
+at all has been detected three seconds into the session, it calls
+`XRSession.initiateRoomCapture()`, which launches room setup from inside the
+session. It is a Meta extension, so it is feature-detected and silently skipped
+elsewhere, and — per Meta's contract — it is only ever called **once per
+session**.
+
+If a user reports "it never asked to scan my room" and the geometry does not
+match, they are almost certainly running against a stale scan from another
+space; re-running Space Setup for the current room fixes it.
+
+### Reading the `environmentDebug` overlay
+
+The overlay colours surfaces by what the runtime says they are, so a glance
+(or a screenshot) tells you which surface is wrong rather than just "wireframe
+somewhere":
+
+- **green** — the floor plane
+- **blue** — walls
+- **purple** — the ceiling
+- **orange** — labelled things: tables, couches, shelves, doors, windows
+- **faint gray** — the raw room scan (the "global mesh")
+
+Two things about the global mesh are easy to misread. It is deliberately
+COARSE — a couch becomes a melted lump, a wall gets bumps — because it is a
+collision mesh, not a picture; those lumps are not phantom objects. And under
+`environmentPhysics: "auto"` it is what the physics uses, with the flat planes
+skipped entirely, so orange furniture boxes and gray scan lumps can legitimately
+overlap the same real object.
+
+The overlay is depth-tested like ordinary geometry — the near wall hides the
+far one — but it never *writes* depth, so it cannot occlude actual content. (It
+was briefly rendered X-ray; on device, the dense global scan at low opacity
+fuses into translucent sheets, and without a depth test the walls' sheets drew
+through everything and stacked in front of the user's face.)
+
+For identifying a mystery shape, `XREnvironmentSystem.getDebugSnapshot(world)`
+returns a census of every spawned surface — kind, semantic label, position and
+approximate radius.
+
+One artifact is handled for you: three.js's depth-occlusion pass draws a
+head-locked mesh that writes only depth, but leaves its colour output
+undefined — on Quest hardware that painted the depth camera's view over the
+screen as faint contour bands that follow the head. `XRSessionSystem` forces
+`colorWrite` off on that mesh every frame, so enabling `depthOcclusion` costs
+no visual artifacts.
+
+### Lost tracking freezes a surface — it never follows the head
+
+When the headset loses tracking of a surface, WebXR does not stop reporting a
+pose — it reports an *emulated* one: the last known pose held relative to the
+viewer, i.e. glued to the head. Applying that to room geometry would make the
+surface — and its collider — follow the user around, shoving physics objects
+out of the way. The engine instead freezes the surface at its last tracked
+pose, resumes when tracking returns, and never creates a surface from an
+emulated pose in the first place. The diagnostics census marks such surfaces
+with `emulated: true`.
+
+### No artificial locomotion in passthrough
+
+While a session composites over the real world, `XRLocomotionSystem` disables
+ALL artificial locomotion — snap turn, teleport and smooth movement — whatever
+the rig says. Virtual content in mixed reality is registered to the real room;
+rotating or translating the rig swings every virtual object out of alignment
+while the passthrough room stays put, which feels broken rather than like
+movement. The user moves by walking. Note that `snapTurn` is a separate field
+from `locomotion`, so `locomotion: "none"` alone would NOT have stopped the
+stick from turning the rig — the gate is on the live blend mode.
+
 ## Finding the real room from a script
 
 Detected surfaces carry `XREnvironmentComponent`, so they are queryable like any
@@ -93,6 +199,36 @@ export default class RoomAware extends Behaviour {
 }
 ```
 
+## Hit-testing the real room from a script
+
+With `hitTest` on the rig, `XRHitTestSystem` keeps a ray from the viewer (and
+one per hand) cast against the real room. Systems are resolved by INSTANCE, so
+pass a throwaway one to `getSystem`:
+
+```ts
+import { Behaviour, XRHitTestSystem } from "@relu-interactives/spatial-ecs";
+import { TransformComponent } from "@relu-interactives/spatial-ecs/components";
+
+export default class PlaceOnRealSurface extends Behaviour {
+  onSelectStart() {
+    const hitTest = this.world.getSystem(new XRHitTestSystem());
+    // "viewer" = where the user is looking; "left"/"right" = hand rays.
+    const hit = hitTest.getHitResult("viewer");
+    if (!hit) return; // nothing granted, or no surface under the ray yet
+
+    const transform = this.world.getComponent(this.entityId, TransformComponent);
+    if (transform) {
+      transform.position.x = hit.position.x;
+      transform.position.y = hit.position.y;
+      transform.position.z = hit.position.z;
+    }
+  }
+}
+```
+
+`getHitResult` returns null until the runtime grants hit-testing and the ray
+actually meets a surface — always handle that case.
+
 ## Anchors
 
 A headset keeps revising where it thinks the room is. Content at fixed scene
@@ -106,6 +242,27 @@ on, and it stays put relative to the room.
 Because the anchor owns the transform, moving the entity by script or physics has
 no lasting effect — disable the component first, move it, then re-enable.
 
+From a script the toggle is the whole API:
+
+```ts
+import { Behaviour } from "@relu-interactives/spatial-ecs";
+import { XRAnchorComponent } from "@relu-interactives/spatial-ecs/components";
+
+export default class ReleaseOnGrab extends Behaviour {
+  onGrab() {
+    // Let the hand move it: release the weld for the duration of the grab.
+    const anchor = this.world.getComponent(this.entityId, XRAnchorComponent);
+    if (anchor) anchor.enabled = false;
+  }
+
+  onRelease() {
+    // Re-anchor wherever the user left it.
+    const anchor = this.world.getComponent(this.entityId, XRAnchorComponent);
+    if (anchor) anchor.enabled = true;
+  }
+}
+```
+
 Anchors last for the session only. Persisting one across launches needs
 `requestPersistentHandle`, which is a draft extension and not supported here.
 

package/dist/docs/guide/vr-and-hand-poses.md

@@ -185,6 +185,35 @@ onXRSessionStart(capabilities: XrCapabilities) {
 
 Two rules worth internalising: **move the rig, never the camera** (the headset owns the camera while presenting and overwrites it every frame), and give any destructive gesture a longer **Hold Time** so it cannot fire in passing.
 
+### Reading grab state from a script
+
+`XRInteractionSystem.getGrabInfo(entityId)` reports who is holding an entity —
+hand count, primary/secondary handedness, live hand separation (two-hand grabs)
+and the attach-point ids in use. Systems resolve by instance:
+
+```ts
+import {
+  Behaviour,
+  XRInputSystem,
+  XRInteractionSystem,
+} from "@relu-interactives/spatial-ecs";
+
+export default class TwoHandChargeUp extends Behaviour {
+  onUpdate() {
+    const interaction = this.world.getSystem(
+      new XRInteractionSystem(new XRInputSystem()),
+    );
+    const grab = interaction.getGrabInfo(this.entityId);
+    if (!grab || grab.hands < 2 || grab.handSeparation === null) return;
+
+    // Example: pull both hands apart to "charge" — separation is metres.
+    console.log("charge level", Math.min(1, grab.handSeparation / 0.8));
+  }
+}
+```
+
+Returns null while nothing holds the entity, so gate on it every frame.
+
 ## Hand poses
 
 Add an **XR Hand Pose** component and list the poses to watch for. Each is either a built-in preset or a custom set of finger curls.

package/dist/docs/guide/world-api.md

@@ -102,7 +102,7 @@ Both methods accept `Vec3` (`THREE.Vector3` or `{ x, y, z }`) and return `Inters
 
 | Method | Purpose |
 | --- | --- |
-| `getSystem(SystemClass)` | Resolve a registered system instance. |
+| `getSystem(instance)` | Resolve the REGISTERED system matching the argument's class — pass a throwaway instance (`world.getSystem(new XRHitTestSystem())`), never the class itself. |
 | `hasSystem(SystemClass)` | `true` if the system is registered. |
 | `getObjectManagementSystem()` | Shorthand for the built-in `ObjectManagementSystem`. Use this to spawn or destroy entities at runtime (`requestCreate`, `createNow`, `deleteEntity`). |
 | `getEnvironmentComponent()` | Returns the singleton [`Environment`](./scene/environment) component, or `undefined` if not yet initialized. Mutate `values` to change skybox, fog, or gravity at runtime. |

package/dist/{index-n7SeH119.d.ts → index-BphKHaV-.d.ts}

@@ -647,8 +647,29 @@ declare class XRSessionSystem implements EcsSystem {
     private readonly inputSystem;
     /** Y compensation applied when a floor-relative space was requested but not granted. */
     private appliedOriginOffset;
+    /** The live session, so an async floor probe can tell if it has been replaced. */
+    private session;
     constructor(inputSystem?: XRInputSystem);
     update(world: World): void;
+    /**
+     * Stop three's depth-occlusion pass from writing COLOR.
+     *
+     * With depth sensing granted, three draws a full-screen, head-locked mesh
+     * every frame whose fragment shader assigns ONLY `gl_FragDepth` — that is
+     * the whole occlusion mechanism — but its material never sets
+     * `colorWrite: false`, so the colour output is UNDEFINED per GLSL. Desktop
+     * drivers happen to zero it, which is why this ships unnoticed; Quest's
+     * mobile GPU frequently leaves the last-computed temporary in the register —
+     * the sampled DEPTH value — so the pass paints the depth camera's view over
+     * the screen as faint head-locked contour bands. On device that reads as "a
+     * large wireframe dome that follows my head".
+     *
+     * The mesh is created lazily inside three's render, so this is re-checked
+     * every frame rather than hooked once; the write is idempotent and the
+     * check is two property reads. Depth-only is the mesh's entire purpose —
+     * occlusion is unaffected.
+     */
+    private muteDepthOcclusionColor;
     private handleSessionStart;
     /**
      * Apply the rig's authored tracking origin, and compensate when the device
@@ -690,12 +711,55 @@ declare class XREnvironmentSystem implements EcsSystem {
     private readonly meshes;
     private presenting;
     private debugVisible;
-    update(world: World): void;
+    /** Seconds presenting, used to decide the room has genuinely nothing in it. */
+    private elapsed;
+    /** `initiateRoomCapture` may be called only ONCE per session. */
+    private roomCaptureRequested;
+    update(world: World, deltaTime?: number): void;
+    /**
+     * Ask the runtime to scan the room when it turns out we have no room.
+     *
+     * Plane and mesh detection do NOT scan anything — they replay a scan the user
+     * captured earlier through the system's own room setup. Most people have
+     * never run it, and those who have may have run it in a DIFFERENT room, so a
+     * mixed-reality experience opens against either nothing at all or someone
+     * else's furniture, with no prompt and no error.
+     *
+     * `XRSession.initiateRoomCapture()` (Meta's extension) launches that setup
+     * from inside the session, and exists precisely for this case. Contract, from
+     * Meta's docs: callable ONCE per session, and only worth calling once we are
+     * confident the room really is empty — the arrays populate a beat after the
+     * session starts, so a check on frame one would fire for everyone.
+     */
+    private maybeCaptureRoom;
     /** How many pieces of real geometry are currently colliding. Script-facing. */
     getDetectedCount(): {
         planes: number;
         meshes: number;
     };
+    /**
+     * A census of every surface currently spawned: what it is, where it sits and
+     * roughly how big it is. Built for the template's post-session readout, so a
+     * mystery shape on the headset can be looked up by its label instead of
+     * guessed at — a small-radius entry at head height IS the mystery shape, and
+     * a clean list means whatever was seen did not come from this system.
+     */
+    getDebugSnapshot(world: World): {
+        kind: "plane" | "mesh";
+        label: string | null;
+        position: {
+            x: number;
+            y: number;
+            z: number;
+        };
+        radius: number;
+        /** Tracking currently lost — pose frozen at the last known place. */
+        emulated: boolean;
+    }[];
+    /** Create the no-scan floor once; it lives in the ordinary plane map. */
+    private ensureSyntheticFloor;
+    /** Real scene data exists (or arrived) — the stand-in must go. */
+    private removeSyntheticFloor;
     private findRig;
     private syncPlanes;
     private syncMeshes;

package/dist/{index-CBdnIyI8.d.cts → index-CH7qMsQT.d.cts}

@@ -647,8 +647,29 @@ declare class XRSessionSystem implements EcsSystem {
     private readonly inputSystem;
     /** Y compensation applied when a floor-relative space was requested but not granted. */
     private appliedOriginOffset;
+    /** The live session, so an async floor probe can tell if it has been replaced. */
+    private session;
     constructor(inputSystem?: XRInputSystem);
     update(world: World): void;
+    /**
+     * Stop three's depth-occlusion pass from writing COLOR.
+     *
+     * With depth sensing granted, three draws a full-screen, head-locked mesh
+     * every frame whose fragment shader assigns ONLY `gl_FragDepth` — that is
+     * the whole occlusion mechanism — but its material never sets
+     * `colorWrite: false`, so the colour output is UNDEFINED per GLSL. Desktop
+     * drivers happen to zero it, which is why this ships unnoticed; Quest's
+     * mobile GPU frequently leaves the last-computed temporary in the register —
+     * the sampled DEPTH value — so the pass paints the depth camera's view over
+     * the screen as faint head-locked contour bands. On device that reads as "a
+     * large wireframe dome that follows my head".
+     *
+     * The mesh is created lazily inside three's render, so this is re-checked
+     * every frame rather than hooked once; the write is idempotent and the
+     * check is two property reads. Depth-only is the mesh's entire purpose —
+     * occlusion is unaffected.
+     */
+    private muteDepthOcclusionColor;
     private handleSessionStart;
     /**
      * Apply the rig's authored tracking origin, and compensate when the device
@@ -690,12 +711,55 @@ declare class XREnvironmentSystem implements EcsSystem {
     private readonly meshes;
     private presenting;
     private debugVisible;
-    update(world: World): void;
+    /** Seconds presenting, used to decide the room has genuinely nothing in it. */
+    private elapsed;
+    /** `initiateRoomCapture` may be called only ONCE per session. */
+    private roomCaptureRequested;
+    update(world: World, deltaTime?: number): void;
+    /**
+     * Ask the runtime to scan the room when it turns out we have no room.
+     *
+     * Plane and mesh detection do NOT scan anything — they replay a scan the user
+     * captured earlier through the system's own room setup. Most people have
+     * never run it, and those who have may have run it in a DIFFERENT room, so a
+     * mixed-reality experience opens against either nothing at all or someone
+     * else's furniture, with no prompt and no error.
+     *
+     * `XRSession.initiateRoomCapture()` (Meta's extension) launches that setup
+     * from inside the session, and exists precisely for this case. Contract, from
+     * Meta's docs: callable ONCE per session, and only worth calling once we are
+     * confident the room really is empty — the arrays populate a beat after the
+     * session starts, so a check on frame one would fire for everyone.
+     */
+    private maybeCaptureRoom;
     /** How many pieces of real geometry are currently colliding. Script-facing. */
     getDetectedCount(): {
         planes: number;
         meshes: number;
     };
+    /**
+     * A census of every surface currently spawned: what it is, where it sits and
+     * roughly how big it is. Built for the template's post-session readout, so a
+     * mystery shape on the headset can be looked up by its label instead of
+     * guessed at — a small-radius entry at head height IS the mystery shape, and
+     * a clean list means whatever was seen did not come from this system.
+     */
+    getDebugSnapshot(world: World): {
+        kind: "plane" | "mesh";
+        label: string | null;
+        position: {
+            x: number;
+            y: number;
+            z: number;
+        };
+        radius: number;
+        /** Tracking currently lost — pose frozen at the last known place. */
+        emulated: boolean;
+    }[];
+    /** Create the no-scan floor once; it lives in the ordinary plane map. */
+    private ensureSyntheticFloor;
+    /** Real scene data exists (or arrived) — the stand-in must go. */
+    private removeSyntheticFloor;
     private findRig;
     private syncPlanes;
     private syncMeshes;