@ssheleg/xr-dev 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (39) hide show
  1. package/CHANGELOG.md +143 -0
  2. package/LICENSE +21 -0
  3. package/README.md +98 -0
  4. package/SECURITY.md +39 -0
  5. package/bin/xr-dev.js +186 -0
  6. package/package.json +56 -0
  7. package/plugins/xr-dev/.claude-plugin/plugin.json +37 -0
  8. package/plugins/xr-dev/skills/quest-lifecycle/SKILL.md +98 -0
  9. package/plugins/xr-dev/skills/quest-lifecycle/references/engine-paths.md +38 -0
  10. package/plugins/xr-dev/skills/quest-lifecycle/references/immersive-design.md +45 -0
  11. package/plugins/xr-dev/skills/quest-lifecycle/references/stage-map.md +60 -0
  12. package/plugins/xr-dev/skills/quest-native/SKILL.md +191 -0
  13. package/plugins/xr-dev/skills/quest-native/references/doc-map.md +90 -0
  14. package/plugins/xr-dev/skills/quest-native/references/frame-loop.md +92 -0
  15. package/plugins/xr-dev/skills/quest-native/references/manifest-and-gradle.md +111 -0
  16. package/plugins/xr-dev/skills/quest-native/references/mixed-reality.md +46 -0
  17. package/plugins/xr-dev/skills/quest-native/references/project-playbook.md +56 -0
  18. package/plugins/xr-dev/skills/quest-perf/SKILL.md +128 -0
  19. package/plugins/xr-dev/skills/quest-perf/references/capture-playbook.md +102 -0
  20. package/plugins/xr-dev/skills/quest-perf/references/mobile-rendering.md +56 -0
  21. package/plugins/xr-dev/skills/quest-perf/references/rendering-playbook.md +51 -0
  22. package/plugins/xr-dev/skills/quest-spatial/SKILL.md +170 -0
  23. package/plugins/xr-dev/skills/quest-spatial/references/budgets-and-traps.md +73 -0
  24. package/plugins/xr-dev/skills/quest-spatial/references/build-and-audit.md +36 -0
  25. package/plugins/xr-dev/skills/quest-spatial/references/docs-map.md +91 -0
  26. package/plugins/xr-dev/skills/quest-spatial/references/hybrid-activities.md +31 -0
  27. package/plugins/xr-dev/skills/quest-spatial/references/samples-map.md +60 -0
  28. package/plugins/xr-dev/skills/quest-store/SKILL.md +145 -0
  29. package/plugins/xr-dev/skills/quest-store/references/launch-and-growth.md +96 -0
  30. package/plugins/xr-dev/skills/quest-store/references/production-readiness.md +56 -0
  31. package/plugins/xr-dev/skills/quest-store/references/store-asset-production.md +48 -0
  32. package/plugins/xr-dev/skills/quest-store/references/vrc-checklist.md +193 -0
  33. package/plugins/xr-dev/skills/quest-tooling/SKILL.md +158 -0
  34. package/plugins/xr-dev/skills/quest-tooling/references/research-navigation.md +47 -0
  35. package/plugins/xr-dev/skills/quest-tooling/references/source-research.md +58 -0
  36. package/plugins/xr-dev/skills/quest-tooling/references/tool-matrix.md +54 -0
  37. package/plugins/xr-dev/skills/quest-webxr/SKILL.md +141 -0
  38. package/plugins/xr-dev/skills/quest-webxr/references/pwa-packaging.md +85 -0
  39. package/plugins/xr-dev/skills/quest-webxr/references/runtime-delivery.md +40 -0
@@ -0,0 +1,111 @@
1
+ # The manifest and Gradle contract, both halves
2
+
3
+ **Read this when** writing or reviewing an `AndroidManifest.xml` or `build.gradle`
4
+ for a native Quest app, or when a build is rejected for `VRC.Quest.Packaging.1`
5
+ or `.4`.
6
+
7
+ Sources: [native manifest](https://developers.meta.com/horizon/documentation/native/android/mobile-native-manifest/)
8
+ (page dated 2026-09-01) and [release manifest](https://developers.meta.com/horizon/resources/publish-mobile-manifest/)
9
+ (2026-08-31), rechecked 2026-09-21. **Development and release requirements are two different documents.**
10
+
11
+ ## Contents
12
+
13
+ - Development manifest
14
+ - Release manifest (the one review enforces)
15
+ - SDK levels, with the 2026 rule
16
+ - Gradle and the OpenXR loader
17
+ - Supported devices and compatibility mode
18
+
19
+ ## Development manifest
20
+
21
+ ```xml
22
+ <manifest xmlns:android="http://schemas.android.com/apk/res/android"
23
+ package="<packagename>">
24
+ <uses-feature android:name="android.hardware.vr.headtracking"
25
+ android:required="true" android:version="1" />
26
+ <application android:label="@string/app_name"
27
+ android:theme="@android:style/Theme.Black.NoTitleBar.Fullscreen">
28
+ <activity android:name="android.app.NativeActivity"
29
+ android:screenOrientation="landscape"
30
+ android:configChanges="density|keyboard|keyboardHidden|navigation|orientation|screenLayout|screenSize|uiMode"
31
+ android:excludeFromRecents="false">
32
+ <intent-filter>
33
+ <action android:name="android.intent.action.MAIN" />
34
+ <category android:name="android.intent.category.LAUNCHER" />
35
+ <category android:name="com.oculus.intent.category.VR" />
36
+ </intent-filter>
37
+ </activity>
38
+ </application>
39
+ </manifest>
40
+ ```
41
+
42
+ - `android.hardware.vr.headtracking` `required="true"` is also what v2 signing
43
+ and store review expect from an immersive app.
44
+ - **Never** add `android:noHistory`.
45
+ - Landscape and the black fullscreen theme are comfort requirements during app
46
+ transitions, not style.
47
+
48
+ ## Release manifest — the delta
49
+
50
+ | Element | Release requirement |
51
+ |---|---|
52
+ | `android:debuggable` | `false` or absent — a release build |
53
+ | `installLocation` | `auto` (or `0`); anything else needs store-team approval |
54
+ | `android:label` | the app's name, **unique on the platform** |
55
+ | `android.hardware.vr.headtracking` | `required="true"` for immersive; omitted or `false` for a 2D panel app |
56
+ | `android:excludeFromRecents` | `true` in the release launch activity |
57
+ | intent filter | `MAIN` + `LAUNCHER`; current Meta OpenXR release example adds `com.oculus.intent.category.VR` |
58
+ | `com.oculus.supportedDevices` | `quest2\|questpro\|quest3\|quest3s` as a `meta-data` element in `application` |
59
+
60
+ The snippet is a manifest segment, not a complete generated project. Verify
61
+ merged APK output and preserve additional categories required by the chosen
62
+ SDK/cross-runtime target; the Meta category is not a universal OpenXR spec rule.
63
+
64
+ Non-conformance fails `VRC.Quest.Packaging.1` and/or `.4` — while still
65
+ installing happily when sideloaded, which is what makes it a late surprise.
66
+
67
+ ## SDK levels, with the 2026 rule
68
+
69
+ Recommended for in-lifecycle devices (Quest 2, Quest Pro, Quest 3 family):
70
+
71
+ | | minSdkVersion | targetSdkVersion | compileSdkVersion |
72
+ |---|---|---|---|
73
+ | Recommended | 32 | 34 | 34 |
74
+ | Legal range | 29–34 | 32–34 immersive, 32–36 for 2D | ≥ targetSdkVersion |
75
+
76
+ **Apps created since 1 March 2026 must set `targetSdkVersion` to 34.** Horizon
77
+ OS has been Android 14 (SDK 34) since HzOS v76 in April 2025; older releases
78
+ were Android 12 (SDK 32). To gate on a Horizon OS version rather than an Android
79
+ one, use "Requiring Minimum OS Versions"
80
+ (`documentation/native/min-os-versions`), not `minSdkVersion`.
81
+
82
+ `maxSdkVersion` is supported by Android and **not recommended** on Quest.
83
+
84
+ ## Gradle and the OpenXR loader
85
+
86
+ ```gradle
87
+ android {
88
+ compileSdk 34
89
+ defaultConfig { minSdk 32; targetSdk 34; ndk { abiFilters 'arm64-v8a' } }
90
+ buildFeatures { prefab true } // required for the loader's prefab
91
+ externalNativeBuild { cmake { cppFlags '-std=c++20' } }
92
+ }
93
+ dependencies {
94
+ implementation 'org.khronos.openxr:openxr_loader_for_android:<version>'
95
+ }
96
+ ```
97
+
98
+ - Only `arm64-v8a` ships: `VRC.Quest.Packaging.6` requires 64-bit binaries.
99
+ - **Loader below 1.0.34 crashes**; apps on the Khronos loader also crash on
100
+ Horizon OS older than v62, and non-Quest-1 users below v62 never see the
101
+ update in the Store.
102
+ - Pin the loader version in whatever lock file the project keeps **and** in
103
+ `build.gradle`, then check they agree — a drifted lock is a release note
104
+ describing a build that does not exist.
105
+
106
+ ## Supported devices and compatibility mode
107
+
108
+ A headset whose model is absent from `com.oculus.supportedDevices` runs the app
109
+ in **compatibility mode** and reports itself as a previous generation. That is a
110
+ safety net, not a plan: query capabilities (extension enumeration) rather than
111
+ device models, and add new identifiers as they appear.
@@ -0,0 +1,46 @@
1
+ # Mixed reality capabilities and lifecycle
2
+
3
+ **Read this when** implementing passthrough, room understanding, anchors, hands/body/eyes, shared physical space, camera processing or AI input on Quest.
4
+
5
+ Sources checked 2026-09-21. Select a feature by the app's need and runtime support, not by “all SDK features.” The acceptance checks below are recommended project tests, not a claim that every row is a Store rule.
6
+
7
+ ## Capability contract
8
+
9
+ For each feature record: API/SDK and source date; required versus optional; runtime support; manifest declaration; current permission grant; account/device restrictions; data lifetime; fallback; test scene and evidence. An available extension and a granted Android permission are separate facts. Inspect valid/tracked pose flags before using coordinates.
10
+
11
+ | Feature | Choose it for | Failure path to test |
12
+ |---|---|---|
13
+ | Displayed passthrough | Showing the real world behind/through virtual content | Unavailable session/mode, focus loss, transition back to VR; do not assume access to camera pixels |
14
+ | Scene/MRUK room model | Semantic surfaces, placement, generated room gameplay | Permission denied, no Space Setup, incomplete scan, changed room, rescan and unsupported semantics |
15
+ | Environment depth | Occlusion against the physical world | Unsupported/invalid depth; fall back visually, never treat it as authoritative collision/safety geometry |
16
+ | Spatial anchors | Stable real-world placement and later localization | Save/share/load failure, localization timeout, changed room, deleted anchor |
17
+ | Shared anchors/colocation | Aligning multiple users' coordinate systems | Anchor alignment success with network-state failure and vice versa; stale alignment and host loss |
18
+ | Hands/controllers | Interaction, gestures and haptics | Switching input source, tracking quality, occluded hands, controller disconnect, reserved system gesture |
19
+ | Body/face/eye data | Supported avatar/interaction requirements | Capability unavailable or consent refused; retain a usable avatar/input alternative |
20
+ | Raw passthrough camera | CV/ML, object recognition, QR/reference acquisition | Missing grant/device/account support, camera busy, invalid frame, changed resolution, resume after pause |
21
+
22
+ Primary maps: [passthrough](https://developers.meta.com/horizon/essentials/horizon-os-passthrough/), [spatial data permission](https://developers.meta.com/horizon/documentation/unity/unity-spatial-data-perm/), [MRUK features](https://developers.meta.com/horizon/documentation/unreal/unreal-mr-utility-kit-features/), [Shared Spatial Anchors sample](https://github.com/oculus-samples/Unity-SharedSpatialAnchors), [native feature samples](https://developers.meta.com/horizon/documentation/native/native-openxr-sdk-sample/).
23
+
24
+ ## Camera facts that commonly get mixed together
25
+
26
+ Displayed passthrough, raw forward camera frames, environment depth, a saved scene mesh and MediaProjection/casting are distinct products. Their images, coordinate frames, permissions and availability are not interchangeable. An image that looks like the headset view may omit UI or cover a different field of view.
27
+
28
+ The [camera overview](https://developers.meta.com/horizon/documentation/spatial-sdk/spatial-sdk-pca-overview/) describes Quest 3/3S, Horizon OS v74+, Camera2-based access and either CAMERA or HEADSET_CAMERA. The [native camera page](https://developers.meta.com/horizon/documentation/native/android/pca-native-documentation/) also says both permissions are needed, while [MRUK's component](https://developers.meta.com/horizon/reference/mruk/v85/class_meta_x_r_passthrough_camera_access/) names HEADSET_CAMERA. Treat this as a documented discrepancy: use the chosen component's versioned requirements, inspect the merged manifest, request the minimum supported grant and verify a fresh install. Record unresolved conflicts; do not silently broaden permissions.
29
+
30
+ Query camera identity, supported output sizes and calibration metadata; never hardcode camera index, resolution or the assumption that it covers the user's full view. Keep capture timestamps and convert through the correct camera/head/world transforms before placing content. Close images, sessions, readers and GPU resources according to the API lifecycle; bound queues and drop stale frames rather than accumulating latency.
31
+
32
+ The overview states that the Passthrough Camera API is unsupported in XR Simulator. It also warns that parental restrictions may not be applied to an MQDH-installed app: developer sideloading cannot certify the release-channel age/account path. Validate that path separately. The [public-release announcement](https://developers.meta.com/horizon/blog/new-era-mixed-reality-passthrough-camera-api-machine-learning-computer-vision/) permits Store publication; do not repeat the obsolete blanket claim that camera apps cannot ship.
33
+
34
+ Camera data is Device User Data under the linked Meta policy. Explain capture/off-device use, minimize retention and redact diagnostics; apply the product's consent/privacy flow. Permission to render passthrough is not permission to upload room images to an AI service. Test denial/revocation without stranding the user.
35
+
36
+ ## Shared physical play is more than anchors
37
+
38
+ Keep local tracking space, shared world transform and authoritative game simulation distinct. Share anchor identities/alignment under the chosen SDK; use an actual networking solution for object state, ownership, prediction/interpolation and reconnection. [Group Presence](https://developers.meta.com/horizon/documentation/native/ps-group-presence-overview/) reports destinations/joinability; it does not synchronize physics. Exercise invite/deep-link joining when the app is cold, active or already in another session.
39
+
40
+ Test two real accounts and devices in representative rooms, including non-joinable/private states. Bound avatar/network updates independently of the render loop. Include mute/block/report and data handling when the product has social/UGC features, under current platform policy.
41
+
42
+ ## Comfort and interaction review
43
+
44
+ Consume the project's scenario set. Cover seated/standing reach, left/right-handed input, snap/smooth movement preferences, controller/hand alternatives, text angular size and distance, subtitles and non-audio feedback. World-lock stable content; use head-locked content deliberately. Do not force camera motion or hide tracking failure behind a seemingly stable world. Respect system focus/gestures and the OS boundary.
45
+
46
+ Use [Meta design guidance](https://developers.meta.com/horizon/design/) and a human headset review for comfort. Automated input, screenshots and static reach calculations cannot certify comfort or environmental safety.
@@ -0,0 +1,56 @@
1
+ # Project discovery, implementation and audit
2
+
3
+ **Read this when** starting a Quest project, entering an unfamiliar repository, planning a vertical slice or deciding which engine/platform procedure to use.
4
+
5
+ Checked 2026-09-21. Procedures below are this pack's engineering contract; linked platform behavior comes from the named primary sources. A source audit is useful without an engine, MCP or headset; label unavailable checks NOT-RUN.
6
+
7
+ ## Establish the target before choosing libraries
8
+
9
+ Record in the consuming project's existing architecture/evidence location:
10
+
11
+ | Field | Inspect or decide |
12
+ |---|---|
13
+ | Deliverable | Standalone Horizon OS APK, PC OpenXR executable over Link, browser WebXR, Android panel/hybrid, or a separate Horizon Worlds experience |
14
+ | Source | Commit, dirty changes, project files, build scripts, asset import configuration and dependency locks |
15
+ | Engine | Native/Spatial/Unity/Unreal/Godot version, renderer, plugin/SDK versions and matching build/export tools |
16
+ | Runtime | Device and Horizon OS, OpenXR runtime/loader, refresh rate, supported capabilities, permissions and account state |
17
+ | Product | VR/MR mode, seated/standing/roomscale, input options, offline behavior, save/account/network needs |
18
+ | Delivery | Package ID, signing owner, app ID/creation date, channel, supported devices and asset target profile |
19
+
20
+ Do not use “Oculus” alone as a build target. Legacy Rift/PC or Go tutorials, standalone Quest and Meta's current Android surface have different rendering, lifecycle and distribution paths. Link testing is not a standalone APK test. Android API level is not a Horizon OS feature version.
21
+
22
+ ## Engine handoff
23
+
24
+ | Evidence | Procedure and primary entry |
25
+ |---|---|
26
+ | CMake/NDK/NativeActivity/OpenXR | This skill; [Meta samples](https://github.com/meta-quest/Meta-OpenXR-SDK) and [Khronos SDK source](https://github.com/KhronosGroup/OpenXR-SDK-Source) |
27
+ | Kotlin + `com.meta.spatial` | `quest-spatial`; [Spatial samples](https://github.com/meta-quest/Meta-Spatial-SDK-Samples) |
28
+ | Unity ProjectSettings/Packages/Assets | Discover available `hz-unity-*` skills; otherwise [Unity getting started](https://developers.meta.com/horizon/documentation/unity/unity-tutorial-hello-vr/), locked packages, scene/prefab inspection, EditMode/PlayMode and Android build |
29
+ | Unreal `.uproject`/Plugins/Source | [Meta compatibility matrix](https://developers.meta.com/horizon/documentation/unreal/unreal-compatibility-matrix/); choose Epic+plugin versus Meta fork deliberately; inspect assets/Blueprint/C++, cook/package and device render path |
30
+ | Godot `project.godot`/scenes/resources | [Godot XR setup](https://docs.godotengine.org/en/stable/tutorials/xr/setting_up_xr.html) and [Android export](https://docs.godotengine.org/en/stable/tutorials/xr/deploying_to_android.html); match engine/templates/vendor extension and renderer |
31
+ | Browser application | `quest-webxr`; native OpenXR extensions are not browser APIs |
32
+
33
+ Use engine companions if installed; never invent their tools. This platform skill supplies requirements and evidence, not Unity APIs inside a C++ project. For Godot, the setup/deployment pages currently disagree on Mobile versus Compatibility recommendations for Quest. Pin a release and test the required renderer/extensions on target hardware before choosing. Native engine compilation is needed only for an actual engine modification or custom template requirement.
34
+
35
+ ## Build a representative vertical slice
36
+
37
+ 1. Preserve the existing project. Reproduce its current build before scaffolding missing pieces.
38
+ 2. Implement launch → tracked view/input → one real interaction → save/resume → exit. Add one MR capability only if it is required by the brief.
39
+ 3. Import one representative source asset; check scale, axes, pivot, collision, material extensions, texture formats, skin weights and animation. Keep the editable master and import settings.
40
+ 4. Keep gameplay simulation separate from rendering and asynchronous I/O. Budget physics, AI, navigation, animation, audio and asset loading as well as draw work. Avoid per-frame allocations and unbounded event subscriptions.
41
+ 5. Exercise focus loss, permission refusal, tracking loss, input-source changes, background/resume and a second entitled account where applicable. Reset global/static state between sessions.
42
+ 6. Produce the actual target build, install/cold-start it and capture representative sustained behavior. A successful editor, simulator or headless import check proves only that surface.
43
+
44
+ ## Audit output
45
+
46
+ For each finding record severity, source location, observed/expected behavior, reproduction, affected targets, bounded fix and verification. Separate measured defect, source-risk, documented upstream limitation and NOT-RUN. Do not award a whole-project score from file names or a single screenshot.
47
+
48
+ Suggested evidence files, only if the project has no equivalent: `docs/xr/target.md`, `docs/xr/capabilities.json`, `docs/xr/verification.md`. Do not create a competing registry when the project's existing plan/scenario/evidence owners already hold these facts.
49
+
50
+ ## Runtime AI is a separate latency and data path
51
+
52
+ Generation servers such as [vLLM-Omni](https://github.com/vllm-project/vllm-omni) can supply prepared speech/images/video or asynchronous NPC services. They do not provide collision-ready geometry or a replacement for the OpenXR frame loop. Route production assets through the configured asset/job owner, such as Asset Foundry when present; use delivered manifests and target gates.
53
+
54
+ Keep requests asynchronous, bound the queue and payload, cancel/ignore obsolete results using scene/session IDs, and cache approved fallbacks. Continue tracking/rendering and essential interactions when offline. Benchmark network/queue/inference/audio-playout separately from frame time; no universal latency claim follows from “streaming.” For camera/microphone input, apply the MR permission/data contract before any off-device transfer.
55
+
56
+ For small on-device CV, evaluate [Meta's Unity Inference Engine sample](https://developers.meta.com/horizon/documentation/unity/unity-pca-sentis/) against the exact model/operators and headset GPU budget. A model loading successfully does not prove it coexists with 90 Hz rendering.
@@ -0,0 +1,128 @@
1
+ ---
2
+ name: quest-perf
3
+ description: >-
4
+ Use when a Meta Quest / Horizon OS app misses frames, judders, overheats or
5
+ must be profiled — the frame budget per refresh rate, logcat stats, the GPU%
6
+ number that lies during frame drops, Dynamic Resolution, foveation,
7
+ Application SpaceWarp, and capturing with OVR Metrics, Perfetto, simpleperf or
8
+ RenderDoc. Engine-independent. Triggers - "dropped frames" / "просадка кадров", "judder" /
9
+ "джаддер", "Quest performance" / "производительность на Quest", "GPU bound" /
10
+ "упёрлись в GPU", "profile the headset" / "профилировать шлем", "perfetto trace"
11
+ / "трейс perfetto", "simpleperf", "RenderDoc", "OVR Metrics", "stale frames",
12
+ "SpaceWarp", "foveated rendering" / "фовеальный рендеринг", "72 vs 90 Hz",
13
+ "app is hot" / "шлем греется". NOT for wiring the render loop itself
14
+ (quest-native), not for Store performance VRCs as a submission gate
15
+ (quest-store), and not for desktop or mobile-phone profiling.
16
+ license: MIT
17
+ compatibility: Any agent can read this workflow. Live source checks need network; build, device, profiling and Store actions need the named installed tools and accounts. Missing capabilities use the inline fallback and leave dependent checks unverified.
18
+ metadata:
19
+ version: "0.3.0"
20
+ ---
21
+
22
+ # Quest performance: the budget, the lie, and the capture
23
+
24
+ Performance work on Horizon OS fails in one of two ways: optimising without a
25
+ capture, or reading the capture wrong. This skill fixes the second first, because
26
+ the most-quoted number on the platform — `GPU%` — is misleading exactly when it
27
+ matters.
28
+
29
+ For a whole-product roadmap or stage audit, use `quest-lifecycle`; a single technical task stays with this owner. If absent, identify the current stage, its evidence and the next prerequisite inline.
30
+
31
+ Read `references/mobile-rendering.md` when choosing the render pipeline or investigating tile, render-pass, geometry, overdraw and asset costs.
32
+
33
+ ## The budget is a deadline, not a target
34
+
35
+ | Refresh rate | Time per frame |
36
+ |---|---|
37
+ | 72 Hz | 13.9 ms |
38
+ | 90 Hz | 11.1 ms |
39
+ | 120 Hz | 8.3 ms |
40
+
41
+ CPU, render-thread, GPU and compositor work form a pipeline and can overlap
42
+ across frames. Compare the critical path and synchronization against the
43
+ deadline; do not add independent CPU/GPU samples as a universal frame-time
44
+ formula. Missed submissions may invoke reprojection, with visible artifacts
45
+ that depend on motion, depth and the selected runtime path.
46
+
47
+ ## The GPU% lie, and the 50% threshold
48
+
49
+ When an app drops to half rate the GPU finishes early and idles, so utilisation
50
+ *falls* while the app is firmly GPU-bound.
51
+
52
+ ```text
53
+ FPS=36/72 Stale=36 GPU%=0.65 App=18.05ms
54
+ ```
55
+
56
+ - `FPS=36/72` — half rate: every frame shown twice.
57
+ - `App=18.05ms` — the real per-frame GPU time, against a 13.9 ms budget.
58
+ - `GPU%=0.65` — measured across the doubled 27.8 ms interval, which is why it
59
+ reads 65% while the app needs a **23% cut**, not the 35% "headroom" it implies.
60
+
61
+ **Rule: read `App` in milliseconds against the refresh-rate budget. Only trust
62
+ `GPU%` near saturation.** To climb back out of half rate, GPU work must fit one
63
+ refresh interval — i.e. utilisation must fall **below ~50%** while at half rate.
64
+
65
+ If `App` is inside budget and `Stale` remains non-zero, inspect CPU submission,
66
+ synchronization and compositor timing in a trace. That symptom alone does not
67
+ prove a CPU bottleneck. The worked half-rate example above assumes the stated
68
+ metric definitions and no intentional SpaceWarp mode; verify both first.
69
+
70
+ ## Select a lever from the measured bottleneck
71
+
72
+ Read `references/rendering-playbook.md` when choosing multiview, render scale,
73
+ foveation, layers, assets, shaders or reprojection. It links each experiment to
74
+ its prerequisites, visual failure cases and target-device evidence.
75
+
76
+ 1. **Dynamic Resolution** — tune render scale through the engine/SDK's supported
77
+ control path; verify actual behavior rather than assuming the OS enables it.
78
+ 2. **Foveation** — distinguish fixed from eye-tracked; test supported levels and
79
+ quality. A simple shader can cost more with FFR than without it.
80
+ 3. **Refresh rate** — request a rate the representative workload sustains; observe
81
+ actual runtime and thermal behavior rather than promising an automatic rate change.
82
+ 4. **Application SpaceWarp** — a supported depth/motion-vector integration with
83
+ artifact and latency tests, not an automatic rescue or guaranteed percentage gain.
84
+
85
+ ## Capture chain — what each tool answers
86
+
87
+ | Question | Tool | How |
88
+ |---|---|---|
89
+ | Am I missing frames at all, and where | logcat VrApi stats (`FPS`, `Stale`, `App`, `GPU%`) | `metavr log` / `adb logcat` |
90
+ | Live numbers inside the headset | OVR Metrics Tool (`ovrmetric` APK) | `metavr tools install ovrmetric`, then enable its overlay |
91
+ | CPU vs GPU, thread timeline, stalls | Perfetto | `metavr` MCP `start_perfetto_capture` → `stop_perfetto_capture` → `analyze_trace`, or the CLI equivalents |
92
+ | Which C/C++ functions burn the CPU | simpleperf | Android NDK's simpleperf against the running package |
93
+ | Which draw call, shader or render stage burns the GPU | RenderDoc Meta fork | `metavr tools install renderdoc`; draw-call trace, render-stage trace, Vulkan shader stats |
94
+
95
+ Meta ships task skills for two of these — `hz-perfetto-debug` and
96
+ `hz-simpleperf-debug`, plus `hz-vr-debug` for general on-device debugging. When
97
+ they are installed, this skill decides *what to measure* and they drive the tool.
98
+
99
+ `references/capture-playbook.md` holds the exact sequences, including what to
100
+ capture for a report that another agent can act on.
101
+
102
+ ## Method, so a second capture means something
103
+
104
+ 1. Reproduce on a **release build on a real headset**. A simulator run
105
+ (`xrsim`, `spatialsim`) checks supported simulated behavior, not headset performance.
106
+ 2. Note refresh rate and the resulting budget before looking at anything else.
107
+ 3. Capture with the app in the *worst* scene, not the menu.
108
+ 4. Change **one** thing; re-capture; compare `App` ms, not impressions.
109
+ 5. Record the pair (before/after `App`, `Stale`, scene, build id) in the
110
+ project's own notes — a performance claim with no capture is an opinion.
111
+
112
+ ## Gotchas that cost a debugging round
113
+
114
+ - **A thermally throttled headset invalidates the comparison.** Long sessions
115
+ drift; let it cool or state the temperature state with the number.
116
+ - **Dev-mode overlays and logging cost frames.** Measure with them off, then
117
+ turn them on only to read the numbers you cannot get otherwise.
118
+ - **Counter semantics depend on capture/runtime mode.** Inspect timestamps and
119
+ reprojection state before interpreting a stale-frame count as dropped work
120
+ or pipeline latency.
121
+ - **Draw calls are a CPU cost and fill is a GPU cost.** Cutting draw calls on a
122
+ fill-bound frame moves nothing, which is why the capture comes first.
123
+ - **WebXR has its own counters** — draw-call metrics in the browser docs; see
124
+ `quest-webxr` before applying native advice to a web build.
125
+
126
+ *Frame budgets, the 65%/18.05 ms worked example and the recovery mechanisms were
127
+ read from `documentation/native/android/os-missed-frames.md` on 2026-09-20;
128
+ re-verify with `metavr docs search "missed frames"`.*
@@ -0,0 +1,102 @@
1
+ # Capture playbook — the exact sequences
2
+
3
+ **Read this when** a performance problem needs evidence rather than an opinion,
4
+ or when a capture has to be handed to another agent or another person.
5
+
6
+ Commands verified against `metavr` 1.3.2.2.2 on 2026-09-20 (`metavr perf --help`
7
+ and its subcommands). Every one of them needs a **connected headset**; a
8
+ simulator answers behaviour questions only.
9
+
10
+ ## Contents
11
+
12
+ - Step 0: the numbers before any tool
13
+ - Perfetto through metavr
14
+ - simpleperf: CPU hotspots and what is actually bound
15
+ - RenderDoc: the GPU side
16
+ - OVR Metrics Tool: live, in the headset
17
+ - What a usable capture report contains
18
+
19
+ ## Step 0: the numbers before any tool
20
+
21
+ ```bash
22
+ metavr device list # the headset is attached at all
23
+ metavr device battery # a throttling device invalidates the run
24
+ metavr app launch <package>
25
+ metavr log | grep -E 'FPS=|Stale=|App='
26
+ ```
27
+
28
+ Read `App` (ms) against the refresh-rate budget, `Stale` against the refresh
29
+ rate, and only then `GPU%`. The interpretation — including why `GPU%` reads low
30
+ exactly when the app is GPU-bound — is in the skill body.
31
+
32
+ ## Perfetto through metavr
33
+
34
+ ```bash
35
+ # one-shot, fixed duration
36
+ metavr perf capture --mode vr --duration 10000 --app <package> -o before
37
+
38
+ # or bracket an interaction by hand
39
+ metavr perf start --mode custom --gpu-render-stage --cpu-scheduling --xr-runtime
40
+ # … reproduce the worst scene in the headset …
41
+ metavr perf stop
42
+
43
+ metavr perf analyze-trace --focus frames # overview | gpu | cpu | frames | threads
44
+ metavr perf compare before after # a delta report, after one change
45
+ metavr perf open # ui.perfetto.dev, for a human
46
+ metavr perf query "<SQL>" # trace processor, for a specific question
47
+ ```
48
+
49
+ Modes: `standard`, `gpu`, `cpu`, `lightweight`, `full`, `vr`, `custom`. Start
50
+ with `vr`; go to `custom` when a specific data source is missing. The `--gpu-*`,
51
+ `--cpu-scheduling`, `--xr-runtime` and `--vulkan-layer` switches only apply to
52
+ `custom`.
53
+
54
+ Meta's `hz-perfetto-debug` skill drives this tool in depth — use it once the
55
+ capture exists.
56
+
57
+ ## simpleperf: CPU hotspots and what is actually bound
58
+
59
+ ```bash
60
+ metavr perf simpleperf classify # CPU-bound, memory-bound or I/O-bound, from PMU counters
61
+ metavr perf simpleperf record # cycle-sampled hotspots
62
+ metavr perf simpleperf kernel-overhead # kernel vs userspace, per thread
63
+ ```
64
+
65
+ `classify` first: optimising a memory-bound loop as if it were compute is the
66
+ most common wasted day. Meta's `hz-simpleperf-debug` covers the deeper flags.
67
+
68
+ ## RenderDoc: the GPU side
69
+
70
+ `metavr tools install renderdoc` (Meta's fork). Three captures answer three
71
+ different questions:
72
+
73
+ | Question | Capture |
74
+ |---|---|
75
+ | Which draw call costs what | draw-call trace |
76
+ | Which render stage dominates | render-stage trace |
77
+ | Is a shader the problem | Vulkan shader stats |
78
+
79
+ Docs: `ts-renderdoc-for-oculus`, `ts-renderdoc-capture`, `ts-renderdoc-drawcall`,
80
+ `ts-renderdoc-renderstage`, `ts-renderdoc-shaderstats`.
81
+
82
+ ## OVR Metrics Tool: live, in the headset
83
+
84
+ `metavr tools install ovrmetric` installs the OVRMonitorMetricsService APK;
85
+ enable its overlay on the device to watch FPS, stale frames and utilisation
86
+ while playing. Best for "does this scene ever recover", worst for precise
87
+ numbers — the overlay itself costs a little.
88
+
89
+ ## What a usable capture report contains
90
+
91
+ A report another agent can act on names all seven:
92
+
93
+ 1. build id and whether it is a **release** build;
94
+ 2. device model and Horizon OS version;
95
+ 3. requested refresh rate, and therefore the budget in ms;
96
+ 4. the scene and the interaction reproduced;
97
+ 5. `App` ms, `Stale`, `FPS=x/y`, `GPU%` — as read, not summarised;
98
+ 6. battery/thermal state at capture time;
99
+ 7. the trace file path, so the claim can be re-checked.
100
+
101
+ A performance claim missing (1), (3) or (7) cannot be verified later and will
102
+ be re-measured from scratch.
@@ -0,0 +1,56 @@
1
+ # Mobile rendering: experiments before prescriptions
2
+
3
+ **Read this when**: choosing an art/render pipeline or diagnosing costs that desktop intuition
4
+ misses. Sources checked 2026-09-21; no fixed scene-wide triangle/draw-call/texture
5
+ budget applies to every device, content type or renderer.
6
+
7
+ ## Establish the workload
8
+
9
+ Record device/OS, graphics API, render pipeline, stereo mode, refresh target,
10
+ resolution, FFR/ETFR, MSAA, layers, lighting, animation/skinning, active cameras,
11
+ content and thermal state. Capture representative CPU/GPU/frame/memory baselines.
12
+ Use the same view and input sequence before/after; compare visual quality too.
13
+
14
+ ## Tile-based GPU decisions
15
+
16
+ - Render-pass transitions and intermediate targets can cause costly stores,
17
+ resolves and reads from shared memory. Inventory postprocessing, camera stacks,
18
+ portals/reflections and copies before reducing asset quality indiscriminately.
19
+ - Test pass fusion/subpass or direct rendering where supported by the actual
20
+ engine/backend. FFR/MSAA behavior depends on the target path; a screenshot of
21
+ an enabled setting does not prove it applies to the final rendering.
22
+ - MSAA can be relatively efficient in tile memory, but higher sample counts still
23
+ cost memory/bandwidth and interactions matter. Benchmark against the chosen
24
+ resolution and shader workload rather than blanket-disable or max it.
25
+ - Expensive position/vertex work and triangles crossing many tiles affect binning.
26
+ Inspect RenderDoc Meta Fork tile/render-stage evidence when available. Do not
27
+ assume tile dimensions or quote an unverified per-triangle execution formula.
28
+ - Compare transparent/alpha-blended overdraw, particle layers and screen coverage;
29
+ reduce what profiling implicates. Measure baked versus realtime lighting,
30
+ shadows, shader variants and material complexity on the target.
31
+ - For assets, measure compressed and resident textures, mip use, mesh/skin/animation
32
+ cost, LOD transitions, collisions, loading/upload stalls and memory peaks.
33
+ Normal maps or impostors trade geometry for other costs; verify the trade.
34
+
35
+ ## Advanced experiments and stop rules
36
+
37
+ Use the existing rendering playbook for multiview, foveation, dynamic resolution,
38
+ composition layers, symmetric projection, late latching and SpaceWarp. Every
39
+ proposal states prerequisite, expected bottleneck, quality risk, capture and
40
+ revert criterion. Do not enable multiple techniques then attribute a gain to one.
41
+ Low app GPU utilization alone is not spare frame budget; distinguish app render,
42
+ compositor, CPU scheduling and thermal behavior.
43
+
44
+ Keep spectator/capture cameras out of normal gameplay budgets unless shipped.
45
+ Profile representative sustained use and transitions, not just an empty scene.
46
+ A simulator can exercise logic/API availability; it cannot certify headset GPU,
47
+ comfort, thermal or Store performance. Never claim a fixed sum of CPU and GPU
48
+ intervals when stages overlap.
49
+
50
+ ## Primary reading
51
+
52
+ - [Tile rendering](https://developers.meta.com/horizon/documentation/unity/gpu-tiled/): architecture and binning.
53
+ - [Improved algorithms](https://developers.meta.com/horizon/documentation/unity/gpu-improved-algorithms/): mobile-friendly tradeoffs.
54
+ - [Impaired algorithms](https://developers.meta.com/horizon/documentation/unity/gpu-impaired-algorithms/): pass/resolve and vertex costs; verify its unresolved formula separately.
55
+ - [Graphics pipeline](https://developers.meta.com/horizon/design/design-graphic-rendering-pipeline/): vocabulary, not a device feature support matrix.
56
+ - [Display](https://developers.meta.com/horizon/design/display/): stereo, color and capture presentation.
@@ -0,0 +1,51 @@
1
+ # Rendering decisions and controlled experiments
2
+
3
+ **Read this when** choosing a render path, diagnosing an expensive scene, importing assets or integrating foveation, compositor layers, late latching or SpaceWarp.
4
+
5
+ Checked 2026-09-21. The tests below are engineering guidance. Use the engine/SDK's versioned documentation for switches; a native Vulkan extension name is not a Unity setting.
6
+
7
+ ## Capture the right workload
8
+
9
+ Record build hash, engine/packages, graphics API, device/OS, refresh rate, render scale, CPU/render-thread/GPU times, waits, dropped frames, memory and sustained thermal state. Include scene/input, warmup and capture overhead. Compare the same workload before/after; average FPS hides spikes and half-rate operation.
10
+
11
+ CPU and GPU can process different frames concurrently. Diagnose the critical path and deadline, not a sum of independent durations. GPU idle time may be waiting for CPU submission; a CPU wait may be waiting for the GPU. Compositor scheduling, sync and asset uploads can also produce missed frames. [Unity profiler markers](https://docs.unity3d.com/6000.0/Documentation/Manual/profiler-markers.html) documents these distinctions.
12
+
13
+ ## Select an experiment from the evidence
14
+
15
+ | Suspected cost | Controlled change | Check for damage |
16
+ |---|---|---|
17
+ | CPU draw submission | Batching/instancing, culling, compatible multiview | Correct per-eye transforms/shaders, visibility, materials and animation |
18
+ | Pixel/fragment cost | Reduce render scale, shader complexity, transparent coverage or FFR | Text/edges, central/peripheral quality, scene readability |
19
+ | Bandwidth/tile stores | Reduce intermediate targets, resolves and full-screen passes; compatible subsampled layout | Alpha, MSAA/depth resolves, post effects and capture |
20
+ | Geometry/animation | LOD, skin/bone budgets, animation update policy | Silhouette, deformation, popping and physics consistency |
21
+ | Memory/residency | Mips/compression, bounded streaming, release unused assets | Unsupported importer extensions, texture color space, transition hitching |
22
+ | CPU simulation/GC | Fixed-step budget, pooled allocations, bounded AI/pathfinding/I/O | Gameplay determinism, collision and response latency |
23
+ | Load/warmup | Async loading, shader/pipeline warmup, progressive scene entry | First-use spikes and OS-compliant loading feedback |
24
+
25
+ Quest's mobile/tile GPU behavior differs from desktop preview. Prefer the engine's supported mobile renderer; desktop Lumen/Nanite/post-processing assumptions are not standalone evidence. Consult [Meta Vulkan/OpenGL guidance](https://developers.meta.com/horizon/documentation/unreal/os-vulkan-opengl/) and the exact engine compatibility matrix rather than applying one graphics API rule to every engine.
26
+
27
+ ## Stereo, foveation and dynamic resolution
28
+
29
+ [Unreal Mobile Multi-View](https://developers.meta.com/horizon/documentation/unreal/unreal-multi-view/) reduces CPU stereo submission overhead and is a prerequisite there for SpaceWarp/late latching; it does not make pixel shading free. Check the equivalent Unity/native path, shaders and per-view resources.
30
+
31
+ [FFR](https://developers.meta.com/horizon/essentials/fixed-foveated-rendering/) is fixed-center and distinct from eye-tracked foveation. It helps appropriate fill-heavy workloads but can cost more than it saves for simple shaders. Test quality levels and compatible subsampled layout on each supported target. Query ETFR support rather than inferring it from a model name. Text that must remain sharp may belong on compositor layers.
32
+
33
+ Dynamic resolution changes render scale, not the simulation budget. Verify the selected engine/SDK's control path and interaction with manual scale, FFR and frame-rate policy. Record hysteresis/quality limits and sustained behavior. A feature being available does not mean it is already enabled by the OS.
34
+
35
+ [Symmetric projection](https://developers.meta.com/horizon/documentation/native/android/os-symmetric-projection/) changes eye frusta/render areas and can combine with supported Vulkan foveation/optimization paths. Preserve correct view/projection math and test clipping/per-eye edges. Evaluate it after a working baseline; do not hand-modify matrices merely because a tutorial reports a gain.
36
+
37
+ ## Layers, alpha and color
38
+
39
+ [Compositor layers](https://developers.meta.com/horizon/essentials/compositor-layers/) can improve text/UI/video clarity, but have a rendering, memory and per-layer cost. Choose overlay versus underlay deliberately; verify alpha semantics, depth/occlusion and transparent ordering. Do not keep unused zero-alpha layers alive as a free placeholder. Confirm linear/sRGB conversions, supported swapchain formats and the selected engine's premultiplication convention; compare a color/alpha test pattern in the headset.
40
+
41
+ For exported assets inspect post-import triangles/vertices, material slots/draw submissions, texture residency, mips, decoder support and bounds. A smaller compressed file may decode slowly or require an unavailable extension. Preserve masters, measure the scene after import and use the project's target profile instead of a universal polygon limit.
42
+
43
+ ## Reprojection and latency
44
+
45
+ TimeWarp, application SpaceWarp and application rendering are different stages. [Application SpaceWarp](https://developers.meta.com/horizon/documentation/unreal/unreal-asw/) requires a supported engine/runtime integration, including depth/motion data. Test thin geometry, disocclusion, transparency, particles, UI and controller motion. No advertised percentage is a guaranteed budget increase for this scene.
46
+
47
+ Late latching/prediction affects pose-to-display latency; it cannot repair an incorrect simulation, invalid pose or blocking network call. Verify engine support and input path. Retain full-rate rendering as a measured comparison when evaluating half-rate synthesis.
48
+
49
+ ## Deliver evidence, not a list of toggles
50
+
51
+ For each optimization retain settings diff, trace/capture, metric distributions, visual regressions and an explanation of the bottleneck addressed. Report a sustained device result or NOT-RUN; simulator/remote PC rendering is not a headset thermal test. Change one variable per experiment unless a documented dependency requires a group.