sceneview-mcp 3.2.0 → 3.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/dist/guides.js ADDED
@@ -0,0 +1,603 @@
1
+ /**
2
+ * guides.ts
3
+ *
4
+ * Static content for the platform roadmap, best practices, and AR setup tools.
5
+ */
6
+ // ─── Platform Roadmap ─────────────────────────────────────────────────────────
7
+ export const PLATFORM_ROADMAP = `# SceneView Multi-Platform Roadmap
8
+
9
+ ## Current Status (v3.3.0)
10
+
11
+ | Platform | Status | Artifact | Renderer |
12
+ |----------|--------|----------|----------|
13
+ | **Android (Compose)** | Stable | \`io.github.sceneview:sceneview:3.3.0\` | Filament |
14
+ | **Android (AR)** | Stable | \`io.github.sceneview:arsceneview:3.3.0\` | Filament + ARCore |
15
+ | **iOS (SwiftUI)** | Alpha | SceneViewSwift SPM \`from: "3.3.0"\` | RealityKit + ARKit |
16
+ | **macOS (SwiftUI)** | Alpha | SceneViewSwift SPM (in Package.swift) | RealityKit |
17
+ | **visionOS (SwiftUI)** | Alpha | SceneViewSwift SPM (in Package.swift) | RealityKit |
18
+ | **KMP Core** | Stable | \`io.github.sceneview:sceneview-core:3.3.0\` | N/A (shared logic) |
19
+
20
+ ## Architecture: Native Renderers per Platform
21
+
22
+ \`\`\`
23
+ Android: Filament (OpenGL ES / Vulkan)
24
+ Apple: RealityKit (Metal)
25
+ Shared: sceneview-core (KMP) — math, collision, geometry, animations, physics
26
+ \`\`\`
27
+
28
+ KMP shares **logic**, not **rendering**. Each platform uses its native renderer.
29
+
30
+ ## Android — What Ships Today
31
+
32
+ - **3D rendering** via Google Filament: PBR materials, HDR environments, glTF/GLB models, post-processing.
33
+ - **AR** via ARCore: plane detection, hit testing, anchors, cloud anchors, augmented images, depth, light estimation, point cloud.
34
+ - **Compose-native DSL**: all nodes are \`@Composable\` functions inside \`Scene { }\` or \`ARScene { }\`.
35
+ - **26+ node types**: ModelNode, LightNode, AnchorNode, CameraNode, TextNode, PathNode, ViewNode, PlaneNode, SphereNode, CylinderNode, CubeNode, DynamicSkyNode, FogNode, ReflectionProbeNode, PhysicsNode, BillboardNode, LineNode, and more.
36
+
37
+ ## Apple — Alpha (SceneViewSwift)
38
+
39
+ - **3D + AR** in SwiftUI via RealityKit — iOS 17+ / macOS 14+ / visionOS 1+
40
+ - Node types: ModelNode, AnchorNode, GeometryNode, LightNode, CameraNode, ImageNode, VideoNode, PhysicsNode, AugmentedImageNode
41
+ - PBR material system with textures
42
+ - Swift Package Manager distribution
43
+ - Consumable by: Swift native, Flutter (PlatformView), React Native (Fabric), KMP Compose (UIKitView)
44
+
45
+ ## KMP Core (\`sceneview-core\`)
46
+
47
+ Shared Kotlin Multiplatform module providing:
48
+ - Collision system (Ray, Box, Sphere, Intersections)
49
+ - Triangulation (Earcut, Delaunator)
50
+ - Geometry generation (Cube, Sphere, Cylinder, Plane, Path, Line, Shape)
51
+ - Animation (Spring, Property, Interpolation, SmoothTransform)
52
+ - Physics simulation
53
+ - Scene graph, math utilities
54
+
55
+ ## Upcoming
56
+
57
+ - **v3.4.0**: SceneViewSwift stabilization, API parity with Android core nodes
58
+ - **v3.5.0**: KMP core XCFramework consumption in SceneViewSwift
59
+ - **v4.0.0**: Android XR, visionOS spatial computing, cross-framework bridges (Flutter, React Native)
60
+
61
+ ## How to Stay Updated
62
+
63
+ - **GitHub:** https://github.com/SceneView/sceneview
64
+ - **Releases:** https://github.com/SceneView/sceneview/releases
65
+ - **Website:** https://sceneview.github.io
66
+ `;
67
+ // ─── Best Practices ───────────────────────────────────────────────────────────
68
+ const PERFORMANCE_PRACTICES = `## Performance Best Practices
69
+
70
+ ### Model Optimization
71
+ - **Use GLB over glTF** for production — GLB is a single binary file, faster to load than multi-file glTF.
72
+ - **Compress with Draco or Meshopt** — reduces model size by 70-90%. Filament supports both.
73
+ - **Limit polygon count** — aim for <100K triangles per model on mobile. Use LODs for complex scenes.
74
+ - **Texture size** — keep textures at 1024x1024 or smaller. Use KTX2 with Basis Universal compression.
75
+ - **Reduce draw calls** — merge meshes in your 3D editor. Each material = 1 draw call minimum.
76
+
77
+ ### Runtime Performance
78
+ - **Reuse engines and loaders** — always use \`rememberEngine()\`, \`rememberModelLoader(engine)\`, etc. Creating multiple engines wastes GPU memory.
79
+ - **Limit simultaneous model loads** — \`rememberModelInstance\` is async, but loading 10+ models simultaneously can spike memory.
80
+ - **Use \`scaleToUnits\`** — avoids runtime bounding-box computation when you know the desired size.
81
+ - **Avoid per-frame allocations** — in \`onFrame\` callbacks, reuse objects instead of creating new Position/Rotation instances.
82
+ - **Profile with Android GPU Inspector** — Filament renders via OpenGL ES / Vulkan; AGI shows exactly where GPU time goes.
83
+
84
+ ### Environment & Lighting
85
+ - **Use small HDR environments** — \`sky_2k.hdr\` (2K) is usually sufficient. 4K+ HDRs waste GPU memory on mobile.
86
+ - **Prefer \`ENVIRONMENTAL_HDR\`** for AR — gives the most realistic lighting but costs more. Use \`DISABLED\` if not needed.
87
+ - **Limit shadow-casting lights** — each shadow-casting light adds a depth render pass.
88
+ `;
89
+ const ARCHITECTURE_PRACTICES = `## Architecture Best Practices
90
+
91
+ ### Compose Integration
92
+ - **Treat Scene/ARScene like any Compose layout** — it participates in the Compose lifecycle. Don't fight it with imperative code.
93
+ - **State hoisting** — hoist anchor state, model selection, and UI state to the parent composable. The Scene should be a pure renderer.
94
+ - **ViewModel for business logic** — keep AR session state (anchors, detected images) in a ViewModel. Pass it down to the Scene composable.
95
+ - **Side effects** — use \`LaunchedEffect\` and \`DisposableEffect\` for async operations, not raw coroutines in composables.
96
+
97
+ ### Project Structure
98
+ \`\`\`
99
+ app/
100
+ src/main/
101
+ assets/
102
+ models/ # GLB/glTF files
103
+ environments/ # HDR environment maps
104
+ materials/ # Custom .filamat materials
105
+ kotlin/
106
+ ui/
107
+ scene/
108
+ SceneScreen.kt # Compose screen with Scene { }
109
+ SceneViewModel.kt # State management
110
+ ar/
111
+ ARScreen.kt # Compose screen with ARScene { }
112
+ ARViewModel.kt # Anchor and session state
113
+ \`\`\`
114
+
115
+ ### Error Handling
116
+ - **Always null-check \`rememberModelInstance\`** — it returns \`null\` while loading and if the asset fails to load.
117
+ - **Show loading indicators** — wrap the Scene with a loading overlay keyed on the model instance being null.
118
+ - **Handle AR session failures** — ARCore may not be installed or the device may not support AR. Check \`ArCoreApk.getInstance().checkAvailability(context)\`.
119
+ `;
120
+ const MEMORY_PRACTICES = `## Memory Management Best Practices
121
+
122
+ ### Lifecycle Rules
123
+ - **Never call \`engine.destroy()\` manually** when using \`rememberEngine()\` — it handles cleanup on composition disposal.
124
+ - **Never call \`destroy()\` on nodes** created as composables — the composition lifecycle manages them.
125
+ - **Destroy order matters** for imperative code: MaterialInstance first, then Texture, then Engine. Reversing this causes SIGABRT.
126
+
127
+ ### Reducing Memory Usage
128
+ - **Release unused model instances** — if you swap models, the old instance is released when recomposition removes its \`ModelNode\`.
129
+ - **Limit concurrent scenes** — each \`Scene\` composable creates its own Filament View and Renderer. Avoid multiple visible scenes.
130
+ - **Watch for Bitmap leaks** — if loading textures from Bitmaps, ensure they are recycled after Filament consumes them.
131
+ - **Use \`rememberEnvironment\`** — it caches the HDR environment and releases it on disposal. Don't load environments in \`LaunchedEffect\`.
132
+ `;
133
+ const THREADING_PRACTICES = `## Threading Best Practices
134
+
135
+ ### Swift / RealityKit Threading Rules
136
+
137
+ **RealityKit is \`@MainActor\`-bound.** All entity mutations must happen on the main thread.
138
+
139
+ #### Safe Patterns (Swift)
140
+ - **In SwiftUI:** Use \`.task { }\` — it is \`@MainActor\`-isolated by default.
141
+ - **Standalone functions:** Annotate with \`@MainActor\` when modifying entities.
142
+ - **Model loading:** \`ModelNode.load()\` is \`async throws\` — call it with \`try await\` inside a \`.task { }\` block.
143
+
144
+ #### Anti-Patterns (Swift)
145
+ \`\`\`swift
146
+ // WRONG — detached task is not @MainActor
147
+ Task.detached {
148
+ let model = try await ModelNode.load("car.usdz")
149
+ model.entity.position = .init(x: 0, y: 1, z: 0) // Race condition!
150
+ }
151
+
152
+ // CORRECT — .task { } is @MainActor in SwiftUI
153
+ .task {
154
+ do {
155
+ let model = try await ModelNode.load("car.usdz")
156
+ model.entity.position = .init(x: 0, y: 1, z: 0) // Safe — main actor
157
+ } catch {
158
+ print("Load failed: \\(error)")
159
+ }
160
+ }
161
+ \`\`\`
162
+
163
+ ---
164
+
165
+ ### Android / Filament Threading Rules
166
+
167
+ ### The Golden Rule
168
+ **All Filament JNI calls must run on the main thread.** This includes:
169
+ - \`modelLoader.createModelInstance()\`
170
+ - \`materialLoader.createMaterial()\`
171
+ - \`engine.createTexture()\`
172
+ - \`Texture.Builder().build(engine)\`
173
+ - Any \`engine.*\` call
174
+
175
+ ### Safe Patterns
176
+ - **In composables:** Use \`rememberModelInstance(modelLoader, path)\` — it handles threading internally.
177
+ - **In ViewModels:** Use \`modelLoader.loadModelInstanceAsync(path)\` — it posts the JNI call to the main thread.
178
+ - **Manual threading:** Wrap in \`withContext(Dispatchers.Main) { }\`.
179
+
180
+ ### Anti-Patterns
181
+ \`\`\`kotlin
182
+ // WRONG — Filament JNI call on IO thread → crash
183
+ viewModelScope.launch(Dispatchers.IO) {
184
+ val model = modelLoader.createModelInstance(buffer) // SIGABRT!
185
+ }
186
+
187
+ // CORRECT — load data on IO, create model on Main
188
+ viewModelScope.launch(Dispatchers.IO) {
189
+ val buffer = loadFromNetwork(url)
190
+ withContext(Dispatchers.Main) {
191
+ val model = modelLoader.createModelInstance(buffer) // Safe
192
+ }
193
+ }
194
+ \`\`\`
195
+
196
+ ### ARCore Threading
197
+ - \`Session.update()\` must run on the main thread (SceneView handles this via \`onSessionUpdated\`).
198
+ - \`hitResult.createAnchor()\` is safe on the main thread (called from \`onTouchEvent\` which runs on main).
199
+ - Cloud anchor hosting/resolving is async — SceneView's \`CloudAnchorNode\` handles the callbacks correctly.
200
+ `;
201
+ export const BEST_PRACTICES = {
202
+ all: [
203
+ "# SceneView Best Practices\n",
204
+ PERFORMANCE_PRACTICES,
205
+ ARCHITECTURE_PRACTICES,
206
+ MEMORY_PRACTICES,
207
+ THREADING_PRACTICES,
208
+ ].join("\n---\n\n"),
209
+ performance: `# SceneView Best Practices\n\n${PERFORMANCE_PRACTICES}`,
210
+ architecture: `# SceneView Best Practices\n\n${ARCHITECTURE_PRACTICES}`,
211
+ memory: `# SceneView Best Practices\n\n${MEMORY_PRACTICES}`,
212
+ threading: `# SceneView Best Practices\n\n${THREADING_PRACTICES}`,
213
+ };
214
+ // ─── AR Setup Guide ───────────────────────────────────────────────────────────
215
+ // ─── Troubleshooting Guide ────────────────────────────────────────────────────
216
+ export const TROUBLESHOOTING_GUIDE = `# SceneView Troubleshooting Guide
217
+
218
+ ## Common Crashes
219
+
220
+ ### SIGABRT / Native Crash on Destroy
221
+ **Cause:** Destroying Filament resources in wrong order or from wrong thread.
222
+ **Fix:**
223
+ - Never call \`engine.destroy()\` manually when using \`rememberEngine()\`
224
+ - Never call \`node.destroy()\` on composable nodes — Compose handles lifecycle
225
+ - If using imperative API: destroy MaterialInstance before Texture, then Engine last
226
+ - All Filament calls must be on the main thread
227
+
228
+ ### Model Not Showing / Invisible
229
+ **Check list:**
230
+ 1. \`rememberModelInstance\` returns null while loading — are you null-checking it?
231
+ 2. Is the model path correct? Assets go in \`src/main/assets/models/\`
232
+ 3. Is \`scaleToUnits\` too small or too large? Try \`1.0f\`
233
+ 4. Is there a light in the scene? Add: \`LightNode(type = LightNode.Type.DIRECTIONAL)\`
234
+ 5. Is the camera pointing at the model? Default camera is at origin looking -Z
235
+
236
+ ### AR Camera Feed Shows but No Planes Detected
237
+ **Fix:**
238
+ - Ensure camera permission is granted at runtime (not just in manifest)
239
+ - Point the device at a textured surface (plain white walls are hard to track)
240
+ - Check \`planeFindingMode\` is not \`DISABLED\`
241
+ - Wait 2-3 seconds for ARCore to initialize tracking
242
+
243
+ ### "ARCore not installed" on Device
244
+ **Fix:**
245
+ - Ensure \`<meta-data android:name="com.google.ar.core" android:value="required" />\` is in manifest
246
+ - Install Google Play Services for AR from Play Store
247
+ - Some emulators don't support ARCore — test on a real device
248
+
249
+ ### Build Fails: "Cannot find Filament material"
250
+ **Fix:**
251
+ - Pre-compiled materials are in \`src/main/assets/materials/\`
252
+ - Don't delete assets when cleaning — they're checked into the repository
253
+ - If \`filamentPluginEnabled=true\` in gradle.properties, you need the Filament desktop tools installed
254
+
255
+ ### Gradle Sync Fails
256
+ **Fix:**
257
+ - Ensure Java 17 is set: \`compileOptions { sourceCompatibility = JavaVersion.VERSION_17 }\`
258
+ - Use compatible AGP/Gradle versions (AGP 8.11.1 + Gradle 8.11.1)
259
+ - Clear Gradle cache: \`./gradlew clean && rm -rf ~/.gradle/caches\`
260
+
261
+ ## Performance Issues
262
+
263
+ ### Low FPS / Jank
264
+ - Reduce model polygon count (< 100K triangles)
265
+ - Use KTX2 compressed textures (1024x1024 max)
266
+ - Avoid per-frame allocations in \`onFrame\` callbacks
267
+ - Disable post-processing if not needed: \`Scene(postProcessing = false)\`
268
+ - Profile with Android GPU Inspector
269
+
270
+ ### High Memory Usage
271
+ - Don't create multiple Engine instances — reuse \`rememberEngine()\`
272
+ - Limit concurrent model loads (max 3-4 simultaneously)
273
+ - Use smaller HDR environments (2K, not 4K)
274
+ - Release unused environments: let them leave composition
275
+
276
+ ## AR-Specific Issues
277
+
278
+ ### Anchor Drift / Objects Moving
279
+ - Use \`AnchorNode\` instead of setting \`worldPosition\` manually
280
+ - ARCore anchors are world-locked; plain nodes follow the coordinate system which shifts during tracking
281
+ - For persistent anchors across sessions, use \`CloudAnchorNode\`
282
+
283
+ ### AR Objects Too Bright / Overexposed
284
+ - Set \`toneMapper = ToneMapper.Linear\` on the AR scene view
285
+ - Default tone mapping enhances contrast which looks wrong on the camera feed
286
+
287
+ ### Augmented Images Not Detected
288
+ - Image must be >= 300x300 pixels with good contrast and detail
289
+ - Specify physical width when registering: \`addImage("name", bitmap, 0.15f)\` (meters)
290
+ - Only one image database per session — add all images before starting
291
+
292
+ ---
293
+
294
+ ## iOS-Specific Issues (SceneViewSwift)
295
+
296
+ ### Model Not Showing in SceneView
297
+ **Check list:**
298
+ 1. Is the model format USDZ or .reality? RealityKit does not support GLB/glTF natively.
299
+ 2. Is \`ModelNode.load()\` called with \`try await\`? It's async — if you forget \`await\`, the model won't load.
300
+ 3. Are you adding \`.entity\` (not the wrapper)? \`root.addChild(model.entity)\`, not \`root.addChild(model)\`.
301
+ 4. Check the model path — it must be in the app bundle (add to Xcode project, ensure "Copy Bundle Resources" includes it).
302
+ 5. Is the model too large or too small? Try \`.scaleToUnits(1.0)\`.
303
+
304
+ ### AR Camera Shows Black Screen (iOS)
305
+ **Fix:**
306
+ - Check Info.plist has \`NSCameraUsageDescription\` — without it the app crashes or shows a black screen.
307
+ - Ensure the device supports AR: \`ARWorldTrackingConfiguration.isSupported\`.
308
+ - Test on a real device — the simulator does not support ARKit camera.
309
+
310
+ ### ARSceneView Crash on macOS / visionOS
311
+ **Fix:**
312
+ - \`ARSceneView\` uses \`ARView\` which is iOS-only. Use \`SceneView\` for 3D on macOS/visionOS.
313
+ - For visionOS AR, use \`ARKitSession\` with \`RealityView\` directly.
314
+
315
+ ### Swift Concurrency Warnings
316
+ **Fix:**
317
+ - If you get "Non-sendable type" warnings, note that SceneViewSwift node types are marked \`@unchecked Sendable\` — RealityKit entities are main-actor-bound.
318
+ - Always load models inside \`.task { }\` (which is \`@MainActor\`) or annotate functions with \`@MainActor\`.
319
+
320
+ ### SPM Package Resolution Fails
321
+ **Fix:**
322
+ - Ensure Xcode 15.0+ (required for iOS 17 / visionOS targets).
323
+ - Clean derived data: Xcode → Product → Clean Build Folder, then File → Packages → Reset Package Caches.
324
+ - Check the URL is exactly: \`https://github.com/SceneView/sceneview\`
325
+
326
+ ### Image Tracking Not Working (iOS)
327
+ **Fix:**
328
+ - Reference images need high contrast and detail — avoid solid colors or simple patterns.
329
+ - Specify \`physicalWidth\` in meters when creating the reference image.
330
+ - Only one set of detection images per AR session configuration.
331
+ - The image must be physically present and visible to the camera — screenshots don't track well.
332
+ `;
333
+ export const AR_SETUP_GUIDE = `# SceneView AR — Complete Setup Guide (Android + iOS)
334
+
335
+ ---
336
+
337
+ # iOS AR Setup (ARKit + RealityKit)
338
+
339
+ ## 1. SPM Dependency
340
+
341
+ \`\`\`swift
342
+ .package(url: "https://github.com/SceneView/sceneview", from: "3.3.0")
343
+ \`\`\`
344
+
345
+ ## 2. Info.plist — Camera Permission
346
+
347
+ \`\`\`xml
348
+ <key>NSCameraUsageDescription</key>
349
+ <string>This app uses the camera for augmented reality.</string>
350
+ \`\`\`
351
+
352
+ Without this entry, the app will crash on camera access.
353
+
354
+ ## 3. Minimum Platform
355
+
356
+ AR requires **iOS 17.0+**. ARKit is not available on macOS or visionOS via \`ARSceneView\` (visionOS uses \`ARKitSession\` directly).
357
+
358
+ ## 4. Basic AR Template
359
+
360
+ \`\`\`swift
361
+ import SwiftUI
362
+ import SceneViewSwift
363
+ import RealityKit
364
+
365
+ struct MyARView: View {
366
+ @State private var model: ModelNode?
367
+
368
+ var body: some View {
369
+ ARSceneView(
370
+ planeDetection: .horizontal,
371
+ showCoachingOverlay: true,
372
+ onTapOnPlane: { position, arView in
373
+ guard let model else { return }
374
+ let anchor = AnchorNode.world(position: position)
375
+ anchor.add(model.entity.clone(recursive: true))
376
+ arView.scene.addAnchor(anchor.entity)
377
+ }
378
+ )
379
+ .edgesIgnoringSafeArea(.all)
380
+ .task {
381
+ model = try? await ModelNode.load("models/object.usdz")
382
+ model?.scaleToUnits(0.3)
383
+ }
384
+ }
385
+ }
386
+ \`\`\`
387
+
388
+ ## 5. AR Configuration Options
389
+
390
+ | Parameter | Options | Default |
391
+ |-----------|---------|---------|
392
+ | \`planeDetection\` | \`.none\`, \`.horizontal\`, \`.vertical\`, \`.both\` | \`.horizontal\` |
393
+ | \`showPlaneOverlay\` | \`true\` / \`false\` | \`true\` |
394
+ | \`showCoachingOverlay\` | \`true\` / \`false\` | \`true\` |
395
+ | \`imageTrackingDatabase\` | \`Set<ARReferenceImage>?\` | \`nil\` |
396
+
397
+ ## 6. Image Tracking (iOS)
398
+
399
+ \`\`\`swift
400
+ let images = AugmentedImageNode.createImageDatabase([
401
+ AugmentedImageNode.ReferenceImage(
402
+ name: "poster",
403
+ image: UIImage(named: "poster_ref")!,
404
+ physicalWidth: 0.3
405
+ )
406
+ ])
407
+
408
+ ARSceneView(
409
+ imageTrackingDatabase: images,
410
+ onImageDetected: { name, anchor, arView in
411
+ let cube = GeometryNode.cube(size: 0.1, color: .blue)
412
+ anchor.add(cube.entity)
413
+ arView.scene.addAnchor(anchor.entity)
414
+ }
415
+ )
416
+ \`\`\`
417
+
418
+ ---
419
+
420
+ # Android AR Setup (ARCore + Filament)
421
+
422
+ ## 1. Gradle Dependencies
423
+
424
+ \`\`\`kotlin
425
+ // build.gradle.kts (app module)
426
+ dependencies {
427
+ implementation("io.github.sceneview:arsceneview:3.3.0")
428
+ // arsceneview includes sceneview transitively — no need to add both
429
+ }
430
+ \`\`\`
431
+
432
+ ## 2. AndroidManifest.xml
433
+
434
+ \`\`\`xml
435
+ <manifest xmlns:android="http://schemas.android.com/apk/res/android">
436
+
437
+ <!-- Required: camera access for AR -->
438
+ <uses-permission android:name="android.permission.CAMERA" />
439
+
440
+ <!-- Required: declares AR hardware dependency -->
441
+ <uses-feature android:name="android.hardware.camera.ar" android:required="true" />
442
+
443
+ <!-- Optional: internet for cloud anchors -->
444
+ <uses-permission android:name="android.permission.INTERNET" />
445
+
446
+ <application>
447
+ <!-- Required: tells Play Store this app needs ARCore -->
448
+ <!-- Use "required" to block installs on non-AR devices -->
449
+ <!-- Use "optional" to allow install but degrade gracefully -->
450
+ <meta-data android:name="com.google.ar.core" android:value="required" />
451
+ </application>
452
+ </manifest>
453
+ \`\`\`
454
+
455
+ ### AR Required vs Optional
456
+
457
+ | Value | Behavior |
458
+ |-------|----------|
459
+ | \`"required"\` | App only visible in Play Store on ARCore-supported devices. ARCore auto-installed. |
460
+ | \`"optional"\` | App visible to all devices. You must check ARCore availability at runtime. |
461
+
462
+ For optional AR, check availability before showing AR features:
463
+ \`\`\`kotlin
464
+ val availability = ArCoreApk.getInstance().checkAvailability(context)
465
+ if (availability.isSupported) { /* show AR */ }
466
+ \`\`\`
467
+
468
+ ## 3. Session Configuration Options
469
+
470
+ \`\`\`kotlin
471
+ ARScene(
472
+ engine = engine,
473
+ modelLoader = modelLoader,
474
+ sessionConfiguration = { session, config ->
475
+
476
+ // ── Depth ─────────────────────────────────────────────
477
+ // Enables occlusion (virtual objects hidden behind real ones)
478
+ config.depthMode =
479
+ if (session.isDepthModeSupported(Config.DepthMode.AUTOMATIC))
480
+ Config.DepthMode.AUTOMATIC
481
+ else Config.DepthMode.DISABLED
482
+
483
+ // ── Light Estimation ──────────────────────────────────
484
+ // ENVIRONMENTAL_HDR: most realistic (reflects real lighting on models)
485
+ // AMBIENT_INTENSITY: lighter weight, just brightness + color temp
486
+ // DISABLED: no lighting adjustment
487
+ config.lightEstimationMode = Config.LightEstimationMode.ENVIRONMENTAL_HDR
488
+
489
+ // ── Plane Detection ───────────────────────────────────
490
+ // HORIZONTAL: floors, tables
491
+ // VERTICAL: walls
492
+ // HORIZONTAL_AND_VERTICAL: both
493
+ config.planeFindingMode = Config.PlaneFindingMode.HORIZONTAL_AND_VERTICAL
494
+
495
+ // ── Instant Placement ─────────────────────────────────
496
+ // Allows placing objects before planes are fully detected
497
+ config.instantPlacementMode = Config.InstantPlacementMode.LOCAL_Y_UP
498
+
499
+ // ── Cloud Anchors ─────────────────────────────────────
500
+ // Required for cross-device persistent anchors
501
+ config.cloudAnchorMode = Config.CloudAnchorMode.ENABLED
502
+
503
+ // ── Augmented Images ──────────────────────────────────
504
+ // Register reference images for detection
505
+ val imageDb = AugmentedImageDatabase(session)
506
+ imageDb.addImage("my-image", bitmap, 0.15f) // 15cm physical width
507
+ config.augmentedImageDatabase = imageDb
508
+
509
+ // ── Focus Mode ────────────────────────────────────────
510
+ config.focusMode = Config.FocusMode.AUTO
511
+ }
512
+ ) {
513
+ // AR content goes here
514
+ }
515
+ \`\`\`
516
+
517
+ ## 4. Common AR Patterns
518
+
519
+ ### Tap-to-Place with Anchor
520
+ \`\`\`kotlin
521
+ var anchor by remember { mutableStateOf<Anchor?>(null) }
522
+
523
+ ARScene(
524
+ engine = engine,
525
+ modelLoader = modelLoader,
526
+ planeRenderer = true,
527
+ onTouchEvent = { event, hitResult ->
528
+ if (event.action == MotionEvent.ACTION_UP && hitResult != null) {
529
+ anchor = hitResult.createAnchor()
530
+ }
531
+ true
532
+ }
533
+ ) {
534
+ anchor?.let { a ->
535
+ AnchorNode(anchor = a) {
536
+ modelInstance?.let { instance ->
537
+ ModelNode(
538
+ modelInstance = instance,
539
+ scaleToUnits = 0.5f,
540
+ isEditable = true // pinch-to-scale + drag-to-rotate
541
+ )
542
+ }
543
+ }
544
+ }
545
+ }
546
+ \`\`\`
547
+
548
+ ### Surface Cursor (HitResultNode)
549
+ \`\`\`kotlin
550
+ ARScene(
551
+ engine = engine,
552
+ modelLoader = modelLoader,
553
+ planeRenderer = true
554
+ ) {
555
+ HitResultNode(engine = engine) {
556
+ modelInstance?.let { instance ->
557
+ ModelNode(modelInstance = instance, scaleToUnits = 0.3f)
558
+ }
559
+ }
560
+ }
561
+ \`\`\`
562
+
563
+ ### Augmented Image Detection
564
+ \`\`\`kotlin
565
+ var images by remember { mutableStateOf<Map<String, AugmentedImage>>(emptyMap()) }
566
+
567
+ ARScene(
568
+ engine = engine,
569
+ sessionConfiguration = { session, config ->
570
+ config.addAugmentedImage(session, "poster", posterBitmap)
571
+ },
572
+ onSessionUpdated = { _, frame ->
573
+ frame.getUpdatedAugmentedImages().forEach { img ->
574
+ images = images + (img.name to img)
575
+ }
576
+ }
577
+ ) {
578
+ for ((_, image) in images) {
579
+ AugmentedImageNode(augmentedImage = image) {
580
+ ModelNode(modelInstance = instance, scaleToUnits = 0.1f)
581
+ }
582
+ }
583
+ }
584
+ \`\`\`
585
+
586
+ ## 5. Permissions
587
+
588
+ Camera permission must be requested at runtime (Android 6.0+). SceneView does **not** handle this — you must request it before showing ARScene:
589
+
590
+ \`\`\`kotlin
591
+ val cameraPermission = rememberLauncherForActivityResult(
592
+ ActivityResultContracts.RequestPermission()
593
+ ) { granted -> if (granted) showAR = true }
594
+
595
+ LaunchedEffect(Unit) {
596
+ cameraPermission.launch(Manifest.permission.CAMERA)
597
+ }
598
+
599
+ if (showAR) {
600
+ ARScene(engine = engine, modelLoader = modelLoader) { /* ... */ }
601
+ }
602
+ \`\`\`
603
+ `;