react-native-image-stitcher 0.16.0 → 0.16.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/CHANGELOG.md +47 -0
  2. package/README.md +25 -10
  3. package/android/src/main/cpp/image_stitcher_jni.cpp +52 -7
  4. package/android/src/main/cpp/keyframe_gate_jni.cpp +15 -8
  5. package/android/src/main/java/io/imagestitcher/rn/IncrementalStitcher.kt +135 -59
  6. package/android/src/main/java/io/imagestitcher/rn/RNSARCameraView.kt +7 -36
  7. package/cpp/keyframe_gate.cpp +54 -15
  8. package/cpp/keyframe_gate.hpp +33 -0
  9. package/cpp/stitcher.cpp +481 -87
  10. package/cpp/stitcher.hpp +52 -0
  11. package/dist/camera/Camera.d.ts +13 -0
  12. package/dist/camera/Camera.js +9 -64
  13. package/dist/camera/CaptureFrameCounterOverlay.js +12 -1
  14. package/dist/camera/CaptureMemoryPill.d.ts +15 -7
  15. package/dist/camera/CaptureMemoryPill.js +34 -9
  16. package/dist/camera/PanoramaBandOverlay.d.ts +2 -1
  17. package/dist/camera/PanoramaBandOverlay.js +9 -3
  18. package/dist/camera/PanoramaSettings.js +22 -25
  19. package/dist/camera/RectCropPreview.d.ts +3 -29
  20. package/dist/camera/RectCropPreview.js +20 -130
  21. package/dist/stitching/incremental.d.ts +29 -0
  22. package/dist/stitching/useIncrementalStitcher.d.ts +7 -1
  23. package/dist/stitching/useIncrementalStitcher.js +7 -1
  24. package/ios/Sources/RNImageStitcher/IncrementalStitcher.swift +103 -26
  25. package/ios/Sources/RNImageStitcher/IncrementalStitcherBridge.m +4 -0
  26. package/ios/Sources/RNImageStitcher/IncrementalStitcherBridge.swift +15 -0
  27. package/ios/Sources/RNImageStitcher/OpenCVStitcher.mm +21 -1
  28. package/package.json +1 -1
  29. package/src/camera/Camera.tsx +21 -70
  30. package/src/camera/CaptureFrameCounterOverlay.tsx +15 -1
  31. package/src/camera/CaptureMemoryPill.tsx +33 -9
  32. package/src/camera/PanoramaBandOverlay.tsx +9 -1
  33. package/src/camera/PanoramaSettings.ts +22 -25
  34. package/src/camera/RectCropPreview.tsx +38 -220
  35. package/src/stitching/incremental.ts +29 -0
  36. package/src/stitching/useIncrementalStitcher.ts +13 -0
  37. package/android/src/main/java/io/imagestitcher/rn/TransferredNV21.kt +0 -100
package/CHANGELOG.md CHANGED
@@ -14,6 +14,53 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
14
14
  > during 0.x are bumped to a new MINOR (e.g., 0.1 → 0.2), and the
15
15
  > upgrade path is documented in this CHANGELOG.
16
16
 
17
+ ## [0.16.1] — 2026-06-16
18
+
19
+ ### Changed — high-level `cv::Stitcher` is now the default pipeline
20
+
21
+ The batch finalize now drives OpenCV's high-level `cv::Stitcher`
22
+ (PANORAMA) on both platforms instead of the hand-rolled `cv::detail`
23
+ ("manual") path. In testing it produced consistently better seams and
24
+ lower, more stable peak memory. This is a **behaviour change, not an
25
+ API change** — the public surface (`<Camera>`, the hooks, the finalize
26
+ options) is unchanged; only the stitched output and memory profile
27
+ differ.
28
+
29
+ The warper is chosen per-capture (pure function of the selected lens +
30
+ pan direction), always `PANORAMA`:
31
+
32
+ | Lens | Mode A (vertical pan) | Mode B (horizontal pan) |
33
+ | ----- | --------------------- | ----------------------- |
34
+ | 1× | plane | cylindrical |
35
+ | 0.5× | spherical | spherical |
36
+
37
+ The lens comes from the explicit `1x` / `0.5x` the user selected
38
+ (plumbed through the finalize options); the previous FOV-from-intrinsics
39
+ heuristic was unreliable on multi-camera devices and is gone, along with
40
+ the now-redundant rotation-vs-translation (ex-SCANS) branch.
41
+
42
+ ### Added — production memory hardening on the high-level path
43
+
44
+ The OOM guards that previously only covered the manual path were ported
45
+ across, so the new default is memory-safe under pressure:
46
+
47
+ - pre-stitch RSS headroom abort (also works on iOS now via the
48
+ `phys_footprint` probe, which revives the runtime-pressure router);
49
+ - RAM-aware compositing resolution;
50
+ - two-phase `estimateTransform` → project the warp canvas → abort if
51
+ degenerate, downscale or route to the bounded spherical warper if
52
+ over budget;
53
+ - a full C++ catch ladder + a JNI backstop so an allocation failure can
54
+ no longer cross the C-ABI and abort the process;
55
+ - a warper→spherical rescue (high-level) with the manual `PANORAMA` ↔
56
+ `SCANS` mode-fallback preserved for the iOS manual callers.
57
+
58
+ ### Fixed
59
+
60
+ - The native allocator is purged after each stitch, and on Android the
61
+ OpenCV worker pool is pinned to one thread, eliminating the per-stitch
62
+ RSS creep observed on the manual path.
63
+
17
64
  ## [0.16.0] — 2026-06-15
18
65
 
19
66
  ### Added — first-time-user panorama capture GUIDANCE
package/README.md CHANGED
@@ -353,21 +353,32 @@ omitted keys fall back to the English default. This covers the rotate prompt,
353
353
  the pan hint, the live "too fast" cue, the lateral-drift popups, the crop-editor
354
354
  buttons, the **capture-status banner**, and the **crop-editor warning banners**:
355
355
 
356
- | Key | Group | English default |
356
+ Each default is the **exact, complete** source string — translate it verbatim
357
+ (keep the `{included}` / `{requested}` / `{percent}` placeholders in
358
+ `warnLowFrameUtilization`), or `import { DEFAULT_GUIDANCE_COPY }` to seed your
359
+ catalogue programmatically.
360
+
361
+ | Key | Where it appears | English default (translate verbatim) |
357
362
  | --- | --- | --- |
358
363
  | `rotateToLandscape` | rotate prompt | `Rotate to landscape` |
359
364
  | `rotateToPortrait` | rotate prompt | `Rotate to portrait` |
360
- | `panHint` | pan how-to | `Pan slowly top to bottom` |
361
- | `tooFast` | speed cue | `Moving too fast — slow down` |
362
- | `lateralStopTitle` / `lateralStopBody` / `lateralStopDismiss` | lateral popup (stitched) | `Keep the pan straight` / … / `Got it` |
363
- | `lateralWrongDirectionTitle` / `lateralWrongDirectionBody` | lateral popup (too few frames) | `Follow the arrow` / |
364
- | `cropConfirm` / `cropReset` / `cropUseOriginal` / `cropRetake` | crop buttons | `Crop` / `Reset` / `Use original` / `Retake` |
365
- | `previewConfirm` | preview-only accept button (`showPreview`) | `Confirm` |
365
+ | `panHint` | pan how-to overlay | `Pan slowly top to bottom` |
366
+ | `tooFast` | speed-cue pill | `Moving too fast — slow down` |
367
+ | `lateralStopTitle` | lateral-drift popup (stitched) | `Keep the pan straight` |
368
+ | `lateralStopBody` | lateral-drift popup (stitched) | `You moved sideways. Pan in one direction only — we stitched what you captured.` |
369
+ | `lateralStopDismiss` | lateral-drift popup button | `Got it` |
370
+ | `lateralWrongDirectionTitle` | lateral-drift popup (too few frames) | `Follow the arrow` |
371
+ | `lateralWrongDirectionBody` | lateral-drift popup (too few frames) | `You moved the phone the wrong way. Pan slowly in the direction the arrow shows, in one straight line.` |
372
+ | `cropConfirm` | crop-editor button | `Crop` |
373
+ | `cropReset` | crop-editor button | `Reset` |
374
+ | `cropUseOriginal` | crop-editor button | `Use original` |
375
+ | `cropRetake` | crop-editor button | `Retake` |
376
+ | `previewConfirm` | preview accept button (`showPreview`) | `Confirm` |
366
377
  | `statusRecording` | status banner | `Hold steady — pan slowly` |
367
378
  | `statusStitching` | status banner | `Stitching panorama…` |
368
- | `warnLowFrameUtilization` | crop warning **(template)** | `Only {included} of {requested} captured frames ({percent}%) could be used — …` |
369
- | `warnLateralDriftFinalize` | crop warning | `Capture stopped early because the phone drifted sideways — …` |
370
- | `warnHighPanSpeed` | crop warning | `The capture was taken faster than the recommended pace — …` |
379
+ | `warnLowFrameUtilization` | crop warning **(template)** | `Only {included} of {requested} captured frames ({percent}%) could be used — the panorama may be incomplete. Pan more slowly and steadily next time.` |
380
+ | `warnLateralDriftFinalize` | crop warning | `Capture stopped early because the phone drifted sideways — only the part captured before the drift was stitched.` |
381
+ | `warnHighPanSpeed` | crop warning | `The capture was taken faster than the recommended pace — the result may not be the best. Pan more slowly next time.` |
371
382
 
372
383
  > **Templates:** `warnLowFrameUtilization` is interpolated at runtime — your
373
384
  > translation must keep the `{included}`, `{requested}` and `{percent}`
@@ -433,6 +444,10 @@ function CaptureScreen() {
433
444
  seed your translation catalogue from the source strings, and
434
445
  `RECOVERABLE_STITCH_GUIDANCE` exposes the built-in error copy for the same reason.
435
446
 
447
+ > **Full worked example** — a Spanish `es.json` catalogue (both surfaces) plus a
448
+ > host language-setting that switches the copy at runtime: see the
449
+ > [Internationalization guide](https://bhargavkanda.github.io/react-native-image-stitcher/docs/i18n#worked-example-spanish-with-a-dynamic-language-setting).
450
+
436
451
  ### Migration from 0.13.x
437
452
 
438
453
  - **Removed:** the `panGuide` and `panoramaGuidance` props (the
@@ -97,7 +97,11 @@ double procRssMB() {
97
97
  * static_cast<double>(sysconf(_SC_PAGE_SIZE)) / (1024.0 * 1024.0);
98
98
  }
99
99
 
100
- void purgeNativeAllocator() {
100
+ // Returns the POST-purge RSS in MB (the leak-plateau "floor"), or -1 when the
101
+ // diagnostic reads are gated off. The mallopt(M_PURGE) CALL is UNCONDITIONAL —
102
+ // it's the leak fix; only its before/after READS + the log are gated (3A), so a
103
+ // release build pays nothing while debug builds surface memFloor.
104
+ double purgeNativeAllocator(bool profiling) {
101
105
  using MalloptFn = int (*)(int, int);
102
106
  // Resolve mallopt at runtime (API-26 symbol; minSdk 24). Prefer an explicit
103
107
  // libc.so handle — RTLD_DEFAULT from a dlopen'd .so doesn't always reach
@@ -108,8 +112,9 @@ void purgeNativeAllocator() {
108
112
  if (s == nullptr) s = dlsym(RTLD_DEFAULT, "mallopt");
109
113
  return reinterpret_cast<MalloptFn>(s);
110
114
  }();
111
- const double before = procRssMB();
112
- if (fn != nullptr) fn(M_PURGE, 0);
115
+ const double before = profiling ? procRssMB() : -1.0;
116
+ if (fn != nullptr) fn(M_PURGE, 0); // the fix — always runs
117
+ if (!profiling) return -1.0;
113
118
  const double after = procRssMB();
114
119
  // Diagnostic: shows whether mallopt resolved and how much RSS the purge
115
120
  // actually returned to the OS. If mallopt=MISSING → dlsym failed; if
@@ -117,6 +122,7 @@ void purgeNativeAllocator() {
117
122
  // leak) and M_PURGE can't help.
118
123
  LOGI("[memstat] purge: mallopt=%s rss %.1f -> %.1f MB",
119
124
  (fn != nullptr) ? "ok" : "MISSING", before, after);
125
+ return after; // memFloor — the post-purge plateau metric
120
126
  }
121
127
 
122
128
  } // namespace
@@ -189,6 +195,9 @@ Java_io_imagestitcher_rn_BatchStitcher_nativeStitchFramePaths(
189
195
  // re-arms the manual pipeline's dynamic plane→spherical fallback/divergence
190
196
  // switch (they only fire when warperType != "spherical").
191
197
  cfg.useManualPipeline = (useManualPipeline == JNI_TRUE);
198
+ // 2026-06-16 — memory profiling (DEV). Gated by the compile flag (debug-on,
199
+ // release-off); Android leaves memProbeFn null so rss_mb() uses /proc.
200
+ cfg.enableMemoryProfiling = (RNIS_MEMORY_PROFILING != 0);
192
201
  if (cfg.warperType.empty()) cfg.warperType = "spherical";
193
202
  if (cfg.registrationResolMP <= 0.0) {
194
203
  cfg.registrationResolMP = 0.6;
@@ -210,13 +219,49 @@ Java_io_imagestitcher_rn_BatchStitcher_nativeStitchFramePaths(
210
219
 
211
220
  const std::string outPath = jstring_to_string(env, outputPath);
212
221
 
213
- retailens::StitchResult result = retailens::stitchFramePaths(
214
- paths, outPath, cfg, &androidLogBridge);
222
+ // 2026-06-16 (review #1) — backstop try/catch at the JNI C-ABI boundary.
223
+ // stitchFramePaths now has its own catch ladders (high-level + manual), so
224
+ // this should never fire — but a C++ exception crossing into JNI is UB
225
+ // (std::terminate/SIGABRT), so we NEVER let one through: convert any escape
226
+ // into a Java exception the Kotlin layer can catch.
227
+ retailens::StitchResult result;
228
+ try {
229
+ result = retailens::stitchFramePaths(
230
+ paths, outPath, cfg, &androidLogBridge);
231
+ } catch (const std::exception& e) {
232
+ throw_runtime(env, std::string("native stitch crashed: ") + e.what());
233
+ return nullptr;
234
+ } catch (...) {
235
+ throw_runtime(env, "native stitch crashed (unknown exception)");
236
+ return nullptr;
237
+ }
215
238
 
216
239
  // Return the stitch's freed native memory to the OS so the native-heap RSS
217
240
  // baseline doesn't ratchet up ~10-15 MB per capture (see purgeNativeAllocator).
218
- // Applies to BOTH pipelines (they share the OpenCV/bionic allocator).
219
- purgeNativeAllocator();
241
+ // Applies to BOTH pipelines (they share the OpenCV/bionic allocator). The
242
+ // post-purge RSS is the leak-plateau "floor" — append it to debugSummary so
243
+ // it rides the existing nativeLastDebugSummary() path to JS (no new bridge).
244
+ const double memFloor = purgeNativeAllocator(RNIS_MEMORY_PROFILING != 0);
245
+ if ((RNIS_MEMORY_PROFILING != 0) && memFloor >= 0.0) {
246
+ char fbuf[40];
247
+ std::snprintf(fbuf, sizeof(fbuf), ";memFloor=%.1f", memFloor);
248
+ if (!result.debugSummary.empty()) result.debugSummary += fbuf;
249
+ // 2026-06-16 — one authoritative per-stitch memory line to logcat (the
250
+ // sampler peak otherwise only rides debugSummary to the on-screen
251
+ // overlay). pipe/warp/mode lets each line be attributed to a preview
252
+ // tab: pipe=manual warp=plane mode=panorama = "As captured" primary;
253
+ // pipe=highlevel warp=plane = HL·Plane; warp=spherical = HL·Sph;
254
+ // mode=scans = SCANS. Grep `[memstat] record:` to harvest all of them.
255
+ LOGI("[memstat] record: pipe=%s warp=%s mode=%s before=%.1f peak=%.1f "
256
+ "after=%.1f floor=%.1f src=%s frames=%d/%d",
257
+ cfg.useManualPipeline ? "manual" : "highlevel",
258
+ cfg.warperType.c_str(),
259
+ (result.stitchModeUsed == retailens::StitchMode::Scans)
260
+ ? "scans" : "panorama",
261
+ result.memBeforeMB, result.memPeakMB, result.memAfterMB, memFloor,
262
+ result.memSource.c_str(),
263
+ result.framesIncluded, result.framesRequested);
264
+ }
220
265
 
221
266
  if (!result.success) {
222
267
  const std::string msg = "Stitch failed: " + result.errorMessage +
@@ -311,23 +311,30 @@ Java_io_imagestitcher_rn_KeyframeGate_nativeEvaluateWithFrame(
311
311
  }
312
312
  }
313
313
 
314
- // Pin the byte[] for the duration of the gate evaluate. Use
315
- // GetPrimitiveArrayCritical (zero-copy, JVM pins the GC) over
316
- // GetByteArrayElements (may copy on some VMs) because at 30-60
317
- // Hz of 2 MB Y-planes, the copy cost adds up. Evaluate is
318
- // ~1-5 ms so the pin window is short. Always paired with
319
- // ReleasePrimitiveArrayCritical even on the error paths below.
314
+ // 2026-06-16 (audit #4) — pin the byte[] ONLY to INGEST it.
315
+ // GetPrimitiveArrayCritical is zero-copy (the JVM pins the GC) — preferred
316
+ // over GetByteArrayElements (which may copy a 2 MB Y-plane at 30-60 Hz) —
317
+ // but it blocks the GC for the pin's whole life. So we keep that life to a
318
+ // single downscale: ingestWorkingFrame() reads the pinned bytes into an
319
+ // OWNED working frame, we Release IMMEDIATELY, then evaluateWithWorkingMat()
320
+ // runs the heavy OpenCV (goodFeaturesToTrack / optical flow) with the pin
321
+ // already gone — no longer stalling the GC or the frame-rate producer
322
+ // thread. Always paired with ReleasePrimitiveArrayCritical, even on errors.
320
323
  retailens::KeyframeGateDecision d;
321
324
  if (grayBytes && grayWidth > 0 && grayHeight > 0 && grayStride >= grayWidth) {
322
325
  void* raw = env->GetPrimitiveArrayCritical(grayBytes, nullptr);
323
326
  if (raw) {
324
- d = gate(handle)->evaluateWithFrame(
325
- pose, planePtr,
327
+ gate(handle)->ingestWorkingFrame(
326
328
  static_cast<const uint8_t*>(raw),
327
329
  static_cast<int32_t>(grayWidth),
328
330
  static_cast<int32_t>(grayHeight),
329
331
  static_cast<int32_t>(grayStride));
330
332
  env->ReleasePrimitiveArrayCritical(grayBytes, raw, JNI_ABORT);
333
+ // Pin released — heavy OpenCV now runs outside the critical section.
334
+ d = gate(handle)->evaluateWithWorkingMat(
335
+ pose, planePtr,
336
+ static_cast<int32_t>(grayWidth),
337
+ static_cast<int32_t>(grayHeight));
331
338
  } else {
332
339
  // GetPrimitiveArrayCritical failed (rare, but defensive).
333
340
  // Fall back to pose-only path so we degrade gracefully
@@ -633,7 +633,8 @@ class IncrementalStitcher(
633
633
  val wasBatchKeyframe = batchKeyframeMode
634
634
  val keyframePathsSnapshot = batchKeyframePaths.toList()
635
635
  val captureOrientationSnapshot = batchCaptureOrientation
636
- val warperTypeSnapshot = batchWarperType
636
+ // batchWarperType (settings) is superseded by the high-level warper tree
637
+ // (pickHighLevelWarper) below — kept as a field for back-compat, unused here.
637
638
  val blenderTypeSnapshot = batchBlenderType
638
639
  val seamFinderTypeSnapshot = batchSeamFinderType
639
640
  val useInscribedRectCropSnapshot = batchUseInscribedRectCrop
@@ -664,11 +665,31 @@ class IncrementalStitcher(
664
665
  // falls back to pose data only. Always non-negative.
665
666
  val imuTranslationMetres = (options.getDoubleOrDefault("imuTranslationMetres", 0.0) ?: 0.0)
666
667
  .coerceAtLeast(0.0)
668
+ // Resolve once so the dev readout gets the SAME tMeters / ratio / rRadians
669
+ // that drove the decision — and gets them even when the mode was forced
670
+ // (informative: shows what auto WOULD have picked).
671
+ val autoResolution = resolveStitchModeAuto(firstPose, lastPose, imuTranslationMetres)
667
672
  val stitchModeResolved: String = when (batchStitchMode) {
668
673
  "panorama" -> "panorama"
669
674
  "scans" -> "scans"
670
- else -> resolveStitchModeAuto(firstPose, lastPose, imuTranslationMetres)
675
+ else -> autoResolution.mode
671
676
  }
677
+ // Surface the gyro rotation + translation + decision ratio for EVERY
678
+ // capture (the forced modes skip the auto decision, but the dev preview
679
+ // still reads these to tune the panorama-vs-SCANS threshold).
680
+ val rRadiansResolved: Double = autoResolution.rRadians
681
+ val tMetersResolved: Double = autoResolution.tMeters
682
+ val decisionRatioResolved: Double = autoResolution.ratio
683
+ // 2026-06-16 — HIGH-LEVEL ACROSS THE BOARD. Pick the warper from the
684
+ // (motion, Mode A/B, zoom) tree and always run cv::Stitcher PANORAMA
685
+ // (useManualPipeline=false at the stitchSync call below). stitchModeResolved
686
+ // is now only the MOTION classifier feeding the tree + the dev readout;
687
+ // the actual stitch mode is always panorama. Zoom comes from the EXPLICIT
688
+ // lens label the user selected ('1x'|'0.5x') — the reliable signal (FOV
689
+ // from intrinsics was unreliable: multi-cam 0.5x doesn't change fx, and
690
+ // the non-AR path may supply fx=0 → FOV defaulted to 65° → never 0.5x).
691
+ val lensOpt = options.getString("lens") ?: "1x"
692
+ val highLevelWarper = pickHighLevelWarper(captureOrientationSnapshot, lensOpt)
672
693
  android.util.Log.i(
673
694
  "IncrementalStitcher",
674
695
  "finalize stitch-mode: configured=$batchStitchMode resolved=$stitchModeResolved " +
@@ -714,12 +735,13 @@ class IncrementalStitcher(
714
735
  keyframePathsSnapshot.toTypedArray(),
715
736
  outputPath,
716
737
  quality,
717
- warperTypeSnapshot,
738
+ highLevelWarper, // tree-chosen (was batchWarperType)
718
739
  blenderTypeSnapshot,
719
740
  seamFinderTypeSnapshot,
720
741
  captureOrientationSnapshot,
721
742
  useInscribedRectCropSnapshot,
722
- stitchMode = stitchModeResolved,
743
+ stitchMode = "panorama", // always high-level PANORAMA
744
+ useManualPipeline = false, // high level across the board
723
745
  )
724
746
  // 2026-05-15 (D) — dims layout from native JNI:
725
747
  // [0] width, [1] height, [2] framesRequested,
@@ -748,6 +770,11 @@ class IncrementalStitcher(
748
770
  // resolved cv::Stitcher mode so JS can surface it
749
771
  // on the output preview + debug toast.
750
772
  map.putString("stitchModeResolved", stitchModeResolved)
773
+ map.putDouble("rRadians", rRadiansResolved)
774
+ // Dev tuning readout — translation magnitude + the auto
775
+ // decision ratio that drove panorama-vs-SCANS.
776
+ map.putDouble("tMeters", tMetersResolved)
777
+ map.putDouble("decisionRatio", decisionRatioResolved)
751
778
  // 2026-06-15 (iOS parity) — the exact keyframe JPEG
752
779
  // paths used for this stitch, so JS can re-stitch
753
780
  // them ON DEMAND via refinePanorama (the high-level
@@ -875,36 +902,13 @@ class IncrementalStitcher(
875
902
  grayHeight: Int,
876
903
  grayStride: Int,
877
904
  onAccept: (targetPath: String) -> Boolean,
878
- // 2026-05-21 (v0.3) — only required when batchKeyframeMode
879
- // is false (the legacy hybrid/firstwins live-engine path,
880
- // which feeds JPEG paths into addFrameAtPath for each ARCore
881
- // frame). Pass null when batchKeyframeMode is true; the
882
- // batch path uses `grayData` + `onAccept` instead. Modern
883
- // callers prefer `nv21PixelData` below`legacyJpegPath` is
884
- // kept only as a defensive fallback for older call sites
885
- // that have not yet been migrated.
886
- legacyJpegPath: String? = null,
887
- // F8.6 — pixel-data path for live engines. When supplied
888
- // (and `batchKeyframeMode == false`), takes precedence over
889
- // `legacyJpegPath`: the live engine ingests via
890
- // `addFramePixelData` (NV21 → BGR Mat in-process) instead of
891
- // `addFrameAtPath` (JPEG decode round-trip). Saves ~30-50 ms
892
- // per accepted frame on a mid-tier device. Pass null to use
893
- // the legacy JPEG path.
894
- //
895
- // OWNERSHIP: wrapped in `TransferredNV21` (audit #4A,
896
- // v0.10.0). The wrapper enforces single-use: the engine
897
- // calls `.takeOnce()` on the producer thread before
898
- // dispatching to `workScope`; subsequent attempts to extract
899
- // the bytes throw. Callers MUST construct a fresh
900
- // `TransferredNV21` per frame and MUST NOT hand the same
901
- // instance to two consumers (e.g., a sync gate-eval + an
902
- // async workScope.launch). The Frame Processor plugin and
903
- // the AR camera view both allocate fresh NV21 arrays per
904
- // frame; the wrapper is a defensive-programming guard.
905
- nv21PixelData: TransferredNV21? = null,
906
- nv21PixelWidth: Int = 0,
907
- nv21PixelHeight: Int = 0,
905
+ // 2026-06-16 (audit #8/L3) — the live-engine ingest params
906
+ // (legacyJpegPath / nv21PixelData / nv21PixelWidth/Height) were
907
+ // removed here. The live engines were archived in 2026-06, so the
908
+ // only remaining path is batch-keyframe (always on), which ingests via
909
+ // `grayData` + `onAccept`. The TransferredNV21 ownership wrapper had no
910
+ // live consumer (takeOnce() called nowhere verified by grep) and is
911
+ // deleted along with these params.
908
912
  ) {
909
913
  // ── V16 batch-keyframe: AR-driven path ─────────────────────
910
914
  //
@@ -1213,21 +1217,6 @@ class IncrementalStitcher(
1213
1217
  grayWidth = width,
1214
1218
  grayHeight = height,
1215
1219
  grayStride = yRowStride,
1216
- // F8.6 — pass the already-packed NV21 so the live
1217
- // engine branch (hybrid / firstwins) can ingest via
1218
- // `addFramePixelData` instead of JPEG-decoding a
1219
- // separately-written path. Batch-keyframe mode
1220
- // ignores these (it uses `grayData` + `onAccept`).
1221
- //
1222
- // v0.10.0 audit #4A — wrap in TransferredNV21 so the
1223
- // engine takes ownership exactly once on the producer
1224
- // thread (engine calls `.takeOnce()` before workScope).
1225
- // Misuse (handing this same instance to two consumers)
1226
- // throws at the second `.takeOnce()` site, not silently
1227
- // corrupting frames.
1228
- nv21PixelData = TransferredNV21(nv21Bytes),
1229
- nv21PixelWidth = width,
1230
- nv21PixelHeight = height,
1231
1220
  onAccept = { targetPath ->
1232
1221
  // Synchronous JPEG encode via the existing
1233
1222
  // YuvImageConverter (also used by RNSARCameraView's
@@ -1737,6 +1726,33 @@ class IncrementalStitcher(
1737
1726
  }
1738
1727
  }
1739
1728
 
1729
+ /**
1730
+ * Total physical RAM in MB. Lets the DEV memory pill derive RAM-aware
1731
+ * pressure bands instead of the iPhone-fixed 1500/2200 MB thresholds (which
1732
+ * never trip on a 4 GB Android phone that jetsams ~1.3 GB — false comfort).
1733
+ * Reads `_SC_PHYS_PAGES × _SC_PAGE_SIZE` (TOTAL + stable across runs, unlike
1734
+ * the rate-limited ActivityManager path). -1.0 on failure.
1735
+ */
1736
+ @ReactMethod
1737
+ fun getDeviceTotalRamMB(promise: Promise) {
1738
+ try {
1739
+ val pages = android.system.Os.sysconf(android.system.OsConstants._SC_PHYS_PAGES)
1740
+ val pageSize =
1741
+ android.system.Os.sysconf(android.system.OsConstants._SC_PAGESIZE)
1742
+ if (pages <= 0 || pageSize <= 0) {
1743
+ promise.resolve(-1.0)
1744
+ return
1745
+ }
1746
+ promise.resolve(pages.toDouble() * pageSize.toDouble() / (1024.0 * 1024.0))
1747
+ } catch (t: Throwable) {
1748
+ android.util.Log.w(
1749
+ "IncrementalStitcher",
1750
+ "getDeviceTotalRamMB: failed: ${t.message}",
1751
+ )
1752
+ promise.resolve(-1.0)
1753
+ }
1754
+ }
1755
+
1740
1756
  /**
1741
1757
  * Release the C++ KeyframeGate heap allocation when RN tears
1742
1758
  * down the bridge module (e.g. on a JS reload). Without this,
@@ -1928,6 +1944,24 @@ class IncrementalStitcher(
1928
1944
  *
1929
1945
  * Returns "panorama" or "scans" — never "auto".
1930
1946
  */
1947
+ /**
1948
+ * Result of [resolveStitchModeAuto]: the chosen mode PLUS the gyro rotation
1949
+ * magnitude that drove the decision. rRadians is surfaced to JS (the dev
1950
+ * 3-tab preview shows it) so the panorama-vs-SCANS rotation threshold can be
1951
+ * tuned from real captures. rRadians is 0.0 only on the no-pose fallbacks
1952
+ * (non-AR with no pose data) — there is no gyro-derived rotation to report.
1953
+ */
1954
+ private data class StitchModeResolution(
1955
+ val mode: String,
1956
+ val rRadians: Double,
1957
+ // tMeters = translation magnitude (m) that fed the ratio; ratio = the
1958
+ // tScore/(tScore+rScore) decision value (>=0.55 → SCANS). Surfaced to the
1959
+ // dev readout so the panorama-vs-SCANS threshold can be tuned from real
1960
+ // captures, alongside rRadians.
1961
+ val tMeters: Double,
1962
+ val ratio: Double,
1963
+ )
1964
+
1931
1965
  private fun resolveStitchModeAuto(
1932
1966
  firstPose: DoubleArray?,
1933
1967
  lastPose: DoubleArray?,
@@ -1935,14 +1969,16 @@ class IncrementalStitcher(
1935
1969
  // translation in METRES. Used as a fallback when pose-derived
1936
1970
  // translation is 0 (non-AR mode).
1937
1971
  imuTranslationMetres: Double = 0.0,
1938
- ): String {
1972
+ ): StitchModeResolution {
1939
1973
  if (firstPose == null || lastPose == null) {
1940
1974
  // No pose data at all — fall back on the IMU signal. IMU
1941
1975
  // > 5 cm hints SCANS; everything else hints PANORAMA.
1942
- return if (imuTranslationMetres > 0.05) "scans" else "panorama"
1976
+ return StitchModeResolution(
1977
+ if (imuTranslationMetres > 0.05) "scans" else "panorama", 0.0, 0.0, 0.0)
1943
1978
  }
1944
1979
  if (firstPose.size != 7 || lastPose.size != 7) {
1945
- return if (imuTranslationMetres > 0.05) "scans" else "panorama"
1980
+ return StitchModeResolution(
1981
+ if (imuTranslationMetres > 0.05) "scans" else "panorama", 0.0, 0.0, 0.0)
1946
1982
  }
1947
1983
 
1948
1984
  // Translation magnitude (Euclidean, in metres) — pose-derived.
@@ -1962,11 +1998,7 @@ class IncrementalStitcher(
1962
1998
  // conventions; rotated by the pose quaternion gives the world-
1963
1999
  // frame forward direction. Angle between the first and last
1964
2000
  // camera-forward vectors is the total rotation around any axis.
1965
- val fwdFirst = qrotForward(firstPose[3], firstPose[4], firstPose[5], firstPose[6])
1966
- val fwdLast = qrotForward(lastPose[3], lastPose[4], lastPose[5], lastPose[6])
1967
- val dot = (fwdFirst[0] * fwdLast[0] + fwdFirst[1] * fwdLast[1] + fwdFirst[2] * fwdLast[2])
1968
- .coerceIn(-1.0, 1.0)
1969
- val rRadians = kotlin.math.acos(dot)
2001
+ val rRadians = rotationRadians(firstPose, lastPose)
1970
2002
 
1971
2003
  // Normalisation: 10 cm of translation ≈ 1 rad of rotation as
1972
2004
  // "equivalent magnitude" for the ratio. Empirically: shelf
@@ -1977,7 +2009,7 @@ class IncrementalStitcher(
1977
2009
  val tScore = tMeters / 0.10
1978
2010
  val rScore = rRadians / 1.00
1979
2011
  val denom = tScore + rScore
1980
- if (denom <= 1e-9) return "panorama" // no motion either way
2012
+ if (denom <= 1e-9) return StitchModeResolution("panorama", rRadians, tMeters, 0.0) // no motion
1981
2013
  val ratio = tScore / denom
1982
2014
 
1983
2015
  // 2026-06-15 — LOW-ROTATION GUARD. The gyro rotation (rRadians) is
@@ -2000,7 +2032,51 @@ class IncrementalStitcher(
2000
2032
  "ratio=${"%.3f".format(ratio)} " +
2001
2033
  "rotGuard=$lowRotationGuard → $mode",
2002
2034
  )
2003
- return mode
2035
+ return StitchModeResolution(mode, rRadians, tMeters, ratio)
2036
+ }
2037
+
2038
+ /**
2039
+ * 2026-06-16 — high-level warper decision tree (the pipeline is now ALWAYS
2040
+ * high-level cv::Stitcher PANORAMA — useManualPipeline=false). Warper is a
2041
+ * pure function of (lens, pan direction); the rotation-vs-translation
2042
+ * (ex-SCANS) distinction was DROPPED as redundant — at 1x the same
2043
+ * direction-based warpers serve both, and 0.5x is always spherical. Inputs:
2044
+ * orientation = capture hold ("landscape*" = Mode A vertical pan;
2045
+ * "portrait*" = Mode B horizontal pan)
2046
+ * lens = the EXPLICIT lens the user selected ("0.5x" ultra-wide |
2047
+ * "1x" wide). Reliable zoom signal (FOV-from-intrinsics was
2048
+ * unreliable — multi-cam 0.5x reaches the ultra-wide by zoom
2049
+ * without changing fx, and the non-AR path may supply fx=0).
2050
+ *
2051
+ * 0.5x ultra-wide → spherical (bounded both axes; any pan)
2052
+ * 1x + Mode A (vertical) → plane
2053
+ * 1x + Mode B (horizontal) → cylindrical
2054
+ *
2055
+ * Quality-preferred warper; the C++ memory ladder force-falls to spherical
2056
+ * (and downscales compositingResol) under pressure.
2057
+ */
2058
+ private fun pickHighLevelWarper(
2059
+ orientation: String,
2060
+ lens: String,
2061
+ ): String {
2062
+ if (lens == "0.5x") return "spherical" // ultra-wide → always spherical
2063
+ val verticalPanModeA = orientation.startsWith("landscape")
2064
+ return if (verticalPanModeA) "plane" else "cylindrical" // 1x: A→plane, B→cylindrical
2065
+ }
2066
+
2067
+ /**
2068
+ * Gyro rotation magnitude (radians) between two 7-element poses
2069
+ * `[tx,ty,tz,qx,qy,qz,qw]` — the angle between the camera-forward vectors.
2070
+ * Returns 0.0 if either pose is missing/malformed (non-AR with no pose).
2071
+ * Shared by [resolveStitchModeAuto] and the finalize `rRadians` readout (DRY).
2072
+ */
2073
+ private fun rotationRadians(firstPose: DoubleArray?, lastPose: DoubleArray?): Double {
2074
+ if (firstPose == null || lastPose == null) return 0.0
2075
+ if (firstPose.size != 7 || lastPose.size != 7) return 0.0
2076
+ val f = qrotForward(firstPose[3], firstPose[4], firstPose[5], firstPose[6])
2077
+ val l = qrotForward(lastPose[3], lastPose[4], lastPose[5], lastPose[6])
2078
+ val dot = (f[0] * l[0] + f[1] * l[1] + f[2] * l[2]).coerceIn(-1.0, 1.0)
2079
+ return kotlin.math.acos(dot)
2004
2080
  }
2005
2081
 
2006
2082
  /**
@@ -612,26 +612,13 @@ class RNSARCameraView @JvmOverloads constructor(
612
612
  val rotationForEncode = if (lastDisplayRotation >= 0)
613
613
  lastDisplayRotation else android.view.Surface.ROTATION_0
614
614
 
615
- // F8.6 (v0.6) the eager JPEG encode for live-engine mode
616
- // is gone. Pass the already-packed NV21 directly via
617
- // `nv21PixelData`; the engine's new `addFramePixelData`
618
- // path builds the BGR cv::Mat in-process via cvtColor,
619
- // skipping the JPEG decode round-trip downstream. In
620
- // batch-keyframe mode the engine ignores `nv21PixelData`
621
- // (it uses `grayData` + `onAccept` lazily); no behaviour
622
- // change there.
623
- //
624
- // (Was: eager JPEG encode for non-batch-keyframe modes,
625
- // written to `tmpJpegFile`, passed as `legacyJpegPath`.
626
- // See the v0.3 / F8.6 entries in CHANGELOG.md.)
627
- //
628
- // Synchronous engine ingest. The ARCore Image ownership
629
- // contract requires the engine to consume the TransferredNV21
630
- // before ARCore recycles the Image, so this runs inline. Only
631
- // ingest when the host has actively engaged capture
632
- // (`setIncrementalIngestionActive(true)`). (The v0.8.0 worklet-
633
- // runtime `runFirstParty` indirection + host-worklet fan-out
634
- // were archived in the 2026-06 batch-keyframe cleanup.)
615
+ // Batch-keyframe ingest. The gate reads the Y plane of the packed
616
+ // NV21 synchronously (grayData) and the lazy onAccept JPEG-encodes only
617
+ // accepted frames — no eager encode, no live-engine pixel-data path
618
+ // (the live engines + the TransferredNV21 ownership wrapper were removed
619
+ // in the 2026-06 cleanup; see audit #8). Runs inline so the gate read
620
+ // completes before ARCore recycles the Image. Only ingest when the host
621
+ // has actively engaged capture (`setIncrementalIngestionActive(true)`).
635
622
  if (ingestActive) {
636
623
  module.ingestFromARCameraView(
637
624
  tx = tArr[0].toDouble(),
@@ -654,22 +641,6 @@ class RNSARCameraView @JvmOverloads constructor(
654
641
  grayWidth = packed.width,
655
642
  grayHeight = packed.height,
656
643
  grayStride = packed.width,
657
- legacyJpegPath = null,
658
- // F8.6 — pixel-data path for live engines. Batch-
659
- // keyframe mode ignores these (bails earlier).
660
- //
661
- // v0.10.0 audit #4A — wrap `packed.nv21` in
662
- // TransferredNV21 so ownership is enforced at runtime.
663
- // The AR caller passes the SAME `packed.nv21` array as
664
- // both `grayData` (sync, gate-eval read) and
665
- // `nv21PixelData` (async, engine ingest). Today no race
666
- // because grayData is consumed inside evaluateWithFrame
667
- // before workScope.launch fires; the wrapper makes a
668
- // future refactor that reorders consumption fail loudly
669
- // instead of silently corrupting frames.
670
- nv21PixelData = TransferredNV21(packed.nv21),
671
- nv21PixelWidth = packed.width,
672
- nv21PixelHeight = packed.height,
673
644
  onAccept = { targetPath ->
674
645
  // Lazy JPEG encode. Runs ONLY if the C++ KeyframeGate
675
646
  // accepted the frame. Encodes from the pre-packed