react-native-image-stitcher 0.16.0 → 0.16.2

package/CHANGELOG.md

@@ -14,6 +14,86 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 > during 0.x are bumped to a new MINOR (e.g., 0.1 → 0.2), and the
 > upgrade path is documented in this CHANGELOG.
 
+## [0.16.2] — 2026-06-17
+
+### Added — reuse the bundled OpenCV from your host app's native code (Android)
+
+A host app's own native (C++/NDK) code can now reuse the **same** custom
+OpenCV this library bundles (4.10.0, arm64-v8a) — **including `cv::Stitcher`**
+— with no second copy of `libopencv_java4.so` in the APK.
+
+The Android build now publishes the location of its vendored OpenCV SDK via
+`rootProject.ext.rnisOpenCVDir` (and `rnisOpenCVAndroidSdkDir`). A consumer
+points its `externalNativeBuild` at `-DOpenCV_DIR=${rootProject.ext.rnisOpenCVDir}`,
+calls `find_package(OpenCV)`, and links the shared `opencv_java` (core /
+imgproc / calib3d / … resolved at runtime from the already-shipped `.so`)
+plus the whole-archived static `opencv_stitching` (`cv::Stitcher`). A
+build-verified consumer ships in the example app
+(`example/android/app/src/main/cpp/`).
+
+This is additive — no public API or runtime-behaviour change. AGP
+`prefabPublishing` was evaluated and is unworkable for prebuilt OpenCV
+(prefab only exports libraries the module itself builds), so OpenCV's own
+first-class CMake package is used instead. iOS reuse (the vendored
+`opencv2.xcframework`) is unchanged.
+
+### Docs
+
+Documentation site refreshed: an easier **Getting started**, a complete
+**`<Camera>` API** reference (every prop, the v0.16 guidance params —
+`rectCrop` / `showPreview` / `panMode` / `panGuidance` / `maxPanDurationMs` /
+`panTooFastThreshold` / `lateralBudgetCm` / `guidanceCopy` — and the
+`stitcher` / `frameSelection` settings-JSON tables), a fully-loaded
+**Complete example**, the v0.16 **Capture result & errors** union, and new
+**Sharing OpenCV** / **Bring your own OpenCV** guides.
+
+## [0.16.1] — 2026-06-16
+
+### Changed — high-level `cv::Stitcher` is now the default pipeline
+
+The batch finalize now drives OpenCV's high-level `cv::Stitcher`
+(PANORAMA) on both platforms instead of the hand-rolled `cv::detail`
+("manual") path.  In testing it produced consistently better seams and
+lower, more stable peak memory.  This is a **behaviour change, not an
+API change** — the public surface (`<Camera>`, the hooks, the finalize
+options) is unchanged; only the stitched output and memory profile
+differ.
+
+The warper is chosen per-capture (pure function of the selected lens +
+pan direction), always `PANORAMA`:
+
+| Lens  | Mode A (vertical pan) | Mode B (horizontal pan) |
+| ----- | --------------------- | ----------------------- |
+| 1×    | plane                 | cylindrical             |
+| 0.5×  | spherical             | spherical               |
+
+The lens comes from the explicit `1x` / `0.5x` the user selected
+(plumbed through the finalize options); the previous FOV-from-intrinsics
+heuristic was unreliable on multi-camera devices and is gone, along with
+the now-redundant rotation-vs-translation (ex-SCANS) branch.
+
+### Added — production memory hardening on the high-level path
+
+The OOM guards that previously only covered the manual path were ported
+across, so the new default is memory-safe under pressure:
+
+- pre-stitch RSS headroom abort (also works on iOS now via the
+  `phys_footprint` probe, which revives the runtime-pressure router);
+- RAM-aware compositing resolution;
+- two-phase `estimateTransform` → project the warp canvas → abort if
+  degenerate, downscale or route to the bounded spherical warper if
+  over budget;
+- a full C++ catch ladder + a JNI backstop so an allocation failure can
+  no longer cross the C-ABI and abort the process;
+- a warper→spherical rescue (high-level) with the manual `PANORAMA` ↔
+  `SCANS` mode-fallback preserved for the iOS manual callers.
+
+### Fixed
+
+- The native allocator is purged after each stitch, and on Android the
+  OpenCV worker pool is pinned to one thread, eliminating the per-stitch
+  RSS creep observed on the manual path.
+
 ## [0.16.0] — 2026-06-15
 
 ### Added — first-time-user panorama capture GUIDANCE

package/README.md

@@ -29,8 +29,8 @@ Peer dependencies (the host app provides these):
   "react": ">=18.0.0",
   "react-native": ">=0.72.0",
   "react-native-vision-camera": ">=4.7.0",
+  "react-native-worklets-core": ">=1.3.0",
   "react-native-sensors": ">=7.0.0",
-  "expo-sensors": ">=14.0.0",
   "react-native-safe-area-context": ">=4.0.0"
 }
 ```
@@ -71,50 +71,32 @@ cd android && ./gradlew :app:assembleDebug   # Android
 > See [Orientation support](#orientation-support) for the full story
 > (landscape *is* supported on iOS if you need it).
 
-The minimum: resolve camera permission, then mount `<Camera>` with an
-`onCapture` handler.
+The minimum: mount `<Camera>` with an `onCapture` handler. It fires once
+per capture attempt — gate on `result.ok` before reading the output.
 
 ```tsx
-import {
-  Camera,
-  type CameraCaptureResult,
-  type CameraError,
-} from 'react-native-image-stitcher';
+import { Camera, type CameraCaptureResult } from 'react-native-image-stitcher';
 
 export function CaptureScreen() {
-  const handleCapture = (result: CameraCaptureResult) => {
-    // `onCapture` fires on success AND failure — gate on `ok` first.
-    if (!result.ok) {
-      console.warn('capture failed:', result.error.code, result.error.message);
-      return;
-    }
-    // Non-fatal quality signals (e.g. <70% of frames used). Always present.
-    if (result.warnings.length > 0) {
-      console.warn('warnings:', result.warnings.map((w) => w.code));
-    }
-    if (result.type === 'photo') {
-      console.log('Photo:', result.uri, result.width, result.height);
-    } else {
-      console.log(
-        'Panorama:',
-        result.uri,
-        `${result.framesIncluded}/${result.framesRequested} frames`,
-        `stitched as ${result.stitchModeResolved ?? 'n/a'}`,
-      );
-    }
-  };
-
   return (
     <Camera
-      onCapture={handleCapture}
-      // onError still fires on failure too (an unchanged mirror of the
-      // ok:false result above).
-      onError={(err: CameraError) => console.warn(err.code, err.message)}
+      onCapture={(result: CameraCaptureResult) => {
+        if (!result.ok) {
+          console.warn('capture failed:', result.error.code);
+          return;
+        }
+        // result.type is 'photo' or 'panorama'; both carry uri/width/height.
+        console.log(result.type, result.uri, result.width, result.height);
+      }}
     />
   );
 }
 ```
 
+> **Camera permission is the host's job.** The SDK never requests it for
+> you — resolve it (e.g. with vision-camera's `useCameraPermission`)
+> before mounting `<Camera>`.
+
 ### A complete capture screen
 
 A realistic screen: requests permission up front, shows a capture
@@ -353,21 +335,32 @@ omitted keys fall back to the English default. This covers the rotate prompt,
 the pan hint, the live "too fast" cue, the lateral-drift popups, the crop-editor
 buttons, the **capture-status banner**, and the **crop-editor warning banners**:
 
-| Key | Group | English default |
+Each default is the **exact, complete** source string — translate it verbatim
+(keep the `{included}` / `{requested}` / `{percent}` placeholders in
+`warnLowFrameUtilization`), or `import { DEFAULT_GUIDANCE_COPY }` to seed your
+catalogue programmatically.
+
+| Key | Where it appears | English default (translate verbatim) |
 | --- | --- | --- |
 | `rotateToLandscape` | rotate prompt | `Rotate to landscape` |
 | `rotateToPortrait` | rotate prompt | `Rotate to portrait` |
-| `panHint` | pan how-to | `Pan slowly top to bottom` |
-| `tooFast` | speed cue | `Moving too fast — slow down` |
-| `lateralStopTitle` / `lateralStopBody` / `lateralStopDismiss` | lateral popup (stitched) | `Keep the pan straight` / … / `Got it` |
-| `lateralWrongDirectionTitle` / `lateralWrongDirectionBody` | lateral popup (too few frames) | `Follow the arrow` / … |
-| `cropConfirm` / `cropReset` / `cropUseOriginal` / `cropRetake` | crop buttons | `Crop` / `Reset` / `Use original` / `Retake` |
-| `previewConfirm` | preview-only accept button (`showPreview`) | `Confirm` |
+| `panHint` | pan how-to overlay | `Pan slowly top to bottom` |
+| `tooFast` | speed-cue pill | `Moving too fast — slow down` |
+| `lateralStopTitle` | lateral-drift popup (stitched) | `Keep the pan straight` |
+| `lateralStopBody` | lateral-drift popup (stitched) | `You moved sideways. Pan in one direction only — we stitched what you captured.` |
+| `lateralStopDismiss` | lateral-drift popup button | `Got it` |
+| `lateralWrongDirectionTitle` | lateral-drift popup (too few frames) | `Follow the arrow` |
+| `lateralWrongDirectionBody` | lateral-drift popup (too few frames) | `You moved the phone the wrong way. Pan slowly in the direction the arrow shows, in one straight line.` |
+| `cropConfirm` | crop-editor button | `Crop` |
+| `cropReset` | crop-editor button | `Reset` |
+| `cropUseOriginal` | crop-editor button | `Use original` |
+| `cropRetake` | crop-editor button | `Retake` |
+| `previewConfirm` | preview accept button (`showPreview`) | `Confirm` |
 | `statusRecording` | status banner | `Hold steady — pan slowly` |
 | `statusStitching` | status banner | `Stitching panorama…` |
-| `warnLowFrameUtilization` | crop warning **(template)** | `Only {included} of {requested} captured frames ({percent}%) could be used — …` |
-| `warnLateralDriftFinalize` | crop warning | `Capture stopped early because the phone drifted sideways — …` |
-| `warnHighPanSpeed` | crop warning | `The capture was taken faster than the recommended pace — …` |
+| `warnLowFrameUtilization` | crop warning **(template)** | `Only {included} of {requested} captured frames ({percent}%) could be used — the panorama may be incomplete. Pan more slowly and steadily next time.` |
+| `warnLateralDriftFinalize` | crop warning | `Capture stopped early because the phone drifted sideways — only the part captured before the drift was stitched.` |
+| `warnHighPanSpeed` | crop warning | `The capture was taken faster than the recommended pace — the result may not be the best. Pan more slowly next time.` |
 
 > **Templates:** `warnLowFrameUtilization` is interpolated at runtime — your
 > translation must keep the `{included}`, `{requested}` and `{percent}`
@@ -433,6 +426,10 @@ function CaptureScreen() {
 seed your translation catalogue from the source strings, and
 `RECOVERABLE_STITCH_GUIDANCE` exposes the built-in error copy for the same reason.
 
+> **Full worked example** — a Spanish `es.json` catalogue (both surfaces) plus a
+> host language-setting that switches the copy at runtime: see the
+> [Internationalization guide](https://bhargavkanda.github.io/react-native-image-stitcher/docs/i18n#worked-example-spanish-with-a-dynamic-language-setting).
+
 ### Migration from 0.13.x
 
 - **Removed:** the `panGuide` and `panoramaGuidance` props (the

package/android/build.gradle

@@ -112,6 +112,40 @@ android {
         prefab true
     }
 
+    // ── Host OpenCV reuse: why NOT prefab publishing ─────────────────
+    //
+    // Goal: let a HOST app's own native (C++/NDK) code reuse the SAME
+    // custom OpenCV (4.10.0, arm64-v8a) this AAR already bundles — both
+    // libopencv_java4.so (cv::Mat, imgproc, features2d, calib3d, flann,
+    // photo, video …) AND cv::Stitcher (which lives in the static
+    // archive libopencv_stitching.a, NOT in the fat .so).
+    //
+    // We evaluated AGP `prefabPublishing` first (the idiomatic AAR way)
+    // and it CANNOT carry this OpenCV.  Empirically verified: AGP's
+    // prefab modules only export libraries the module's own
+    // externalNativeBuild PRODUCES (here: just `image_stitcher`).  A
+    // prebuilt jniLib (.so) or a prebuilt static archive (.a) is not an
+    // accepted prefab `libraryName` — the configure fails with
+    // `[CXX1404] did not find implicitly required targets`.  So neither
+    // libopencv_java4.so nor libopencv_stitching.a can ride a prefab.
+    //
+    // Instead we expose the bundled OpenCV's OWN first-class CMake
+    // package — which already defines every module (incl. opencv_stitching
+    // as a STATIC IMPORTED target and opencv_java as a SHARED IMPORTED
+    // target) — to consumers via a rootProject ext property.  A host
+    // points its externalNativeBuild at `-DOpenCV_DIR=<that dir>`, does
+    // `find_package(OpenCV REQUIRED)`, then links the SHARED `opencv_java`
+    // (cv::Mat & friends resolved at runtime from the AAR's already-shipped
+    // libopencv_java4.so — NO second copy) plus the whole-archived STATIC
+    // `opencv_stitching` (cv::Stitcher, a small private copy since it isn't
+    // in the fat .so).  This is the idiomatic Android OpenCV-consumption
+    // path.  See example/android/app for the working consumer.
+    //
+    // Set unconditionally at configure time so it's readable from the
+    // host app module regardless of project evaluation order.
+    rootProject.ext.rnisOpenCVAndroidSdkDir = "$opencvSdkDir/native"
+    rootProject.ext.rnisOpenCVDir = "$opencvSdkDir/native/jni"
+
     // ── JNI shim build path ─────────────────────────────────────────
     // Gradle compiles cpp/image_stitcher_jni.cpp into
     // libimage_stitcher.so for the ABIs filtered above.  The shim

package/android/src/main/cpp/image_stitcher_jni.cpp

@@ -97,7 +97,11 @@ double procRssMB() {
         * static_cast<double>(sysconf(_SC_PAGE_SIZE)) / (1024.0 * 1024.0);
 }
 
-void purgeNativeAllocator() {
+// Returns the POST-purge RSS in MB (the leak-plateau "floor"), or -1 when the
+// diagnostic reads are gated off.  The mallopt(M_PURGE) CALL is UNCONDITIONAL —
+// it's the leak fix; only its before/after READS + the log are gated (3A), so a
+// release build pays nothing while debug builds surface memFloor.
+double purgeNativeAllocator(bool profiling) {
     using MalloptFn = int (*)(int, int);
     // Resolve mallopt at runtime (API-26 symbol; minSdk 24).  Prefer an explicit
     // libc.so handle — RTLD_DEFAULT from a dlopen'd .so doesn't always reach
@@ -108,8 +112,9 @@ void purgeNativeAllocator() {
         if (s == nullptr) s = dlsym(RTLD_DEFAULT, "mallopt");
         return reinterpret_cast<MalloptFn>(s);
     }();
-    const double before = procRssMB();
-    if (fn != nullptr) fn(M_PURGE, 0);
+    const double before = profiling ? procRssMB() : -1.0;
+    if (fn != nullptr) fn(M_PURGE, 0);          // the fix — always runs
+    if (!profiling) return -1.0;
     const double after = procRssMB();
     // Diagnostic: shows whether mallopt resolved and how much RSS the purge
     // actually returned to the OS.  If mallopt=MISSING → dlsym failed; if
@@ -117,6 +122,7 @@ void purgeNativeAllocator() {
     // leak) and M_PURGE can't help.
     LOGI("[memstat] purge: mallopt=%s rss %.1f -> %.1f MB",
          (fn != nullptr) ? "ok" : "MISSING", before, after);
+    return after;  // memFloor — the post-purge plateau metric
 }
 
 }  // namespace
@@ -189,6 +195,9 @@ Java_io_imagestitcher_rn_BatchStitcher_nativeStitchFramePaths(
     // re-arms the manual pipeline's dynamic plane→spherical fallback/divergence
     // switch (they only fire when warperType != "spherical").
     cfg.useManualPipeline = (useManualPipeline == JNI_TRUE);
+    // 2026-06-16 — memory profiling (DEV).  Gated by the compile flag (debug-on,
+    // release-off); Android leaves memProbeFn null so rss_mb() uses /proc.
+    cfg.enableMemoryProfiling = (RNIS_MEMORY_PROFILING != 0);
     if (cfg.warperType.empty()) cfg.warperType = "spherical";
     if (cfg.registrationResolMP <= 0.0) {
         cfg.registrationResolMP = 0.6;
@@ -210,13 +219,49 @@ Java_io_imagestitcher_rn_BatchStitcher_nativeStitchFramePaths(
 
     const std::string outPath = jstring_to_string(env, outputPath);
 
-    retailens::StitchResult result = retailens::stitchFramePaths(
-        paths, outPath, cfg, &androidLogBridge);
+    // 2026-06-16 (review #1) — backstop try/catch at the JNI C-ABI boundary.
+    // stitchFramePaths now has its own catch ladders (high-level + manual), so
+    // this should never fire — but a C++ exception crossing into JNI is UB
+    // (std::terminate/SIGABRT), so we NEVER let one through: convert any escape
+    // into a Java exception the Kotlin layer can catch.
+    retailens::StitchResult result;
+    try {
+        result = retailens::stitchFramePaths(
+            paths, outPath, cfg, &androidLogBridge);
+    } catch (const std::exception& e) {
+        throw_runtime(env, std::string("native stitch crashed: ") + e.what());
+        return nullptr;
+    } catch (...) {
+        throw_runtime(env, "native stitch crashed (unknown exception)");
+        return nullptr;
+    }
 
     // Return the stitch's freed native memory to the OS so the native-heap RSS
     // baseline doesn't ratchet up ~10-15 MB per capture (see purgeNativeAllocator).
-    // Applies to BOTH pipelines (they share the OpenCV/bionic allocator).
-    purgeNativeAllocator();
+    // Applies to BOTH pipelines (they share the OpenCV/bionic allocator).  The
+    // post-purge RSS is the leak-plateau "floor" — append it to debugSummary so
+    // it rides the existing nativeLastDebugSummary() path to JS (no new bridge).
+    const double memFloor = purgeNativeAllocator(RNIS_MEMORY_PROFILING != 0);
+    if ((RNIS_MEMORY_PROFILING != 0) && memFloor >= 0.0) {
+        char fbuf[40];
+        std::snprintf(fbuf, sizeof(fbuf), ";memFloor=%.1f", memFloor);
+        if (!result.debugSummary.empty()) result.debugSummary += fbuf;
+        // 2026-06-16 — one authoritative per-stitch memory line to logcat (the
+        // sampler peak otherwise only rides debugSummary to the on-screen
+        // overlay).  pipe/warp/mode lets each line be attributed to a preview
+        // tab: pipe=manual warp=plane mode=panorama = "As captured" primary;
+        // pipe=highlevel warp=plane = HL·Plane; warp=spherical = HL·Sph;
+        // mode=scans = SCANS.  Grep `[memstat] record:` to harvest all of them.
+        LOGI("[memstat] record: pipe=%s warp=%s mode=%s before=%.1f peak=%.1f "
+             "after=%.1f floor=%.1f src=%s frames=%d/%d",
+             cfg.useManualPipeline ? "manual" : "highlevel",
+             cfg.warperType.c_str(),
+             (result.stitchModeUsed == retailens::StitchMode::Scans)
+                 ? "scans" : "panorama",
+             result.memBeforeMB, result.memPeakMB, result.memAfterMB, memFloor,
+             result.memSource.c_str(),
+             result.framesIncluded, result.framesRequested);
+    }
 
     if (!result.success) {
         const std::string msg = "Stitch failed: " + result.errorMessage +

package/android/src/main/cpp/keyframe_gate_jni.cpp

@@ -311,23 +311,30 @@ Java_io_imagestitcher_rn_KeyframeGate_nativeEvaluateWithFrame(
         }
     }
 
-    // Pin the byte[] for the duration of the gate evaluate.  Use
-    // GetPrimitiveArrayCritical (zero-copy, JVM pins the GC) over
-    // GetByteArrayElements (may copy on some VMs) because at 30-60
-    // Hz of 2 MB Y-planes, the copy cost adds up.  Evaluate is
-    // ~1-5 ms so the pin window is short.  Always paired with
-    // ReleasePrimitiveArrayCritical even on the error paths below.
+    // 2026-06-16 (audit #4) — pin the byte[] ONLY to INGEST it.
+    // GetPrimitiveArrayCritical is zero-copy (the JVM pins the GC) — preferred
+    // over GetByteArrayElements (which may copy a 2 MB Y-plane at 30-60 Hz) —
+    // but it blocks the GC for the pin's whole life.  So we keep that life to a
+    // single downscale: ingestWorkingFrame() reads the pinned bytes into an
+    // OWNED working frame, we Release IMMEDIATELY, then evaluateWithWorkingMat()
+    // runs the heavy OpenCV (goodFeaturesToTrack / optical flow) with the pin
+    // already gone — no longer stalling the GC or the frame-rate producer
+    // thread.  Always paired with ReleasePrimitiveArrayCritical, even on errors.
     retailens::KeyframeGateDecision d;
     if (grayBytes && grayWidth > 0 && grayHeight > 0 && grayStride >= grayWidth) {
         void* raw = env->GetPrimitiveArrayCritical(grayBytes, nullptr);
         if (raw) {
-            d = gate(handle)->evaluateWithFrame(
-                pose, planePtr,
+            gate(handle)->ingestWorkingFrame(
                 static_cast<const uint8_t*>(raw),
                 static_cast<int32_t>(grayWidth),
                 static_cast<int32_t>(grayHeight),
                 static_cast<int32_t>(grayStride));
             env->ReleasePrimitiveArrayCritical(grayBytes, raw, JNI_ABORT);
+            // Pin released — heavy OpenCV now runs outside the critical section.
+            d = gate(handle)->evaluateWithWorkingMat(
+                pose, planePtr,
+                static_cast<int32_t>(grayWidth),
+                static_cast<int32_t>(grayHeight));
         } else {
             // GetPrimitiveArrayCritical failed (rare, but defensive).
             // Fall back to pose-only path so we degrade gracefully