react-native-skia-webgpu-video 0.1.0-beta.1 → 0.1.0-beta.3

package/LICENSE

@@ -1,7 +1,6 @@
 MIT License
 
 Copyright (c) 2026 Sebastien Hecart
-Portions Copyright (c) 2024 Azzapp
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

package/README.md

@@ -6,7 +6,7 @@
   <a href="https://github.com/batical/react-native-skia-webgpu-video/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/batical/react-native-skia-webgpu-video/actions/workflows/ci.yml/badge.svg?branch=main" /></a>
   <a href="https://www.npmjs.com/package/react-native-skia-webgpu-video"><img alt="npm" src="https://img.shields.io/npm/v/react-native-skia-webgpu-video/beta" /></a>
   <a href="#status"><img alt="Status: beta" src="https://img.shields.io/badge/status-beta-f59e0b" /></a>
-  <img alt="React Native Skia 3.7.0" src="https://img.shields.io/badge/Skia-3.7.0-38bdf8" />
+  <img alt="React Native Skia 3.7.1" src="https://img.shields.io/badge/Skia-3.7.1-38bdf8" />
   <img alt="React Native WebGPU 0.17.0" src="https://img.shields.io/badge/WebGPU-0.17.0-a78bfa" />
   <a href="LICENSE"><img alt="MIT license" src="https://img.shields.io/badge/license-MIT-34d399" /></a>
 </p>
@@ -27,7 +27,7 @@ A new Skia 3 / WebGPU frame-interop layer, written from scratch with explicit re
 - **Audio compositions** with clip audio, separate tracks, and per-item volume.
 - **Custom GPU processing** through an export frame processor and an optional WebGPU resource scope.
 
-> **Beta.** The current Skia 3.7.0 / WebGPU 0.17.0 build has run on an iPhone 15 Pro and a Pixel 8a (export speed, pixel checks). Not yet qualified: memory over long sessions (last checked on 3.0.6), HDR, audible output and A/V synchronization; two 4K cases have known issues. See [status](#status) before adopting this in production.
+> **Beta.** The Skia 3.7.0 / WebGPU 0.17.0 build has run on an iPhone 15 Pro and a Pixel 8a (export speed, pixel checks); 3.7.1, which only changes iOS presentation on ProMotion displays, passed the native tests, smoke runs and the Pixel pixel check. Not yet qualified: memory over long sessions (last checked on 3.0.6), HDR, audible output and A/V synchronization; two 4K cases have known issues. See [status](#status) before adopting this in production.
 
 ## Installation
 
@@ -37,7 +37,7 @@ A new Skia 3 / WebGPU frame-interop layer, written from scratch with explicit re
 | --- | --- |
 | React Native | `0.86.2` tested; declared peer range `>=0.83 <0.87` |
 | React | `19.2.3` tested |
-| React Native Skia | **`react-native-skia@3.7.0`** |
+| React Native Skia | **`react-native-skia@3.7.1`** |
 | React Native WebGPU | **`react-native-webgpu@0.17.0`** (both resolve `react-native-webgpu-dawn@157.0.0`) |
 | Reanimated / Worklets | **`4.5.3` / `0.11.3`** |
 | Native runtime | New Architecture and Hermes |
@@ -45,12 +45,14 @@ A new Skia 3 / WebGPU frame-interop layer, written from scratch with explicit re
 
 A native rebuild is required. Expo Go cannot load this module. Skia and WebGPU now share Dawn through the `react-native-webgpu-dawn` npm package: `node` must be on the `PATH` during `pod install` and Gradle builds, and both packages must resolve the same version of it. For iOS, use the deployment target required by your React Native and Skia versions; the example targets iOS 16.4 or later.
 
+On ProMotion iPhones, add `CADisableMinimumFrameDurationOnPhone` (Boolean, `YES`) to your app's `Info.plist`: without it, iOS holds every frame callback of the app, the preview's included, at 60 Hz. On Android 15 and later, a playing preview asks the display for its highest refresh rate itself.
+
 ### Install
 
 The beta is published on npm under the `beta` tag:
 
 ```sh
-npm install --save-exact react-native-skia@3.7.0 react-native-webgpu@0.17.0 react-native-reanimated@4.5.3 react-native-worklets@0.11.3
+npm install --save-exact react-native-skia@3.7.1 react-native-webgpu@0.17.0 react-native-reanimated@4.5.3 react-native-worklets@0.11.3
 npm install --save-exact react-native-skia-webgpu-video@beta
 ```
 
@@ -203,14 +205,28 @@ This budget covers **tracked video resources**, not all memory held by codecs, S
 
 | Step | iOS | Android |
 | --- | --- | --- |
-| Decode | AVFoundation, BGRA IOSurface-backed buffers, no extra copy (`alwaysCopiesSampleData = NO`) | API 29+, 8-bit SDR H.264/HEVC at the encoded size: `ImageReader` YUV buffers. Otherwise: EGL render into an RGBA AHardwareBuffer (or a CPU scoped-copy fallback) |
-| Into Skia, preview | WebGPU blit into a reusable texture, then an owned Graphite snapshot | WebGPU blit (YUV→RGB, crop and rotation in one pass), then an owned Graphite snapshot |
-| Into Skia, export | Sampled in place through the frame's IOSurface: no copy, no per-clip GPU wait | Same as preview |
+| Decode | AVFoundation, IOSurface-backed buffers, no extra copy (`alwaysCopiesSampleData = NO`): the decoder's NV12 (`framePixelFormat: 'yuv'`, the default), or BGRA converted by VideoToolbox (`'rgba'`) | API 29+, 8-bit SDR H.264/HEVC at the encoded size: `ImageReader` YUV buffers (the default). Otherwise, or with `'rgba'`: EGL render into an RGBA AHardwareBuffer (or a CPU scoped-copy fallback) |
+| Into Skia, preview | NV12: a YUV→RGB render pass into a reusable texture. BGRA: a WebGPU blit into it. Then an owned Graphite snapshot | WebGPU blit (YUV→RGB, crop and rotation in one pass), then an owned Graphite snapshot |
+| Into Skia, export | NV12: converted into a per-clip texture, as on Android. BGRA: sampled in place through the frame's IOSurface, no copy | Same as preview |
 | Export output | Skia draws into the encoder's CVPixelBuffer (IOSurface) | Skia draws into an AHardwareBuffer lent by the encoder; EGL samples it into the codec Surface after its sync-fd |
 | Fallback | CPU readback, with a warning | CPU readback, with a warning |
 
 Every path waits for checked GPU completion before releasing source frames or handing a buffer to the encoder. Scaled, HDR and 10-bit Android sources keep the EGL decode path.
 
+### Frame pixel format
+
+`framePixelFormat` chooses how a composition's frames are decoded, for its preview and its export:
+
+```ts
+const composition = { duration, items, framePixelFormat: 'yuv' }; // 'auto' (default) | 'yuv' | 'rgba'
+```
+
+- `yuv`: the decoder's own YUV 4:2:0 buffers, converted to RGB on the GPU when a frame is drawn. A decoded frame takes 1.5 bytes a pixel instead of 4 (12.4 MB instead of 33.2 MB in 4K), so the decoder keeps more frames ahead in the same memory.
+- `rgba`: frames converted to RGB as they are decoded (iOS: by VideoToolbox; Android: by an OpenGL pass), which an iOS GPU export samples in place.
+- `auto`: `yuv`, on both platforms.
+
+Both convert with the frame's own YCbCr matrix and range, HDR tone mapped to Rec.709 by the platform. One difference: on iOS, `rgba` also converts a file tagged with other primaries or another SDR transfer function (Display P3, sRGB) to Rec.709, as AVFoundation does; `yuv`, like Android in both formats, takes its samples as Rec.709. Camera footage (Rec.709 or HLG) renders the same in both. See the [measurements](docs/FRAME_PIXEL_FORMAT.md) before choosing.
+
 For advanced exports, `createFrameProcessor` supports asynchronous preparation at each composition timestamp. The optional `react-native-skia-webgpu-video/gpu` entry point provides a resource scope whose asynchronous disposal waits for tracked work. See the [WebGPU cube example](examples/webgpu-overlay.ts).
 
 These are integration building blocks. Three.js adapters, Core ML models, and automatic Neural Engine acceleration are not included. Browser video support is not provided by this native library.
@@ -251,11 +267,12 @@ Reproduce on your devices with the example app: `node scripts/test-ios-device-ab
 
 | Area | Current evidence |
 | --- | --- |
-| iOS | iPhone 15 Pro: all export cases above passed in every run (72/72), with output sizes identical to 3.6.1 ([verification](migration/skia-3.7.0/verification.md)). Native XCTests were last run on the simulator with 3.6.1 (50/50); the iOS native sources have not changed since. |
-| Android | Pixel 8a: CPU vs GPU decode comparison 10/10 cases (max channel error 1, including rotation, portrait and 4K); export checks of a color pattern and of a decoded rotated clip match CPU readback. With Dawn 157 the errors stay within ±1, as with 3.6.1 ([verification](migration/skia-3.7.0/verification.md#pixels)). Instrumented tests were last run on 3.4.4 (97/102; the 5 failures predate it). |
-| JavaScript / types | 217 tests in 10 suites, typecheck, package tests and 112 benchmark-harness tests pass; CI runs them on every pull request. |
+| Frame accuracy | At 60 Hz, the preview draws the frame due at 99–100% of vsyncs in 30, 24 and 60 fps on both phones, and 97–100% in 4K with `yuv` frames on the Pixel 8a. On a warm iPhone at 120 Hz: 73–95% in 1080p, 77–90% in 4K 60 fps. Every exported frame holds the right source frame, in both frame pixel formats ([beta.3 verification](docs/VERIFICATION_0.1.0-beta.3.md)). |
+| iOS | iPhone 15 Pro: 61/61 benchmark cases pass with `yuv` frames, including the three that stopped on the video memory budget with BGRA; 56/58 native XCTests pass, the other two being opt-in diagnostics ([beta.3 verification](docs/VERIFICATION_0.1.0-beta.3.md)). |
+| Android | Pixel 8a: CPU vs GPU decode comparison 10/10 cases (max channel error 1, including rotation, portrait and 4K); export checks of a color pattern and of a decoded rotated clip match CPU readback. With Dawn 157 the errors stay within ±1, as with 3.6.1 ([verification](migration/skia-3.7.0/verification.md#pixels)). Instrumented tests: 126/127 on beta.3 (the failure is a known 4K throughput test, see the [verification](docs/VERIFICATION_0.1.0-beta.3.md#tests-natifs)). |
+| JavaScript / types | 237 tests in 11 suites, typecheck, package tests and 122 benchmark-harness tests pass; CI runs them on every pull request. |
 | Image / audio fidelity | Sampled SDR pixels and an exported color pattern are checked on Android; iOS exported files have the same byte size as with 3.0.6. Whole-image fidelity, HDR, audible output and A/V synchronization remain unqualified. |
-| Known issues | A 4K lazy composition seek times out in 1 of 3 runs on Pixel 8a, and three simultaneous 4K clips stop on the video memory budget during iPhone playback; both happen before and after this release. With `ImageReader`, 1:1 exports are 5–16% larger files for matching sampled pixels; with 3.7.0 they are up to 5.7% smaller than with 3.6.1, cause not confirmed. |
+| Known issues | On Pixel 8a: a seek while playing a 4K sequence leaves the clock stalled, a paused seek into a 120-clip lazy montage does not show its frame, and Android 17 reports a missing file in words the harness does not recognize; all three predate beta.3. An iOS export with `yuv` frames spends 2–4 ms more per frame than with `rgba`. With `ImageReader`, 1:1 exports are 5–16% larger files for matching sampled pixels. |
 | Long sessions | Not yet re-run since 3.0.6; the 3.0.6 results below are the latest. |
 
 ### Earlier results (Skia 3.0.6)
@@ -273,6 +290,10 @@ The eight-clip 4K lifecycle case remained memory intensive: sampled peaks reache
 
 A subsequent export-worklet fix reduces repeated-session allocation overhead. In matched two-montage Release runs, memory after 60 seconds idle fell from **426 to 258 MiB physical footprint on iPhone 15 Pro**, and from **738 to 577 MiB PSS on Pixel 8a**. These are different platform metrics, not a cross-device comparison. Six consecutive iPhone workloads and four Android workloads passed with flatter or decreasing late-run readings; bounded tests do not prove zero leaks. See the [native memory investigation](docs/NATIVE_MEMORY_INVESTIGATION_306.md) for checkpoints and limits.
 
+## Built with it
+
+No projects listed yet. Using this library in an app or an open-source project? [Share it in Show and tell](https://github.com/batical/react-native-skia-webgpu-video/discussions/22) and it may be featured here.
+
 ## Development
 
 ```sh

package/android/BUILD_VALIDATION.md

@@ -1,6 +1,6 @@
 # Android validation and frame ownership
 
-The Android example in `../example/android` pins React Native 0.86.2, Skia 3.7.0, WebGPU 0.17.0, the shared `react-native-webgpu-dawn` 157.0.0 (chrome-m157) and Graphite Android 157.0.0; Gradle needs `node` to resolve that Dawn package. Export renders into encoder-lent AHardwareBuffers (`android/cpp/EncoderTargets.cpp`) sampled into the codec Surface by EGL, with CPU readback as the fallback. On API 29+, 8-bit SDR H.264/HEVC clips decoded at their encoded size go through an `ImageReader`, whose YUV buffers WebGPU converts, crops and rotates; since 3.6.1 each buffer is imported once, and exports sample the converted frame in place. The 3.7.0 build was assembled and run on a Pixel 8a on 2026-10-19 ([verification](../migration/skia-3.7.0/verification.md)), after the 3.6.1 build of 2026-10-16 ([verification](../migration/skia-3.6.1/verification.md)) and the 3.4.4 build of 2026-10-14/15 ([verification](../migration/skia-3.4.4/verification.md)); the evidence below is from Skia 3.0.6 / WebGPU 0.12.1 / Graphite 154.1.0. It uses Java 17, Gradle 9.3.1, Android SDK 36 and NDK 27.1.12297006. The library requires API 28; a consuming application with a lower minimum receives an explicit Gradle configuration error. The default example ABI is arm64-v8a; `RNSKV_ANDROID_ABI` selects another ABI for the validation script.
+The Android example in `../example/android` pins React Native 0.86.2, Skia 3.7.1, WebGPU 0.17.0, the shared `react-native-webgpu-dawn` 157.0.0 (chrome-m157) and Graphite Android 157.0.0; Gradle needs `node` to resolve that Dawn package. Export renders into encoder-lent AHardwareBuffers (`android/cpp/EncoderTargets.cpp`) sampled into the codec Surface by EGL, with CPU readback as the fallback. On API 29+, 8-bit SDR H.264/HEVC clips decoded at their encoded size go through an `ImageReader`, whose YUV buffers WebGPU converts, crops and rotates; since 3.6.1 each buffer is imported once, and exports sample the converted frame in place. The 3.7.0 build was assembled and run on a Pixel 8a on 2026-10-19 ([verification](../migration/skia-3.7.0/verification.md)), after the 3.6.1 build of 2026-10-16 ([verification](../migration/skia-3.6.1/verification.md)) and the 3.4.4 build of 2026-10-14/15 ([verification](../migration/skia-3.4.4/verification.md)); the evidence below is from Skia 3.0.6 / WebGPU 0.12.1 / Graphite 154.1.0. It uses Java 17, Gradle 9.3.1, Android SDK 36 and NDK 27.1.12297006. The library requires API 28; a consuming application with a lower minimum receives an explicit Gradle configuration error. The default example ABI is arm64-v8a; `RNSKV_ANDROID_ABI` selects another ABI for the validation script.
 
 Run `android/scripts/validate.sh build` from the repository. It assembles the example Debug and unsigned Release APKs, library AAR, standalone instrumentation APK and JVM tests. Release uses the installed `hermes-compiler` package, and both native and Metro build concurrency are bounded to two workers. The Release APK is unsigned: device installation requires signing a separate local qualification artifact. The project supplies no publishing or release signing configuration.
 

package/android/cpp/VideoComposition.cpp

@@ -150,6 +150,18 @@ VideoComposition::fromJSIObject(jsi::Runtime& runtime,
   auto composition = VideoComposition::create(duration, items);
   composition->setFieldValue(VideoComposition::javaClassStatic()->getField<jboolean>(
       "hardwareBufferEnabled"), static_cast<jboolean>(NativeHardwareBuffer::isEnabled()));
+  if (jsComposition.hasProperty(runtime, "framePixelFormat")) {
+    auto formatProp = jsComposition.getProperty(runtime, "framePixelFormat");
+    std::string format = formatProp.isString() ? formatProp.asString(runtime).utf8(runtime) : "";
+    if (!formatProp.isUndefined() && format != "auto" && format != "yuv" && format != "rgba")
+      throw jsi::JSError(runtime, "framePixelFormat must be 'auto', 'yuv' or 'rgba'");
+    // 'auto' and 'yuv' are the same on Android: YUV wherever the track allows.
+    if (format == "rgba") {
+      composition->setFieldValue(
+          VideoComposition::javaClassStatic()->getField<jboolean>("rgbaFrames"),
+          (jboolean)JNI_TRUE);
+    }
+  }
   if (jsComposition.hasProperty(runtime, "lazyDecoders")) {
     auto lazyProp = jsComposition.getProperty(runtime, "lazyDecoders");
     if (lazyProp.isBool() && lazyProp.getBool()) {

package/android/cpp/VideoCompositionFramesExtractor.cpp

@@ -54,6 +54,12 @@ jlong VideoCompositionFramesExtractor::getCurrentPosition() const {
   return getCurrentPositionMethod(self());
 }
 
+jlong VideoCompositionFramesExtractor::getPresentedPosition() const {
+  static const auto getPresentedPositionMethod =
+      getClass()->getMethod<jlong()>("getPresentedPosition");
+  return getPresentedPositionMethod(self());
+}
+
 jlong VideoCompositionFramesExtractor::getFramesVersion() const {
   static const auto getFramesVersionMethod =
       getClass()->getMethod<jlong()>("getFramesVersion");

package/android/cpp/VideoCompositionFramesExtractor.h

@@ -33,6 +33,7 @@ public:
   jboolean getIsPlaying() const;
 
   jlong getCurrentPosition() const;
+  jlong getPresentedPosition() const;
 
   jlong getFramesVersion() const;
 

package/android/cpp/VideoCompositionFramesExtractorHostObject.cpp

@@ -172,7 +172,7 @@ jsi::Value VideoCompositionFramesExtractorHostObject::get(
   } else if (propName == "currentTime") {
     std::lock_guard<std::recursive_mutex> lock(playerMutex);
     return {released.test() ? 0
-                            : (double)player->getCurrentPosition() / 1000000.0};
+                            : (double)player->getPresentedPosition() / 1000000.0};
   } else if (propName == "framesVersion") {
     std::lock_guard<std::recursive_mutex> lock(playerMutex);
     return {released.test() || !prepared.test()

package/android/src/main/java/com/azzapp/rnskv/PreviewFrameRateVote.java

@@ -0,0 +1,104 @@
+package com.azzapp.rnskv;
+
+import android.app.Activity;
+import android.os.Build;
+import android.os.SystemClock;
+import android.util.Log;
+import android.view.Display;
+import android.view.View;
+import android.view.ViewGroup;
+
+import com.facebook.react.bridge.ReactApplicationContext;
+
+import java.util.HashSet;
+import java.util.Set;
+
+/**
+ * Asks the display for its highest refresh rate while a preview plays, through
+ * the React Native Skia canvases of the activity (Android 15+). Main thread.
+ *
+ * Android picks the refresh rate from what the app presents. A preview that
+ * presents at 30 fps for a while (30 fps frames drawn as they come) brings a
+ * 60 Hz display down to 30 Hz, and it stays there: the next clip, at 60 or
+ * 24 fps, can only present at 30 fps and never shows the display it needs
+ * more. On a Pixel 8a, one 60 fps frame in two was never shown, and 24 fps
+ * came out irregular on a fifth of its frames. A vote tells the display.
+ *
+ * Only views that draw cast their vote: one on the window's root view is never
+ * counted. The player does not know which canvas shows it, so all of the Skia
+ * TextureView canvases vote; those that do not draw cast nothing. SurfaceView
+ * canvases vote for themselves, and the vote asks what theirs asks: the highest
+ * rate a seamless switch reaches, 120 Hz where the user allows it, so that
+ * what drawFrame animates over the video is not held to 60 Hz. Not the clips'
+ * rate: a 24 fps vote brought the display to 30 Hz, where 24 fps judders.
+ */
+final class PreviewFrameRateVote {
+  private static final String TAG = "PreviewFrameRateVote";
+
+  private static final String SKIA_TEXTURE_VIEW = "com.reactnative.skia.SkiaTextureView";
+
+  // Canvases mounted while playing get the vote at the next refresh.
+  private static final long REFRESH_MILLIS = 1000;
+
+  private static final Set<PreviewFrameRateVote> voting = new HashSet<>();
+
+  private static long lastAppliedMillis = 0;
+
+  /** Whether the player is playing, at each vsync it collects frames on. */
+  void update(boolean playing) {
+    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.VANILLA_ICE_CREAM) {
+      return;
+    }
+    if (playing ? voting.add(this) : voting.remove(this)) {
+      apply();
+    } else if (playing && SystemClock.uptimeMillis() - lastAppliedMillis >= REFRESH_MILLIS) {
+      apply();
+    }
+  }
+
+  /** The player is released. */
+  void withdraw() {
+    update(false);
+  }
+
+  private static void apply() {
+    lastAppliedMillis = SystemClock.uptimeMillis();
+    ReactApplicationContext context = ReactNativeSkiaVideoModule.currentReactApplicationContext();
+    Activity activity = context != null ? context.getCurrentActivity() : null;
+    if (activity == null || activity.getWindow() == null) {
+      return;
+    }
+    float rate = voting.isEmpty() ? View.REQUESTED_FRAME_RATE_CATEGORY_DEFAULT
+      : maxRefreshRate(activity.getWindow().getDecorView().getDisplay());
+    try {
+      vote(activity.getWindow().getDecorView(), rate);
+    } catch (RuntimeException e) {
+      Log.w(TAG, "Could not vote the refresh rate, leaving it to the display", e);
+    }
+  }
+
+  // The highest rate a seamless switch from the current mode reaches.
+  private static float maxRefreshRate(Display display) {
+    if (display == null) {
+      return View.REQUESTED_FRAME_RATE_CATEGORY_HIGH;
+    }
+    Display.Mode mode = display.getMode();
+    float rate = mode.getRefreshRate();
+    for (float alternative : mode.getAlternativeRefreshRates()) {
+      rate = Math.max(rate, alternative);
+    }
+    return rate;
+  }
+
+  private static void vote(View view, float rate) {
+    if (SKIA_TEXTURE_VIEW.equals(view.getClass().getName())) {
+      view.setRequestedFrameRate(rate);
+    }
+    if (view instanceof ViewGroup) {
+      ViewGroup group = (ViewGroup) view;
+      for (int i = 0; i < group.getChildCount(); i++) {
+        vote(group.getChildAt(i), rate);
+      }
+    }
+  }
+}

package/android/src/main/java/com/azzapp/rnskv/VideoComposition.java

@@ -14,6 +14,10 @@ public class VideoComposition {
   private boolean lazyDecoders = false;
   private boolean hardwareBufferEnabled = false;
   public boolean isHardwareBufferEnabled() { return hardwareBufferEnabled; }
+  // framePixelFormat: 'rgba': every frame through the GL path, none through
+  // ImageReader's YUV buffers. Set from the C++ side.
+  private boolean rgbaFrames = false;
+  public boolean isRgbaFrames() { return rgbaFrames; }
 
   public VideoComposition(
     double duration,

package/android/src/main/java/com/azzapp/rnskv/VideoCompositionDecoder.java

@@ -687,10 +687,12 @@ public class VideoCompositionDecoder {
   /**
    * Hardware frames at the encoded size from an 8-bit SDR track skip the GL
    * render pass: the codec writes into ImageReader buffers WebGPU samples.
-   * A capped or explicit size keeps the GL path, which scales while drawing.
+   * A capped or explicit size keeps the GL path, which scales while drawing,
+   * as does `framePixelFormat: 'rgba'`.
    */
   private boolean wantsNativeImages(VideoComposition.Item item, VideoCompositionItemDecoder decoder) {
-    if (!composition.isHardwareBufferEnabled() || !decoder.isEightBitSdr()) return false;
+    if (composition.isRgbaFrames() || !composition.isHardwareBufferEnabled() || !decoder.isEightBitSdr())
+      return false;
     int[] size = FrameSize.of(item.getWidth(), item.getHeight(), item.getMaxLongSide(),
       decoder.getVideoWidth(), decoder.getVideoHeight(), 0);
     return size[0] == decoder.getVideoWidth() && size[1] == decoder.getVideoHeight();

package/android/src/main/java/com/azzapp/rnskv/VideoCompositionFramesExtractor.java

@@ -32,6 +32,7 @@ public class VideoCompositionFramesExtractor {
   private static final int PLAYBACK_LOOP = 4;
   private static final int PLAYBACK_SEEK = 5;
   private static final int PLAYBACK_RELEASE = 6;
+  private static final int PLAYBACK_VSYNC_RELEASE = 7;
 
   private final VideoComposition composition;
 
@@ -57,6 +58,8 @@ public class VideoCompositionFramesExtractor {
   private volatile boolean isPlaying = false;
 
   private volatile long startTime = 0;
+  // The position playback last started or was sought to while playing.
+  private volatile long playStartPosition = 0;
   private volatile long pausePosition = 0;
   private boolean isEOS = false;
   private boolean completeDispatched = false;
@@ -71,6 +74,7 @@ public class VideoCompositionFramesExtractor {
     public void doFrame(long frameTimeNanos) {
       if (releasing) {
         unwatchDisplay();
+        frameRateVote.withdraw();
         return;
       }
       recordedVsyncNanos = frameTimeNanos;
@@ -79,18 +83,21 @@ public class VideoCompositionFramesExtractor {
   };
   // The display the preview is on; null until the first collection, or when unknown.
   private DisplayVsyncPeriod vsyncPeriod;
+  // The display's highest refresh rate while playing.
+  private final PreviewFrameRateVote frameRateVote = new PreviewFrameRateVote();
   // When that thread last released frames; the playback loop stands back meanwhile.
   private volatile long lastVsyncReleaseNanos = 0;
+  // The vsync the frames collected last are shown at; 0 until the first.
+  private volatile long collectedVsyncNanos = 0;
   // The release itself runs on the playback thread: the decoder's locks are held
   // there for whole codec flushes on a seek, and the UI thread must never
   // wait on them (measured: 65 ms frames when it did).
   private volatile long vsyncReleasePosition = 0;
   private final AtomicBoolean vsyncReleasePosted = new AtomicBoolean(false);
-  private final Runnable vsyncRelease = this::runVsyncRelease;
 
-  private void runVsyncRelease() {
+  private void vsyncReleaseInternal() {
     vsyncReleasePosted.set(false);
-    if (!releasing) decoder.render(vsyncReleasePosition);
+    decoder.render(vsyncReleasePosition);
   }
   private static final long VSYNC_RELEASE_TIMEOUT_NANOS = 100_000_000L;
 
@@ -187,11 +194,13 @@ public class VideoCompositionFramesExtractor {
       watchDisplay();
       return;
     }
+    frameRateVote.update(isPlaying);
     long period = vsyncPeriod != null ? vsyncPeriod.getPeriodNanos() : 0;
     if (period == 0 || recordedVsyncNanos == 0) return;
     // This frame's vsync, whether the recorder has run in this frame yet or not.
     long since = Math.max(0, System.nanoTime() - recordedVsyncNanos);
     long vsyncNanos = recordedVsyncNanos + (since / period) * period;
+    collectedVsyncNanos = vsyncNanos;
     if (vsyncNanos == lastVsyncNanos) return;
     lastVsyncNanos = vsyncNanos;
     long vsyncPeriodNanos = period;
@@ -201,7 +210,9 @@ public class VideoCompositionFramesExtractor {
     lastVsyncReleaseNanos = System.nanoTime();
     vsyncReleasePosition = position;
     if (vsyncReleasePosted.compareAndSet(false, true)) {
-      handler.post(vsyncRelease);
+      // A message, not a posted runnable: it goes through handleMessage's
+      // releasing check and turns a throw into an error event.
+      handler.sendEmptyMessage(PLAYBACK_VSYNC_RELEASE);
     }
   }
 
@@ -241,6 +252,27 @@ public class VideoCompositionFramesExtractor {
     return isPlaying ? microTime() - startTime : pausePosition;
   }
 
+  /**
+   * The position the frames collected in this vsync were released for: the
+   * clock at the vsync rather than at the call, a few milliseconds later.
+   * drawFrame gets it as its time, so that the time it draws at and the frames
+   * it draws agree: read at the call, the time ran up to a frame ahead of the
+   * frames, at 60 fps on most vsyncs. Without vsync release (no display, or
+   * a stalled collector), the clock at the call.
+   *
+   * @return the position in microseconds
+   */
+  public long getPresentedPosition() {
+    long vsyncNanos = collectedVsyncNanos;
+    if (!isPlaying || vsyncNanos == 0
+      || System.nanoTime() - lastVsyncReleaseNanos > VSYNC_RELEASE_TIMEOUT_NANOS) {
+      return getCurrentPosition();
+    }
+    // Never before the play or seek that started this run: the vsync can
+    // precede it by up to a period.
+    return Math.max(vsyncNanos / 1000 - startTime, playStartPosition);
+  }
+
   /**
    * @return the version of the frames returned by {@link #decodeCompositionFrames},
    * see {@link VideoCompositionDecoder#getFramesVersion()}
@@ -312,6 +344,7 @@ public class VideoCompositionFramesExtractor {
       pausePosition = 0;
       seekInternal(0);
     }
+    playStartPosition = pausePosition;
     startTime = microTime() - pausePosition;
     isPlaying = true;
     pausePosition = 0;
@@ -379,6 +412,7 @@ public class VideoCompositionFramesExtractor {
       audioPlayer.seekTo(position);
     }
     if (isPlaying) {
+      playStartPosition = position;
       startTime = microTime() - position;
     } else {
       pausePosition = position;
@@ -438,6 +472,10 @@ public class VideoCompositionFramesExtractor {
             releaseInternal();
             return true;
           }
+          case PLAYBACK_VSYNC_RELEASE -> {
+            vsyncReleaseInternal();
+            return true;
+          }
           default -> {
             return false;
           }

package/docs/FRAME_PIXEL_FORMAT.md

@@ -0,0 +1,124 @@
+# YUV ou RGBA : le format des frames décodées
+
+Mesures du 21 octobre 2026 sur **iPhone 15 Pro** (iOS 27.0.1) et **Pixel 8a** (Android 17), builds Release, Skia 3.7.1 / WebGPU 0.17.0. Ce document explique l'option `framePixelFormat`, ce que chaque format coûte et rend, et les pièges trouvés en la mesurant. Le détail des runs est dans le [rapport de vérification beta.3](VERIFICATION_0.1.0-beta.3.md).
+
+## En bref
+
+- **`auto` vaut `yuv` sur les deux plateformes** ; `rgba` reste disponible.
+- **Android** décode en YUV depuis le début (`ImageReader`), et c'est nettement le meilleur choix. Forcer `rgba` (la passe OpenGL) divise par deux à trois la vitesse des exports 4K, et fait perdre près des deux tiers des frames d'une preview 4K 60 fps.
+- **iOS** décodait jusqu'ici en BGRA (VideoToolbox convertit chaque frame). En NV12, une frame 4K pèse **12,4 Mo au lieu de 33,2 Mo**. Le décodeur garde cinq frames d'avance au lieu d'une ou deux, la preview 4K perd moins de frames, et les cas qui dépassaient le budget mémoire passent. Seul coût mesuré : **environ 2 à 4 ms de plus par frame à l'export**, visible sur les exports longs ou à petite sortie.
+- **Le rendu est le même** pour la vidéo de caméra (Rec.709 ou HLG). Un fichier étiqueté Display P3 ou avec la fonction de transfert sRGB est converti en Rec.709 par `rgba`, pas par `yuv`.
+
+## L'option
+
+```ts
+const composition = { duration, items, framePixelFormat: 'yuv' }; // 'auto' | 'yuv' | 'rgba'
+```
+
+| | iOS | Android |
+| --- | --- | --- |
+| `yuv` | AVAssetReader sort le NV12 du décodeur. La preview l'importe comme texture externe à chaque frame et le convertit en RGB dans une passe de rendu ; l'export le convertit dans une texture par clip | `ImageReader` (pistes 8 bits SDR à leur taille encodée ; les autres restent en `rgba`), converti en RGB par la même passe |
+| `rgba` | VideoToolbox convertit en BGRA ; l'export GPU échantillonne l'IOSurface en place, sans copie | Passe OpenGL vers un `AHardwareBuffer` RGBA |
+| `auto` (défaut) | `yuv` | `yuv` |
+
+L'option ne concerne que les compositions (preview et export) : le lecteur d'un clip seul, `useVideoPlayer`, reste en BGRA sur iOS.
+
+La conversion YUV → RGB applique la matrice YCbCr et la plage de chaque buffer, sans convertir la fonction de transfert ni les primaires : c'est ce qui évite d'éclaircir les tons moyens à chaque export (voir `yuvBlit.ts`). Le HDR est d'abord ramené en SDR Rec.709 par la plateforme, dans les deux formats.
+
+## Mesures
+
+### Décodeur seul (iOS)
+
+XCTest `DecodeThroughputTests` : AVAssetReader lit les 360 frames d'un clip HEVC 4K 60 fps, sans React Native ni GPU, en alternant les formats sur trois tours. Téléphone en état thermique *serious*.
+
+| Format | Débit | CPU par frame | Taille d'une frame |
+| --- | --- | --- | --- |
+| BGRA + propriétés Rec.709 | 105–127 fps | 0,42–0,47 ms | 33,2 Mo |
+| NV12 + propriétés Rec.709 | 93–139 fps | 0,45–0,55 ms | 12,4 Mo |
+| NV12 brut | 120–137 fps | 0,47–0,62 ms | 12,4 Mo |
+
+La conversion BGRA est faite par le matériel et ne coûte presque rien en débit. La différence est la **mémoire**, et ce qu'elle permet : la file du décodeur de la preview est plafonnée à 64 Mo, soit une ou deux frames d'avance en BGRA 4K, cinq en NV12.
+
+### Preview
+
+Profil `sync` : chaque frame porte son numéro, lu à chaque vsync. « Exact » : part des vsyncs qui montrent la frame due ; « frames » : frames distinctes affichées.
+
+**Pixel 8a**, 60 Hz, état thermique *nominal* :
+
+| Cas | `yuv` | `rgba` |
+| --- | --- | --- |
+| HEVC 4K 60 fps | **96,9 % exact**, 344/360 frames | 18,4 %, 132/360 |
+| HEVC 4K 30 fps | **100 %**, 179/180 | 38,1 %, 123/180 |
+| H.264 4K 30 fps | **98,8 %**, 177/180 | 38,4 %, 128/180 |
+| Vraie vidéo 4K 60 fps (fleurs, 60 Mbit/s) | **335/360** | 145/360 |
+| HEVC 1080p 60 fps | 99,4 % | 92,7 % |
+| 30 fps, 24 fps, coupe | 99,7–100 % | 99,7–100 % |
+
+**iPhone 15 Pro**, 120 Hz, quatre runs alternés `rgba` / `yuv` / `rgba` / `yuv`, tous en état *serious* (le téléphone n'a pas refroidi entre les runs ; l'alternance répartit cette chaleur sur les deux formats). Deux valeurs par format :
+
+| Cas | `yuv` | `rgba` |
+| --- | --- | --- |
+| HEVC 4K 60 fps (frames, sans lecture du numéro) | **311, 294** /360 | 241, 261 |
+| HEVC 4K 60 fps (exact, une frame sur quatre lue) | **77 %, 90 %** | 46 %, 48 % |
+| Vraie vidéo 4K 60 fps (frames) | **270, 327** /360 | 234, 284 |
+| HEVC 4K 30 fps du benchmark (frames) | **235, 239** /240 | 223, 224 |
+| Vraie vidéo 4K 30 fps (frames) | 173, 178 /180 | 173, 169 |
+| 30, 60 fps et HEVC 1080p 60 fps (exact) | 73–95 % | 71–91 % |
+
+En 1080p et en 30 fps, les écarts entre formats restent dans la dispersion d'un run à l'autre, très forte sur un téléphone chaud.
+
+### Export
+
+Chaque frame exportée a été relue : la bonne frame source dans tous les cas, dans les deux formats, sur les deux téléphones.
+
+| Cas | Pixel `yuv` | Pixel `rgba` | iPhone `yuv` | iPhone `rgba` |
+| --- | --- | --- | --- | --- |
+| 4K 60 → 4K 60 | **5,0 s** | 15,6 s | 4,2–4,3 s | 4,1–4,5 s |
+| 4K 30 → 4K 30 | **3,2 s** | 8,7 s | 2,3 s | 2,4–2,7 s |
+| 4K 60 → 1080p 30 | **2,4–2,9 s** | 8,0–8,2 s | 1,4–2,2 s | 1,7 s |
+| HEVC 1080p 60 → 640×360 | 4,1 s | 8,9 s | 0,9–1,4 s | **0,5 s** |
+| Montage de 120 clips (suite `full`) | **63,5 s** | 95,9 s | 18,1 s | **14,0 s** |
+
+Sur iOS, l'export en `yuv` importe chaque frame NV12 et la convertit dans une texture, là où le BGRA est lu en place. Ce surcoût d'environ 2 à 4 ms par frame est invisible en 4K, où le décodage domine, mais pèse sur les sorties petites ou les exports longs. Pour un export iOS où la vitesse prime sur la mémoire, `framePixelFormat: 'rgba'` reste le plus rapide.
+
+### Mémoire
+
+Pics de la suite `full` (une répétition par format). Pixel : PSS ; iPhone : empreinte physique.
+
+| Cas | Pixel `yuv` | Pixel `rgba` | iPhone `yuv` | iPhone `rgba` |
+| --- | --- | --- | --- | --- |
+| Lecture continue 4K | **438–458 Mio** | 554–602 | **336–343 Mio** | 400–405 |
+| Clip seul 4K | 698–707 | 795–830 | **601–639** | 657–677 |
+| 8 clips 4K, paresseux | **766–785** | 1 020–1 037 | **571–573** | 670–683 |
+| 3 clips 4K simultanés | 1 507 | 1 314 | **734, passe** | arrêté par le budget |
+| 8 clips 4K, eager | 1 562, passe | arrêté par le budget | 1 089, passe | arrêté par le budget |
+
+Les cas « eager » qui s'arrêtaient sur le budget en `rgba` vont jusqu'au bout en `yuv`, d'où une mémoire plus haute sur ceux-là.
+
+## Rendu
+
+Trois contrôles, la même frame décodée dans chaque format :
+
+- **Mire SMPTE** (H.264 et HEVC 1080p, étiquetées Rec.709) comparée à ffmpeg : écart maximal de **0 niveau en `yuv`, 1 en `rgba`**, sur les deux téléphones.
+- **Vraie vidéo** : trois clips tirés de photos d'appareil (panoramique, grain, débits de téléphone, étiquetés Rec.709) comparés à ffmpeg sur 24 points à trois instants. Écart moyen de **0,07 à 0,16 en `yuv`** sur les deux téléphones (max 1). En `rgba` : 0,15 à 0,48 sur le Pixel, 0,5 à 0,8 sur l'iPhone (max 5).
+- **Fixtures du benchmark** (rotations, HDR HLG 10 bits, portrait, 638×358, slow motion) : `yuv` contre `rgba`, écart moyen de 0 à 0,8 niveau. Les maxima de 5 à 17 niveaux sont ponctuels, sur des bords nets entre aplats saturés, où VideoToolbox (ou la passe OpenGL) et la passe GPU n'interpolent pas la chrominance de la même façon.
+
+## Pièges trouvés
+
+**VideoToolbox réétiquette sans convertir.** Avec une sortie NV12, demander les propriétés Rec.709 fait étiqueter le buffer en BT.709 sans convertir les échantillons. Pour une vidéo SD sans étiquette, codée en BT.601, le GPU appliquait alors la mauvaise matrice : le bleu d'un jaune décalé de 11 niveaux, 4,7 en moyenne sur un clip 320×240. Le XCTest `ColorMatrixDiagnosticsTests` le montre : le BGRA de VideoToolbox correspond au NV12 converti en BT.601 (écart 0,4), pas en BT.709 (6,3). Correction : en NV12, seules les pistes HDR ou BT.2020 reçoivent ces propriétés (pour le tone mapping) ; les autres gardent l'étiquette réelle du décodeur, que le GPU applique. Écart après correction : max 1 (638×358) et 3 (320×240), moyenne 0,15 à 0,32.
+
+**Les étiquettes Display P3 et sRGB.** `rgba` convertit vers Rec.709 ce que le fichier déclare (primaires P3, fonction de transfert sRGB) : VideoToolbox sur iOS, la passe OpenGL sur Android. `yuv` n'applique que la matrice et la plage. Sur un clip étiqueté P3 (fixture `tagged-p3-flowers`), `yuv` reste à 0,2 niveau de ffmpeg (qui ignore aussi les primaires), `rgba` s'en écarte de 5,8 en moyenne sur l'iPhone (max 26) et de 1,3 sur le Pixel (max 16). La vidéo de caméra n'est pas concernée ; certains enregistrements d'écran ou exports d'éditeurs le sont.
+
+**Les fixtures héritaient des étiquettes des photos.** ffmpeg recopiait dans les clips de test les primaires P3, la fonction de transfert sRGB ou le profil ICC des photos sources, malgré `-color_primaries bt709`. Le script force désormais les étiquettes avec `setparams` et retire le profil ICC, puis les vérifie avec `ffprobe`.
+
+**La file de décodage iOS tournait en priorité *utility*.** Sur un iPhone chaud, iOS la reléguait sur les cœurs basse consommation : des gels de 300 à 500 ms. Elle tourne désormais en *user-interactive* ; le même run passe de 56 à 244 frames sur 360 en 4K 60 fps.
+
+## Recommandation
+
+Garder `auto` (`yuv`). Choisir `rgba` sur iOS seulement pour :
+- un export où la vitesse compte plus que la mémoire ;
+- des fichiers étiquetés Display P3 ou sRGB dont on veut la conversion colorimétrique.
+
+Pistes pour la suite :
+- réduire le coût de l'import NV12 à l'export sur iOS, par exemple en réutilisant les imports d'un même IOSurface ;
+- mesurer à froid, sur plusieurs répétitions, les écarts 1080p de l'iPhone qui restent dans le bruit.

package/docs/UPSTREAM_NOTES.md

@@ -1,5 +1,7 @@
 # Points amont à revoir lors d'une mise à jour Skia
 
+**Mise à jour du 20 octobre 2026 :** **`react-native-skia@3.7.1`**, toujours avec `react-native-webgpu@0.17.0` et Dawn/Graphite 157.0.0. La 3.7.1 ne contient qu'une correction iOS ([#4273](https://github.com/wcandillon/react-native-skia/pull/4273)) : `SkiaView.mm` présente désormais à la pleine fréquence de l'écran sur les appareils ProMotion. Les sept fichiers modifiés par le patch local sont identiques octet pour octet à ceux de la 3.7.0 : le patch s'applique tel quel, seuls sa version et son nom changent. Nos PR amont [#4274](https://github.com/wcandillon/react-native-skia/pull/4274), [#4275](https://github.com/wcandillon/react-native-skia/pull/4275) et [#4276](https://github.com/wcandillon/react-native-skia/pull/4276) sont encore ouvertes : le patch ne rétrécit pas. Vérifié le 20 octobre sur appareils, en Release : iPhone 15 Pro, tests natifs 50/50 et profil smoke 15/15 (l'app déclare Skia 3.7.1, écran à 120 Hz) ; Pixel 8a, contrôle `interop` 10/10 (erreur maximale par canal de 1, comme en 3.7.0, chemins GPU et readback d'accord à l'export) et smoke 15/15. Les mesures d'export de la 3.7.0 restent la référence : le code d'export ne change pas.
+
 **Migration du 19 octobre 2026 :** le code et les dépendances déclarées ciblent désormais **`react-native-skia@3.7.0` / `react-native-webgpu@0.17.0`**, sur Graphite **157.0.0** et **`react-native-webgpu-dawn@157.0.0`** (Skia `chrome/m157`). Les deux paquets doivent monter ensemble : Gradle et le podspec refusent deux Dawn différents. Le patch local garde 61 de ses 62 éditions ; seule tombe l'include `<stdexcept>`, désormais présent en amont. Le Dawn 157 demande trois images FIFO à la swapchain Android, ce qui supprime l'attente du thread principal dans `dequeueBuffer` sur Android 10 ; il garde aussi la robustesse des shaders sur Android, nos WGSL compris. L'[audit](../migration/skia-3.7.0/upstream-audit.md) détaille ces changements ; la [vérification](../migration/skia-3.7.0/verification.md) donne les mesures sur iPhone 15 Pro et Pixel 8a. Après une mise à jour, supprimer le bundle JS généré par Gradle avant de construire un APK de qualification : sa tâche ne voit pas le `node_modules` racine.
 
 **Migration du 16 octobre 2026 :** le code et les dépendances déclarées ciblaient alors **`react-native-skia@3.6.1` / `react-native-webgpu@0.16.0`**, toujours sur Graphite **154.3.0** et **`react-native-webgpu-dawn@154.3.0`** : les binaires Skia et Dawn ne changent pas. L'[audit](../migration/skia-3.6.1/upstream-audit.md) détaille les changements amont, la régression de couleurs des frames décodées Android introduite par WebGPU 0.15 et sa correction, puis l'import en place à l'export Android. La [vérification](../migration/skia-3.6.1/verification.md) donne les mesures sur iPhone 15 Pro et Pixel 8a. **À chaque mise à jour WebGPU, relancer le contrôle `interop` du Pixel** : il compare les pixels décodés et exportés au chemin CPU, et c'est lui qui a révélé cette régression.