expo-video-subtitle 0.8.0 → 0.8.1

package/CHANGELOG.md

@@ -7,6 +7,41 @@ Changes inherited from the upstream base version are not repeated here — see t
 [upstream changelog](https://github.com/expo/expo/blob/main/packages/expo-video/CHANGELOG.md)
 for the history of everything that isn't subtitle related.
 
+## 0.8.1
+
+### Fixed
+
+- **A plain `.vtt` track sat flush against the bottom edge and ignored `bottomOffset` (Android).**
+  `.srt` and `.ass` tracks in the same player were fine.
+
+  Media3's `WebvttCueParser` leaves a cue with no `line:` setting unpositioned, but that is not what
+  reaches the screen. `WebvttParser.parse` hands its cues through `WebvttSubtitle.getCues`, which
+  runs the WebVTT spec's snap-to-lines stacking on every cue, unconditionally — a lone cue comes out
+  as `line = -1, LINE_TYPE_NUMBER`. `SubtitlePainter` then takes its number-line branch, where `-1`
+  is `parentBottom - textHeight` and `bottomPaddingFraction` is never read; the painter only consults
+  it when `line` is unset. So a WebVTT cue landed `parentHeight × padding` lower than the identical
+  SubRip cue — 54px on a 1080-tall frame at the app's 5% — and no offset could move it.
+
+  SubRip never had this because `SubripParser` leaves `line` alone. ASS had it and was already fixed:
+  `SsaCueNormalizer` strips the placement `SsaParser` stamps on. WebVTT was the one format with the
+  stamp and no normalizer. It now goes through `StackingSubtitleParser` behind `WebvttCueNormalizer`,
+  which clears a negative line number and the derived centre `position`, so a plain cue takes the
+  app's padding and simultaneous cues stack — exactly as `.srt` does.
+
+  Verified against the 1.9.0 artefacts the app ships, from source and from decompiled bytecode; the
+  logic is byte for byte identical to 1.8.0, so this was never a version question.
+
+  One honest limit: after `getCues` has run, a parser-stacked `line = -1` is indistinguishable from
+  an author's explicit `line:-1`, so both are reset. `line:-1` means "the last line", and "the last
+  line, lifted by the app's padding" is where every other format's bottom cue already goes;
+  percentages and non-negative line numbers are untouched.
+
+### Why it surfaced now
+
+Nothing changed in this path for a long time. Until `SUBTITLE_PRESERVE_FORMAT` was switched on, every
+anikoto sidecar was converted to SubRip before it reached the app, so the WebVTT path was never
+exercised. Storing the `.vtt` as-is is what put the first real WebVTT track in front of it.
+
 ## 0.8.0
 
 ### Added

package/FORK.md

@@ -29,6 +29,7 @@ Measured against the published `expo-video@56.1.4` tarball. `+`/`-` are added/re
 | `android/…/records/SubtitleStyle.kt` | 152 | Record → `CaptionStyleCompat`; hex colour parsing; font-family resolution (`res/font` → `ReactFontManager`); the `applyEmbeddedStyles` flag, which `SubtitleUtils` reads directly rather than `toCaptionStyle` and which defaults to `true` since 0.5.0. |
 | `android/…/utils/StackingSubtitleParser.kt` | 342 | `SubtitleParser` that segments the cue timeline and merges simultaneous **unpositioned** cues into one stacked block, plus the `SubtitleParser.Factory` that routes SubRip and ASS/SSA to it and wraps every parser in `FilteringSubtitleParser`. ASS gets there through `SsaCueNormalizer` as the `prepare` step — without it `SsaParser` sets both `line` and `position` on every cue and they would all take the passthrough branch. Assigns each cue a vertical slot it keeps for its whole time on screen — a finished cue's line is held blank (`\u200B`) so the cues above it do not drop. Merges through a `SpannableStringBuilder` so cue spans survive. |
 | `android/…/utils/SsaCueNormalizer.kt` | 93 | Clears the `line`/`position` Media3's `SsaParser` fills in on cues the ASS author never placed, so `bottomOffset` and cue stacking work on `.ass` the way they already do on `.srt`. The test is exact rather than heuristic: those values are a pure function of the cue's anchors, so matching both defaults for the bottom-centre anchors *is* "no positioning tag". Also drops events on a style the author made invisible (`Fontsize 0`, or a primary colour whose inverted alpha byte is `FF`) — that is where typesetters park working notes, and nothing downstream can carry "invisible". |
+| `android/…/utils/WebvttCueNormalizer.kt` | 61 | Clears the `line = -1 - i` that Media3's `WebvttSubtitle.getCues` stamps on every unpositioned WebVTT cue, plus the derived centre `position`, so `.vtt` takes the app's bottom padding and stacks like `.srt`. **Not exact the way the SSA one is**: once `getCues` has run, a parser-stacked `-1` and an author's `line:-1` are the same cue, so both are reset — a defensible reading, since `line:-1` means "the last line". Percentages and non-negative numbers are untouched. |
 | `android/…/utils/AssSignParser.kt` | 505 | Splits an ASS script into the typeset events this package draws and a copy of the script with those lines removed for Media3's parser. One pass, no matching step, so an event is in exactly one of the two outputs and cannot be drawn twice. Pure Kotlin with no Android imports, which is what makes it unit-testable on the JVM. Reads `\pos`, `\move`, `\org`, `\an`, `\frz`, `\frx`, `\fry`, `\fscx/y`, `\fs`, `\c`, `\alpha`, `\b`, `\i`, `\u` and the style columns behind them. The tag scanner takes the longest name first — `\c` must not swallow `\clip`, `\b` must not swallow `\bord`, `\blur` or `\be` — and reads a leading digit as part of the name for `\1c` and `\4a`. |
 | `android/…/utils/AssSignOverlay.kt` | 330 | Draws those signs on a canvas in front of the captions, with `Matrix` for rotation and scale and `Camera` for the two 3-D axes. Drives itself from the player's clock on a `Choreographer` callback, because signs do not ride the cue stream and nothing else would tell it when one starts; 40 ms events need frame granularity anyway. **Media3 cannot do this**: `SubtitlePainter` has no rotate, skew, Matrix or Camera call, and `Cue` has no rotation field. Sits in the same parent as `SubtitleView` — the content frame, sized to the video, which is why `\pos` survives letterboxing. Lays text out on change rather than per frame, because frame-by-frame typesetting changes the on-screen set every 40 ms. |
 | `android/…/utils/AssSignStore.kt` | 61 | Hands parsed signs from the parser to the overlay, keyed by the track's `Format.id`. **Not** the cue stream: `CueEncoder` marshals every cue through `Parcel`, and a custom span that is neither a framework `ParcelableSpan` nor one of Media3's three `CustomSpanBundler` types is dropped without a word — which is how 0.7.0 shipped with signs that parsed correctly and never appeared. Staying out of the cue stream also keeps 12,616 sign events from putting ~26,000 boundaries through the O(n²) timeline pass. |

package/android/src/main/java/expo/modules/video/utils/StackingSubtitleParser.kt

@@ -364,8 +364,8 @@ class StackingSsaParser(
 }
 
 /**
- * Routes SubRip and ASS/SSA through the stacking parsers so simultaneous cues stack, delegates every
- * other subtitle format to Media3's default handling, and wraps the result in
+ * Routes SubRip, ASS/SSA and WebVTT through the stacking parsers so simultaneous cues stack and the
+ * app's bottom padding applies, delegates every other subtitle format to Media3's default handling, and wraps the result in
  * [FilteringSubtitleParser] so the appearance a track asks for never overrides the app's own
  * subtitle style — except for cues the track positions itself, which keep theirs.
  */
@@ -377,8 +377,8 @@ class ExpoVideoSubtitleParserFactory : SubtitleParser.Factory {
     format.isSubrip() || defaultFactory.supportsFormat(format)
 
   override fun getCueReplacementBehavior(format: Format): Int =
-    if (format.isSubrip() || format.isSsa()) {
-      // Both stacking parsers emit whole-screen segments, whatever the delegate's own behaviour is.
+    if (format.isSubrip() || format.isSsa() || format.isWebvtt()) {
+      // Every stacking parser emits whole-screen segments, whatever the delegate's own behaviour is.
       Format.CUE_REPLACEMENT_BEHAVIOR_REPLACE
     } else {
       defaultFactory.getCueReplacementBehavior(format)
@@ -393,6 +393,13 @@ class ExpoVideoSubtitleParserFactory : SubtitleParser.Factory {
       StackingSsaParser(defaultFactory.create(format), format.id),
       keepAppearanceOfPositionedCues = true,
     )
+    // WebVTT gets the stacking parser too, with the placement Media3 stamps on every unpositioned
+    // cue taken back off first. Without that, a plain `.vtt` sat flush against the bottom edge and
+    // ignored `bottomOffset` entirely — see WebvttCueNormalizer. Not the sign carve-out: a
+    // positioned WebVTT cue is a cue setting on ordinary dialogue, not typesetting.
+    format.isWebvtt() -> FilteringSubtitleParser(
+      StackingSubtitleParser(defaultFactory.create(format), WebvttCueNormalizer::normalize)
+    )
     else -> FilteringSubtitleParser(defaultFactory.create(format))
   }
 
@@ -401,4 +408,7 @@ class ExpoVideoSubtitleParserFactory : SubtitleParser.Factory {
 
   private fun Format.isSsa(): Boolean =
     MimeTypes.TEXT_SSA.equals(sampleMimeType, ignoreCase = true)
+
+  private fun Format.isWebvtt(): Boolean =
+    MimeTypes.TEXT_VTT.equals(sampleMimeType, ignoreCase = true)
 }

package/android/src/main/java/expo/modules/video/utils/WebvttCueNormalizer.kt

@@ -0,0 +1,64 @@
+package expo.modules.video.utils
+
+import androidx.annotation.OptIn
+import androidx.media3.common.text.Cue
+import androidx.media3.common.util.UnstableApi
+
+/**
+ * Undoes the placement Media3's WebVTT path stamps on every cue that had none, so a `.vtt` track
+ * behaves like a `.srt` one: it takes the app's bottom padding, and simultaneous cues stack.
+ *
+ * ### What Media3 does
+ *
+ * `WebvttCueParser` leaves a cue with no `line:` setting at `line = DIMEN_UNSET`. That is not what
+ * reaches the screen. `WebvttParser.parse` wraps its cues in a `WebvttSubtitle` and hands them
+ * through `LegacySubtitleUtil`, and `WebvttSubtitle.getCues` runs the spec's snap-to-lines stacking
+ * on every query, unconditionally:
+ *
+ * ```java
+ * // Steps 4 - 10 of https://www.w3.org/TR/webvtt1/#cue-computed-line
+ * currentCues.add(cue.buildUpon().setLine((float) (-1 - i), Cue.LINE_TYPE_NUMBER).build());
+ * ```
+ *
+ * A lone cue gets `line = -1`. `SubtitlePainter` then takes its `LINE_TYPE_NUMBER` branch, where
+ * `line = -1` is `parentBottom - textHeight` — flush against the bottom edge — and
+ * `bottomPaddingFraction` is never read; the painter only consults it when `line` is unset. So a
+ * plain WebVTT track sits `parentHeight × padding` lower than the identical SubRip track, and
+ * `SubtitleSource.bottomOffset` has no effect on it at all. Verified against the 1.9.0 artefacts the
+ * app ships, byte for byte identical to 1.8.0.
+ *
+ * `position` gets the same treatment: with no setting, `derivePosition(CENTER)` yields `0.5` with
+ * `ANCHOR_TYPE_MIDDLE`. That is harmless for the painter but it makes [StackingSubtitleParser] read
+ * the cue as author-placed and pass it through unstacked.
+ *
+ * ### Why this cannot be exact, and why that is acceptable
+ *
+ * [SsaCueNormalizer] can recognise "the parser filled this in" precisely, because Media3's SSA
+ * defaults are a pure function of the anchors. WebVTT's are not: once `getCues` has run, a cue the
+ * parser stacked to `line = -1` is identical to one whose author wrote `line:-1`. Both are reset
+ * here. That is a defensible reading rather than a loss — `line:-1` means "the last line", and
+ * "the last line, lifted by the app's padding" is where every other format's bottom cue already
+ * goes. An author who wants an exact placement writes a percentage, and percentages are untouched.
+ *
+ * Only negative line *numbers* qualify. A fraction (`line:80%`), a non-negative number (`line:3`,
+ * counted from the top) or an explicit `position` off centre is a placement someone chose.
+ */
+@OptIn(UnstableApi::class)
+internal object WebvttCueNormalizer {
+  private const val DEFAULT_POSITION = 0.5f
+  private const val EPSILON = 0.0001f
+
+  fun normalize(cue: Cue): Cue {
+    val stackedLine = cue.lineType == Cue.LINE_TYPE_NUMBER && cue.line != Cue.DIMEN_UNSET && cue.line < 0f
+    val derivedPosition = cue.positionAnchor == Cue.ANCHOR_TYPE_MIDDLE &&
+      cue.position != Cue.DIMEN_UNSET &&
+      kotlin.math.abs(cue.position - DEFAULT_POSITION) < EPSILON
+
+    if (!stackedLine && !derivedPosition) return cue
+
+    val builder = cue.buildUpon()
+    if (stackedLine) builder.setLine(Cue.DIMEN_UNSET, Cue.LINE_TYPE_FRACTION)
+    if (derivedPosition) builder.setPosition(Cue.DIMEN_UNSET)
+    return builder.build()
+  }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "expo-video-subtitle",
-  "version": "0.8.0",
+  "version": "0.8.1",
   "description": "A cross-platform, performant video component for React Native and Expo with external subtitle sideloading and custom subtitle styling.",
   "keywords": [
     "expo",