expo-video-subtitle 0.7.6 → 0.8.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,86 @@ Changes inherited from the upstream base version are not repeated here — see t
7
7
  [upstream changelog](https://github.com/expo/expo/blob/main/packages/expo-video/CHANGELOG.md)
8
8
  for the history of everything that isn't subtitle related.
9
9
 
10
+ ## 0.8.1
11
+
12
+ ### Fixed
13
+
14
+ - **A plain `.vtt` track sat flush against the bottom edge and ignored `bottomOffset` (Android).**
15
+ `.srt` and `.ass` tracks in the same player were fine.
16
+
17
+ Media3's `WebvttCueParser` leaves a cue with no `line:` setting unpositioned, but that is not what
18
+ reaches the screen. `WebvttParser.parse` hands its cues through `WebvttSubtitle.getCues`, which
19
+ runs the WebVTT spec's snap-to-lines stacking on every cue, unconditionally — a lone cue comes out
20
+ as `line = -1, LINE_TYPE_NUMBER`. `SubtitlePainter` then takes its number-line branch, where `-1`
21
+ is `parentBottom - textHeight` and `bottomPaddingFraction` is never read; the painter only consults
22
+ it when `line` is unset. So a WebVTT cue landed `parentHeight × padding` lower than the identical
23
+ SubRip cue — 54px on a 1080-tall frame at the app's 5% — and no offset could move it.
24
+
25
+ SubRip never had this because `SubripParser` leaves `line` alone. ASS had it and was already fixed:
26
+ `SsaCueNormalizer` strips the placement `SsaParser` stamps on. WebVTT was the one format with the
27
+ stamp and no normalizer. It now goes through `StackingSubtitleParser` behind `WebvttCueNormalizer`,
28
+ which clears a negative line number and the derived centre `position`, so a plain cue takes the
29
+ app's padding and simultaneous cues stack — exactly as `.srt` does.
30
+
31
+ Verified against the 1.9.0 artefacts the app ships, from source and from decompiled bytecode; the
32
+ logic is byte for byte identical to 1.8.0, so this was never a version question.
33
+
34
+ One honest limit: after `getCues` has run, a parser-stacked `line = -1` is indistinguishable from
35
+ an author's explicit `line:-1`, so both are reset. `line:-1` means "the last line", and "the last
36
+ line, lifted by the app's padding" is where every other format's bottom cue already goes;
37
+ percentages and non-negative line numbers are untouched.
38
+
39
+ ### Why it surfaced now
40
+
41
+ Nothing changed in this path for a long time. Until `SUBTITLE_PRESERVE_FORMAT` was switched on, every
42
+ anikoto sidecar was converted to SubRip before it reached the app, so the WebVTT path was never
43
+ exercised. Storing the `.vtt` as-is is what put the first real WebVTT track in front of it.
44
+
45
+ ## 0.8.0
46
+
47
+ ### Added
48
+
49
+ - **ASS signs are drawn with their transforms on iOS too.** AVFoundation renders a sideloaded
50
+ WebVTT track itself, and WebVTT can express no rotation, no scale and no per-cue opacity;
51
+ `AVTextStyleRule` has no attribute for any of them either. So the same split the Android side
52
+ makes: `SubtitleConverter.splitAss` takes the typeset events out of the script before the WebVTT
53
+ is built, and `AssSignOverlayView` draws them in `AVPlayerViewController.contentOverlayView`,
54
+ sized to `videoBounds` and driven by a `CADisplayLink`.
55
+
56
+ Signs travel on the `AVPlayerItem` that owns them rather than through a store of their own. A
57
+ store would have to be keyed by something both ends can see, and the only identity a sideloaded
58
+ track keeps inside a composition is its language — so two players loading different videos in the
59
+ same language would overwrite each other, and the next episode would inherit the previous one's
60
+ typesetting. The item is the identity, and the overlay re-resolves whenever the item or the
61
+ selected language changes.
62
+
63
+ Read and honoured: `\pos`, `\move` (at its destination), `\org`, `\an` and the legacy `\a`,
64
+ `\frz`, `\fscx`, `\fscy`, `\fs`, `\c` / `\1c`, `\alpha` / `\1a`, `\b`, `\i`, `\u`, and the
65
+ style columns behind them.
66
+
67
+ ### Known limitations on iOS
68
+
69
+ - `\frx` and `\fry` are not applied. `CGAffineTransform` is two-dimensional, and the values are
70
+ almost always under 3° in practice; `\frz` is what carries the effect.
71
+ - Signs do not appear in Picture in Picture. The system renders that window from the video layer
72
+ alone, not from the view hierarchy `contentOverlayView` belongs to.
73
+ - `\t` animation, `\fad`, `\clip`, `\blur`, `\be`, `\fax` shear, vector drawings and karaoke are
74
+ dropped, as on Android. Fonts fall back to the system face.
75
+
76
+ ### Fixed
77
+
78
+ - **Non-finite numbers are refused.** `Double("nan")` and `Double("inf")` parse, and a subtitle is
79
+ untrusted scraper output; either one reaching a transform makes a sign silently fail to draw,
80
+ which looks exactly like a sign that was never parsed. Every number out of a tag or a style column
81
+ now goes through a finiteness check.
82
+
83
+ ### Changed
84
+
85
+ - The override-block scanner reads a leading digit as part of a tag name, so `\1c`, `\1a` and `\4a`
86
+ are recognised. The WebVTT path ignores them as before — 91 existing checks confirm its output is
87
+ unchanged — but the sign path needs them.
88
+ - The Swift harness is at 119 checks, up from 91.
89
+
10
90
  ## 0.7.6
11
91
 
12
92
  ### Fixed
package/FORK.md CHANGED
@@ -29,6 +29,7 @@ Measured against the published `expo-video@56.1.4` tarball. `+`/`-` are added/re
29
29
  | `android/…/records/SubtitleStyle.kt` | 152 | Record → `CaptionStyleCompat`; hex colour parsing; font-family resolution (`res/font` → `ReactFontManager`); the `applyEmbeddedStyles` flag, which `SubtitleUtils` reads directly rather than `toCaptionStyle` and which defaults to `true` since 0.5.0. |
30
30
  | `android/…/utils/StackingSubtitleParser.kt` | 342 | `SubtitleParser` that segments the cue timeline and merges simultaneous **unpositioned** cues into one stacked block, plus the `SubtitleParser.Factory` that routes SubRip and ASS/SSA to it and wraps every parser in `FilteringSubtitleParser`. ASS gets there through `SsaCueNormalizer` as the `prepare` step — without it `SsaParser` sets both `line` and `position` on every cue and they would all take the passthrough branch. Assigns each cue a vertical slot it keeps for its whole time on screen — a finished cue's line is held blank (`\u200B`) so the cues above it do not drop. Merges through a `SpannableStringBuilder` so cue spans survive. |
31
31
  | `android/…/utils/SsaCueNormalizer.kt` | 93 | Clears the `line`/`position` Media3's `SsaParser` fills in on cues the ASS author never placed, so `bottomOffset` and cue stacking work on `.ass` the way they already do on `.srt`. The test is exact rather than heuristic: those values are a pure function of the cue's anchors, so matching both defaults for the bottom-centre anchors *is* "no positioning tag". Also drops events on a style the author made invisible (`Fontsize 0`, or a primary colour whose inverted alpha byte is `FF`) — that is where typesetters park working notes, and nothing downstream can carry "invisible". |
32
+ | `android/…/utils/WebvttCueNormalizer.kt` | 61 | Clears the `line = -1 - i` that Media3's `WebvttSubtitle.getCues` stamps on every unpositioned WebVTT cue, plus the derived centre `position`, so `.vtt` takes the app's bottom padding and stacks like `.srt`. **Not exact the way the SSA one is**: once `getCues` has run, a parser-stacked `-1` and an author's `line:-1` are the same cue, so both are reset — a defensible reading, since `line:-1` means "the last line". Percentages and non-negative numbers are untouched. |
32
33
  | `android/…/utils/AssSignParser.kt` | 505 | Splits an ASS script into the typeset events this package draws and a copy of the script with those lines removed for Media3's parser. One pass, no matching step, so an event is in exactly one of the two outputs and cannot be drawn twice. Pure Kotlin with no Android imports, which is what makes it unit-testable on the JVM. Reads `\pos`, `\move`, `\org`, `\an`, `\frz`, `\frx`, `\fry`, `\fscx/y`, `\fs`, `\c`, `\alpha`, `\b`, `\i`, `\u` and the style columns behind them. The tag scanner takes the longest name first — `\c` must not swallow `\clip`, `\b` must not swallow `\bord`, `\blur` or `\be` — and reads a leading digit as part of the name for `\1c` and `\4a`. |
33
34
  | `android/…/utils/AssSignOverlay.kt` | 330 | Draws those signs on a canvas in front of the captions, with `Matrix` for rotation and scale and `Camera` for the two 3-D axes. Drives itself from the player's clock on a `Choreographer` callback, because signs do not ride the cue stream and nothing else would tell it when one starts; 40 ms events need frame granularity anyway. **Media3 cannot do this**: `SubtitlePainter` has no rotate, skew, Matrix or Camera call, and `Cue` has no rotation field. Sits in the same parent as `SubtitleView` — the content frame, sized to the video, which is why `\pos` survives letterboxing. Lays text out on change rather than per frame, because frame-by-frame typesetting changes the on-screen set every 40 ms. |
34
35
  | `android/…/utils/AssSignStore.kt` | 61 | Hands parsed signs from the parser to the overlay, keyed by the track's `Format.id`. **Not** the cue stream: `CueEncoder` marshals every cue through `Parcel`, and a custom span that is neither a framework `ParcelableSpan` nor one of Media3's three `CustomSpanBundler` types is dropped without a word — which is how 0.7.0 shipped with signs that parsed correctly and never appeared. Staying out of the cue stream also keeps 12,616 sign events from putting ~26,000 boundaries through the O(n²) timeline pass. |
@@ -364,8 +364,8 @@ class StackingSsaParser(
364
364
  }
365
365
 
366
366
  /**
367
- * Routes SubRip and ASS/SSA through the stacking parsers so simultaneous cues stack, delegates every
368
- * other subtitle format to Media3's default handling, and wraps the result in
367
+ * Routes SubRip, ASS/SSA and WebVTT through the stacking parsers so simultaneous cues stack and the
368
+ * app's bottom padding applies, delegates every other subtitle format to Media3's default handling, and wraps the result in
369
369
  * [FilteringSubtitleParser] so the appearance a track asks for never overrides the app's own
370
370
  * subtitle style — except for cues the track positions itself, which keep theirs.
371
371
  */
@@ -377,8 +377,8 @@ class ExpoVideoSubtitleParserFactory : SubtitleParser.Factory {
377
377
  format.isSubrip() || defaultFactory.supportsFormat(format)
378
378
 
379
379
  override fun getCueReplacementBehavior(format: Format): Int =
380
- if (format.isSubrip() || format.isSsa()) {
381
- // Both stacking parsers emit whole-screen segments, whatever the delegate's own behaviour is.
380
+ if (format.isSubrip() || format.isSsa() || format.isWebvtt()) {
381
+ // Every stacking parser emits whole-screen segments, whatever the delegate's own behaviour is.
382
382
  Format.CUE_REPLACEMENT_BEHAVIOR_REPLACE
383
383
  } else {
384
384
  defaultFactory.getCueReplacementBehavior(format)
@@ -393,6 +393,13 @@ class ExpoVideoSubtitleParserFactory : SubtitleParser.Factory {
393
393
  StackingSsaParser(defaultFactory.create(format), format.id),
394
394
  keepAppearanceOfPositionedCues = true,
395
395
  )
396
+ // WebVTT gets the stacking parser too, with the placement Media3 stamps on every unpositioned
397
+ // cue taken back off first. Without that, a plain `.vtt` sat flush against the bottom edge and
398
+ // ignored `bottomOffset` entirely — see WebvttCueNormalizer. Not the sign carve-out: a
399
+ // positioned WebVTT cue is a cue setting on ordinary dialogue, not typesetting.
400
+ format.isWebvtt() -> FilteringSubtitleParser(
401
+ StackingSubtitleParser(defaultFactory.create(format), WebvttCueNormalizer::normalize)
402
+ )
396
403
  else -> FilteringSubtitleParser(defaultFactory.create(format))
397
404
  }
398
405
 
@@ -401,4 +408,7 @@ class ExpoVideoSubtitleParserFactory : SubtitleParser.Factory {
401
408
 
402
409
  private fun Format.isSsa(): Boolean =
403
410
  MimeTypes.TEXT_SSA.equals(sampleMimeType, ignoreCase = true)
411
+
412
+ private fun Format.isWebvtt(): Boolean =
413
+ MimeTypes.TEXT_VTT.equals(sampleMimeType, ignoreCase = true)
404
414
  }
@@ -0,0 +1,64 @@
1
+ package expo.modules.video.utils
2
+
3
+ import androidx.annotation.OptIn
4
+ import androidx.media3.common.text.Cue
5
+ import androidx.media3.common.util.UnstableApi
6
+
7
+ /**
8
+ * Undoes the placement Media3's WebVTT path stamps on every cue that had none, so a `.vtt` track
9
+ * behaves like a `.srt` one: it takes the app's bottom padding, and simultaneous cues stack.
10
+ *
11
+ * ### What Media3 does
12
+ *
13
+ * `WebvttCueParser` leaves a cue with no `line:` setting at `line = DIMEN_UNSET`. That is not what
14
+ * reaches the screen. `WebvttParser.parse` wraps its cues in a `WebvttSubtitle` and hands them
15
+ * through `LegacySubtitleUtil`, and `WebvttSubtitle.getCues` runs the spec's snap-to-lines stacking
16
+ * on every query, unconditionally:
17
+ *
18
+ * ```java
19
+ * // Steps 4 - 10 of https://www.w3.org/TR/webvtt1/#cue-computed-line
20
+ * currentCues.add(cue.buildUpon().setLine((float) (-1 - i), Cue.LINE_TYPE_NUMBER).build());
21
+ * ```
22
+ *
23
+ * A lone cue gets `line = -1`. `SubtitlePainter` then takes its `LINE_TYPE_NUMBER` branch, where
24
+ * `line = -1` is `parentBottom - textHeight` — flush against the bottom edge — and
25
+ * `bottomPaddingFraction` is never read; the painter only consults it when `line` is unset. So a
26
+ * plain WebVTT track sits `parentHeight × padding` lower than the identical SubRip track, and
27
+ * `SubtitleSource.bottomOffset` has no effect on it at all. Verified against the 1.9.0 artefacts the
28
+ * app ships, byte for byte identical to 1.8.0.
29
+ *
30
+ * `position` gets the same treatment: with no setting, `derivePosition(CENTER)` yields `0.5` with
31
+ * `ANCHOR_TYPE_MIDDLE`. That is harmless for the painter but it makes [StackingSubtitleParser] read
32
+ * the cue as author-placed and pass it through unstacked.
33
+ *
34
+ * ### Why this cannot be exact, and why that is acceptable
35
+ *
36
+ * [SsaCueNormalizer] can recognise "the parser filled this in" precisely, because Media3's SSA
37
+ * defaults are a pure function of the anchors. WebVTT's are not: once `getCues` has run, a cue the
38
+ * parser stacked to `line = -1` is identical to one whose author wrote `line:-1`. Both are reset
39
+ * here. That is a defensible reading rather than a loss — `line:-1` means "the last line", and
40
+ * "the last line, lifted by the app's padding" is where every other format's bottom cue already
41
+ * goes. An author who wants an exact placement writes a percentage, and percentages are untouched.
42
+ *
43
+ * Only negative line *numbers* qualify. A fraction (`line:80%`), a non-negative number (`line:3`,
44
+ * counted from the top) or an explicit `position` off centre is a placement someone chose.
45
+ */
46
+ @OptIn(UnstableApi::class)
47
+ internal object WebvttCueNormalizer {
48
+ private const val DEFAULT_POSITION = 0.5f
49
+ private const val EPSILON = 0.0001f
50
+
51
+ fun normalize(cue: Cue): Cue {
52
+ val stackedLine = cue.lineType == Cue.LINE_TYPE_NUMBER && cue.line != Cue.DIMEN_UNSET && cue.line < 0f
53
+ val derivedPosition = cue.positionAnchor == Cue.ANCHOR_TYPE_MIDDLE &&
54
+ cue.position != Cue.DIMEN_UNSET &&
55
+ kotlin.math.abs(cue.position - DEFAULT_POSITION) < EPSILON
56
+
57
+ if (!stackedLine && !derivedPosition) return cue
58
+
59
+ val builder = cue.buildUpon()
60
+ if (stackedLine) builder.setLine(Cue.DIMEN_UNSET, Cue.LINE_TYPE_FRACTION)
61
+ if (derivedPosition) builder.setPosition(Cue.DIMEN_UNSET)
62
+ return builder.build()
63
+ }
64
+ }
@@ -0,0 +1,264 @@
1
+ import Foundation
2
+
3
+ /// One typeset event, in script coordinates. `AssSignOverlayView` maps it onto the video rect.
4
+ ///
5
+ /// The Android side has the same shape, field for field, and for the same reason: this is what a
6
+ /// renderer needs to put a sign on a placard, and AVFoundation can carry almost none of it.
7
+ struct AssSign: Equatable {
8
+ let startMs: Int
9
+ let endMs: Int
10
+ let text: String
11
+
12
+ let x: Double
13
+ let y: Double
14
+ let alignment: Int
15
+
16
+ /// Clockwise, having already been flipped from ASS's counter-clockwise measure.
17
+ let rotationZ: Double
18
+ let rotationX: Double
19
+ let rotationY: Double
20
+ let pivotX: Double
21
+ let pivotY: Double
22
+
23
+ let scaleX: Double
24
+ let scaleY: Double
25
+ let fontSize: Double
26
+ let colour: Int
27
+ let alpha: Double
28
+ let outlineColour: Int
29
+ let outlineWidth: Double
30
+ let bold: Bool
31
+ let italic: Bool
32
+ let underline: Bool
33
+
34
+ /// The coordinate space the values above are expressed in.
35
+ let playResX: Double
36
+ let playResY: Double
37
+ }
38
+
39
+ extension AssSign {
40
+ /// `AVMediaSelectionOption` reports a full locale identifier (`id_ID`, `pt-BR`) where the track was
41
+ /// given a bare language tag. Comparing the language subtag is what makes the two ends meet.
42
+ static func normalizeLanguage(_ language: String) -> String {
43
+ let lowered = language.lowercased()
44
+ if let cut = lowered.firstIndex(where: { $0 == "-" || $0 == "_" }) {
45
+ return String(lowered[lowered.startIndex..<cut])
46
+ }
47
+ return lowered
48
+ }
49
+ }
50
+
51
+ /// A script split into the part this package draws and the part AVFoundation draws.
52
+ struct AssSplit {
53
+ /// Typeset events, drawn by `AssSignOverlayView`.
54
+ let signs: [AssSign]
55
+ /// The script with every sign, drawing and invisible event cut out, for the WebVTT converter.
56
+ let dialogueOnly: String
57
+ }
58
+
59
+ extension SubtitleConverter {
60
+
61
+ /// Takes the typesetting out of a script and hands back what is left.
62
+ ///
63
+ /// ### Why signs cannot go through WebVTT
64
+ ///
65
+ /// AVFoundation renders a `.text` track itself, and WebVTT has no rotation, no scale and no
66
+ /// per-cue opacity to express. `AVTextStyleRule` has no attribute for any of it either. A sign
67
+ /// tilted to lie on a placard comes out flat, which reads as pasted over the picture rather than
68
+ /// part of it — in one episode of Jujutsu Kaisen S3, 3,723 events carry `\frz`, 1,304 of them past
69
+ /// 5°.
70
+ ///
71
+ /// So the same split the Android side makes: typeset events to us, dialogue to the platform, and
72
+ /// the sign lines removed from the copy the platform parses. There is no matching step to get
73
+ /// wrong, because an event is in exactly one of the two outputs.
74
+ ///
75
+ /// ### What counts as a sign
76
+ ///
77
+ /// Anything the author placed: an event carrying `\pos` or `\move`, or whose effective alignment
78
+ /// is not bottom-centre. The complement is dialogue, and AVFoundation is welcome to it.
79
+ static func splitAss(_ ass: String) -> AssSplit {
80
+ let normalized = normalizeNewlines(ass)
81
+ let script = AssScript(parsing: normalized)
82
+ let lines = normalized.components(separatedBy: "\n")
83
+
84
+ var signs: [AssSign] = []
85
+ var removed = Set<Int>()
86
+
87
+ for event in script.events {
88
+ guard event.endMs > event.startMs else { continue }
89
+ let style = script.styles[event.styleName]
90
+
91
+ // A style that draws nothing is where a typesetter parks working notes. Neither renderer
92
+ // should show it, and the WebVTT converter drops it too — cut here so both agree.
93
+ if style?.invisible == true {
94
+ removed.insert(event.lineIndex)
95
+ continue
96
+ }
97
+
98
+ guard let sign = buildSign(event: event, style: style, script: script) else { continue }
99
+
100
+ switch sign {
101
+ case .drop:
102
+ removed.insert(event.lineIndex)
103
+ case .sign(let value):
104
+ signs.append(value)
105
+ removed.insert(event.lineIndex)
106
+ case .dialogue:
107
+ break
108
+ }
109
+ }
110
+
111
+ let dialogueOnly = removed.isEmpty
112
+ ? normalized
113
+ : lines.enumerated().filter { !removed.contains($0.offset) }.map(\.element).joined(separator: "\n")
114
+
115
+ return AssSplit(signs: signs, dialogueOnly: dialogueOnly)
116
+ }
117
+
118
+ private enum SignResolution {
119
+ case sign(AssSign)
120
+ case dialogue
121
+ case drop
122
+ }
123
+
124
+ private static func buildSign(
125
+ event: AssScript.Event,
126
+ style: AssScript.Style?,
127
+ script: AssScript
128
+ ) -> SignResolution? {
129
+ var alignment: Int = style?.alignment ?? 2
130
+ var position: (x: Double, y: Double)?
131
+ var origin: (x: Double, y: Double)?
132
+ var rotationZ: Double = style?.angle ?? 0
133
+ var rotationX: Double = 0
134
+ var rotationY: Double = 0
135
+ var scaleX: Double = style?.scaleX ?? 1
136
+ var scaleY: Double = style?.scaleY ?? 1
137
+ var fontSize: Double = style?.fontSize ?? 48
138
+ var colour: Int = style?.colour ?? 0xFFFFFF
139
+ var alpha: Double = style?.alpha ?? 1
140
+ var bold: Bool = style?.bold ?? false
141
+ var italic: Bool = style?.italic ?? false
142
+ var underline: Bool = style?.underline ?? false
143
+ var isDrawing = false
144
+
145
+ for block in overrideBlocks(in: event.text) {
146
+ for tag in parseOverrideBlock(block) {
147
+ switch tag {
148
+ case .alignment(let value): alignment = value
149
+ case .position(let x, let y): position = (x, y)
150
+ case .origin(let x, let y): origin = (x, y)
151
+ case .rotationZ(let value): rotationZ = value
152
+ case .rotationX(let value): rotationX = value
153
+ case .rotationY(let value): rotationY = value
154
+ case .scaleX(let value): scaleX = value
155
+ case .scaleY(let value): scaleY = value
156
+ case .fontSize(let value): fontSize = value
157
+ case .primaryColour(let value): colour = value
158
+ case .primaryAlpha(let value): alpha = value
159
+ case .bold(let on): bold = on
160
+ case .italic(let on): italic = on
161
+ case .underline(let on): underline = on
162
+ case .drawing: isDrawing = true
163
+ case .reset: break
164
+ }
165
+ }
166
+ }
167
+
168
+ if isDrawing { return .drop }
169
+
170
+ let text = strippedText(event.text)
171
+ if text.trimmingCharacters(in: CharacterSet.whitespacesAndNewlines).isEmpty { return .drop }
172
+
173
+ // The complement of what AVFoundation lays out at the bottom by itself.
174
+ guard position != nil || alignment != 2 else { return .dialogue }
175
+
176
+ // Style margins only. A per-event margin override would need the Event to carry one, and it
177
+ // only ever matters for a sign with no `\pos` at all — which in a typeset script is close to
178
+ // none of them. Media3 ignores per-event margins outright, so both platforms agree here.
179
+ let marginL: Double = style?.marginL ?? 0
180
+ let marginR: Double = style?.marginR ?? 0
181
+ let marginV: Double = style?.marginV ?? 0
182
+
183
+ let x: Double = position?.x ?? {
184
+ switch (alignment - 1) % 3 {
185
+ case 0: return marginL
186
+ case 2: return script.playResX - marginR
187
+ default: return script.playResX / 2
188
+ }
189
+ }()
190
+ let y: Double = position?.y ?? {
191
+ switch (alignment - 1) / 3 {
192
+ case 0: return script.playResY - marginV
193
+ case 2: return marginV
194
+ default: return script.playResY / 2
195
+ }
196
+ }()
197
+
198
+ return .sign(AssSign(
199
+ startMs: event.startMs,
200
+ endMs: event.endMs,
201
+ text: text,
202
+ x: x,
203
+ y: y,
204
+ alignment: alignment,
205
+ // ASS measures counter-clockwise; every renderer here turns clockwise.
206
+ rotationZ: -rotationZ,
207
+ rotationX: rotationX,
208
+ rotationY: -rotationY,
209
+ pivotX: origin?.x ?? x,
210
+ pivotY: origin?.y ?? y,
211
+ scaleX: scaleX,
212
+ scaleY: scaleY,
213
+ fontSize: fontSize,
214
+ colour: colour,
215
+ alpha: alpha,
216
+ outlineColour: style?.outlineColour ?? 0x000000,
217
+ outlineWidth: style?.outlineWidth ?? 0,
218
+ bold: bold,
219
+ italic: italic,
220
+ underline: underline,
221
+ playResX: script.playResX,
222
+ playResY: script.playResY
223
+ ))
224
+ }
225
+
226
+ /// The contents of every `{...}` block, in order.
227
+ private static func overrideBlocks(in text: String) -> [String] {
228
+ var blocks: [String] = []
229
+ let chars = Array(text)
230
+ var i = 0
231
+ while i < chars.count {
232
+ guard chars[i] == "{" else { i += 1; continue }
233
+ guard let close = chars[(i + 1)...].firstIndex(of: "}") else { break }
234
+ blocks.append(String(chars[(i + 1)..<close]))
235
+ i = close + 1
236
+ }
237
+ return blocks
238
+ }
239
+
240
+ /// Override blocks out, ASS escapes in.
241
+ private static func strippedText(_ raw: String) -> String {
242
+ var out = ""
243
+ let chars = Array(raw)
244
+ var i = 0
245
+ while i < chars.count {
246
+ if chars[i] == "{", let close = chars[(i + 1)...].firstIndex(of: "}") {
247
+ i = close + 1
248
+ continue
249
+ }
250
+ if chars[i] == "\\", i + 1 < chars.count {
251
+ switch chars[i + 1] {
252
+ case "N", "n": out.append("\n"); i += 2; continue
253
+ // A hard space: U+00A0, not U+0020. A plain space would let the layout break the line
254
+ // exactly where the typesetter wrote the escape to stop it doing so.
255
+ case "h": out.append("\u{00A0}"); i += 2; continue
256
+ default: break
257
+ }
258
+ }
259
+ out.append(chars[i])
260
+ i += 1
261
+ }
262
+ return out
263
+ }
264
+ }
@@ -0,0 +1,302 @@
1
+ import AVKit
2
+ import ExpoModulesCore
3
+ import UIKit
4
+
5
+ /// Draws ASS typesetting — signs, placards, on-screen labels — with the transforms the script asks
6
+ /// for.
7
+ ///
8
+ /// AVFoundation cannot: it renders a `.text` track itself, WebVTT can express no rotation, scale or
9
+ /// per-cue opacity, and `AVTextStyleRule` has no attribute for any of them. So `SubtitleConverter`
10
+ /// takes those events out of the WebVTT it builds and leaves them in `AssSignStore`, and this view
11
+ /// draws them.
12
+ ///
13
+ /// ### Where it sits
14
+ ///
15
+ /// In `AVPlayerViewController.contentOverlayView`, which is the frame Apple provides between the
16
+ /// video and the transport controls. Sized to `videoBounds` on every frame, so `\pos` — which is in
17
+ /// `PlayResX`/`PlayResY` space — maps onto the picture rather than onto the player, and survives
18
+ /// letterboxing and rotation without any arithmetic of its own.
19
+ ///
20
+ /// ### Why it drives itself from the clock
21
+ ///
22
+ /// Signs do not ride the subtitle track, so nothing announces when one starts. It reads the player's
23
+ /// time on each frame instead, which is also the only granularity that works: frame-by-frame
24
+ /// typesetting writes one event per frame, and a sign on screen for 40 ms has to arrive and leave on
25
+ /// the frame it was authored for.
26
+ final class AssSignOverlayView: UIView {
27
+
28
+ private weak var playerViewController: AVPlayerViewController?
29
+ private var displayLink: CADisplayLink?
30
+
31
+ private var signs: [AssSign] = []
32
+ private var active: [AssSign] = []
33
+ /// What `signs` was resolved from. Both halves matter: the item changes when the next episode
34
+ /// loads, the language when the viewer switches track.
35
+ private var resolvedItem: ObjectIdentifier?
36
+ private var resolvedLanguage: String?
37
+ private var lastLookup: CFTimeInterval = 0
38
+
39
+ /// Rebuilt only when the on-screen set changes, not per frame.
40
+ private var rendered: [RenderedSign] = []
41
+
42
+ private struct RenderedSign {
43
+ let text: NSAttributedString
44
+ let size: CGSize
45
+ let origin: CGPoint
46
+ let transform: CGAffineTransform
47
+ }
48
+
49
+ // MARK: - Lifecycle
50
+
51
+ init(playerViewController: AVPlayerViewController) {
52
+ self.playerViewController = playerViewController
53
+ super.init(frame: .zero)
54
+ isUserInteractionEnabled = false
55
+ backgroundColor = .clear
56
+ contentMode = .redraw
57
+ }
58
+
59
+ @available(*, unavailable)
60
+ required init?(coder: NSCoder) {
61
+ fatalError("init(coder:) is not used")
62
+ }
63
+
64
+ /// Adds the overlay to the controller's content overlay, or returns the one already there.
65
+ @discardableResult
66
+ static func install(in controller: AVPlayerViewController) -> AssSignOverlayView? {
67
+ guard let host = controller.contentOverlayView else { return nil }
68
+
69
+ for subview in host.subviews {
70
+ if let existing = subview as? AssSignOverlayView { return existing }
71
+ }
72
+
73
+ let overlay = AssSignOverlayView(playerViewController: controller)
74
+ host.addSubview(overlay)
75
+ overlay.start()
76
+ return overlay
77
+ }
78
+
79
+ static func remove(from controller: AVPlayerViewController) {
80
+ guard let host = controller.contentOverlayView else { return }
81
+ for subview in host.subviews {
82
+ guard let overlay = subview as? AssSignOverlayView else { continue }
83
+ overlay.stop()
84
+ overlay.removeFromSuperview()
85
+ }
86
+ }
87
+
88
+ private func start() {
89
+ guard displayLink == nil else { return }
90
+ let link = CADisplayLink(target: self, selector: #selector(tick))
91
+ link.add(to: .main, forMode: .common)
92
+ displayLink = link
93
+ }
94
+
95
+ private func stop() {
96
+ displayLink?.invalidate()
97
+ displayLink = nil
98
+ clearResolved()
99
+ }
100
+
101
+ override func willMove(toWindow newWindow: UIWindow?) {
102
+ super.willMove(toWindow: newWindow)
103
+ if newWindow == nil {
104
+ displayLink?.invalidate()
105
+ displayLink = nil
106
+ } else {
107
+ start()
108
+ }
109
+ }
110
+
111
+ // MARK: - The clock
112
+
113
+ @objc private func tick() {
114
+ guard let controller = playerViewController, let player = controller.player else { return }
115
+
116
+ // The picture moves under us: rotation, full screen, and a letterbox that changes with the
117
+ // video's own aspect all land here rather than in any positioning arithmetic.
118
+ let bounds = controller.videoBounds
119
+ if bounds != frame, bounds.width > 0, bounds.height > 0 {
120
+ frame = bounds
121
+ // Every measurement in `prepare` is in view pixels, so all of it is stale after a resize.
122
+ rendered = []
123
+ setNeedsDisplay()
124
+ }
125
+
126
+ // Re-resolved on a timer rather than only when empty. Gating on "no signs yet" is the same
127
+ // mistake twice: it never fires again once a video with typesetting has loaded, so the next
128
+ // episode inherits the previous one's signs and matches them against its own timeline. The
129
+ // Android side shipped exactly that and it took two releases to unpick.
130
+ let now = CACurrentMediaTime()
131
+ if now - lastLookup >= AssSignOverlayView.lookupInterval {
132
+ lastLookup = now
133
+ resolveSigns(for: player)
134
+ }
135
+ if signs.isEmpty { return }
136
+
137
+ let positionMs = Int(CMTimeGetSeconds(player.currentTime()) * 1000)
138
+ setActive(signs.filter { positionMs >= $0.startMs && positionMs < $0.endMs })
139
+ }
140
+
141
+ private func resolveSigns(for player: AVPlayer) {
142
+ guard let item = player.currentItem else {
143
+ if resolvedItem != nil { clearResolved() }
144
+ return
145
+ }
146
+
147
+ let identity = ObjectIdentifier(item)
148
+ let language = selectedSubtitleLanguage(of: item)
149
+ guard identity != resolvedItem || language != resolvedLanguage else { return }
150
+
151
+ resolvedItem = identity
152
+ resolvedLanguage = language
153
+
154
+ // Typesetting rides on the item that owns it, so there is no key to mismatch and no store to go
155
+ // stale: a different item simply has different signs.
156
+ let found = (item as? VideoPlayerItem)
157
+ .flatMap { playerItem in language.map { playerItem.assSigns[AssSign.normalizeLanguage($0)] } }
158
+ ?? nil
159
+
160
+ signs = found ?? []
161
+ rendered = []
162
+ setActive([])
163
+ log.info("[expo-video] ASS signs for \(language ?? "nil"): \(signs.count)")
164
+ }
165
+
166
+ private func clearResolved() {
167
+ resolvedItem = nil
168
+ resolvedLanguage = nil
169
+ signs = []
170
+ rendered = []
171
+ setActive([])
172
+ }
173
+
174
+ /// The locale of the selected legible option, which is the only identity a sideloaded track keeps
175
+ /// once it is inside the composition.
176
+ private func selectedSubtitleLanguage(of item: AVPlayerItem) -> String? {
177
+ guard let group = item.asset.mediaSelectionGroup(forMediaCharacteristic: .legible),
178
+ let option = item.currentMediaSelection.selectedMediaOption(in: group) else {
179
+ return nil
180
+ }
181
+ return option.locale?.identifier ?? option.extendedLanguageTag
182
+ }
183
+
184
+ private func setActive(_ next: [AssSign]) {
185
+ // Value equality across the whole sign, not a (time, text) stand-in: two events can share both
186
+ // and differ in where they sit, which is the case a frame-by-frame animation is made of.
187
+ guard next != active else { return }
188
+ active = next
189
+ rendered = []
190
+ setNeedsDisplay()
191
+ }
192
+
193
+ // MARK: - Drawing
194
+
195
+ override func draw(_ rect: CGRect) {
196
+ guard let context = UIGraphicsGetCurrentContext(), bounds.width > 0, bounds.height > 0 else { return }
197
+
198
+ if rendered.isEmpty && !active.isEmpty {
199
+ rendered = active.compactMap(prepare)
200
+ }
201
+
202
+ for sign in rendered {
203
+ context.saveGState()
204
+ context.concatenate(sign.transform)
205
+ sign.text.draw(at: sign.origin)
206
+ context.restoreGState()
207
+ }
208
+ }
209
+
210
+ private func prepare(_ sign: AssSign) -> RenderedSign? {
211
+ let alpha = max(0, min(1, sign.alpha))
212
+ if alpha == 0 { return nil }
213
+
214
+ // Script space to view space. Both axes scale independently, so a script authored at a different
215
+ // aspect ratio than the video still lands where it was drawn.
216
+ let scaleToViewX = bounds.width / CGFloat(sign.playResX)
217
+ let scaleToViewY = bounds.height / CGFloat(sign.playResY)
218
+
219
+ let attributed = attributedText(for: sign, scaleToViewY: scaleToViewY, alpha: alpha)
220
+ let size = attributed.boundingRect(
221
+ with: CGSize(width: CGFloat.greatestFiniteMagnitude, height: CGFloat.greatestFiniteMagnitude),
222
+ options: [.usesLineFragmentOrigin, .usesFontLeading],
223
+ context: nil
224
+ ).size
225
+
226
+ let anchorX = CGFloat(sign.x) * scaleToViewX
227
+ let anchorY = CGFloat(sign.y) * scaleToViewY
228
+ // \pos names the point the cue's alignment corner sits on, so the box is offset from it.
229
+ let left: CGFloat
230
+ switch (sign.alignment - 1) % 3 {
231
+ case 0: left = anchorX
232
+ case 2: left = anchorX - size.width
233
+ default: left = anchorX - size.width / 2
234
+ }
235
+ let top: CGFloat
236
+ switch (sign.alignment - 1) / 3 {
237
+ case 0: top = anchorY - size.height
238
+ case 2: top = anchorY
239
+ default: top = anchorY - size.height / 2
240
+ }
241
+
242
+ // \org moves the centre of rotation away from the text, which is how a sign stays glued to a
243
+ // placard pivoting off-screen. Without it every rotation would spin about the text's own middle.
244
+ let pivotX = CGFloat(sign.pivotX) * scaleToViewX
245
+ let pivotY = CGFloat(sign.pivotY) * scaleToViewY
246
+
247
+ // Scale first, then turn — ASS scales the glyph and then rotates it, and with an anisotropic
248
+ // scale the two do not commute. `concatenating` applies the receiver first, so the order reads
249
+ // in the order it happens.
250
+ let transform = CGAffineTransform(translationX: -pivotX, y: -pivotY)
251
+ .concatenating(CGAffineTransform(scaleX: CGFloat(sign.scaleX), y: CGFloat(sign.scaleY)))
252
+ .concatenating(CGAffineTransform(rotationAngle: CGFloat(sign.rotationZ) * .pi / 180))
253
+ .concatenating(CGAffineTransform(translationX: pivotX, y: pivotY))
254
+
255
+ return RenderedSign(text: attributed, size: size, origin: CGPoint(x: left, y: top), transform: transform)
256
+ }
257
+
258
+ private func attributedText(for sign: AssSign, scaleToViewY: CGFloat, alpha: Double) -> NSAttributedString {
259
+ let pointSize = max(1, CGFloat(sign.fontSize) * scaleToViewY)
260
+ var font = UIFont.systemFont(ofSize: pointSize, weight: sign.bold ? .bold : .regular)
261
+ if sign.italic, let descriptor = font.fontDescriptor.withSymbolicTraits(.traitItalic) {
262
+ font = UIFont(descriptor: descriptor, size: pointSize)
263
+ }
264
+
265
+ let paragraph = NSMutableParagraphStyle()
266
+ switch (sign.alignment - 1) % 3 {
267
+ case 0: paragraph.alignment = .left
268
+ case 2: paragraph.alignment = .right
269
+ default: paragraph.alignment = .center
270
+ }
271
+
272
+ var attributes: [NSAttributedString.Key: Any] = [
273
+ .font: font,
274
+ .foregroundColor: colour(sign.colour, alpha: alpha),
275
+ .paragraphStyle: paragraph
276
+ ]
277
+ if sign.underline {
278
+ attributes[.underlineStyle] = NSUnderlineStyle.single.rawValue
279
+ }
280
+ if sign.outlineWidth > 0 {
281
+ attributes[.strokeColor] = colour(sign.outlineColour, alpha: alpha)
282
+ // Negative means stroke *and* fill. A positive width would draw the outline only, leaving the
283
+ // letters hollow.
284
+ attributes[.strokeWidth] = -sign.outlineWidth * 2
285
+ }
286
+
287
+ return NSAttributedString(string: sign.text, attributes: attributes)
288
+ }
289
+
290
+ /// ASS colours carry no alpha byte; opacity travels separately.
291
+ private func colour(_ rgb: Int, alpha: Double) -> UIColor {
292
+ UIColor(
293
+ red: CGFloat((rgb >> 16) & 0xFF) / 255,
294
+ green: CGFloat((rgb >> 8) & 0xFF) / 255,
295
+ blue: CGFloat(rgb & 0xFF) / 255,
296
+ alpha: CGFloat(alpha)
297
+ )
298
+ }
299
+
300
+ /// How often to ask the store for signs that have not been parsed yet.
301
+ private static let lookupInterval: CFTimeInterval = 0.25
302
+ }
@@ -285,12 +285,14 @@ internal struct RenderedSubtitleText {
285
285
  var isDrawing: Bool
286
286
  }
287
287
 
288
- private enum InlineToken {
288
+ internal enum InlineToken {
289
289
  case text(String)
290
290
  case markup(String)
291
291
  }
292
292
 
293
- private extension SubtitleConverter {
293
+ // Internal rather than private: `AssSign.swift` reuses the override-block scanner and the colour
294
+ // parser, and reimplementing either would be two copies that have to agree about ASS forever.
295
+ internal extension SubtitleConverter {
294
296
  /// Escapes only the parts of a cue that came from the source text.
295
297
  ///
296
298
  /// Order matters: `&` first, then `<`, then `>`. Escaping `&` last would double-escape the one in
@@ -370,6 +372,9 @@ private extension SubtitleConverter {
370
372
  case .underline(let on): on ? openTag("u") : closeTag("u")
371
373
  case .drawing: isDrawing = true
372
374
  case .reset: closeAll()
375
+ // The transform tags are for signs, which never reach this converter — a sign is taken
376
+ // out of the script before the WebVTT is built, because WebVTT can express none of them.
377
+ default: break
373
378
  }
374
379
  }
375
380
  i = end + 1
@@ -479,6 +484,50 @@ private extension SubtitleConverter {
479
484
  case underline(Bool)
480
485
  case drawing
481
486
  case reset
487
+ // Read for signs only. The WebVTT path ignores every one of them, because WebVTT can express
488
+ // none of it — which is the whole reason signs are drawn separately.
489
+ case origin(Double, Double)
490
+ case rotationZ(Double)
491
+ case rotationX(Double)
492
+ case rotationY(Double)
493
+ case scaleX(Double)
494
+ case scaleY(Double)
495
+ case fontSize(Double)
496
+ case primaryColour(Int)
497
+ case primaryAlpha(Double)
498
+ case bold(Bool)
499
+ }
500
+
501
+ struct AssColour {
502
+ let rgb: Int
503
+ let alpha: Double
504
+ }
505
+
506
+ /// `&HAABBGGRR&` or `&HBBGGRR&`, with or without the ampersands.
507
+ ///
508
+ /// The bytes are reversed against RGB and the alpha byte is inverted — `FF` is fully transparent.
509
+ /// A six-digit value carries no alpha byte and is therefore opaque.
510
+ static func parseAssColour(_ value: String, alphaOnly: Bool = false) -> AssColour? {
511
+ var text = value.trimmingCharacters(in: .whitespaces)
512
+ if text.hasSuffix("&") { text = String(text.dropLast()) }
513
+ if text.lowercased().hasPrefix("&h") { text = String(text.dropFirst(2)) }
514
+ guard !text.isEmpty, let hex = UInt32(text, radix: 16) else { return nil }
515
+
516
+ if alphaOnly || text.count <= 2 {
517
+ return AssColour(rgb: 0xFFFFFF, alpha: 1 - Double(hex & 0xFF) / 255)
518
+ }
519
+
520
+ let red: UInt32 = hex & 0xFF
521
+ let green: UInt32 = (hex >> 8) & 0xFF
522
+ let blue: UInt32 = (hex >> 16) & 0xFF
523
+ let rgb: Int = Int(red) << 16 | Int(green) << 8 | Int(blue)
524
+
525
+ var alpha: Double = 1
526
+ if text.count > 6 {
527
+ let transparency: UInt32 = (hex >> 24) & 0xFF
528
+ alpha = 1 - Double(transparency) / 255
529
+ }
530
+ return AssColour(rgb: rgb, alpha: alpha)
482
531
  }
483
532
 
484
533
  /// Scans one `{...}` block left to right.
@@ -487,6 +536,14 @@ private extension SubtitleConverter {
487
536
  /// one pass and the `\i1` keeps its position within the text. Reading the tag name as a run of
488
537
  /// letters is also what makes the longest-match problem disappear: `\c` cannot eat `\clip` and
489
538
  /// `\b` cannot eat `\bord`, because those parse as the names "clip" and "bord".
539
+ /// A number, or nothing. `Double` parses "nan" and "inf", and neither belongs in a transform.
540
+ static func finite(_ text: String) -> Double? {
541
+ guard let value = Double(text.trimmingCharacters(in: CharacterSet.whitespaces)), value.isFinite else {
542
+ return nil
543
+ }
544
+ return value
545
+ }
546
+
490
547
  static func parseOverrideBlock(_ block: String) -> [OverrideTag] {
491
548
  var tags: [OverrideTag] = []
492
549
  let chars = Array(block)
@@ -497,6 +554,10 @@ private extension SubtitleConverter {
497
554
  i += 1
498
555
 
499
556
  var name = ""
557
+ // A leading digit belongs to the name: \1c, \1a and \4a all start with one. Reading only
558
+ // letters left those as the empty name, which was harmless while colour was being dropped and
559
+ // is not once signs need it.
560
+ while i < chars.count, chars[i].isNumber { name.append(chars[i]); i += 1 }
500
561
  while i < chars.count, chars[i].isLetter { name.append(chars[i]); i += 1 }
501
562
 
502
563
  var argument = ""
@@ -526,9 +587,7 @@ private extension SubtitleConverter {
526
587
  tags.append(.alignment(value))
527
588
  }
528
589
  case "pos":
529
- let parts = argument.split(separator: ",").map {
530
- Double($0.trimmingCharacters(in: .whitespaces)) ?? 0
531
- }
590
+ let parts = argument.split(separator: ",").compactMap { finite(String($0)) }
532
591
  if parts.count >= 2 { tags.append(.position(parts[0], parts[1])) }
533
592
  case "i": tags.append(.italic(argument.trimmingCharacters(in: .whitespaces) == "1"))
534
593
  case "u": tags.append(.underline(argument.trimmingCharacters(in: .whitespaces) == "1"))
@@ -539,6 +598,34 @@ private extension SubtitleConverter {
539
598
  tags.append(.drawing)
540
599
  }
541
600
  case "r": tags.append(.reset)
601
+ case "move":
602
+ // A move ends where it ends; drawing it at the destination beats drawing it at the start.
603
+ let parts = argument.split(separator: ",").compactMap { finite(String($0)) }
604
+ if parts.count >= 4 { tags.append(.position(parts[2], parts[3])) }
605
+ case "org":
606
+ let parts = argument.split(separator: ",").compactMap { finite(String($0)) }
607
+ if parts.count >= 2 { tags.append(.origin(parts[0], parts[1])) }
608
+ // `Double("nan")` and `Double("inf")` parse. A subtitle is untrusted scraper output, and a
609
+ // non-finite value carried into a transform makes the sign silently fail to draw — so every
610
+ // number here goes through `finite`.
611
+ case "frz", "fr":
612
+ if let value = finite(argument) { tags.append(.rotationZ(value)) }
613
+ case "frx":
614
+ if let value = finite(argument) { tags.append(.rotationX(value)) }
615
+ case "fry":
616
+ if let value = finite(argument) { tags.append(.rotationY(value)) }
617
+ case "fscx":
618
+ if let value = finite(argument) { tags.append(.scaleX(value / 100)) }
619
+ case "fscy":
620
+ if let value = finite(argument) { tags.append(.scaleY(value / 100)) }
621
+ case "fs":
622
+ if let value = finite(argument), value > 0 { tags.append(.fontSize(value)) }
623
+ case "c", "1c":
624
+ if let colour = parseAssColour(argument) { tags.append(.primaryColour(colour.rgb)) }
625
+ case "alpha", "1a":
626
+ if let colour = parseAssColour(argument, alphaOnly: true) { tags.append(.primaryAlpha(colour.alpha)) }
627
+ case "b":
628
+ tags.append(.bold(argument.trimmingCharacters(in: .whitespaces) != "0"))
542
629
  default: break
543
630
  }
544
631
  }
@@ -610,16 +697,35 @@ private extension SubtitleConverter {
610
697
  /// Header fields are read from their own `Format:` lines rather than assumed, because encoders
611
698
  /// reorder them and reading the Text column one position off produces a file that still parses and
612
699
  /// is entirely wrong.
613
- private struct AssScript {
700
+ internal struct AssScript {
614
701
  struct Style {
702
+
615
703
  var alignment: Int
616
704
  var italic: Bool
617
705
  var underline: Bool
618
706
  /// The author asked for this style to draw nothing. See `isInvisibleStyle`.
619
707
  var invisible: Bool
708
+ // Read for signs. The WebVTT path uses none of it: colour, size and weight are the app's to
709
+ // decide there, and a sign is the one case where they are not.
710
+ var fontSize: Double = 48
711
+ var colour: Int = 0xFFFFFF
712
+ var alpha: Double = 1
713
+ var outlineColour: Int = 0x000000
714
+ var outlineWidth: Double = 0
715
+ var bold: Bool = false
716
+ var scaleX: Double = 1
717
+ var scaleY: Double = 1
718
+ var angle: Double = 0
719
+ var marginL: Double = 0
720
+ var marginR: Double = 0
721
+ var marginV: Double = 0
620
722
  }
621
723
 
622
724
  struct Event {
725
+ /// Index of the line this came from, so a sign can be cut out of the script handed to the
726
+ /// WebVTT converter. Mirrors the split on Android; it is what makes drawing an event twice
727
+ /// structurally impossible rather than merely avoided.
728
+ var lineIndex: Int = -1
623
729
  var startMs: Int
624
730
  var endMs: Int
625
731
  var styleName: String
@@ -639,7 +745,7 @@ private struct AssScript {
639
745
  var sawPlayResX = false
640
746
  var sawPlayResY = false
641
747
 
642
- for rawLine in content.components(separatedBy: "\n") {
748
+ for (lineIndex, rawLine) in content.components(separatedBy: "\n").enumerated() {
643
749
  let line = rawLine.trimmingCharacters(in: .whitespaces)
644
750
  if line.isEmpty || line.hasPrefix(";") { continue }
645
751
 
@@ -685,7 +791,19 @@ private struct AssScript {
685
791
  : ((1...9).contains(rawAlignment) ? rawAlignment : 2),
686
792
  italic: field("italic") == "-1",
687
793
  underline: field("underline") == "-1",
688
- invisible: Self.isInvisibleStyle(fontSize: field("fontsize"), primaryColour: field("primarycolour"))
794
+ invisible: Self.isInvisibleStyle(fontSize: field("fontsize"), primaryColour: field("primarycolour")),
795
+ fontSize: field("fontsize").flatMap(SubtitleConverter.finite) ?? 48,
796
+ colour: field("primarycolour").flatMap { SubtitleConverter.parseAssColour($0)?.rgb } ?? 0xFFFFFF,
797
+ alpha: field("primarycolour").flatMap { SubtitleConverter.parseAssColour($0)?.alpha } ?? 1,
798
+ outlineColour: field("outlinecolour").flatMap { SubtitleConverter.parseAssColour($0)?.rgb } ?? 0x000000,
799
+ outlineWidth: field("outline").flatMap(SubtitleConverter.finite) ?? 0,
800
+ bold: field("bold") == "-1",
801
+ scaleX: (field("scalex").flatMap(SubtitleConverter.finite) ?? 100) / 100,
802
+ scaleY: (field("scaley").flatMap(SubtitleConverter.finite) ?? 100) / 100,
803
+ angle: field("angle").flatMap(SubtitleConverter.finite) ?? 0,
804
+ marginL: field("marginl").flatMap(SubtitleConverter.finite) ?? 0,
805
+ marginR: field("marginr").flatMap(SubtitleConverter.finite) ?? 0,
806
+ marginV: field("marginv").flatMap(SubtitleConverter.finite) ?? 0
689
807
  )
690
808
  }
691
809
 
@@ -697,7 +815,8 @@ private struct AssScript {
697
815
  } else if key.caseInsensitiveCompare("Dialogue") == .orderedSame {
698
816
  // Only Dialogue reaches the screen. Comment carries typesetting notes and unused
699
817
  // alternates, and Picture/Sound/Movie/Command are not text at all.
700
- guard let event = Self.parseEvent(value, fields: eventFields) else { continue }
818
+ guard var event = Self.parseEvent(value, fields: eventFields) else { continue }
819
+ event.lineIndex = lineIndex
701
820
  events.append(event)
702
821
  }
703
822
 
@@ -7,6 +7,13 @@ class VideoPlayerItem: AVPlayerItem {
7
7
  let urlAsset: VideoAsset
8
8
  let videoSource: VideoSource
9
9
  let isHls: Bool
10
+ /// ASS typesetting taken out of this item's subtitle tracks, keyed by language subtag.
11
+ ///
12
+ /// Carried on the item rather than in a store of its own, because the item *is* the identity the
13
+ /// overlay needs: two players can be loading different videos at once, and a store keyed by
14
+ /// anything a subtitle track keeps inside a composition — which is only its language — would have
15
+ /// them overwrite each other.
16
+ let assSigns: [String: [AssSign]]
10
17
  var videoTracks: [VideoTrack] {
11
18
  get async {
12
19
  return await tracksLoadingTask?.value ?? []
@@ -24,6 +31,8 @@ class VideoPlayerItem: AVPlayerItem {
24
31
  let asset = VideoAsset(url: url, videoSource: videoSource)
25
32
  self.urlAsset = asset
26
33
  self.isHls = asset.effectivePlaybackURL.isHLS || asset.effectiveContentType == .hls
34
+ // This initializer builds no composition, so it sideloads nothing and has no typesetting.
35
+ self.assSigns = [:]
27
36
  super.init(asset: urlAsset, automaticallyLoadedAssetKeys: nil)
28
37
  self.createTracksLoadingTask()
29
38
  }
@@ -51,15 +60,18 @@ class VideoPlayerItem: AVPlayerItem {
51
60
  // (non-HLS, non-DRM) sources — HLS declares its own subtitle renditions, and a composition would
52
61
  // bypass the DRM/resource-loader path used by protected content.
53
62
  var playbackAsset: AVAsset = asset
63
+ var signs: [String: [AssSign]] = [:]
54
64
  if let subtitleTracks = videoSource.subtitleTracks, !subtitleTracks.isEmpty, !isHls, videoSource.drm == nil {
55
- if let composition = await VideoPlayerSubtitleSideload.buildComposition(
65
+ if let sideloaded = await VideoPlayerSubtitleSideload.buildComposition(
56
66
  from: asset,
57
67
  subtitleTracks: subtitleTracks,
58
68
  headers: videoSource.headers
59
69
  ) {
60
- playbackAsset = composition
70
+ playbackAsset = sideloaded.composition
71
+ signs = sideloaded.signsByLanguage
61
72
  }
62
73
  }
74
+ self.assSigns = signs
63
75
 
64
76
  super.init(asset: playbackAsset, automaticallyLoadedAssetKeys: nil)
65
77
  self.createTracksLoadingTask()
@@ -13,14 +13,27 @@ import ExpoModulesCore
13
13
  /// pipeline already understands. Only WebVTT is natively supported, so `SRT` files are converted first, and
14
14
  /// remote files are downloaded to a temporary location because `insertTimeRange` requires a readable local track.
15
15
  internal enum VideoPlayerSubtitleSideload {
16
+ /// A composition plus the typesetting taken out of its subtitle tracks, keyed by language.
17
+ ///
18
+ /// The signs travel with the composition rather than through a store of their own. A store would
19
+ /// have to be keyed by something both ends can see, and the only identity a sideloaded track keeps
20
+ /// inside a composition is its language — which means two players loading different videos in the
21
+ /// same language would overwrite each other, and a second episode would inherit the first one's
22
+ /// signs. Handing them back here makes the `AVPlayerItem` itself the identity.
23
+ struct SideloadResult {
24
+ let composition: AVMutableComposition
25
+ let signsByLanguage: [String: [AssSign]]
26
+ }
27
+
16
28
  static func buildComposition(
17
29
  from asset: AVURLAsset,
18
30
  subtitleTracks: [SubtitleSource],
19
31
  headers: [String: String]?
20
- ) async -> AVMutableComposition? {
32
+ ) async -> SideloadResult? {
21
33
  do {
22
34
  let mainDuration = try await asset.load(.duration)
23
35
  let composition = AVMutableComposition()
36
+ var signsByLanguage: [String: [AssSign]] = [:]
24
37
 
25
38
  // Copy the original audio + video tracks.
26
39
  let tracks = try await asset.load(.tracks)
@@ -54,7 +67,7 @@ internal enum VideoPlayerSubtitleSideload {
54
67
 
55
68
  var didAddSubtitle = false
56
69
  for subtitle in subtitleTracks {
57
- guard let localURL = await resolveLocalVttURL(for: subtitle, headers: headers) else {
70
+ guard let localURL = await resolveLocalVttURL(for: subtitle, headers: headers, signs: &signsByLanguage) else {
58
71
  continue
59
72
  }
60
73
  let subtitleAsset = AVURLAsset(url: localURL)
@@ -94,7 +107,7 @@ internal enum VideoPlayerSubtitleSideload {
94
107
  }
95
108
  }
96
109
 
97
- return didAddSubtitle ? composition : nil
110
+ return didAddSubtitle ? SideloadResult(composition: composition, signsByLanguage: signsByLanguage) : nil
98
111
  } catch {
99
112
  log.warn("[expo-video] Failed to build subtitle composition: \(error.localizedDescription)")
100
113
  return nil
@@ -108,7 +121,11 @@ internal enum VideoPlayerSubtitleSideload {
108
121
  /// also preserved the colours and weights a WebVTT file carries in `<c.yellow>` and `<b>`, which
109
122
  /// would then beat `subtitleStyle`. The temp-file cache means the work happens once per
110
123
  /// (uri, language, format, offset).
111
- private static func resolveLocalVttURL(for subtitle: SubtitleSource, headers: [String: String]?) async -> URL? {
124
+ private static func resolveLocalVttURL(
125
+ for subtitle: SubtitleSource,
126
+ headers: [String: String]?,
127
+ signs: inout [String: [AssSign]]
128
+ ) async -> URL? {
112
129
  guard let uri = subtitle.uri else {
113
130
  return nil
114
131
  }
@@ -130,7 +147,15 @@ internal enum VideoPlayerSubtitleSideload {
130
147
  }
131
148
 
132
149
  switch subtitle.resolvedFormat {
133
- case .ass: text = SubtitleConverter.assToVtt(text)
150
+ case .ass:
151
+ // Typesetting is taken out here rather than converted. WebVTT can express no rotation, scale
152
+ // or per-cue opacity, and AVFoundation renders the track itself, so a sign that went through
153
+ // would come out flat. `AssSignOverlayView` draws them instead.
154
+ let split = SubtitleConverter.splitAss(text)
155
+ if !split.signs.isEmpty {
156
+ signs[AssSign.normalizeLanguage(subtitle.language ?? "und")] = split.signs
157
+ }
158
+ text = SubtitleConverter.assToVtt(split.dialogueOnly)
134
159
  case .srt: text = SubtitleConverter.srtToVtt(text)
135
160
  case .vtt: text = SubtitleConverter.sanitizeVtt(text)
136
161
  }
@@ -11,6 +11,14 @@ public final class VideoView: ExpoView, AVPlayerViewControllerDelegate {
11
11
  playerViewController.player = player?.ref
12
12
  // Forward any subtitle style set before the player was attached.
13
13
  player?.subtitleStyle = subtitleStyle
14
+ // ASS typesetting is drawn by us, not by AVFoundation, which can express none of its
15
+ // transforms. Installed whenever a player arrives and torn down with the view; the overlay
16
+ // sits idle and costs a display-link tick until a track with signs is actually loaded.
17
+ if player != nil {
18
+ AssSignOverlayView.install(in: playerViewController)
19
+ } else {
20
+ AssSignOverlayView.remove(from: playerViewController)
21
+ }
14
22
  }
15
23
  }
16
24
 
@@ -74,6 +82,7 @@ public final class VideoView: ExpoView, AVPlayerViewControllerDelegate {
74
82
  deinit {
75
83
  VideoManager.shared.unregister(videoView: self)
76
84
  removeFirstFrameObserver()
85
+ AssSignOverlayView.remove(from: playerViewController)
77
86
  }
78
87
 
79
88
  func enterFullscreen() {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "expo-video-subtitle",
3
- "version": "0.7.6",
3
+ "version": "0.8.1",
4
4
  "description": "A cross-platform, performant video component for React Native and Expo with external subtitle sideloading and custom subtitle styling.",
5
5
  "keywords": [
6
6
  "expo",