expo-video-subtitle 0.6.0 → 0.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,76 @@ Changes inherited from the upstream base version are not repeated here — see t
7
7
  [upstream changelog](https://github.com/expo/expo/blob/main/packages/expo-video/CHANGELOG.md)
8
8
  for the history of everything that isn't subtitle related.
9
9
 
10
+ ## 0.7.1
11
+
12
+ ### Fixed
13
+
14
+ - **Playing an `.ass` track crashed the player (Android).** `Regex(""""\\{([^}]*)}""")` in the sign
15
+ parser left the closing brace unescaped. `java.util.regex` accepts that; Android compiles regular
16
+ expressions with **ICU**, which does not, so the pattern threw `PatternSyntaxException` out of a
17
+ static initialiser on the extractor's loading thread and took playback down with it —
18
+ `ExceptionInInitializerError` on the first ASS subtitle the app opened.
19
+
20
+ The whole class of bug is invisible to this module's JVM tests, because the offending pattern
21
+ compiles there. `RegexPortabilityTest` now reads the sources instead and fails on any regex
22
+ literal with a brace that is neither escaped nor inside a character class.
23
+
24
+ - **A subtitle this package cannot read no longer stops the video.** `StackingSsaParser.split` runs
25
+ on the extractor's loading thread, where a thrown exception is not a failed subtitle — it is a
26
+ dead player. It is now wrapped end to end, and a track that fails falls back to Media3's own ASS
27
+ handling, which draws the captions flat. Subtitle files are untrusted scraper output; plain
28
+ captions are the honest failure, a black screen is not.
29
+
30
+ - **A multi-line sign follows its own alignment.** Every block was laid out centred, so the second
31
+ line of a left- or right-anchored placard pulled to the middle.
32
+
33
+ ## 0.7.0
34
+
35
+ ### Added
36
+
37
+ - **ASS signs are drawn with the transforms the script asks for (Android).** A typeset sign is
38
+ placed on the artwork, and the artwork is usually not square to the screen: in one episode of
39
+ Jujutsu Kaisen S3, 3,723 events carry `\frz`, 1,304 of them past 5° and 104 past 45°, because the
40
+ placards they sit on are drawn at an angle. Rendered flat, the translation reads as pasted over
41
+ the picture instead of lying on it.
42
+
43
+ Media3 cannot do this and has no room to. `SubtitlePainter` contains no call to `rotate`, `skew`,
44
+ `Matrix` or `Camera`; `Cue` carries no rotation field, only `shearDegrees`, which the painter
45
+ ignores; and `SsaParser` discards every override block except `\pos`, `\move` and `\an`.
46
+
47
+ So signs leave Media3's hands entirely. `AssSignParser` splits a script in two — typeset events
48
+ to us, dialogue to the delegate, with the sign lines cut out of the copy the delegate parses, so
49
+ double-drawing is structurally impossible rather than merely avoided. Each sign rides the cue
50
+ stream as a zero-width placeholder carrying an `AssSignSpan`, which buys Media3's timing for free,
51
+ and `AssSignOverlay` draws it on a canvas in front of the captions.
52
+
53
+ Read and honoured: `\pos`, `\move` (at its destination), `\org`, `\an` and the legacy `\a`,
54
+ `\frz`, `\frx`, `\fry`, `\fscx`, `\fscy`, `\fs`, `\c` / `\1c`, `\alpha` / `\1a`, `\b`,
55
+ `\i`, `\u`, style `Alignment`, `Fontsize`, `PrimaryColour`, `OutlineColour`, `Outline`,
56
+ `ScaleX/Y`, `Angle` and the margins, plus `PlayResX` / `PlayResY` and VSFilter's 4:3 rule when one
57
+ is missing.
58
+
59
+ The overlay sits in the player's content frame, which is sized to the video rather than the view,
60
+ so positions survive letterboxing.
61
+
62
+ ### Known limitations
63
+
64
+ - `\t` animation, `\fad`, `\clip`, `\blur`, `\be`, `\fax` shear, vector drawings and karaoke are
65
+ still dropped. Reproducing those means libass, not a larger subset of it.
66
+ - Fonts fall back to the system face: a script naming `Bogart Semibold` gets the default at the
67
+ right size and weight, not that typeface.
68
+ - `\frx` and `\fry` are applied with an unverified sign convention. They are almost always tiny —
69
+ 3,363 of 5,126 `\frx` values and 2,715 of 3,579 `\fry` values in that episode are under 3° — so a
70
+ wrong sign is subtle rather than visible. `\frz` is what carries the effect and is verified
71
+ against the source line it came from.
72
+ - iOS still draws signs flat. AVFoundation renders WebVTT itself and has no rotation to give.
73
+
74
+ ### Changed
75
+
76
+ - The module now has JVM unit tests (`./gradlew :expo-video-subtitle:testDebugUnitTest`). The sign
77
+ parser is pure Kotlin with no Android imports precisely so it can be tested this way: its tag
78
+ scanner and colour handling fail by drawing a sign in the wrong place, which nothing else catches.
79
+
10
80
  ## 0.6.0
11
81
 
12
82
  ### Fixed
package/FORK.md CHANGED
@@ -29,6 +29,9 @@ Measured against the published `expo-video@56.1.4` tarball. `+`/`-` are added/re
29
29
  | `android/…/records/SubtitleStyle.kt` | 152 | Record → `CaptionStyleCompat`; hex colour parsing; font-family resolution (`res/font` → `ReactFontManager`); the `applyEmbeddedStyles` flag, which `SubtitleUtils` reads directly rather than `toCaptionStyle` and which defaults to `true` since 0.5.0. |
30
30
  | `android/…/utils/StackingSubtitleParser.kt` | 342 | `SubtitleParser` that segments the cue timeline and merges simultaneous **unpositioned** cues into one stacked block, plus the `SubtitleParser.Factory` that routes SubRip and ASS/SSA to it and wraps every parser in `FilteringSubtitleParser`. ASS gets there through `SsaCueNormalizer` as the `prepare` step — without it `SsaParser` sets both `line` and `position` on every cue and they would all take the passthrough branch. Assigns each cue a vertical slot it keeps for its whole time on screen — a finished cue's line is held blank (`\u200B`) so the cues above it do not drop. Merges through a `SpannableStringBuilder` so cue spans survive. |
31
31
  | `android/…/utils/SsaCueNormalizer.kt` | 93 | Clears the `line`/`position` Media3's `SsaParser` fills in on cues the ASS author never placed, so `bottomOffset` and cue stacking work on `.ass` the way they already do on `.srt`. The test is exact rather than heuristic: those values are a pure function of the cue's anchors, so matching both defaults for the bottom-centre anchors *is* "no positioning tag". Also drops events on a style the author made invisible (`Fontsize 0`, or a primary colour whose inverted alpha byte is `FF`) — that is where typesetters park working notes, and nothing downstream can carry "invisible". |
32
+ | `android/…/utils/AssSignParser.kt` | 505 | Splits an ASS script into the typeset events this package draws and a copy of the script with those lines removed for Media3's parser. One pass, no matching step, so an event is in exactly one of the two outputs and cannot be drawn twice. Pure Kotlin with no Android imports, which is what makes it unit-testable on the JVM. Reads `\pos`, `\move`, `\org`, `\an`, `\frz`, `\frx`, `\fry`, `\fscx/y`, `\fs`, `\c`, `\alpha`, `\b`, `\i`, `\u` and the style columns behind them. The tag scanner takes the longest name first — `\c` must not swallow `\clip`, `\b` must not swallow `\bord`, `\blur` or `\be` — and reads a leading digit as part of the name for `\1c` and `\4a`. |
33
+ | `android/…/utils/AssSignOverlay.kt` | 235 | Draws those signs on a canvas in front of the captions, with `Matrix` for rotation and scale and `Camera` for the two 3-D axes. **Media3 cannot do this**: `SubtitlePainter` has no rotate, skew, Matrix or Camera call, and `Cue` has no rotation field. Sits in the same parent as `SubtitleView` — the content frame, sized to the video, which is why `\pos` survives letterboxing. Lays text out on change rather than per frame, because frame-by-frame typesetting changes the on-screen set every 40 ms. |
34
+ | `android/…/utils/AssSignSpan.kt` | 24 | Carries one parsed sign through Media3's cue stream to the overlay, on a zero-width placeholder so `SubtitleView` draws nothing. Riding the cue stream is what buys the timing for free. |
32
35
  | `android/…/utils/RoundedSubtitleBackground.kt` | 352 | Draws `subtitleStyle.backgroundRadius` as rounded boxes, which Media3 cannot do at all: it paints `backgroundColor` as a framework `BackgroundColorSpan` and `windowColor` with `canvas.drawRect`, neither takes a radius, and `SubtitlePainter`/`CanvasSubtitleOutput` are package-private final while `SubtitleView` is final. A second `SubtitleView` is inserted behind `PlayerView`'s own, fed from `Player.Listener.onCues`, drawing a `LineBackgroundSpan` per line. **Its `EDGE_TYPE_NONE` is load-bearing** — `SubtitlePainter` copies the cue text for an edge pass and draws both layouts when the edge type is outline/raised/depressed, which would paint the box twice and darken a semi-transparent colour. State hangs off the view's `tag`, never a map in the companion object; see the `find` KDoc for why even a `WeakHashMap` leaks here. |
33
36
  | `FORK.md`, `LICENSE` | — | This guide; MIT licence (upstream copyright preserved). |
34
37
  | `eslint.config.js` | 4 | Verbatim copy of `expo-module-scripts/templates/eslint.config.js`. **Recreate it if a sync drops it** — `expo-module configure` treats it as an *optional* template and only syncs it when it already exists, so without this file `npm run lint` dies with "ESLint couldn't find an eslint.config.js". |
@@ -26,6 +26,11 @@ dependencies {
26
26
  implementation "androidx.media3:media3-ui:${androidxMedia3Version}"
27
27
  implementation "androidx.media3:media3-datasource-okhttp:${androidxMedia3Version}"
28
28
 
29
+ // The ASS sign parser is pure Kotlin with no Android imports, which is what makes it worth
30
+ // testing on the JVM: its tag scanner and colour handling are the parts that fail silently on a
31
+ // device, by drawing a sign in the wrong place rather than by crashing.
32
+ testImplementation 'junit:junit:4.13.2'
33
+
29
34
  def fragment_version = "1.8.9"
30
35
  implementation "androidx.fragment:fragment:$fragment_version"
31
36
  implementation "androidx.fragment:fragment-ktx:$fragment_version"
@@ -0,0 +1,276 @@
1
+ package expo.modules.video.utils
2
+
3
+ import android.annotation.SuppressLint
4
+ import android.content.Context
5
+ import android.graphics.Camera
6
+ import android.graphics.Canvas
7
+ import android.graphics.Color
8
+ import android.graphics.Matrix
9
+ import android.graphics.Paint
10
+ import android.graphics.Typeface
11
+ import android.text.Layout
12
+ import android.text.StaticLayout
13
+ import android.text.TextPaint
14
+ import android.view.View
15
+ import android.view.ViewGroup
16
+ import androidx.annotation.OptIn
17
+ import androidx.media3.common.Player
18
+ import androidx.media3.common.text.Cue
19
+ import androidx.media3.common.text.CueGroup
20
+ import androidx.media3.common.util.UnstableApi
21
+ import androidx.media3.ui.PlayerView
22
+ import androidx.media3.ui.SubtitleView
23
+
24
+ /**
25
+ * Draws ASS typesetting — signs, placards, on-screen labels — with the transforms the script asks
26
+ * for.
27
+ *
28
+ * `SubtitleView` cannot: `SubtitlePainter` has no rotation, and `Cue` has no field to carry one.
29
+ * So [AssSignParser] takes those events out of Media3's cue stream, wraps each in an [AssSignSpan],
30
+ * and this view draws them itself.
31
+ *
32
+ * ### Where it sits
33
+ *
34
+ * As a sibling of `PlayerView`'s own `SubtitleView`, in front of it. That parent is the content
35
+ * frame, which is sized to the *video*, not to the player — which is the whole reason positions
36
+ * survive letterboxing. `\pos` is in `PlayResX`/`PlayResY` space and maps onto this view's bounds
37
+ * directly. [RoundedSubtitleBackground] takes the same route for the same reason.
38
+ *
39
+ * ### Why a canvas and not child views
40
+ *
41
+ * A `TextView` per sign would work — Android view transforms map onto ASS almost one for one — but
42
+ * a heavily typeset episode puts dozens of signs on screen across a scene and each would cost a
43
+ * measure and layout pass. Drawing straight to the canvas costs one `StaticLayout` per sign per
44
+ * frame it changes, and it is the only way to stroke the outline the way ASS means it: the border
45
+ * drawn *behind* the fill, not as a shadow.
46
+ */
47
+ @SuppressLint("ViewConstructor")
48
+ @OptIn(UnstableApi::class)
49
+ internal class AssSignOverlay(context: Context) : View(context), Player.Listener {
50
+
51
+ private val camera = Camera()
52
+ private val matrix = Matrix()
53
+
54
+ private var signs: List<AssSign> = emptyList()
55
+ private var prepared: List<PreparedSign> = emptyList()
56
+ private var player: Player? = null
57
+
58
+ init {
59
+ setWillNotDraw(false)
60
+ }
61
+
62
+ override fun onCues(cueGroup: CueGroup) {
63
+ setSigns(cueGroup.cues)
64
+ }
65
+
66
+ private fun setSigns(cues: List<Cue>) {
67
+ val next = cues.mapNotNull(::signOf)
68
+ // Reference equality on the list contents is enough: the parser produces one AssSign per event
69
+ // and hands out the same instance for every segment the event spans.
70
+ if (next.size == signs.size && next.indices.all { next[it] === signs[it] }) return
71
+ signs = next
72
+ rebuild()
73
+ }
74
+
75
+ override fun onSizeChanged(w: Int, h: Int, oldw: Int, oldh: Int) {
76
+ super.onSizeChanged(w, h, oldw, oldh)
77
+ // Every measurement below is in view pixels, so all of it is stale after a resize — which
78
+ // happens on every rotation and on entering full screen.
79
+ rebuild()
80
+ }
81
+
82
+ /**
83
+ * Lays the signs out once per change rather than once per frame.
84
+ *
85
+ * Frame-by-frame typesetting changes the on-screen set every 40 ms, and `onDraw` can run more
86
+ * often than that for reasons of the view system's own. Text layout is the expensive half of
87
+ * drawing, and none of it depends on anything that varies between frames.
88
+ */
89
+ private fun rebuild() {
90
+ prepared = if (width == 0 || height == 0) emptyList() else signs.mapNotNull(::prepare)
91
+ invalidate()
92
+ }
93
+
94
+ fun attachTo(player: Player?) {
95
+ if (this.player === player) return
96
+ this.player?.removeListener(this)
97
+ this.player = player
98
+ player?.addListener(this)
99
+ // A listener only hears about the *next* cue change, so seed from what is on screen already.
100
+ setSigns(player?.currentCues?.cues ?: emptyList())
101
+ }
102
+
103
+ fun detach() {
104
+ player?.removeListener(this)
105
+ player = null
106
+ signs = emptyList()
107
+ rebuild()
108
+ }
109
+
110
+ override fun onDraw(canvas: Canvas) {
111
+ for (sign in prepared) {
112
+ draw(canvas, sign)
113
+ }
114
+ }
115
+
116
+ private fun draw(canvas: Canvas, sign: PreparedSign) {
117
+ canvas.save()
118
+ canvas.concat(sign.matrix)
119
+ canvas.translate(sign.left, sign.top)
120
+ sign.stroke?.draw(canvas)
121
+ sign.fill.draw(canvas)
122
+ canvas.restore()
123
+ }
124
+
125
+ /** Everything about one sign that does not change between frames. */
126
+ private class PreparedSign(
127
+ val matrix: Matrix,
128
+ val left: Float,
129
+ val top: Float,
130
+ val fill: StaticLayout,
131
+ val stroke: StaticLayout?,
132
+ )
133
+
134
+ private fun prepare(sign: AssSign): PreparedSign? {
135
+ // Script space to view space. Both axes are scaled independently so a script authored at a
136
+ // different aspect ratio than the video still lands where it was drawn.
137
+ val scaleToViewX = width / sign.playResX
138
+ val scaleToViewY = height / sign.playResY
139
+
140
+ val alpha = (sign.alpha * 255f).toInt().coerceIn(0, 255)
141
+ if (alpha == 0) return null
142
+
143
+ // A fresh paint per sign: StaticLayout keeps the reference it was built with, so a shared
144
+ // mutable one would repaint every sign already laid out in this pass.
145
+ val fillPaint = TextPaint(Paint.ANTI_ALIAS_FLAG)
146
+ fillPaint.applySign(sign, scaleToViewY)
147
+ fillPaint.color = Color.argb(alpha, Color.red(sign.colour or ALPHA_MASK), Color.green(sign.colour or ALPHA_MASK), Color.blue(sign.colour or ALPHA_MASK))
148
+
149
+ // Wrapping is the script's job, not ours: a sign is placed by hand and its line breaks are
150
+ // written into the text as \N. A width of "as wide as it wants" reproduces that.
151
+ val textWidth = Layout.getDesiredWidth(sign.text, fillPaint).toInt() + 1
152
+ // A multi-line sign wraps its lines the way its own alignment says, not always centred: a
153
+ // left-anchored placard reads wrong with its second line pulled to the middle.
154
+ val blockAlignment = when ((sign.alignment - 1) % 3) {
155
+ 0 -> Layout.Alignment.ALIGN_NORMAL
156
+ 2 -> Layout.Alignment.ALIGN_OPPOSITE
157
+ else -> Layout.Alignment.ALIGN_CENTER
158
+ }
159
+ val layout = StaticLayout.Builder
160
+ .obtain(sign.text, 0, sign.text.length, fillPaint, textWidth)
161
+ .setAlignment(blockAlignment)
162
+ .setIncludePad(false)
163
+ .build()
164
+
165
+ val anchorX = sign.x * scaleToViewX
166
+ val anchorY = sign.y * scaleToViewY
167
+ // \pos names the point the cue's *alignment corner* sits on, so the box is offset from it.
168
+ val left = anchorX - when ((sign.alignment - 1) % 3) {
169
+ 0 -> 0f
170
+ 2 -> layout.width.toFloat()
171
+ else -> layout.width / 2f
172
+ }
173
+ val top = anchorY - when ((sign.alignment - 1) / 3) {
174
+ 0 -> layout.height.toFloat()
175
+ 2 -> 0f
176
+ else -> layout.height / 2f
177
+ }
178
+
179
+ // \org moves the centre of rotation away from the text, which is how a sign stays glued to a
180
+ // placard that is itself pivoting off-screen. Without it every rotation would spin about the
181
+ // text's own middle and drift off the artwork.
182
+ val pivotX = sign.pivotX * scaleToViewX
183
+ val pivotY = sign.pivotY * scaleToViewY
184
+
185
+ matrix.reset()
186
+ if (sign.rotationX != 0f || sign.rotationY != 0f) {
187
+ camera.save()
188
+ camera.rotateX(sign.rotationX)
189
+ camera.rotateY(sign.rotationY)
190
+ camera.getMatrix(matrix)
191
+ camera.restore()
192
+ // Camera transforms about the origin, so the pivot has to be walked to it and back.
193
+ matrix.preTranslate(-pivotX, -pivotY)
194
+ matrix.postTranslate(pivotX, pivotY)
195
+ }
196
+ matrix.postScale(sign.scaleX, sign.scaleY, pivotX, pivotY)
197
+ matrix.postRotate(sign.rotationZ, pivotX, pivotY)
198
+
199
+ val stroke = if (sign.outlineWidth <= 0f) null else {
200
+ val strokePaint = TextPaint(Paint.ANTI_ALIAS_FLAG)
201
+ strokePaint.style = Paint.Style.STROKE
202
+ strokePaint.applySign(sign, scaleToViewY)
203
+ strokePaint.strokeWidth = sign.outlineWidth * scaleToViewY * 2f
204
+ strokePaint.color = Color.argb(alpha, Color.red(sign.outlineColour or ALPHA_MASK), Color.green(sign.outlineColour or ALPHA_MASK), Color.blue(sign.outlineColour or ALPHA_MASK))
205
+ // The stroke is centred on the glyph edge, so half of it lands inside the letter. Drawing the
206
+ // fill afterwards covers that half and leaves the outline the width the script asked for.
207
+ StaticLayout.Builder
208
+ .obtain(sign.text, 0, sign.text.length, strokePaint, textWidth)
209
+ .setAlignment(blockAlignment)
210
+ .setIncludePad(false)
211
+ .build()
212
+ }
213
+
214
+ return PreparedSign(Matrix(matrix), left, top, layout, stroke)
215
+ }
216
+
217
+ private fun TextPaint.applySign(sign: AssSign, scaleToViewY: Float) {
218
+ textSize = sign.fontSize * scaleToViewY
219
+ typeface = Typeface.create(
220
+ Typeface.DEFAULT,
221
+ when {
222
+ sign.bold && sign.italic -> Typeface.BOLD_ITALIC
223
+ sign.bold -> Typeface.BOLD
224
+ sign.italic -> Typeface.ITALIC
225
+ else -> Typeface.NORMAL
226
+ },
227
+ )
228
+ isUnderlineText = sign.underline
229
+ }
230
+
231
+ companion object {
232
+ /** ASS colours are stored without an alpha byte; opacity travels separately. */
233
+ private const val ALPHA_MASK = 0xFF000000.toInt()
234
+
235
+ private fun signOf(cue: Cue): AssSign? {
236
+ val text = cue.text as? android.text.Spanned ?: return null
237
+ return text.getSpans(0, text.length, AssSignSpan::class.java).firstOrNull()?.sign
238
+ }
239
+
240
+ /**
241
+ * Inserts the overlay in front of `PlayerView`'s subtitle view, or returns the one already
242
+ * there.
243
+ *
244
+ * Returns null when the player view has no subtitle view — a configuration this package does
245
+ * not produce, but `PlayerView` allows.
246
+ */
247
+ /** Removes the overlay from [playerView], dropping its listener registration with it. */
248
+ fun remove(playerView: PlayerView) {
249
+ val parent = playerView.subtitleView?.parent as? ViewGroup ?: return
250
+ for (index in 0 until parent.childCount) {
251
+ val overlay = parent.getChildAt(index) as? AssSignOverlay ?: continue
252
+ overlay.detach()
253
+ parent.removeView(overlay)
254
+ return
255
+ }
256
+ }
257
+
258
+ fun install(playerView: PlayerView): AssSignOverlay? {
259
+ val subtitleView: SubtitleView = playerView.subtitleView ?: return null
260
+ val parent = subtitleView.parent as? ViewGroup ?: return null
261
+
262
+ for (index in 0 until parent.childCount) {
263
+ (parent.getChildAt(index) as? AssSignOverlay)?.let { return it }
264
+ }
265
+
266
+ val overlay = AssSignOverlay(playerView.context)
267
+ // Directly in front of the captions: a sign belongs to the picture, and the dialogue box
268
+ // should not be able to cover it.
269
+ parent.addView(overlay, parent.indexOfChild(subtitleView) + 1, ViewGroup.LayoutParams(
270
+ ViewGroup.LayoutParams.MATCH_PARENT,
271
+ ViewGroup.LayoutParams.MATCH_PARENT,
272
+ ))
273
+ return overlay
274
+ }
275
+ }
276
+ }
@@ -0,0 +1,510 @@
1
+ package expo.modules.video.utils
2
+
3
+ /**
4
+ * Reads the typesetting out of an ASS script: the events that are drawn *onto the picture* rather
5
+ * than laid out at the bottom, and the transforms that put them there.
6
+ *
7
+ * ### Why this exists at all
8
+ *
9
+ * Media3 cannot render a rotated caption. `SubtitlePainter` contains no call to `rotate`, `skew`,
10
+ * `Matrix` or `Camera`, and `Cue` has no rotation field — only `shearDegrees`, which the painter
11
+ * ignores. `SsaParser` never reads the tags either: `SsaStyle.Overrides` parses `\pos`, `\move` and
12
+ * `\an` and discards every other override block.
13
+ *
14
+ * That is fine for dialogue and wrong for signs. In one episode of Jujutsu Kaisen S3, 3,723 events
15
+ * carry `\frz` — 1,304 of them past 5°, 104 past 45° — because the placards they sit on are drawn
16
+ * at an angle and the translation has to lie on the placard. Rendered flat, the text reads as
17
+ * pasted over the artwork instead of part of it.
18
+ *
19
+ * So signs are taken out of Media3's hands. This parser produces them; [AssSignOverlay] draws them
20
+ * with Android's own view transforms, which map onto ASS almost one for one.
21
+ *
22
+ * ### What counts as a sign
23
+ *
24
+ * Anything the author placed: an event carrying `\pos` or `\move`, or whose effective alignment is
25
+ * not bottom-centre. The complement — no positioning tag, alignment 2 — is dialogue, and is exactly
26
+ * the set [SsaCueNormalizer] recognises, so the two agree on every event and nothing is drawn twice
27
+ * or dropped.
28
+ *
29
+ * ### What is deliberately not read
30
+ *
31
+ * `\t` animation, `\fad`, `\clip`, `\blur`, `\be`, vector drawings and karaoke. Those need a real
32
+ * ASS renderer. This is a subset chosen because it is what the eye notices first: where the text
33
+ * is, which way it faces, how big it is, and whether it is fading.
34
+ */
35
+ internal object AssSignParser {
36
+
37
+ private const val DEFAULT_PLAY_RES_X = 384f
38
+ private const val DEFAULT_PLAY_RES_Y = 288f
39
+
40
+ /**
41
+ * Matches one override block, `{…}`. The contents are scanned separately.
42
+ *
43
+ * Both braces are escaped. A bare closing brace is legal in `java.util.regex` and a syntax error
44
+ * in ICU, which is the engine Android actually uses — so the unescaped form compiles and passes
45
+ * on a JVM test run, then throws `PatternSyntaxException` from a static initialiser on device and
46
+ * takes the playback thread down with it.
47
+ */
48
+ private val BRACES = Regex("""\{([^}]*)\}""")
49
+
50
+ /**
51
+ * Splits a script into the part this package draws and the part Media3 draws.
52
+ *
53
+ * Both come out of one pass. Handing the delegate a copy with the sign lines removed is what
54
+ * makes double-drawing structurally impossible — there is no matching step to get wrong, because
55
+ * an event is in exactly one of the two outputs.
56
+ */
57
+ fun parse(content: String): AssScript {
58
+ var playResX = DEFAULT_PLAY_RES_X
59
+ var playResY = DEFAULT_PLAY_RES_Y
60
+ var sawX = false
61
+ var sawY = false
62
+
63
+ val styles = HashMap<String, AssStyle>()
64
+ val events = ArrayList<RawEvent>()
65
+ val sourceLines = content.split('\n')
66
+ // Line numbers of the Dialogue events, so the ones this package takes over can be cut out of
67
+ // the copy handed to the delegate.
68
+ val eventLines = ArrayList<Int>()
69
+
70
+ var section = ""
71
+ var styleFields: List<String> = emptyList()
72
+ var eventFields: List<String> = emptyList()
73
+ var legacyStyles = false
74
+
75
+ for ((lineNumber, rawLine) in sourceLines.withIndex()) {
76
+ val line = rawLine.trim()
77
+ if (line.isEmpty() || line.startsWith(";")) continue
78
+
79
+ if (line.startsWith("[") && line.endsWith("]")) {
80
+ section = line.lowercase()
81
+ legacyStyles = section == "[v4 styles]"
82
+ continue
83
+ }
84
+
85
+ // First colon only: a Title: may contain colons of its own.
86
+ val colon = line.indexOf(':')
87
+ if (colon == -1) continue
88
+ val key = line.substring(0, colon).trim()
89
+ val value = line.substring(colon + 1).trim()
90
+
91
+ when (section) {
92
+ "[script info]" -> when {
93
+ key.equals("PlayResX", true) -> value.toFloatOrNull()?.takeIf { it > 0 }?.let { playResX = it; sawX = true }
94
+ key.equals("PlayResY", true) -> value.toFloatOrNull()?.takeIf { it > 0 }?.let { playResY = it; sawY = true }
95
+ }
96
+
97
+ "[v4+ styles]", "[v4 styles]" -> when {
98
+ key.equals("Format", true) -> styleFields = value.split(',').map { it.trim().lowercase() }
99
+ key.equals("Style", true) && styleFields.isNotEmpty() ->
100
+ readStyle(value, styleFields, legacyStyles)?.let { styles[it.first] = it.second }
101
+ }
102
+
103
+ "[events]" -> when {
104
+ key.equals("Format", true) -> eventFields = value.split(',').map { it.trim().lowercase() }
105
+ // Only Dialogue reaches the screen. Comment holds notes and unused alternates.
106
+ key.equals("Dialogue", true) && eventFields.isNotEmpty() ->
107
+ readEvent(value, eventFields)?.let { events.add(it); eventLines.add(lineNumber) }
108
+ }
109
+ }
110
+ }
111
+
112
+ // VSFilter's rule: one dimension given, derive the other at 4:3.
113
+ if (sawX && !sawY) playResY = playResX * 3f / 4f
114
+ if (sawY && !sawX) playResX = playResY * 4f / 3f
115
+
116
+ val signs = ArrayList<AssSign>()
117
+ val removed = HashSet<Int>()
118
+ for ((index, event) in events.withIndex()) {
119
+ when (val resolution = resolve(event, styles, playResX, playResY)) {
120
+ is Resolution.Sign -> {
121
+ signs.add(resolution.sign)
122
+ removed.add(eventLines[index])
123
+ }
124
+ // Invisible styles, vector drawings and events with nothing to read. Media3 would render
125
+ // the first two as text.
126
+ Resolution.Drop -> removed.add(eventLines[index])
127
+ Resolution.Dialogue -> Unit
128
+ }
129
+ }
130
+
131
+ val dialogueOnly = if (removed.isEmpty()) {
132
+ content
133
+ } else {
134
+ sourceLines.filterIndexed { index, _ -> index !in removed }.joinToString("\n")
135
+ }
136
+
137
+ return AssScript(playResX, playResY, signs, dialogueOnly)
138
+ }
139
+
140
+ private sealed interface Resolution {
141
+ /** Typeset by the author; drawn by [AssSignOverlay]. */
142
+ data class Sign(val sign: AssSign) : Resolution
143
+
144
+ /** Bottom-centre with no positioning tag; left to Media3. */
145
+ data object Dialogue : Resolution
146
+
147
+ /** Renders nothing, or renders something no viewer was meant to read. */
148
+ data object Drop : Resolution
149
+ }
150
+
151
+ // ---------------------------------------------------------------------------------------------
152
+ // Styles
153
+ // ---------------------------------------------------------------------------------------------
154
+
155
+ private fun readStyle(value: String, fields: List<String>, legacy: Boolean): Pair<String, AssStyle>? {
156
+ val parts = value.split(',').map { it.trim() }
157
+ fun field(name: String): String? = fields.indexOf(name).takeIf { it >= 0 && it < parts.size }?.let { parts[it] }
158
+
159
+ val name = field("name") ?: return null
160
+ val rawAlignment = field("alignment")?.toIntOrNull() ?: 2
161
+ val alignment = if (legacy) legacyAlignment(rawAlignment) else rawAlignment.takeIf { it in 1..9 } ?: 2
162
+
163
+ val fontSize = field("fontsize")?.toFloatOrNull() ?: 48f
164
+ val primary = parseAssColour(field("primarycolour"))
165
+ val outline = parseAssColour(field("outlinecolour"))
166
+
167
+ return name to AssStyle(
168
+ alignment = alignment,
169
+ fontSize = fontSize,
170
+ colour = primary?.rgb ?: 0xFFFFFF,
171
+ alpha = primary?.alpha ?: 1f,
172
+ outlineColour = outline?.rgb ?: 0x000000,
173
+ outlineAlpha = outline?.alpha ?: 1f,
174
+ outlineWidth = field("outline")?.toFloatOrNull() ?: 0f,
175
+ bold = field("bold") == "-1",
176
+ italic = field("italic") == "-1",
177
+ underline = field("underline") == "-1",
178
+ scaleX = (field("scalex")?.toFloatOrNull() ?: 100f) / 100f,
179
+ scaleY = (field("scaley")?.toFloatOrNull() ?: 100f) / 100f,
180
+ angle = field("angle")?.toFloatOrNull() ?: 0f,
181
+ marginL = field("marginl")?.toFloatOrNull() ?: 0f,
182
+ marginR = field("marginr")?.toFloatOrNull() ?: 0f,
183
+ marginV = field("marginv")?.toFloatOrNull() ?: 0f,
184
+ // Fontsize 0 or a fully transparent primary colour is how a typesetter parks working notes
185
+ // in a script they still have to ship. See SsaCueNormalizer for the same rule on the Media3
186
+ // side; both have to agree or the notes come back on one path and not the other.
187
+ invisible = fontSize <= 0f || (primary?.alpha ?: 1f) == 0f,
188
+ )
189
+ }
190
+
191
+ /** SSA v4 packs the vertical into bits (+4 top, +8 middle) instead of using the numpad. */
192
+ private fun legacyAlignment(raw: Int): Int {
193
+ val horizontal = raw and 0x3
194
+ val h = when (horizontal) {
195
+ 1 -> 1; 2 -> 2; 3 -> 3
196
+ else -> 2
197
+ }
198
+ return when {
199
+ raw and 0x8 != 0 -> h + 3 // middle
200
+ raw and 0x4 != 0 -> h + 6 // top
201
+ else -> h // bottom
202
+ }
203
+ }
204
+
205
+ // ---------------------------------------------------------------------------------------------
206
+ // Events
207
+ // ---------------------------------------------------------------------------------------------
208
+
209
+ private fun readEvent(value: String, fields: List<String>): RawEvent? {
210
+ val textIndex = fields.indexOf("text")
211
+ if (textIndex == -1) return null
212
+
213
+ // The Text column is last by definition and may contain commas, so split on the first N only
214
+ // and take the whole remainder. Splitting on every comma is how a line of dialogue loses its
215
+ // second half.
216
+ val parts = value.split(',', limit = fields.size)
217
+ if (parts.size <= textIndex) return null
218
+ fun field(name: String): String? = fields.indexOf(name).takeIf { it >= 0 && it < parts.size }?.let { parts[it].trim() }
219
+
220
+ val start = parseTimeUs(field("start") ?: return null) ?: return null
221
+ val end = parseTimeUs(field("end") ?: return null) ?: return null
222
+ if (end <= start) return null
223
+
224
+ return RawEvent(
225
+ startUs = start,
226
+ endUs = end,
227
+ styleName = field("style") ?: "Default",
228
+ marginL = field("marginl")?.toFloatOrNull() ?: 0f,
229
+ marginR = field("marginr")?.toFloatOrNull() ?: 0f,
230
+ marginV = field("marginv")?.toFloatOrNull() ?: 0f,
231
+ raw = parts[textIndex],
232
+ )
233
+ }
234
+
235
+ /** `H:MM:SS.cc`, centiseconds. */
236
+ private fun parseTimeUs(value: String): Long? {
237
+ val parts = value.split(':')
238
+ if (parts.size != 3) return null
239
+ val h = parts[0].trim().toLongOrNull() ?: return null
240
+ val m = parts[1].trim().toLongOrNull() ?: return null
241
+ val s = parts[2].trim().toDoubleOrNull() ?: return null
242
+ return ((h * 3600 + m * 60) * 1_000_000L) + (s * 1_000_000.0).toLong()
243
+ }
244
+
245
+ // ---------------------------------------------------------------------------------------------
246
+ // Overrides
247
+ // ---------------------------------------------------------------------------------------------
248
+
249
+ private fun resolve(event: RawEvent, styles: Map<String, AssStyle>, resX: Float, resY: Float): Resolution {
250
+ val style = styles[event.styleName] ?: AssStyle.DEFAULT
251
+ if (style.invisible) return Resolution.Drop
252
+
253
+ var alignment = style.alignment
254
+ var posX: Float? = null
255
+ var posY: Float? = null
256
+ var orgX: Float? = null
257
+ var orgY: Float? = null
258
+ var frz = style.angle
259
+ var frx = 0f
260
+ var fry = 0f
261
+ var fscx = style.scaleX
262
+ var fscy = style.scaleY
263
+ var fontSize = style.fontSize
264
+ var colour = style.colour
265
+ var alpha = style.alpha
266
+ var bold = style.bold
267
+ var italic = style.italic
268
+ var underline = style.underline
269
+ var drawing = false
270
+
271
+ for (block in BRACES.findAll(event.raw)) {
272
+ val body = block.groupValues[1]
273
+ for (tag in scanTags(body)) {
274
+ when (tag.name) {
275
+ "an" -> tag.arg.toIntOrNull()?.takeIf { it in 1..9 }?.let { alignment = it }
276
+ "a" -> tag.arg.toIntOrNull()?.let { alignment = legacyAlignment(it) }
277
+ "pos" -> readPoint(tag.arg)?.let { posX = it.first; posY = it.second }
278
+ // A move ends where it ends; drawing it at the destination beats drawing it at the start.
279
+ "move" -> readPoint(tag.arg, from = 2)?.let { posX = it.first; posY = it.second }
280
+ "org" -> readPoint(tag.arg)?.let { orgX = it.first; orgY = it.second }
281
+ "frz", "fr" -> tag.arg.toFloatOrNull()?.let { frz = it }
282
+ "frx" -> tag.arg.toFloatOrNull()?.let { frx = it }
283
+ "fry" -> tag.arg.toFloatOrNull()?.let { fry = it }
284
+ "fscx" -> tag.arg.toFloatOrNull()?.let { fscx = it / 100f }
285
+ "fscy" -> tag.arg.toFloatOrNull()?.let { fscy = it / 100f }
286
+ "fs" -> tag.arg.toFloatOrNull()?.takeIf { it > 0 }?.let { fontSize = it }
287
+ "c", "1c" -> parseAssColour(tag.arg)?.let { colour = it.rgb }
288
+ "alpha", "1a" -> parseAssColour(tag.arg, alphaOnly = true)?.let { alpha = it.alpha }
289
+ "b" -> bold = tag.arg.trim() != "0"
290
+ "i" -> italic = tag.arg.trim() == "1"
291
+ "u" -> underline = tag.arg.trim() == "1"
292
+ // A \p with a non-zero argument switches the text into vector drawing commands. Digits
293
+ // are required so \pos and \pbo are not mistaken for it.
294
+ "p" -> if ((tag.arg.toIntOrNull() ?: 0) > 0) drawing = true
295
+ }
296
+ }
297
+ }
298
+
299
+ if (drawing) return Resolution.Drop
300
+
301
+ val text = strip(event.raw)
302
+ if (text.isBlank()) return Resolution.Drop
303
+
304
+ val hasPosition = posX != null && posY != null
305
+ // The complement of SsaCueNormalizer's dialogue test. Anything it would clear stays with
306
+ // Media3; everything else is a sign and is drawn here.
307
+ if (!hasPosition && alignment == 2) return Resolution.Dialogue
308
+
309
+ val marginL = if (event.marginL != 0f) event.marginL else style.marginL
310
+ val marginR = if (event.marginR != 0f) event.marginR else style.marginR
311
+ val marginV = if (event.marginV != 0f) event.marginV else style.marginV
312
+
313
+ val x = posX ?: when ((alignment - 1) % 3) {
314
+ 0 -> marginL
315
+ 2 -> resX - marginR
316
+ else -> resX / 2f
317
+ }
318
+ val y = posY ?: when ((alignment - 1) / 3) {
319
+ 0 -> resY - marginV
320
+ 2 -> marginV
321
+ else -> resY / 2f
322
+ }
323
+
324
+ return Resolution.Sign(AssSign(
325
+ startUs = event.startUs,
326
+ endUs = event.endUs,
327
+ text = text,
328
+ x = x,
329
+ y = y,
330
+ alignment = alignment,
331
+ // ASS measures counter-clockwise, Android clockwise.
332
+ rotationZ = -frz,
333
+ // The two 3-D axes are a guess at the sign convention and are unverified on a device. They
334
+ // are kept because they are almost always tiny — in the episode this was built against, 3,363
335
+ // of 5,126 \frx values and 2,715 of 3,579 \fry values are under 3° — so getting a sign wrong
336
+ // is a subtle wrongness rather than a visible one. \frz is the one that carries the effect.
337
+ rotationX = frx,
338
+ rotationY = -fry,
339
+ pivotX = orgX ?: x,
340
+ pivotY = orgY ?: y,
341
+ scaleX = fscx,
342
+ scaleY = fscy,
343
+ fontSize = fontSize,
344
+ colour = colour,
345
+ alpha = alpha,
346
+ outlineColour = style.outlineColour,
347
+ outlineWidth = style.outlineWidth,
348
+ bold = bold,
349
+ italic = italic,
350
+ underline = underline,
351
+ playResX = resX,
352
+ playResY = resY,
353
+ ))
354
+ }
355
+
356
+ private data class Tag(val name: String, val arg: String)
357
+
358
+ /**
359
+ * Reads `\name arg` pairs out of one override block, left to right.
360
+ *
361
+ * Longest name wins, which is the whole reason this is a scanner and not a list of regexes: `\c`
362
+ * must not swallow `\clip`, and `\b` must not swallow `\bord`, `\blur` or `\be`.
363
+ */
364
+ private fun scanTags(body: String): List<Tag> {
365
+ val tags = ArrayList<Tag>()
366
+ var i = 0
367
+ while (i < body.length) {
368
+ if (body[i] != '\\') { i++; continue }
369
+ i++
370
+ val nameStart = i
371
+ // A leading digit is part of the name, not the argument: \1c, \3a and \4c all start with
372
+ // one. Letters then run to the end of the name, longest first, which is what keeps \c from
373
+ // swallowing \clip and \b from swallowing \bord, \blur and \be.
374
+ while (i < body.length && body[i].isDigit()) i++
375
+ while (i < body.length && body[i].isLetter()) i++
376
+ val name = body.substring(nameStart, i).lowercase()
377
+ if (name.isEmpty()) continue
378
+
379
+ val argStart = i
380
+ if (i < body.length && body[i] == '(') {
381
+ var depth = 0
382
+ while (i < body.length) {
383
+ if (body[i] == '(') depth++
384
+ if (body[i] == ')') { depth--; if (depth == 0) { i++; break } }
385
+ i++
386
+ }
387
+ tags.add(Tag(name, body.substring(argStart + 1, (i - 1).coerceAtLeast(argStart + 1))))
388
+ } else {
389
+ while (i < body.length && body[i] != '\\') i++
390
+ tags.add(Tag(name, body.substring(argStart, i)))
391
+ }
392
+ }
393
+ return tags
394
+ }
395
+
396
+ private fun readPoint(arg: String, from: Int = 0): Pair<Float, Float>? {
397
+ val parts = arg.split(',').map { it.trim() }
398
+ if (parts.size < from + 2) return null
399
+ val x = parts[from].toFloatOrNull() ?: return null
400
+ val y = parts[from + 1].toFloatOrNull() ?: return null
401
+ return x to y
402
+ }
403
+
404
+ /** Override blocks out, ASS escapes in. Mirrors what `SsaParser` does to the same string. */
405
+ private fun strip(raw: String): String =
406
+ BRACES.replace(raw, "")
407
+ .replace("\\N", "\n")
408
+ .replace("\\n", "\n")
409
+ .replace("\\h", " ")
410
+
411
+ internal data class AssColour(val rgb: Int, val alpha: Float)
412
+
413
+ /**
414
+ * `&HAABBGGRR&` or `&HBBGGRR&`, with or without the ampersands, or a bare decimal.
415
+ *
416
+ * The alpha byte is inverted in ASS — `FF` is fully transparent — and is absent from a six-digit
417
+ * value, which is therefore opaque.
418
+ */
419
+ internal fun parseAssColour(value: String?, alphaOnly: Boolean = false): AssColour? {
420
+ var text = value?.trim()?.removeSuffix("&") ?: return null
421
+ text = if (text.startsWith("&H", true)) text.substring(2) else text
422
+ if (text.isEmpty()) return null
423
+
424
+ val hex = text.toLongOrNull(16) ?: return null
425
+ if (alphaOnly || text.length <= 2) {
426
+ return AssColour(rgb = 0xFFFFFF, alpha = 1f - ((hex and 0xFF) / 255f))
427
+ }
428
+
429
+ val rgb = (((hex and 0xFF) shl 16) or (((hex shr 8) and 0xFF) shl 8) or ((hex shr 16) and 0xFF)).toInt()
430
+ val alpha = if (text.length > 6) 1f - (((hex shr 24) and 0xFF) / 255f) else 1f
431
+ return AssColour(rgb, alpha)
432
+ }
433
+
434
+ private data class RawEvent(
435
+ val startUs: Long,
436
+ val endUs: Long,
437
+ val styleName: String,
438
+ val marginL: Float,
439
+ val marginR: Float,
440
+ val marginV: Float,
441
+ val raw: String,
442
+ )
443
+ }
444
+
445
+ internal data class AssStyle(
446
+ val alignment: Int,
447
+ val fontSize: Float,
448
+ val colour: Int,
449
+ val alpha: Float,
450
+ val outlineColour: Int,
451
+ val outlineAlpha: Float,
452
+ val outlineWidth: Float,
453
+ val bold: Boolean,
454
+ val italic: Boolean,
455
+ val underline: Boolean,
456
+ val scaleX: Float,
457
+ val scaleY: Float,
458
+ val angle: Float,
459
+ val marginL: Float,
460
+ val marginR: Float,
461
+ val marginV: Float,
462
+ val invisible: Boolean,
463
+ ) {
464
+ companion object {
465
+ val DEFAULT = AssStyle(
466
+ alignment = 2, fontSize = 48f, colour = 0xFFFFFF, alpha = 1f,
467
+ outlineColour = 0x000000, outlineAlpha = 1f, outlineWidth = 0f,
468
+ bold = false, italic = false, underline = false,
469
+ scaleX = 1f, scaleY = 1f, angle = 0f,
470
+ marginL = 0f, marginR = 0f, marginV = 0f, invisible = false,
471
+ )
472
+ }
473
+ }
474
+
475
+ /** One typeset event, in script coordinates. [AssSignOverlay] maps it onto the video rect. */
476
+ internal data class AssSign(
477
+ val startUs: Long,
478
+ val endUs: Long,
479
+ val text: String,
480
+ val x: Float,
481
+ val y: Float,
482
+ val alignment: Int,
483
+ val rotationZ: Float,
484
+ val rotationX: Float,
485
+ val rotationY: Float,
486
+ val pivotX: Float,
487
+ val pivotY: Float,
488
+ val scaleX: Float,
489
+ val scaleY: Float,
490
+ val fontSize: Float,
491
+ val colour: Int,
492
+ val alpha: Float,
493
+ val outlineColour: Int,
494
+ val outlineWidth: Float,
495
+ val bold: Boolean,
496
+ val italic: Boolean,
497
+ val underline: Boolean,
498
+ /** The coordinate space x, y, pivot and fontSize are expressed in. */
499
+ val playResX: Float,
500
+ val playResY: Float,
501
+ )
502
+
503
+ internal data class AssScript(
504
+ val playResX: Float,
505
+ val playResY: Float,
506
+ /** Typeset events, drawn by [AssSignOverlay]. */
507
+ val signs: List<AssSign>,
508
+ /** The script with every sign, drawing and invisible event cut out, for Media3's own parser. */
509
+ val dialogueOnly: String,
510
+ )
@@ -0,0 +1,23 @@
1
+ package expo.modules.video.utils
2
+
3
+ /**
4
+ * Carries one parsed ASS sign through Media3's cue stream to [AssSignOverlay].
5
+ *
6
+ * The cue itself is a single zero-width character, so `SubtitleView` lays it out and draws nothing
7
+ * while the overlay reads the sign off this span and draws it properly. Riding the cue stream is
8
+ * what buys the timing for free: Media3 already decides which cues are on screen at each moment,
9
+ * and a sign is on screen exactly as long as its cue is.
10
+ *
11
+ * A marker span rather than a field on `Cue` because `Cue` is final and carries no room for one —
12
+ * it has `shearDegrees` and nothing else transform-shaped, and `SubtitlePainter` ignores even that.
13
+ */
14
+ internal class AssSignSpan(val sign: AssSign)
15
+
16
+ /**
17
+ * The cue text a sign rides on.
18
+ *
19
+ * A zero-width space still occupies a line of height in `SubtitleView`, which is harmless for a
20
+ * positioned cue and keeps the span anchored to something. Distinct from [HELD_SLOT_PLACEHOLDER]
21
+ * only in intent; both are invisible.
22
+ */
23
+ internal const val SIGN_CUE_PLACEHOLDER = "​"
@@ -191,6 +191,9 @@ class RoundedSubtitleBackground private constructor(private val backgroundView:
191
191
  */
192
192
  private fun toBackgroundCue(cue: Cue): Cue? {
193
193
  val text = cue.text ?: return null // Bitmap cues carry no text to put a box behind.
194
+ // A sign's cue is a zero-width placeholder standing in for text drawn by `AssSignOverlay`.
195
+ // There is no glyph here to sit behind, and the box would be the only thing on screen.
196
+ if (text is Spanned && text.getSpans(0, text.length, AssSignSpan::class.java).isNotEmpty()) return null
194
197
  val stripped = if (keepMetricSpans && text is Spanned) {
195
198
  SpannableString(text).also { spannable ->
196
199
  for (span in spannable.getSpans(0, spannable.length, Any::class.java)) {
@@ -1,12 +1,15 @@
1
1
  package expo.modules.video.utils
2
2
 
3
+ import android.text.SpannableString
3
4
  import android.text.SpannableStringBuilder
5
+ import android.text.Spanned
4
6
  import androidx.annotation.OptIn
5
7
  import androidx.media3.common.C
6
8
  import androidx.media3.common.Format
7
9
  import androidx.media3.common.MimeTypes
8
10
  import androidx.media3.common.text.Cue
9
11
  import androidx.media3.common.util.Consumer
12
+ import androidx.media3.common.util.Log
10
13
  import androidx.media3.common.util.UnstableApi
11
14
  import androidx.media3.extractor.text.CuesWithTiming
12
15
  import androidx.media3.extractor.text.DefaultSubtitleParserFactory
@@ -73,6 +76,26 @@ open class StackingSubtitleParser(
73
76
  private val prepare: (Cue) -> Cue? = { it },
74
77
  ) : SubtitleParser {
75
78
 
79
+ /**
80
+ * A chance for a subclass to take some events out of the delegate's hands and supply its own cues
81
+ * for them. Returns the bytes the delegate should parse plus the cues the subclass is drawing.
82
+ *
83
+ * Default is to change nothing.
84
+ */
85
+ protected open fun split(data: ByteArray, offset: Int, length: Int): Split =
86
+ Split(data, offset, length, emptyList())
87
+
88
+ /** [split]'s result: what the delegate parses, and what the subclass contributes. */
89
+ protected class Split(
90
+ val data: ByteArray,
91
+ val offset: Int,
92
+ val length: Int,
93
+ val ownCues: List<PreparedCue>,
94
+ )
95
+
96
+ /** One cue a subclass supplies, with the timing the delegate would otherwise have derived. */
97
+ protected class PreparedCue(val cue: Cue, val startUs: Long, val endUs: Long)
98
+
76
99
  // Every emitted segment is a complete description of what should be on screen for its interval,
77
100
  // so segments replace each other rather than being merged by the renderer.
78
101
  override fun getCueReplacementBehavior(): Int = Format.CUE_REPLACEMENT_BEHAVIOR_REPLACE
@@ -89,9 +112,13 @@ open class StackingSubtitleParser(
89
112
  output: Consumer<CuesWithTiming>
90
113
  ) {
91
114
  val entries = mutableListOf<Entry>()
115
+ val split = split(data, offset, length)
116
+ for (prepared in split.ownCues) {
117
+ entries.add(Entry(prepared.cue, prepared.startUs, prepared.endUs, entries.size))
118
+ }
92
119
  // Always ask the delegate for everything; `outputOptions` is applied to the merged result below,
93
120
  // because a segment can span cues that fall on both sides of the requested start time.
94
- delegate.parse(data, offset, length, SubtitleParser.OutputOptions.allCues()) { cuesWithTiming ->
121
+ delegate.parse(split.data, split.offset, split.length, SubtitleParser.OutputOptions.allCues()) { cuesWithTiming ->
95
122
  val startUs = cuesWithTiming.startTimeUs
96
123
  val endUs = resolveEndUs(cuesWithTiming)
97
124
  for (cue in cuesWithTiming.cues) {
@@ -291,7 +318,16 @@ open class StackingSubtitleParser(
291
318
  class StackingSubripParser : StackingSubtitleParser(SubripParser())
292
319
 
293
320
  /**
294
- * ASS and SSA, via [SsaCueNormalizer].
321
+ * ASS and SSA.
322
+ *
323
+ * Splits the script in two before anything else runs. [AssSignParser] takes the typeset events —
324
+ * anything the author placed — and the delegate gets a copy with those lines removed, so the two
325
+ * renderers can never draw the same event. Signs come back as cues carrying an [AssSignSpan], which
326
+ * rides Media3's own timing all the way to [AssSignOverlay]; the cue text itself is a zero-width
327
+ * character, so `SubtitleView` lays it out and draws nothing.
328
+ *
329
+ * Falls back to plain delegation when the script yields no signs, which is every `.ass` that is
330
+ * only dialogue, and when the bytes are not decodable as text.
295
331
  *
296
332
  * The delegate is built by the caller rather than constructed here because an SSA track muxed into
297
333
  * a container carries its `[V4+ Styles]` header in the format's initialization data, and only
@@ -299,7 +335,50 @@ class StackingSubripParser : StackingSubtitleParser(SubripParser())
299
335
  */
300
336
  @OptIn(UnstableApi::class)
301
337
  class StackingSsaParser(delegate: SubtitleParser) :
302
- StackingSubtitleParser(delegate, SsaCueNormalizer::normalize)
338
+ StackingSubtitleParser(delegate, SsaCueNormalizer::normalize) {
339
+
340
+ override fun split(data: ByteArray, offset: Int, length: Int): Split {
341
+ // Nothing in here is worth a crash, and the whole body is inside the guard rather than just the
342
+ // parse. This runs on the extractor's loading thread, where an exception does not fail the
343
+ // subtitle — it takes playback down with it, which is how a stray `}` in a regex turned into a
344
+ // dead player on an episode that had been playing fine.
345
+ //
346
+ // A subtitle file is untrusted input from a scraper. The honest failure for one this cannot
347
+ // read is captions drawn Media3's own way, flat, rather than no video at all.
348
+ return runCatching {
349
+ val script = AssSignParser.parse(String(data, offset, length, Charsets.UTF_8))
350
+ if (script.signs.isEmpty()) {
351
+ Split(data, offset, length, emptyList())
352
+ } else {
353
+ val dialogue = script.dialogueOnly.toByteArray(Charsets.UTF_8)
354
+ Split(dialogue, 0, dialogue.size, script.signs.map(::toEntry))
355
+ }
356
+ }.getOrElse { error ->
357
+ Log.w(TAG, "Sign rendering unavailable for this track; falling back to Media3", error)
358
+ Split(data, offset, length, emptyList())
359
+ }
360
+ }
361
+
362
+ private companion object {
363
+ const val TAG = "StackingSsaParser"
364
+ }
365
+
366
+ private fun toEntry(sign: AssSign): PreparedCue {
367
+ val text = SpannableString(SIGN_CUE_PLACEHOLDER)
368
+ text.setSpan(AssSignSpan(sign), 0, text.length, Spanned.SPAN_EXCLUSIVE_EXCLUSIVE)
369
+
370
+ // Positioned so the stacking pass leaves it alone and the span filter keeps it intact. The
371
+ // values are only a rough placement for the invisible placeholder; the overlay reads the real
372
+ // coordinates off the span.
373
+ val cue = Cue.Builder()
374
+ .setText(text)
375
+ .setPosition((sign.x / sign.playResX).coerceIn(0f, 1f))
376
+ .setLine((sign.y / sign.playResY).coerceIn(0f, 1f), Cue.LINE_TYPE_FRACTION)
377
+ .build()
378
+
379
+ return PreparedCue(cue, sign.startUs, sign.endUs)
380
+ }
381
+ }
303
382
 
304
383
  /**
305
384
  * Routes SubRip and ASS/SSA through the stacking parsers so simultaneous cues stack, delegates every
@@ -98,6 +98,12 @@ object SubtitleUtils {
98
98
  if (!roundedBackgroundApplied) {
99
99
  RoundedSubtitleBackground.remove(playerView)
100
100
  }
101
+
102
+ // Independent of every style decision above: an ASS sign is drawn where and how the script
103
+ // says, not how the viewer's caption settings say, so nothing here applies to it. Installed
104
+ // unconditionally because the track can be switched to an `.ass` at any point, and rebinding
105
+ // on each reconfigure is what carries it across a player swap.
106
+ AssSignOverlay.install(playerView)?.attachTo(playerView.player)
101
107
  }
102
108
  }
103
109
 
@@ -108,6 +114,7 @@ object SubtitleUtils {
108
114
  */
109
115
  fun releaseSubtitleView(playerView: PlayerView) {
110
116
  RoundedSubtitleBackground.remove(playerView)
117
+ AssSignOverlay.remove(playerView)
111
118
  }
112
119
 
113
120
  /**
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "expo-video-subtitle",
3
- "version": "0.6.0",
3
+ "version": "0.7.1",
4
4
  "description": "A cross-platform, performant video component for React Native and Expo with external subtitle sideloading and custom subtitle styling.",
5
5
  "keywords": [
6
6
  "expo",