expo-video-subtitle 0.6.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +47 -0
- package/FORK.md +3 -0
- package/android/build.gradle +5 -0
- package/android/src/main/java/expo/modules/video/utils/AssSignOverlay.kt +269 -0
- package/android/src/main/java/expo/modules/video/utils/AssSignParser.kt +503 -0
- package/android/src/main/java/expo/modules/video/utils/AssSignSpan.kt +23 -0
- package/android/src/main/java/expo/modules/video/utils/RoundedSubtitleBackground.kt +3 -0
- package/android/src/main/java/expo/modules/video/utils/StackingSubtitleParser.kt +66 -3
- package/android/src/main/java/expo/modules/video/utils/SubtitleUtils.kt +7 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,53 @@ Changes inherited from the upstream base version are not repeated here — see t
|
|
|
7
7
|
[upstream changelog](https://github.com/expo/expo/blob/main/packages/expo-video/CHANGELOG.md)
|
|
8
8
|
for the history of everything that isn't subtitle related.
|
|
9
9
|
|
|
10
|
+
## 0.7.0
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- **ASS signs are drawn with the transforms the script asks for (Android).** A typeset sign is
|
|
15
|
+
placed on the artwork, and the artwork is usually not square to the screen: in one episode of
|
|
16
|
+
Jujutsu Kaisen S3, 3,723 events carry `\frz`, 1,304 of them past 5° and 104 past 45°, because the
|
|
17
|
+
placards they sit on are drawn at an angle. Rendered flat, the translation reads as pasted over
|
|
18
|
+
the picture instead of lying on it.
|
|
19
|
+
|
|
20
|
+
Media3 cannot do this and has no room to. `SubtitlePainter` contains no call to `rotate`, `skew`,
|
|
21
|
+
`Matrix` or `Camera`; `Cue` carries no rotation field, only `shearDegrees`, which the painter
|
|
22
|
+
ignores; and `SsaParser` discards every override block except `\pos`, `\move` and `\an`.
|
|
23
|
+
|
|
24
|
+
So signs leave Media3's hands entirely. `AssSignParser` splits a script in two — typeset events
|
|
25
|
+
to us, dialogue to the delegate, with the sign lines cut out of the copy the delegate parses, so
|
|
26
|
+
double-drawing is structurally impossible rather than merely avoided. Each sign rides the cue
|
|
27
|
+
stream as a zero-width placeholder carrying an `AssSignSpan`, which buys Media3's timing for free,
|
|
28
|
+
and `AssSignOverlay` draws it on a canvas in front of the captions.
|
|
29
|
+
|
|
30
|
+
Read and honoured: `\pos`, `\move` (at its destination), `\org`, `\an` and the legacy `\a`,
|
|
31
|
+
`\frz`, `\frx`, `\fry`, `\fscx`, `\fscy`, `\fs`, `\c` / `\1c`, `\alpha` / `\1a`, `\b`,
|
|
32
|
+
`\i`, `\u`, style `Alignment`, `Fontsize`, `PrimaryColour`, `OutlineColour`, `Outline`,
|
|
33
|
+
`ScaleX/Y`, `Angle` and the margins, plus `PlayResX` / `PlayResY` and VSFilter's 4:3 rule when one
|
|
34
|
+
is missing.
|
|
35
|
+
|
|
36
|
+
The overlay sits in the player's content frame, which is sized to the video rather than the view,
|
|
37
|
+
so positions survive letterboxing.
|
|
38
|
+
|
|
39
|
+
### Known limitations
|
|
40
|
+
|
|
41
|
+
- `\t` animation, `\fad`, `\clip`, `\blur`, `\be`, `\fax` shear, vector drawings and karaoke are
|
|
42
|
+
still dropped. Reproducing those means libass, not a larger subset of it.
|
|
43
|
+
- Fonts fall back to the system face: a script naming `Bogart Semibold` gets the default at the
|
|
44
|
+
right size and weight, not that typeface.
|
|
45
|
+
- `\frx` and `\fry` are applied with an unverified sign convention. They are almost always tiny —
|
|
46
|
+
3,363 of 5,126 `\frx` values and 2,715 of 3,579 `\fry` values in that episode are under 3° — so a
|
|
47
|
+
wrong sign is subtle rather than visible. `\frz` is what carries the effect and is verified
|
|
48
|
+
against the source line it came from.
|
|
49
|
+
- iOS still draws signs flat. AVFoundation renders WebVTT itself and has no rotation to give.
|
|
50
|
+
|
|
51
|
+
### Changed
|
|
52
|
+
|
|
53
|
+
- The module now has JVM unit tests (`./gradlew :expo-video-subtitle:testDebugUnitTest`). The sign
|
|
54
|
+
parser is pure Kotlin with no Android imports precisely so it can be tested this way: its tag
|
|
55
|
+
scanner and colour handling fail by drawing a sign in the wrong place, which nothing else catches.
|
|
56
|
+
|
|
10
57
|
## 0.6.0
|
|
11
58
|
|
|
12
59
|
### Fixed
|
package/FORK.md
CHANGED
|
@@ -29,6 +29,9 @@ Measured against the published `expo-video@56.1.4` tarball. `+`/`-` are added/re
|
|
|
29
29
|
| `android/…/records/SubtitleStyle.kt` | 152 | Record → `CaptionStyleCompat`; hex colour parsing; font-family resolution (`res/font` → `ReactFontManager`); the `applyEmbeddedStyles` flag, which `SubtitleUtils` reads directly rather than `toCaptionStyle` and which defaults to `true` since 0.5.0. |
|
|
30
30
|
| `android/…/utils/StackingSubtitleParser.kt` | 342 | `SubtitleParser` that segments the cue timeline and merges simultaneous **unpositioned** cues into one stacked block, plus the `SubtitleParser.Factory` that routes SubRip and ASS/SSA to it and wraps every parser in `FilteringSubtitleParser`. ASS gets there through `SsaCueNormalizer` as the `prepare` step — without it `SsaParser` sets both `line` and `position` on every cue and they would all take the passthrough branch. Assigns each cue a vertical slot it keeps for its whole time on screen — a finished cue's line is held blank (`\u200B`) so the cues above it do not drop. Merges through a `SpannableStringBuilder` so cue spans survive. |
|
|
31
31
|
| `android/…/utils/SsaCueNormalizer.kt` | 93 | Clears the `line`/`position` Media3's `SsaParser` fills in on cues the ASS author never placed, so `bottomOffset` and cue stacking work on `.ass` the way they already do on `.srt`. The test is exact rather than heuristic: those values are a pure function of the cue's anchors, so matching both defaults for the bottom-centre anchors *is* "no positioning tag". Also drops events on a style the author made invisible (`Fontsize 0`, or a primary colour whose inverted alpha byte is `FF`) — that is where typesetters park working notes, and nothing downstream can carry "invisible". |
|
|
32
|
+
| `android/…/utils/AssSignParser.kt` | 505 | Splits an ASS script into the typeset events this package draws and a copy of the script with those lines removed for Media3's parser. One pass, no matching step, so an event is in exactly one of the two outputs and cannot be drawn twice. Pure Kotlin with no Android imports, which is what makes it unit-testable on the JVM. Reads `\pos`, `\move`, `\org`, `\an`, `\frz`, `\frx`, `\fry`, `\fscx/y`, `\fs`, `\c`, `\alpha`, `\b`, `\i`, `\u` and the style columns behind them. The tag scanner takes the longest name first — `\c` must not swallow `\clip`, `\b` must not swallow `\bord`, `\blur` or `\be` — and reads a leading digit as part of the name for `\1c` and `\4a`. |
|
|
33
|
+
| `android/…/utils/AssSignOverlay.kt` | 235 | Draws those signs on a canvas in front of the captions, with `Matrix` for rotation and scale and `Camera` for the two 3-D axes. **Media3 cannot do this**: `SubtitlePainter` has no rotate, skew, Matrix or Camera call, and `Cue` has no rotation field. Sits in the same parent as `SubtitleView` — the content frame, sized to the video, which is why `\pos` survives letterboxing. Lays text out on change rather than per frame, because frame-by-frame typesetting changes the on-screen set every 40 ms. |
|
|
34
|
+
| `android/…/utils/AssSignSpan.kt` | 24 | Carries one parsed sign through Media3's cue stream to the overlay, on a zero-width placeholder so `SubtitleView` draws nothing. Riding the cue stream is what buys the timing for free. |
|
|
32
35
|
| `android/…/utils/RoundedSubtitleBackground.kt` | 352 | Draws `subtitleStyle.backgroundRadius` as rounded boxes, which Media3 cannot do at all: it paints `backgroundColor` as a framework `BackgroundColorSpan` and `windowColor` with `canvas.drawRect`, neither takes a radius, and `SubtitlePainter`/`CanvasSubtitleOutput` are package-private final while `SubtitleView` is final. A second `SubtitleView` is inserted behind `PlayerView`'s own, fed from `Player.Listener.onCues`, drawing a `LineBackgroundSpan` per line. **Its `EDGE_TYPE_NONE` is load-bearing** — `SubtitlePainter` copies the cue text for an edge pass and draws both layouts when the edge type is outline/raised/depressed, which would paint the box twice and darken a semi-transparent colour. State hangs off the view's `tag`, never a map in the companion object; see the `find` KDoc for why even a `WeakHashMap` leaks here. |
|
|
33
36
|
| `FORK.md`, `LICENSE` | — | This guide; MIT licence (upstream copyright preserved). |
|
|
34
37
|
| `eslint.config.js` | 4 | Verbatim copy of `expo-module-scripts/templates/eslint.config.js`. **Recreate it if a sync drops it** — `expo-module configure` treats it as an *optional* template and only syncs it when it already exists, so without this file `npm run lint` dies with "ESLint couldn't find an eslint.config.js". |
|
package/android/build.gradle
CHANGED
|
@@ -26,6 +26,11 @@ dependencies {
|
|
|
26
26
|
implementation "androidx.media3:media3-ui:${androidxMedia3Version}"
|
|
27
27
|
implementation "androidx.media3:media3-datasource-okhttp:${androidxMedia3Version}"
|
|
28
28
|
|
|
29
|
+
// The ASS sign parser is pure Kotlin with no Android imports, which is what makes it worth
|
|
30
|
+
// testing on the JVM: its tag scanner and colour handling are the parts that fail silently on a
|
|
31
|
+
// device, by drawing a sign in the wrong place rather than by crashing.
|
|
32
|
+
testImplementation 'junit:junit:4.13.2'
|
|
33
|
+
|
|
29
34
|
def fragment_version = "1.8.9"
|
|
30
35
|
implementation "androidx.fragment:fragment:$fragment_version"
|
|
31
36
|
implementation "androidx.fragment:fragment-ktx:$fragment_version"
|
|
@@ -0,0 +1,269 @@
|
|
|
1
|
+
package expo.modules.video.utils
|
|
2
|
+
|
|
3
|
+
import android.annotation.SuppressLint
|
|
4
|
+
import android.content.Context
|
|
5
|
+
import android.graphics.Camera
|
|
6
|
+
import android.graphics.Canvas
|
|
7
|
+
import android.graphics.Color
|
|
8
|
+
import android.graphics.Matrix
|
|
9
|
+
import android.graphics.Paint
|
|
10
|
+
import android.graphics.Typeface
|
|
11
|
+
import android.text.Layout
|
|
12
|
+
import android.text.StaticLayout
|
|
13
|
+
import android.text.TextPaint
|
|
14
|
+
import android.view.View
|
|
15
|
+
import android.view.ViewGroup
|
|
16
|
+
import androidx.annotation.OptIn
|
|
17
|
+
import androidx.media3.common.Player
|
|
18
|
+
import androidx.media3.common.text.Cue
|
|
19
|
+
import androidx.media3.common.text.CueGroup
|
|
20
|
+
import androidx.media3.common.util.UnstableApi
|
|
21
|
+
import androidx.media3.ui.PlayerView
|
|
22
|
+
import androidx.media3.ui.SubtitleView
|
|
23
|
+
|
|
24
|
+
/**
|
|
25
|
+
* Draws ASS typesetting — signs, placards, on-screen labels — with the transforms the script asks
|
|
26
|
+
* for.
|
|
27
|
+
*
|
|
28
|
+
* `SubtitleView` cannot: `SubtitlePainter` has no rotation, and `Cue` has no field to carry one.
|
|
29
|
+
* So [AssSignParser] takes those events out of Media3's cue stream, wraps each in an [AssSignSpan],
|
|
30
|
+
* and this view draws them itself.
|
|
31
|
+
*
|
|
32
|
+
* ### Where it sits
|
|
33
|
+
*
|
|
34
|
+
* As a sibling of `PlayerView`'s own `SubtitleView`, in front of it. That parent is the content
|
|
35
|
+
* frame, which is sized to the *video*, not to the player — which is the whole reason positions
|
|
36
|
+
* survive letterboxing. `\pos` is in `PlayResX`/`PlayResY` space and maps onto this view's bounds
|
|
37
|
+
* directly. [RoundedSubtitleBackground] takes the same route for the same reason.
|
|
38
|
+
*
|
|
39
|
+
* ### Why a canvas and not child views
|
|
40
|
+
*
|
|
41
|
+
* A `TextView` per sign would work — Android view transforms map onto ASS almost one for one — but
|
|
42
|
+
* a heavily typeset episode puts dozens of signs on screen across a scene and each would cost a
|
|
43
|
+
* measure and layout pass. Drawing straight to the canvas costs one `StaticLayout` per sign per
|
|
44
|
+
* frame it changes, and it is the only way to stroke the outline the way ASS means it: the border
|
|
45
|
+
* drawn *behind* the fill, not as a shadow.
|
|
46
|
+
*/
|
|
47
|
+
@SuppressLint("ViewConstructor")
|
|
48
|
+
@OptIn(UnstableApi::class)
|
|
49
|
+
internal class AssSignOverlay(context: Context) : View(context), Player.Listener {
|
|
50
|
+
|
|
51
|
+
private val camera = Camera()
|
|
52
|
+
private val matrix = Matrix()
|
|
53
|
+
|
|
54
|
+
private var signs: List<AssSign> = emptyList()
|
|
55
|
+
private var prepared: List<PreparedSign> = emptyList()
|
|
56
|
+
private var player: Player? = null
|
|
57
|
+
|
|
58
|
+
init {
|
|
59
|
+
setWillNotDraw(false)
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
override fun onCues(cueGroup: CueGroup) {
|
|
63
|
+
setSigns(cueGroup.cues)
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
private fun setSigns(cues: List<Cue>) {
|
|
67
|
+
val next = cues.mapNotNull(::signOf)
|
|
68
|
+
// Reference equality on the list contents is enough: the parser produces one AssSign per event
|
|
69
|
+
// and hands out the same instance for every segment the event spans.
|
|
70
|
+
if (next.size == signs.size && next.indices.all { next[it] === signs[it] }) return
|
|
71
|
+
signs = next
|
|
72
|
+
rebuild()
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
override fun onSizeChanged(w: Int, h: Int, oldw: Int, oldh: Int) {
|
|
76
|
+
super.onSizeChanged(w, h, oldw, oldh)
|
|
77
|
+
// Every measurement below is in view pixels, so all of it is stale after a resize — which
|
|
78
|
+
// happens on every rotation and on entering full screen.
|
|
79
|
+
rebuild()
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
/**
|
|
83
|
+
* Lays the signs out once per change rather than once per frame.
|
|
84
|
+
*
|
|
85
|
+
* Frame-by-frame typesetting changes the on-screen set every 40 ms, and `onDraw` can run more
|
|
86
|
+
* often than that for reasons of the view system's own. Text layout is the expensive half of
|
|
87
|
+
* drawing, and none of it depends on anything that varies between frames.
|
|
88
|
+
*/
|
|
89
|
+
private fun rebuild() {
|
|
90
|
+
prepared = if (width == 0 || height == 0) emptyList() else signs.mapNotNull(::prepare)
|
|
91
|
+
invalidate()
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
fun attachTo(player: Player?) {
|
|
95
|
+
if (this.player === player) return
|
|
96
|
+
this.player?.removeListener(this)
|
|
97
|
+
this.player = player
|
|
98
|
+
player?.addListener(this)
|
|
99
|
+
// A listener only hears about the *next* cue change, so seed from what is on screen already.
|
|
100
|
+
setSigns(player?.currentCues?.cues ?: emptyList())
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
fun detach() {
|
|
104
|
+
player?.removeListener(this)
|
|
105
|
+
player = null
|
|
106
|
+
signs = emptyList()
|
|
107
|
+
rebuild()
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
override fun onDraw(canvas: Canvas) {
|
|
111
|
+
for (sign in prepared) {
|
|
112
|
+
draw(canvas, sign)
|
|
113
|
+
}
|
|
114
|
+
}
|
|
115
|
+
|
|
116
|
+
private fun draw(canvas: Canvas, sign: PreparedSign) {
|
|
117
|
+
canvas.save()
|
|
118
|
+
canvas.concat(sign.matrix)
|
|
119
|
+
canvas.translate(sign.left, sign.top)
|
|
120
|
+
sign.stroke?.draw(canvas)
|
|
121
|
+
sign.fill.draw(canvas)
|
|
122
|
+
canvas.restore()
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
/** Everything about one sign that does not change between frames. */
|
|
126
|
+
private class PreparedSign(
|
|
127
|
+
val matrix: Matrix,
|
|
128
|
+
val left: Float,
|
|
129
|
+
val top: Float,
|
|
130
|
+
val fill: StaticLayout,
|
|
131
|
+
val stroke: StaticLayout?,
|
|
132
|
+
)
|
|
133
|
+
|
|
134
|
+
private fun prepare(sign: AssSign): PreparedSign? {
|
|
135
|
+
// Script space to view space. Both axes are scaled independently so a script authored at a
|
|
136
|
+
// different aspect ratio than the video still lands where it was drawn.
|
|
137
|
+
val scaleToViewX = width / sign.playResX
|
|
138
|
+
val scaleToViewY = height / sign.playResY
|
|
139
|
+
|
|
140
|
+
val alpha = (sign.alpha * 255f).toInt().coerceIn(0, 255)
|
|
141
|
+
if (alpha == 0) return null
|
|
142
|
+
|
|
143
|
+
// A fresh paint per sign: StaticLayout keeps the reference it was built with, so a shared
|
|
144
|
+
// mutable one would repaint every sign already laid out in this pass.
|
|
145
|
+
val fillPaint = TextPaint(Paint.ANTI_ALIAS_FLAG)
|
|
146
|
+
fillPaint.applySign(sign, scaleToViewY)
|
|
147
|
+
fillPaint.color = Color.argb(alpha, Color.red(sign.colour or ALPHA_MASK), Color.green(sign.colour or ALPHA_MASK), Color.blue(sign.colour or ALPHA_MASK))
|
|
148
|
+
|
|
149
|
+
// Wrapping is the script's job, not ours: a sign is placed by hand and its line breaks are
|
|
150
|
+
// written into the text as \N. A width of "as wide as it wants" reproduces that.
|
|
151
|
+
val textWidth = Layout.getDesiredWidth(sign.text, fillPaint).toInt() + 1
|
|
152
|
+
val layout = StaticLayout.Builder
|
|
153
|
+
.obtain(sign.text, 0, sign.text.length, fillPaint, textWidth)
|
|
154
|
+
.setAlignment(Layout.Alignment.ALIGN_CENTER)
|
|
155
|
+
.setIncludePad(false)
|
|
156
|
+
.build()
|
|
157
|
+
|
|
158
|
+
val anchorX = sign.x * scaleToViewX
|
|
159
|
+
val anchorY = sign.y * scaleToViewY
|
|
160
|
+
// \pos names the point the cue's *alignment corner* sits on, so the box is offset from it.
|
|
161
|
+
val left = anchorX - when ((sign.alignment - 1) % 3) {
|
|
162
|
+
0 -> 0f
|
|
163
|
+
2 -> layout.width.toFloat()
|
|
164
|
+
else -> layout.width / 2f
|
|
165
|
+
}
|
|
166
|
+
val top = anchorY - when ((sign.alignment - 1) / 3) {
|
|
167
|
+
0 -> layout.height.toFloat()
|
|
168
|
+
2 -> 0f
|
|
169
|
+
else -> layout.height / 2f
|
|
170
|
+
}
|
|
171
|
+
|
|
172
|
+
// \org moves the centre of rotation away from the text, which is how a sign stays glued to a
|
|
173
|
+
// placard that is itself pivoting off-screen. Without it every rotation would spin about the
|
|
174
|
+
// text's own middle and drift off the artwork.
|
|
175
|
+
val pivotX = sign.pivotX * scaleToViewX
|
|
176
|
+
val pivotY = sign.pivotY * scaleToViewY
|
|
177
|
+
|
|
178
|
+
matrix.reset()
|
|
179
|
+
if (sign.rotationX != 0f || sign.rotationY != 0f) {
|
|
180
|
+
camera.save()
|
|
181
|
+
camera.rotateX(sign.rotationX)
|
|
182
|
+
camera.rotateY(sign.rotationY)
|
|
183
|
+
camera.getMatrix(matrix)
|
|
184
|
+
camera.restore()
|
|
185
|
+
// Camera transforms about the origin, so the pivot has to be walked to it and back.
|
|
186
|
+
matrix.preTranslate(-pivotX, -pivotY)
|
|
187
|
+
matrix.postTranslate(pivotX, pivotY)
|
|
188
|
+
}
|
|
189
|
+
matrix.postScale(sign.scaleX, sign.scaleY, pivotX, pivotY)
|
|
190
|
+
matrix.postRotate(sign.rotationZ, pivotX, pivotY)
|
|
191
|
+
|
|
192
|
+
val stroke = if (sign.outlineWidth <= 0f) null else {
|
|
193
|
+
val strokePaint = TextPaint(Paint.ANTI_ALIAS_FLAG)
|
|
194
|
+
strokePaint.style = Paint.Style.STROKE
|
|
195
|
+
strokePaint.applySign(sign, scaleToViewY)
|
|
196
|
+
strokePaint.strokeWidth = sign.outlineWidth * scaleToViewY * 2f
|
|
197
|
+
strokePaint.color = Color.argb(alpha, Color.red(sign.outlineColour or ALPHA_MASK), Color.green(sign.outlineColour or ALPHA_MASK), Color.blue(sign.outlineColour or ALPHA_MASK))
|
|
198
|
+
// The stroke is centred on the glyph edge, so half of it lands inside the letter. Drawing the
|
|
199
|
+
// fill afterwards covers that half and leaves the outline the width the script asked for.
|
|
200
|
+
StaticLayout.Builder
|
|
201
|
+
.obtain(sign.text, 0, sign.text.length, strokePaint, textWidth)
|
|
202
|
+
.setAlignment(Layout.Alignment.ALIGN_CENTER)
|
|
203
|
+
.setIncludePad(false)
|
|
204
|
+
.build()
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
return PreparedSign(Matrix(matrix), left, top, layout, stroke)
|
|
208
|
+
}
|
|
209
|
+
|
|
210
|
+
private fun TextPaint.applySign(sign: AssSign, scaleToViewY: Float) {
|
|
211
|
+
textSize = sign.fontSize * scaleToViewY
|
|
212
|
+
typeface = Typeface.create(
|
|
213
|
+
Typeface.DEFAULT,
|
|
214
|
+
when {
|
|
215
|
+
sign.bold && sign.italic -> Typeface.BOLD_ITALIC
|
|
216
|
+
sign.bold -> Typeface.BOLD
|
|
217
|
+
sign.italic -> Typeface.ITALIC
|
|
218
|
+
else -> Typeface.NORMAL
|
|
219
|
+
},
|
|
220
|
+
)
|
|
221
|
+
isUnderlineText = sign.underline
|
|
222
|
+
}
|
|
223
|
+
|
|
224
|
+
companion object {
|
|
225
|
+
/** ASS colours are stored without an alpha byte; opacity travels separately. */
|
|
226
|
+
private const val ALPHA_MASK = 0xFF000000.toInt()
|
|
227
|
+
|
|
228
|
+
private fun signOf(cue: Cue): AssSign? {
|
|
229
|
+
val text = cue.text as? android.text.Spanned ?: return null
|
|
230
|
+
return text.getSpans(0, text.length, AssSignSpan::class.java).firstOrNull()?.sign
|
|
231
|
+
}
|
|
232
|
+
|
|
233
|
+
/**
|
|
234
|
+
* Inserts the overlay in front of `PlayerView`'s subtitle view, or returns the one already
|
|
235
|
+
* there.
|
|
236
|
+
*
|
|
237
|
+
* Returns null when the player view has no subtitle view — a configuration this package does
|
|
238
|
+
* not produce, but `PlayerView` allows.
|
|
239
|
+
*/
|
|
240
|
+
/** Removes the overlay from [playerView], dropping its listener registration with it. */
|
|
241
|
+
fun remove(playerView: PlayerView) {
|
|
242
|
+
val parent = playerView.subtitleView?.parent as? ViewGroup ?: return
|
|
243
|
+
for (index in 0 until parent.childCount) {
|
|
244
|
+
val overlay = parent.getChildAt(index) as? AssSignOverlay ?: continue
|
|
245
|
+
overlay.detach()
|
|
246
|
+
parent.removeView(overlay)
|
|
247
|
+
return
|
|
248
|
+
}
|
|
249
|
+
}
|
|
250
|
+
|
|
251
|
+
fun install(playerView: PlayerView): AssSignOverlay? {
|
|
252
|
+
val subtitleView: SubtitleView = playerView.subtitleView ?: return null
|
|
253
|
+
val parent = subtitleView.parent as? ViewGroup ?: return null
|
|
254
|
+
|
|
255
|
+
for (index in 0 until parent.childCount) {
|
|
256
|
+
(parent.getChildAt(index) as? AssSignOverlay)?.let { return it }
|
|
257
|
+
}
|
|
258
|
+
|
|
259
|
+
val overlay = AssSignOverlay(playerView.context)
|
|
260
|
+
// Directly in front of the captions: a sign belongs to the picture, and the dialogue box
|
|
261
|
+
// should not be able to cover it.
|
|
262
|
+
parent.addView(overlay, parent.indexOfChild(subtitleView) + 1, ViewGroup.LayoutParams(
|
|
263
|
+
ViewGroup.LayoutParams.MATCH_PARENT,
|
|
264
|
+
ViewGroup.LayoutParams.MATCH_PARENT,
|
|
265
|
+
))
|
|
266
|
+
return overlay
|
|
267
|
+
}
|
|
268
|
+
}
|
|
269
|
+
}
|
|
@@ -0,0 +1,503 @@
|
|
|
1
|
+
package expo.modules.video.utils
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* Reads the typesetting out of an ASS script: the events that are drawn *onto the picture* rather
|
|
5
|
+
* than laid out at the bottom, and the transforms that put them there.
|
|
6
|
+
*
|
|
7
|
+
* ### Why this exists at all
|
|
8
|
+
*
|
|
9
|
+
* Media3 cannot render a rotated caption. `SubtitlePainter` contains no call to `rotate`, `skew`,
|
|
10
|
+
* `Matrix` or `Camera`, and `Cue` has no rotation field — only `shearDegrees`, which the painter
|
|
11
|
+
* ignores. `SsaParser` never reads the tags either: `SsaStyle.Overrides` parses `\pos`, `\move` and
|
|
12
|
+
* `\an` and discards every other override block.
|
|
13
|
+
*
|
|
14
|
+
* That is fine for dialogue and wrong for signs. In one episode of Jujutsu Kaisen S3, 3,723 events
|
|
15
|
+
* carry `\frz` — 1,304 of them past 5°, 104 past 45° — because the placards they sit on are drawn
|
|
16
|
+
* at an angle and the translation has to lie on the placard. Rendered flat, the text reads as
|
|
17
|
+
* pasted over the artwork instead of part of it.
|
|
18
|
+
*
|
|
19
|
+
* So signs are taken out of Media3's hands. This parser produces them; [AssSignOverlay] draws them
|
|
20
|
+
* with Android's own view transforms, which map onto ASS almost one for one.
|
|
21
|
+
*
|
|
22
|
+
* ### What counts as a sign
|
|
23
|
+
*
|
|
24
|
+
* Anything the author placed: an event carrying `\pos` or `\move`, or whose effective alignment is
|
|
25
|
+
* not bottom-centre. The complement — no positioning tag, alignment 2 — is dialogue, and is exactly
|
|
26
|
+
* the set [SsaCueNormalizer] recognises, so the two agree on every event and nothing is drawn twice
|
|
27
|
+
* or dropped.
|
|
28
|
+
*
|
|
29
|
+
* ### What is deliberately not read
|
|
30
|
+
*
|
|
31
|
+
* `\t` animation, `\fad`, `\clip`, `\blur`, `\be`, vector drawings and karaoke. Those need a real
|
|
32
|
+
* ASS renderer. This is a subset chosen because it is what the eye notices first: where the text
|
|
33
|
+
* is, which way it faces, how big it is, and whether it is fading.
|
|
34
|
+
*/
|
|
35
|
+
internal object AssSignParser {
|
|
36
|
+
|
|
37
|
+
private const val DEFAULT_PLAY_RES_X = 384f
|
|
38
|
+
private const val DEFAULT_PLAY_RES_Y = 288f
|
|
39
|
+
|
|
40
|
+
/** Matches one override block, `{…}`. The contents are scanned separately. */
|
|
41
|
+
private val BRACES = Regex("""\{([^}]*)}""")
|
|
42
|
+
|
|
43
|
+
/**
|
|
44
|
+
* Splits a script into the part this package draws and the part Media3 draws.
|
|
45
|
+
*
|
|
46
|
+
* Both come out of one pass. Handing the delegate a copy with the sign lines removed is what
|
|
47
|
+
* makes double-drawing structurally impossible — there is no matching step to get wrong, because
|
|
48
|
+
* an event is in exactly one of the two outputs.
|
|
49
|
+
*/
|
|
50
|
+
fun parse(content: String): AssScript {
|
|
51
|
+
var playResX = DEFAULT_PLAY_RES_X
|
|
52
|
+
var playResY = DEFAULT_PLAY_RES_Y
|
|
53
|
+
var sawX = false
|
|
54
|
+
var sawY = false
|
|
55
|
+
|
|
56
|
+
val styles = HashMap<String, AssStyle>()
|
|
57
|
+
val events = ArrayList<RawEvent>()
|
|
58
|
+
val sourceLines = content.split('\n')
|
|
59
|
+
// Line numbers of the Dialogue events, so the ones this package takes over can be cut out of
|
|
60
|
+
// the copy handed to the delegate.
|
|
61
|
+
val eventLines = ArrayList<Int>()
|
|
62
|
+
|
|
63
|
+
var section = ""
|
|
64
|
+
var styleFields: List<String> = emptyList()
|
|
65
|
+
var eventFields: List<String> = emptyList()
|
|
66
|
+
var legacyStyles = false
|
|
67
|
+
|
|
68
|
+
for ((lineNumber, rawLine) in sourceLines.withIndex()) {
|
|
69
|
+
val line = rawLine.trim()
|
|
70
|
+
if (line.isEmpty() || line.startsWith(";")) continue
|
|
71
|
+
|
|
72
|
+
if (line.startsWith("[") && line.endsWith("]")) {
|
|
73
|
+
section = line.lowercase()
|
|
74
|
+
legacyStyles = section == "[v4 styles]"
|
|
75
|
+
continue
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
// First colon only: a Title: may contain colons of its own.
|
|
79
|
+
val colon = line.indexOf(':')
|
|
80
|
+
if (colon == -1) continue
|
|
81
|
+
val key = line.substring(0, colon).trim()
|
|
82
|
+
val value = line.substring(colon + 1).trim()
|
|
83
|
+
|
|
84
|
+
when (section) {
|
|
85
|
+
"[script info]" -> when {
|
|
86
|
+
key.equals("PlayResX", true) -> value.toFloatOrNull()?.takeIf { it > 0 }?.let { playResX = it; sawX = true }
|
|
87
|
+
key.equals("PlayResY", true) -> value.toFloatOrNull()?.takeIf { it > 0 }?.let { playResY = it; sawY = true }
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
"[v4+ styles]", "[v4 styles]" -> when {
|
|
91
|
+
key.equals("Format", true) -> styleFields = value.split(',').map { it.trim().lowercase() }
|
|
92
|
+
key.equals("Style", true) && styleFields.isNotEmpty() ->
|
|
93
|
+
readStyle(value, styleFields, legacyStyles)?.let { styles[it.first] = it.second }
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
"[events]" -> when {
|
|
97
|
+
key.equals("Format", true) -> eventFields = value.split(',').map { it.trim().lowercase() }
|
|
98
|
+
// Only Dialogue reaches the screen. Comment holds notes and unused alternates.
|
|
99
|
+
key.equals("Dialogue", true) && eventFields.isNotEmpty() ->
|
|
100
|
+
readEvent(value, eventFields)?.let { events.add(it); eventLines.add(lineNumber) }
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
// VSFilter's rule: one dimension given, derive the other at 4:3.
|
|
106
|
+
if (sawX && !sawY) playResY = playResX * 3f / 4f
|
|
107
|
+
if (sawY && !sawX) playResX = playResY * 4f / 3f
|
|
108
|
+
|
|
109
|
+
val signs = ArrayList<AssSign>()
|
|
110
|
+
val removed = HashSet<Int>()
|
|
111
|
+
for ((index, event) in events.withIndex()) {
|
|
112
|
+
when (val resolution = resolve(event, styles, playResX, playResY)) {
|
|
113
|
+
is Resolution.Sign -> {
|
|
114
|
+
signs.add(resolution.sign)
|
|
115
|
+
removed.add(eventLines[index])
|
|
116
|
+
}
|
|
117
|
+
// Invisible styles, vector drawings and events with nothing to read. Media3 would render
|
|
118
|
+
// the first two as text.
|
|
119
|
+
Resolution.Drop -> removed.add(eventLines[index])
|
|
120
|
+
Resolution.Dialogue -> Unit
|
|
121
|
+
}
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
val dialogueOnly = if (removed.isEmpty()) {
|
|
125
|
+
content
|
|
126
|
+
} else {
|
|
127
|
+
sourceLines.filterIndexed { index, _ -> index !in removed }.joinToString("\n")
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
return AssScript(playResX, playResY, signs, dialogueOnly)
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
private sealed interface Resolution {
|
|
134
|
+
/** Typeset by the author; drawn by [AssSignOverlay]. */
|
|
135
|
+
data class Sign(val sign: AssSign) : Resolution
|
|
136
|
+
|
|
137
|
+
/** Bottom-centre with no positioning tag; left to Media3. */
|
|
138
|
+
data object Dialogue : Resolution
|
|
139
|
+
|
|
140
|
+
/** Renders nothing, or renders something no viewer was meant to read. */
|
|
141
|
+
data object Drop : Resolution
|
|
142
|
+
}
|
|
143
|
+
|
|
144
|
+
// ---------------------------------------------------------------------------------------------
|
|
145
|
+
// Styles
|
|
146
|
+
// ---------------------------------------------------------------------------------------------
|
|
147
|
+
|
|
148
|
+
private fun readStyle(value: String, fields: List<String>, legacy: Boolean): Pair<String, AssStyle>? {
|
|
149
|
+
val parts = value.split(',').map { it.trim() }
|
|
150
|
+
fun field(name: String): String? = fields.indexOf(name).takeIf { it >= 0 && it < parts.size }?.let { parts[it] }
|
|
151
|
+
|
|
152
|
+
val name = field("name") ?: return null
|
|
153
|
+
val rawAlignment = field("alignment")?.toIntOrNull() ?: 2
|
|
154
|
+
val alignment = if (legacy) legacyAlignment(rawAlignment) else rawAlignment.takeIf { it in 1..9 } ?: 2
|
|
155
|
+
|
|
156
|
+
val fontSize = field("fontsize")?.toFloatOrNull() ?: 48f
|
|
157
|
+
val primary = parseAssColour(field("primarycolour"))
|
|
158
|
+
val outline = parseAssColour(field("outlinecolour"))
|
|
159
|
+
|
|
160
|
+
return name to AssStyle(
|
|
161
|
+
alignment = alignment,
|
|
162
|
+
fontSize = fontSize,
|
|
163
|
+
colour = primary?.rgb ?: 0xFFFFFF,
|
|
164
|
+
alpha = primary?.alpha ?: 1f,
|
|
165
|
+
outlineColour = outline?.rgb ?: 0x000000,
|
|
166
|
+
outlineAlpha = outline?.alpha ?: 1f,
|
|
167
|
+
outlineWidth = field("outline")?.toFloatOrNull() ?: 0f,
|
|
168
|
+
bold = field("bold") == "-1",
|
|
169
|
+
italic = field("italic") == "-1",
|
|
170
|
+
underline = field("underline") == "-1",
|
|
171
|
+
scaleX = (field("scalex")?.toFloatOrNull() ?: 100f) / 100f,
|
|
172
|
+
scaleY = (field("scaley")?.toFloatOrNull() ?: 100f) / 100f,
|
|
173
|
+
angle = field("angle")?.toFloatOrNull() ?: 0f,
|
|
174
|
+
marginL = field("marginl")?.toFloatOrNull() ?: 0f,
|
|
175
|
+
marginR = field("marginr")?.toFloatOrNull() ?: 0f,
|
|
176
|
+
marginV = field("marginv")?.toFloatOrNull() ?: 0f,
|
|
177
|
+
// Fontsize 0 or a fully transparent primary colour is how a typesetter parks working notes
|
|
178
|
+
// in a script they still have to ship. See SsaCueNormalizer for the same rule on the Media3
|
|
179
|
+
// side; both have to agree or the notes come back on one path and not the other.
|
|
180
|
+
invisible = fontSize <= 0f || (primary?.alpha ?: 1f) == 0f,
|
|
181
|
+
)
|
|
182
|
+
}
|
|
183
|
+
|
|
184
|
+
/** SSA v4 packs the vertical into bits (+4 top, +8 middle) instead of using the numpad. */
|
|
185
|
+
private fun legacyAlignment(raw: Int): Int {
|
|
186
|
+
val horizontal = raw and 0x3
|
|
187
|
+
val h = when (horizontal) {
|
|
188
|
+
1 -> 1; 2 -> 2; 3 -> 3
|
|
189
|
+
else -> 2
|
|
190
|
+
}
|
|
191
|
+
return when {
|
|
192
|
+
raw and 0x8 != 0 -> h + 3 // middle
|
|
193
|
+
raw and 0x4 != 0 -> h + 6 // top
|
|
194
|
+
else -> h // bottom
|
|
195
|
+
}
|
|
196
|
+
}
|
|
197
|
+
|
|
198
|
+
// ---------------------------------------------------------------------------------------------
|
|
199
|
+
// Events
|
|
200
|
+
// ---------------------------------------------------------------------------------------------
|
|
201
|
+
|
|
202
|
+
private fun readEvent(value: String, fields: List<String>): RawEvent? {
|
|
203
|
+
val textIndex = fields.indexOf("text")
|
|
204
|
+
if (textIndex == -1) return null
|
|
205
|
+
|
|
206
|
+
// The Text column is last by definition and may contain commas, so split on the first N only
|
|
207
|
+
// and take the whole remainder. Splitting on every comma is how a line of dialogue loses its
|
|
208
|
+
// second half.
|
|
209
|
+
val parts = value.split(',', limit = fields.size)
|
|
210
|
+
if (parts.size <= textIndex) return null
|
|
211
|
+
fun field(name: String): String? = fields.indexOf(name).takeIf { it >= 0 && it < parts.size }?.let { parts[it].trim() }
|
|
212
|
+
|
|
213
|
+
val start = parseTimeUs(field("start") ?: return null) ?: return null
|
|
214
|
+
val end = parseTimeUs(field("end") ?: return null) ?: return null
|
|
215
|
+
if (end <= start) return null
|
|
216
|
+
|
|
217
|
+
return RawEvent(
|
|
218
|
+
startUs = start,
|
|
219
|
+
endUs = end,
|
|
220
|
+
styleName = field("style") ?: "Default",
|
|
221
|
+
marginL = field("marginl")?.toFloatOrNull() ?: 0f,
|
|
222
|
+
marginR = field("marginr")?.toFloatOrNull() ?: 0f,
|
|
223
|
+
marginV = field("marginv")?.toFloatOrNull() ?: 0f,
|
|
224
|
+
raw = parts[textIndex],
|
|
225
|
+
)
|
|
226
|
+
}
|
|
227
|
+
|
|
228
|
+
/** `H:MM:SS.cc`, centiseconds. */
|
|
229
|
+
private fun parseTimeUs(value: String): Long? {
|
|
230
|
+
val parts = value.split(':')
|
|
231
|
+
if (parts.size != 3) return null
|
|
232
|
+
val h = parts[0].trim().toLongOrNull() ?: return null
|
|
233
|
+
val m = parts[1].trim().toLongOrNull() ?: return null
|
|
234
|
+
val s = parts[2].trim().toDoubleOrNull() ?: return null
|
|
235
|
+
return ((h * 3600 + m * 60) * 1_000_000L) + (s * 1_000_000.0).toLong()
|
|
236
|
+
}
|
|
237
|
+
|
|
238
|
+
// ---------------------------------------------------------------------------------------------
|
|
239
|
+
// Overrides
|
|
240
|
+
// ---------------------------------------------------------------------------------------------
|
|
241
|
+
|
|
242
|
+
private fun resolve(event: RawEvent, styles: Map<String, AssStyle>, resX: Float, resY: Float): Resolution {
|
|
243
|
+
val style = styles[event.styleName] ?: AssStyle.DEFAULT
|
|
244
|
+
if (style.invisible) return Resolution.Drop
|
|
245
|
+
|
|
246
|
+
var alignment = style.alignment
|
|
247
|
+
var posX: Float? = null
|
|
248
|
+
var posY: Float? = null
|
|
249
|
+
var orgX: Float? = null
|
|
250
|
+
var orgY: Float? = null
|
|
251
|
+
var frz = style.angle
|
|
252
|
+
var frx = 0f
|
|
253
|
+
var fry = 0f
|
|
254
|
+
var fscx = style.scaleX
|
|
255
|
+
var fscy = style.scaleY
|
|
256
|
+
var fontSize = style.fontSize
|
|
257
|
+
var colour = style.colour
|
|
258
|
+
var alpha = style.alpha
|
|
259
|
+
var bold = style.bold
|
|
260
|
+
var italic = style.italic
|
|
261
|
+
var underline = style.underline
|
|
262
|
+
var drawing = false
|
|
263
|
+
|
|
264
|
+
for (block in BRACES.findAll(event.raw)) {
|
|
265
|
+
val body = block.groupValues[1]
|
|
266
|
+
for (tag in scanTags(body)) {
|
|
267
|
+
when (tag.name) {
|
|
268
|
+
"an" -> tag.arg.toIntOrNull()?.takeIf { it in 1..9 }?.let { alignment = it }
|
|
269
|
+
"a" -> tag.arg.toIntOrNull()?.let { alignment = legacyAlignment(it) }
|
|
270
|
+
"pos" -> readPoint(tag.arg)?.let { posX = it.first; posY = it.second }
|
|
271
|
+
// A move ends where it ends; drawing it at the destination beats drawing it at the start.
|
|
272
|
+
"move" -> readPoint(tag.arg, from = 2)?.let { posX = it.first; posY = it.second }
|
|
273
|
+
"org" -> readPoint(tag.arg)?.let { orgX = it.first; orgY = it.second }
|
|
274
|
+
"frz", "fr" -> tag.arg.toFloatOrNull()?.let { frz = it }
|
|
275
|
+
"frx" -> tag.arg.toFloatOrNull()?.let { frx = it }
|
|
276
|
+
"fry" -> tag.arg.toFloatOrNull()?.let { fry = it }
|
|
277
|
+
"fscx" -> tag.arg.toFloatOrNull()?.let { fscx = it / 100f }
|
|
278
|
+
"fscy" -> tag.arg.toFloatOrNull()?.let { fscy = it / 100f }
|
|
279
|
+
"fs" -> tag.arg.toFloatOrNull()?.takeIf { it > 0 }?.let { fontSize = it }
|
|
280
|
+
"c", "1c" -> parseAssColour(tag.arg)?.let { colour = it.rgb }
|
|
281
|
+
"alpha", "1a" -> parseAssColour(tag.arg, alphaOnly = true)?.let { alpha = it.alpha }
|
|
282
|
+
"b" -> bold = tag.arg.trim() != "0"
|
|
283
|
+
"i" -> italic = tag.arg.trim() == "1"
|
|
284
|
+
"u" -> underline = tag.arg.trim() == "1"
|
|
285
|
+
// A \p with a non-zero argument switches the text into vector drawing commands. Digits
|
|
286
|
+
// are required so \pos and \pbo are not mistaken for it.
|
|
287
|
+
"p" -> if ((tag.arg.toIntOrNull() ?: 0) > 0) drawing = true
|
|
288
|
+
}
|
|
289
|
+
}
|
|
290
|
+
}
|
|
291
|
+
|
|
292
|
+
if (drawing) return Resolution.Drop
|
|
293
|
+
|
|
294
|
+
val text = strip(event.raw)
|
|
295
|
+
if (text.isBlank()) return Resolution.Drop
|
|
296
|
+
|
|
297
|
+
val hasPosition = posX != null && posY != null
|
|
298
|
+
// The complement of SsaCueNormalizer's dialogue test. Anything it would clear stays with
|
|
299
|
+
// Media3; everything else is a sign and is drawn here.
|
|
300
|
+
if (!hasPosition && alignment == 2) return Resolution.Dialogue
|
|
301
|
+
|
|
302
|
+
val marginL = if (event.marginL != 0f) event.marginL else style.marginL
|
|
303
|
+
val marginR = if (event.marginR != 0f) event.marginR else style.marginR
|
|
304
|
+
val marginV = if (event.marginV != 0f) event.marginV else style.marginV
|
|
305
|
+
|
|
306
|
+
val x = posX ?: when ((alignment - 1) % 3) {
|
|
307
|
+
0 -> marginL
|
|
308
|
+
2 -> resX - marginR
|
|
309
|
+
else -> resX / 2f
|
|
310
|
+
}
|
|
311
|
+
val y = posY ?: when ((alignment - 1) / 3) {
|
|
312
|
+
0 -> resY - marginV
|
|
313
|
+
2 -> marginV
|
|
314
|
+
else -> resY / 2f
|
|
315
|
+
}
|
|
316
|
+
|
|
317
|
+
return Resolution.Sign(AssSign(
|
|
318
|
+
startUs = event.startUs,
|
|
319
|
+
endUs = event.endUs,
|
|
320
|
+
text = text,
|
|
321
|
+
x = x,
|
|
322
|
+
y = y,
|
|
323
|
+
alignment = alignment,
|
|
324
|
+
// ASS measures counter-clockwise, Android clockwise.
|
|
325
|
+
rotationZ = -frz,
|
|
326
|
+
// The two 3-D axes are a guess at the sign convention and are unverified on a device. They
|
|
327
|
+
// are kept because they are almost always tiny — in the episode this was built against, 3,363
|
|
328
|
+
// of 5,126 \frx values and 2,715 of 3,579 \fry values are under 3° — so getting a sign wrong
|
|
329
|
+
// is a subtle wrongness rather than a visible one. \frz is the one that carries the effect.
|
|
330
|
+
rotationX = frx,
|
|
331
|
+
rotationY = -fry,
|
|
332
|
+
pivotX = orgX ?: x,
|
|
333
|
+
pivotY = orgY ?: y,
|
|
334
|
+
scaleX = fscx,
|
|
335
|
+
scaleY = fscy,
|
|
336
|
+
fontSize = fontSize,
|
|
337
|
+
colour = colour,
|
|
338
|
+
alpha = alpha,
|
|
339
|
+
outlineColour = style.outlineColour,
|
|
340
|
+
outlineWidth = style.outlineWidth,
|
|
341
|
+
bold = bold,
|
|
342
|
+
italic = italic,
|
|
343
|
+
underline = underline,
|
|
344
|
+
playResX = resX,
|
|
345
|
+
playResY = resY,
|
|
346
|
+
))
|
|
347
|
+
}
|
|
348
|
+
|
|
349
|
+
private data class Tag(val name: String, val arg: String)
|
|
350
|
+
|
|
351
|
+
/**
|
|
352
|
+
* Reads `\name arg` pairs out of one override block, left to right.
|
|
353
|
+
*
|
|
354
|
+
* Longest name wins, which is the whole reason this is a scanner and not a list of regexes: `\c`
|
|
355
|
+
* must not swallow `\clip`, and `\b` must not swallow `\bord`, `\blur` or `\be`.
|
|
356
|
+
*/
|
|
357
|
+
private fun scanTags(body: String): List<Tag> {
|
|
358
|
+
val tags = ArrayList<Tag>()
|
|
359
|
+
var i = 0
|
|
360
|
+
while (i < body.length) {
|
|
361
|
+
if (body[i] != '\\') { i++; continue }
|
|
362
|
+
i++
|
|
363
|
+
val nameStart = i
|
|
364
|
+
// A leading digit is part of the name, not the argument: \1c, \3a and \4c all start with
|
|
365
|
+
// one. Letters then run to the end of the name, longest first, which is what keeps \c from
|
|
366
|
+
// swallowing \clip and \b from swallowing \bord, \blur and \be.
|
|
367
|
+
while (i < body.length && body[i].isDigit()) i++
|
|
368
|
+
while (i < body.length && body[i].isLetter()) i++
|
|
369
|
+
val name = body.substring(nameStart, i).lowercase()
|
|
370
|
+
if (name.isEmpty()) continue
|
|
371
|
+
|
|
372
|
+
val argStart = i
|
|
373
|
+
if (i < body.length && body[i] == '(') {
|
|
374
|
+
var depth = 0
|
|
375
|
+
while (i < body.length) {
|
|
376
|
+
if (body[i] == '(') depth++
|
|
377
|
+
if (body[i] == ')') { depth--; if (depth == 0) { i++; break } }
|
|
378
|
+
i++
|
|
379
|
+
}
|
|
380
|
+
tags.add(Tag(name, body.substring(argStart + 1, (i - 1).coerceAtLeast(argStart + 1))))
|
|
381
|
+
} else {
|
|
382
|
+
while (i < body.length && body[i] != '\\') i++
|
|
383
|
+
tags.add(Tag(name, body.substring(argStart, i)))
|
|
384
|
+
}
|
|
385
|
+
}
|
|
386
|
+
return tags
|
|
387
|
+
}
|
|
388
|
+
|
|
389
|
+
private fun readPoint(arg: String, from: Int = 0): Pair<Float, Float>? {
|
|
390
|
+
val parts = arg.split(',').map { it.trim() }
|
|
391
|
+
if (parts.size < from + 2) return null
|
|
392
|
+
val x = parts[from].toFloatOrNull() ?: return null
|
|
393
|
+
val y = parts[from + 1].toFloatOrNull() ?: return null
|
|
394
|
+
return x to y
|
|
395
|
+
}
|
|
396
|
+
|
|
397
|
+
/** Override blocks out, ASS escapes in. Mirrors what `SsaParser` does to the same string. */
|
|
398
|
+
private fun strip(raw: String): String =
|
|
399
|
+
BRACES.replace(raw, "")
|
|
400
|
+
.replace("\\N", "\n")
|
|
401
|
+
.replace("\\n", "\n")
|
|
402
|
+
.replace("\\h", " ")
|
|
403
|
+
|
|
404
|
+
internal data class AssColour(val rgb: Int, val alpha: Float)
|
|
405
|
+
|
|
406
|
+
/**
|
|
407
|
+
* `&HAABBGGRR&` or `&HBBGGRR&`, with or without the ampersands, or a bare decimal.
|
|
408
|
+
*
|
|
409
|
+
* The alpha byte is inverted in ASS — `FF` is fully transparent — and is absent from a six-digit
|
|
410
|
+
* value, which is therefore opaque.
|
|
411
|
+
*/
|
|
412
|
+
internal fun parseAssColour(value: String?, alphaOnly: Boolean = false): AssColour? {
|
|
413
|
+
var text = value?.trim()?.removeSuffix("&") ?: return null
|
|
414
|
+
text = if (text.startsWith("&H", true)) text.substring(2) else text
|
|
415
|
+
if (text.isEmpty()) return null
|
|
416
|
+
|
|
417
|
+
val hex = text.toLongOrNull(16) ?: return null
|
|
418
|
+
if (alphaOnly || text.length <= 2) {
|
|
419
|
+
return AssColour(rgb = 0xFFFFFF, alpha = 1f - ((hex and 0xFF) / 255f))
|
|
420
|
+
}
|
|
421
|
+
|
|
422
|
+
val rgb = (((hex and 0xFF) shl 16) or (((hex shr 8) and 0xFF) shl 8) or ((hex shr 16) and 0xFF)).toInt()
|
|
423
|
+
val alpha = if (text.length > 6) 1f - (((hex shr 24) and 0xFF) / 255f) else 1f
|
|
424
|
+
return AssColour(rgb, alpha)
|
|
425
|
+
}
|
|
426
|
+
|
|
427
|
+
private data class RawEvent(
|
|
428
|
+
val startUs: Long,
|
|
429
|
+
val endUs: Long,
|
|
430
|
+
val styleName: String,
|
|
431
|
+
val marginL: Float,
|
|
432
|
+
val marginR: Float,
|
|
433
|
+
val marginV: Float,
|
|
434
|
+
val raw: String,
|
|
435
|
+
)
|
|
436
|
+
}
|
|
437
|
+
|
|
438
|
+
internal data class AssStyle(
|
|
439
|
+
val alignment: Int,
|
|
440
|
+
val fontSize: Float,
|
|
441
|
+
val colour: Int,
|
|
442
|
+
val alpha: Float,
|
|
443
|
+
val outlineColour: Int,
|
|
444
|
+
val outlineAlpha: Float,
|
|
445
|
+
val outlineWidth: Float,
|
|
446
|
+
val bold: Boolean,
|
|
447
|
+
val italic: Boolean,
|
|
448
|
+
val underline: Boolean,
|
|
449
|
+
val scaleX: Float,
|
|
450
|
+
val scaleY: Float,
|
|
451
|
+
val angle: Float,
|
|
452
|
+
val marginL: Float,
|
|
453
|
+
val marginR: Float,
|
|
454
|
+
val marginV: Float,
|
|
455
|
+
val invisible: Boolean,
|
|
456
|
+
) {
|
|
457
|
+
companion object {
|
|
458
|
+
val DEFAULT = AssStyle(
|
|
459
|
+
alignment = 2, fontSize = 48f, colour = 0xFFFFFF, alpha = 1f,
|
|
460
|
+
outlineColour = 0x000000, outlineAlpha = 1f, outlineWidth = 0f,
|
|
461
|
+
bold = false, italic = false, underline = false,
|
|
462
|
+
scaleX = 1f, scaleY = 1f, angle = 0f,
|
|
463
|
+
marginL = 0f, marginR = 0f, marginV = 0f, invisible = false,
|
|
464
|
+
)
|
|
465
|
+
}
|
|
466
|
+
}
|
|
467
|
+
|
|
468
|
+
/** One typeset event, in script coordinates. [AssSignOverlay] maps it onto the video rect. */
|
|
469
|
+
internal data class AssSign(
|
|
470
|
+
val startUs: Long,
|
|
471
|
+
val endUs: Long,
|
|
472
|
+
val text: String,
|
|
473
|
+
val x: Float,
|
|
474
|
+
val y: Float,
|
|
475
|
+
val alignment: Int,
|
|
476
|
+
val rotationZ: Float,
|
|
477
|
+
val rotationX: Float,
|
|
478
|
+
val rotationY: Float,
|
|
479
|
+
val pivotX: Float,
|
|
480
|
+
val pivotY: Float,
|
|
481
|
+
val scaleX: Float,
|
|
482
|
+
val scaleY: Float,
|
|
483
|
+
val fontSize: Float,
|
|
484
|
+
val colour: Int,
|
|
485
|
+
val alpha: Float,
|
|
486
|
+
val outlineColour: Int,
|
|
487
|
+
val outlineWidth: Float,
|
|
488
|
+
val bold: Boolean,
|
|
489
|
+
val italic: Boolean,
|
|
490
|
+
val underline: Boolean,
|
|
491
|
+
/** The coordinate space x, y, pivot and fontSize are expressed in. */
|
|
492
|
+
val playResX: Float,
|
|
493
|
+
val playResY: Float,
|
|
494
|
+
)
|
|
495
|
+
|
|
496
|
+
internal data class AssScript(
|
|
497
|
+
val playResX: Float,
|
|
498
|
+
val playResY: Float,
|
|
499
|
+
/** Typeset events, drawn by [AssSignOverlay]. */
|
|
500
|
+
val signs: List<AssSign>,
|
|
501
|
+
/** The script with every sign, drawing and invisible event cut out, for Media3's own parser. */
|
|
502
|
+
val dialogueOnly: String,
|
|
503
|
+
)
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
package expo.modules.video.utils
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* Carries one parsed ASS sign through Media3's cue stream to [AssSignOverlay].
|
|
5
|
+
*
|
|
6
|
+
* The cue itself is a single zero-width character, so `SubtitleView` lays it out and draws nothing
|
|
7
|
+
* while the overlay reads the sign off this span and draws it properly. Riding the cue stream is
|
|
8
|
+
* what buys the timing for free: Media3 already decides which cues are on screen at each moment,
|
|
9
|
+
* and a sign is on screen exactly as long as its cue is.
|
|
10
|
+
*
|
|
11
|
+
* A marker span rather than a field on `Cue` because `Cue` is final and carries no room for one —
|
|
12
|
+
* it has `shearDegrees` and nothing else transform-shaped, and `SubtitlePainter` ignores even that.
|
|
13
|
+
*/
|
|
14
|
+
internal class AssSignSpan(val sign: AssSign)
|
|
15
|
+
|
|
16
|
+
/**
|
|
17
|
+
* The cue text a sign rides on.
|
|
18
|
+
*
|
|
19
|
+
* A zero-width space still occupies a line of height in `SubtitleView`, which is harmless for a
|
|
20
|
+
* positioned cue and keeps the span anchored to something. Distinct from [HELD_SLOT_PLACEHOLDER]
|
|
21
|
+
* only in intent; both are invisible.
|
|
22
|
+
*/
|
|
23
|
+
internal const val SIGN_CUE_PLACEHOLDER = ""
|
|
@@ -191,6 +191,9 @@ class RoundedSubtitleBackground private constructor(private val backgroundView:
|
|
|
191
191
|
*/
|
|
192
192
|
private fun toBackgroundCue(cue: Cue): Cue? {
|
|
193
193
|
val text = cue.text ?: return null // Bitmap cues carry no text to put a box behind.
|
|
194
|
+
// A sign's cue is a zero-width placeholder standing in for text drawn by `AssSignOverlay`.
|
|
195
|
+
// There is no glyph here to sit behind, and the box would be the only thing on screen.
|
|
196
|
+
if (text is Spanned && text.getSpans(0, text.length, AssSignSpan::class.java).isNotEmpty()) return null
|
|
194
197
|
val stripped = if (keepMetricSpans && text is Spanned) {
|
|
195
198
|
SpannableString(text).also { spannable ->
|
|
196
199
|
for (span in spannable.getSpans(0, spannable.length, Any::class.java)) {
|
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
package expo.modules.video.utils
|
|
2
2
|
|
|
3
|
+
import android.text.SpannableString
|
|
3
4
|
import android.text.SpannableStringBuilder
|
|
5
|
+
import android.text.Spanned
|
|
4
6
|
import androidx.annotation.OptIn
|
|
5
7
|
import androidx.media3.common.C
|
|
6
8
|
import androidx.media3.common.Format
|
|
@@ -73,6 +75,26 @@ open class StackingSubtitleParser(
|
|
|
73
75
|
private val prepare: (Cue) -> Cue? = { it },
|
|
74
76
|
) : SubtitleParser {
|
|
75
77
|
|
|
78
|
+
/**
|
|
79
|
+
* A chance for a subclass to take some events out of the delegate's hands and supply its own cues
|
|
80
|
+
* for them. Returns the bytes the delegate should parse plus the cues the subclass is drawing.
|
|
81
|
+
*
|
|
82
|
+
* Default is to change nothing.
|
|
83
|
+
*/
|
|
84
|
+
protected open fun split(data: ByteArray, offset: Int, length: Int): Split =
|
|
85
|
+
Split(data, offset, length, emptyList())
|
|
86
|
+
|
|
87
|
+
/** [split]'s result: what the delegate parses, and what the subclass contributes. */
|
|
88
|
+
protected class Split(
|
|
89
|
+
val data: ByteArray,
|
|
90
|
+
val offset: Int,
|
|
91
|
+
val length: Int,
|
|
92
|
+
val ownCues: List<PreparedCue>,
|
|
93
|
+
)
|
|
94
|
+
|
|
95
|
+
/** One cue a subclass supplies, with the timing the delegate would otherwise have derived. */
|
|
96
|
+
protected class PreparedCue(val cue: Cue, val startUs: Long, val endUs: Long)
|
|
97
|
+
|
|
76
98
|
// Every emitted segment is a complete description of what should be on screen for its interval,
|
|
77
99
|
// so segments replace each other rather than being merged by the renderer.
|
|
78
100
|
override fun getCueReplacementBehavior(): Int = Format.CUE_REPLACEMENT_BEHAVIOR_REPLACE
|
|
@@ -89,9 +111,13 @@ open class StackingSubtitleParser(
|
|
|
89
111
|
output: Consumer<CuesWithTiming>
|
|
90
112
|
) {
|
|
91
113
|
val entries = mutableListOf<Entry>()
|
|
114
|
+
val split = split(data, offset, length)
|
|
115
|
+
for (prepared in split.ownCues) {
|
|
116
|
+
entries.add(Entry(prepared.cue, prepared.startUs, prepared.endUs, entries.size))
|
|
117
|
+
}
|
|
92
118
|
// Always ask the delegate for everything; `outputOptions` is applied to the merged result below,
|
|
93
119
|
// because a segment can span cues that fall on both sides of the requested start time.
|
|
94
|
-
delegate.parse(data, offset, length, SubtitleParser.OutputOptions.allCues()) { cuesWithTiming ->
|
|
120
|
+
delegate.parse(split.data, split.offset, split.length, SubtitleParser.OutputOptions.allCues()) { cuesWithTiming ->
|
|
95
121
|
val startUs = cuesWithTiming.startTimeUs
|
|
96
122
|
val endUs = resolveEndUs(cuesWithTiming)
|
|
97
123
|
for (cue in cuesWithTiming.cues) {
|
|
@@ -291,7 +317,16 @@ open class StackingSubtitleParser(
|
|
|
291
317
|
class StackingSubripParser : StackingSubtitleParser(SubripParser())
|
|
292
318
|
|
|
293
319
|
/**
|
|
294
|
-
* ASS and SSA
|
|
320
|
+
* ASS and SSA.
|
|
321
|
+
*
|
|
322
|
+
* Splits the script in two before anything else runs. [AssSignParser] takes the typeset events —
|
|
323
|
+
* anything the author placed — and the delegate gets a copy with those lines removed, so the two
|
|
324
|
+
* renderers can never draw the same event. Signs come back as cues carrying an [AssSignSpan], which
|
|
325
|
+
* rides Media3's own timing all the way to [AssSignOverlay]; the cue text itself is a zero-width
|
|
326
|
+
* character, so `SubtitleView` lays it out and draws nothing.
|
|
327
|
+
*
|
|
328
|
+
* Falls back to plain delegation when the script yields no signs, which is every `.ass` that is
|
|
329
|
+
* only dialogue, and when the bytes are not decodable as text.
|
|
295
330
|
*
|
|
296
331
|
* The delegate is built by the caller rather than constructed here because an SSA track muxed into
|
|
297
332
|
* a container carries its `[V4+ Styles]` header in the format's initialization data, and only
|
|
@@ -299,7 +334,35 @@ class StackingSubripParser : StackingSubtitleParser(SubripParser())
|
|
|
299
334
|
*/
|
|
300
335
|
@OptIn(UnstableApi::class)
|
|
301
336
|
class StackingSsaParser(delegate: SubtitleParser) :
|
|
302
|
-
StackingSubtitleParser(delegate, SsaCueNormalizer::normalize)
|
|
337
|
+
StackingSubtitleParser(delegate, SsaCueNormalizer::normalize) {
|
|
338
|
+
|
|
339
|
+
override fun split(data: ByteArray, offset: Int, length: Int): Split {
|
|
340
|
+
val content = runCatching { String(data, offset, length, Charsets.UTF_8) }.getOrNull()
|
|
341
|
+
?: return Split(data, offset, length, emptyList())
|
|
342
|
+
|
|
343
|
+
val script = AssSignParser.parse(content)
|
|
344
|
+
if (script.signs.isEmpty()) return Split(data, offset, length, emptyList())
|
|
345
|
+
|
|
346
|
+
val dialogue = script.dialogueOnly.toByteArray(Charsets.UTF_8)
|
|
347
|
+
return Split(dialogue, 0, dialogue.size, script.signs.map(::toEntry))
|
|
348
|
+
}
|
|
349
|
+
|
|
350
|
+
private fun toEntry(sign: AssSign): PreparedCue {
|
|
351
|
+
val text = SpannableString(SIGN_CUE_PLACEHOLDER)
|
|
352
|
+
text.setSpan(AssSignSpan(sign), 0, text.length, Spanned.SPAN_EXCLUSIVE_EXCLUSIVE)
|
|
353
|
+
|
|
354
|
+
// Positioned so the stacking pass leaves it alone and the span filter keeps it intact. The
|
|
355
|
+
// values are only a rough placement for the invisible placeholder; the overlay reads the real
|
|
356
|
+
// coordinates off the span.
|
|
357
|
+
val cue = Cue.Builder()
|
|
358
|
+
.setText(text)
|
|
359
|
+
.setPosition((sign.x / sign.playResX).coerceIn(0f, 1f))
|
|
360
|
+
.setLine((sign.y / sign.playResY).coerceIn(0f, 1f), Cue.LINE_TYPE_FRACTION)
|
|
361
|
+
.build()
|
|
362
|
+
|
|
363
|
+
return PreparedCue(cue, sign.startUs, sign.endUs)
|
|
364
|
+
}
|
|
365
|
+
}
|
|
303
366
|
|
|
304
367
|
/**
|
|
305
368
|
* Routes SubRip and ASS/SSA through the stacking parsers so simultaneous cues stack, delegates every
|
|
@@ -98,6 +98,12 @@ object SubtitleUtils {
|
|
|
98
98
|
if (!roundedBackgroundApplied) {
|
|
99
99
|
RoundedSubtitleBackground.remove(playerView)
|
|
100
100
|
}
|
|
101
|
+
|
|
102
|
+
// Independent of every style decision above: an ASS sign is drawn where and how the script
|
|
103
|
+
// says, not how the viewer's caption settings say, so nothing here applies to it. Installed
|
|
104
|
+
// unconditionally because the track can be switched to an `.ass` at any point, and rebinding
|
|
105
|
+
// on each reconfigure is what carries it across a player swap.
|
|
106
|
+
AssSignOverlay.install(playerView)?.attachTo(playerView.player)
|
|
101
107
|
}
|
|
102
108
|
}
|
|
103
109
|
|
|
@@ -108,6 +114,7 @@ object SubtitleUtils {
|
|
|
108
114
|
*/
|
|
109
115
|
fun releaseSubtitleView(playerView: PlayerView) {
|
|
110
116
|
RoundedSubtitleBackground.remove(playerView)
|
|
117
|
+
AssSignOverlay.remove(playerView)
|
|
111
118
|
}
|
|
112
119
|
|
|
113
120
|
/**
|
package/package.json
CHANGED