expo-video-subtitle 0.7.6 → 0.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +80 -0
- package/FORK.md +1 -0
- package/android/src/main/java/expo/modules/video/utils/StackingSubtitleParser.kt +14 -4
- package/android/src/main/java/expo/modules/video/utils/WebvttCueNormalizer.kt +64 -0
- package/ios/AssSign.swift +264 -0
- package/ios/AssSignOverlayView.swift +302 -0
- package/ios/SubtitleConverterText.swift +128 -9
- package/ios/VideoPlayerItem.swift +14 -2
- package/ios/VideoPlayerSubtitleSideload.swift +30 -5
- package/ios/VideoView.swift +9 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,86 @@ Changes inherited from the upstream base version are not repeated here — see t
|
|
|
7
7
|
[upstream changelog](https://github.com/expo/expo/blob/main/packages/expo-video/CHANGELOG.md)
|
|
8
8
|
for the history of everything that isn't subtitle related.
|
|
9
9
|
|
|
10
|
+
## 0.8.1
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
|
|
14
|
+
- **A plain `.vtt` track sat flush against the bottom edge and ignored `bottomOffset` (Android).**
|
|
15
|
+
`.srt` and `.ass` tracks in the same player were fine.
|
|
16
|
+
|
|
17
|
+
Media3's `WebvttCueParser` leaves a cue with no `line:` setting unpositioned, but that is not what
|
|
18
|
+
reaches the screen. `WebvttParser.parse` hands its cues through `WebvttSubtitle.getCues`, which
|
|
19
|
+
runs the WebVTT spec's snap-to-lines stacking on every cue, unconditionally — a lone cue comes out
|
|
20
|
+
as `line = -1, LINE_TYPE_NUMBER`. `SubtitlePainter` then takes its number-line branch, where `-1`
|
|
21
|
+
is `parentBottom - textHeight` and `bottomPaddingFraction` is never read; the painter only consults
|
|
22
|
+
it when `line` is unset. So a WebVTT cue landed `parentHeight × padding` lower than the identical
|
|
23
|
+
SubRip cue — 54px on a 1080-tall frame at the app's 5% — and no offset could move it.
|
|
24
|
+
|
|
25
|
+
SubRip never had this because `SubripParser` leaves `line` alone. ASS had it and was already fixed:
|
|
26
|
+
`SsaCueNormalizer` strips the placement `SsaParser` stamps on. WebVTT was the one format with the
|
|
27
|
+
stamp and no normalizer. It now goes through `StackingSubtitleParser` behind `WebvttCueNormalizer`,
|
|
28
|
+
which clears a negative line number and the derived centre `position`, so a plain cue takes the
|
|
29
|
+
app's padding and simultaneous cues stack — exactly as `.srt` does.
|
|
30
|
+
|
|
31
|
+
Verified against the 1.9.0 artefacts the app ships, from source and from decompiled bytecode; the
|
|
32
|
+
logic is byte for byte identical to 1.8.0, so this was never a version question.
|
|
33
|
+
|
|
34
|
+
One honest limit: after `getCues` has run, a parser-stacked `line = -1` is indistinguishable from
|
|
35
|
+
an author's explicit `line:-1`, so both are reset. `line:-1` means "the last line", and "the last
|
|
36
|
+
line, lifted by the app's padding" is where every other format's bottom cue already goes;
|
|
37
|
+
percentages and non-negative line numbers are untouched.
|
|
38
|
+
|
|
39
|
+
### Why it surfaced now
|
|
40
|
+
|
|
41
|
+
Nothing changed in this path for a long time. Until `SUBTITLE_PRESERVE_FORMAT` was switched on, every
|
|
42
|
+
anikoto sidecar was converted to SubRip before it reached the app, so the WebVTT path was never
|
|
43
|
+
exercised. Storing the `.vtt` as-is is what put the first real WebVTT track in front of it.
|
|
44
|
+
|
|
45
|
+
## 0.8.0
|
|
46
|
+
|
|
47
|
+
### Added
|
|
48
|
+
|
|
49
|
+
- **ASS signs are drawn with their transforms on iOS too.** AVFoundation renders a sideloaded
|
|
50
|
+
WebVTT track itself, and WebVTT can express no rotation, no scale and no per-cue opacity;
|
|
51
|
+
`AVTextStyleRule` has no attribute for any of them either. So the same split the Android side
|
|
52
|
+
makes: `SubtitleConverter.splitAss` takes the typeset events out of the script before the WebVTT
|
|
53
|
+
is built, and `AssSignOverlayView` draws them in `AVPlayerViewController.contentOverlayView`,
|
|
54
|
+
sized to `videoBounds` and driven by a `CADisplayLink`.
|
|
55
|
+
|
|
56
|
+
Signs travel on the `AVPlayerItem` that owns them rather than through a store of their own. A
|
|
57
|
+
store would have to be keyed by something both ends can see, and the only identity a sideloaded
|
|
58
|
+
track keeps inside a composition is its language — so two players loading different videos in the
|
|
59
|
+
same language would overwrite each other, and the next episode would inherit the previous one's
|
|
60
|
+
typesetting. The item is the identity, and the overlay re-resolves whenever the item or the
|
|
61
|
+
selected language changes.
|
|
62
|
+
|
|
63
|
+
Read and honoured: `\pos`, `\move` (at its destination), `\org`, `\an` and the legacy `\a`,
|
|
64
|
+
`\frz`, `\fscx`, `\fscy`, `\fs`, `\c` / `\1c`, `\alpha` / `\1a`, `\b`, `\i`, `\u`, and the
|
|
65
|
+
style columns behind them.
|
|
66
|
+
|
|
67
|
+
### Known limitations on iOS
|
|
68
|
+
|
|
69
|
+
- `\frx` and `\fry` are not applied. `CGAffineTransform` is two-dimensional, and the values are
|
|
70
|
+
almost always under 3° in practice; `\frz` is what carries the effect.
|
|
71
|
+
- Signs do not appear in Picture in Picture. The system renders that window from the video layer
|
|
72
|
+
alone, not from the view hierarchy `contentOverlayView` belongs to.
|
|
73
|
+
- `\t` animation, `\fad`, `\clip`, `\blur`, `\be`, `\fax` shear, vector drawings and karaoke are
|
|
74
|
+
dropped, as on Android. Fonts fall back to the system face.
|
|
75
|
+
|
|
76
|
+
### Fixed
|
|
77
|
+
|
|
78
|
+
- **Non-finite numbers are refused.** `Double("nan")` and `Double("inf")` parse, and a subtitle is
|
|
79
|
+
untrusted scraper output; either one reaching a transform makes a sign silently fail to draw,
|
|
80
|
+
which looks exactly like a sign that was never parsed. Every number out of a tag or a style column
|
|
81
|
+
now goes through a finiteness check.
|
|
82
|
+
|
|
83
|
+
### Changed
|
|
84
|
+
|
|
85
|
+
- The override-block scanner reads a leading digit as part of a tag name, so `\1c`, `\1a` and `\4a`
|
|
86
|
+
are recognised. The WebVTT path ignores them as before — 91 existing checks confirm its output is
|
|
87
|
+
unchanged — but the sign path needs them.
|
|
88
|
+
- The Swift harness is at 119 checks, up from 91.
|
|
89
|
+
|
|
10
90
|
## 0.7.6
|
|
11
91
|
|
|
12
92
|
### Fixed
|
package/FORK.md
CHANGED
|
@@ -29,6 +29,7 @@ Measured against the published `expo-video@56.1.4` tarball. `+`/`-` are added/re
|
|
|
29
29
|
| `android/…/records/SubtitleStyle.kt` | 152 | Record → `CaptionStyleCompat`; hex colour parsing; font-family resolution (`res/font` → `ReactFontManager`); the `applyEmbeddedStyles` flag, which `SubtitleUtils` reads directly rather than `toCaptionStyle` and which defaults to `true` since 0.5.0. |
|
|
30
30
|
| `android/…/utils/StackingSubtitleParser.kt` | 342 | `SubtitleParser` that segments the cue timeline and merges simultaneous **unpositioned** cues into one stacked block, plus the `SubtitleParser.Factory` that routes SubRip and ASS/SSA to it and wraps every parser in `FilteringSubtitleParser`. ASS gets there through `SsaCueNormalizer` as the `prepare` step — without it `SsaParser` sets both `line` and `position` on every cue and they would all take the passthrough branch. Assigns each cue a vertical slot it keeps for its whole time on screen — a finished cue's line is held blank (`\u200B`) so the cues above it do not drop. Merges through a `SpannableStringBuilder` so cue spans survive. |
|
|
31
31
|
| `android/…/utils/SsaCueNormalizer.kt` | 93 | Clears the `line`/`position` Media3's `SsaParser` fills in on cues the ASS author never placed, so `bottomOffset` and cue stacking work on `.ass` the way they already do on `.srt`. The test is exact rather than heuristic: those values are a pure function of the cue's anchors, so matching both defaults for the bottom-centre anchors *is* "no positioning tag". Also drops events on a style the author made invisible (`Fontsize 0`, or a primary colour whose inverted alpha byte is `FF`) — that is where typesetters park working notes, and nothing downstream can carry "invisible". |
|
|
32
|
+
| `android/…/utils/WebvttCueNormalizer.kt` | 61 | Clears the `line = -1 - i` that Media3's `WebvttSubtitle.getCues` stamps on every unpositioned WebVTT cue, plus the derived centre `position`, so `.vtt` takes the app's bottom padding and stacks like `.srt`. **Not exact the way the SSA one is**: once `getCues` has run, a parser-stacked `-1` and an author's `line:-1` are the same cue, so both are reset — a defensible reading, since `line:-1` means "the last line". Percentages and non-negative numbers are untouched. |
|
|
32
33
|
| `android/…/utils/AssSignParser.kt` | 505 | Splits an ASS script into the typeset events this package draws and a copy of the script with those lines removed for Media3's parser. One pass, no matching step, so an event is in exactly one of the two outputs and cannot be drawn twice. Pure Kotlin with no Android imports, which is what makes it unit-testable on the JVM. Reads `\pos`, `\move`, `\org`, `\an`, `\frz`, `\frx`, `\fry`, `\fscx/y`, `\fs`, `\c`, `\alpha`, `\b`, `\i`, `\u` and the style columns behind them. The tag scanner takes the longest name first — `\c` must not swallow `\clip`, `\b` must not swallow `\bord`, `\blur` or `\be` — and reads a leading digit as part of the name for `\1c` and `\4a`. |
|
|
33
34
|
| `android/…/utils/AssSignOverlay.kt` | 330 | Draws those signs on a canvas in front of the captions, with `Matrix` for rotation and scale and `Camera` for the two 3-D axes. Drives itself from the player's clock on a `Choreographer` callback, because signs do not ride the cue stream and nothing else would tell it when one starts; 40 ms events need frame granularity anyway. **Media3 cannot do this**: `SubtitlePainter` has no rotate, skew, Matrix or Camera call, and `Cue` has no rotation field. Sits in the same parent as `SubtitleView` — the content frame, sized to the video, which is why `\pos` survives letterboxing. Lays text out on change rather than per frame, because frame-by-frame typesetting changes the on-screen set every 40 ms. |
|
|
34
35
|
| `android/…/utils/AssSignStore.kt` | 61 | Hands parsed signs from the parser to the overlay, keyed by the track's `Format.id`. **Not** the cue stream: `CueEncoder` marshals every cue through `Parcel`, and a custom span that is neither a framework `ParcelableSpan` nor one of Media3's three `CustomSpanBundler` types is dropped without a word — which is how 0.7.0 shipped with signs that parsed correctly and never appeared. Staying out of the cue stream also keeps 12,616 sign events from putting ~26,000 boundaries through the O(n²) timeline pass. |
|
|
@@ -364,8 +364,8 @@ class StackingSsaParser(
|
|
|
364
364
|
}
|
|
365
365
|
|
|
366
366
|
/**
|
|
367
|
-
* Routes SubRip
|
|
368
|
-
* other subtitle format to Media3's default handling, and wraps the result in
|
|
367
|
+
* Routes SubRip, ASS/SSA and WebVTT through the stacking parsers so simultaneous cues stack and the
|
|
368
|
+
* app's bottom padding applies, delegates every other subtitle format to Media3's default handling, and wraps the result in
|
|
369
369
|
* [FilteringSubtitleParser] so the appearance a track asks for never overrides the app's own
|
|
370
370
|
* subtitle style — except for cues the track positions itself, which keep theirs.
|
|
371
371
|
*/
|
|
@@ -377,8 +377,8 @@ class ExpoVideoSubtitleParserFactory : SubtitleParser.Factory {
|
|
|
377
377
|
format.isSubrip() || defaultFactory.supportsFormat(format)
|
|
378
378
|
|
|
379
379
|
override fun getCueReplacementBehavior(format: Format): Int =
|
|
380
|
-
if (format.isSubrip() || format.isSsa()) {
|
|
381
|
-
//
|
|
380
|
+
if (format.isSubrip() || format.isSsa() || format.isWebvtt()) {
|
|
381
|
+
// Every stacking parser emits whole-screen segments, whatever the delegate's own behaviour is.
|
|
382
382
|
Format.CUE_REPLACEMENT_BEHAVIOR_REPLACE
|
|
383
383
|
} else {
|
|
384
384
|
defaultFactory.getCueReplacementBehavior(format)
|
|
@@ -393,6 +393,13 @@ class ExpoVideoSubtitleParserFactory : SubtitleParser.Factory {
|
|
|
393
393
|
StackingSsaParser(defaultFactory.create(format), format.id),
|
|
394
394
|
keepAppearanceOfPositionedCues = true,
|
|
395
395
|
)
|
|
396
|
+
// WebVTT gets the stacking parser too, with the placement Media3 stamps on every unpositioned
|
|
397
|
+
// cue taken back off first. Without that, a plain `.vtt` sat flush against the bottom edge and
|
|
398
|
+
// ignored `bottomOffset` entirely — see WebvttCueNormalizer. Not the sign carve-out: a
|
|
399
|
+
// positioned WebVTT cue is a cue setting on ordinary dialogue, not typesetting.
|
|
400
|
+
format.isWebvtt() -> FilteringSubtitleParser(
|
|
401
|
+
StackingSubtitleParser(defaultFactory.create(format), WebvttCueNormalizer::normalize)
|
|
402
|
+
)
|
|
396
403
|
else -> FilteringSubtitleParser(defaultFactory.create(format))
|
|
397
404
|
}
|
|
398
405
|
|
|
@@ -401,4 +408,7 @@ class ExpoVideoSubtitleParserFactory : SubtitleParser.Factory {
|
|
|
401
408
|
|
|
402
409
|
private fun Format.isSsa(): Boolean =
|
|
403
410
|
MimeTypes.TEXT_SSA.equals(sampleMimeType, ignoreCase = true)
|
|
411
|
+
|
|
412
|
+
private fun Format.isWebvtt(): Boolean =
|
|
413
|
+
MimeTypes.TEXT_VTT.equals(sampleMimeType, ignoreCase = true)
|
|
404
414
|
}
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
package expo.modules.video.utils
|
|
2
|
+
|
|
3
|
+
import androidx.annotation.OptIn
|
|
4
|
+
import androidx.media3.common.text.Cue
|
|
5
|
+
import androidx.media3.common.util.UnstableApi
|
|
6
|
+
|
|
7
|
+
/**
|
|
8
|
+
* Undoes the placement Media3's WebVTT path stamps on every cue that had none, so a `.vtt` track
|
|
9
|
+
* behaves like a `.srt` one: it takes the app's bottom padding, and simultaneous cues stack.
|
|
10
|
+
*
|
|
11
|
+
* ### What Media3 does
|
|
12
|
+
*
|
|
13
|
+
* `WebvttCueParser` leaves a cue with no `line:` setting at `line = DIMEN_UNSET`. That is not what
|
|
14
|
+
* reaches the screen. `WebvttParser.parse` wraps its cues in a `WebvttSubtitle` and hands them
|
|
15
|
+
* through `LegacySubtitleUtil`, and `WebvttSubtitle.getCues` runs the spec's snap-to-lines stacking
|
|
16
|
+
* on every query, unconditionally:
|
|
17
|
+
*
|
|
18
|
+
* ```java
|
|
19
|
+
* // Steps 4 - 10 of https://www.w3.org/TR/webvtt1/#cue-computed-line
|
|
20
|
+
* currentCues.add(cue.buildUpon().setLine((float) (-1 - i), Cue.LINE_TYPE_NUMBER).build());
|
|
21
|
+
* ```
|
|
22
|
+
*
|
|
23
|
+
* A lone cue gets `line = -1`. `SubtitlePainter` then takes its `LINE_TYPE_NUMBER` branch, where
|
|
24
|
+
* `line = -1` is `parentBottom - textHeight` — flush against the bottom edge — and
|
|
25
|
+
* `bottomPaddingFraction` is never read; the painter only consults it when `line` is unset. So a
|
|
26
|
+
* plain WebVTT track sits `parentHeight × padding` lower than the identical SubRip track, and
|
|
27
|
+
* `SubtitleSource.bottomOffset` has no effect on it at all. Verified against the 1.9.0 artefacts the
|
|
28
|
+
* app ships, byte for byte identical to 1.8.0.
|
|
29
|
+
*
|
|
30
|
+
* `position` gets the same treatment: with no setting, `derivePosition(CENTER)` yields `0.5` with
|
|
31
|
+
* `ANCHOR_TYPE_MIDDLE`. That is harmless for the painter but it makes [StackingSubtitleParser] read
|
|
32
|
+
* the cue as author-placed and pass it through unstacked.
|
|
33
|
+
*
|
|
34
|
+
* ### Why this cannot be exact, and why that is acceptable
|
|
35
|
+
*
|
|
36
|
+
* [SsaCueNormalizer] can recognise "the parser filled this in" precisely, because Media3's SSA
|
|
37
|
+
* defaults are a pure function of the anchors. WebVTT's are not: once `getCues` has run, a cue the
|
|
38
|
+
* parser stacked to `line = -1` is identical to one whose author wrote `line:-1`. Both are reset
|
|
39
|
+
* here. That is a defensible reading rather than a loss — `line:-1` means "the last line", and
|
|
40
|
+
* "the last line, lifted by the app's padding" is where every other format's bottom cue already
|
|
41
|
+
* goes. An author who wants an exact placement writes a percentage, and percentages are untouched.
|
|
42
|
+
*
|
|
43
|
+
* Only negative line *numbers* qualify. A fraction (`line:80%`), a non-negative number (`line:3`,
|
|
44
|
+
* counted from the top) or an explicit `position` off centre is a placement someone chose.
|
|
45
|
+
*/
|
|
46
|
+
@OptIn(UnstableApi::class)
|
|
47
|
+
internal object WebvttCueNormalizer {
|
|
48
|
+
private const val DEFAULT_POSITION = 0.5f
|
|
49
|
+
private const val EPSILON = 0.0001f
|
|
50
|
+
|
|
51
|
+
fun normalize(cue: Cue): Cue {
|
|
52
|
+
val stackedLine = cue.lineType == Cue.LINE_TYPE_NUMBER && cue.line != Cue.DIMEN_UNSET && cue.line < 0f
|
|
53
|
+
val derivedPosition = cue.positionAnchor == Cue.ANCHOR_TYPE_MIDDLE &&
|
|
54
|
+
cue.position != Cue.DIMEN_UNSET &&
|
|
55
|
+
kotlin.math.abs(cue.position - DEFAULT_POSITION) < EPSILON
|
|
56
|
+
|
|
57
|
+
if (!stackedLine && !derivedPosition) return cue
|
|
58
|
+
|
|
59
|
+
val builder = cue.buildUpon()
|
|
60
|
+
if (stackedLine) builder.setLine(Cue.DIMEN_UNSET, Cue.LINE_TYPE_FRACTION)
|
|
61
|
+
if (derivedPosition) builder.setPosition(Cue.DIMEN_UNSET)
|
|
62
|
+
return builder.build()
|
|
63
|
+
}
|
|
64
|
+
}
|
|
@@ -0,0 +1,264 @@
|
|
|
1
|
+
import Foundation
|
|
2
|
+
|
|
3
|
+
/// One typeset event, in script coordinates. `AssSignOverlayView` maps it onto the video rect.
|
|
4
|
+
///
|
|
5
|
+
/// The Android side has the same shape, field for field, and for the same reason: this is what a
|
|
6
|
+
/// renderer needs to put a sign on a placard, and AVFoundation can carry almost none of it.
|
|
7
|
+
struct AssSign: Equatable {
|
|
8
|
+
let startMs: Int
|
|
9
|
+
let endMs: Int
|
|
10
|
+
let text: String
|
|
11
|
+
|
|
12
|
+
let x: Double
|
|
13
|
+
let y: Double
|
|
14
|
+
let alignment: Int
|
|
15
|
+
|
|
16
|
+
/// Clockwise, having already been flipped from ASS's counter-clockwise measure.
|
|
17
|
+
let rotationZ: Double
|
|
18
|
+
let rotationX: Double
|
|
19
|
+
let rotationY: Double
|
|
20
|
+
let pivotX: Double
|
|
21
|
+
let pivotY: Double
|
|
22
|
+
|
|
23
|
+
let scaleX: Double
|
|
24
|
+
let scaleY: Double
|
|
25
|
+
let fontSize: Double
|
|
26
|
+
let colour: Int
|
|
27
|
+
let alpha: Double
|
|
28
|
+
let outlineColour: Int
|
|
29
|
+
let outlineWidth: Double
|
|
30
|
+
let bold: Bool
|
|
31
|
+
let italic: Bool
|
|
32
|
+
let underline: Bool
|
|
33
|
+
|
|
34
|
+
/// The coordinate space the values above are expressed in.
|
|
35
|
+
let playResX: Double
|
|
36
|
+
let playResY: Double
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
extension AssSign {
|
|
40
|
+
/// `AVMediaSelectionOption` reports a full locale identifier (`id_ID`, `pt-BR`) where the track was
|
|
41
|
+
/// given a bare language tag. Comparing the language subtag is what makes the two ends meet.
|
|
42
|
+
static func normalizeLanguage(_ language: String) -> String {
|
|
43
|
+
let lowered = language.lowercased()
|
|
44
|
+
if let cut = lowered.firstIndex(where: { $0 == "-" || $0 == "_" }) {
|
|
45
|
+
return String(lowered[lowered.startIndex..<cut])
|
|
46
|
+
}
|
|
47
|
+
return lowered
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
/// A script split into the part this package draws and the part AVFoundation draws.
|
|
52
|
+
struct AssSplit {
|
|
53
|
+
/// Typeset events, drawn by `AssSignOverlayView`.
|
|
54
|
+
let signs: [AssSign]
|
|
55
|
+
/// The script with every sign, drawing and invisible event cut out, for the WebVTT converter.
|
|
56
|
+
let dialogueOnly: String
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
extension SubtitleConverter {
|
|
60
|
+
|
|
61
|
+
/// Takes the typesetting out of a script and hands back what is left.
|
|
62
|
+
///
|
|
63
|
+
/// ### Why signs cannot go through WebVTT
|
|
64
|
+
///
|
|
65
|
+
/// AVFoundation renders a `.text` track itself, and WebVTT has no rotation, no scale and no
|
|
66
|
+
/// per-cue opacity to express. `AVTextStyleRule` has no attribute for any of it either. A sign
|
|
67
|
+
/// tilted to lie on a placard comes out flat, which reads as pasted over the picture rather than
|
|
68
|
+
/// part of it — in one episode of Jujutsu Kaisen S3, 3,723 events carry `\frz`, 1,304 of them past
|
|
69
|
+
/// 5°.
|
|
70
|
+
///
|
|
71
|
+
/// So the same split the Android side makes: typeset events to us, dialogue to the platform, and
|
|
72
|
+
/// the sign lines removed from the copy the platform parses. There is no matching step to get
|
|
73
|
+
/// wrong, because an event is in exactly one of the two outputs.
|
|
74
|
+
///
|
|
75
|
+
/// ### What counts as a sign
|
|
76
|
+
///
|
|
77
|
+
/// Anything the author placed: an event carrying `\pos` or `\move`, or whose effective alignment
|
|
78
|
+
/// is not bottom-centre. The complement is dialogue, and AVFoundation is welcome to it.
|
|
79
|
+
static func splitAss(_ ass: String) -> AssSplit {
|
|
80
|
+
let normalized = normalizeNewlines(ass)
|
|
81
|
+
let script = AssScript(parsing: normalized)
|
|
82
|
+
let lines = normalized.components(separatedBy: "\n")
|
|
83
|
+
|
|
84
|
+
var signs: [AssSign] = []
|
|
85
|
+
var removed = Set<Int>()
|
|
86
|
+
|
|
87
|
+
for event in script.events {
|
|
88
|
+
guard event.endMs > event.startMs else { continue }
|
|
89
|
+
let style = script.styles[event.styleName]
|
|
90
|
+
|
|
91
|
+
// A style that draws nothing is where a typesetter parks working notes. Neither renderer
|
|
92
|
+
// should show it, and the WebVTT converter drops it too — cut here so both agree.
|
|
93
|
+
if style?.invisible == true {
|
|
94
|
+
removed.insert(event.lineIndex)
|
|
95
|
+
continue
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
guard let sign = buildSign(event: event, style: style, script: script) else { continue }
|
|
99
|
+
|
|
100
|
+
switch sign {
|
|
101
|
+
case .drop:
|
|
102
|
+
removed.insert(event.lineIndex)
|
|
103
|
+
case .sign(let value):
|
|
104
|
+
signs.append(value)
|
|
105
|
+
removed.insert(event.lineIndex)
|
|
106
|
+
case .dialogue:
|
|
107
|
+
break
|
|
108
|
+
}
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
let dialogueOnly = removed.isEmpty
|
|
112
|
+
? normalized
|
|
113
|
+
: lines.enumerated().filter { !removed.contains($0.offset) }.map(\.element).joined(separator: "\n")
|
|
114
|
+
|
|
115
|
+
return AssSplit(signs: signs, dialogueOnly: dialogueOnly)
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
private enum SignResolution {
|
|
119
|
+
case sign(AssSign)
|
|
120
|
+
case dialogue
|
|
121
|
+
case drop
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
private static func buildSign(
|
|
125
|
+
event: AssScript.Event,
|
|
126
|
+
style: AssScript.Style?,
|
|
127
|
+
script: AssScript
|
|
128
|
+
) -> SignResolution? {
|
|
129
|
+
var alignment: Int = style?.alignment ?? 2
|
|
130
|
+
var position: (x: Double, y: Double)?
|
|
131
|
+
var origin: (x: Double, y: Double)?
|
|
132
|
+
var rotationZ: Double = style?.angle ?? 0
|
|
133
|
+
var rotationX: Double = 0
|
|
134
|
+
var rotationY: Double = 0
|
|
135
|
+
var scaleX: Double = style?.scaleX ?? 1
|
|
136
|
+
var scaleY: Double = style?.scaleY ?? 1
|
|
137
|
+
var fontSize: Double = style?.fontSize ?? 48
|
|
138
|
+
var colour: Int = style?.colour ?? 0xFFFFFF
|
|
139
|
+
var alpha: Double = style?.alpha ?? 1
|
|
140
|
+
var bold: Bool = style?.bold ?? false
|
|
141
|
+
var italic: Bool = style?.italic ?? false
|
|
142
|
+
var underline: Bool = style?.underline ?? false
|
|
143
|
+
var isDrawing = false
|
|
144
|
+
|
|
145
|
+
for block in overrideBlocks(in: event.text) {
|
|
146
|
+
for tag in parseOverrideBlock(block) {
|
|
147
|
+
switch tag {
|
|
148
|
+
case .alignment(let value): alignment = value
|
|
149
|
+
case .position(let x, let y): position = (x, y)
|
|
150
|
+
case .origin(let x, let y): origin = (x, y)
|
|
151
|
+
case .rotationZ(let value): rotationZ = value
|
|
152
|
+
case .rotationX(let value): rotationX = value
|
|
153
|
+
case .rotationY(let value): rotationY = value
|
|
154
|
+
case .scaleX(let value): scaleX = value
|
|
155
|
+
case .scaleY(let value): scaleY = value
|
|
156
|
+
case .fontSize(let value): fontSize = value
|
|
157
|
+
case .primaryColour(let value): colour = value
|
|
158
|
+
case .primaryAlpha(let value): alpha = value
|
|
159
|
+
case .bold(let on): bold = on
|
|
160
|
+
case .italic(let on): italic = on
|
|
161
|
+
case .underline(let on): underline = on
|
|
162
|
+
case .drawing: isDrawing = true
|
|
163
|
+
case .reset: break
|
|
164
|
+
}
|
|
165
|
+
}
|
|
166
|
+
}
|
|
167
|
+
|
|
168
|
+
if isDrawing { return .drop }
|
|
169
|
+
|
|
170
|
+
let text = strippedText(event.text)
|
|
171
|
+
if text.trimmingCharacters(in: CharacterSet.whitespacesAndNewlines).isEmpty { return .drop }
|
|
172
|
+
|
|
173
|
+
// The complement of what AVFoundation lays out at the bottom by itself.
|
|
174
|
+
guard position != nil || alignment != 2 else { return .dialogue }
|
|
175
|
+
|
|
176
|
+
// Style margins only. A per-event margin override would need the Event to carry one, and it
|
|
177
|
+
// only ever matters for a sign with no `\pos` at all — which in a typeset script is close to
|
|
178
|
+
// none of them. Media3 ignores per-event margins outright, so both platforms agree here.
|
|
179
|
+
let marginL: Double = style?.marginL ?? 0
|
|
180
|
+
let marginR: Double = style?.marginR ?? 0
|
|
181
|
+
let marginV: Double = style?.marginV ?? 0
|
|
182
|
+
|
|
183
|
+
let x: Double = position?.x ?? {
|
|
184
|
+
switch (alignment - 1) % 3 {
|
|
185
|
+
case 0: return marginL
|
|
186
|
+
case 2: return script.playResX - marginR
|
|
187
|
+
default: return script.playResX / 2
|
|
188
|
+
}
|
|
189
|
+
}()
|
|
190
|
+
let y: Double = position?.y ?? {
|
|
191
|
+
switch (alignment - 1) / 3 {
|
|
192
|
+
case 0: return script.playResY - marginV
|
|
193
|
+
case 2: return marginV
|
|
194
|
+
default: return script.playResY / 2
|
|
195
|
+
}
|
|
196
|
+
}()
|
|
197
|
+
|
|
198
|
+
return .sign(AssSign(
|
|
199
|
+
startMs: event.startMs,
|
|
200
|
+
endMs: event.endMs,
|
|
201
|
+
text: text,
|
|
202
|
+
x: x,
|
|
203
|
+
y: y,
|
|
204
|
+
alignment: alignment,
|
|
205
|
+
// ASS measures counter-clockwise; every renderer here turns clockwise.
|
|
206
|
+
rotationZ: -rotationZ,
|
|
207
|
+
rotationX: rotationX,
|
|
208
|
+
rotationY: -rotationY,
|
|
209
|
+
pivotX: origin?.x ?? x,
|
|
210
|
+
pivotY: origin?.y ?? y,
|
|
211
|
+
scaleX: scaleX,
|
|
212
|
+
scaleY: scaleY,
|
|
213
|
+
fontSize: fontSize,
|
|
214
|
+
colour: colour,
|
|
215
|
+
alpha: alpha,
|
|
216
|
+
outlineColour: style?.outlineColour ?? 0x000000,
|
|
217
|
+
outlineWidth: style?.outlineWidth ?? 0,
|
|
218
|
+
bold: bold,
|
|
219
|
+
italic: italic,
|
|
220
|
+
underline: underline,
|
|
221
|
+
playResX: script.playResX,
|
|
222
|
+
playResY: script.playResY
|
|
223
|
+
))
|
|
224
|
+
}
|
|
225
|
+
|
|
226
|
+
/// The contents of every `{...}` block, in order.
|
|
227
|
+
private static func overrideBlocks(in text: String) -> [String] {
|
|
228
|
+
var blocks: [String] = []
|
|
229
|
+
let chars = Array(text)
|
|
230
|
+
var i = 0
|
|
231
|
+
while i < chars.count {
|
|
232
|
+
guard chars[i] == "{" else { i += 1; continue }
|
|
233
|
+
guard let close = chars[(i + 1)...].firstIndex(of: "}") else { break }
|
|
234
|
+
blocks.append(String(chars[(i + 1)..<close]))
|
|
235
|
+
i = close + 1
|
|
236
|
+
}
|
|
237
|
+
return blocks
|
|
238
|
+
}
|
|
239
|
+
|
|
240
|
+
/// Override blocks out, ASS escapes in.
|
|
241
|
+
private static func strippedText(_ raw: String) -> String {
|
|
242
|
+
var out = ""
|
|
243
|
+
let chars = Array(raw)
|
|
244
|
+
var i = 0
|
|
245
|
+
while i < chars.count {
|
|
246
|
+
if chars[i] == "{", let close = chars[(i + 1)...].firstIndex(of: "}") {
|
|
247
|
+
i = close + 1
|
|
248
|
+
continue
|
|
249
|
+
}
|
|
250
|
+
if chars[i] == "\\", i + 1 < chars.count {
|
|
251
|
+
switch chars[i + 1] {
|
|
252
|
+
case "N", "n": out.append("\n"); i += 2; continue
|
|
253
|
+
// A hard space: U+00A0, not U+0020. A plain space would let the layout break the line
|
|
254
|
+
// exactly where the typesetter wrote the escape to stop it doing so.
|
|
255
|
+
case "h": out.append("\u{00A0}"); i += 2; continue
|
|
256
|
+
default: break
|
|
257
|
+
}
|
|
258
|
+
}
|
|
259
|
+
out.append(chars[i])
|
|
260
|
+
i += 1
|
|
261
|
+
}
|
|
262
|
+
return out
|
|
263
|
+
}
|
|
264
|
+
}
|
|
@@ -0,0 +1,302 @@
|
|
|
1
|
+
import AVKit
|
|
2
|
+
import ExpoModulesCore
|
|
3
|
+
import UIKit
|
|
4
|
+
|
|
5
|
+
/// Draws ASS typesetting — signs, placards, on-screen labels — with the transforms the script asks
|
|
6
|
+
/// for.
|
|
7
|
+
///
|
|
8
|
+
/// AVFoundation cannot: it renders a `.text` track itself, WebVTT can express no rotation, scale or
|
|
9
|
+
/// per-cue opacity, and `AVTextStyleRule` has no attribute for any of them. So `SubtitleConverter`
|
|
10
|
+
/// takes those events out of the WebVTT it builds and leaves them in `AssSignStore`, and this view
|
|
11
|
+
/// draws them.
|
|
12
|
+
///
|
|
13
|
+
/// ### Where it sits
|
|
14
|
+
///
|
|
15
|
+
/// In `AVPlayerViewController.contentOverlayView`, which is the frame Apple provides between the
|
|
16
|
+
/// video and the transport controls. Sized to `videoBounds` on every frame, so `\pos` — which is in
|
|
17
|
+
/// `PlayResX`/`PlayResY` space — maps onto the picture rather than onto the player, and survives
|
|
18
|
+
/// letterboxing and rotation without any arithmetic of its own.
|
|
19
|
+
///
|
|
20
|
+
/// ### Why it drives itself from the clock
|
|
21
|
+
///
|
|
22
|
+
/// Signs do not ride the subtitle track, so nothing announces when one starts. It reads the player's
|
|
23
|
+
/// time on each frame instead, which is also the only granularity that works: frame-by-frame
|
|
24
|
+
/// typesetting writes one event per frame, and a sign on screen for 40 ms has to arrive and leave on
|
|
25
|
+
/// the frame it was authored for.
|
|
26
|
+
final class AssSignOverlayView: UIView {
|
|
27
|
+
|
|
28
|
+
private weak var playerViewController: AVPlayerViewController?
|
|
29
|
+
private var displayLink: CADisplayLink?
|
|
30
|
+
|
|
31
|
+
private var signs: [AssSign] = []
|
|
32
|
+
private var active: [AssSign] = []
|
|
33
|
+
/// What `signs` was resolved from. Both halves matter: the item changes when the next episode
|
|
34
|
+
/// loads, the language when the viewer switches track.
|
|
35
|
+
private var resolvedItem: ObjectIdentifier?
|
|
36
|
+
private var resolvedLanguage: String?
|
|
37
|
+
private var lastLookup: CFTimeInterval = 0
|
|
38
|
+
|
|
39
|
+
/// Rebuilt only when the on-screen set changes, not per frame.
|
|
40
|
+
private var rendered: [RenderedSign] = []
|
|
41
|
+
|
|
42
|
+
private struct RenderedSign {
|
|
43
|
+
let text: NSAttributedString
|
|
44
|
+
let size: CGSize
|
|
45
|
+
let origin: CGPoint
|
|
46
|
+
let transform: CGAffineTransform
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
// MARK: - Lifecycle
|
|
50
|
+
|
|
51
|
+
init(playerViewController: AVPlayerViewController) {
|
|
52
|
+
self.playerViewController = playerViewController
|
|
53
|
+
super.init(frame: .zero)
|
|
54
|
+
isUserInteractionEnabled = false
|
|
55
|
+
backgroundColor = .clear
|
|
56
|
+
contentMode = .redraw
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
@available(*, unavailable)
|
|
60
|
+
required init?(coder: NSCoder) {
|
|
61
|
+
fatalError("init(coder:) is not used")
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
/// Adds the overlay to the controller's content overlay, or returns the one already there.
|
|
65
|
+
@discardableResult
|
|
66
|
+
static func install(in controller: AVPlayerViewController) -> AssSignOverlayView? {
|
|
67
|
+
guard let host = controller.contentOverlayView else { return nil }
|
|
68
|
+
|
|
69
|
+
for subview in host.subviews {
|
|
70
|
+
if let existing = subview as? AssSignOverlayView { return existing }
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
let overlay = AssSignOverlayView(playerViewController: controller)
|
|
74
|
+
host.addSubview(overlay)
|
|
75
|
+
overlay.start()
|
|
76
|
+
return overlay
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
static func remove(from controller: AVPlayerViewController) {
|
|
80
|
+
guard let host = controller.contentOverlayView else { return }
|
|
81
|
+
for subview in host.subviews {
|
|
82
|
+
guard let overlay = subview as? AssSignOverlayView else { continue }
|
|
83
|
+
overlay.stop()
|
|
84
|
+
overlay.removeFromSuperview()
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
private func start() {
|
|
89
|
+
guard displayLink == nil else { return }
|
|
90
|
+
let link = CADisplayLink(target: self, selector: #selector(tick))
|
|
91
|
+
link.add(to: .main, forMode: .common)
|
|
92
|
+
displayLink = link
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
private func stop() {
|
|
96
|
+
displayLink?.invalidate()
|
|
97
|
+
displayLink = nil
|
|
98
|
+
clearResolved()
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
override func willMove(toWindow newWindow: UIWindow?) {
|
|
102
|
+
super.willMove(toWindow: newWindow)
|
|
103
|
+
if newWindow == nil {
|
|
104
|
+
displayLink?.invalidate()
|
|
105
|
+
displayLink = nil
|
|
106
|
+
} else {
|
|
107
|
+
start()
|
|
108
|
+
}
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
// MARK: - The clock
|
|
112
|
+
|
|
113
|
+
@objc private func tick() {
|
|
114
|
+
guard let controller = playerViewController, let player = controller.player else { return }
|
|
115
|
+
|
|
116
|
+
// The picture moves under us: rotation, full screen, and a letterbox that changes with the
|
|
117
|
+
// video's own aspect all land here rather than in any positioning arithmetic.
|
|
118
|
+
let bounds = controller.videoBounds
|
|
119
|
+
if bounds != frame, bounds.width > 0, bounds.height > 0 {
|
|
120
|
+
frame = bounds
|
|
121
|
+
// Every measurement in `prepare` is in view pixels, so all of it is stale after a resize.
|
|
122
|
+
rendered = []
|
|
123
|
+
setNeedsDisplay()
|
|
124
|
+
}
|
|
125
|
+
|
|
126
|
+
// Re-resolved on a timer rather than only when empty. Gating on "no signs yet" is the same
|
|
127
|
+
// mistake twice: it never fires again once a video with typesetting has loaded, so the next
|
|
128
|
+
// episode inherits the previous one's signs and matches them against its own timeline. The
|
|
129
|
+
// Android side shipped exactly that and it took two releases to unpick.
|
|
130
|
+
let now = CACurrentMediaTime()
|
|
131
|
+
if now - lastLookup >= AssSignOverlayView.lookupInterval {
|
|
132
|
+
lastLookup = now
|
|
133
|
+
resolveSigns(for: player)
|
|
134
|
+
}
|
|
135
|
+
if signs.isEmpty { return }
|
|
136
|
+
|
|
137
|
+
let positionMs = Int(CMTimeGetSeconds(player.currentTime()) * 1000)
|
|
138
|
+
setActive(signs.filter { positionMs >= $0.startMs && positionMs < $0.endMs })
|
|
139
|
+
}
|
|
140
|
+
|
|
141
|
+
private func resolveSigns(for player: AVPlayer) {
|
|
142
|
+
guard let item = player.currentItem else {
|
|
143
|
+
if resolvedItem != nil { clearResolved() }
|
|
144
|
+
return
|
|
145
|
+
}
|
|
146
|
+
|
|
147
|
+
let identity = ObjectIdentifier(item)
|
|
148
|
+
let language = selectedSubtitleLanguage(of: item)
|
|
149
|
+
guard identity != resolvedItem || language != resolvedLanguage else { return }
|
|
150
|
+
|
|
151
|
+
resolvedItem = identity
|
|
152
|
+
resolvedLanguage = language
|
|
153
|
+
|
|
154
|
+
// Typesetting rides on the item that owns it, so there is no key to mismatch and no store to go
|
|
155
|
+
// stale: a different item simply has different signs.
|
|
156
|
+
let found = (item as? VideoPlayerItem)
|
|
157
|
+
.flatMap { playerItem in language.map { playerItem.assSigns[AssSign.normalizeLanguage($0)] } }
|
|
158
|
+
?? nil
|
|
159
|
+
|
|
160
|
+
signs = found ?? []
|
|
161
|
+
rendered = []
|
|
162
|
+
setActive([])
|
|
163
|
+
log.info("[expo-video] ASS signs for \(language ?? "nil"): \(signs.count)")
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
private func clearResolved() {
|
|
167
|
+
resolvedItem = nil
|
|
168
|
+
resolvedLanguage = nil
|
|
169
|
+
signs = []
|
|
170
|
+
rendered = []
|
|
171
|
+
setActive([])
|
|
172
|
+
}
|
|
173
|
+
|
|
174
|
+
/// The locale of the selected legible option, which is the only identity a sideloaded track keeps
|
|
175
|
+
/// once it is inside the composition.
|
|
176
|
+
private func selectedSubtitleLanguage(of item: AVPlayerItem) -> String? {
|
|
177
|
+
guard let group = item.asset.mediaSelectionGroup(forMediaCharacteristic: .legible),
|
|
178
|
+
let option = item.currentMediaSelection.selectedMediaOption(in: group) else {
|
|
179
|
+
return nil
|
|
180
|
+
}
|
|
181
|
+
return option.locale?.identifier ?? option.extendedLanguageTag
|
|
182
|
+
}
|
|
183
|
+
|
|
184
|
+
private func setActive(_ next: [AssSign]) {
|
|
185
|
+
// Value equality across the whole sign, not a (time, text) stand-in: two events can share both
|
|
186
|
+
// and differ in where they sit, which is the case a frame-by-frame animation is made of.
|
|
187
|
+
guard next != active else { return }
|
|
188
|
+
active = next
|
|
189
|
+
rendered = []
|
|
190
|
+
setNeedsDisplay()
|
|
191
|
+
}
|
|
192
|
+
|
|
193
|
+
// MARK: - Drawing
|
|
194
|
+
|
|
195
|
+
override func draw(_ rect: CGRect) {
|
|
196
|
+
guard let context = UIGraphicsGetCurrentContext(), bounds.width > 0, bounds.height > 0 else { return }
|
|
197
|
+
|
|
198
|
+
if rendered.isEmpty && !active.isEmpty {
|
|
199
|
+
rendered = active.compactMap(prepare)
|
|
200
|
+
}
|
|
201
|
+
|
|
202
|
+
for sign in rendered {
|
|
203
|
+
context.saveGState()
|
|
204
|
+
context.concatenate(sign.transform)
|
|
205
|
+
sign.text.draw(at: sign.origin)
|
|
206
|
+
context.restoreGState()
|
|
207
|
+
}
|
|
208
|
+
}
|
|
209
|
+
|
|
210
|
+
private func prepare(_ sign: AssSign) -> RenderedSign? {
|
|
211
|
+
let alpha = max(0, min(1, sign.alpha))
|
|
212
|
+
if alpha == 0 { return nil }
|
|
213
|
+
|
|
214
|
+
// Script space to view space. Both axes scale independently, so a script authored at a different
|
|
215
|
+
// aspect ratio than the video still lands where it was drawn.
|
|
216
|
+
let scaleToViewX = bounds.width / CGFloat(sign.playResX)
|
|
217
|
+
let scaleToViewY = bounds.height / CGFloat(sign.playResY)
|
|
218
|
+
|
|
219
|
+
let attributed = attributedText(for: sign, scaleToViewY: scaleToViewY, alpha: alpha)
|
|
220
|
+
let size = attributed.boundingRect(
|
|
221
|
+
with: CGSize(width: CGFloat.greatestFiniteMagnitude, height: CGFloat.greatestFiniteMagnitude),
|
|
222
|
+
options: [.usesLineFragmentOrigin, .usesFontLeading],
|
|
223
|
+
context: nil
|
|
224
|
+
).size
|
|
225
|
+
|
|
226
|
+
let anchorX = CGFloat(sign.x) * scaleToViewX
|
|
227
|
+
let anchorY = CGFloat(sign.y) * scaleToViewY
|
|
228
|
+
// \pos names the point the cue's alignment corner sits on, so the box is offset from it.
|
|
229
|
+
let left: CGFloat
|
|
230
|
+
switch (sign.alignment - 1) % 3 {
|
|
231
|
+
case 0: left = anchorX
|
|
232
|
+
case 2: left = anchorX - size.width
|
|
233
|
+
default: left = anchorX - size.width / 2
|
|
234
|
+
}
|
|
235
|
+
let top: CGFloat
|
|
236
|
+
switch (sign.alignment - 1) / 3 {
|
|
237
|
+
case 0: top = anchorY - size.height
|
|
238
|
+
case 2: top = anchorY
|
|
239
|
+
default: top = anchorY - size.height / 2
|
|
240
|
+
}
|
|
241
|
+
|
|
242
|
+
// \org moves the centre of rotation away from the text, which is how a sign stays glued to a
|
|
243
|
+
// placard pivoting off-screen. Without it every rotation would spin about the text's own middle.
|
|
244
|
+
let pivotX = CGFloat(sign.pivotX) * scaleToViewX
|
|
245
|
+
let pivotY = CGFloat(sign.pivotY) * scaleToViewY
|
|
246
|
+
|
|
247
|
+
// Scale first, then turn — ASS scales the glyph and then rotates it, and with an anisotropic
|
|
248
|
+
// scale the two do not commute. `concatenating` applies the receiver first, so the order reads
|
|
249
|
+
// in the order it happens.
|
|
250
|
+
let transform = CGAffineTransform(translationX: -pivotX, y: -pivotY)
|
|
251
|
+
.concatenating(CGAffineTransform(scaleX: CGFloat(sign.scaleX), y: CGFloat(sign.scaleY)))
|
|
252
|
+
.concatenating(CGAffineTransform(rotationAngle: CGFloat(sign.rotationZ) * .pi / 180))
|
|
253
|
+
.concatenating(CGAffineTransform(translationX: pivotX, y: pivotY))
|
|
254
|
+
|
|
255
|
+
return RenderedSign(text: attributed, size: size, origin: CGPoint(x: left, y: top), transform: transform)
|
|
256
|
+
}
|
|
257
|
+
|
|
258
|
+
private func attributedText(for sign: AssSign, scaleToViewY: CGFloat, alpha: Double) -> NSAttributedString {
|
|
259
|
+
let pointSize = max(1, CGFloat(sign.fontSize) * scaleToViewY)
|
|
260
|
+
var font = UIFont.systemFont(ofSize: pointSize, weight: sign.bold ? .bold : .regular)
|
|
261
|
+
if sign.italic, let descriptor = font.fontDescriptor.withSymbolicTraits(.traitItalic) {
|
|
262
|
+
font = UIFont(descriptor: descriptor, size: pointSize)
|
|
263
|
+
}
|
|
264
|
+
|
|
265
|
+
let paragraph = NSMutableParagraphStyle()
|
|
266
|
+
switch (sign.alignment - 1) % 3 {
|
|
267
|
+
case 0: paragraph.alignment = .left
|
|
268
|
+
case 2: paragraph.alignment = .right
|
|
269
|
+
default: paragraph.alignment = .center
|
|
270
|
+
}
|
|
271
|
+
|
|
272
|
+
var attributes: [NSAttributedString.Key: Any] = [
|
|
273
|
+
.font: font,
|
|
274
|
+
.foregroundColor: colour(sign.colour, alpha: alpha),
|
|
275
|
+
.paragraphStyle: paragraph
|
|
276
|
+
]
|
|
277
|
+
if sign.underline {
|
|
278
|
+
attributes[.underlineStyle] = NSUnderlineStyle.single.rawValue
|
|
279
|
+
}
|
|
280
|
+
if sign.outlineWidth > 0 {
|
|
281
|
+
attributes[.strokeColor] = colour(sign.outlineColour, alpha: alpha)
|
|
282
|
+
// Negative means stroke *and* fill. A positive width would draw the outline only, leaving the
|
|
283
|
+
// letters hollow.
|
|
284
|
+
attributes[.strokeWidth] = -sign.outlineWidth * 2
|
|
285
|
+
}
|
|
286
|
+
|
|
287
|
+
return NSAttributedString(string: sign.text, attributes: attributes)
|
|
288
|
+
}
|
|
289
|
+
|
|
290
|
+
/// ASS colours carry no alpha byte; opacity travels separately.
|
|
291
|
+
private func colour(_ rgb: Int, alpha: Double) -> UIColor {
|
|
292
|
+
UIColor(
|
|
293
|
+
red: CGFloat((rgb >> 16) & 0xFF) / 255,
|
|
294
|
+
green: CGFloat((rgb >> 8) & 0xFF) / 255,
|
|
295
|
+
blue: CGFloat(rgb & 0xFF) / 255,
|
|
296
|
+
alpha: CGFloat(alpha)
|
|
297
|
+
)
|
|
298
|
+
}
|
|
299
|
+
|
|
300
|
+
/// How often to ask the store for signs that have not been parsed yet.
|
|
301
|
+
private static let lookupInterval: CFTimeInterval = 0.25
|
|
302
|
+
}
|
|
@@ -285,12 +285,14 @@ internal struct RenderedSubtitleText {
|
|
|
285
285
|
var isDrawing: Bool
|
|
286
286
|
}
|
|
287
287
|
|
|
288
|
-
|
|
288
|
+
internal enum InlineToken {
|
|
289
289
|
case text(String)
|
|
290
290
|
case markup(String)
|
|
291
291
|
}
|
|
292
292
|
|
|
293
|
-
private
|
|
293
|
+
// Internal rather than private: `AssSign.swift` reuses the override-block scanner and the colour
|
|
294
|
+
// parser, and reimplementing either would be two copies that have to agree about ASS forever.
|
|
295
|
+
internal extension SubtitleConverter {
|
|
294
296
|
/// Escapes only the parts of a cue that came from the source text.
|
|
295
297
|
///
|
|
296
298
|
/// Order matters: `&` first, then `<`, then `>`. Escaping `&` last would double-escape the one in
|
|
@@ -370,6 +372,9 @@ private extension SubtitleConverter {
|
|
|
370
372
|
case .underline(let on): on ? openTag("u") : closeTag("u")
|
|
371
373
|
case .drawing: isDrawing = true
|
|
372
374
|
case .reset: closeAll()
|
|
375
|
+
// The transform tags are for signs, which never reach this converter — a sign is taken
|
|
376
|
+
// out of the script before the WebVTT is built, because WebVTT can express none of them.
|
|
377
|
+
default: break
|
|
373
378
|
}
|
|
374
379
|
}
|
|
375
380
|
i = end + 1
|
|
@@ -479,6 +484,50 @@ private extension SubtitleConverter {
|
|
|
479
484
|
case underline(Bool)
|
|
480
485
|
case drawing
|
|
481
486
|
case reset
|
|
487
|
+
// Read for signs only. The WebVTT path ignores every one of them, because WebVTT can express
|
|
488
|
+
// none of it — which is the whole reason signs are drawn separately.
|
|
489
|
+
case origin(Double, Double)
|
|
490
|
+
case rotationZ(Double)
|
|
491
|
+
case rotationX(Double)
|
|
492
|
+
case rotationY(Double)
|
|
493
|
+
case scaleX(Double)
|
|
494
|
+
case scaleY(Double)
|
|
495
|
+
case fontSize(Double)
|
|
496
|
+
case primaryColour(Int)
|
|
497
|
+
case primaryAlpha(Double)
|
|
498
|
+
case bold(Bool)
|
|
499
|
+
}
|
|
500
|
+
|
|
501
|
+
struct AssColour {
|
|
502
|
+
let rgb: Int
|
|
503
|
+
let alpha: Double
|
|
504
|
+
}
|
|
505
|
+
|
|
506
|
+
/// `&HAABBGGRR&` or `&HBBGGRR&`, with or without the ampersands.
|
|
507
|
+
///
|
|
508
|
+
/// The bytes are reversed against RGB and the alpha byte is inverted — `FF` is fully transparent.
|
|
509
|
+
/// A six-digit value carries no alpha byte and is therefore opaque.
|
|
510
|
+
static func parseAssColour(_ value: String, alphaOnly: Bool = false) -> AssColour? {
|
|
511
|
+
var text = value.trimmingCharacters(in: .whitespaces)
|
|
512
|
+
if text.hasSuffix("&") { text = String(text.dropLast()) }
|
|
513
|
+
if text.lowercased().hasPrefix("&h") { text = String(text.dropFirst(2)) }
|
|
514
|
+
guard !text.isEmpty, let hex = UInt32(text, radix: 16) else { return nil }
|
|
515
|
+
|
|
516
|
+
if alphaOnly || text.count <= 2 {
|
|
517
|
+
return AssColour(rgb: 0xFFFFFF, alpha: 1 - Double(hex & 0xFF) / 255)
|
|
518
|
+
}
|
|
519
|
+
|
|
520
|
+
let red: UInt32 = hex & 0xFF
|
|
521
|
+
let green: UInt32 = (hex >> 8) & 0xFF
|
|
522
|
+
let blue: UInt32 = (hex >> 16) & 0xFF
|
|
523
|
+
let rgb: Int = Int(red) << 16 | Int(green) << 8 | Int(blue)
|
|
524
|
+
|
|
525
|
+
var alpha: Double = 1
|
|
526
|
+
if text.count > 6 {
|
|
527
|
+
let transparency: UInt32 = (hex >> 24) & 0xFF
|
|
528
|
+
alpha = 1 - Double(transparency) / 255
|
|
529
|
+
}
|
|
530
|
+
return AssColour(rgb: rgb, alpha: alpha)
|
|
482
531
|
}
|
|
483
532
|
|
|
484
533
|
/// Scans one `{...}` block left to right.
|
|
@@ -487,6 +536,14 @@ private extension SubtitleConverter {
|
|
|
487
536
|
/// one pass and the `\i1` keeps its position within the text. Reading the tag name as a run of
|
|
488
537
|
/// letters is also what makes the longest-match problem disappear: `\c` cannot eat `\clip` and
|
|
489
538
|
/// `\b` cannot eat `\bord`, because those parse as the names "clip" and "bord".
|
|
539
|
+
/// A number, or nothing. `Double` parses "nan" and "inf", and neither belongs in a transform.
|
|
540
|
+
static func finite(_ text: String) -> Double? {
|
|
541
|
+
guard let value = Double(text.trimmingCharacters(in: CharacterSet.whitespaces)), value.isFinite else {
|
|
542
|
+
return nil
|
|
543
|
+
}
|
|
544
|
+
return value
|
|
545
|
+
}
|
|
546
|
+
|
|
490
547
|
static func parseOverrideBlock(_ block: String) -> [OverrideTag] {
|
|
491
548
|
var tags: [OverrideTag] = []
|
|
492
549
|
let chars = Array(block)
|
|
@@ -497,6 +554,10 @@ private extension SubtitleConverter {
|
|
|
497
554
|
i += 1
|
|
498
555
|
|
|
499
556
|
var name = ""
|
|
557
|
+
// A leading digit belongs to the name: \1c, \1a and \4a all start with one. Reading only
|
|
558
|
+
// letters left those as the empty name, which was harmless while colour was being dropped and
|
|
559
|
+
// is not once signs need it.
|
|
560
|
+
while i < chars.count, chars[i].isNumber { name.append(chars[i]); i += 1 }
|
|
500
561
|
while i < chars.count, chars[i].isLetter { name.append(chars[i]); i += 1 }
|
|
501
562
|
|
|
502
563
|
var argument = ""
|
|
@@ -526,9 +587,7 @@ private extension SubtitleConverter {
|
|
|
526
587
|
tags.append(.alignment(value))
|
|
527
588
|
}
|
|
528
589
|
case "pos":
|
|
529
|
-
let parts = argument.split(separator: ",").
|
|
530
|
-
Double($0.trimmingCharacters(in: .whitespaces)) ?? 0
|
|
531
|
-
}
|
|
590
|
+
let parts = argument.split(separator: ",").compactMap { finite(String($0)) }
|
|
532
591
|
if parts.count >= 2 { tags.append(.position(parts[0], parts[1])) }
|
|
533
592
|
case "i": tags.append(.italic(argument.trimmingCharacters(in: .whitespaces) == "1"))
|
|
534
593
|
case "u": tags.append(.underline(argument.trimmingCharacters(in: .whitespaces) == "1"))
|
|
@@ -539,6 +598,34 @@ private extension SubtitleConverter {
|
|
|
539
598
|
tags.append(.drawing)
|
|
540
599
|
}
|
|
541
600
|
case "r": tags.append(.reset)
|
|
601
|
+
case "move":
|
|
602
|
+
// A move ends where it ends; drawing it at the destination beats drawing it at the start.
|
|
603
|
+
let parts = argument.split(separator: ",").compactMap { finite(String($0)) }
|
|
604
|
+
if parts.count >= 4 { tags.append(.position(parts[2], parts[3])) }
|
|
605
|
+
case "org":
|
|
606
|
+
let parts = argument.split(separator: ",").compactMap { finite(String($0)) }
|
|
607
|
+
if parts.count >= 2 { tags.append(.origin(parts[0], parts[1])) }
|
|
608
|
+
// `Double("nan")` and `Double("inf")` parse. A subtitle is untrusted scraper output, and a
|
|
609
|
+
// non-finite value carried into a transform makes the sign silently fail to draw — so every
|
|
610
|
+
// number here goes through `finite`.
|
|
611
|
+
case "frz", "fr":
|
|
612
|
+
if let value = finite(argument) { tags.append(.rotationZ(value)) }
|
|
613
|
+
case "frx":
|
|
614
|
+
if let value = finite(argument) { tags.append(.rotationX(value)) }
|
|
615
|
+
case "fry":
|
|
616
|
+
if let value = finite(argument) { tags.append(.rotationY(value)) }
|
|
617
|
+
case "fscx":
|
|
618
|
+
if let value = finite(argument) { tags.append(.scaleX(value / 100)) }
|
|
619
|
+
case "fscy":
|
|
620
|
+
if let value = finite(argument) { tags.append(.scaleY(value / 100)) }
|
|
621
|
+
case "fs":
|
|
622
|
+
if let value = finite(argument), value > 0 { tags.append(.fontSize(value)) }
|
|
623
|
+
case "c", "1c":
|
|
624
|
+
if let colour = parseAssColour(argument) { tags.append(.primaryColour(colour.rgb)) }
|
|
625
|
+
case "alpha", "1a":
|
|
626
|
+
if let colour = parseAssColour(argument, alphaOnly: true) { tags.append(.primaryAlpha(colour.alpha)) }
|
|
627
|
+
case "b":
|
|
628
|
+
tags.append(.bold(argument.trimmingCharacters(in: .whitespaces) != "0"))
|
|
542
629
|
default: break
|
|
543
630
|
}
|
|
544
631
|
}
|
|
@@ -610,16 +697,35 @@ private extension SubtitleConverter {
|
|
|
610
697
|
/// Header fields are read from their own `Format:` lines rather than assumed, because encoders
|
|
611
698
|
/// reorder them and reading the Text column one position off produces a file that still parses and
|
|
612
699
|
/// is entirely wrong.
|
|
613
|
-
|
|
700
|
+
internal struct AssScript {
|
|
614
701
|
struct Style {
|
|
702
|
+
|
|
615
703
|
var alignment: Int
|
|
616
704
|
var italic: Bool
|
|
617
705
|
var underline: Bool
|
|
618
706
|
/// The author asked for this style to draw nothing. See `isInvisibleStyle`.
|
|
619
707
|
var invisible: Bool
|
|
708
|
+
// Read for signs. The WebVTT path uses none of it: colour, size and weight are the app's to
|
|
709
|
+
// decide there, and a sign is the one case where they are not.
|
|
710
|
+
var fontSize: Double = 48
|
|
711
|
+
var colour: Int = 0xFFFFFF
|
|
712
|
+
var alpha: Double = 1
|
|
713
|
+
var outlineColour: Int = 0x000000
|
|
714
|
+
var outlineWidth: Double = 0
|
|
715
|
+
var bold: Bool = false
|
|
716
|
+
var scaleX: Double = 1
|
|
717
|
+
var scaleY: Double = 1
|
|
718
|
+
var angle: Double = 0
|
|
719
|
+
var marginL: Double = 0
|
|
720
|
+
var marginR: Double = 0
|
|
721
|
+
var marginV: Double = 0
|
|
620
722
|
}
|
|
621
723
|
|
|
622
724
|
struct Event {
|
|
725
|
+
/// Index of the line this came from, so a sign can be cut out of the script handed to the
|
|
726
|
+
/// WebVTT converter. Mirrors the split on Android; it is what makes drawing an event twice
|
|
727
|
+
/// structurally impossible rather than merely avoided.
|
|
728
|
+
var lineIndex: Int = -1
|
|
623
729
|
var startMs: Int
|
|
624
730
|
var endMs: Int
|
|
625
731
|
var styleName: String
|
|
@@ -639,7 +745,7 @@ private struct AssScript {
|
|
|
639
745
|
var sawPlayResX = false
|
|
640
746
|
var sawPlayResY = false
|
|
641
747
|
|
|
642
|
-
for rawLine in content.components(separatedBy: "\n") {
|
|
748
|
+
for (lineIndex, rawLine) in content.components(separatedBy: "\n").enumerated() {
|
|
643
749
|
let line = rawLine.trimmingCharacters(in: .whitespaces)
|
|
644
750
|
if line.isEmpty || line.hasPrefix(";") { continue }
|
|
645
751
|
|
|
@@ -685,7 +791,19 @@ private struct AssScript {
|
|
|
685
791
|
: ((1...9).contains(rawAlignment) ? rawAlignment : 2),
|
|
686
792
|
italic: field("italic") == "-1",
|
|
687
793
|
underline: field("underline") == "-1",
|
|
688
|
-
invisible: Self.isInvisibleStyle(fontSize: field("fontsize"), primaryColour: field("primarycolour"))
|
|
794
|
+
invisible: Self.isInvisibleStyle(fontSize: field("fontsize"), primaryColour: field("primarycolour")),
|
|
795
|
+
fontSize: field("fontsize").flatMap(SubtitleConverter.finite) ?? 48,
|
|
796
|
+
colour: field("primarycolour").flatMap { SubtitleConverter.parseAssColour($0)?.rgb } ?? 0xFFFFFF,
|
|
797
|
+
alpha: field("primarycolour").flatMap { SubtitleConverter.parseAssColour($0)?.alpha } ?? 1,
|
|
798
|
+
outlineColour: field("outlinecolour").flatMap { SubtitleConverter.parseAssColour($0)?.rgb } ?? 0x000000,
|
|
799
|
+
outlineWidth: field("outline").flatMap(SubtitleConverter.finite) ?? 0,
|
|
800
|
+
bold: field("bold") == "-1",
|
|
801
|
+
scaleX: (field("scalex").flatMap(SubtitleConverter.finite) ?? 100) / 100,
|
|
802
|
+
scaleY: (field("scaley").flatMap(SubtitleConverter.finite) ?? 100) / 100,
|
|
803
|
+
angle: field("angle").flatMap(SubtitleConverter.finite) ?? 0,
|
|
804
|
+
marginL: field("marginl").flatMap(SubtitleConverter.finite) ?? 0,
|
|
805
|
+
marginR: field("marginr").flatMap(SubtitleConverter.finite) ?? 0,
|
|
806
|
+
marginV: field("marginv").flatMap(SubtitleConverter.finite) ?? 0
|
|
689
807
|
)
|
|
690
808
|
}
|
|
691
809
|
|
|
@@ -697,7 +815,8 @@ private struct AssScript {
|
|
|
697
815
|
} else if key.caseInsensitiveCompare("Dialogue") == .orderedSame {
|
|
698
816
|
// Only Dialogue reaches the screen. Comment carries typesetting notes and unused
|
|
699
817
|
// alternates, and Picture/Sound/Movie/Command are not text at all.
|
|
700
|
-
guard
|
|
818
|
+
guard var event = Self.parseEvent(value, fields: eventFields) else { continue }
|
|
819
|
+
event.lineIndex = lineIndex
|
|
701
820
|
events.append(event)
|
|
702
821
|
}
|
|
703
822
|
|
|
@@ -7,6 +7,13 @@ class VideoPlayerItem: AVPlayerItem {
|
|
|
7
7
|
let urlAsset: VideoAsset
|
|
8
8
|
let videoSource: VideoSource
|
|
9
9
|
let isHls: Bool
|
|
10
|
+
/// ASS typesetting taken out of this item's subtitle tracks, keyed by language subtag.
|
|
11
|
+
///
|
|
12
|
+
/// Carried on the item rather than in a store of its own, because the item *is* the identity the
|
|
13
|
+
/// overlay needs: two players can be loading different videos at once, and a store keyed by
|
|
14
|
+
/// anything a subtitle track keeps inside a composition — which is only its language — would have
|
|
15
|
+
/// them overwrite each other.
|
|
16
|
+
let assSigns: [String: [AssSign]]
|
|
10
17
|
var videoTracks: [VideoTrack] {
|
|
11
18
|
get async {
|
|
12
19
|
return await tracksLoadingTask?.value ?? []
|
|
@@ -24,6 +31,8 @@ class VideoPlayerItem: AVPlayerItem {
|
|
|
24
31
|
let asset = VideoAsset(url: url, videoSource: videoSource)
|
|
25
32
|
self.urlAsset = asset
|
|
26
33
|
self.isHls = asset.effectivePlaybackURL.isHLS || asset.effectiveContentType == .hls
|
|
34
|
+
// This initializer builds no composition, so it sideloads nothing and has no typesetting.
|
|
35
|
+
self.assSigns = [:]
|
|
27
36
|
super.init(asset: urlAsset, automaticallyLoadedAssetKeys: nil)
|
|
28
37
|
self.createTracksLoadingTask()
|
|
29
38
|
}
|
|
@@ -51,15 +60,18 @@ class VideoPlayerItem: AVPlayerItem {
|
|
|
51
60
|
// (non-HLS, non-DRM) sources — HLS declares its own subtitle renditions, and a composition would
|
|
52
61
|
// bypass the DRM/resource-loader path used by protected content.
|
|
53
62
|
var playbackAsset: AVAsset = asset
|
|
63
|
+
var signs: [String: [AssSign]] = [:]
|
|
54
64
|
if let subtitleTracks = videoSource.subtitleTracks, !subtitleTracks.isEmpty, !isHls, videoSource.drm == nil {
|
|
55
|
-
if let
|
|
65
|
+
if let sideloaded = await VideoPlayerSubtitleSideload.buildComposition(
|
|
56
66
|
from: asset,
|
|
57
67
|
subtitleTracks: subtitleTracks,
|
|
58
68
|
headers: videoSource.headers
|
|
59
69
|
) {
|
|
60
|
-
playbackAsset = composition
|
|
70
|
+
playbackAsset = sideloaded.composition
|
|
71
|
+
signs = sideloaded.signsByLanguage
|
|
61
72
|
}
|
|
62
73
|
}
|
|
74
|
+
self.assSigns = signs
|
|
63
75
|
|
|
64
76
|
super.init(asset: playbackAsset, automaticallyLoadedAssetKeys: nil)
|
|
65
77
|
self.createTracksLoadingTask()
|
|
@@ -13,14 +13,27 @@ import ExpoModulesCore
|
|
|
13
13
|
/// pipeline already understands. Only WebVTT is natively supported, so `SRT` files are converted first, and
|
|
14
14
|
/// remote files are downloaded to a temporary location because `insertTimeRange` requires a readable local track.
|
|
15
15
|
internal enum VideoPlayerSubtitleSideload {
|
|
16
|
+
/// A composition plus the typesetting taken out of its subtitle tracks, keyed by language.
|
|
17
|
+
///
|
|
18
|
+
/// The signs travel with the composition rather than through a store of their own. A store would
|
|
19
|
+
/// have to be keyed by something both ends can see, and the only identity a sideloaded track keeps
|
|
20
|
+
/// inside a composition is its language — which means two players loading different videos in the
|
|
21
|
+
/// same language would overwrite each other, and a second episode would inherit the first one's
|
|
22
|
+
/// signs. Handing them back here makes the `AVPlayerItem` itself the identity.
|
|
23
|
+
struct SideloadResult {
|
|
24
|
+
let composition: AVMutableComposition
|
|
25
|
+
let signsByLanguage: [String: [AssSign]]
|
|
26
|
+
}
|
|
27
|
+
|
|
16
28
|
static func buildComposition(
|
|
17
29
|
from asset: AVURLAsset,
|
|
18
30
|
subtitleTracks: [SubtitleSource],
|
|
19
31
|
headers: [String: String]?
|
|
20
|
-
) async ->
|
|
32
|
+
) async -> SideloadResult? {
|
|
21
33
|
do {
|
|
22
34
|
let mainDuration = try await asset.load(.duration)
|
|
23
35
|
let composition = AVMutableComposition()
|
|
36
|
+
var signsByLanguage: [String: [AssSign]] = [:]
|
|
24
37
|
|
|
25
38
|
// Copy the original audio + video tracks.
|
|
26
39
|
let tracks = try await asset.load(.tracks)
|
|
@@ -54,7 +67,7 @@ internal enum VideoPlayerSubtitleSideload {
|
|
|
54
67
|
|
|
55
68
|
var didAddSubtitle = false
|
|
56
69
|
for subtitle in subtitleTracks {
|
|
57
|
-
guard let localURL = await resolveLocalVttURL(for: subtitle, headers: headers) else {
|
|
70
|
+
guard let localURL = await resolveLocalVttURL(for: subtitle, headers: headers, signs: &signsByLanguage) else {
|
|
58
71
|
continue
|
|
59
72
|
}
|
|
60
73
|
let subtitleAsset = AVURLAsset(url: localURL)
|
|
@@ -94,7 +107,7 @@ internal enum VideoPlayerSubtitleSideload {
|
|
|
94
107
|
}
|
|
95
108
|
}
|
|
96
109
|
|
|
97
|
-
return didAddSubtitle ? composition : nil
|
|
110
|
+
return didAddSubtitle ? SideloadResult(composition: composition, signsByLanguage: signsByLanguage) : nil
|
|
98
111
|
} catch {
|
|
99
112
|
log.warn("[expo-video] Failed to build subtitle composition: \(error.localizedDescription)")
|
|
100
113
|
return nil
|
|
@@ -108,7 +121,11 @@ internal enum VideoPlayerSubtitleSideload {
|
|
|
108
121
|
/// also preserved the colours and weights a WebVTT file carries in `<c.yellow>` and `<b>`, which
|
|
109
122
|
/// would then beat `subtitleStyle`. The temp-file cache means the work happens once per
|
|
110
123
|
/// (uri, language, format, offset).
|
|
111
|
-
private static func resolveLocalVttURL(
|
|
124
|
+
private static func resolveLocalVttURL(
|
|
125
|
+
for subtitle: SubtitleSource,
|
|
126
|
+
headers: [String: String]?,
|
|
127
|
+
signs: inout [String: [AssSign]]
|
|
128
|
+
) async -> URL? {
|
|
112
129
|
guard let uri = subtitle.uri else {
|
|
113
130
|
return nil
|
|
114
131
|
}
|
|
@@ -130,7 +147,15 @@ internal enum VideoPlayerSubtitleSideload {
|
|
|
130
147
|
}
|
|
131
148
|
|
|
132
149
|
switch subtitle.resolvedFormat {
|
|
133
|
-
case .ass:
|
|
150
|
+
case .ass:
|
|
151
|
+
// Typesetting is taken out here rather than converted. WebVTT can express no rotation, scale
|
|
152
|
+
// or per-cue opacity, and AVFoundation renders the track itself, so a sign that went through
|
|
153
|
+
// would come out flat. `AssSignOverlayView` draws them instead.
|
|
154
|
+
let split = SubtitleConverter.splitAss(text)
|
|
155
|
+
if !split.signs.isEmpty {
|
|
156
|
+
signs[AssSign.normalizeLanguage(subtitle.language ?? "und")] = split.signs
|
|
157
|
+
}
|
|
158
|
+
text = SubtitleConverter.assToVtt(split.dialogueOnly)
|
|
134
159
|
case .srt: text = SubtitleConverter.srtToVtt(text)
|
|
135
160
|
case .vtt: text = SubtitleConverter.sanitizeVtt(text)
|
|
136
161
|
}
|
package/ios/VideoView.swift
CHANGED
|
@@ -11,6 +11,14 @@ public final class VideoView: ExpoView, AVPlayerViewControllerDelegate {
|
|
|
11
11
|
playerViewController.player = player?.ref
|
|
12
12
|
// Forward any subtitle style set before the player was attached.
|
|
13
13
|
player?.subtitleStyle = subtitleStyle
|
|
14
|
+
// ASS typesetting is drawn by us, not by AVFoundation, which can express none of its
|
|
15
|
+
// transforms. Installed whenever a player arrives and torn down with the view; the overlay
|
|
16
|
+
// sits idle and costs a display-link tick until a track with signs is actually loaded.
|
|
17
|
+
if player != nil {
|
|
18
|
+
AssSignOverlayView.install(in: playerViewController)
|
|
19
|
+
} else {
|
|
20
|
+
AssSignOverlayView.remove(from: playerViewController)
|
|
21
|
+
}
|
|
14
22
|
}
|
|
15
23
|
}
|
|
16
24
|
|
|
@@ -74,6 +82,7 @@ public final class VideoView: ExpoView, AVPlayerViewControllerDelegate {
|
|
|
74
82
|
deinit {
|
|
75
83
|
VideoManager.shared.unregister(videoView: self)
|
|
76
84
|
removeFirstFrameObserver()
|
|
85
|
+
AssSignOverlayView.remove(from: playerViewController)
|
|
77
86
|
}
|
|
78
87
|
|
|
79
88
|
func enterFullscreen() {
|
package/package.json
CHANGED