expo-video-subtitle 0.4.1 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +132 -4
- package/FORK.md +58 -30
- package/README.md +41 -17
- package/android/src/main/java/expo/modules/video/records/SubtitleSource.kt +12 -2
- package/android/src/main/java/expo/modules/video/records/SubtitleStyle.kt +9 -4
- package/android/src/main/java/expo/modules/video/utils/SsaCueNormalizer.kt +93 -0
- package/android/src/main/java/expo/modules/video/utils/StackingSubtitleParser.kt +55 -13
- package/android/src/main/java/expo/modules/video/utils/SubtitleSpanFilter.kt +156 -0
- package/android/src/main/java/expo/modules/video/utils/SubtitleUtils.kt +12 -6
- package/build/VideoPlayer.types.d.ts +18 -5
- package/build/VideoPlayer.types.d.ts.map +1 -1
- package/build/VideoPlayer.types.js.map +1 -1
- package/build/VideoView.types.d.ts +7 -12
- package/build/VideoView.types.d.ts.map +1 -1
- package/build/VideoView.types.js.map +1 -1
- package/ios/Records/SubtitleSource.swift +26 -5
- package/ios/Records/SubtitleStyle.swift +8 -0
- package/ios/SubtitleConverterText.swift +781 -0
- package/ios/VideoPlayerSubtitleSideload.swift +40 -48
- package/package.json +1 -1
- package/src/VideoPlayer.types.ts +18 -5
- package/src/VideoView.types.ts +7 -12
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,132 @@ Changes inherited from the upstream base version are not repeated here — see t
|
|
|
7
7
|
[upstream changelog](https://github.com/expo/expo/blob/main/packages/expo-video/CHANGELOG.md)
|
|
8
8
|
for the history of everything that isn't subtitle related.
|
|
9
9
|
|
|
10
|
+
## 0.6.0
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
|
|
14
|
+
- **ASS dialogue keeps working with `bottomOffset` and with cue stacking again (Android).** Media3's
|
|
15
|
+
`SsaParser` calls `setLine` and `setPosition` on *every* cue, including the bottom-centre dialogue
|
|
16
|
+
the author placed nothing on. `SubtitlePainter` applies `bottomPaddingFraction` only to cues whose
|
|
17
|
+
`line` is unset, so `SubtitleSource.bottomOffset` silently stopped moving anything on an `.ass`
|
|
18
|
+
track, and `StackingSubtitleParser` treated every cue as deliberately placed and left simultaneous
|
|
19
|
+
speakers drawing on top of each other.
|
|
20
|
+
|
|
21
|
+
A new `SsaCueNormalizer` clears the placement Media3 filled in. Recognising it is exact rather
|
|
22
|
+
than a guess: the values are a pure function of the cue's anchors, so a cue matching both defaults
|
|
23
|
+
for the bottom-centre anchors is precisely a cue with no positioning tag. Anything the author did
|
|
24
|
+
place — `\an8`, `\pos`, a style aligned elsewhere — keeps everything and still bypasses stacking.
|
|
25
|
+
|
|
26
|
+
- **Signs keep the colours they were typeset in (Android).** The span filter dropped colour from
|
|
27
|
+
every cue so the app's own subtitle style would win, which is right for dialogue and wrong for a
|
|
28
|
+
sign: typesetting is drawn onto the artwork in colours chosen to sit on it, and the dark lettering
|
|
29
|
+
on the white placards in Jujutsu Kaisen S3E3 became white-on-white. Cues an ASS track positions
|
|
30
|
+
itself now keep their appearance; unpositioned dialogue still loses its own and follows the
|
|
31
|
+
viewer's settings. Only ASS opts in — it is the one format where positioning reliably means
|
|
32
|
+
"sign" rather than a cue setting on ordinary dialogue.
|
|
33
|
+
|
|
34
|
+
- **Typesetters' working notes stay hidden.** A script ships its production notes on a style that
|
|
35
|
+
renders nothing: JJK S3E3 carries `Style: f0ld3r-invisible` with `Fontsize 0` and every colour
|
|
36
|
+
`&HFF……`, holding 158 events of "--mahito", "#1", "check in frame 6212". libass honours both
|
|
37
|
+
signals. Media3 turns a transparent primary colour into a `ForegroundColorSpan` with alpha 0,
|
|
38
|
+
which the span filter then strips, so the note came back in the viewer's colour at the viewer's
|
|
39
|
+
size; the iOS converter never read either field, and WebVTT cannot express opacity anyway. Events
|
|
40
|
+
on such a style are now dropped on both platforms, before anything can make them visible.
|
|
41
|
+
|
|
42
|
+
### Known limitations
|
|
43
|
+
|
|
44
|
+
- Sign **font size** is still the app's, not the script's. Media3 does read `Fontsize` into
|
|
45
|
+
`Cue.textSize`, but the filter clears it so the two subtitle views measure the same glyphs; giving
|
|
46
|
+
it back to signs needs the rounded-background view checked on a device first.
|
|
47
|
+
- Blur, borders, shadows, rotation and `\t` animation are still dropped. Matching libass output
|
|
48
|
+
exactly needs libass, not Media3.
|
|
49
|
+
|
|
50
|
+
## 0.5.0
|
|
51
|
+
|
|
52
|
+
### Added
|
|
53
|
+
|
|
54
|
+
- **`SubtitleSource.format` accepts `'ass'` and `'ssa'`.** Advanced SubStation Alpha is what anime
|
|
55
|
+
releases actually ship, and it is the only common format that can place a caption anywhere on
|
|
56
|
+
screen rather than at the bottom.
|
|
57
|
+
|
|
58
|
+
The two platforms get there differently. **Android** renders the original file: Media3's
|
|
59
|
+
`DefaultSubtitleParserFactory` already accepts `MimeTypes.TEXT_SSA`, so this is a MIME mapping and
|
|
60
|
+
nothing more. **iOS** converts to WebVTT inside the module at load time, because `AVFoundation`
|
|
61
|
+
reads only WebVTT as a `.text` track.
|
|
62
|
+
|
|
63
|
+
What survives: `\an1`-`\an9` and the legacy `\a`, `\pos`, `\N` / `\n` / `\h`, italic and
|
|
64
|
+
underline. What is dropped: colour, font size, font weight, karaoke timings, transforms, rotations,
|
|
65
|
+
borders and shadows, `\move`, `\fad`, `\clip`, vector drawings (`\p1`), and `Comment:` events.
|
|
66
|
+
|
|
67
|
+
On iOS, a `\an8` sign lands at `line:5%`, matching Media3's `SsaParser.DEFAULT_MARGIN` so the same
|
|
68
|
+
sign sits in the same place on both platforms.
|
|
69
|
+
|
|
70
|
+
- **iOS:** `.srt` files carrying `{\anN}` leaked from an ASS converter now position correctly instead
|
|
71
|
+
of printing the tag on screen. The SubRip path was a two-line regex over timestamps; it is now a cue
|
|
72
|
+
block parser, because a tag found in a cue's *text* has to become a setting on the *timing line*
|
|
73
|
+
above it.
|
|
74
|
+
|
|
75
|
+
- **iOS:** subtitle files that are not UTF-8 are decoded instead of being dropped without a message.
|
|
76
|
+
The ladder is BOM sniff, UTF-8, then Windows-1252/1251, Shift-JIS, Big5, GB 18030, and finally
|
|
77
|
+
Latin-1, which never fails. Previously `String(data:encoding:.utf8)` returned nil for a Windows-1251
|
|
78
|
+
file and the whole track vanished silently.
|
|
79
|
+
|
|
80
|
+
- **`scripts/verify-subtitle-converter.sh`** compiles the shipped converter against 87 assertions and
|
|
81
|
+
runs them. The tag table, the `\an` to cue-settings mapping and the escaping order are all pure
|
|
82
|
+
string transforms, and all three fail silently on a device.
|
|
83
|
+
|
|
84
|
+
### Changed
|
|
85
|
+
|
|
86
|
+
- **Colour, font size and font weight carried by a subtitle file are now always ignored, on both
|
|
87
|
+
platforms.** `subtitleStyle` is the only thing that decides them. This is deliberate and there is no
|
|
88
|
+
way to opt out: a track shipping yellow 72pt bold text should not override the size a user chose in
|
|
89
|
+
your app's settings.
|
|
90
|
+
|
|
91
|
+
On Android this moved into the parser (`SubtitleSpanFilter`), because `SubtitleView`'s own
|
|
92
|
+
`applyEmbeddedStyles` flag is all-or-nothing and, when off, strips italic along with everything else.
|
|
93
|
+
Note it never touches `line`, `position` or `textAlignment`, so a track's placement survives either
|
|
94
|
+
setting.
|
|
95
|
+
|
|
96
|
+
- **`subtitleStyle.applyEmbeddedStyles` now defaults to `true`** and means "keep the track's italic,
|
|
97
|
+
underline and strikethrough". If you want the old flattened look, set it to `false` explicitly.
|
|
98
|
+
|
|
99
|
+
- **iOS:** subtitle text is now HTML-escaped, so a cue containing `<`, `&` or `-->` renders literally
|
|
100
|
+
instead of being read as markup or as a cue boundary.
|
|
101
|
+
|
|
102
|
+
- **iOS:** a sideloaded `.vtt` is no longer passed through untouched. It goes through the same
|
|
103
|
+
sanitising pass, because `<c.yellow>` and `<b>` are a WebVTT file's colour and weight and belong to
|
|
104
|
+
`subtitleStyle` like every other track's do.
|
|
105
|
+
|
|
106
|
+
- **iOS:** the cached rendition's `formatVersion` moved to `v3` and the cache key now includes the
|
|
107
|
+
format, so an install that has already played a track picks up the newly converted file. Without the
|
|
108
|
+
bump the change appears not to work on exactly the devices used to test it.
|
|
109
|
+
|
|
110
|
+
### Known limitations
|
|
111
|
+
|
|
112
|
+
- **Strikethrough renders on Android only.** WebVTT has no strikethrough tag and CoreMedia has no text
|
|
113
|
+
markup attribute for one, so there is no iOS workaround to reach for.
|
|
114
|
+
|
|
115
|
+
- **`bottomOffset` does not move ASS cues on Android.** `SsaParser` gives every cue an explicit line
|
|
116
|
+
and `SubtitlePainter` only applies `bottomPaddingFraction` to cues without one. On iOS the converter
|
|
117
|
+
emits no `line:` for bottom-centre cues, so ordinary dialogue still shifts there. Fixing Android
|
|
118
|
+
means a custom SSA parser that can tell a style default from a real `\pos`.
|
|
119
|
+
|
|
120
|
+
- **Simultaneous ASS cues at the same alignment can overlap on Android.** Media3 does no collision
|
|
121
|
+
avoidance between positioned cues; libass does. Two speakers talking over each other in an `.ass`
|
|
122
|
+
track will overprint.
|
|
123
|
+
|
|
124
|
+
- **Android renders ASS italic and underline from the `[V4+ Styles]` `Style:` line, not from inline
|
|
125
|
+
`{\i1}`.** Media3's `SsaStyle.Overrides` parses only `\an`, `\pos` and `\move` and then deletes
|
|
126
|
+
every remaining `{...}` block. iOS handles both.
|
|
127
|
+
|
|
128
|
+
- **`position:`, `align:` and the `,center` line alignment are newly relied on and unverified on
|
|
129
|
+
device.** Only `line:N%,end` has been confirmed (iOS 26.5). CoreMedia exposes a matching attribute
|
|
130
|
+
for each WebVTT cue setting, which is the reason to expect them to work, but there is no attribute
|
|
131
|
+
for line-align or position-align, so AVFoundation resolves those internally and internal resolution
|
|
132
|
+
is exactly the kind of thing that can be partial. The mapping emits nothing at all for `\an2`, omits
|
|
133
|
+
the default `,start`, and avoids `position-align` and `size:` entirely, so the common case is
|
|
134
|
+
byte-shaped like the output already known to work.
|
|
135
|
+
|
|
10
136
|
## 0.4.1
|
|
11
137
|
|
|
12
138
|
### Fixed
|
|
@@ -24,10 +150,12 @@ for the history of everything that isn't subtitle related.
|
|
|
24
150
|
This also fixes `bottomOffset: 0`, which as a bare `line:100%` put the top of the cue box at the bottom
|
|
25
151
|
edge of the video and hid the subtitle completely. It now means "sit at the bottom", matching Android.
|
|
26
152
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
153
|
+
AVFoundation honours both parts — confirmed on iOS 26.5, where the caption's bottom edge lands at the
|
|
154
|
+
requested percentage and clears a burned-in subtitle exactly as the Android side does at the same
|
|
155
|
+
value. Worth knowing for older targets, though: a WebVTT parser that predates the alignment component
|
|
156
|
+
discards the entire `line` setting rather than ignoring the component, so on such a parser captions
|
|
157
|
+
would be left unpositioned instead of top-anchored. Bottom anchoring cannot be expressed any other
|
|
158
|
+
way — it is the same setting — so confirm the offset still lands on the oldest OS you support.
|
|
31
159
|
|
|
32
160
|
- **iOS:** the cached rendition of a rewritten subtitle is keyed on the format version as well as on the
|
|
33
161
|
URL, language and offset. Without it, an install that had already rendered a given offset would be
|
package/FORK.md
CHANGED
|
@@ -20,12 +20,15 @@ Measured against the published `expo-video@56.1.4` tarball. `+`/`-` are added/re
|
|
|
20
20
|
|
|
21
21
|
| File | Lines | Purpose |
|
|
22
22
|
| --- | --- | --- |
|
|
23
|
-
| `ios/Records/SubtitleSource.swift` |
|
|
23
|
+
| `ios/Records/SubtitleSource.swift` | 61 | Record for one sideloaded subtitle; `resolvedFormat` / `isSrt` / `isAss` detection; `bottomOffset`, which lives here rather than on `SubtitleStyle` because the composition is built from a `VideoSource` alone, before any `VideoView` has supplied a style. |
|
|
24
24
|
| `ios/Records/SubtitleStyle.swift` | 118 | Record for caption styling; hex→ARGB parsing; builds `[AVTextStyleRule]`. |
|
|
25
|
-
| `ios/VideoPlayerSubtitleSideload.swift` |
|
|
26
|
-
| `
|
|
27
|
-
| `android/…/
|
|
28
|
-
| `android/…/
|
|
25
|
+
| `ios/VideoPlayerSubtitleSideload.swift` | 282 | Builds the `AVMutableComposition`, downloads remote subs, SRT→WebVTT converter, and `applyLinePosition`, which writes a WebVTT `line:N%,end` cue setting per cue for `bottomOffset` — the only way to position captions AVFoundation renders itself. **The trailing `,end` is the line alignment component, not a `line-align` setting** (there is no such setting; the six are `vertical`/`line`/`position`/`size`/`align`/`region`), and it is what anchors the percentage to the bottom edge, matching Android. A parser predating it drops the whole `line` setting, so there is no graceful degradation — verify on device. Cue timing lines are matched by **timestamp regex, not by searching for `-->`**: subtitle text may contain an arrow, and appending a cue setting to it prints the setting on screen. The temp-file hash includes `bottomOffset` *and* a `formatVersion`, joined on **NUL** — the offset so a changed value does not reuse the previous rendition, the version so an upgrade that changes the written form does not reuse the previous release's, and NUL because a hyphen occurs inside both BCP-47 tags and URLs and let two subtitles collide. |
|
|
26
|
+
| `ios/SubtitleConverterText.swift` | 566 | ASS/SSA → WebVTT, the reworked SubRip → WebVTT cue-block parser, WebVTT sanitising, and the non-UTF-8 decode ladder. Header fields are read from the script's own `Format:` lines, never assumed: encoders reorder them, and reading the Text column one position off yields a file that still parses and is entirely wrong. Override blocks are walked by a **scanner, not a list of regexes**, and the tag name is read as a run of letters — that is what stops `\c` from eating `\clip` and `\b` from eating `\bord`, and what makes `\p` require its digit so `\pos` and `\pbo` can never be mistaken for a vector drawing. Text is escaped **before** markup is inserted (`&` then `<` then `>`), which also turns a dialogue `-->` into `-->` so WebVTT cannot read a text line as a cue boundary. |
|
|
27
|
+
| `android/…/utils/SubtitleSpanFilter.kt` | 135 | Strips colour, size, family and weight from every cue of every format, keeping italic/underline/strikethrough and all positioning, plus `FilteringSubtitleParser` which applies it to whatever a delegate emits. **Load-bearing fact, verified against Media3 1.9.0 bytecode:** `SubtitleViewUtils.removeAllEmbeddedStyling` touches only `clearWindowColor`, `getText`/`setText` and `setTextSize` — never `line`, `position` or `textAlignment`. That is why the filter can live in the parser and the view flag can stay on; if an upstream sync changes it, this whole design has to be revisited. Uses an **allowlist**, because a denylist misses `TextAppearanceSpan` (colour + size + family in one span). `StyleSpan` needs a rewrite rather than a keep/drop: `BOLD_ITALIC` has to come back as plain `ITALIC` over the same range. |
|
|
28
|
+
| `android/…/records/SubtitleSource.kt` | 63 | Record → `MediaItem.SubtitleConfiguration` + MIME resolution. |
|
|
29
|
+
| `android/…/records/SubtitleStyle.kt` | 152 | Record → `CaptionStyleCompat`; hex colour parsing; font-family resolution (`res/font` → `ReactFontManager`); the `applyEmbeddedStyles` flag, which `SubtitleUtils` reads directly rather than `toCaptionStyle` and which defaults to `true` since 0.5.0. |
|
|
30
|
+
| `android/…/utils/StackingSubtitleParser.kt` | 342 | `SubtitleParser` that segments the cue timeline and merges simultaneous **unpositioned** cues into one stacked block, plus the `SubtitleParser.Factory` that routes SubRip and ASS/SSA to it and wraps every parser in `FilteringSubtitleParser`. ASS gets there through `SsaCueNormalizer` as the `prepare` step — without it `SsaParser` sets both `line` and `position` on every cue and they would all take the passthrough branch. Assigns each cue a vertical slot it keeps for its whole time on screen — a finished cue's line is held blank (`\u200B`) so the cues above it do not drop. Merges through a `SpannableStringBuilder` so cue spans survive. |
|
|
31
|
+
| `android/…/utils/SsaCueNormalizer.kt` | 93 | Clears the `line`/`position` Media3's `SsaParser` fills in on cues the ASS author never placed, so `bottomOffset` and cue stacking work on `.ass` the way they already do on `.srt`. The test is exact rather than heuristic: those values are a pure function of the cue's anchors, so matching both defaults for the bottom-centre anchors *is* "no positioning tag". Also drops events on a style the author made invisible (`Fontsize 0`, or a primary colour whose inverted alpha byte is `FF`) — that is where typesetters park working notes, and nothing downstream can carry "invisible". |
|
|
29
32
|
| `android/…/utils/RoundedSubtitleBackground.kt` | 352 | Draws `subtitleStyle.backgroundRadius` as rounded boxes, which Media3 cannot do at all: it paints `backgroundColor` as a framework `BackgroundColorSpan` and `windowColor` with `canvas.drawRect`, neither takes a radius, and `SubtitlePainter`/`CanvasSubtitleOutput` are package-private final while `SubtitleView` is final. A second `SubtitleView` is inserted behind `PlayerView`'s own, fed from `Player.Listener.onCues`, drawing a `LineBackgroundSpan` per line. **Its `EDGE_TYPE_NONE` is load-bearing** — `SubtitlePainter` copies the cue text for an edge pass and draws both layouts when the edge type is outline/raised/depressed, which would paint the box twice and darken a semi-transparent colour. State hangs off the view's `tag`, never a map in the companion object; see the `find` KDoc for why even a `WeakHashMap` leaks here. |
|
|
30
33
|
| `FORK.md`, `LICENSE` | — | This guide; MIT licence (upstream copyright preserved). |
|
|
31
34
|
| `eslint.config.js` | 4 | Verbatim copy of `expo-module-scripts/templates/eslint.config.js`. **Recreate it if a sync drops it** — `expo-module configure` treats it as an *optional* template and only syncs it when it already exists, so without this file `npm run lint` dies with "ESLint couldn't find an eslint.config.js". |
|
|
@@ -36,8 +39,8 @@ Measured against the published `expo-video@56.1.4` tarball. `+`/`-` are added/re
|
|
|
36
39
|
|
|
37
40
|
| File | Δ | What to re-apply |
|
|
38
41
|
| --- | --- | --- |
|
|
39
|
-
| `src/VideoPlayer.types.ts` | +
|
|
40
|
-
| `src/VideoView.types.ts` | +
|
|
42
|
+
| `src/VideoPlayer.types.ts` | +92 | `subtitleTracks?: SubtitleSource[]` on `VideoSourceObject`; the exported `SubtitleSource` type, including `bottomOffset` and the `'vtt' | 'srt' | 'ass' | 'ssa'` format union. |
|
|
43
|
+
| `src/VideoView.types.ts` | +123 | `subtitleStyle?: SubtitleStyle` on `VideoViewProps`; `SubtitleStyle` + `SubtitleEdgeType` types. |
|
|
41
44
|
| `src/index.ts` | +2 | Export `SubtitleStyle`, `SubtitleEdgeType` (`SubtitleSource` rides on `export type *`). |
|
|
42
45
|
| `src/VideoPlayer.web.tsx` | +5 / −1 | Hoist `JSON.stringify(source)` out of the `useMemo` dependency list into a `sourceKey` const. No behaviour change — it was already recomputed every render — but `react-hooks/use-memo` rejects non-simple expressions in a dependency array, and it was the single lint **error** in the package. |
|
|
43
46
|
| `src/ts-declarations/react-native-assets.d.ts` | +22 / −1 | Replace upstream's `../../../expo-asset/…` reference (monorepo-only) with standalone `declare module` shims. **Always needed outside the Expo monorepo.** |
|
|
@@ -47,7 +50,7 @@ Measured against the published `expo-video@56.1.4` tarball. `+`/`-` are added/re
|
|
|
47
50
|
| `ios/VideoView.swift` | +8 | `subtitleStyle` property; forward it when `player` is (re)assigned. |
|
|
48
51
|
| `ios/VideoModule.swift` | +4 | `Prop("subtitleStyle")`. |
|
|
49
52
|
| `android/…/records/VideoSource.kt` | +9 / −2 | `subtitleTracks` field; `setSubtitleConfigurations(…)` in `toMediaItem()`; subtitle key in `toMediaId()`. |
|
|
50
|
-
| `android/…/utils/SubtitleUtils.kt` | +92 / −21 | `configureSubtitleView(…, customStyle)` overload and a `styleProvider` on the captioning listener. Keeps the original behaviour when no custom style is set. Passes `context` to `toCaptionStyle` for font lookup. Drives `setApplyEmbeddedStyles` from `customStyle.applyEmbeddedStyles` (upstream hardcodes `false`). |
|
|
53
|
+
| `android/…/utils/SubtitleUtils.kt` | +92 / −21 | `configureSubtitleView(…, customStyle)` overload and a `styleProvider` on the captioning listener. Keeps the original behaviour when no custom style is set. Passes `context` to `toCaptionStyle` for font lookup. Drives `setApplyEmbeddedStyles` from `customStyle.applyEmbeddedStyles`, defaulting to `true` (upstream hardcodes `false`). The resolved value is shared with `RoundedSubtitleBackground`: `StyleSpan` is a `MetricAffectingSpan`, so italic measures wider and a disagreement draws the box too narrow. |
|
|
51
54
|
| `android/…/VideoView.kt` | +16 / −5 | `subtitleStyle` property; pass it at all 4 `configureSubtitleView` call sites + the captioning listener. |
|
|
52
55
|
| `android/…/VideoModule.kt` | +4 | `Prop("subtitleStyle")`. |
|
|
53
56
|
| `android/…/utils/DataSourceUtils.kt` | +5 | `setSubtitleParserFactory(ExpoVideoSubtitleParserFactory())` on the `DefaultMediaSourceFactory`. Note this file declares `package expo.modules.video` despite living in `utils/`, so the factory needs an explicit import. |
|
|
@@ -182,7 +185,8 @@ npm pack --dry-run # expect build/, plugin/build/, LICENSE, native sources
|
|
|
182
185
|
# NO dev config (.prettierrc, eslint.config.js, *.tsbuildinfo) — see §1
|
|
183
186
|
```
|
|
184
187
|
|
|
185
|
-
The file count is a blunt but effective regression check: **
|
|
188
|
+
The file count is a blunt but effective regression check: **257 files** as of 0.5.0 (255 at 0.4.1, plus
|
|
189
|
+
`ios/SubtitleConverterText.swift` and `android/…/utils/SubtitleSpanFilter.kt`; 254 at 0.2.2, plus
|
|
186
190
|
`RoundedSubtitleBackground.kt` in 0.3.0). If a change moves
|
|
187
191
|
it, diff the list against the previous release and confirm every difference is intended —
|
|
188
192
|
`npm pack --dry-run --json` gives a machine-readable list. Both directions matter: an unexpected
|
|
@@ -197,33 +201,57 @@ the config went missing than that the code is genuinely misformatted — check t
|
|
|
197
201
|
|
|
198
202
|
### iOS — running the subtitle converter without a host app
|
|
199
203
|
|
|
200
|
-
`
|
|
201
|
-
|
|
202
|
-
`applyLinePosition` are pure string transforms whose failure mode is a silently malformed cue rather
|
|
203
|
-
than a crash — a cue setting appended to a *text* line prints on screen, and a percentage outside
|
|
204
|
-
`0`–`100` can cost the whole cue.
|
|
205
|
-
|
|
206
|
-
Extract the shipped source rather than retyping it, so the test cannot drift from what ships:
|
|
204
|
+
The converter imports only `Foundation`, so it compiles and runs without a host app. There is a
|
|
205
|
+
script for it:
|
|
207
206
|
|
|
208
207
|
```bash
|
|
209
|
-
|
|
210
|
-
> /tmp/swiftcheck/Converter.swift
|
|
211
|
-
sed -i '' '1i\
|
|
212
|
-
import Foundation
|
|
213
|
-
' /tmp/swiftcheck/Converter.swift
|
|
214
|
-
# Assertions go in main.swift — Swift allows top-level code only in a file with that name.
|
|
215
|
-
swiftc -o converttest Converter.swift main.swift && ./converttest
|
|
208
|
+
./scripts/verify-subtitle-converter.sh
|
|
216
209
|
```
|
|
217
210
|
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
211
|
+
It extracts the shipped source rather than copying it — `awk` for the `SubtitleConverter` enum out of
|
|
212
|
+
`ios/VideoPlayerSubtitleSideload.swift`, plus `ios/SubtitleConverterText.swift` with its
|
|
213
|
+
`ExpoModulesCore` import stripped and `log` stubbed — so the assertions cannot drift from what ships.
|
|
214
|
+
87 of them live in `scripts/subtitle-converter-checks.swift`.
|
|
215
|
+
|
|
216
|
+
**`scripts/` must stay out of `ios/`.** The podspec globs `ios/**/*.{h,m,swift}`, so a checks file
|
|
217
|
+
with top-level code placed under `ios/` would be compiled into every consumer's pod and fail their
|
|
218
|
+
build. It is also listed in `.npmignore`.
|
|
219
|
+
|
|
220
|
+
This is worth exercising on every change, because everything it covers is a pure string transform
|
|
221
|
+
whose failure mode is a silently malformed cue rather than a crash: a cue setting appended to a
|
|
222
|
+
*text* line prints on screen, a percentage outside `0`–`100` can cost the whole cue, and a
|
|
223
|
+
mis-scanned override block can eat the dialogue around it.
|
|
224
|
+
|
|
225
|
+
Cases the harness keeps covered: all nine `\an` values and the legacy `\a`; `\pos` scaling against
|
|
226
|
+
`PlayResX`/`PlayResY`, including the 4:3 derivation when only one axis is declared; the full
|
|
227
|
+
drop-list of appearance tags; longest-match (`\clip` is not `\c`, `\bord` is not `\b`); whole-event
|
|
228
|
+
drops for `Comment:`, `{\p1}` drawings and tags-only lines; escaping order, including a dialogue
|
|
229
|
+
`-->`; commas inside the ASS Text column; a reordered `[Events] Format:` line; the SubRip path with a
|
|
230
|
+
leaked `{\an8}`, `<font color>` and `<b>`; WebVTT `<c.…>` and STYLE-block sanitising; the
|
|
231
|
+
`applyLinePosition` interaction for positioned, bottom-centre and side-only cues; and the encoding
|
|
232
|
+
ladder.
|
|
222
233
|
|
|
223
234
|
**A passing harness does not mean the cue renders where you asked.** It checks the bytes written, not
|
|
224
|
-
whether AVFoundation honours them.
|
|
225
|
-
|
|
226
|
-
|
|
235
|
+
whether AVFoundation honours them. That is the part of this package with no automated coverage at all
|
|
236
|
+
— confirm it against a real player with a *two-line* cue, which is the case the alignment component
|
|
237
|
+
exists for.
|
|
238
|
+
|
|
239
|
+
Baseline: as of 0.4.1, AVFoundation honours `line:` **and** the `,end` alignment component on iOS 26.5,
|
|
240
|
+
placing the caption's bottom edge at the requested percentage.
|
|
241
|
+
|
|
242
|
+
As of 0.5.0 the ASS mapping newly relies on **`position:`, `align:` and the `,center` line alignment,
|
|
243
|
+
and none of the three is confirmed on device.** CoreMedia exposes a matching text-markup attribute for
|
|
244
|
+
each WebVTT cue setting, which is the reason to expect them to work, but there is no attribute for
|
|
245
|
+
line-align or position-align — AVFoundation resolves those internally, and internal resolution is
|
|
246
|
+
exactly the kind of thing that can be partial. Verify `\an5` (`,center`) and `\an7`
|
|
247
|
+
(`position:` + `align:`) on the oldest supported OS as well as the newest; the deployment target is
|
|
248
|
+
16.4. If they turn out unsupported, the fallback is to keep the vertical mapping and drop the
|
|
249
|
+
horizontal one, which puts `\an7`/`\an8`/`\an9` all at top-centre and still delivers most of the
|
|
250
|
+
value.
|
|
251
|
+
|
|
252
|
+
Re-confirm all of this after an upstream sync or when adding support for an older deployment target —
|
|
253
|
+
the failure is silent, and for `line:` it is total rather than partial (a parser that rejects the
|
|
254
|
+
alignment component drops the whole setting).
|
|
227
255
|
|
|
228
256
|
### Android — compiling the parser without a host app
|
|
229
257
|
|
package/README.md
CHANGED
|
@@ -60,6 +60,7 @@ const player = useVideoPlayer({
|
|
|
60
60
|
subtitleTracks: [
|
|
61
61
|
{ uri: 'https://cdn.example.com/en.vtt', language: 'en', label: 'English', default: true },
|
|
62
62
|
{ uri: 'https://cdn.example.com/id.srt', language: 'id', label: 'Indonesia', format: 'srt' },
|
|
63
|
+
{ uri: 'https://cdn.example.com/signs.ass', language: 'ja', label: 'Signs', format: 'ass' },
|
|
63
64
|
],
|
|
64
65
|
});
|
|
65
66
|
|
|
@@ -67,12 +68,14 @@ const player = useVideoPlayer({
|
|
|
67
68
|
player.subtitleTrack = player.availableSubtitleTracks[0];
|
|
68
69
|
```
|
|
69
70
|
|
|
70
|
-
`SubtitleSource` fields: `uri` (local `file://` or remote `http(s)://`), `language` (BCP-47 / ISO 639), optional `label`, optional `format` (`'vtt' | 'srt'`, inferred from the extension when omitted), optional `default`, and optional `bottomOffset` (iOS — see [Moving subtitles off burned-in captions](#moving-subtitles-off-burned-in-captions)).
|
|
71
|
+
`SubtitleSource` fields: `uri` (local `file://` or remote `http(s)://`), `language` (BCP-47 / ISO 639), optional `label`, optional `format` (`'vtt' | 'srt' | 'ass' | 'ssa'`, inferred from the extension when omitted), optional `default`, and optional `bottomOffset` (iOS — see [Moving subtitles off burned-in captions](#moving-subtitles-off-burned-in-captions)).
|
|
72
|
+
|
|
73
|
+
Pass `format` whenever you know it. A URL that serves the file without an extension — a Directus asset at `/assets/<uuid>`, say — leaves nothing to infer from, and the WebVTT default would decode a SubRip or ASS file to zero cues in silence.
|
|
71
74
|
|
|
72
75
|
| | Android | iOS |
|
|
73
76
|
| --- | --- | --- |
|
|
74
77
|
| Mechanism | `MediaItem.SubtitleConfiguration` (Media3 merges into a `MergingMediaSource`) | `AVMutableComposition` (copies the video/audio tracks and inserts a `.text` track) |
|
|
75
|
-
| Formats | VTT, SRT, TTML natively | VTT natively; **SRT
|
|
78
|
+
| Formats | VTT, SRT, ASS/SSA, TTML natively | VTT natively; **SRT and ASS/SSA are auto-converted to VTT**; remote files are **downloaded to a local cache first** |
|
|
76
79
|
|
|
77
80
|
> **iOS limitations:** sideloading only runs on the asynchronous loading path (the default `VideoPlayer` constructor and `replaceAsync`, not the synchronous `replace`), and only for progressive (non-HLS, non-DRM) sources. HLS/DASH already carry their own subtitle renditions.
|
|
78
81
|
|
|
@@ -145,12 +148,20 @@ That last row is worth knowing: subtitles carrying their own placement — SubRi
|
|
|
145
148
|
cues with a `line:` setting — were positioned deliberately by whoever wrote them, so neither platform
|
|
146
149
|
moves them.
|
|
147
150
|
|
|
151
|
+
**ASS/SSA is the case where the two platforms diverge.** Media3's `SsaParser` gives *every* cue an
|
|
152
|
+
explicit line, so `subtitleStyle.bottomOffset` moves nothing on an ASS track on Android. On iOS the
|
|
153
|
+
converter deliberately emits no `line:` for bottom-centre cues (`\an2`, the default for dialogue),
|
|
154
|
+
so ordinary dialogue still shifts while genuine `\an8` signs stay put. Fixing the Android side means
|
|
155
|
+
a custom SSA parser that can tell a style default from a real `\pos`; it is not in this release.
|
|
156
|
+
|
|
148
157
|
The iOS cue setting is written as `line:N%,end`. The trailing `,end` is WebVTT's line alignment
|
|
149
158
|
component, and it is what makes the percentage refer to the bottom edge of the cue box — so the two
|
|
150
|
-
platforms anchor the same way and one value is correct on both.
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
159
|
+
platforms anchor the same way and one value is correct on both. AVFoundation honours both parts,
|
|
160
|
+
confirmed on iOS 26.5.
|
|
161
|
+
|
|
162
|
+
For older targets, note that a WebVTT parser predating that component discards the whole `line` setting
|
|
163
|
+
rather than ignoring the component, which would leave captions unpositioned; there is no form that
|
|
164
|
+
degrades more gently, since both behaviours come from the same setting.
|
|
154
165
|
|
|
155
166
|
## Styling subtitles
|
|
156
167
|
|
|
@@ -170,7 +181,7 @@ Pass a `subtitleStyle` prop to `VideoView`. It applies to whichever subtitle tra
|
|
|
170
181
|
edgeType: 'outline', // 'none' | 'outline' | 'dropShadow' | 'raised' | 'depressed'
|
|
171
182
|
edgeColor: '#000000',
|
|
172
183
|
bottomOffset: 0.1, // Android only — fraction of the view height (0–1)
|
|
173
|
-
applyEmbeddedStyles:
|
|
184
|
+
applyEmbeddedStyles: true, // Android only — keeps the track's italic/underline; default true
|
|
174
185
|
}}
|
|
175
186
|
/>
|
|
176
187
|
```
|
|
@@ -205,21 +216,34 @@ the region the caption occupies, the background is the box that hugs the text.
|
|
|
205
216
|
|
|
206
217
|
### Inline formatting from the track
|
|
207
218
|
|
|
208
|
-
Subtitle files carry formatting of their own
|
|
209
|
-
|
|
210
|
-
|
|
219
|
+
Subtitle files carry formatting of their own: `<b>`, `<i>`, `<u>` and `<font color>` in `SRT`, `<c.yellow>`
|
|
220
|
+
and `<b>` in `WebVTT`, and a whole styling language in `ASS`. The module splits it in two.
|
|
221
|
+
|
|
222
|
+
**Colour, font size, font family and weight are always discarded, on both platforms.** They are removed
|
|
223
|
+
while the file is being parsed, before any view exists, so `subtitleStyle` is the single thing deciding
|
|
224
|
+
how captions look. There is no way to opt back into a track's own colours — a subtitle shipping yellow
|
|
225
|
+
72pt bold text cannot override the size your user picked.
|
|
226
|
+
|
|
227
|
+
**Italic, underline and strikethrough are kept**, and `applyEmbeddedStyles: false` is the blunt switch
|
|
228
|
+
that drops them too:
|
|
211
229
|
|
|
212
230
|
```tsx
|
|
213
|
-
<VideoView player={player} subtitleStyle={{ textColor: '#FFFFFF', applyEmbeddedStyles:
|
|
231
|
+
<VideoView player={player} subtitleStyle={{ textColor: '#FFFFFF', applyEmbeddedStyles: false }} />
|
|
214
232
|
```
|
|
215
233
|
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
234
|
+
A track's own placement is unaffected either way, so an `\an8` sign still sits at the top.
|
|
235
|
+
|
|
236
|
+
Two asymmetries worth knowing:
|
|
237
|
+
|
|
238
|
+
| | Android | iOS |
|
|
239
|
+
| --- | --- | --- |
|
|
240
|
+
| Strikethrough | renders | **cannot render** — WebVTT has no strikethrough tag and CoreMedia has no attribute for one |
|
|
241
|
+
| Italic/underline in ASS | from the `[V4+ Styles]` `Style:` line only | from the style line **and** inline `{\i1}` |
|
|
242
|
+
| `applyEmbeddedStyles` | honoured | ignored; italic and underline always render |
|
|
220
243
|
|
|
221
|
-
|
|
222
|
-
|
|
244
|
+
The Android ASS limitation is Media3's: `SsaStyle.Overrides` parses only `\an`, `\pos` and `\move`,
|
|
245
|
+
then `stripStyleOverrides` deletes every remaining `{...}` block, so an inline `{\i1}` never reaches
|
|
246
|
+
the renderer.
|
|
223
247
|
|
|
224
248
|
### Fonts
|
|
225
249
|
|
|
@@ -37,14 +37,24 @@ class SubtitleSource(
|
|
|
37
37
|
return builder.build()
|
|
38
38
|
}
|
|
39
39
|
|
|
40
|
+
/**
|
|
41
|
+
* ASS and SSA both map to [MimeTypes.TEXT_SSA] ("text/x-ssa"), which Media3's
|
|
42
|
+
* `DefaultSubtitleParserFactory` already accepts, so no parser of our own is
|
|
43
|
+
* needed for them. The `else` branch defaults to WebVTT, which means an
|
|
44
|
+
* unrecognised extension decodes to zero cues rather than erroring: the
|
|
45
|
+
* caller should pass [format] whenever it knows, and a Directus asset URL
|
|
46
|
+
* (`/assets/<uuid>`) carries no extension to fall back on.
|
|
47
|
+
*/
|
|
40
48
|
private fun resolveMimeType(uri: Uri): String {
|
|
41
49
|
return when (format?.lowercase()) {
|
|
42
|
-
"vtt" -> MimeTypes.TEXT_VTT
|
|
43
|
-
"srt" -> MimeTypes.APPLICATION_SUBRIP
|
|
50
|
+
"vtt", "webvtt" -> MimeTypes.TEXT_VTT
|
|
51
|
+
"srt", "subrip" -> MimeTypes.APPLICATION_SUBRIP
|
|
52
|
+
"ass", "ssa" -> MimeTypes.TEXT_SSA
|
|
44
53
|
else -> {
|
|
45
54
|
val path = uri.toString().substringBefore('?').lowercase()
|
|
46
55
|
when {
|
|
47
56
|
path.endsWith(".srt") -> MimeTypes.APPLICATION_SUBRIP
|
|
57
|
+
path.endsWith(".ass") || path.endsWith(".ssa") -> MimeTypes.TEXT_SSA
|
|
48
58
|
path.endsWith(".ttml") || path.endsWith(".dfxp") -> MimeTypes.APPLICATION_TTML
|
|
49
59
|
else -> MimeTypes.TEXT_VTT
|
|
50
60
|
}
|
|
@@ -35,11 +35,16 @@ class SubtitleStyle(
|
|
|
35
35
|
*/
|
|
36
36
|
@Field val backgroundRadius: Float? = null,
|
|
37
37
|
/**
|
|
38
|
-
* Whether the
|
|
39
|
-
* [expo.modules.video.utils.SubtitleUtils.configureSubtitleView] rather than
|
|
40
|
-
* because it maps to a [androidx.media3.ui.SubtitleView] flag and not to a
|
|
38
|
+
* Whether the italic, underline and strikethrough carried by the subtitle track itself are kept.
|
|
39
|
+
* Consumed by [expo.modules.video.utils.SubtitleUtils.configureSubtitleView] rather than
|
|
40
|
+
* [toCaptionStyle], because it maps to a [androidx.media3.ui.SubtitleView] flag and not to a
|
|
41
|
+
* caption style property.
|
|
42
|
+
*
|
|
43
|
+
* Colour, font size, font family and weight are never taken from the track regardless of this
|
|
44
|
+
* flag: [expo.modules.video.utils.SubtitleSpanFilter] strips them in the parser, before any view
|
|
45
|
+
* exists. That is what makes `true` a safe default here, which it was not before.
|
|
41
46
|
*/
|
|
42
|
-
@Field val applyEmbeddedStyles: Boolean =
|
|
47
|
+
@Field val applyEmbeddedStyles: Boolean = true
|
|
43
48
|
) : Record, Serializable {
|
|
44
49
|
/**
|
|
45
50
|
* Builds a [CaptionStyleCompat], falling back to [base] for any unspecified property so that
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
package expo.modules.video.utils
|
|
2
|
+
|
|
3
|
+
import android.graphics.Color
|
|
4
|
+
import android.text.Spanned
|
|
5
|
+
import android.text.style.ForegroundColorSpan
|
|
6
|
+
import androidx.annotation.OptIn
|
|
7
|
+
import androidx.media3.common.text.Cue
|
|
8
|
+
import androidx.media3.common.util.UnstableApi
|
|
9
|
+
import kotlin.math.abs
|
|
10
|
+
|
|
11
|
+
/**
|
|
12
|
+
* Undoes two things Media3's `SsaParser` does to every ASS cue, so the rest of this package can
|
|
13
|
+
* treat an `.ass` track the way it treats every other format.
|
|
14
|
+
*
|
|
15
|
+
* ### Dialogue gets its position taken back off
|
|
16
|
+
*
|
|
17
|
+
* `SsaParser.createCue` ends with an `else` branch that runs for every cue *without* a `\pos` or
|
|
18
|
+
* `\move`: it calls `setPosition` and `setLine` with values derived purely from the cue's
|
|
19
|
+
* alignment. So an ordinary bottom-centre dialogue line — the overwhelming majority of a subtitle
|
|
20
|
+
* file — arrives carrying `line = 0.95` and `position = 0.5` even though the author placed nothing.
|
|
21
|
+
*
|
|
22
|
+
* Two features break on that. `SubtitlePainter` applies `bottomPaddingFraction` only to cues whose
|
|
23
|
+
* `line` is unset, so `SubtitleSource.bottomOffset` (the app's `subtitle_offset_percent`, used to
|
|
24
|
+
* lift softsubs clear of burnt-in hardsubs) silently stops moving anything. And
|
|
25
|
+
* [StackingSubtitleParser] treats a positioned cue as deliberately placed and passes it through
|
|
26
|
+
* unstacked, so two simultaneous speakers draw on top of each other.
|
|
27
|
+
*
|
|
28
|
+
* Recognising "the parser filled this in" is exact rather than a guess: the values are a pure
|
|
29
|
+
* function of the anchors, so a cue matching *both* defaults for the bottom-centre anchors is
|
|
30
|
+
* precisely a cue that had no positioning tag. Clearing them puts an ASS dialogue cue in the same
|
|
31
|
+
* shape as a SubRip one, which is the shape both features already work on.
|
|
32
|
+
*
|
|
33
|
+
* A cue the author did place — `\an8`, `\pos`, a style aligned anywhere but bottom-centre — fails
|
|
34
|
+
* the check and keeps everything. That is the signs-and-typesetting case, and it must not move.
|
|
35
|
+
*
|
|
36
|
+
* ### Cues the author made invisible are dropped
|
|
37
|
+
*
|
|
38
|
+
* Typesetters keep their working notes in the script on a style that renders nothing: JJK S3E3
|
|
39
|
+
* ships `Style: f0ld3r-invisible` with `Fontsize 0` and every colour `&HFF……`, holding 158 events
|
|
40
|
+
* of production notes ("--mahito", "#1", "check in frame 6212").
|
|
41
|
+
*
|
|
42
|
+
* libass honours both signals and shows nothing. Media3 turns the fully transparent primary colour
|
|
43
|
+
* into a `ForegroundColorSpan` with alpha 0 — and [SubtitleSpanFilter] then strips colour spans,
|
|
44
|
+
* because the app owns the subtitle colour. The note would come back in the viewer's own colour, at
|
|
45
|
+
* the viewer's own size, on screen. Dropping the cue here, before any of that, is the only place
|
|
46
|
+
* the alpha still means what the author meant.
|
|
47
|
+
*/
|
|
48
|
+
@OptIn(UnstableApi::class)
|
|
49
|
+
internal object SsaCueNormalizer {
|
|
50
|
+
/** `SsaParser.DEFAULT_MARGIN`. Kept in sync by [sitsWhereTheParserPutIt]'s test. */
|
|
51
|
+
private const val DEFAULT_MARGIN = 0.05f
|
|
52
|
+
|
|
53
|
+
/** The values are computed, not parsed, so they compare exactly; this only guards float drift. */
|
|
54
|
+
private const val EPSILON = 0.0001f
|
|
55
|
+
|
|
56
|
+
fun normalize(cue: Cue): Cue? {
|
|
57
|
+
if (isInvisible(cue)) return null
|
|
58
|
+
if (!sitsWhereTheParserPutIt(cue)) return cue
|
|
59
|
+
|
|
60
|
+
return cue.buildUpon()
|
|
61
|
+
.setLine(Cue.DIMEN_UNSET, Cue.LINE_TYPE_FRACTION)
|
|
62
|
+
.setPosition(Cue.DIMEN_UNSET)
|
|
63
|
+
.build()
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
/**
|
|
67
|
+
* True when the cue's placement is exactly what `SsaParser` computes for the bottom-centre
|
|
68
|
+
* anchors, which is to say the author placed nothing.
|
|
69
|
+
*/
|
|
70
|
+
private fun sitsWhereTheParserPutIt(cue: Cue): Boolean =
|
|
71
|
+
cue.lineType == Cue.LINE_TYPE_FRACTION &&
|
|
72
|
+
cue.lineAnchor == Cue.ANCHOR_TYPE_END &&
|
|
73
|
+
cue.positionAnchor == Cue.ANCHOR_TYPE_MIDDLE &&
|
|
74
|
+
abs(cue.line - (1f - DEFAULT_MARGIN)) < EPSILON &&
|
|
75
|
+
abs(cue.position - 0.5f) < EPSILON
|
|
76
|
+
|
|
77
|
+
/**
|
|
78
|
+
* Either of the two ways an ASS style says "draw nothing": a fully transparent primary colour, or
|
|
79
|
+
* a zero font size. Both have to cover the whole cue — a transparent run *inside* a line is a
|
|
80
|
+
* typesetting effect, not a hidden line.
|
|
81
|
+
*/
|
|
82
|
+
private fun isInvisible(cue: Cue): Boolean {
|
|
83
|
+
val text = cue.text ?: return false
|
|
84
|
+
if (cue.textSize == 0f) return true
|
|
85
|
+
if (text !is Spanned || text.isEmpty()) return false
|
|
86
|
+
|
|
87
|
+
return text.getSpans(0, text.length, ForegroundColorSpan::class.java).any { span ->
|
|
88
|
+
Color.alpha(span.foregroundColor) == 0 &&
|
|
89
|
+
text.getSpanStart(span) == 0 &&
|
|
90
|
+
text.getSpanEnd(span) == text.length
|
|
91
|
+
}
|
|
92
|
+
}
|
|
93
|
+
}
|