easycaptionkit 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -5
- package/package.json +1 -1
- package/skills/easycaptionkit/SKILL.md +8 -3
- package/skills/easycaptionkit/scripts/easycaptionkit.js +12 -5
- package/skills/easycaptionkit/scripts/lib/boxes.js +139 -0
- package/skills/easycaptionkit/scripts/lib/caption.js +71 -40
- package/skills/easycaptionkit/scripts/lib/chrome.js +94 -42
- package/skills/easycaptionkit/scripts/lib/ffmpeg.js +97 -91
- package/skills/easycaptionkit/scripts/lib/look.js +1 -1
- package/skills/easycaptionkit/scripts/lib/programs.js +478 -0
- package/skills/easycaptionkit/scripts/lib/render.js +26 -7
- package/skills/easycaptionkit/scripts/lib/report.js +85 -21
- package/skills/easycaptionkit/scripts/lib/subtitles.js +47 -18
- package/skills/easycaptionkit/scripts/lib/version.js +1 -1
- package/skills/easycaptionkit/scripts/lib/words.js +8 -1
package/README.md
CHANGED
|
@@ -29,7 +29,7 @@ easycaptionkit-demo/
|
|
|
29
29
|
```
|
|
30
30
|
|
|
31
31
|
- Three styles: `karaoke` shows each caption whole and lights the word being spoken, `line` shows each caption whole in one color, `word` shows one word at a time, lit.
|
|
32
|
-
- With easyfontkit's `fonts.css`, the captions are drawn with the site's own font files, the ones in its folder, its heading font first. Chrome reports which font drew each letter, and CAPTIONS.md says so: a Japanese or Arabic caption over a Latin-only brand font is drawn partly in a system font, and the file tells you how much.
|
|
32
|
+
- With easyfontkit's `fonts.css`, the captions are drawn with the site's own font files, the ones in its folder, its heading font first. Chrome reports which font drew each letter, and CAPTIONS.md says so: a Japanese or Arabic caption over a Latin-only brand font is drawn partly in a system font, and the file tells you how much. A letter Chrome finds no font for on the machine is drawn as a box, as Japanese is on a server with no Japanese font; on Linux, where fontconfig can tell, CAPTIONS.md names those letters and says to install a font that has them, such as Noto Sans CJK, or to give `--fonts` an easyfontkit folder whose font has them.
|
|
33
33
|
- With an easybrandkit folder, the spoken word is lit in the brand's button, link or logo color. A color too dark to read as letters on the outline (a deep green, a navy) lights the word on a rounded patch of it instead. Without `--fonts` or `--font`, the brand's heading font is asked for by name, which this machine may not have; CAPTIONS.md says which font drew the letters wherever Chrome or fontconfig can tell.
|
|
34
34
|
- Captions are cut at sentence ends, commas and pauses, into at most two lines of what fits the safe width, up to 42 characters, by default (`--lines` and `--max-chars` change both), and held a little longer when they would flash by. In Japanese no caption or line starts with a particle or a small kana, or splits a word, when another break fits.
|
|
35
35
|
- The sound is copied as it is, so its loudness does not change. Sound that MP4 players do not take, like a camera's PCM in a MOV, becomes AAC at the same rate.
|
|
@@ -37,6 +37,17 @@ easycaptionkit-demo/
|
|
|
37
37
|
- The captions run as long as the video's sound where the sound runs on past its picture, in the subtitle files too, and as long as its frames where it states no length, as a WebM a browser recorded does. A file whose frames and sound all end well before the length it states, as an interrupted download leaves it, is captioned as far as its frames go, and CAPTIONS.md says so.
|
|
38
38
|
- CAPTIONS.md carries the license facts easyfontkit read for the font that was burned in, and the closest free font when it is not open. Whether to use it is your call. When the font has none of the letters, as a Latin font has no Japanese, CAPTIONS.md says that no letter of it is in the video and names the fonts of the machine that drew them.
|
|
39
39
|
|
|
40
|
+
## Requirements
|
|
41
|
+
|
|
42
|
+
- Node.js 22 or newer.
|
|
43
|
+
- ffmpeg and ffprobe 4.4 or newer, Ubuntu 22.04's 4.4.2 included. To burn the captions, ffmpeg needs an H.264 encoder: libx264, the one every test has used, or VideoToolbox, OpenH264 or Media Foundation, which no test has run with. `--libass` needs it built with libass as well, which Homebrew's default ffmpeg is not; `subtitles` needs neither, and with `--video` it uses ffprobe alone. Apart from libass's ass filter, the filters it uses are ffmpeg's own, and so is the AAC encoder. `brew install ffmpeg` and `winget install ffmpeg` give a new enough one, and so does `sudo apt install ffmpeg` on Ubuntu 22.04, Debian 12 or newer. Set `FFMPEG_PATH` to use another build, as a path or a name on your PATH, and `FFPROBE_PATH` for an ffprobe that is not next to ffmpeg or on your PATH. Before any work, a run checks that ffmpeg and ffprobe start and answer as ffmpeg and ffprobe, ffprobe alone in `subtitles --video`; one that does not stops the run with what the system or the program said, and how to get one. When the system is short of open files, processes or memory to start one, the run says so and what to do. Then, before it draws anything, a run that burns captions checks ffmpeg's version number, its H.264 encoder and, for `--libass`, libass, and says what is missing; an ffmpeg that fails or is stopped when asked for its encoders or its filters stops the run with how it ended. An ffmpeg or ffprobe that gives no answer in two minutes when asked, as on a machine short of memory, stops the run too, and the message says to run it again. It checks no version number of ffprobe's.
|
|
44
|
+
- Google Chrome or Chromium, which draws the captions; `subtitles` starts none. Chrome runs in its sandbox unless easycaptionkit runs as root, where the sandbox does not start: on Linux, Google Chrome's own package sets the sandbox up, and a container has to let it start, as Docker does with `--security-opt seccomp=unconfined`. It has been tested with Google Chrome only: 155 on Ubuntu 26.04, 157 on Ubuntu 22.04 and the ones GitHub's ubuntu-22.04, ubuntu-24.04 and ubuntu-latest runners come with. No test has run with Chromium, or with Edge or Brave, which easycaptionkit also takes. It looks for all four on your PATH and where macOS installs them; on Windows it looks only where Chrome and Edge install themselves. Set `CHROME_PATH` to use another build, as a path or a name on your PATH. Before any work, a run that draws with Chrome checks that it starts and gives a version number when asked with `--version`, except on Windows, where Chrome prints none, and one that does not stops the run in the same way. With `--libass` ffmpeg draws the captions instead, and a run with no browser, or with one that does not start or gives no version number, makes plainer sheet.png and safe-zones.png.
|
|
45
|
+
- A font that has the letters of your captions, on the machine or in `--fonts`. A letter no font has is drawn as a box, as Japanese is on a server with no Japanese font; `fonts-noto-cjk` gives Debian and Ubuntu one.
|
|
46
|
+
- Linux, where it is tested: on Ubuntu 22.04 with its ffmpeg 4.4.2, on Ubuntu 24.04 with its ffmpeg 6.1 and on GitHub's ubuntu-latest runner, on every push and before every release. This release has also run on Ubuntu 26.04 with ffmpeg 8.0. It is written for macOS and Windows as well, but it has not been run on either.
|
|
47
|
+
- Room on the disk for the files of a run and those of the run before it, as the earlier files stay until the new ones are in place. captioned.mp4 is a new encode of the whole video and can come out larger than the video given. Until a run ends, it also keeps the pictures Chrome drew of the captions, a PNG of each state and an animated PNG of them all, in its hidden folder in the out folder. In the karaoke style, at the default letter size and two lines, they take about 75 KB a word on a 1080x1920 video, 150 KB on a 1920x1080 one and 360 KB on a 3840x2160 one, some 1.5 GB for an hour of speech at 1920x1080. With the largest letters and the most lines the options allow, `--size 20` and `--lines 3`, they take about twice as much, and in the line and word styles and with `--box` less. Each Chrome that easycaptionkit starts keeps a profile of 3 to 7 MB in the temp folder.
|
|
48
|
+
- Memory for ffmpeg's encode, more for a video of more pixels, more again for an HDR one, which stays in 10 bits, and on a machine with more cores. For a 1920x1080 or 1080x1920 video, ffmpeg took about 0.6 GB on 2 cores and 1.5 GB on 24, as much for ten minutes of video as for 30 seconds, and 1 GB and 2.4 GB for one in HDR. For a 3840x2160 video it took about 2 GB on 2 cores and 5 GB on 24, and 3.6 GB and 8.6 GB for one in HDR. The cores are the ones ffmpeg may run on, and a CPU quota does not lower their count: in a systemd scope held to 2 CPUs by CPUQuota=200% on a 24 core machine, where `nproc` said 2, the encode of a 1920x1080 video took as much as on all 24 cores, and 0.6 GB once `taskset` held it to 2 cores. A container held to 2 CPUs by `docker run --cpus=2` or by a Kubernetes CPU limit is held by such a quota, while `docker run --cpuset-cpus` holds it to cores, as taskset does. Chrome takes about 0.3 GB while it draws, before the encode and after it, and Node.js about 0.1 GB.
|
|
49
|
+
- A run needs no network access and no API key: easycaptionkit reads files on your disk, the browser runs offline with the font files there, and nothing leaves your machine. Installing it needs the network, and `npx easycaptionkit@latest` asks npm for the newest version each time it runs. There are no npm dependencies, and `subtitles` without `--video` needs Node.js alone.
|
|
50
|
+
|
|
40
51
|
## Use it with your agent
|
|
41
52
|
|
|
42
53
|
easycaptionkit ships as an [Agent Skill](https://agentskills.io). Install it once and your agent runs it by itself when a video needs captions:
|
|
@@ -96,7 +107,7 @@ npx easycaptionkit@latest styles [--json]
|
|
|
96
107
|
|
|
97
108
|
easycaptionkit reads files, not addresses: a video or words given as `https://...` is refused, so download it first. `--brand` and `--fonts` take what easybrandkit and easyfontkit make from a site, so a site's address there is refused with words that name the kit to make it with.
|
|
98
109
|
|
|
99
|
-
`subtitles` writes captions.srt, captions.vtt and captions.ass from the words alone, with words.json, captions.json and CAPTIONS.md, and changes no video; `--video` gives it a video's size and length. captions.ass is laid out for that frame, its lines cut to what fits across it at the letter size, so it can have more captions than captions.srt and captions.vtt, whose lines take up to 42 characters unless `--max-chars` gives another number. `styles` lists the styles, the options and every place's margins with their sources.
|
|
110
|
+
`subtitles` writes captions.srt, captions.vtt and captions.ass from the words alone, with words.json, captions.json and CAPTIONS.md, and changes no video; `--video` gives it a video's size and length, and without it a demo's captions end where its steps.json says the video does. captions.ass is laid out for that frame, its lines cut to what fits across it at the letter size, so it can have more captions than captions.srt and captions.vtt, whose lines take up to 42 characters unless `--max-chars` gives another number. `styles` lists the styles, the options and every place's margins with their sources.
|
|
100
111
|
|
|
101
112
|
Without `--out`, a run goes to `easycaptionkit-<video name>`, or to `easycaptionkit-<video name>-2` and on when that folder holds the captions of another video by the same name, such as a second `demo.mp4`. A subtitles run goes to `easycaptionkit-subtitles-<video name>`, or `easycaptionkit-subtitles-<words name>` without `--video`, with `-2` and on in the same way. The name is the file's in lower case, its letters and digits in any script with a dash for anything else, and accents taken off Latin letters only; a name longer than 60 letters or 180 bytes is cut and ends in a short hash of the whole name. A run replaces the files an earlier run wrote in its folder, says so when that run was made from another file, and leaves anything else there alone. Each run's captions.json and words.json name the other files it wrote, with their sizes and SHA-256. A run stops before any work when the folder holds a file by one of the names it writes that they do not name, or that has changed since, such as another tool's words.json, a sheet.png of your own or a link, or one of the files the run reads, and says to pick another `--out`. A subtitles run also stops at a folder that holds the captioned video of an earlier run, which its new files would no longer match. The one file read and written over is a words.json from an earlier run, fixed by hand; if the edit breaks the JSON, the run says at which line and column. Once its words are fixed, a run that reads other words, such as the first command again, stops rather than replace them, and says how to keep them or start over. A file that comes in by one of the names while a run is going, or one there whose content changes, stays as it is: the run leaves the folder as it was and puts its own files in `easycaptionkit-new` inside it.
|
|
102
113
|
|
|
@@ -111,8 +122,6 @@ npx easycaptionkit@latest subtitles transcript.json --video film.mp4
|
|
|
111
122
|
|
|
112
123
|
Installed globally (`npm install -g easycaptionkit`), the command is `easycaptionkit`. npx keeps the copy an earlier run fetched, so `npx easycaptionkit@latest` is how you get a newer version.
|
|
113
124
|
|
|
114
|
-
It needs Node 22 or newer, ffmpeg 4.4 or newer and Chrome, Chromium, Edge or Brave. `brew install ffmpeg` and `winget install ffmpeg` give a new enough ffmpeg, and so does `sudo apt install ffmpeg` on Ubuntu 22.04, Debian 12 or newer; the tests run on Ubuntu 22.04's ffmpeg 4.4 and on the latest Ubuntu's, before every release too. Set `FFMPEG_PATH` and `CHROME_PATH` to pick them, as a path or a name on your PATH, and `FFPROBE_PATH` for an ffprobe that is not next to ffmpeg or on your PATH. There are no npm dependencies, and nothing leaves your machine: the browser runs offline with the font files on your disk.
|
|
115
|
-
|
|
116
125
|
## Where the words come from
|
|
117
126
|
|
|
118
127
|
`--words` takes the words with their times from any of these, told apart by what is in the file, saved as UTF-8 or UTF-16:
|
|
@@ -147,7 +156,7 @@ Only ads have published margins, and apps move their buttons. Each figure carrie
|
|
|
147
156
|
4. The PNGs go into one animated PNG, each for exactly as long as its state lasts, to the millisecond. ffmpeg reads it as one stream and lays it over the video in a single encode, H.264 with x264 at CRF 18 where ffmpeg has x264, with every frame at its own time, or at a steady rate where the video has too few frames for the captions, as above. On an SDR video the caption colors are converted with the video's own color matrix, so a brand color lands on the same color. A picture stored as RGB, as PNG or QuickTime Animation in a MOV is, or coded in YCgCo, is converted to the matrix players take for its size first, and captioned.mp4 names the matrix it holds.
|
|
148
157
|
5. Writes the subtitle files, takes frames of up to 24 captions from the result for sheet.png, draws safe-zones.png, and moves everything into the folder at once. A run that fails, or is stopped, leaves the folder as it was. A run killed outright (`kill -9`, or the system when memory runs out) cannot tidy up: its hidden `.easycaptionkit-work-` folder stays in the out folder, where an ffmpeg it started can go on writing captioned.mp4 to the end, and the Chrome profile it was drawing with stays in the temporary folder. The next run into that folder removes the work folder, and a later run of the same user that starts Chrome removes the profile once no process names it and it has not changed for a day, except on Windows, where it stays in the temporary folder until Windows or you clear it.
|
|
149
158
|
|
|
150
|
-
With `--libass`, ffmpeg draws the captions itself from captions.ass with its ass filter, and where Chrome is missing or cannot start, ffmpeg makes plainer pictures too, and CAPTIONS.md says why. That needs an ffmpeg built with libass, which Homebrew's default ffmpeg is not. The kit gives the ass filter libass's complex shaping, which joins Arabic letters and forms the conjuncts of Devanagari and Bengali as Chrome does. ffmpeg's subtitles filter shapes them simply and draws Arabic in a font without its presentation forms, such as Noto Kufi Arabic, as boxes, so to burn captions.ass yourself, use `ass=captions.ass:shaping=complex`. When its font comes from fonts.css, give libass the folder of that file too, easyfontkit's `files`, as `ass=captions.ass:fontsdir=../easyfontkit-acme.com/files:shaping=complex` does from the captions' folder, or libass draws another font in its place; CAPTIONS.md gives the folder. captions.ass names the video's color matrix and range in its YCbCr Matrix line, as TV.709 for most HD videos, so players and ffmpeg 8's ass filter convert its colors as the video's own; for a video in BT.2020's matrix, as HDR video is, one stored as RGB, and one in YCgCo or another matrix ASS has no name for, it says None, with which they take the video's matrix. Older ass filters, such as 4.4's, convert every color as BT.601 does, so the kit gives them the colors that come out as the video's own, and a full range picture in the one form they draw full range colors on, so its white is white; a color BT.601 cannot reach that way, such as a pure green on an HD video, is drawn as the nearest one it can, and CAPTIONS.md says so. libass lays every line out from the left unless the style's Encoding is -1, so captions in Arabic, Hebrew or another script written from the right have -1 there, and libass, like Chrome, takes each line's direction from its first letter, so a line of Arabic has its full stop at the left. libass finds a font file in the kit's folder by the family name in the file, which fonts.css can give otherwise (easyfontkit calls InterVariable Inter), so captions.ass names it as the file does. libass draws a variable font at its default weight rather than the bold, and CAPTIONS.md says so; a highlight too dark for letters becomes a thick border of that color instead of a patch. libass draws no color emoji either, so the video shows a box for an emoji this machine has no black and white font for. CAPTIONS.md says which on Linux, where fontconfig can tell, and elsewhere names the emoji as ones that can show as boxes. libass makes a font as tall as its OS/2 heights rather than its em, so captions.ass gives the font the size whose em is the letter size CAPTIONS.md names. libass measures no letters either, so a line whose width, added up from the widths of its letters in the font, puts it past the safe space is drawn smaller, and captions.ass draws it smaller the same way, as capitals often need. The heights and widths come from the font file where the kit knows it: the one fonts.css names, or on Linux the one fontconfig gives for the name. libass draws a letter that font lacks, as a Latin font lacks Japanese, with another font, sized by that font's own heights, so on Linux captions.ass sets such letters in the font fontconfig gives for them, and an Arabic letter in the first Arabic family Chrome is given that this machine has, such as Noto Sans Arabic, at the size that makes them as big as the others. Elsewhere the numbers are Arial Bold's, and a letter the font does not have is guessed, so such a line can still reach past the safe space. Where Chrome drew the captions, captions.ass takes the size Chrome measured. Without `--libass`, a run with no Chrome, or with one that cannot start here (in a container without its sandbox, say), stops before the encode and says so.
|
|
159
|
+
With `--libass`, ffmpeg draws the captions itself from captions.ass with its ass filter, and where Chrome is missing or cannot start, ffmpeg makes plainer pictures too, and CAPTIONS.md says why. That needs an ffmpeg built with libass, which Homebrew's default ffmpeg is not. The kit gives the ass filter libass's complex shaping, which joins Arabic letters and forms the conjuncts of Devanagari and Bengali as Chrome does. ffmpeg's subtitles filter shapes them simply and draws Arabic in a font without its presentation forms, such as Noto Kufi Arabic, as boxes, so to burn captions.ass yourself, use `ass=captions.ass:shaping=complex`. When its font comes from fonts.css, give libass the folder of that file too, easyfontkit's `files`, as `ass=captions.ass:fontsdir=../easyfontkit-acme.com/files:shaping=complex` does from the captions' folder, or libass draws another font in its place; CAPTIONS.md gives the folder. captions.ass names the video's color matrix and range in its YCbCr Matrix line, as TV.709 for most HD videos, so players and ffmpeg 8's ass filter convert its colors as the video's own; for a video in BT.2020's matrix, as HDR video is, one stored as RGB, and one in YCgCo or another matrix ASS has no name for, it says None, with which they take the video's matrix. Older ass filters, such as 4.4's, convert every color as BT.601 does, so the kit gives them the colors that come out as the video's own, and a full range picture in the one form they draw full range colors on, so its white is white; a color BT.601 cannot reach that way, such as a pure green on an HD video, is drawn as the nearest one it can, and CAPTIONS.md says so. libass lays every line out from the left unless the style's Encoding is -1, so captions in Arabic, Hebrew or another script written from the right have -1 there, and libass, like Chrome, takes each line's direction from its first letter, so a line of Arabic has its full stop at the left. libass finds a font file in the kit's folder by the family name in the file, which fonts.css can give otherwise (easyfontkit calls InterVariable Inter), so captions.ass names it as the file does. libass draws a variable font at its default weight rather than the bold, and CAPTIONS.md says so; a highlight too dark for letters becomes a thick border of that color instead of a patch. libass draws no color emoji either, so the video shows a box for an emoji this machine has no black and white font for, and for any letter no font on it has. CAPTIONS.md says which on Linux, where fontconfig can tell, with what to do, and elsewhere names the emoji as ones that can show as boxes. libass makes a font as tall as its OS/2 heights rather than its em, so captions.ass gives the font the size whose em is the letter size CAPTIONS.md names. libass measures no letters either, so a line whose width, added up from the widths of its letters in the font, puts it past the safe space is drawn smaller, and captions.ass draws it smaller the same way, as capitals often need. The heights and widths come from the font file where the kit knows it: the one fonts.css names, or on Linux the one fontconfig gives for the name. libass draws a letter that font lacks, as a Latin font lacks Japanese, with another font, sized by that font's own heights, so on Linux captions.ass sets such letters in the font fontconfig gives for them, and an Arabic letter in the first Arabic family Chrome is given that this machine has, such as Noto Sans Arabic, at the size that makes them as big as the others. Elsewhere the numbers are Arial Bold's, and a letter the font does not have is guessed, so such a line can still reach past the safe space. Where Chrome drew the captions, captions.ass takes the size Chrome measured. Without `--libass`, a run with no Chrome, or with one that cannot start here (in a container without its sandbox, say), stops before the encode and says so.
|
|
151
160
|
|
|
152
161
|
## Limits
|
|
153
162
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "easycaptionkit",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.3",
|
|
4
4
|
"description": "Burn captions into a video in the product's own fonts and colors, word by word or line by line, kept clear of each app's buttons, with the SRT, VTT and ASS files to go with it.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "Emin Budak",
|
|
@@ -32,7 +32,7 @@ node <skill>/scripts/easycaptionkit.js subtitles transcript.json --video video.m
|
|
|
32
32
|
|
|
33
33
|
## Use the result
|
|
34
34
|
|
|
35
|
-
1. Read `CAPTIONS.md` first. It says which font drew the letters (and how many letters fell back to another font, as Japanese or Arabic text over a Latin brand font does), where the captions sit and why, and which captions go faster than people read.
|
|
35
|
+
1. Read `CAPTIONS.md` first. It says which font drew the letters (and how many letters fell back to another font, as Japanese or Arabic text over a Latin brand font does, and which show as boxes because no font on the machine has them), where the captions sit and why, and which captions go faster than people read.
|
|
36
36
|
2. Look at `sheet.png`: frames of `captioned.mp4` for up to 24 captions, spread over the video. Check that no caption covers text or a face in the picture; if one does, run again with `--position middle` or `top`. A video with more captions than the sheet shows has every caption's time in `captions.json`; take a frame at the times that matter.
|
|
37
37
|
3. Show `safe-zones.png` to the user: a frame under each app's buttons, the margins tinted, the safe space outlined.
|
|
38
38
|
4. Hand over `captioned.mp4`. It has every frame of the video at its own time, or, where the video holds a frame too long for the captions or has few frames a second (CAPTIONS.md says so), every frame at a steady rate, and its sound copied, or made AAC when MP4 players do not take it. Give `captions.srt` or `captions.vtt` to places that take a captions file.
|
|
@@ -42,7 +42,7 @@ node <skill>/scripts/easycaptionkit.js subtitles transcript.json --video video.m
|
|
|
42
42
|
## When it goes wrong
|
|
43
43
|
|
|
44
44
|
- "reads files, not addresses": the video or the words were given as a link. Download it first and give the file. "--brand takes an easybrandkit folder" or "--fonts takes an easyfontkit folder": a site's address was given for the brand or the fonts. Make the folder from that site with easybrandkit or easyfontkit and give it.
|
|
45
|
-
- "Every word starts after the video ends": the words belong to another video, or their times are in milliseconds in a shape the kit does not know. Convert them to seconds in a plain list of `{ "text", "start", "end" }`.
|
|
45
|
+
- "Every word starts after the video ends": the words belong to another video, or their times are in milliseconds in a shape the kit does not know. Convert them to seconds in a plain list of `{ "text", "start", "end" }`. Where the message says the words file gives the video's length, as a demo's steps.json does on a subtitles run with no `--video`, the times in that file or that length are wrong; `--video` goes by the video's own length.
|
|
46
46
|
- A caption is drawn smaller than the others: one of its lines was wider than the safe space (with `--libass`, by an estimate of its width). Lower `--max-chars` for long lines. A single long word, like a link, fits only at a lower `--size`; or shorten it in `words.json`.
|
|
47
47
|
- "Pick another --out": the folder holds a file by one of the names the kit writes that it did not write itself (another tool's words.json, or a link, say) or that has changed since it wrote it, or one of the files the run reads, or, for a subtitles run, the captioned video of an earlier run, which new subtitle files would no longer match. Give a new folder with `--out`.
|
|
48
48
|
- "holds words fixed by hand": the folder's words.json has fixes this run would lose, since it reads another words file. The message gives the command that keeps them, the one refused with `<out>/words.json` as its words; or move `<out>/words.json` away to start again from the words file. A subtitles run into the folder of a captioned video is refused whatever its words, and when they were fixed there, the message gives the command that makes subtitles of them in a folder of their own.
|
|
@@ -50,8 +50,13 @@ node <skill>/scripts/easycaptionkit.js subtitles transcript.json --video video.m
|
|
|
50
50
|
- Without `--out`, a second video by the same name as an earlier one (every easyscreenshotkit demo is demo.mp4) goes to `easycaptionkit-demo-2`, and the summary says so. A subtitles run goes to `easycaptionkit-subtitles-<video name>`, or `easycaptionkit-subtitles-<words name>` without `--video`, in the same way. With `--out`, a run into the folder of another video's captions replaces them and says whose they were.
|
|
51
51
|
- CAPTIONS.md says the brand font was asked for by name and this machine does not have it: give easyfontkit's folder with `--fonts`. When it says Chrome drew no letter with the file of a fonts.css font, that file lacks the letters of these words, as a cut for another script does.
|
|
52
52
|
- The font license in CAPTIONS.md is commercial: tell the user before the video goes out. The file names the closest free font; `--font "<that font>"` uses it once it is installed. When CAPTIONS.md says no letter of the font is in the video, as with Japanese or Arabic words over a Latin font, its license does not reach the video: the fonts that drew the letters came with the machine.
|
|
53
|
-
- With `--libass`, CAPTIONS.md can say the video shows boxes for some emoji: libass draws no color emoji. Run again without `--libass` to have Chrome draw them.
|
|
53
|
+
- With `--libass`, CAPTIONS.md can say the video shows boxes for some emoji: libass draws no color emoji. Run again without `--libass` to have Chrome draw them.
|
|
54
|
+
- CAPTIONS.md says Chrome found no font for some letters, or that no font libass can use has them: the video shows a box for each, as it does for Japanese on a server with no Japanese font. Install a font that has them, such as Noto Sans CJK (`fonts-noto-cjk` on Debian and Ubuntu), or give `--fonts` an easyfontkit folder whose font has them, and run again. Do not hand over a video with boxes in its captions.
|
|
54
55
|
- No Chrome on the machine: the run stops. `--libass` draws the captions with ffmpeg's libass instead, which needs an ffmpeg built with libass (Homebrew's default ffmpeg is not).
|
|
56
|
+
- "easycaptionkit needs ffmpeg", "easycaptionkit needs ffprobe" or "easycaptionkit needs Chrome": the run stopped before any work, as it found none, or the one it found or the variable names could not be started, did not answer as one when asked its version or, for ffmpeg, failed when asked its encoders or its filters. The message names that file, by `FFMPEG_PATH`, `FFPROBE_PATH`, `CHROME_PATH` or PATH, and gives what is at its path, as a folder, or what the system said, or the first line the program printed and how it ended: tell the user, and ask them to install the program or to point the variable at one that runs, and never guess a path. Where it is Chrome, `--libass` draws the captions without it.
|
|
57
|
+
- "easycaptionkit could not start ffmpeg:", or ffprobe or Chrome, and what the system is short of: the system could not start a program then, as when too many files are open, and the message says what to do, such as raising the limit on open files or running it again once fewer processes run. The program is not at fault, so do not ask the user to install another.
|
|
58
|
+
- "easycaptionkit could not start ffmpeg, <its path>, again", or ffprobe or Chrome: the program answered when the run began, and the system could not start it later in the run, as when it was taken away or changed in the meantime. Tell the user what the system said, and run again: a program that is still wrong then stops the run before any work, with what is wrong with it.
|
|
59
|
+
- "easycaptionkit could not check ffmpeg", or ffprobe or Chrome: the program gave no answer in two minutes when asked its version or, for ffmpeg, its encoders or its filters, as on a machine short of memory. Run again, and if it gives no answer again, tell the user, with the path the message names.
|
|
55
60
|
- "Chrome could not start its sandbox here", or another reason Chrome stopped as it started (in a container, or on a CI runner without user namespaces): the run stops before the encode. `--libass` draws the captions without Chrome, and ffmpeg makes plainer sheet.png and safe-zones.png, which CAPTIONS.md notes.
|
|
56
61
|
- A run killed outright (`kill -9`, or the system when memory runs out) cannot tidy up: its hidden `.easycaptionkit-work-` folder stays in the out folder, where an ffmpeg it started can go on writing captioned.mp4, and its Chrome profile stays in the temporary folder. The captioned.mp4 in that folder is no result to hand over: run again. The next run into the folder removes the work folder, and a later run of the same user that starts Chrome removes the profile once no process names it and it has not changed for a day, except on Windows, where it stays in the temporary folder until Windows or the user clears it.
|
|
57
62
|
- The safe zones are the apps' ad guides or estimates, each with its source and date. Tell the user to check one post on a phone.
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
const path = require("node:path");
|
|
5
5
|
const version = require("./lib/version");
|
|
6
6
|
const { PLACES, PLACE_NAMES } = require("./lib/places");
|
|
7
|
-
const { drewWith, setsAny, setsNone, machineHas, shown, soundText } = require("./lib/report");
|
|
7
|
+
const { drewWith, setsAny, setsNone, boxedIn, machineHas, shown, soundText } = require("./lib/report");
|
|
8
8
|
const { variableFace, sameName } = require("./lib/look");
|
|
9
9
|
|
|
10
10
|
const USAGE = `Usage: easycaptionkit <video> --words <file> [options]
|
|
@@ -297,7 +297,12 @@ function summary({ dir, facts: f }) {
|
|
|
297
297
|
// sets them in, which can be all of them, as Japanese over a Latin font.
|
|
298
298
|
const fallbacks = libass ? f.assFallbacks || [] : [];
|
|
299
299
|
const none = libass && setsNone(f);
|
|
300
|
-
|
|
300
|
+
// Where letters show as boxes, which a note names, the fonts named
|
|
301
|
+
// drew the others, and the font lacks more letters than they drew.
|
|
302
|
+
const boxed = boxedIn(f);
|
|
303
|
+
const them = boxed ? "all but the boxes" : "them";
|
|
304
|
+
const the = boxed ? "" : "the ";
|
|
305
|
+
if (!f.look.family) font = libass ? `Arial, asked for by name${m && !sameName(m.family, "Arial") ? ` (fontconfig gives ${m.family})` : ""}` : f.fontFirst ? (drawn.length && !firstDrew ? `${f.fontFirst}, which has none of these letters: ${list(drawn.map((x) => x.family))} drew ${them}` : f.fontFirst) : drawn.length ? `a common sans (${list(drawn.map((x) => x.family))} drew it)` : "a common sans";
|
|
301
306
|
else if (libass && variableFace(f.look) && setsAny(f)) font = `${f.look.family} from fonts.css, which libass draws at its variable file's default weight, not at ${f.look.weight}`;
|
|
302
307
|
else if (libass && f.look.from === "machine") font = `${f.look.family} ${f.look.weight}, asked for by name${!m ? "; libass draws whatever this machine gives for that name" : sameName(m.family, f.look.family) ? "" : ` (not on this machine: fontconfig gives ${m.family})`}`;
|
|
303
308
|
else font = `${f.look.family} ${f.look.weight}${f.look.from === "fonts.css" ? " from fonts.css" : ""}`;
|
|
@@ -307,7 +312,7 @@ function summary({ dir, facts: f }) {
|
|
|
307
312
|
const families = list(fallbacks.map((x) => x.family));
|
|
308
313
|
const set = fallbacks.reduce((a, b) => a + b.letters, 0);
|
|
309
314
|
if (none) font += `, ${renamed ? "and which" : "which"} has none of these letters: captions.ass sets them in ${families}`;
|
|
310
|
-
else if (fallbacks.length) font += `; captions.ass sets the
|
|
315
|
+
else if (fallbacks.length) font += `; captions.ass sets ${the}${set} ${set === 1 ? "letter" : "letters"} it lacks in ${families}`;
|
|
311
316
|
const missing = drawn.length && f.look.family && !drawn.some((x) => drewWith(x, f.look));
|
|
312
317
|
const why = f.look.from !== "machine" ? "its file drew nothing" : machineHas(f) ? "it has none of these letters" : "not on this machine";
|
|
313
318
|
// With Chrome, the fonts that drew the letters the font lacks, where
|
|
@@ -315,8 +320,10 @@ function summary({ dir, facts: f }) {
|
|
|
315
320
|
const own = libass ? [] : drawn.filter((x) => (f.look.family ? drewWith(x, f.look) : x.family === f.fontFirst));
|
|
316
321
|
const others = own.length ? drawn.filter((x) => !own.includes(x)) : [];
|
|
317
322
|
const rest = others.reduce((a, b) => a + b.glyphs, 0);
|
|
318
|
-
const filled = others.length ? `; Chrome drew the
|
|
319
|
-
|
|
323
|
+
const filled = others.length ? `; Chrome drew ${the}${rest} ${rest === 1 ? "letter" : "letters"} it lacks with ${list(others.map((x) => (others.length > 1 ? `${x.family} (${x.glyphs})` : x.family)))}` : "";
|
|
324
|
+
// Every letter a box: the font asked for by name drew none of them either.
|
|
325
|
+
const gone = !libass && boxed && !drawn.length && f.look.family && f.look.from === "machine" && !machineHas(f);
|
|
326
|
+
lines.push(` Font ${font}${missing ? ` (${why}: ${list(drawn.map((x) => x.family))} drew ${them})` : gone ? " (not on this machine)" : filled}`);
|
|
320
327
|
if (f.style !== "line") lines.push(` Lit word ${f.look.highlightMode !== "patch" ? f.look.highlight : libass ? `in a thick border of ${f.look.highlight}` : `on a patch of ${f.look.highlight}`}`);
|
|
321
328
|
lines.push(` Video captioned.mp4, ${o.width}x${o.height}, ${o.fps} fps, ${sec(o.duration)}, ${soundText(o.audio)}`);
|
|
322
329
|
}
|
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
|
|
3
|
+
// The letters a video shows as boxes. Chrome draws a letter it finds no
|
|
4
|
+
// font for with the missing glyph of the first font of the list, a box,
|
|
5
|
+
// and names that font as the one that drew it: on an Ubuntu with no
|
|
6
|
+
// Japanese font, Liberation Sans for every letter of a Japanese caption.
|
|
7
|
+
// So each font Chrome names is asked whether it has each letter, a font
|
|
8
|
+
// of fonts.css through its own files and any other through the letters
|
|
9
|
+
// fontconfig lists for it, and a letter none of them has is a box. libass
|
|
10
|
+
// looks through fontconfig for a font for each letter its own font lacks,
|
|
11
|
+
// and draws a box where no font on the machine has one. Without
|
|
12
|
+
// fontconfig, as on macOS and Windows, nothing here can say, and nothing
|
|
13
|
+
// is said.
|
|
14
|
+
|
|
15
|
+
const { spawnSync } = require("node:child_process");
|
|
16
|
+
const { clusters } = require("./text");
|
|
17
|
+
const { fcName } = require("./look");
|
|
18
|
+
|
|
19
|
+
// What fc-list prints for a pattern in a format, or "" where there is no
|
|
20
|
+
// fc-list or it fails. The letters of every font of a machine with many
|
|
21
|
+
// fonts take some megabytes.
|
|
22
|
+
function fcList(pattern, format) {
|
|
23
|
+
let r;
|
|
24
|
+
try {
|
|
25
|
+
r = spawnSync("fc-list", ["-f", format, ...(pattern ? [pattern] : [])], { encoding: "utf8", timeout: 20_000, maxBuffer: 256 * 1024 * 1024 });
|
|
26
|
+
} catch {
|
|
27
|
+
return "";
|
|
28
|
+
}
|
|
29
|
+
return r.error || r.status !== 0 ? "" : r.stdout || "";
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
// The code points of fontconfig's charsets, as it prints them ("20-7e
|
|
33
|
+
// a0-17f 192"), all of them together, as a lookup: has(code). The ranges
|
|
34
|
+
// are put in order and joined, and a code point is found by halves.
|
|
35
|
+
function letterSet(charsets) {
|
|
36
|
+
const ranges = [];
|
|
37
|
+
for (const charset of charsets) {
|
|
38
|
+
for (const part of charset.split(/\s+/)) {
|
|
39
|
+
const m = /^([0-9a-f]{1,6})(?:-([0-9a-f]{1,6}))?$/i.exec(part);
|
|
40
|
+
if (m) ranges.push([parseInt(m[1], 16), parseInt(m[2] || m[1], 16)]);
|
|
41
|
+
}
|
|
42
|
+
}
|
|
43
|
+
ranges.sort((a, b) => a[0] - b[0]);
|
|
44
|
+
const starts = [];
|
|
45
|
+
const ends = [];
|
|
46
|
+
for (const [from, to] of ranges) {
|
|
47
|
+
const last = ends.length - 1;
|
|
48
|
+
if (last >= 0 && from <= ends[last] + 1) ends[last] = Math.max(ends[last], to);
|
|
49
|
+
else {
|
|
50
|
+
starts.push(from);
|
|
51
|
+
ends.push(to);
|
|
52
|
+
}
|
|
53
|
+
}
|
|
54
|
+
return (code) => {
|
|
55
|
+
let lo = 0;
|
|
56
|
+
let hi = starts.length - 1;
|
|
57
|
+
let found = -1;
|
|
58
|
+
while (lo <= hi) {
|
|
59
|
+
const mid = (lo + hi) >> 1;
|
|
60
|
+
if (starts[mid] <= code) {
|
|
61
|
+
found = mid;
|
|
62
|
+
lo = mid + 1;
|
|
63
|
+
} else hi = mid - 1;
|
|
64
|
+
}
|
|
65
|
+
return found >= 0 && code <= ends[found];
|
|
66
|
+
};
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
// The letters a font Chrome names has, as fontconfig lists them for its
|
|
70
|
+
// PostScript name, else for its family: has(code), or null where
|
|
71
|
+
// fontconfig gives nothing for it or cannot be asked. `list` stands in
|
|
72
|
+
// for fc-list in the tests.
|
|
73
|
+
function fontLetters({ family = "", postScriptName = "" }, { platform = process.platform, list = fcList } = {}) {
|
|
74
|
+
if (platform !== "linux") return null;
|
|
75
|
+
const charsets = (pattern) => list(pattern, "%{charset}\\n").split("\n").filter((line) => line.trim());
|
|
76
|
+
let found = postScriptName ? charsets(`:postscriptname=${fcName(postScriptName)}`) : [];
|
|
77
|
+
if (!found.length && family) found = charsets(`:family=${fcName(family)}`);
|
|
78
|
+
return found.length ? letterSet(found) : null;
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
// The letters of every font on this machine, as fontconfig lists them:
|
|
82
|
+
// has(code), or null where fontconfig cannot be asked. libass finds a
|
|
83
|
+
// font through fontconfig for a letter its own font lacks, among all of
|
|
84
|
+
// them, so a letter none of them has is a box.
|
|
85
|
+
function machineLetters({ platform = process.platform, list = fcList } = {}) {
|
|
86
|
+
if (platform !== "linux") return null;
|
|
87
|
+
const text = list("", "%{charset}\\n");
|
|
88
|
+
return text.trim() ? letterSet(text.split("\n")) : null;
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
// A letter as a reader sees it, one or more code points, is in a font
|
|
92
|
+
// when the font has each of them but those that draw nothing (a joiner, a
|
|
93
|
+
// variation selector, the object mark U+FFFC), as they are, composed or
|
|
94
|
+
// decomposed: HarfBuzz draws é from e and its accent in a font that has
|
|
95
|
+
// only those, and a no-break hyphen with the plain one of a font that has
|
|
96
|
+
// only that, as Liberation Sans is from version 2.1.
|
|
97
|
+
const DRAWS_NOTHING = /^[\p{Default_Ignorable_Code_Point}\ufffc]$/u;
|
|
98
|
+
function hasLetter(has, letter) {
|
|
99
|
+
for (const form of new Set([letter, letter.normalize("NFC"), letter.normalize("NFD"), letter.replace(/\u2011/g, "\u2010")])) {
|
|
100
|
+
if (Array.from(form).every((p) => DRAWS_NOTHING.test(p) || has(p))) return true;
|
|
101
|
+
}
|
|
102
|
+
return false;
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
// The letters of `letters` Chrome drew as boxes: those no font it names
|
|
106
|
+
// for them has, of [{ family, postScriptName, custom, glyphs }]. A font of
|
|
107
|
+
// fonts.css, which Chrome names as custom, is asked through `kit(c)`,
|
|
108
|
+
// kitLetters' answer for a code point. Where a font named cannot be asked,
|
|
109
|
+
// as without fontconfig, or Chrome names none, nothing can say: none.
|
|
110
|
+
// `lookUp` stands in for fontLetters in the tests.
|
|
111
|
+
function chromeBoxes(letters, fonts, { kit = null, lookUp = fontLetters } = {}) {
|
|
112
|
+
const drew = fonts.filter((font) => font.glyphs > 0);
|
|
113
|
+
if (!drew.length) return [];
|
|
114
|
+
const asks = [];
|
|
115
|
+
for (const font of drew) {
|
|
116
|
+
if (font.custom) {
|
|
117
|
+
if (!kit) return [];
|
|
118
|
+
asks.push(kit);
|
|
119
|
+
continue;
|
|
120
|
+
}
|
|
121
|
+
const has = lookUp(font);
|
|
122
|
+
if (!has) return [];
|
|
123
|
+
asks.push((p) => has(p.codePointAt(0)));
|
|
124
|
+
}
|
|
125
|
+
return letters.filter((letter) => !/^\s+$/.test(letter) && !asks.some((has) => hasLetter(has, letter)));
|
|
126
|
+
}
|
|
127
|
+
|
|
128
|
+
// Each letter of the captions that is one of `boxed`, with the number of
|
|
129
|
+
// its caption: [{ letter, cue }], as assLetters gives the ones libass has
|
|
130
|
+
// no font for.
|
|
131
|
+
function boxesIn(cues, boxed) {
|
|
132
|
+
const set = new Set(boxed);
|
|
133
|
+
if (!set.size) return [];
|
|
134
|
+
const out = [];
|
|
135
|
+
for (const cue of cues) for (const line of cue.lines) for (const c of clusters(line)) if (set.has(c)) out.push({ letter: c, cue: cue.n });
|
|
136
|
+
return out;
|
|
137
|
+
}
|
|
138
|
+
|
|
139
|
+
module.exports = { fcList, letterSet, fontLetters, machineLetters, hasLetter, chromeBoxes, boxesIn };
|
|
@@ -12,15 +12,16 @@ const crypto = require("node:crypto");
|
|
|
12
12
|
const { loadWords, handFixed, wordsSha256, FORMATS, MOST_OPENED, DEEPEST } = require("./words");
|
|
13
13
|
const { cjkLanguage, arabicIn, scriptsIn } = require("./text");
|
|
14
14
|
const { buildCues, states: makeStates, readingSpeed } = require("./cues");
|
|
15
|
-
const { srt, vtt, ass, assFont, estimatedScales, shrunkOf, assLetters, SRT_SPECIAL } = require("./subtitles");
|
|
15
|
+
const { srt, vtt, ass, assFont, kitLetters, estimatedScales, shrunkOf, assLetters, SRT_SPECIAL } = require("./subtitles");
|
|
16
|
+
const { machineLetters, chromeBoxes, boxesIn } = require("./boxes");
|
|
16
17
|
const { captionLayout, charsThatFit, defaultPlaces, shapeOf } = require("./places");
|
|
17
18
|
const { chooseLook, variableFace, machineFont, fallbackFont } = require("./look");
|
|
18
19
|
const { fontLike } = require("./fontfile");
|
|
19
20
|
const ffmpeg = require("./ffmpeg");
|
|
20
|
-
const {
|
|
21
|
+
const { chromeReady, chromeProblem, tempProblem } = require("./chrome");
|
|
21
22
|
const { renderStates } = require("./render");
|
|
22
23
|
const { safeZonesPng, sheetPng, plainPictures, shotOf } = require("./draw");
|
|
23
|
-
const { captionsMarkdown, captionsJson, subtitlesMarkdown, arg, subtitlesCommand, shown } = require("./report");
|
|
24
|
+
const { captionsMarkdown, captionsJson, subtitlesMarkdown, arg, subtitlesCommand, shown, captionsOf, lettersOf, boxesText, installText } = require("./report");
|
|
24
25
|
const { jsonShape, jsonProblem, decodeText } = require("./files");
|
|
25
26
|
const { stopAll } = require("./running");
|
|
26
27
|
const version = require("./version");
|
|
@@ -823,15 +824,18 @@ function notStill(video, name) {
|
|
|
823
824
|
|
|
824
825
|
// The words, cut to the video: words after its end are left out, a word
|
|
825
826
|
// that runs past it is cut there.
|
|
826
|
-
function fitWords(read, duration) {
|
|
827
|
+
function fitWords(read, duration, givenBy = "") {
|
|
827
828
|
const notes = [];
|
|
828
829
|
if (!Number.isFinite(duration)) return { words: read.words, notes };
|
|
829
830
|
const words = read.words.filter((w) => w.start < duration - 0.05).map((w) => ({ ...w, end: Math.min(w.end, duration) }));
|
|
830
831
|
const dropped = read.words.length - words.length;
|
|
831
|
-
|
|
832
|
-
|
|
832
|
+
// `givenBy`: the file that gives the length where no video was given,
|
|
833
|
+
// which says nothing of whether its own words belong to its video.
|
|
834
|
+
const end = `${duration.toFixed(2)} s${givenBy ? `, as ${givenBy} gives its length` : ""}`;
|
|
835
|
+
if (dropped) notes.push(`${dropped} ${dropped === 1 ? "word starts" : "words start"} after the video ends (${end}) and ${dropped === 1 ? "was" : "were"} left out.${givenBy ? "" : " Check that the words belong to this video."}`);
|
|
836
|
+
if (!words.length) throw new Error(`Every word starts after the video ends (${end}). ${givenBy ? "The times in that file or the length it gives are wrong" : "The words file belongs to another video, or its times are in another unit"}`);
|
|
833
837
|
const last = words[words.length - 1].end;
|
|
834
|
-
if (last < duration * 0.5 && duration > 20) notes.push(`The words end at ${last.toFixed(1)} s, before half of the video (${duration.toFixed(1)} s). Check that they belong to this video.`);
|
|
838
|
+
if (!givenBy && last < duration * 0.5 && duration > 20) notes.push(`The words end at ${last.toFixed(1)} s, before half of the video (${duration.toFixed(1)} s). Check that they belong to this video.`);
|
|
835
839
|
return { words, notes };
|
|
836
840
|
}
|
|
837
841
|
|
|
@@ -859,29 +863,25 @@ function specialNote(cues) {
|
|
|
859
863
|
|
|
860
864
|
// The letters libass drew boxes for, since this machine has no font it
|
|
861
865
|
// can draw them with, as notes: emoji, which libass draws in no color
|
|
862
|
-
// font, and any other letters. `missing` and `unsure`
|
|
863
|
-
// where nothing says which font an emoji goes in, as on
|
|
864
|
-
// the note says it can be a box.
|
|
866
|
+
// font, and any other letters, with what to do. `missing` and `unsure`
|
|
867
|
+
// are assLetters': where nothing says which font an emoji goes in, as on
|
|
868
|
+
// macOS and Windows, the note says it can be a box.
|
|
865
869
|
const EMOJI = /[\p{Extended_Pictographic}\p{Regional_Indicator}]/u;
|
|
866
870
|
function missingNotes(missing, unsure = []) {
|
|
867
|
-
const where = (items) => {
|
|
868
|
-
const cues = [...new Set(items.map((m) => m.cue))];
|
|
869
|
-
return `${cues.length === 1 ? "caption" : "captions"} ${cues.length > 6 ? `${cues.slice(0, 6).join(", ")} and ${cues.length - 6} more` : list(cues.map(String))}`;
|
|
870
|
-
};
|
|
871
|
-
const which = (items) => {
|
|
872
|
-
const letters = [...new Set(items.map((m) => m.letter))];
|
|
873
|
-
return letters.length > 8 ? `${letters.slice(0, 8).join(" ")} and ${letters.length - 8} more` : letters.join(" ");
|
|
874
|
-
};
|
|
875
871
|
const boxes = (n) => (n === 1 ? "a box for it" : "boxes for them");
|
|
876
872
|
const emoji = missing.filter((m) => EMOJI.test(m.letter));
|
|
877
873
|
const other = missing.filter((m) => !EMOJI.test(m.letter));
|
|
878
874
|
const notes = [];
|
|
879
|
-
if (emoji.length) notes.push(`libass draws no color emoji, and this machine has no other font that draws ${emoji.length === 1 ? "the emoji" : `the ${emoji.length} emoji`} in ${
|
|
880
|
-
if (other.length) notes.push(
|
|
881
|
-
if (unsure.length) notes.push(`libass draws no color emoji, so ${unsure.length === 1 ? "the emoji" : `the ${unsure.length} emoji`} in ${
|
|
875
|
+
if (emoji.length) notes.push(`libass draws no color emoji, and this machine has no other font that draws ${emoji.length === 1 ? "the emoji" : `the ${emoji.length} emoji`} in ${captionsOf(emoji)} (${lettersOf(emoji)}) whole, so the video shows ${boxes(emoji.length)}. Leave out --libass to have Chrome draw ${emoji.length === 1 ? "it" : "them"}.`);
|
|
876
|
+
if (other.length) notes.push(`${boxesText(other, "libass")} ${installText(other)}`);
|
|
877
|
+
if (unsure.length) notes.push(`libass draws no color emoji, so ${unsure.length === 1 ? "the emoji" : `the ${unsure.length} emoji`} in ${captionsOf(unsure)} (${lettersOf(unsure)}) can show as ${unsure.length === 1 ? "a box" : "boxes"}: nothing on this machine says which font libass draws ${unsure.length === 1 ? "it" : "them"} with. Leave out --libass to have Chrome draw ${unsure.length === 1 ? "it" : "them"}.`);
|
|
882
878
|
return notes;
|
|
883
879
|
}
|
|
884
880
|
|
|
881
|
+
// The letters Chrome drew as boxes, [{ letter, cue }], as a note: which,
|
|
882
|
+
// and what to do.
|
|
883
|
+
const boxesNotes = (boxes) => (boxes.length ? [`${boxesText(boxes, "chrome")} ${installText(boxes)}`] : []);
|
|
884
|
+
|
|
885
885
|
// The folder of a fonts.css font's files, which ffmpeg's ass filter finds
|
|
886
886
|
// the font of captions.ass in when it is its fontsdir, from the output
|
|
887
887
|
// folder `dir`, where captions.ass is; else "".
|
|
@@ -945,15 +945,22 @@ async function captionVideo(videoFile, options) {
|
|
|
945
945
|
writable(dir);
|
|
946
946
|
const notes = recover(path.resolve(dir), dir);
|
|
947
947
|
const view = guard(dir, { video: videoFile, timeline: timelineOf(videoFile), words: options.words, brand: options.brand, fonts: options.fonts });
|
|
948
|
-
const bins = ffmpeg.tools();
|
|
949
948
|
const useLibass = Boolean(options.libass);
|
|
950
|
-
if (!useLibass && !findChrome()) throw new Error(missingChrome());
|
|
951
949
|
// Chrome keeps its profile in the temporary folder.
|
|
952
950
|
const temp = useLibass ? "" : tempProblem();
|
|
953
951
|
if (temp) throw new Error(`${temp}. Set it to a folder, or unset it`);
|
|
954
|
-
|
|
952
|
+
// Each program the run uses is asked its version before the video is
|
|
953
|
+
// read, Chrome while ffmpeg is: one that cannot be started or does not
|
|
954
|
+
// answer as itself stops the run with what the system or the program
|
|
955
|
+
// said, ffmpeg's first.
|
|
956
|
+
const chrome = useLibass ? null : chromeReady();
|
|
957
|
+
if (chrome) chrome.catch(() => {});
|
|
958
|
+
const tools = await ffmpeg.ready();
|
|
959
|
+
if (chrome) await chrome;
|
|
960
|
+
const bins = { ffmpeg: tools.ffmpeg, ffprobe: tools.ffprobe, places: tools.places };
|
|
961
|
+
const [video, caps] = await Promise.all([ffmpeg.readVideo(videoFile, bins), ffmpeg.capabilities(bins, tools.version)]);
|
|
955
962
|
notStill(video, videoFile);
|
|
956
|
-
if (!caps.encoder) throw new Error("This ffmpeg has no H.264 encoder (libx264, VideoToolbox or
|
|
963
|
+
if (!caps.encoder) throw new Error("This ffmpeg has no H.264 encoder (libx264, VideoToolbox, OpenH264 or Media Foundation), so it cannot write the captioned MP4");
|
|
957
964
|
if (useLibass && !caps.libass) throw new Error("This ffmpeg has no libass (its ass filter), so --libass cannot draw the captions. Leave out --libass to draw them with Chrome, or install an ffmpeg built with libass");
|
|
958
965
|
if (!useLibass && !caps.overlay) throw new Error("This ffmpeg has no overlay filter, so it cannot lay the captions over the video");
|
|
959
966
|
const read = await loadWords(options.words);
|
|
@@ -967,7 +974,7 @@ async function captionVideo(videoFile, options) {
|
|
|
967
974
|
notes.push(...place.notes);
|
|
968
975
|
const fitted = fitWords(read, video.duration);
|
|
969
976
|
notes.push(...fitted.notes);
|
|
970
|
-
if (read.video && Math.abs(read.video.duration - video.duration) > 0.5) notes.push(`steps.json is for a video ${read.video.duration.toFixed(1)} s long, and this one is ${video.duration.toFixed(1)} s. The captions follow the steps' times.`);
|
|
977
|
+
if (read.format === "steps" && read.video && Math.abs(read.video.duration - video.duration) > 0.5) notes.push(`steps.json is for a video ${read.video.duration.toFixed(1)} s long, and this one is ${video.duration.toFixed(1)} s. The captions follow the steps' times.`);
|
|
971
978
|
const words = fitted.words;
|
|
972
979
|
log(`Read ${count(words.length, "word", "words")} from ${shown(path.basename(options.words))} (${read.about})`);
|
|
973
980
|
const cues = buildCues(words, { lines, maxChars, hard: read.hard, duration: video.duration });
|
|
@@ -1021,8 +1028,14 @@ async function captionVideo(videoFile, options) {
|
|
|
1021
1028
|
// The font captions.ass names, as libass and players that read ASS
|
|
1022
1029
|
// make it: a font asked for by name is what this machine gives for it.
|
|
1023
1030
|
const libassFont = look.from === "fonts.css" ? null : machineFont(look.family || "Arial", look.weight);
|
|
1024
|
-
|
|
1031
|
+
// libass finds a font through fontconfig for each letter its own font
|
|
1032
|
+
// lacks: one no font on this machine has, as fontconfig lists them, is
|
|
1033
|
+
// a box, which needs no asking.
|
|
1034
|
+
const every = useLibass ? machineLetters() : null;
|
|
1035
|
+
const font = assFont(look, libassFont, { fallback: givenFor(look), lacks: every ? (code) => !every(code) : null });
|
|
1025
1036
|
const letters = assLetters(cues, font);
|
|
1037
|
+
// The letters the video shows as boxes, [{ letter, cue }].
|
|
1038
|
+
let boxes = useLibass ? letters.missing : [];
|
|
1026
1039
|
// captions.ass names the video's own matrix and range for players; the
|
|
1027
1040
|
// file libass burns can name others, and give other colors (assColors).
|
|
1028
1041
|
const assFile = (scales, { level = 1, matrix = color.header, tint = null } = {}) => ass(cues, words, { states, style: options.style, look: { ...look, tool: TOOL }, place, title: titled, level, scales, font, matrix, tint });
|
|
@@ -1066,14 +1079,16 @@ async function captionVideo(videoFile, options) {
|
|
|
1066
1079
|
} else {
|
|
1067
1080
|
log(`Drawing ${count(states.length, "caption state", "caption states")} in Chrome`);
|
|
1068
1081
|
try {
|
|
1069
|
-
drawn = await renderStates({ cues, words, states, style: options.style, look, place, duration: video.duration, dir: run.work, log });
|
|
1082
|
+
drawn = await renderStates({ cues, words, states, style: options.style, look, place, duration: video.duration, dir: run.work, log, boxed: (all, fonts) => chromeBoxes(all, fonts, { kit: kitLetters(look) }) });
|
|
1070
1083
|
} catch (err) {
|
|
1071
1084
|
// A Chrome that cannot start here (no usable sandbox, as in a
|
|
1072
1085
|
// container) stops the run before the encode; libass needs none. A
|
|
1073
1086
|
// Chrome the system stopped as memory ran out can draw with fewer
|
|
1074
|
-
// jobs at once, or a smaller video.
|
|
1075
|
-
|
|
1076
|
-
|
|
1087
|
+
// jobs at once, or a smaller video. A message that ends with a full
|
|
1088
|
+
// stop, as one of what to do does, keeps only that one.
|
|
1089
|
+
const said = err.message.replace(/\.$/, "");
|
|
1090
|
+
if (err.chrome && err.signal === "SIGKILL") err.message = `${said}. Run again with fewer jobs at once or with a smaller video, or pass --libass to have ffmpeg draw the captions without Chrome`;
|
|
1091
|
+
else if (err.chrome && err.start) err.message = `${said}. Pass --libass to have ffmpeg draw the captions without Chrome`;
|
|
1077
1092
|
throw err;
|
|
1078
1093
|
}
|
|
1079
1094
|
fs.writeFileSync(path.join(run.work, "captions.ass"), assFile(drawn.scales));
|
|
@@ -1082,6 +1097,8 @@ async function captionVideo(videoFile, options) {
|
|
|
1082
1097
|
notes.push(...made.notes);
|
|
1083
1098
|
sound = made.tracks;
|
|
1084
1099
|
frameTags = made.frameTags;
|
|
1100
|
+
boxes = boxesIn(cues, drawn.boxes);
|
|
1101
|
+
notes.push(...boxesNotes(boxes));
|
|
1085
1102
|
if (drawn.failedFonts.length) notes.push(`Chrome could not load the font ${[...new Set(drawn.failedFonts)].join(", ")}; the next font in the list drew the captions.`);
|
|
1086
1103
|
if (drawn.shrunk) notes.push(`${drawn.shrunk} ${drawn.shrunk === 1 ? `${unit[0]} was` : `${unit[1]} were`} wider than the safe space and drawn smaller to fit, ${drawn.shrunk === 1 ? "at" : "the smallest at"} ${percentOf(drawn.smallest)}% of the size.`);
|
|
1087
1104
|
}
|
|
@@ -1118,8 +1135,10 @@ async function captionVideo(videoFile, options) {
|
|
|
1118
1135
|
const sheetFile = path.join(run.work, "sheet.png");
|
|
1119
1136
|
// Chrome lays the pictures out. Where there is none, or it cannot start
|
|
1120
1137
|
// or stops, ffmpeg makes plainer ones: the captions are in the video
|
|
1121
|
-
// by now, and the pictures only show them.
|
|
1122
|
-
|
|
1138
|
+
// by now, and the pictures only show them. A run that drew them with
|
|
1139
|
+
// libass asks Chrome its version here, which a run that drew them with
|
|
1140
|
+
// Chrome did before any work, with the temporary folder.
|
|
1141
|
+
let plain = useLibass ? (await chromeProblem()) || tempProblem() : "";
|
|
1123
1142
|
if (!plain) {
|
|
1124
1143
|
try {
|
|
1125
1144
|
await safeZonesPng({ bins, video, output: out, zones: place.zones, cue: widest, times, file: zonesFile, dir: run.work, frameTags });
|
|
@@ -1152,6 +1171,7 @@ async function captionVideo(videoFile, options) {
|
|
|
1152
1171
|
states: drawn ? drawn.states : states.length,
|
|
1153
1172
|
fontsDrawn: drawn ? drawn.fonts : [],
|
|
1154
1173
|
fontFirst: drawn ? drawn.first : "",
|
|
1174
|
+
boxes,
|
|
1155
1175
|
libassFont: useLibass ? libassFont : null,
|
|
1156
1176
|
// The name captions.ass gives the font, from its file for fonts.css,
|
|
1157
1177
|
// and the fonts it sets the letters in that the font lacks.
|
|
@@ -1172,7 +1192,7 @@ async function captionVideo(videoFile, options) {
|
|
|
1172
1192
|
fs.writeFileSync(path.join(run.work, "CAPTIONS.md"), captionsMarkdown(facts));
|
|
1173
1193
|
names.push("CAPTIONS.md");
|
|
1174
1194
|
const files = writtenFiles(run.work, names);
|
|
1175
|
-
fs.writeFileSync(path.join(run.work, "words.json"), JSON.stringify(wordsJson(words, read, files, first), null, 2) + "\n");
|
|
1195
|
+
fs.writeFileSync(path.join(run.work, "words.json"), JSON.stringify(wordsJson(words, read, files, first, video.duration), null, 2) + "\n");
|
|
1176
1196
|
const json = captionsJson({ ...facts, files: { ...files, ...writtenFiles(run.work, ["words.json"]) } });
|
|
1177
1197
|
fs.writeFileSync(path.join(run.work, "captions.json"), JSON.stringify(json, null, 2) + "\n");
|
|
1178
1198
|
names.push("words.json", "captions.json");
|
|
@@ -1266,12 +1286,17 @@ function describe(v, rate = null, frames = 0) {
|
|
|
1266
1286
|
// firstFormat's), which a run that reads the file again keeps. A word
|
|
1267
1287
|
// without a time of its own says how it got one: spread over its line by
|
|
1268
1288
|
// length, or placed between its neighbors, as the file gave it none.
|
|
1269
|
-
|
|
1289
|
+
// Words of a demo's steps keep `end`, where this run ended the captions:
|
|
1290
|
+
// the video's length, or the one steps.json gives without a video. That
|
|
1291
|
+
// is `demo`, where a later subtitles run with no video ends them too.
|
|
1292
|
+
function wordsJson(words, read, files, first, end) {
|
|
1270
1293
|
const items = words.map((w) => ({ text: w.text, start: round(w.start), end: round(w.end), ...(w.seg >= 0 ? { seg: w.seg } : {}), ...(w.timed ? {} : w.placed ? { placed: true } : { spread: true }) }));
|
|
1294
|
+
const demo = read.video && Number.isFinite(end) ? end : 0;
|
|
1271
1295
|
return {
|
|
1272
1296
|
tool: TOOL,
|
|
1273
1297
|
from: FORMATS[first] || read.about,
|
|
1274
1298
|
hard: read.hard,
|
|
1299
|
+
...(demo > 0 ? { demo: { duration: round(demo) } } : {}),
|
|
1275
1300
|
// The other files the run wrote, so a folder whose captions.json went
|
|
1276
1301
|
// is still known as the run's.
|
|
1277
1302
|
files,
|
|
@@ -1307,6 +1332,9 @@ async function makeSubtitles(wordsFile, options) {
|
|
|
1307
1332
|
writable(dir);
|
|
1308
1333
|
const notes = recover(path.resolve(dir), dir);
|
|
1309
1334
|
const view = guard(dir, { words: wordsFile, video: options.video, brand: options.brand, fonts: options.fonts }, { subtitles: true, options });
|
|
1335
|
+
// The video is read with ffprobe alone, which is asked its version first;
|
|
1336
|
+
// no ffmpeg is started, so nothing at ffmpeg's path stops the run.
|
|
1337
|
+
const bins = options.video ? await ffmpeg.readyProbe() : null;
|
|
1310
1338
|
const read = await loadWords(wordsFile);
|
|
1311
1339
|
const first = firstFormat(read, origin);
|
|
1312
1340
|
const look = chooseLook({ ...options, ...scriptsOf(read.words) });
|
|
@@ -1314,7 +1342,7 @@ async function makeSubtitles(wordsFile, options) {
|
|
|
1314
1342
|
notes.push(...read.notes, ...look.notes);
|
|
1315
1343
|
let video = null;
|
|
1316
1344
|
if (options.video) {
|
|
1317
|
-
video = await ffmpeg.readVideo(options.video,
|
|
1345
|
+
video = await ffmpeg.readVideo(options.video, bins);
|
|
1318
1346
|
notStill(video, options.video);
|
|
1319
1347
|
notes.push(...video.notes);
|
|
1320
1348
|
}
|
|
@@ -1323,11 +1351,14 @@ async function makeSubtitles(wordsFile, options) {
|
|
|
1323
1351
|
const maxChars = options.maxChars || 42;
|
|
1324
1352
|
const { place, lines } = cueLayout({ ...options, maxChars }, width, height, places);
|
|
1325
1353
|
notes.push(...place.notes);
|
|
1326
|
-
|
|
1354
|
+
// No caption runs past the video's end: the video's, or with no video
|
|
1355
|
+
// the one steps.json gives the demo's video, as words.json keeps it.
|
|
1356
|
+
const end = video ? video.duration : read.video ? read.video.duration : Infinity;
|
|
1357
|
+
const fitted = fitWords(read, end, video ? "" : shown(path.basename(wordsFile)));
|
|
1327
1358
|
notes.push(...fitted.notes);
|
|
1328
1359
|
const words = fitted.words;
|
|
1329
1360
|
log(`Read ${count(words.length, "word", "words")} from ${shown(path.basename(wordsFile))} (${read.about})`);
|
|
1330
|
-
const shape = { lines: options.style === "word" ? 1 : lines, hard: read.hard, duration:
|
|
1361
|
+
const shape = { lines: options.style === "word" ? 1 : lines, hard: read.hard, duration: end };
|
|
1331
1362
|
const cues = buildCues(words, { ...shape, maxChars });
|
|
1332
1363
|
// Words said after the picture ends, while the sound runs on, are heard.
|
|
1333
1364
|
if (video && video.soundPast && cues.some((cue) => cue.end > video.picture.end + 0.05)) notes.push(`The video's picture ends at ${video.picture.end.toFixed(2)} s, and its sound runs on to ${video.duration.toFixed(2)} s. The captions go on to the end of the sound, over the last frame, which players hold.`);
|
|
@@ -1381,7 +1412,7 @@ async function makeSubtitles(wordsFile, options) {
|
|
|
1381
1412
|
};
|
|
1382
1413
|
fs.writeFileSync(path.join(run.work, "CAPTIONS.md"), subtitlesMarkdown(facts));
|
|
1383
1414
|
const files = writtenFiles(run.work, ["captions.srt", "captions.vtt", "captions.ass", "CAPTIONS.md"]);
|
|
1384
|
-
fs.writeFileSync(path.join(run.work, "words.json"), JSON.stringify(wordsJson(words, read, files, first), null, 2) + "\n");
|
|
1415
|
+
fs.writeFileSync(path.join(run.work, "words.json"), JSON.stringify(wordsJson(words, read, files, first, end), null, 2) + "\n");
|
|
1385
1416
|
const json = captionsJson({ ...facts, files: { ...files, ...writtenFiles(run.work, ["words.json"]) } });
|
|
1386
1417
|
fs.writeFileSync(path.join(run.work, "captions.json"), JSON.stringify(json, null, 2) + "\n");
|
|
1387
1418
|
await heard();
|
|
@@ -1392,4 +1423,4 @@ async function makeSubtitles(wordsFile, options) {
|
|
|
1392
1423
|
}
|
|
1393
1424
|
}
|
|
1394
1425
|
|
|
1395
|
-
module.exports = { captionVideo, makeSubtitles, slug, notAddresses, writable, guard, recover, begin, finish, abandon, frameFor, encodingLine, percentOf, missingNotes, signalsOf, stopOnSignals, stopped, began, OUTPUTS, TOOL, shown, named };
|
|
1426
|
+
module.exports = { captionVideo, makeSubtitles, slug, notAddresses, writable, guard, recover, begin, finish, abandon, frameFor, encodingLine, percentOf, missingNotes, boxesNotes, signalsOf, stopOnSignals, stopped, began, OUTPUTS, TOOL, shown, named };
|