ffmpeg-skill 2.5.1 → 2.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -82,7 +82,7 @@ An agent that "knows FFmpeg" still guesses: it assumes a frame rate, picks a cod
82
82
  - **Structured tools, not shell strings.** Each operation is a script with typed arguments. Nothing runs through a shell; no filter graph is accepted from the caller.
83
83
  - **A contract the agent can read.** `contract --json` states, for every tool, what it takes, what it writes, which FFmpeg components it needs and how the result is verified. The MCP surface is derived from it.
84
84
  - **Verification after execution.** The result is probed, checked against the destination's spec and, when the picture changed, looked at as a contact sheet.
85
- - **Local first.** No cloud, no API keys, no Python dependencies. Optional local transcription is used when a whisper is installed, never required — by `caption.py --transcribe` and, since 1.17, by `silence.py --filler --transcribe`; both take a transcript you already have instead, and both refuse with the install lines rather than guessing.
85
+ - **Local first.** No cloud, no API keys, no Python dependencies. Optional local transcription is used when a speech engine (Parakeet or a whisper) is installed, never required — by `caption.py --transcribe` and, since 1.17, by `silence.py --filler --transcribe`; both take a transcript you already have instead, and both refuse with the install lines rather than guessing. The engine runs on your machine and the audio is never uploaded; parakeet-mlx, faster-whisper and openai-whisper download their model weights the first time they run (parakeet-mlx's default from Hugging Face).
86
86
 
87
87
  ## Quick start
88
88
 
@@ -188,7 +188,7 @@ These are the rules the skill file gives the agent and the code enforces.
188
188
 
189
189
  ## Tools
190
190
 
191
- 42 public tools, all Python 3.9 standard library, all with `--help`, `--dry-run`, `--json`, `--plan FILE` (a dry run written as a plan `render.py` executes later), non-zero exit and a reason on stderr on failure. `--json-brief` (1.11.0) prints the same result document trimmed to what a caller acts on — status, output, `verified`, a compact `summary` of the output probe, the tool's own keys and the command count instead of the command lines — for roughly a third of the bytes; `--json` itself is unchanged. Every re-encoding tool takes `--codec h264|hevc|av1|prores` and `--quality N` (1.8), and every time flag takes seconds, `mm:ss`, `hh:mm:ss.fff` or SMPTE `hh:mm:ss:ff` with an optional `@fps` suffix (1.9).
191
+ 42 public tools, all Python 3.9 standard library, all with `--help`, `--dry-run`, `--json`, `--plan FILE` (a dry run written as a plan `render.py` executes later), non-zero exit and a reason on stderr on failure. `--json-brief` (1.11.0) prints the same result document trimmed to what a caller acts on — status, output, `verified`, a compact `summary` of the output probe, the tool's own keys and the command count instead of the command lines — for roughly a third of the bytes; `--json` itself is unchanged. Every re-encoding tool takes `--codec h264|hevc|av1|prores` and `--quality N` (1.8) and, with `export.py`, `--hw`/`--no-hw` (Apple VideoToolbox on Apple Silicon, opt-in; see `doctor` below), and every time flag takes seconds, `mm:ss`, `hh:mm:ss.fff` or SMPTE `hh:mm:ss:ff` with an optional `@fps` suffix (1.9).
192
192
 
193
193
  **Analysis and inspection**
194
194
 
@@ -356,7 +356,7 @@ npx ffmpeg-skill doctor --json # available / missing / missing_optional / unkn
356
356
 
357
357
  `doctor` reads `ffmpeg -encoders`, `-filters` and `-bsfs` and resolves every capability the contract declares against this machine's build. Three states per capability: `available`, `missing`, `unknown`. Exit 0 when everything required is available, 1 when something required is missing, 2 when nothing is proven missing but a required capability is unknown. With detection on (the default), `contract --json` carries the same lists under `capabilities`. `doctor` also reports fonts: the default drawtext family, and `fonts.scripts` — one `available`/`missing`/`unknown` per writing system (ja, zh, ko, ar, he, hi, th, ru, el) with the file it would use — so "can this machine render Korean captions" is answered before the job, not after. `doctor --json`'s `tools` field folds that down to one answer per tool — `{"caption": {"usable": "no", "missing": ["filter:subtitles"], "fix": "..."}, ...}` — so "is `doctor` overall `ok`" and "can I run `caption.py` on this machine" are answered separately: a plain Homebrew `ffmpeg` is `ok` for tools that don't need `subtitles`/`drawtext`/`zscale`, while `caption`'s own `usable` is `"no"`.
358
358
 
359
- `doctor --json`'s `gpu_encoders` reports which GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) this ffmpeg *build* was compiled with — read from `-encoders` alone, so it proves the capability shipped, not that the GPU/driver on this machine will actually accept a job (that needs a real encode, which `doctor`'s introspection never runs). No tool here uses one yet — every tool still assumes CPU x264/x265 — so this is purely informational and never affects `ok` or any tool's `usable`. GPU-accelerated encoding stays deliberately off the roadmap until there's a real-hardware-verified design for it (build-presence alone is not proof a job will succeed) — not a promised feature, just an honest "not yet, and not without proof it actually works."
359
+ `doctor --json`'s `gpu_encoders` reports which GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) this ffmpeg *build* was compiled with — read from `-encoders` alone, so it proves the capability shipped, not that the GPU/driver on this machine will actually accept a job (that needs a real encode, which `doctor`'s introspection never runs). It never affects `ok` or any tool's `usable`. The one GPU path a tool uses is Apple VideoToolbox, opt-in: `--hw` on every re-encoding tool (h264/hevc/prores on the GPU, AV1 stays on SVT-AV1), `FFMPEG_SKILL_HW=1` to make it this machine's default for everything except `export.py`'s delivery presets (those take `--hw` explicitly, or `render.py --hw` for a whole project). It runs only on Apple Silicon and never by default; it falls back to the CPU encoder — reported in the result's `hw.fallback`/`hw.notes` — only when VideoToolbox itself refuses a job (any other ffmpeg failure is reported as it happened, not retried); every result names the `encoder` that actually ran (`batch.py`, which encodes nothing itself: each `results` row), and a GPU encode adds a `notes` line that its quality is not CRF-equivalent. Measured on an M4 Max, 2–7× faster than x264/x265 `medium`; at matched SSIM (`tests/bench_vt.py`) 1.1–2.9× the bytes on SDR and 1.9–3.5× on HDR, which is why it is a flag and not the default (`docs/design-decisions.md`). `doctor --json`'s `hw` says whether this machine can run it (`platform_ok`) and whether it is the default here (`default_on`).
360
360
 
361
361
  ## Gotchas and best practices
362
362
 
@@ -371,7 +371,7 @@ The short list for humans. The agent-facing version, with the reasoning, is the
371
371
  - **Phrase-aware caption breaking (1.16).** `caption.py`/`graphics.py --wrap phrase` (the default) never breaks inside a word or on the wrong side of a hyphen, never leaves a lone digit, kana or punctuation pair on a line, prefers Japanese sentence ends and particles over a mid-word break, and never ends a line on an article or preposition. All four are penalties over break positions that already fit, so no line is widened and the line count never changes; `--wrap measured` restores 1.15's width-only wrap. The text itself is never rewritten or shortened. Since 1.16.1 a Thai run and a katakana word are never broken inside (Thai writes no space inside a phrase and there is no dictionary: the break goes where you put a space or `|`), and a line wider than the safe width is reported as `overlong` with the fix named.
372
372
  - **Caption size fitted to the cue (1.17, reachable from the templates since 1.17.1).** At a platform caption size a line holds about six em, so an ordinary sentence needs four lines and `--max-lines 2` used to cut it into consecutive cues — half the sentence arriving late. `caption.py --fit-size` (default `auto`) walks the size down until every cue fits, *before* laying the cues out, with a legibility floor of 4.5 % of the frame height (`--min-size`, default 13 ASS units). An explicit `--size` or a `brand.json` size is a statement about the look and is never overridden — which in 1.17.0 also silenced the fitter on every `render.py --template` run, since a template fills the size from the platform table; 1.17.1 marks that size as the default it is (`"fit_size": "on"` in the filled project), so the type shrinks and no cue is split. `--fit-size off` restores that earlier behaviour byte for byte, and the caption text is still never rewritten to make it fit.
373
373
  - **Beat-synced cuts (1.17).** `scenes.py --beats` reports the measured grid — tempo, beat times, and a confidence built from how far the winning autocorrelation lag stands above the others and how many onsets land on it. `cut.py --snap beats` (and a `"snap"` block in a `render.py` project) moves in/out points to the nearest beat within `--snap-tolerance` — and only onto the grid points a measured onset actually marks, never onto the regular grid's continuation through a passage with no music in it. Below `--min-confidence` it **refuses**: a cut point may move to a measured beat and may not appear from one, so speech and ambience get an honest "no steady pulse here" instead of an invented grid.
374
- - **Filler words (1.17).** `silence.py --filler --words transcript.json` removes "um" and "uh" through the same cut graph the silences use. Never without measured word timings — there is no heuristic that finds an "um" without them that would not also cut real speech — and `like`, `tipo` and `cioè` are deliberately not in the built-in lists, because a discourse marker is a content word. Whisper stays optional: `--transcribe` with no engine installed refuses and names the three installs.
374
+ - **Filler words (1.17).** `silence.py --filler --words transcript.json` removes "um" and "uh" through the same cut graph the silences use. Never without measured word timings — there is no heuristic that finds an "um" without them that would not also cut real speech — and `like`, `tipo` and `cioè` are deliberately not in the built-in lists, because a discourse marker is a content word. Whisper stays optional: `--transcribe` with no engine installed refuses and names the five installs.
375
375
  - **Throughput (1.17).** `batch.py --jobs N` runs several files at once, capped at `min(N, cpu_count, 8)` and sharing one `--timeout` budget rather than one per item; the per-item table keeps its order. `render.py --cache DIR` reuses stages whose inputs and arguments did not change, so swapping an export preset re-runs export only. The cache is opt-in with no default directory, and the ffmpeg, skill and contract versions are part of every key, so a cache is never reused across them.
376
376
  - **Audiogram (1.16).** `waveform.py --image cover.png` (or `render.py --template audiogram`) puts the waveform over a still plate for an episode that has no picture, with `--platform` for the frame, `--title` and burnt-in captions. The image is a local file you give: nothing is fetched and no cover art is ever invented.
377
377
  - **Emoji in captions and titles (1.15).** `caption.py`/`graphics.py --emoji-assets DIR` composites a PNG per emoji (Twemoji/Noto naming, `1f389.png`) on top of the text, because drawtext cannot load a colour emoji font at all and an installed one does not prove libass will draw it in colour — `doctor --json`'s `fonts.emoji` answers that from a render probe. Without assets the run still succeeds and reports `mode: mono`. Nothing is ever downloaded.
package/SKILL.md CHANGED
@@ -7,7 +7,7 @@ description: 'Edit video and audio with local FFmpeg from natural-language reque
7
7
 
8
8
  Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`, and delivery templates in `templates/`. This file is enough to do a job: the table below routes the request, and `--help` on the script about to run is the cheapest full flag list. A reference file costs as much as this one; open one only for a question you have: `references/scripts.md` (every flag of all 42 scripts), `references/devices.md` (iPhone HDR, GoPro, DJI, screen recordings, Zoom), `references/gotchas.md` (the long form of the one-line rules at the end). MCP: `tools/list` shows only the core 12 by default; the other 30 (per this table) stay callable by name via `tools/call`, described by `contract --json`; set `FFMPEG_SKILL_MCP_FULL=1` for all 42.
9
9
 
10
- Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (status/output/verified plus a `summary`; prefer on writing steps); `--fast` (preview quality); `--progress`; `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (the dry run as a plan `render.py FILE` runs later; refuses if an input changed). Re-encoding tools also take `--codec h264|hevc|av1|prores` and `--quality N`: unset, SDR is x264, HDR is x265 Main10; `prores` needs `-o NAME.mov`, `h264` refuses HDR (`color.py --to-sdr` first).
10
+ Shared flags (every script): `--dry-run`; `--json` (output path, its probe, the commands run); `--json-brief` (status/output/verified + a `summary`; prefer on writing steps); `--fast` (preview quality); `--progress` (percent/ETA); `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (a dry run `render.py FILE` runs later; refuses if an input changed). Re-encoding tools also take `--hw` (Apple Silicon GPU, big files; `FFMPEG_SKILL_HW=1` = default, not export), `--codec h264|hevc|av1|prores` and `--quality N`: unset, SDR is x264, HDR is x265 Main10; `prores` needs `-o NAME.mov`, `h264` refuses HDR (`color.py --to-sdr` first).
11
11
 
12
12
  Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) may still run ffmpeg/ffprobe, skipping artifacts/side files (`--edl`, `--sheet`, a generated `.ass`); `verify` ignores the flag. Per tool: `contract --json` `dry_run`.
13
13
 
@@ -16,7 +16,7 @@ Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `chec
16
16
  0. **Environment, only on failure.** Never start a job with `doctor`: a broken machine fails with `kind: missing_tool` or an ffmpeg error naming the filter/encoder. Run `python3 <skill-dir>/scripts/_contract.py doctor` (or `npx ffmpeg-skill doctor`) after such a failure, or when asked what the machine can do: read `ok`/`usable`, report the missing capability. `contract --json`'s tool schema is for a *planning* agent, not this workflow.
17
17
  1. **Probe what you must plan from.** Run `probe.py` on each input you plan from — duration, fps, resolution, codecs, channels, `variable_frame_rate_suspected` — and whenever the user asks about a file. No separate probe before every edit: every writing tool's `--json` already carries its input and a probe of the output. Plan from real numbers, never assumptions.
18
18
  2. **Prefer lossless.** If the request can be met without re-encoding (plain cuts on keyframes, remuxing, audio-only changes), do not re-encode. `cut.py` and `loudness.py` stream-copy video by default; `--accurate` on `cut.py` only for frame-exact cuts.
19
- 3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's summary line, for any number in the plan (a dimension there can be a placeholder). Use before long encodes, to report facts. `--fast` is preview quality, `--progress` prints percent/ETA on stderr. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
19
+ 3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's summary line, for any number in the plan (a dimension there can be a placeholder). Use before long encodes, to report facts. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
20
20
  4. **Chain in a sensible order.** A delivery request with no other editing is one template run (`render.py --template NAME INPUT`), not a hand-built chain. Otherwise: colour (HDR→SDR / LUT) → cut → join → silence → fit → caption/overlay → sync → audio → loudness → export. Frame changes before captions, so text is sized for the final frame. Re-encode as few times as possible: intermediates at CRF 18, `export.py` last. **Three or more steps: `render.py` with a project.json.**
21
21
  5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination named (a template run already does). Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix. Judgement rows change content: duration (cut loses material), aspect (crop loses edges), fps (drops motion), loudness (ambience must not be boosted) — fix only when the request implies the answer, else state the choice and its cost. Mention WARNs; do not chase them.
22
22
  6. **Verify the output.** Confirm duration, resolution, fps and audio match the request, from the writing tool's `--json`/`--json-brief` probe or `probe.py`, and report those numbers ("final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo"). A step is done only when the script exited 0 and the output probes as expected: a non-zero exit, a missing or empty file, or a probe that contradicts the request is a failure reported with the script's error message.
@@ -32,7 +32,7 @@ Ask a short question only when the answer changes the output materially and the
32
32
 
33
33
  - **Destination** decides aspect, length limit, loudness and codec; "for Reels" answers all four and names a template. No destination and a plain cut/caption: keep the source format and say so. If the user says "export", "post" or "deliver", ask where.
34
34
  - **Duration** ("make it 60 s") without a method: speed up for ≤1.5× changes, trim otherwise, say which. Ask when the content is a talk (trimming loses words) and the change is large.
35
- - **Captions** without a text source: `--transcribe` if a local whisper exists, else ask for the text or a timed file; never invent dialogue.
35
+ - **Captions** without a text source: `--transcribe` if a local engine exists, else ask for the text or a timed file; never invent dialogue.
36
36
  - **Fonts and brand**: if the user mentions a brand, colours or "our font", ask for or create `brand.json` once, reuse it.
37
37
  - **CJK / non-Latin text**: let the tool pick the font by script (`--font` turns it off); `--lang ja|ko` for Han-only text. `doctor --json` `.fonts.scripts` says what renders here. Tofu is a failed job.
38
38
  - **Crop position** for `--fit crop`: centre by default, but when the request or the source names an off-centre subject ("keep the product on the right", someone visibly off-centre in the sheet) use `--crop-x`/`--crop-y` (0=left/top, 1=right/bottom) instead of guessing centre, or ask which edge to keep.
@@ -97,7 +97,7 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
97
97
  | "caption it with these lines" (text with times) | `caption.py input.mp4 --text cues.txt` |
98
98
  | "keep the subtitles toggleable", "mux in an SRT" | `caption.py input.mp4 --srt subs.srt --mode mux`; repeat `--srt file:lang` for several languages, `.mkv` for more than two |
99
99
  | "the captions are tiny / three lines on a Short", "don't chop the sentence" | `caption.py` shrinks the size until the cue fits `--max-lines` before splitting it (`--fit-size off` keeps `--size` and splits instead, `--min-size` the floor). Keep the template's `--max-lines` (2 on a vertical) and let the size drop; raising it to dodge a shrink stacks two words per line. A word still wider than the column at the floor is sliced at the edge (hyphen preferred), never rewritten, automatic; `broken_inside_word` in the JSON counts it |
100
- | "transcribe it and caption it" | `caption.py input.mp4 --transcribe --animate pop --karaoke` (needs a local whisper; else `--text`) |
100
+ | "transcribe it and caption it" | `caption.py input.mp4 --transcribe --animate pop --karaoke` (Parakeet/whisper; else `--text`) |
101
101
  | "TikTok-style captions with the words popping" | `caption.py input.mp4 --text cues.txt --animate pop --karaoke` |
102
102
  | "our logo top-right", "a watermark" | `overlay.py input.mp4 --image logo.png --position top-right --scale 200` |
103
103
  | "a title for the first 4 seconds" | `overlay.py input.mp4 --text "Title" --position top --start 0 --end 4 --fade 0.4` |
@@ -216,4 +216,4 @@ One line each; open the linked `references/gotchas.md` section when the job is i
216
216
  - `--fit crop` 16:9→9:16 throws away 70% of the width; "60 seconds" by speed or by trim are different answers — say which and why. -> [#reframing-fps-and-duration](references/gotchas.md#reframing-fps-and-duration)
217
217
  - TikTok/Reels cover the bottom fifth and right column with their own UI — templates keep text out of those zones; `look.py --safe tiktok` shows them. -> [#platform-safe-zones](references/gotchas.md#platform-safe-zones)
218
218
  - `scenes.py --highlights` ranks by loudness (or duration), never by meaning: check the sheet before trusting picks. -> [#highlights](references/gotchas.md#highlights)
219
- - Re-encodes use x264 `medium`; chain 3+ in one `render.py` project. -> [#chaining-and-speed](references/gotchas.md#chaining-and-speed)
219
+ - CPU re-encodes: x264 `medium`; chain 3+ in one `render.py` project. -> [#chaining-and-speed](references/gotchas.md#chaining-and-speed)
package/docs/contract.md CHANGED
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
21
21
  | Field | Meaning | Changes when |
22
22
  |---|---|---|
23
23
  | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
24
- | `skill.version` | the npm / package.json version (`2.5.1`) | any release |
24
+ | `skill.version` | the npm / package.json version (`2.7.0`) | any release |
25
25
 
26
26
  A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
27
27
  ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -119,11 +119,15 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
119
119
  "contract_version": "1.0",
120
120
  "deprecated": [],
121
121
  "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
122
- "skill": {"id": "ffmpeg-skill", "version": "2.5.1", "execution_mode": "local", "kind": "execution",
122
+ "skill": {"id": "ffmpeg-skill", "version": "2.7.0", "execution_mode": "local", "kind": "execution",
123
123
  "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
124
124
  "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
125
125
  "requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
126
- "execution": {"shell": false, "arbitrary_executables": false, "network": false, "input_mutation": false}
126
+ "execution": {"shell": false, "arbitrary_executables": false, "network": false,
127
+ "model_downloads": {"when": "...", "in_process": {"faster-whisper": "..."},
128
+ "child_process": {"parakeet-mlx": "...", "openai-whisper": "..."},
129
+ "none": ["whisper.cpp", "parakeet.cpp"], "offline": "..."},
130
+ "input_mutation": false}
127
131
  }
128
132
  ```
129
133
 
@@ -216,6 +220,15 @@ Main10 for HDR), so the flags add no behaviour to a caller that does not pass th
216
220
  encoder each value needs is listed under the tool's optional capabilities (`--codec hevc` and
217
221
  so on). `export.py` refuses `--codec`: its presets decide the codec.
218
222
 
223
+ `--hw` / `--no-hw` are on the same tools and on `export.py`, marked `common`;
224
+ `render.py` and `batch.py` take them for every stage they run. They are two booleans in the
225
+ input schema (`hw`, `no_hw`; MCP sends `--hw` for `hw: true` and `--no-hw` for `no_hw: true`),
226
+ mutually exclusive. `--hw` moves an h264/hevc/prores encode to Apple VideoToolbox (Apple Silicon
227
+ only; AV1 stays on the CPU); `--no-hw` keeps it on the CPU even when `FFMPEG_SKILL_HW=1` makes the
228
+ GPU this machine's default. Neither changes anything for a caller that passes neither on a
229
+ machine without the variable. What ran is in the result's `encoder` and `hw` (see the keys added
230
+ after 2.2.2, and `doctor`'s `gpu_encoders` below).
231
+
219
232
  `--plan FILE` (1.6) is a dry run that also writes a plan document: `{"plan_version": 1,
220
233
  "tool", "argv", "cwd", "inputs": [{"path", "size", "sha256_head_tail"}], "commands",
221
234
  "output", "verify": [{"tool": "probe"}, {"tool": "check", "platform"}], "notes"}`. It
@@ -237,11 +250,17 @@ gives you:
237
250
 
238
251
  | Hint | Tools |
239
252
  |---|---|
240
- | `bit_exact` | probe, check, scenes, look |
253
+ | `bit_exact` | probe, check, scenes, look, metadata, background |
241
254
  | `content_equivalent` (same media, bytes may differ between encoder builds) | every encoding tool, cut, sync, report |
242
255
  | `cached` | batch (content-hash cache, re-runs skip unchanged inputs); render **when `--cache DIR` is given** — the hint stays `content_equivalent` because that is what render is without the flag, and the cache is opt-in |
243
256
  | `environment_dependent` | verify |
244
257
 
258
+ The hints describe the CPU encoders. Under `--hw`, or `FFMPEG_SKILL_HW=1` on the machine, a tool
259
+ that encodes video runs VideoToolbox, whose bytes are not promised identical from run to run:
260
+ treat such a result as `content_equivalent` at best. That includes `background`, whose hint stays
261
+ `bit_exact` within 2.x. The variable belongs to the machine, so the same flags can give x264 bytes
262
+ on one machine and VideoToolbox bytes on another; `encoder` in the result says which.
263
+
245
264
  ## `provides`
246
265
 
247
266
  `provides` lists these 42 tools by a cross-repository Capability id, for
@@ -334,18 +353,40 @@ A project may now carry `"template"` (the name it was filled from) and `"frame":
334
353
  ## Capabilities
335
354
 
336
355
  Names: `ffmpeg`, `ffprobe`, `encoder:<name>`, `filter:<name>`, `bsf:<name>`,
337
- `external:whisper`. `capabilities.required` is the union of every tool's required list;
356
+ `external:whisper`, `external:parakeet`. `capabilities.required` is the union of every tool's required list;
338
357
  `optional` the union of the conditional ones. With detection (the default)
339
358
  `available`, `missing` and `missing_optional` are added from `doctor`, which reads
340
- `ffmpeg -encoders / -filters / -bsfs` and looks for a local whisper. Pass `--static`
359
+ `ffmpeg -encoders / -filters / -bsfs` and looks for a local whisper or Parakeet. Pass `--static`
341
360
  to omit detection. Nothing from the environment other than those lists and the
342
- ffmpeg/ffprobe/python versions is printed; no environment variables, no paths.
361
+ ffmpeg/ffprobe/python versions is printed; no environment variables, no paths (`doctor`'s
362
+ `hw.default_on` is a boolean derived from `FFMPEG_SKILL_HW`, never its value).
343
363
 
344
364
  `external:whisper` is **optional** for two tools since 1.17: `caption.py`
345
365
  (`--transcribe`) and `silence.py` (`--filler --transcribe`). Neither requires
346
366
  it — both take a transcript the caller already has (`--srt`/`--words`), and
347
- both refuse with the same three install lines when asked to make one with no
367
+ both refuse with the same install lines when asked to make one with no
348
368
  engine present. Whisper is never a dependency of this skill.
369
+ `external:parakeet` is the same kind of optional capability for the same two flags:
370
+ parakeet-mlx, or parakeet-cli with a `.gguf` model it can find. Either engine family
371
+ satisfies `--transcribe`; `--engine auto` (the default, or `FFMPEG_SKILL_ASR_ENGINE`) runs
372
+ Parakeet for English speech and Whisper for every other language. When no `--language` is given
373
+ and no detector can name it, a Whisper engine that can run takes it (whisper.cpp is passed
374
+ `-l auto`: its own default is English). Parakeet then runs only on English *assumed* -- when no
375
+ Whisper engine is installed, or every one failed, or `--model` names an English-only Parakeet
376
+ model -- and the result's top-level `notes` says the transcript is wrong if the speech is not
377
+ English; a multilingual (v3) Parakeet model assumes nothing and gets no note. The result's
378
+ `transcription` says which engine, model and routing decision produced the cues (in every
379
+ `--mode`, `mux` included).
380
+ Every engine transcribes on the machine running the skill, and the audio is never uploaded. The
381
+ skill's own code opens no network connection (`execution.network: false`), but an engine may
382
+ fetch its model weights on first use: parakeet-mlx downloads its default
383
+ `mlx-community/parakeet-tdt-0.6b-v2` from Hugging Face and openai-whisper its model from OpenAI,
384
+ each in its own child process. faster-whisper is a Python library the skill runs inside its own
385
+ process, so its first-run download from Hugging Face is made by the skill's process;
386
+ `HF_HUB_OFFLINE=1` keeps it (and parakeet-mlx) on the cached copy after that. parakeet.cpp and
387
+ whisper.cpp read a model file the user downloaded. `execution.model_downloads` says the same in the
388
+ contract, engine by engine, so a caller that sandboxes the network can tell before a first
389
+ `--transcribe` that faster-whisper's fetch comes from the skill's own process.
349
390
 
350
391
  `doctor` has three states per capability. `available` and `missing` come from a listing
351
392
  that was read; `unknown` means the listing that would prove the capability could not be
@@ -380,9 +421,14 @@ for tools that don't need `subtitles`/`drawtext`/`zscale`, but `caption.usable`
380
421
  "parsed"|"unparsed"|"failed"|"missing", "present": [...]}`. It proves the build shipped the
381
422
  capability, not that the GPU/driver on this machine will accept a job (that needs a real
382
423
  encode, which this introspection never runs). No tool declares or requires a GPU encoder, so
383
- `gpu_encoders` never affects `ok` or any tool's `usable` — it exists purely so a caller can ask
384
- the same honest yes/no/unknown question about GPU support that filter/encoder detection already
385
- answers for everything else, without a tool here needing to use one.
424
+ `gpu_encoders` never affects `ok` or any tool's `usable`. `--hw` (and
425
+ `FFMPEG_SKILL_HW=1`, which `export.py`'s delivery presets ignore) puts h264/hevc/prores encodes on
426
+ VideoToolbox on Apple Silicon only, never by default. A job VideoToolbox refuses (the encoder could
427
+ not open, or its compression session failed) is re-encoded on the CPU and said so in the result's
428
+ `hw.fallback`/`hw.notes`; any other failure is not retried. When `FFMPEG_SKILL_HW=1` rather than `--hw` chose the GPU and the GPU ran,
429
+ `hw.notes` also says so, with the file-size cost and `--no-hw` as the remedy for a final
430
+ deliverable. `doctor`'s `hw` field is `{platform_ok, default_on}` — normalised
431
+ booleans, never the environment value.
386
432
 
387
433
  `doctor`'s `fonts` field reports whether the default drawtext font (`caption.py`'s
388
434
  `--animate`/`--karaoke`, `graphics.py`'s templates — `BRAND_DEFAULTS["font"]`, `"DejaVu Sans"`)
@@ -493,7 +539,7 @@ Per-tool keys added in 1.17, all additive:
493
539
  | `filler`, `removed_seconds_total` | `silence.py --filler` | `{lang, source, engine, words, removed, removed_count, removed_seconds, removed_words, word_timings, list, warnings}`. The existing `removed_seconds` is unchanged in name and meaning — the seconds of *silence* removed, which is what it has always held — and `removed_seconds_total` is the additive sibling covering silence plus filler |
494
540
  | `jobs`, `jobs_requested`, `wall_seconds`, `item_seconds_total`, `timed_out` | `batch.py` | the parallelism actually applied and the number asked for, the batch's wall clock, the sum of the per-item times (so the speed-up can be quoted), and whether the shared timeout budget ran out. A timed-out item carries `"skipped": "timeout"` in its result row |
495
541
  | `timeline` | `render.py --export-timeline FILE` | `{format, rate, duration, frames, clips, transition, transition_frames, music, markers, notes, not_exported}`: the written timeline's own numbers (`rate` is exact, `"30000/1001"` for 29.97) and what the project asked for that an editor timeline cannot carry. Nothing is encoded (`commands: []`); `verified` means the file was written and reads back as its format (`verification` step `parse`) -- not that an editor opened it |
496
- | `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
542
+ | `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …), the GPU setting (`FFMPEG_SKILL_HW` and whether `--hw`/`--no-hw` made it explicit) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
497
543
 
498
544
  Per-tool keys added in 1.17.1, all additive:
499
545
 
@@ -524,11 +570,20 @@ Per-tool keys added after 2.2.2, all additive:
524
570
  | `silent` | `join.py` (every run) | `[{index, path, peak_db, at, end}]`: inputs whose whole-file audio peak (volumedetect) is at or below `--silence-threshold` (default -50 dBFS) and that `--on-silent warn` (default) joined anyway, with a `notes` line; `at`/`end` are where the input sits in the output, in seconds (a transition overlaps its neighbours). An input named by `--allow-silent N[,N]` (1-based) is a planned pause: always joined, listed with `intended: true`, no warning or note, whatever `--on-silent` says. `[]` otherwise. `--on-silent fail` names them in the `kind: input` refusal's `problems` (reason `silent (peak -91.0 dBFS)`), `skip` lists them under `skipped`. An input with no audio stream is never silent; `verified` is unaffected |
525
571
  | `short_segments`, `duplicates` | `join.py` (every run) | warnings, never refusals, and the join command is unchanged: `short_segments: [{index, path, duration}]` names each measured input shorter than 2 frames at the join's fps (an audio-only join: shorter than 0.05 s); `duplicates: [{path, indices}]` names a path (compared resolved) listed more than once — repeating a clip can be intended. `[]` when none; each non-empty one adds a `notes` line |
526
572
  | `silent` | `waveform.py` (every run) | `true` when the input audio's whole-file peak is at or below `--silence-threshold` (default -50 dBFS), which draws a flat line; `--on-silent warn` (default) renders it anyway with a `notes` line, `fail` refuses (`kind: input`) before ffmpeg runs. `false` when audible, `null` under `--dry-run` (nothing measured). `verified` is unaffected |
573
+ | `encoder`, `hw`, `notes` | every re-encoding tool, `export.py`, `render.py`, `batch.py` (and its rows), `waveform.py` | `encoder`: the video encoder the last command ran (after any GPU→CPU fallback; under `--dry-run`, the planned one), `copy` for a stream copy; absent when no command encoded video. A tool that runs other tools reports its stages': `render.py` its stages and an executed plan's tool, `waveform.py` the title/caption stage that writes its file (its `commands` include that stage's). `hw` is present when VideoToolbox was asked for (`--hw`, `FFMPEG_SKILL_HW=1`, or a stage or `batch.py` item that reported it) and on `export.py` when `FFMPEG_SKILL_HW=1` is set but its delivery preset does not take it: `{requested, source: flag\|env, used, fallback, notes}`. `requested` says whether this run's encodes were asked onto VideoToolbox (`false` for that export), `used` whether the encoder that ran last is VideoToolbox (`null` under `--dry-run`, which ran nothing, and when the tool encoded nothing itself: `batch.py`, whose top-level `hw` gathers its items' `requested`/`source`/`fallback`/`notes` while each `results` row carries the item's own `encoder`, and `hw`/`notes` when VideoToolbox was asked for), `fallback` whether a job VideoToolbox refused was re-encoded on the CPU, and `notes` why. Only a VideoToolbox failure (the encoder could not open, or its compression session failed) is retried on the CPU; any other failure is reported as it happened, once. Every VideoToolbox encode adds a top-level `notes` line: its quality is not CRF-equivalent (`--quality` / a preset's CRF maps to `-q:v` approximately), and `verified` stays what it is for every tool, the measured properties of the output. A `kind: ffmpeg` failure under `--hw` / `FFMPEG_SKILL_HW` carries `encoder` and `hw` for the command that ran last (a stage's, when `render.py`/`waveform.py` re-raise it); when the command itself failed inside the run (not a tool's own check afterwards, such as `cut.py`'s), its message also names that encoder |
574
+ | `transcription` | `caption.py --transcribe` (every `--mode`); inside `filler` for `silence.py --filler --transcribe` | `{routing, detected_language?, engine, model, language}`: the engine and model that made the transcript; `language` is the language the engine was told -- the one named (`--language`, `--filler-lang`), else the one auto's detector named -- and `null` when the engine identified it itself; `detected_language` is present when auto's whisper.cpp detector named one; `routing` says why `--engine auto` chose the engine (`"requested"` for a named engine), ending `, no Parakeet engine transcribed it` when auto chose Parakeet and a Whisper engine made the transcript after every Parakeet engine failed |
575
+ | `notes` | `silence.py --filler --transcribe` (new for `silence.py`; `caption.py` already had it) | present when `--engine auto` ran the English-only Parakeet model on English *assumed* (no language given or detectable, and no Whisper engine installed or none that succeeded): the transcript is wrong if the speech is not English. `caption.py --transcribe` puts the same line in its `notes` |
527
576
 
528
577
  `caption.py --transcribe` and `silence.py --filler --transcribe`: when a local speech engine ran
529
578
  and produced no cue, the refusal is `"<engine> found no speech in <input>"` with `kind: input`,
530
579
  `reason: "no_speech"` and `engine` in the failure document — distinct from the "no local
531
- speech-to-text engine found" refusal, which now only means no engine was found.
580
+ speech-to-text engine found" refusal, which now only means no engine was found. When `--engine
581
+ auto` found engines and none transcribed, the refusal is `kind: input` with `reason:
582
+ "engine_failed"` (one ran and failed) or `"english_only"` (the only engines found were Parakeet
583
+ engines auto passed over for a language that is not English), `engine` (the first one it names)
584
+ and `engines: [{engine, reason, detail}]`, where `detail` carries an engine's own error line. A
585
+ named `--engine` that is not installed is `kind: missing_tool`; one that is installed and failed
586
+ is `kind: input`, `reason: "engine_failed"`, with `engine` and `detail`.
532
587
 
533
588
  `check.py` also gains an informational `subtitles` row on **every** platform:
534
589
  `PASS` when every soft subtitle stream carries a language tag, `WARN` when one
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ffmpeg-skill",
3
- "version": "2.5.1",
3
+ "version": "2.7.0",
4
4
  "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
5
5
  "keywords": [
6
6
  "ffmpeg",
@@ -267,7 +267,13 @@ the contact sheet (`--sheet`) before treating the picks as final.
267
267
  Anything chained by hand through three re-encodes should be one `render.py`
268
268
  project instead, so the plan is one file and the user can change one number.
269
269
  Re-encodes use x264 `medium`; for long files add `--preset veryfast` to
270
- intermediates and keep the default for the final export.
270
+ intermediates and keep the default for the final export. On Apple Silicon,
271
+ `--hw` (or `FFMPEG_SKILL_HW=1` for the machine) moves h264/hevc/prores encodes to
272
+ VideoToolbox: 2–7× faster, 1.1–2.9× the bytes at the same quality (1.9–3.5× on HDR). The variable
273
+ leaves `export.py`'s delivery presets on the CPU, which is the usual split (GPU
274
+ intermediates, CPU delivery); `render.py --hw` puts the export on the GPU too.
275
+ Check `encoder`/`hw` in the result: a job the GPU refuses (H.264 wider than 4096)
276
+ was re-encoded on the CPU.
271
277
 
272
278
  ## Audio-only files
273
279
 
@@ -1,6 +1,6 @@
1
1
  # Script reference
2
2
 
3
- Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, before anything runs) to replace an existing output unless `--overwrite` is given. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither flag; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
3
+ Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every re-encoding tool and `export.py` also take `--hw`/`--no-hw` (h264/hevc/prores on Apple VideoToolbox, Apple Silicon only, CRF mapped to `-q:v`; AV1 stays on the CPU; a job VideoToolbox itself refuses -- the encoder cannot open or its compression session fails -- is re-encoded on the CPU and reported in `hw.fallback`/`hw.notes`, any other failure is not retried; a GPU encode adds a `notes` line that its quality is not CRF-equivalent; `FFMPEG_SKILL_HW=1` makes it the default for every tool except `export.py`, whose delivery presets need `--hw` itself; `render.py`/`batch.py --hw`/`--no-hw` apply to every stage, export included; results name the `encoder` that ran); every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, before anything runs) to replace an existing output unless `--overwrite` is given. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither `--codec` nor `--quality`; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
4
4
 
5
5
  ## Time grammar (every time-taking flag, 1.9)
6
6
 
@@ -490,7 +490,8 @@ each shorter clip's last frame (with silence) out to the longest.
490
490
  silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [--speech-aware] [-o OUT]
491
491
  silence.py INPUT --filler --words transcript.json [--filler-lang auto|en|ja|es|de|fr|pt|it]
492
492
  [--filler-words FILE] [--filler-extra W,W] [--filler-keep W,W] [--filler-pad 0.02]
493
- [--transcribe] [--filler-list] [--max-cuts 400]
493
+ [--transcribe] [--engine auto|parakeet-mlx|parakeet.cpp|whisper.cpp|faster-whisper|openai-whisper]
494
+ [--filler-list] [--max-cuts 400]
494
495
  ```
495
496
  Runs `silencedetect`, keeps `--margin` seconds of air around speech, drops
496
497
  gaps shorter than `--min-silence`, and re-encodes once with `select`/`aselect`
@@ -533,7 +534,10 @@ Three refusals, all `kind: input`, all before any encode:
533
534
 
534
535
  - `--filler` with neither `--words` nor `--transcribe` → names both flags.
535
536
  - `--transcribe` with no engine on PATH → the same message `caption.py` gives,
536
- with the three install lines.
537
+ with its install lines (Parakeet and whisper engines). Engines that were found
538
+ but did not transcribe are named instead, with why (`reason: engine_failed`,
539
+ or `english_only` for a Parakeet engine auto passed over for `--filler-lang`
540
+ that is not English; see `caption.py --transcribe` below).
537
541
  - `--transcribe` where the engine runs but its build produces no word-level
538
542
  timings → names that engine, says some builds do not support word timestamps,
539
543
  and points at `--words`. (`--transcribe` drives whichever engine is installed
@@ -546,7 +550,11 @@ Three refusals, all `kind: input`, all before any encode:
546
550
 
547
551
  Results: `filler.removed_count`, `filler.removed_seconds`, `filler.removed`
548
552
  (one entry per span), `filler.lang`, `filler.list`, `filler.word_timings` and
549
- `filler.warnings`. The existing `removed_seconds` keeps exactly the meaning it
553
+ `filler.warnings`. With `--transcribe`, `filler.source` is `whisper:ENGINE` or
554
+ `parakeet:ENGINE` and `filler.transcription` says which engine, model, language
555
+ and routing made the words; a top-level `notes` line says when `--engine auto`
556
+ ran the English-only Parakeet model on English assumed (no `--filler-lang`, and
557
+ no Whisper engine that could take it). The existing `removed_seconds` keeps exactly the meaning it
550
558
  has always had — the seconds of **silence** this run removed, the figure the
551
559
  same run would report without `--filler` — and `removed_seconds_total` is the
552
560
  additive sibling covering everything that went. A filler word quiet enough to
@@ -928,12 +936,38 @@ above the cap is clamped with a note, not refused, and both `jobs` and
928
936
  - `--watch` composes: each pass is parallel, passes are sequential.
929
937
 
930
938
  ### caption.py --transcribe — optional local speech-to-text
931
- If `whisper-cli` (whisper.cpp), `faster-whisper` or `whisper` is installed,
932
- `caption.py input.mp4 --transcribe [--language ja] [--model base]` writes the
933
- SRT from the audio and burns it (combine with `--animate pop --karaoke`).
934
- Nothing is downloaded and nothing is required: without an engine it prints
935
- install hints and the user can supply `--text` cues instead. Always tell the
936
- user which engine was used, and treat the transcript as a draft to review.
939
+ If `parakeet-mlx`, `parakeet-cli` (parakeet.cpp, with a `.gguf`), `whisper-cli` (whisper.cpp),
940
+ `faster-whisper` or `whisper` is installed,
941
+ `caption.py input.mp4 --transcribe [--language ja] [--model base] [--engine auto]` writes the
942
+ SRT from the audio and burns it (combine with `--animate pop --karaoke`). `--engine auto` (default,
943
+ or `FFMPEG_SKILL_ASR_ENGINE`) runs Parakeet for English speech -- an English `--language`, else
944
+ English from whisper.cpp's language detector -- and Whisper for any other language. When neither
945
+ can name the language, a Whisper engine that can run goes first; Parakeet runs on English
946
+ *assumed* only when no Whisper engine is installed or every one failed, and the result's
947
+ top-level `notes` then says so: that transcript is wrong for non-English speech, so pass `--language`
948
+ (`--filler-lang` for `silence.py`) when the speech may not be English.
949
+ `--engine parakeet-mlx|parakeet.cpp|whisper.cpp|faster-whisper|openai-whisper` forces one (a
950
+ missing one is `kind: missing_tool`; an installed one that fails is `kind: input`, `reason:
951
+ engine_failed`, with its own error line in `detail`). Without a language whisper.cpp is passed
952
+ `-l auto` (its own default is English), else the named or detected language. `--model` is the
953
+ whisper model; a Parakeet engine takes it only when it names a Parakeet model
954
+ (`mlx-community/parakeet-*` or a `.gguf`), else `PARAKEET_MODEL` / `PARAKEET_CPP_MODEL`, else the
955
+ English `tdt-0.6b-v2`. A non-English `--language` on an English-only Parakeet model is refused; a
956
+ multilingual (v3) model assumes no language, so it gets no English-assumed note. The result's
957
+ `transcription` names the engine, model and routing. `silence.py --filler --transcribe` takes
958
+ the same `--engine`.
959
+ Nothing is required: without an engine it prints install hints and the user can supply `--text`
960
+ cues instead. When engines are installed but none transcribed, the refusal names each and why
961
+ instead (`reason: engine_failed`, or `english_only` for a Parakeet engine auto passed over for
962
+ another language). The skill's own code downloads nothing, and every engine transcribes on this
963
+ machine -- the audio is never uploaded. An engine may fetch its *model weights* the
964
+ first time it runs: parakeet-mlx downloads `mlx-community/parakeet-tdt-0.6b-v2` (or the `--model` /
965
+ `PARAKEET_MODEL` repo) from Hugging Face into its cache; faster-whisper (also from Hugging Face)
966
+ and openai-whisper (from OpenAI) fetch theirs on first use too. faster-whisper is a library that
967
+ runs inside the skill's own process, so its download is made from that process.
968
+ `HF_HUB_OFFLINE=1` keeps the Hugging Face ones on the cached copy afterwards. parakeet.cpp and
969
+ whisper.cpp only read a model file you downloaded. Always tell the user which engine was used,
970
+ and treat the transcript as a draft to review.
937
971
 
938
972
  ### MCP server — the toolkit for any MCP client
939
973
  `python3 mcp/server.py` speaks MCP over stdio; each script is a tool taking