ffmpeg-skill 2.10.4 → 2.10.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -46,7 +46,7 @@ npx ffmpeg-skill
46
46
  </tr>
47
47
  </table>
48
48
 
49
- Left half is the input, right half is what the command produced. **[All 53 before/after demos, with the exact command under each one →](docs/demos.md)** — all of it generated from synthetic footage by `python3 demos/build.py`, so you can rebuild every frame of it yourself.
49
+ Left half is the input, right half is what the command produced. **[All 66 before/after demos, with the exact command under each one →](docs/demos.md)** — all of it generated from synthetic footage by `python3 demos/build.py`, so you can rebuild every frame of it yourself.
50
50
 
51
51
  `ffmpeg-skill` is an [Agent Skill](https://docs.anthropic.com/en/docs/agents-and-tools/agent-skills) for Claude Code, Cursor, Codex and any agent that reads `SKILL.md`. It teaches the agent a fixed workflow (probe → edit losslessly where possible → check → verify) and ships **42 tools** that do the actual work with `ffmpeg` / `ffprobe`: cut, join, silence removal, fit to duration and aspect, captions and karaoke, overlays and motion graphics, HDR → SDR and LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, whole-edit project rendering, batch folders. Every tool is also callable as an MCP tool (`tools/list` advertises a core 12 by default to keep client context small, with `FFMPEG_SKILL_MCP_FULL=1` listing all 42; every tool is reachable by name through `tools/call` either way), and the whole set is described by a machine-readable contract.
52
52
 
@@ -77,7 +77,7 @@ Other repos in the ecosystem — [`media-analysis-skill`](https://github.com/kaj
77
77
  ---
78
78
 
79
79
  **Contents**
80
- [Standalone, and in an ecosystem](#standalone-and-in-an-ecosystem) · [Why](#why) · [Quick start](#quick-start) · [How it works](#how-it-works) · [Design principles](#design-principles) · [Tools](#tools) · [Audio](#audio-is-a-first-class-input) · [Built for agents](#built-for-agents) · [FFmpeg compatibility](#ffmpeg-compatibility) · [Tested on real footage](#tested-on-real-footage) · [Install](#install) · [Requirements](#requirements) · [Development](#development) · [Docs](#docs)
80
+ [Standalone, and in an ecosystem](#standalone-and-in-an-ecosystem) · [Why](#why) · [Quick start](#quick-start) · [How it works](#how-it-works) · [Design principles](#design-principles) · [Tools](#tools) · [Audio](#audio-is-a-first-class-input) · [Built for agents](#built-for-agents) · [FAQ](#faq) · [FFmpeg compatibility](#ffmpeg-compatibility) · [Tested on real footage](#tested-on-real-footage) · [Install](#install) · [Requirements](#requirements) · [Development](#development) · [Docs](#docs)
81
81
 
82
82
  ---
83
83
 
@@ -217,7 +217,7 @@ These are the rules the skill file gives the agent and the code enforces.
217
217
  | `cropdetect.py` | Measure existing black letterbox/pillarbox bars and report the `crop.py`-ready rectangle that removes them — analysis only, writes no file; `--motion-centre` reports the per-second motion centroid instead, for a 9:16 reframe — report only, the calling agent picks the crop, no auto-reframing |
218
218
  | `deinterlace.py` | Deinterlace interlaced source footage (`yadif`), `--mode frame`/`field`, `--parity` |
219
219
  | `denoise.py` | Reduce video noise/grain (`hqdn3d`), `--strength low/medium/high` or individual spatial/temporal overrides |
220
- | `redact.py` | Blur or pixelate an exact pixel rectangle for the whole clip (privacy/compliance redaction) |
220
+ | `redact.py` | Blur, pixelate, or (`--mode delogo`) interpolate away an exact pixel rectangle for the whole clip. The box is given: nothing here finds a face, a plate or a logo, and `delogo` is a border blend, not inpainting |
221
221
  | `sphere.py` | Extract a flat rectilinear viewport from a 360/spherical video (`--yaw --pitch --roll --h-fov --v-fov`); no subject tracking, only the aim you give it |
222
222
  | `straighten.py` | Rotate by an arbitrary angle for horizon correction (`--degrees`, `--fit crop/pad`) — distinct from `fit.py --rotate`'s exact 90-degree turns |
223
223
  | `insert.py` | Turn a still image into a silent, fixed-duration video clip (title card, end slate) at an exact frame size / fps, with an optional Ken Burns zoom/pan |
@@ -389,6 +389,28 @@ The short list for humans. The agent-facing version, with the reasoning, is the
389
389
  - **Outputs are never overwritten silently.** An existing output path is refused (`kind: input`, before anything runs); pass `--overwrite` where a replacement is intended.
390
390
  - **Long chains belong in a plan.** Three hand-chained re-encodes lose quality and are hard to change; `render.py` runs the whole edit from one JSON file, and `--dry-run` shows every ffmpeg command before anything is written.
391
391
 
392
+ ## FAQ
393
+
394
+ ### Can I pass raw ffmpeg arguments, or a filter_complex?
395
+
396
+ Each operation is a script whose flags are its argparse parser, and that parser is what `contract --json` and the MCP `inputSchema` are built from. CI fails the build when those drift from the scripts. A caller-supplied filter graph, `-vf` string, or other ffmpeg option is not an argument any script accepts. `doctor` reads `ffmpeg -filters` (and `-encoders`, `-bsfs`) and reports a filter a tool needs before a job runs into it. MCP does accept an `argv` array, and that array is the named script's own CLI flags, still bound to that script: the script's flags, never an ffmpeg command line and never a shell.
397
+
398
+ ### How does this differ from Remotion or HyperFrames? Can they be used together?
399
+
400
+ Remotion renders a video from React components. HyperFrames renders one from HTML. This repository does neither, and it does not call either of them. It edits a media file you already have, on the machine, with ffmpeg and ffprobe, then probes the file it wrote. A render from either tool is an ordinary input: `probe.py`, `cut.py`, `fit.py`, `export.py` and `check.py` take the file the same way they take any other clip.
401
+
402
+ ### Can it remove burned-in subtitles or logos?
403
+
404
+ A rectangle you already know, sitting still for the whole clip, can be filled from the picture around it with `redact.py --mode delogo` (ffmpeg's `delogo` filter). That is a 1 px border blend. It is not inpainting: a logo that moves, or burned-in subtitles whose position changes, is out of scope. The same known box can be obscured instead, with `--mode blur` or `--mode pixelate`. None of the three finds the box. `delogo` also needs the box at least 1 px inside the frame, and it only interpolates 8-bit YUV, so an HDR or 10-bit source is refused until `color.py --to-sdr`.
405
+
406
+ ### Does it auto-track faces or subjects for reframing?
407
+
408
+ `fit.py --fit crop` centres the crop unless you pass `--crop-x` and `--crop-y`. `crop.py` cuts the rectangle you pass. `cropdetect.py` measures letterbox bars, and `--motion-centre` reports where motion sat each second; it writes no file and picks no crop. `sphere.py` points a 360 view where you aim it. `look.py` writes a contact sheet. Nothing in this repository detects a face, a product or a subject. Choosing what to keep in frame is the caller's.
409
+
410
+ ### Does it encode on the GPU?
411
+
412
+ The encode path a tool will actually take is Apple VideoToolbox, for h264, hevc and prores, and only on Apple Silicon. `--hw` asks for it on that call. `FFMPEG_SKILL_HW=1` makes it the default for every re-encoding tool except `export.py`'s delivery presets, which take `--hw` on their own (or `render.py --hw` for a whole project). AV1 stays on SVT-AV1. nvenc, QSV, VAAPI and AMF are reported by `doctor --json`'s `gpu_encoders` when the ffmpeg build was compiled with them, and no tool encodes with them. One build is the exception to opt-in: with no `libx264` and no `libx265`, and with `h264_videotoolbox` or `hevc_videotoolbox`, those encodes use VideoToolbox without `--hw` and the result says `hw.source` is `"auto"`. `--no-hw` still asks for the CPU encoder. A VideoToolbox job the hardware refuses is re-encoded on the CPU and the result says so (`hw.fallback`).
413
+
392
414
  ## FFmpeg compatibility
393
415
 
394
416
  The tools need FFmpeg 5.0 or later and Python 3.9 or later (standard library only). What CI actually exercises on every pull request is FFmpeg 5.1.1 (static build), 6.1 (Ubuntu apt), 7.1 (Debian trixie apt), 8.x (macOS Homebrew) and 9.x (Windows gyan.dev), on Python 3.9 and 3.13 (the two ends of the supported range). The capability parser has been run against the listings of these builds:
package/SKILL.md CHANGED
@@ -86,6 +86,7 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
86
86
  | "this old footage is interlaced" | `deinterlace.py input.mp4` |
87
87
  | "it's grainy/noisy, clean it up" | `denoise.py input.mp4 --strength medium` |
88
88
  | "blur/pixelate this face/plate" (known box) | `redact.py input.mp4 --x 820 --y 140 --width 240 --height 240 --mode pixelate` |
89
+ | "remove this fixed logo" (known box) | `redact.py input.mp4 --x 40 --y 20 --width 180 --height 60 --mode delogo` |
89
90
  | "stabilize this shaky footage" | `stabilize.py input.mp4` |
90
91
  | "are there black bars on this?" | `cropdetect.py input.mp4` |
91
92
  | "iPhone Dolby Vision clip looks wrong" | `color.py clip.mov --to-sdr` or `--strip-dovi` (keep HDR, drop the DV layer) |
@@ -196,9 +197,9 @@ Notes: send a valid .cube, or say if you want the clip left as is
196
197
  Those filler lines are sentences, not labels: the same report for a Japanese request ends
197
198
  `Check: 検証するものなし` / `Look: 不要`.
198
199
 
199
- A refusal uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. The shortest failure gets all five labels, never headings. A partial result is `Done:` with the shortfall in `Notes:`; a refusal that still delivers something is `Failed:` — never a third label like `Done (partially):`. Quote a failure's `error.hint` in `Notes:` — the flag change a retry needs.
200
+ A refusal uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. A partial result is `Done:` with the shortfall in `Notes:`; a refusal that still delivers something is `Failed:` — never a third label like `Done (partially):`. Quote a failure's `error.hint` in `Notes:`.
200
201
 
201
- Every script prints `{"status": "failed", "error": {"kind": input | ffmpeg | output | missing_tool | timeout | verification | interrupted, "message": ...}}` with `--json` and exits non-zero; quote the message, never paraphrase it.
202
+ A failure's `--json` is `status: failed` plus `error.kind` and `error.message`; quote the message, never paraphrase it.
202
203
 
203
204
  ## Things that look right but are wrong
204
205
 
package/bin/install.js CHANGED
@@ -41,21 +41,39 @@ if (has('--help') || has('-h')) {
41
41
  process.exit(0);
42
42
  }
43
43
 
44
- // `contract` / `doctor` are answered by scripts/_contract.py; everything else installs.
45
- if (args[0] === 'contract' || args[0] === 'doctor') {
46
- // Windows Python installers commonly expose `python`/`py`, not `python3` (only the
47
- // Microsoft Store package does); try python3 first (macOS/Linux convention), then fall
48
- // back so `npx ffmpeg-skill doctor` doesn't silently fail with ENOENT on Windows.
49
- const candidates = process.platform === 'win32' ? ['python3', 'python', 'py'] : ['python3'];
50
- let py;
44
+ // Windows Python installers commonly expose `python`/`py`, not `python3`. The Microsoft Store
45
+ // app-execution alias is a real python3.exe that spawns (no ENOENT) and exits non-zero, so a
46
+ // missing `error` is not proof an interpreter ran. Probe with a version check, then fall through.
47
+ const PYTHON_PROBE = ['-c', 'import sys; raise SystemExit(0 if sys.version_info >= (3, 9) else 1)'];
48
+
49
+ function pythonCandidates(platform) {
50
+ return platform === 'win32' ? ['python3', 'python', 'py'] : ['python3'];
51
+ }
52
+
53
+ function runWithPython(scriptArgs, platform, spawn) {
54
+ const candidates = pythonCandidates(platform);
55
+ const run = spawn || spawnSync;
56
+ let last = null;
51
57
  for (const cmd of candidates) {
52
- py = spawnSync(cmd, [path.join(ROOT, 'scripts', '_contract.py'), ...args], { stdio: 'inherit' });
53
- if (!py.error) break;
58
+ const probe = run(cmd, PYTHON_PROBE, { stdio: 'ignore', timeout: 15000 });
59
+ if (probe.error || probe.status !== 0) {
60
+ last = probe;
61
+ continue;
62
+ }
63
+ return run(cmd, scriptArgs, { stdio: 'inherit' });
54
64
  }
55
- if (py.error) {
56
- console.error(`error: could not find a Python interpreter (tried: ${candidates.join(', ')}). Install Python 3.9+ and ensure it is on PATH.`);
65
+ return { missing: true, error: last && last.error, status: last && last.status, candidates };
66
+ }
67
+
68
+ function main() {
69
+ // `contract` / `doctor` are answered by scripts/_contract.py; everything else installs.
70
+ if (args[0] === 'contract' || args[0] === 'doctor') {
71
+ const py = runWithPython([path.join(ROOT, 'scripts', '_contract.py'), ...args], process.platform, spawnSync);
72
+ if (py.missing) {
73
+ console.error(`error: could not find a Python 3.9+ interpreter (tried: ${py.candidates.join(', ')}). Install Python 3.9+ and ensure it is on PATH.`);
74
+ process.exit(127);
57
75
  }
58
- process.exit(py.error ? 127 : py.status);
76
+ process.exit(py.error ? 127 : (py.status == null ? 1 : py.status));
59
77
  }
60
78
 
61
79
  const home = os.homedir();
@@ -159,3 +177,10 @@ if (!has('--uninstall')) {
159
177
  console.log('\nDone. Ask your agent things like "make this video 60 seconds and add captions".');
160
178
  }
161
179
  process.exit(failed ? 1 : 0);
180
+ }
181
+
182
+ if (require.main === module) {
183
+ main();
184
+ }
185
+
186
+ module.exports = { pythonCandidates, runWithPython };
package/docs/contract.md CHANGED
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
21
21
  | Field | Meaning | Changes when |
22
22
  |---|---|---|
23
23
  | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
24
- | `skill.version` | the npm / package.json version (`2.10.4`) | any release |
24
+ | `skill.version` | the npm / package.json version (`2.10.6`) | any release |
25
25
 
26
26
  A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
27
27
  ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -119,7 +119,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
119
119
  "contract_version": "1.0",
120
120
  "deprecated": [],
121
121
  "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
122
- "skill": {"id": "ffmpeg-skill", "version": "2.10.4", "execution_mode": "local", "kind": "execution",
122
+ "skill": {"id": "ffmpeg-skill", "version": "2.10.6", "execution_mode": "local", "kind": "execution",
123
123
  "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
124
124
  "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
125
125
  "requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
@@ -384,6 +384,12 @@ environment variables (`doctor`'s `hw.default_on` is a boolean derived from `FFM
384
384
  never its value). The two binary paths can sit under a home directory, so read a pasted `doctor`
385
385
  before posting it.
386
386
 
387
+ `filter:delogo` is optional on `redact` (`--mode delogo`). The filter fills a fixed
388
+ rectangle from a 1 px ring of the surrounding picture. It is a border blend,
389
+ not inpainting, and its pixel formats are 8-bit YUV, so an HDR or 10-bit
390
+ source is refused (`color.py --to-sdr` first). `filter:boxblur` stays required
391
+ on `redact`, because `--mode blur` is the default.
392
+
387
393
  `external:whisper` is **optional** for two tools since 1.17: `caption.py`
388
394
  (`--transcribe`) and `silence.py` (`--filler --transcribe`). Neither requires
389
395
  it — both take a transcript the caller already has (`--srt`/`--words`), and
package/mcp/server.py CHANGED
@@ -21,7 +21,7 @@ import os
21
21
  import subprocess
22
22
  import sys
23
23
  from pathlib import Path
24
- from typing import Any, Dict, List
24
+ from typing import Any, Dict, List, Optional
25
25
 
26
26
  HERE = Path(__file__).resolve().parent
27
27
  SCRIPTS = HERE.parent / "scripts"
@@ -90,18 +90,66 @@ def build_argv(name: str, args: Dict[str, Any]) -> List[str]:
90
90
  return argv
91
91
 
92
92
 
93
+ def _timeout_in_argv(argv: List[str]) -> Optional[float]:
94
+ """The last `--timeout` / `--timeout=N` value in a raw argv list.
95
+
96
+ None when the flag is absent or its value is not a number (the child argparse then fails,
97
+ and the outer ceiling stays the environment default)."""
98
+ found: Optional[float] = None
99
+ seen = False
100
+ i = 0
101
+ n = len(argv)
102
+ while i < n:
103
+ token = argv[i]
104
+ if token == "--":
105
+ break
106
+ if token == "--timeout":
107
+ if i + 1 >= n:
108
+ return None
109
+ raw = argv[i + 1]
110
+ i += 2
111
+ elif token.startswith("--timeout="):
112
+ raw = token.split("=", 1)[1]
113
+ i += 1
114
+ else:
115
+ i += 1
116
+ continue
117
+ seen = True
118
+ try:
119
+ found = float(raw)
120
+ except ValueError:
121
+ return None
122
+ return found if seen else None
123
+
124
+
125
+ def timeout_from_call(args: Dict[str, Any]) -> float:
126
+ """Per-call seconds for the MCP outer ceiling.
127
+
128
+ Structured `timeout` is the canonical key. A raw `argv` list ignores every other key, so
129
+ `--timeout` has to be read from that list: `--timeout 0` otherwise left the ceiling at 4x
130
+ the 1800 s default while the child itself ran with no limit. A negative value is 0, matching
131
+ apply_common (0 means no limit)."""
132
+ if isinstance(args.get("argv"), list):
133
+ parsed = _timeout_in_argv([str(a) for a in args["argv"]])
134
+ if parsed is None:
135
+ return _common._env_timeout()
136
+ return max(0.0, parsed)
137
+ try:
138
+ return max(0.0, float(args.get("timeout", _common._env_timeout())))
139
+ except (TypeError, ValueError):
140
+ return _common._env_timeout()
141
+
142
+
93
143
  def call_tool(name: str, args: Dict[str, Any]) -> Dict[str, Any]:
94
144
  script = SCRIPTS / f"{name}.py"
95
145
  if name not in specs() or not script.exists():
96
146
  return {"isError": True, "content": [{"type": "text", "text": f"unknown tool {name}"}]}
97
147
  argv = build_argv(name, args or {})
98
148
  # The child enforces --timeout on each ffmpeg call; this outer ceiling (4x that plus 60 s)
99
- # is the only thing that ends a child hung for any other reason. The caller's own `timeout`
100
- # argument sets both; otherwise FFMPEG_SKILL_TIMEOUT / the 1800 s default.
101
- try:
102
- per_call = float((args or {}).get("timeout", _common._env_timeout()))
103
- except (TypeError, ValueError):
104
- per_call = _common._env_timeout()
149
+ # is the only thing that ends a child hung for any other reason. Structured `timeout` or
150
+ # `--timeout` inside a raw argv list sets both (0 = no limit); otherwise
151
+ # FFMPEG_SKILL_TIMEOUT / the 1800 s default.
152
+ per_call = timeout_from_call(args or {})
105
153
  proc = _common.run_tool([str(script)] + argv, per_call=per_call)
106
154
  stdout = proc.stdout.strip()
107
155
  text = stdout
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ffmpeg-skill",
3
- "version": "2.10.4",
3
+ "version": "2.10.6",
4
4
  "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
5
5
  "keywords": [
6
6
  "ffmpeg",
@@ -91,7 +91,16 @@ max(2 frames, 0.1 s) away from the source's own offset where the picture starts
91
91
  (the keyframe the copy began from; `--start` under `--edit-list`). It is
92
92
  reported, not repaired. `-ss`/`-t` are passed to the microsecond; `--accurate`
93
93
  seeks a second early and trims on the output side, so frames just before a
94
- keyframe are kept. `keyframe_snapped` is true for every stream copy (`precision`
94
+ keyframe are kept. An `--accurate` re-encode of a video does not pass
95
+ `-avoid_negative_ts make_zero`, which shifted the picture by the AAC priming
96
+ (about 21 ms, more when a B-frame decode time is negative) while the sound
97
+ stayed at 0, and `fit.py --fps` then changed the frame count. A start that
98
+ lands on a frame leaves that frame at time 0. A start between frames begins on
99
+ the next source frame and may keep up to one frame of timestamp. A `.mp4`/`.mov`
100
+ edit list hides the priming. A source whose audio starts after its picture keeps
101
+ that offset. An audio-only `--accurate` cut
102
+ still uses `make_zero`; that priming is what `precision: codec_frame` counts
103
+ in the length. `keyframe_snapped` is true for every stream copy (`precision`
95
104
  `packet`); `start_snapped` is measured: true when a copied picture starts more
96
105
  than a frame from `--start`, or when its start could not be measured (an unknown start is
97
106
  never claimed exact); `null` under `--dry-run`.
@@ -272,18 +281,25 @@ Heavier denoising trades fine detail for a cleaner but softer image -- for
272
281
  audio noise reduction use `audio.py --denoise` instead, this tool only
273
282
  touches the picture.
274
283
 
275
- ### redact.py — blur or pixelate an exact rectangle
284
+ ### redact.py — blur, pixelate, or interpolate away an exact rectangle
276
285
  ```
277
- redact.py INPUT --x X --y Y --width W --height H [--mode blur|pixelate]
286
+ redact.py INPUT --x X --y Y --width W --height H [--mode blur|pixelate|delogo]
278
287
  [--blur-strength N] [--block-size N] [-o OUT]
279
288
  ```
280
289
  Same rectangle convention as `crop.py` -- the rectangle must already be
281
290
  known (a saved detection box, a hand-picked region); this tool does not
282
- locate faces or plates itself. `--mode blur` (default) box-blurs the
283
- rectangle; `--mode pixelate` mosaics it into `--block-size`-px blocks, the
284
- more unmistakably-redacted look often wanted for compliance footage. The
285
- rest of the frame, and the whole clip's timeline, are untouched -- to
286
- redact only part of the timeline, `cut.py` the clip into segments first.
291
+ locate faces, plates, logos or burned-in text itself. `--mode blur`
292
+ (default) box-blurs the rectangle; `--mode pixelate` mosaics it into
293
+ `--block-size`-px blocks, the more unmistakably-redacted look often wanted
294
+ for compliance footage. `--mode delogo` runs ffmpeg's `delogo` filter,
295
+ which fills the rectangle from the picture around it: a fixed logo or a
296
+ burned-in line that does not move. That is a border blend, not inpainting,
297
+ and a logo that travels or subtitles that change position are out of
298
+ scope. The filter reads a 1 px ring outside the box, so the box cannot
299
+ touch the frame edge, and it only interpolates 8-bit YUV (an HDR or 10-bit
300
+ source is refused; `color.py --to-sdr` first). The rest of the frame, and
301
+ the whole clip's timeline, are untouched -- to cover only part of the
302
+ timeline, `cut.py` the clip into segments first.
287
303
 
288
304
  ### sphere.py — flat viewport extraction from 360/spherical video
289
305
  ```
@@ -739,10 +755,13 @@ An `overlays[]` entry draws one of `"logo": true`, `"image"`, `"text"` or
739
755
  `"video"` (#326: a picture-in-picture or green-screen layer, `overlay.py
740
756
  --video`), with `position`, `scale`, `opacity`, `start`, `end`, `margin`,
741
757
  `platform` (and `fade`, `box`, `font*` for the still/text kinds) mapping to
742
- overlay.py's flags. `"chromakey": "0x00ff00"`, `"chromakey_similarity"` and
743
- `"chromakey_blend"` key a colour out of the video layer. The stage runs after
744
- `fit`, so `"scale": 480` is 480 px of the delivered frame, and a template's
745
- safe margins apply as they do to a logo. Refused (`kind: input`) before the
758
+ overlay.py's flags. `scale` is the layer width in px for a logo, image or
759
+ video (`"scale": 480` is 480 px of the delivered frame, because the stage
760
+ runs after `fit`). A text entry's size is `font_size`; `scale` on a text-only
761
+ entry is refused (`kind: input`) rather than drawn at the default font size.
762
+ `"chromakey": "0x00ff00"`, `"chromakey_similarity"` and
763
+ `"chromakey_blend"` key a colour out of the video layer. A template's
764
+ safe margins apply as they do to a logo. Also refused before the
746
765
  first stage: `video` together with `logo`/`image`/`text` in one entry (a
747
766
  logo, image and text together still draw the first of them, as before), a
748
767
  `chromakey*` key without `video`, `chromakey_similarity`/`_blend` without
@@ -1448,7 +1467,9 @@ overlay.py INPUT --image PNG [--scale W | --scale-percent P] | --text "..." [--f
1448
1467
  (`scripts/_platforms.py`), so a template's top-left logo clears TikTok's status bar instead of
1449
1468
  sitting 24 px into it; an explicit `--margin` (or a brand `safe_margin`) wins.
1450
1469
  Alpha in PNGs is respected. Fades apply to the overlay only; the video keeps
1451
- playing. `--video` composites a second video as a picture-in-picture layer
1470
+ playing. `--scale` / `--scale-percent` set the width of an image, logo or
1471
+ `--video` layer. They are refused with `--text`: text size is `--font-size`.
1472
+ `--video` composites a second video as a picture-in-picture layer
1452
1473
  (same position/scale/opacity/time-range knobs as `--image`); only the main
1453
1474
  input's audio is kept, the PiP layer's own audio is dropped. `--chromakey`
1454
1475
  (with `--video`) keys out that colour first for green-screen compositing.
@@ -119,10 +119,13 @@ WINDOWS_FONTS = {
119
119
  "ja": [("meiryo.ttc", "Meiryo"), ("YuGothM.ttc", "Yu Gothic Medium"), ("YuGothR.ttc", "Yu Gothic"), ("msgothic.ttc", "MS Gothic")],
120
120
  "ar": [("tahoma.ttf", "Tahoma"), ("arial.ttf", "Arial")],
121
121
  "he": [("tahoma.ttf", "Tahoma"), ("arial.ttf", "Arial")],
122
- "hi": [("mangal.ttf", "Mangal"), ("Nirmala.ttf", "Nirmala UI"), ("NirmalaB.ttf", "Nirmala UI")],
122
+ # Windows 11 ships Nirmala UI as the collection Nirmala.ttc; there is no Nirmala.ttf (#339).
123
+ # The .ttf names stay first so a machine that still has them (older Windows, a copied font)
124
+ # keeps that file. Mangal, when present, stays the Devanagari face it always was.
125
+ "hi": [("mangal.ttf", "Mangal"), ("Nirmala.ttf", "Nirmala UI"), ("Nirmala.ttc", "Nirmala UI"), ("NirmalaB.ttf", "Nirmala UI")],
123
126
  "th": [("leelawui.ttf", "Leelawadee UI"), ("leelawad.ttf", "Leelawadee"), ("tahoma.ttf", "Tahoma")],
124
- "bn": [("Nirmala.ttf", "Nirmala UI"), ("vrinda.ttf", "Vrinda")],
125
- "ta": [("Nirmala.ttf", "Nirmala UI"), ("latha.ttf", "Latha")],
127
+ "bn": [("Nirmala.ttf", "Nirmala UI"), ("Nirmala.ttc", "Nirmala UI"), ("vrinda.ttf", "Vrinda")],
128
+ "ta": [("Nirmala.ttf", "Nirmala UI"), ("Nirmala.ttc", "Nirmala UI"), ("latha.ttf", "Latha")],
126
129
  "lo": [("leelawui.ttf", "Leelawadee UI"), ("DokChamp.ttf", "DokChampa")],
127
130
  "ru": [("arial.ttf", "Arial"), ("segoeui.ttf", "Segoe UI")],
128
131
  "el": [("arial.ttf", "Arial"), ("segoeui.ttf", "Segoe UI")],
@@ -98,8 +98,9 @@ TOOL_META: Dict[str, Dict[str, Any]] = {
98
98
  "cropdetect": dict(role="analysis", inputs=["video asset"], outputs=["detected crop rectangle JSON on stdout (no file)"],
99
99
  required=FF + ["filter:cropdetect"], optional=[],
100
100
  video_required=True, audio_only=False, visual=False, verify=[], produces_artifact=False, idempotency="environment_dependent", deterministic=False),
101
- "redact": dict(role="execution", inputs=["video asset"], outputs=["video artifact with the given pixel rectangle blurred or pixelated"],
102
- required=FF + [X264, AAC, "filter:boxblur"], optional=[HDR_X265],
101
+ "redact": dict(role="execution", inputs=["video asset"], outputs=["video artifact with the given pixel rectangle blurred, pixelated, or interpolated away (--mode delogo)"],
102
+ required=FF + [X264, AAC, "filter:boxblur"],
103
+ optional=[HDR_X265, {"capability": "filter:delogo", "when": "--mode delogo"}],
103
104
  video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
104
105
  "waveform": dict(role="execution", inputs=["audio or video asset (audio track only)"], outputs=["generated waveform or spectrum visualization video artifact"],
105
106
  required=FF + [X264, AAC, "filter:showwaves"], optional=[{"capability": "filter:showspectrum", "when": "--style spectrum"}],
@@ -261,7 +262,7 @@ REENCODE_META: Dict[str, Dict[str, str]] = {
261
262
  "deinterlace": dict(video="always", audio="always", note="the yadif filter always forces a re-encode of the video stream; audio is re-encoded to AAC when present"),
262
263
  "denoise": dict(video="always", audio="always", note="the hqdn3d filter always forces a re-encode of the video stream; audio is re-encoded to AAC when present"),
263
264
  "cropdetect": dict(video="never", audio="never", note="analysis only, no artifact"),
264
- "redact": dict(video="always", audio="always", note="the boxblur/pixelate filter_complex always forces a re-encode of the video stream; audio is re-encoded to AAC when present"),
265
+ "redact": dict(video="always", audio="always", note="the boxblur, pixelate or delogo graph always forces a re-encode of the video stream; audio is re-encoded to AAC when present"),
265
266
  "waveform": dict(video="always", audio="always", note="always encodes a fresh generated visualization clip; the source audio is always re-encoded to AAC"),
266
267
  "sphere": dict(video="always", audio="always", note="the v360 filter always forces a re-encode of the video stream; audio is passed through to AAC unchanged"),
267
268
  "straighten": dict(video="always", audio="always", note="the rotate filter always forces a re-encode of the video stream; audio is re-encoded to AAC when present"),
package/scripts/cut.py CHANGED
@@ -22,6 +22,12 @@ caller never has to trust "it probably cut where I asked" on faith.
22
22
  A single-segment .mp4/.mov copy shifts its timestamps to zero (-avoid_negative_ts
23
23
  make_zero), so the picture starts at the keyframe; --edit-list keeps the MP4 edit
24
24
  list instead, which hides the keyframe's pre-roll so the picture starts at --start.
25
+ An --accurate re-encode of a video does not: make_zero moved the picture by the
26
+ AAC priming (about 21 ms, and further when a B-frame's decode time is negative)
27
+ while the sound stayed at 0. A start that lands on a frame keeps that frame at 0.
28
+ A start between frames begins on the next frame and may keep up to one frame of
29
+ timestamp. The MP4 edit list hides the encoder priming, and a source whose audio
30
+ starts late keeps that offset.
25
31
 
26
32
  A re-encode uses x264 for an SDR source and x265 Main10 for an HDR or BT.2020 one, as every
27
33
  editing tool does; --keep-hevc re-encodes an SDR HEVC source as HEVC (x265 8-bit BT.709, or
@@ -823,7 +829,19 @@ def cut_one(src: str, start: float, end: float, dst: str, reencode: bool, crf: i
823
829
  cmd += ["-sn", "-dn"]
824
830
  if meta.get("subtitle_streams") or meta.get("data_streams"):
825
831
  DROPPED_STREAMS.append(dst)
826
- cmd += encode_args(meta, dst, crf, preset) + ["-avoid_negative_ts", "make_zero", dst]
832
+ cmd += encode_args(meta, dst, crf, preset)
833
+ if audio_only:
834
+ # asetpts already starts the samples at 0. make_zero is what folds the encoder's
835
+ # priming into the container length, which is the delay precision codec_frame reports.
836
+ # A video re-encode must not take it (#336): the earliest timestamp is that priming,
837
+ # or a negative B-frame decode time, so make_zero starts the picture after 0 and
838
+ # leaves the sound at 0. fit --fps then drops or repeats a frame. Without the shift
839
+ # a start on a frame has that frame at 0. A start between frames begins on the next
840
+ # frame and may keep up to one frame of timestamp; moving the picture alone would
841
+ # leave the sound that far ahead. An MP4/MOV edit list hides the priming, and a
842
+ # source whose audio starts late keeps that offset.
843
+ cmd += ["-avoid_negative_ts", "make_zero"]
844
+ cmd += [dst]
827
845
  elif audio_only:
828
846
  # output-side seek: an input seek on a video file lands on the previous video keyframe and on
829
847
  # FLAC/MP3 on a coarse index; reading from the start and dropping packets is exact to the packet
package/scripts/fit.py CHANGED
@@ -198,7 +198,7 @@ def main() -> int:
198
198
  darken = 0.0 if meta["video"].get("bt2020_or_hdr") else BLUR_DARKEN
199
199
  if not darken:
200
200
  info("--fit blur: HDR source, so the blurred background is not dimmed "
201
- "(an eq on PQ/HLG code values is not the -15%% dim it is on SDR); "
201
+ "(an eq on PQ/HLG code values is not the -15% dim it is on SDR); "
202
202
  "run color.py --to-sdr first for the SDR look")
203
203
  vf.append(pad_filters(out_w, out_h, "blur", args.pad_color, args.pad_blur, darken))
204
204
  else:
@@ -116,8 +116,8 @@ def main() -> int:
116
116
  ap.add_argument("--fade", type=float, default=0.0, help="fade-in duration in seconds (at --start or 0); the fade-out happens only at --end")
117
117
  ap.add_argument("--opacity", type=float, default=1.0, help="0..1 (default 1)")
118
118
  img = ap.add_argument_group("image options")
119
- img.add_argument("--scale", type=int, help="scale the image to this width in px (keeps aspect)")
120
- img.add_argument("--scale-percent", type=float, help="scale the image to this %% of the video width")
119
+ img.add_argument("--scale", type=int, help="width in px for an image, logo or video layer (keeps aspect); text size is --font-size")
120
+ img.add_argument("--scale-percent", type=float, help="width as a %% of the video for an image, logo or video layer (keeps aspect); text size is --font-size")
121
121
  txt = ap.add_argument_group("text options")
122
122
  txt.add_argument("--font", default="DejaVu Sans", help="fontconfig font name")
123
123
  txt.add_argument("--font-file", help="explicit .ttf/.otf/.ttc path (use this for CJK fonts)")
@@ -158,6 +158,10 @@ def main() -> int:
158
158
  die("--text is blank -- give the words to draw", kind="input")
159
159
  if not (args.image or args.text or args.video):
160
160
  die("give --image, --text, --logo or --video")
161
+ # --scale is a layer width. On a text draw it used to be accepted and ignored (fontsize stayed
162
+ # at --font-size), so a project that asked for 200 px text completed with the 42 px default.
163
+ if args.text and not (args.image or args.video) and (args.scale is not None or args.scale_percent is not None):
164
+ die("--scale and --scale-percent size an image, logo or video layer; text size is --font-size", kind="input")
161
165
  if args.brand:
162
166
  if args.margin == ap.get_default("margin"):
163
167
  args.margin = int(brand.get("safe_margin", args.margin))
package/scripts/redact.py CHANGED
@@ -1,21 +1,28 @@
1
1
  #!/usr/bin/env python3
2
- """Blur or pixelate an exact pixel rectangle for the whole clip (privacy redaction, license plates, faces).
2
+ """Blur, pixelate, or interpolate away an exact pixel rectangle for the whole clip.
3
3
 
4
4
  {x, y, width, height} are literal pixel offsets and dimensions in the SOURCE
5
5
  frame, the same convention as crop.py -- this tool needs the rectangle
6
6
  already known (a saved detection box, a hand-picked region); it does not
7
- locate faces or plates itself. The rest of the frame is untouched.
7
+ locate faces, plates, logos or burned-in text itself. The rest of the frame
8
+ is untouched.
8
9
 
9
10
  --mode blur (default) applies a strong box blur inside the rectangle;
10
11
  --mode pixelate mosaics it into large blocks -- the more recognisable,
11
12
  unmistakably-redacted look often wanted for compliance/legal footage.
12
- The region stays --mode blur/pixelate for the whole clip; for a region that
13
- only needs covering part of the timeline, cut the clip into segments first
14
- (cut.py) and redact only the relevant one.
13
+ --mode delogo fills the rectangle from the picture around it (ffmpeg's
14
+ delogo filter): a fixed logo or burned-in line, not inpainting, and not a
15
+ region that moves. delogo reads a 1 px ring outside the box, so the box
16
+ cannot touch the frame edge, and it only interpolates 8-bit YUV (an HDR or
17
+ 10-bit source is refused; color.py --to-sdr first).
18
+ The region stays one mode for the whole clip; for a region that only needs
19
+ covering part of the timeline, cut the clip into segments first (cut.py)
20
+ and redact only the relevant one.
15
21
 
16
22
  Examples:
17
23
  python3 redact.py interview.mp4 --x 820 --y 140 --width 240 --height 240
18
24
  python3 redact.py dashcam.mp4 --x 0 --y 900 --width 400 --height 120 --mode pixelate --block-size 16
25
+ python3 redact.py branded.mp4 --x 40 --y 20 --width 180 --height 60 --mode delogo
19
26
  """
20
27
  import argparse
21
28
  import sys
@@ -31,7 +38,8 @@ def main() -> int:
31
38
  ap.add_argument("--y", type=int, required=True, help="top edge of the rectangle, in source pixels")
32
39
  ap.add_argument("--width", type=int, required=True, help="rectangle width in px (must be even)")
33
40
  ap.add_argument("--height", type=int, required=True, help="rectangle height in px (must be even)")
34
- ap.add_argument("--mode", choices=["blur", "pixelate"], default="blur", help="blur (default) or pixelate the rectangle")
41
+ ap.add_argument("--mode", choices=["blur", "pixelate", "delogo"], default="blur",
42
+ help="blur (default), pixelate, or delogo (interpolate a fixed rectangle away; not inpainting, and not a moving region)")
35
43
  ap.add_argument("--blur-strength", type=int, default=20, help="box-blur radius in px, --mode blur only (default 20)")
36
44
  ap.add_argument("--block-size", type=int, default=12, help="mosaic block size in px, --mode pixelate only (default 12)")
37
45
  ap.add_argument("--audio-stream", type=int, default=0,
@@ -64,6 +72,17 @@ def main() -> int:
64
72
  sw, sh = sh, sw
65
73
  if args.x + args.width > sw or args.y + args.height > sh:
66
74
  die(f"redaction rectangle ({args.x},{args.y},{args.width}x{args.height}) exceeds the source frame ({sw}x{sh})")
75
+ if args.mode == "delogo":
76
+ # delogo's band is fixed at 1 px (the option was removed; 5.1 through current all
77
+ # expand the box by that ring and refuse it when the ring leaves the frame).
78
+ vinfo = meta["video"]
79
+ if vinfo.get("bt2020_or_hdr") or (vinfo.get("bit_depth") or 8) > 8:
80
+ die(f"--mode delogo interpolates 8-bit YUV only (this source is {vinfo.get('pix_fmt') or 'not 8-bit'}); "
81
+ "color.py --to-sdr first")
82
+ if args.x < 1 or args.y < 1 or args.x + args.width >= sw or args.y + args.height >= sh:
83
+ die(f"--mode delogo needs a 1 px ring of picture around the rectangle "
84
+ f"({args.x},{args.y},{args.width}x{args.height} in {sw}x{sh}); "
85
+ "a logo on the frame edge is out of range")
67
86
  has_audio = bool(meta.get("audio"))
68
87
  audio_streams = meta.get("audio_streams") or []
69
88
  if audio_streams and not (0 <= args.audio_stream < len(audio_streams)):
@@ -72,17 +91,22 @@ def main() -> int:
72
91
  die("--audio-stream needs an input with audio streams")
73
92
 
74
93
  output = args.output or default_output(args.input, "redact")
75
- crop = f"crop={args.width}:{args.height}:{args.x}:{args.y}"
76
- if args.mode == "blur":
77
- # boxblur refuses a radius above half the plane: the chroma planes of 4:2:0 are half-size,
78
- # so they get their own (halved) radius; a 30 px region with the default 20 used to fail
79
- radius = max(1, min(args.blur_strength, min(args.width, args.height) // 2 - 1))
80
- chroma = max(1, min(radius // 2, min(args.width, args.height) // 4 - 1))
81
- region = f"{crop},boxblur={radius}:{radius}:{chroma}:{radius}"
94
+ if args.mode == "delogo":
95
+ cmd = ffmpeg_base() + ["-i", args.input, "-vf",
96
+ f"delogo=x={args.x}:y={args.y}:w={args.width}:h={args.height}",
97
+ "-map", "0:v:0"]
82
98
  else:
83
- region = f"{crop},scale={max(1, args.width // args.block_size)}:{max(1, args.height // args.block_size)}:flags=neighbor,scale={args.width}:{args.height}:flags=neighbor"
84
- fc = f"[0:v]split=2[base][region];[region]{region}[patched];[base][patched]overlay={args.x}:{args.y}[out]"
85
- cmd = ffmpeg_base() + ["-i", args.input, "-filter_complex", fc, "-map", "[out]"]
99
+ crop = f"crop={args.width}:{args.height}:{args.x}:{args.y}"
100
+ if args.mode == "blur":
101
+ # boxblur refuses a radius above half the plane: the chroma planes of 4:2:0 are half-size,
102
+ # so they get their own (halved) radius; a 30 px region with the default 20 used to fail
103
+ radius = max(1, min(args.blur_strength, min(args.width, args.height) // 2 - 1))
104
+ chroma = max(1, min(radius // 2, min(args.width, args.height) // 4 - 1))
105
+ region = f"{crop},boxblur={radius}:{radius}:{chroma}:{radius}"
106
+ else:
107
+ region = f"{crop},scale={max(1, args.width // args.block_size)}:{max(1, args.height // args.block_size)}:flags=neighbor,scale={args.width}:{args.height}:flags=neighbor"
108
+ fc = f"[0:v]split=2[base][region];[region]{region}[patched];[base][patched]overlay={args.x}:{args.y}[out]"
109
+ cmd = ffmpeg_base() + ["-i", args.input, "-filter_complex", fc, "-map", "[out]"]
86
110
  if has_audio:
87
111
  cmd += ["-map", f"0:a:{args.audio_stream}?"]
88
112
  cmd += video_args(meta, args.crf, args.preset)
package/scripts/render.py CHANGED
@@ -40,7 +40,9 @@ skipped. "brand" points caption/graphics/overlay at a brand.json (fonts,
40
40
  colours, logo, safe margin); {"logo": true} in overlays places the brand logo.
41
41
  An overlay entry draws one of logo, image, text or video (a picture-in-picture
42
42
  layer, overlay.py --video; "chromakey", "chromakey_similarity" and
43
- "chromakey_blend" key a colour out of that video and need "video").
43
+ "chromakey_blend" key a colour out of that video and need "video"). "scale"
44
+ is that layer's width in px for a logo, image or video; a text entry's size
45
+ is "font_size", and "scale" on a text entry is refused.
44
46
 
45
47
  "check" mirrors check.py's own exit code: a delivery-spec FAIL (or check.py
46
48
  itself failing to run) exits 1, same as running check.py directly would --
@@ -436,6 +438,10 @@ def check_overlay(ov: Any, label: str) -> None:
436
438
  tuned = [k for k in ("chromakey_similarity", "chromakey_blend") if ov.get(k) is not None]
437
439
  if tuned:
438
440
  die(f"{label}: {tuned[0]} tunes \"chromakey\", which this entry does not set")
441
+ # scale is a layer width (overlay.py --scale). A text entry that carries it used to complete
442
+ # with the default font size, because the text branch never reads the flag.
443
+ if ov.get("text") and not any(ov.get(k) for k in ("logo", "image", "video")) and ov.get("scale") is not None:
444
+ die(f"{label}: scale sizes an image, logo or video layer; text size is font_size", kind="input")
439
445
 
440
446
 
441
447
  def require_object(obj: Any, label: str) -> None:
@@ -84,9 +84,14 @@ def main() -> int:
84
84
  output = args.output or default_output(args.input, "ramp")
85
85
 
86
86
  vparts, aparts, labels = [], [], []
87
+ # The coverage check above allows the last end to sit up to 0.5 s off the probed duration.
88
+ # That slack is for a caller who rounded the duration, not a reason to redraw the tail at
89
+ # the last factor. The end bound is omitted only when the stated end is the file's end
90
+ # (within 50 ms): a trim a few milliseconds short of the last packet drops that packet.
91
+ last_runs_to_eof = abs(segments[-1][1] - dur) <= 0.05
87
92
  for i, (start, end, factor) in enumerate(segments):
88
93
  vlabel, alabel = f"v{i}", f"a{i}"
89
- end_expr = f"{end:.3f}" if i < len(segments) - 1 else None
94
+ end_expr = f"{end:.3f}" if not (i == len(segments) - 1 and last_runs_to_eof) else None
90
95
  trim = f"trim=start={start:.3f}" + (f":end={end_expr}" if end_expr else "")
91
96
  vparts.append(f"[0:v]{trim},setpts=(PTS-STARTPTS)/{factor:.6f}[{vlabel}]")
92
97
  labels.append(f"[{vlabel}]")
package/scripts/sync.py CHANGED
@@ -36,6 +36,17 @@ from _common import video_args, add_common, apply_common, emit, aac_args, audio_
36
36
 
37
37
  SR = 8000 # decode sample rate
38
38
 
39
+
40
+ def sync_meaning(offset: float) -> str:
41
+ """The sentence a caller reads next to offset_seconds.
42
+
43
+ A zero offset used to take the else branch of `later if offset > 0` and say the second
44
+ file starts earlier. Values that print as 0.000 s (under half a millisecond) say the two
45
+ recordings start together; anything larger keeps later/earlier."""
46
+ if abs(offset) < 5e-4:
47
+ return "second starts together with the reference"
48
+ return "second starts %.3fs %s than reference" % (abs(offset), "later" if offset > 0 else "earlier")
49
+
39
50
  # Scoring constants for coarse alignment. Both come from tests/bench_sync.py on real dialogue and
40
51
  # music with +/-30 s offsets, gain, noise and EQ (see evals/results and CHANGELOG 0.8.0):
41
52
  #
@@ -307,7 +318,7 @@ def main() -> int:
307
318
  "second": args.second,
308
319
  "offset_seconds": offset,
309
320
  "confidence": score,
310
- "meaning": ("second starts %.3fs %s than reference" % (abs(offset), "later" if offset > 0 else "earlier")),
321
+ "meaning": sync_meaning(offset),
311
322
  "sources": sources_block,
312
323
  }
313
324
  if drift_info: