pixi-effects 0.36.0 → 0.38.0

package/CHANGELOG.md

@@ -1,5 +1,34 @@
 # Changelog
 
+## 0.38.0
+
+**Media tools for AI agents: look at, listen to, cut and join footage with no ffmpeg**
+
+- **New: `pixi-effects probe | trim | frames | sound | concat | audio`** (docs: `docs/media.md`, `help media`). `concat a b … -o out` joins clips by copy (refused, naming what differs, when the video or audio codec, shown size, rotation, sample rate, channels or the presence of sound or picture differ; each clip starts where the longest track of the one before it ends, so picture and sound do not drift apart over many joins; the report says where each clip starts), and `audio file -o out.m4a` copies the sound out (AAC → `.m4a`, Opus → `.webm`, any → `.mka`). Both were asked for by the trial below (an agent making a highlights reel could only plan it), run in Node with no browser, are made aside and moved into place, and were played in Chrome (the colours before and after the seam) and run on the iPhone clip. A fresh review of them found, and this release fixes: clips of the same codec with different decoder parameters (profile, level, parameter sets) were joined into a file whose later clips decode as garbage (now refused, naming it; a `trim --exact` cut is such a case), a sound that began later than its picture lost that offset in every clip (one start per clip, not one per track), AAC's encoder-delay packet was copied into the middle of the join as a gap of silence, an output that was the input under another spelling (`A.MP4`) overwrote it on a disk that ignores case, hundreds of inputs held a file each and ran into the descriptor limit, a refused command left an empty new folder, a second sound track was dropped without a word, and `audio` lost the sound's lead-in offset without saying so. Description of the first four: For an agent that is handed a video or a sound: `probe <file|url>` (container, duration, size, per track codec / size as shown / frame rate and whether it varies / rotation / bitrate, key packet times with `--keyframes`, and notes: `no audio`, `rotated 90°`, `variable frame rate`), `frames <file> --at 1,50%,f120,end,"every 2s" [--size 480] [--sheet]` (PNGs of the frames, named by time, and a contact sheet with the times under each frame), `sound <file> [--waveform]` (integrated loudness, true peak, clipping, silences, level over time and a waveform: the numbers `check` gives for a movie, from the same code), `trim <file> <start> <end> -o <out> [--exact]` (a cut by copy, with the real times). Every command: a short report or `--json`; exit 0 done / 1 a problem / 2 usage; a bad file is one sentence naming it, never a stack trace.
+- **Honest about what a cut is.** A copy can only start on a key frame, so `trim` starts on the next one and says it (`asked 3.3–4.9 s, got 4–4.9 s`); a range with no key frame inside is refused with the key frames around it; `--exact` re-encodes in the browser for a start on the frame asked. mediabunny's own default (start exactly, with hidden earlier frames and negative timestamps) plays right in Chrome and ffmpeg (measured by playing the cut and reading the colour of every frame, after a first attempt that seeked a paused `<video>` gave a wrong answer) but other players were not measured, so the cut starts ON the key frame and the report is read back from the written file. A fresh reviewer found that the first version of this left the picture starting 0.7 s late with the sound from 0 (the Chrome test had been blind to it: it read the first frame for every time before the first frame); the test now reads the file's tracks back and fails on a late start. The output is made aside and moved into place, so a refused copy leaves nothing and never empties an existing file.
+- **No ffmpeg anywhere, offline.** `probe` and `trim` by copy run in Node with mediabunny's file source and target, no browser (about 0.1 s). `frames`, `sound` and `trim --exact` decode or re-encode, so they use the headless Chrome that `check` and `render` use; mediabunny comes from this package's own bundle through a local server (Range reads in, PUT out: a long video never goes through CDP). A test runs the four commands with only `node` on `PATH`. If ffmpeg is installed and the job is a long transcode, the docs say to use ffmpeg: measured on one 3.9 MB file on one Mac, a lossless 5.6 s cut took 0.05–0.15 s in ffmpeg and 0.10–0.15 s in mediabunny under Node, and a 5.6 s re-encode 0.6–0.75 s in ffmpeg (the browser re-encode was not timed).
+- **New export: `inspectSound(pcm, { window? })`**, the measurement of `movie.inspectAudio()` for the sound of any file (loudness BS.1770, true peak, clipped samples, silences, level rows; a last window that is only a sliver is not a row). It is also built on its own as `dist/sound.js` with no PixiJS in it, which the media page loads so it works offline (a test follows every import of that file). On a real export the command read the −14.8 LUFS that `check` had said (true peak −3.6 against −3.7: the file is AAC); on a very quiet one −30 against −29.7.
+- Found while building it: mediabunny's `UrlSource` retries a dead URL for ever by default and its delay is in seconds (the command gives up after two retries and says so); its own console lines are silenced so a failure is one line; `getDisplayWidth()` is already the width after a rotation. The staged site no longer carries the tools' browser page (`ai/tools/media-page.html`).
+- **A trial by three fresh agents** (four clips: two of our renders, one with a rotation flag, one with no sound; only these commands and their docs; findings in `pixi-effects-opus/M1..M3`) found, and this release fixes: `trim --exact` files were 70–79 ms longer than reported (now only the picture is re-encoded, the sound is copied byte for byte, and `actual` is read back from the file); an audio bitrate that read 157 kb/s on a file and 55 on its cut (a declared figure against a measured one: both are measured now); a rotated clip's text did not say the stored size or the direction (`180×320 shown (stored 320×180, turned 90° clockwise)`; note that a flag on a landscape picture shows it sideways, which `frames` shows too); `--at every 2s` needed quotes (`every:2s` works, and the error says to quote); `sound` gave whole-dB rows with no times, no loudest stretch, no headroom and no verdict (now: loudest and quietest rows with times, `--window`, and a `reference` line with how many LU under −14 and the headroom to −1 dBTP, whether raising it would need a limiter; the library's own notes start at −24 and −9 LUFS, which the docs now say); `help media` opened with a line of the DSL (an exact heading now crowds out files that only mention the word). Then on one real iPhone clip (HEVC, 1080×1920 stored sideways and turned, variable frame rate ~59.94 fps): `probe`, `frames` (upright), `sound` and the copy cuts (0.2 s, HEVC kept) were right, four cuts matched the source frame at their first and last time, and `--exact` (8 s) turned the HEVC into H.264 without saying so, which it now does (`the picture was re-encoded as avc (the source is hevc)`). Not built: an option to show a rotated video unrotated, re-encoding clips so that different ones can be joined, `convert`, `scenes`, tempo.
+- Spec and plan: `docs/superpowers/specs/2026-10-14-media-tools-design.md`, `docs/superpowers/plans/2026-10-14-media-tools.md`. Not claimed anywhere yet (landing page, FACTS): the trial above used our own renders, not footage from a phone or a camera.
+
+## 0.37.0
+
+**The `canvas` layer: draw it yourself, and what eight trial pieces taught**
+
+- **New: `{ type: 'canvas', draw: (ctx, t, info) => { … } }`.** A picture drawn by your own function of the time into a canvas of the layer's own (cleared before every call, nothing survives into the next), shown like an image at its place in the stack: `initial`, `keyframes`, `filters`, `mask`, `blendMode`, `parent`, `threeD`, transitions and export work on it. Keys: `draw` (required), `setup` (once, may be async; its result is `info.state`, the place to compute a whole simulation), `width` / `height` (px or expression, at most 8192), `resolution` (0.25 to 4), `params` (numbers that keyframes animate as `'params.name'`), `seed` (seeds `info.random()`, which starts again at every call), `smoothing: false` (hard pixels, also when the layer is scaled up: pixel art). `info` is `{ frame, duration, progress, width, height, resolution, frameRate, params, state, random, start, time, canvas }`.
+- **The contract is a function of the time, and `check` holds you to it:** a build-time warning for `Math.random`, `Date.now`, `performance.now` and `new Date()` in `draw`, and `check` (and `movie.review()`, so the Playground) draws every canvas layer twice at four moments and reports a layer whose two pictures differ, naming the moments (`movie.canvasProblems()`). The check draws in two orders (an integrator that moves by the time since the last call is caught, which two draws in a row cannot show) and in a scratch canvas (reading the pixels of the real one made Chrome move it from the GPU to the CPU in the middle of the check, which anti-aliases a few pixels differently), covers inline canvas masks, and `check` lists it under its own `canvas` heading, not as a layout issue. A missing or throwing `draw` / `setup` is said once with the layer's name and the layer is a magenta checkerboard (the next frames are tried again).
+- It cannot see or paint on what is below it, it is not audio, and a spec that holds one holds functions (not pure JSON): an escape hatch like `three` and the `custom` filter. Recipes `canvas-trail`, `canvas-rings`, `canvas-pixel-sprite`, `canvas-life`; a guide section with a live demo; pitfall 115; `help canvas` finds the layer first (a heading that is the word now ranks first).
+- **What eight fresh authors needed (a trial of eight different pieces made with the layer: a botanical plate, a fifths clock, a data race, a brush painting, an elastic exposure, a pixel-art scene, kinetic type, a flow field), and what changed because of it:**
+  - **`info.time`** (the movie's seconds) and **`info.start`** (where the layer starts on the movie's clock, nested compositions counted; inside a time-remapped composition it is that composition's own clock): several canvas layers and the sound share one clock without adding the layer's `at` by hand, which broke quietly when someone moved the layer (6 of 8 pieces did it by hand).
+  - **`info.canvas(w, h)`**: an offscreen canvas for sprites, masks and scratch pictures, with a 2D context made for reading (CPU). Found by tracing a `check` report that blamed `Math.random` for a layer that used none: a canvas made in `setup` with `document.createElement` and drawn with `destination-in` over 96 anti-aliased strokes was drawn differently the first time (up to 128 of 255, 40 418 channel values on a 1280×720 layer) and the same every time after.
+  - **`check` tells the truth about canvas layers**: every moment is drawn once and thrown away before the two orders are compared (a first-use difference of the browser is not held against the layer, and a cache filled on the first call is fine); a layer that differs is reported with how many channel values differ and by how much; when every layer passes, `check` prints a `canvas` line (`2 canvas layer(s) drew the same picture twice at 4 moments (two orders): deterministic · slowest draw 3.2 ms on the CPU`; silence made 7 of 8 authors unsure it had run); `movie.canvasReport()` and `review().canvas` carry the rows. A draw that throws at a checked moment is no longer "deterministic" (its checkerboard was the same twice): the row says it could not draw there. The draw time is measured after a one-pixel read, because a canvas keeps its calls until something reads it (a draw that took 259 ms in the check was the flow-field piece's 3600 lines).
+  - **New exports for `draw`:** `samplePath(d)` (SVG path data walked by length: `.length`, `.at(u)` → `{ x, y, angle }`, `.trim(a, b)` → flat point pieces; for a line that draws on, text along a curve, a petal that falls along a route) and `noise2(x, y, seed)` (smooth 2-D noise). `ease`, `noise`, `random` and `rand` were already exported and are now named in the docs: 6 of 8 authors rewrote easing by hand because nothing said it could be imported.
+  - **Docs:** shared data belongs in module scope (the state `setup` returns is private to its layer), a cache of a pure function is fine, sprites and masks come from `info.canvas`, `ctx.filter` / composite operations / `drawImage` of another canvas / `measureText` all work, a web font is loaded with `await document.fonts.load(…)` in `setup`, **`check` does not see text drawn in a canvas**, the first line of SKILL.md and AGENTS.md no longer says "you never write per-frame code" without the one exception; pitfalls 116–118; recipes `canvas-text-mask` (words as a window onto moving bands: mask first, then `source-atop`), `canvas-text-on-path` and `canvas-bars` (bars that swap places) in a section of their own.
+  - **Not built, on purpose:** values several layers read from one animated source (`composition.params`), reading another layer's picture, text on a path as a layer, pixel-art helpers (framebuffer, dither, palette, bitmap font), `paramAt(name, t)` (a keyframed param read at another time), a square wave for `music`. Each was asked for by one or two pieces.
+- Gallery: five pieces made with the layer in the trial (a botanical plate, a bar-chart race, an elastic long exposure, a pixel cut-scene, a poem in kinetic type): 64 pieces.
+- Spec and plan: `docs/superpowers/specs/2026-10-14-canvas-layer-design.md`. Plugins, registries and maintainer-only fields were considered and dropped: a layer that can draw anything makes them unnecessary.
+
 ## 0.36.0
 
 **Vertical writing, `check` that sees pictures over text, holes in paths, and the fixes found by 18 fresh authors (the claims that were verified on the real build)**

package/README.md

@@ -1,6 +1,6 @@
 # pixi-effects
 
-> **Status**: experimental — current release `0.36.0`. The API may still change between minor versions; see the [CHANGELOG](./CHANGELOG.md) for what each release changed.
+> **Status**: experimental — current release `0.38.0`. The API may still change between minor versions; see the [CHANGELOG](./CHANGELOG.md) for what each release changed.
 
 **[Guide →](https://yjmtmtk.github.io/pixi-effects/guide/)** · **[Gallery →](https://yjmtmtk.github.io/pixi-effects/examples/gallery/)** · 52 portfolio pieces written as plain data by AI models · **[Examples →](https://yjmtmtk.github.io/pixi-effects/examples/)** · 15 numbered examples, a music lab + an in-browser playground.
 
@@ -49,8 +49,8 @@ Drop the imports into an [importmap](https://developer.mozilla.org/docs/Web/HTML
   "imports": {
     "pixi.js":                 "https://esm.sh/pixi.js@8.22.0?bundle-deps",
     "mediabunny":              "https://esm.sh/mediabunny",
-    "pixi-effects":            "https://cdn.jsdelivr.net/npm/pixi-effects@0.36.0/dist/index.js",
-    "pixi-effects/controller": "https://cdn.jsdelivr.net/npm/pixi-effects@0.36.0/dist/Controller.js"
+    "pixi-effects":            "https://cdn.jsdelivr.net/npm/pixi-effects@0.38.0/dist/index.js",
+    "pixi-effects/controller": "https://cdn.jsdelivr.net/npm/pixi-effects@0.38.0/dist/Controller.js"
   }
 }
 </script>
@@ -61,7 +61,7 @@ Drop the imports into an [importmap](https://developer.mozilla.org/docs/Web/HTML
 </script>
 ```
 
-Load the `dist/` files **as they are** (jsDelivr's `/npm/…/dist/…`, or unpkg's `https://unpkg.com/pixi-effects@0.36.0/dist/index.js`) rather than a CDN-rebundled build such as `esm.sh/pixi-effects` or jsDelivr's `+esm`: the entries (`pixi-effects`, `…/controller`, `…/three`) share internal chunks, which only works when each file is served untouched.
+Load the `dist/` files **as they are** (jsDelivr's `/npm/…/dist/…`, or unpkg's `https://unpkg.com/pixi-effects@0.38.0/dist/index.js`) rather than a CDN-rebundled build such as `esm.sh/pixi-effects` or jsDelivr's `+esm`: the entries (`pixi-effects`, `…/controller`, `…/three`) share internal chunks, which only works when each file is served untouched.
 
 > **Using three.js?** Add two more entries (`three` and `pixi-effects/three`) to this importmap — see [Adding three.js](#adding-threejs-optional) below.
 >
@@ -77,7 +77,7 @@ Add two more entries to the importmap above: three.js itself, and the `pixi-effe
   "imports": {
     "...":                     "(everything from the importmap above)",
     "three":                   "https://esm.sh/three@0.178.0",
-    "pixi-effects/three":      "https://cdn.jsdelivr.net/npm/pixi-effects@0.36.0/dist/three.js"
+    "pixi-effects/three":      "https://cdn.jsdelivr.net/npm/pixi-effects@0.38.0/dist/three.js"
   }
 }
 </script>
@@ -176,6 +176,7 @@ This library is designed to be written by AI: a video is plain data, and every m
 
 - **Start a page, and read only what you need:** `npx pixi-effects new my-video` writes `my-video.html` from the template; `npx pixi-effects recipe [id]` lists the tested recipes and prints one with its code; `npx pixi-effects help <word>` prints the sections of the docs about a word (`mask`, `threeD`, `keyframes`, `music` …). An AI starts from the 2 KB [quick start](./skills/pixi-effects/reference/quickstart.md), not from the cheatsheet.
 - **Look at a page with its timeline, and scrub one with the other:** `npx pixi-effects view my-video.html` (it also watches the folder: when the work changes it shows "Updated hh:mm:ss — reload (R)" and never switches by itself; reload comes back to the same frame) (or `node ai/tools/view.mjs …`) opens your browser on a viewer: the page on top, the timeline of every layer under it with a playhead that follows the movie. Click or drag the timeline to seek, click a layer's name to jump to where it starts, Space plays, ← / → step a frame, + / − (or Ctrl/⌘ + wheel) zoom, the names stay in place. For the human who wants to see what the AI made.
+- **Look at, listen to and cut footage you were handed, with no ffmpeg:** `npx pixi-effects probe clip.mp4` (codecs, size, rotation, sound), `frames clip.mp4 --sheet` (a contact sheet of any video), `sound clip.mp4` (loudness, peak, silences), `trim clip.mp4 3 8 -o cut.mp4` (by copy, with the real times), `concat a.mp4 b.mp4 -o both.mp4` (join clips by copy) and `audio clip.mp4 -o sound.m4a` (the sound alone). They use mediabunny, so there is nothing else to install (`frames`, `sound` and `trim --exact` use Chrome). If ffmpeg is installed and the job is a long transcode, use ffmpeg. See [`docs/media.md`](./docs/media.md).
 - **Render a page to a video file, headless:** `npx pixi-effects render my-video.html -o my-video.mp4` (or `node ai/tools/render.mjs …`; Node ≥ 22 and Chrome installed, no dependencies). The container follows the extension (`mp4 webm mov mkv`); `--quality very-low…very-high`, `--query lang=ja` (added to the page URL), `--strict` (exit 1 when the page logged a warning), `--quiet`. It waits for `window.__ready`, runs `movie.render()` and streams the file to disk; exit 0 = written, 1 = the page or the render failed (no file). For scripts, CI and batches; `pixi-effects check` below is the review.
 - **A talk as a PDF:** `npx pixi-effects render my-talk.html -o my-talk.pdf` (one page per page of a `deck()` / `stops` movie; `--all-stops` makes a page of every step). The review tool writes `stops.png` for such a page, one picture per stop.
 - **One-command review for the AI that wrote the page:** `npx pixi-effects check my-video.html` (or `node ai/tools/check.mjs my-video.html`; Node ≥ 22 and Chrome installed, no dependencies). It opens the page in its own headless Chrome and reports warnings, layout (`movie.inspect` over the whole timeline), the soundtrack (`movie.inspectAudio`) and a real export decoded again, and writes a contact sheet PNG and a `timeline.html` (every layer as a bar on a time axis) to look at. Exit code 0 / 1.

package/ai/chat-template.html

@@ -59,9 +59,9 @@
     "imports": {
       "pixi.js":                 "https://cdn.jsdelivr.net/npm/pixi.js@8.22.0/dist/pixi.min.mjs",
       "mediabunny":              "https://cdn.jsdelivr.net/npm/mediabunny@1.61.3/+esm",
-      "pixi-effects":            "https://cdn.jsdelivr.net/npm/pixi-effects@0.36.0/dist/index.js",
-      "pixi-effects/controller": "https://cdn.jsdelivr.net/npm/pixi-effects@0.36.0/dist/Controller.js",
-      "pixi-effects/present":  "https://cdn.jsdelivr.net/npm/pixi-effects@0.36.0/dist/present.js"
+      "pixi-effects":            "https://cdn.jsdelivr.net/npm/pixi-effects@0.38.0/dist/index.js",
+      "pixi-effects/controller": "https://cdn.jsdelivr.net/npm/pixi-effects@0.38.0/dist/Controller.js",
+      "pixi-effects/present":  "https://cdn.jsdelivr.net/npm/pixi-effects@0.38.0/dist/present.js"
     }
   }
   </script>

package/ai/tools/check.mjs

@@ -388,8 +388,10 @@ export async function runCheck(opts, log = console.log) {
     if (opts.onion && opts.onion[1] > info.duration + 1e-9) throw new Error(`--onion ${opts.onion[0]}:${opts.onion[1]} goes past the end: the movie is ${info.duration} s`);
 
     const rv = await cdp.eval(`movie.review(${JSON.stringify({ strict: !!opts.strict })})`);
-    report.inspect = { checkedFrames: rv.frames, issues: rv.problems, review: rv.review };
-    if (rv.problems.length) report.problems.push(`${rv.problems.length} kind(s) of layout issue from movie.inspect`);
+    const layoutProblems = rv.problems.filter(g => g.kind !== 'canvas'), canvasProblems = rv.problems.filter(g => g.kind === 'canvas');
+    report.inspect = { checkedFrames: rv.frames, issues: layoutProblems, review: rv.review, canvas: canvasProblems.map(g => g.message), canvasLayers: rv.canvas ?? [] };
+    if (layoutProblems.length) report.problems.push(`${layoutProblems.length} kind(s) of layout issue from movie.inspect`);
+    if (canvasProblems.length) report.problems.push(`${canvasProblems.length} canvas layer(s) that do not draw the same picture for the same time (movie.canvasProblems)`);
     // fonts: a web font whose file did not load is a problem (the library lists a layer with no available font for review)
     report.fonts = rv.fonts;
     if (rv.fonts.failed.length) report.problems.push(`${rv.fonts.failed.length} web font(s) failed to load: ${rv.fonts.failed.join(', ')} (the layers that use them are drawn in a fallback)`);
@@ -580,6 +582,14 @@ function finish(report, outDir, log) {
     L.push(`  layout    ${g.length ? g.length + ' kind(s) of issue over ' + report.inspect.checkedFrames + ' frames:' : 'no cut-off / off-canvas text over ' + report.inspect.checkedFrames + ' frames (movie.inspect)'}`);
     for (const i of g.slice(0, 8)) L.push(line(i));
     if (g.length > 8) L.push(`            … ${g.length - 8} more (report.json)`);
+    const cv = report.inspect.canvas ?? [];
+    const rows = report.inspect.canvasLayers ?? [];
+    if (cv.length) { L.push(`  canvas    ${cv.length} canvas layer(s) that do not draw the same picture for the same time:`); for (const m of cv.slice(0, 4)) L.push(`            - ${m}`); }
+    if (rows.length) {                                              // silence must not be the only sign that the check ran
+      const good = rows.filter(r => r.ok), slow = [...rows].sort((a, b) => b.drawMs - a.drawMs)[0];
+      const ms = x => x < 10 ? x.toFixed(1) : String(Math.round(x));
+      if (good.length) L.push(`  canvas    ${cv.length ? 'the other ' : ''}${good.length} canvas layer(s) drew the same picture twice at ${good[0].moments.length} moments (two orders): deterministic · slowest draw ${ms(slow.drawMs)} ms on the CPU (${slow.name})`);
+    }
     const rv = report.inspect.review;
     if (rv.length) {
       L.push(`  review    ${rv.length} to look at — text overlaps are often intentional (ghost / glow copies, per-letter boxes, a wipe), and a font that is not available is drawn in a fallback; check them on the contact sheet, or use --strict to fail on them:`);

package/ai/tools/lookup.mjs

@@ -15,7 +15,7 @@ const RECIPES = 'skills/pixi-effects/reference/recipes.md';
 /** What `help` reads, most useful first (the number is a small bonus in the ranking). */
 const CORPUS = [
   ['docs/dsl.md', 10], ['skills/pixi-effects/reference/cheatsheet.md', 8], ['skills/pixi-effects/reference/pitfalls.md', 6],
-  ['docs/api.md', 4], ['skills/pixi-effects/reference/direction.md', 2],
+  ['docs/api.md', 4], ['docs/media.md', 7], ['skills/pixi-effects/reference/direction.md', 2],
 ];
 const MAX_HITS = 5, MAX_LINES_PER_HIT = 30, MAX_OUTPUT_LINES = 170;
 
@@ -117,11 +117,15 @@ export function searchDocs(root, word) {
         return [...fences, ...inline].some(code => re.test(code));
       };
       const call = tokens.some(calls);
-      scored.push({ s, score: (inHeading ? 1000 : 0) + (row ? 1100 : 0) + (call ? 300 : 0) + Math.min(count, 50) + bonus });
+      const exact = tokens.length === 1 && flat(heading) === flat(tokens[0]);                // the heading IS the word
+      scored.push({ s, exact, strong: exact || inHeading || row || call, score: (exact ? 1500 : 0) + (inHeading ? 1000 : 0) + (row ? 1100 : 0) + (call ? 300 : 0) + Math.min(count, 50) + bonus });
     }
   }
   scored.sort((a, b) => b.score - a.score);
-  return scored.slice(0, MAX_HITS).map(({ s }) => {
+  // a section that IS the word (an exact heading) is the answer: other documents that only mention the word are noise next to it (`help media` is the media tools, not a line of the DSL that says "media")
+  const top = scored[0];
+  const kept = top && top.exact ? scored.filter(h => h.s.file === top.s.file || h.strong) : scored;
+  return kept.slice(0, MAX_HITS).map(({ s }) => {
     const body = s.lines.filter((l, i) => i === 0 || l.trim() !== '' || true);
     const shown = body.slice(0, MAX_LINES_PER_HIT);
     return { file: s.file, line: s.line, heading: s.heading, lines: shown, more: Math.max(0, body.length - shown.length) };

package/ai/tools/media-page.html

@@ -0,0 +1,122 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8" />
+  <title>pixi-effects media</title>
+</head>
+<body>
+<!--
+  The browser side of `pixi-effects frames | sound | trim --exact` (ai/tools/media.mjs opens it in the headless Chrome the other tools use).
+  It reads the input by Range requests from /__in/<id>, decodes with mediabunny's own bundle (/__mb/, so it works offline), and hands every file it makes back with
+  PUT /__out/<name>: a long video never goes through CDP. `window.media.<name>(args)` returns a plain object (JSON) or throws a sentence.
+-->
+<script type="module">
+  import * as mb from '/__mb/mediabunny.min.mjs';
+  import { inspectSound } from '/__lib/sound.js';        // the loudness code `check` uses, built with no PixiJS in it
+  mb.Logging.level = mb.LogLevel.Silent;
+
+  const put = async (name, blob) => {
+    const res = await fetch('/__out/' + name, { method: 'PUT', body: blob });
+    if (!res.ok) throw new Error('could not hand ' + name + ' to the command (' + res.status + ')');
+  };
+  const open = (id) => new mb.Input({ source: new mb.UrlSource('/__in/' + id, { getRetryDelay: (n) => (n < 3 ? 0.3 * n : null) }), formats: mb.ALL_FORMATS });
+  const toBlob = (canvas) => (canvas.convertToBlob ? canvas.convertToBlob({ type: 'image/png' }) : new Promise((ok) => canvas.toBlob(ok, 'image/png')));
+  const pad = (n) => String(n).padStart(2, '0');
+
+  window.media = {
+    /** PNGs of the frames at `times` ([{ t, label }]), `size` px wide at most (a small video is not enlarged), and with `sheet` one contact sheet. */
+    async frames({ id, times, size, sheet }) {
+      const input = await open(id);
+      const track = await input.getPrimaryVideoTrack();
+      if (!track) throw new Error('no video track');
+      if (!(await track.canDecode())) throw new Error('this browser cannot decode ' + track.codec + ' (the video codec of the file)');
+      const width = Math.min(size || 480, await track.getDisplayWidth());              // the width as it is shown (after the turn of a rotated video); a small video is not enlarged
+      const sink = new mb.CanvasSink(track, { width });
+      const firstFrame = await track.getFirstTimestamp();
+      const frames = [], kept = [];
+      for (let i = 0; i < times.length; i++) {
+        const wrapped = await sink.getCanvas(Math.max(times[i].t, firstFrame));           // a time before the first frame is the first frame
+        if (!wrapped) throw new Error('there is no frame at ' + times[i].t + ' s');
+        const file = pad(i + 1) + '-' + times[i].label + '.png';
+        await put(file, await toBlob(wrapped.canvas));
+        frames.push({ file, requested: times[i].t, actual: wrapped.timestamp, width: wrapped.canvas.width, height: wrapped.canvas.height });
+        if (sheet) kept.push(wrapped.canvas);
+      }
+      let sheetFile = null;
+      if (sheet && kept.length) {
+        const cols = Math.min(kept.length, 3), rows = Math.ceil(kept.length / cols), cw = kept[0].width, ch = kept[0].height, strip = 22;
+        const c = new OffscreenCanvas(cols * cw, rows * (ch + strip)), g = c.getContext('2d');
+        g.fillStyle = '#10141d'; g.fillRect(0, 0, c.width, c.height);
+        g.font = '14px ui-monospace, Menlo, monospace'; g.textBaseline = 'middle';
+        kept.forEach((k, i) => {
+          const x = (i % cols) * cw, y = Math.floor(i / cols) * (ch + strip);
+          g.drawImage(k, x, y);
+          g.fillStyle = '#e9edf5'; g.fillText(frames[i].actual.toFixed(2) + ' s', x + 8, y + ch + strip / 2);
+        });
+        sheetFile = 'sheet.png';
+        await put(sheetFile, await c.convertToBlob({ type: 'image/png' }));
+      }
+      return { frames, sheet: sheetFile };
+    },
+  };
+  /** Loudness, peak, clipping, silences and level over time of the first audio track (all of it is decoded into memory: the command refuses very long files), and with `waveform` a picture. */
+  window.media.sound = async ({ id, waveform, window: windowSeconds }) => {
+    const input = await open(id);
+    const track = await input.getPrimaryAudioTrack();
+    if (!track) throw new Error('no audio track');
+    if (!(await track.canDecode())) throw new Error('this browser cannot decode ' + track.codec + ' (the audio codec of the file)');
+    const duration = await track.computeDuration();                                   // the sound's own length: a video file is often a little longer than its audio
+    const sink = new mb.AudioBufferSink(track);
+    const parts = [];
+    let rate = 0, channels = 0;
+    for await (const { buffer, timestamp } of sink.buffers()) {
+      rate = buffer.sampleRate; channels = buffer.numberOfChannels;
+      parts.push({ at: Math.round(timestamp * rate), data: Array.from({ length: channels }, (_, c) => buffer.getChannelData(c).slice()) });
+    }
+    if (!parts.length) throw new Error('the audio track has no samples');
+    const total = Math.max(parts.reduce((m, p) => Math.max(m, p.at + p.data[0].length), 0), Math.round(duration * rate));
+    const data = Array.from({ length: channels }, () => new Float32Array(total));
+    for (const p of parts) p.data.forEach((d, c) => data[c].set(d, Math.max(0, p.at)));
+    const pcm = { numberOfChannels: channels, sampleRate: rate, length: total, getChannelData: (c) => data[c] };
+    const report = inspectSound(pcm, { window: windowSeconds });
+    let waveformFile = null;
+    if (waveform) {
+      const W = 1200, H = 240, c = new OffscreenCanvas(W, H), g = c.getContext('2d');
+      g.fillStyle = '#10141d'; g.fillRect(0, 0, W, H);
+      g.fillStyle = '#7fb4ff';
+      const per = Math.max(1, Math.floor(total / W));
+      for (let x = 0; x < W; x++) {
+        let lo = 0, hi = 0;
+        for (let c = 0; c < channels; c++) for (let i = x * per; i < Math.min(total, (x + 1) * per); i++) { const v = data[c][i]; if (v < lo) lo = v; if (v > hi) hi = v; }       // every channel: a sound only on one side is not an empty picture
+        g.fillRect(x, H / 2 - hi * (H / 2 - 4), 1, Math.max(1, (hi - lo) * (H / 2 - 4)));
+      }
+      g.fillStyle = '#e9edf5'; g.font = '13px ui-monospace, Menlo, monospace'; g.fillText('0 s', 6, H - 6); g.fillText(report.duration.toFixed(1) + ' s', W - 60, H - 6);
+      waveformFile = 'waveform.png';
+      await put(waveformFile, await c.convertToBlob({ type: 'image/png' }));
+    }
+    return { ...report, waveform: waveformFile };
+  };
+
+  const FORMATS = { '.mp4': mb.Mp4OutputFormat, '.m4v': mb.Mp4OutputFormat, '.m4a': mb.Mp4OutputFormat, '.mov': mb.MovOutputFormat, '.webm': mb.WebMOutputFormat, '.mkv': mb.MkvOutputFormat, '.mka': mb.MkvOutputFormat };
+
+  /** [start, end] re-encoded (every track), so the cut starts on the frame asked for; the file is held in memory until it is handed back. */
+  window.media.trimExact = async ({ id, start, end, ext, outName }) => {
+    const input = await open(id);
+    const Format = FORMATS[ext];
+    if (!Format) throw new Error('cannot write ' + ext);
+    const target = new mb.BufferTarget();
+    const output = new mb.Output({ format: new Format(), target });
+    // only the PICTURE is re-encoded (that is what makes the start exact); the sound is copied: re-encoding it squeezed a 157 kb/s track to 55 kb/s and left it running 70-80 ms past the picture
+    const conversion = await mb.Conversion.init({ input, output, trim: { start, end }, video: { forceTranscode: true }, showWarnings: false });
+    if (!conversion.isValid || conversion.discardedTracks.length) {
+      throw new Error('this browser cannot re-encode it into ' + ext + ' (' + conversion.discardedTracks.map((d) => d.track.type + ': ' + d.reason).join(', ') + ')');
+    }
+    await conversion.execute();
+    await put(outName, new Blob([target.buffer]));
+    return { bytes: target.buffer.byteLength };
+  };
+
+  window.__mediaReady = true;
+</script>
+</body>
+</html>