@tongue-core/sdk 0.1.0-rc.6 → 0.1.0-rc.8

package/README.md

@@ -56,7 +56,7 @@ processes, from Bun): provide it once, where your application runs its Effects.
 | `describeFilm(film)` | the plan as text (for agents to read) and as a JSON summary |
 | `checkFilm(path)` | problems and warnings as data |
 | `renderStills(film, { frames, scale, outDir })` / `renderSheet(…)` | PNGs drawn by the wasm raster (the player's), or tiled into one labelled sheet |
-| `renderVideo(film, options)` | an MP4 with sound: the whole film or `[from, to)`, at any scale (`out` defaults to `out/<name>.mp4`); `sound` runs [sound processing](#sound-processing-extensions); `onStart` once the request is checked and the raster and encoder are chosen, `onProgress` while it draws. A frame that throws fails the render with `Frame <n>: <message>` |
+| `renderVideo(film, options)` | an MP4 with sound: the whole film or `[from, to)`, at any scale (`out` defaults to `out/<name>.mp4`); `sound` runs [sound processing](#sound-processing-extensions); `onStart` once the request is checked and the raster, encoder and workers are chosen (`budget` says why that many workers: one per physical core, fewer when memory is short), `onProgress` while it draws. A frame that throws fails the render with `Frame <n>: <message>`. The result's `profile` says where the time went: milliseconds per stage of drawing, decoder starts, pieces encoded, and the wall time of each phase |
 | `renderVideoEvents(film, options)` | the same render as a `Stream` of `RenderEvent`s: `Started`, `Frames`, `Sound` (each processor's progress), `Joining`, then `Done` with the result |
 | `parsePoint`, `parseFrame`, `parseRange`, `clipRange`, `spreadFrames`, `everyFrames` | points in a film as people write them (`90`, `3s`, `50%`, `end`, `site`, `site+30`), for a command line or form of your own; bad ones fail with `InvalidRequest` saying how to write them |
 | `videoInputs(film)`, `checkVideoInputs(film)` | where each video is decoded from, and a check that every one is there |
@@ -200,20 +200,28 @@ child process), or it slows the hand-off of frames to the workers.
 ## How a video renders
 
 ```
-coordinator ──segments──▶ worker 1: frames ─▶ raster ─▶ encoder ─▶ seg-0000.mp4
-            ◀─progress──  worker 2 …                              seg-0001.mp4
+coordinator ──segments──▶ worker 1: frames ─▶ raster ─▶ encoder ─▶ piece-000000.mp4
+            ◀─progress──  worker 2 …                              piece-000280.mp4
             soundtrack:   sounds + audio clips ─▶ mix ─▶ AAC        audio.m4a
               with sound processing: stems ─▶ processors (here, cached) ─▶ sum ─▶ master ─▶ limiter ─▶ AAC
             join:         concat + audio ─(stream copy)─▶ film.mp4, frame count probed
 ```
 
-- **Workers** are processes (`src/render/worker.ts`), one per physical core by default. Each loads
-  the film and its raster once and renders every segment it is sent into that segment's own
-  encoder, so the JIT stays warm and glyphs and images stay uploaded.
-- **Segments** are one to two seconds of frames, shrinking towards the end so the last ones finish
-  together. Each is a closed-GOP encode; the join copies streams without re-encoding.
-- **Video clips** are decoded by ffmpeg from their original files, in order, frame after frame,
-  and restarted on jumps; a frame that can't be decoded fails the render ([Footage](#footage)).
+- **Workers** are processes (`src/render/worker.ts`), one per physical core by default, fewer when
+  the memory available wouldn't hold that many (`src/render/budget.ts`). Each loads
+  the film and its raster once and draws every segment it is sent, so the JIT stays warm and glyphs
+  and images stay uploaded.
+- **Segments** are a quarter to half a second of frames: what a worker is given at a time. Each
+  worker starts on its own contiguous run of them; one that finishes early takes over the later
+  half of the longest run left (`src/render/runs.ts`). So every worker draws long contiguous
+  stretches, and its video decoders read straight through instead of seeking.
+- **Pieces** are what is encoded: each contiguous stretch a worker draws goes to one encoder, as one
+  closed-GOP file; the join copies streams without re-encoding. Which pieces a render makes depends
+  on how the work was shared out; the frames don't.
+- **Video clips** are decoded by ffmpeg from their original files, in order, frame after frame, a
+  few frames ahead, through a FIFO that holds ffmpeg back when it gets ahead; restarted on jumps,
+  a window at a time for clips playing backwards, and closed when unused. A frame that can't be
+  decoded fails the render ([Footage](#footage)).
 - **The soundtrack** is mixed in its own process while the workers draw (`soundtrack-process.ts`);
   with sound processing, it writes the stems, the processors run in the coordinator, and it adds
   them back, limits and encodes (`sound/soundtrack.ts`).

package/dist/encode/encoders.js

@@ -2,7 +2,7 @@
 // (VAAPI on AMD and Intel under Linux, NVENC on NVIDIA, Quick Sync on Intel) differ only in the
 // arguments around the input. `available()` runs a short encode through the exact same arguments,
 // so it proves the ffmpeg build, the device and the driver all work. Plain TypeScript: encoders run
-// in the render loop. The settings are the ones measured in docs/render-pipeline.md.
+// in the render loop. The settings are the ones measured in docs/render-performance.md.
 export class EncoderError extends Error {
     name = "EncoderError";
 }
@@ -94,7 +94,7 @@ export async function encoderWorks(ffmpeg, spec, size = { width: 320, height: 24
 }
 export const DEFAULT_SETTINGS = { preset: "veryfast", crf: 14, threads: 2, qp: null, gop: null, x264Params: "" };
 export const X264_PRESETS = ["ultrafast", "superfast", "veryfast", "faster", "fast", "medium", "slow", "slower", "veryslow"];
-/** Constant QP that clears the master-quality bar on our films (see docs/render-pipeline.md). */
+/** Constant QP that clears the master-quality bar on our films. */
 const HARDWARE_QP = { h264: 16, hevc: 18 };
 const profile = (codec) => (codec === "hevc" ? "main" : "high");
 const vaapiDevice = () => process.env["TONGUE_VAAPI_DEVICE"] ?? "/dev/dri/renderD128";

package/dist/index.d.ts

@@ -8,7 +8,7 @@ export { importAssets, type ImportReport } from "./import/importer.js";
 export { checkFilm, type CheckReport, type CheckProblem } from "./check.js";
 export { renderStills, renderSheet, type StillsOptions, type Still } from "./render/stills.js";
 export { renderVideo, renderVideoEvents, defaultWorkers, type VideoOptions } from "./render/pipeline.js";
-export type { RenderEvent, RenderStart, SoundReport, VideoResult } from "./render/events.js";
+export { STAGES, type RenderEvent, type RenderProfile, type RenderStart, type SoundReport, type Stage, type VideoResult } from "./render/events.js";
 export { checkVideoInputs } from "./render/inputs.js";
 export { parsePoint, parseFrame, parseRange, clipRange, spreadFrames, everyFrames, type Planned } from "./points.js";
 export { listBackends, type BackendsReport } from "./backends.js";

package/dist/index.js

@@ -11,6 +11,7 @@ export { importAssets } from "./import/importer.js";
 export { checkFilm } from "./check.js";
 export { renderStills, renderSheet } from "./render/stills.js";
 export { renderVideo, renderVideoEvents, defaultWorkers } from "./render/pipeline.js";
+export { STAGES } from "./render/events.js";
 export { checkVideoInputs } from "./render/inputs.js";
 export { parsePoint, parseFrame, parseRange, clipRange, spreadFrames, everyFrames } from "./points.js";
 export { listBackends } from "./backends.js";

package/dist/project.js

@@ -132,9 +132,13 @@ export const openFilm = Effect.fn("openFilm")(function* (path, options = {}) {
  *  entry, else its source path under the root. */
 export const videoInputs = (film) => {
     const out = {};
-    for (const asset of film.assets)
-        if (asset.kind === "video")
-            out[asset.name] = film.sources?.[asset.name] ?? resolve(film.project.root, asset.source);
+    for (const asset of film.assets) {
+        if (asset.kind !== "video")
+            continue;
+        // Own keys only: a video called "constructor" mustn't find Object's.
+        const given = film.sources && Object.hasOwn(film.sources, asset.name) ? film.sources[asset.name] : undefined;
+        out[asset.name] = given ?? resolve(film.project.root, asset.source);
+    }
     return out;
 };
 /** What a film's outputs are called: the project's folder (or the module's file) name, and the

package/dist/render/budget.d.ts

@@ -0,0 +1,45 @@
+/** Frames the encoder `name` holds with x264 `preset`. */
+export declare const encoderFrames: (name: string, preset: string) => number;
+export interface WorkerBudget {
+    readonly workers: number;
+    /** MiB one worker is expected to use at its peak, with its decoders and encoder. */
+    readonly perWorker: number;
+    /** MiB available when the render started (null where the system doesn't say). */
+    readonly available: number | null;
+    /** What set the number: the request, the cores, the memory, or how many segments there are. */
+    readonly limit: "requested" | "cores" | "memory" | "segments";
+}
+export interface BudgetInput {
+    /** Workers asked for, if any: used as asked. */
+    readonly requested: number | undefined;
+    /** At most this many (the raster's own limit). */
+    readonly most: number;
+    readonly segments: number;
+    /** The frames drawn, and every video's decoded size. */
+    readonly frame: {
+        readonly width: number;
+        readonly height: number;
+    };
+    readonly videos: ReadonlyArray<{
+        readonly width: number;
+        readonly height: number;
+    }>;
+    /** Frames the encoder holds (`encoderFrames`). */
+    readonly encoder: number;
+    /** MiB available now; null where unknown. */
+    readonly available: number | null;
+    readonly cores: number;
+}
+/** MiB a worker is expected to use at its peak. */
+export declare function perWorker(frame: BudgetInput["frame"], videos: BudgetInput["videos"], encoderFrames: number): number;
+/** The workers for a render: as asked, else one per physical core, fewer if memory is short. */
+export declare function budget(input: BudgetInput): WorkerBudget;
+/** MiB the system says can be used now without swapping: Linux's MemAvailable, else free memory. */
+export declare function availableMemory(): number | null;
+/** A video's decoded size from its imported facts (`video/<name>/info.json`), or null. */
+export declare function videoSize(text: string): {
+    readonly width: number;
+    readonly height: number;
+} | null;
+/** Logical processors (two per physical core on most machines). */
+export declare const logicalCores: () => number;

package/dist/render/budget.js

@@ -0,0 +1,83 @@
+import { readFileSync } from "node:fs";
+import { availableParallelism, freemem } from "node:os";
+import { isRecord } from "../guards.js";
+// How many workers a render runs. One per physical core, as long as the machine has the memory for
+// them: a worker's memory is mostly frames, its raster's (the frame in floats, its layers), its
+// video decoders' (ffmpeg holds a few dozen decoded frames of a video) and its encoder's, so it is
+// estimated from the frame size, the film's largest videos and the encoder. The constants are fitted to the
+// render bench's peaks (docs/render-performance.md); they are meant to be a little high. Plain
+// TypeScript: the coordinator calls it once.
+const MiB = 1024 * 1024;
+/** What a worker holds besides frames: Bun, the film, its fonts and images. */
+const WORKER_BASE = 500;
+/** Float frames the raster keeps at the render's size: the frame and its layers. */
+const RASTER_FRAMES = 4;
+/** Decoded frames an ffmpeg decoder holds (its references and threads' frames), and its own size. */
+const DECODER_FRAMES = 40;
+const DECODER_BASE = 30;
+/** Frames an encoder holds: x264's lookahead and references, which grow with its preset (fitted to
+ *  its peaks at 1080p: 250 MB an encoder at veryfast, 445 at medium, 935 at veryslow); a GPU
+ *  encoder's, which are few. A preset not listed counts as the slowest. */
+const X264_FRAMES = { ultrafast: 60, superfast: 60, veryfast: 80, faster: 100, fast: 120, medium: 145, slow: 180, slower: 300, veryslow: 300 };
+const GPU_ENCODER_FRAMES = 50;
+/** Frames the encoder `name` holds with x264 `preset`. */
+export const encoderFrames = (name, preset) => (name === "x264" ? (X264_FRAMES[preset] ?? 300) : GPU_ENCODER_FRAMES);
+/** Videos a worker decodes at once: the two clips of a transition, or a scene's two videos. */
+const VIDEOS_AT_ONCE = 2;
+/** Of the memory available when the render starts, the share its workers may use. */
+const USABLE = 0.8;
+/** MiB a worker is expected to use at its peak. */
+export function perWorker(frame, videos, encoderFrames) {
+    const pixels = frame.width * frame.height;
+    const raster = (pixels * 16 * RASTER_FRAMES) / MiB;
+    const encoder = (pixels * 1.5 * encoderFrames) / MiB;
+    const decoders = [...videos]
+        .map((v) => (v.width * v.height * 1.5 * DECODER_FRAMES) / MiB + DECODER_BASE)
+        .sort((a, b) => b - a)
+        .slice(0, VIDEOS_AT_ONCE)
+        .reduce((sum, mb) => sum + mb, 0);
+    return Math.ceil(WORKER_BASE + raster + encoder + decoders);
+}
+/** The workers for a render: as asked, else one per physical core, fewer if memory is short. */
+export function budget(input) {
+    const each = perWorker(input.frame, input.videos, input.encoder);
+    const cap = (workers, limit) => {
+        const most = Math.max(1, Math.min(input.most, input.segments));
+        if (workers <= most)
+            return { workers: Math.max(1, workers), perWorker: each, available: input.available, limit };
+        return { workers: most, perWorker: each, available: input.available, limit: input.segments < input.most ? "segments" : limit };
+    };
+    if (input.requested !== undefined)
+        return cap(input.requested, "requested");
+    const cores = Math.max(1, Math.round(input.cores / 2));
+    const fit = input.available === null ? cores : Math.max(1, Math.floor((input.available * USABLE) / each));
+    return fit < cores ? cap(fit, "memory") : cap(cores, "cores");
+}
+/** MiB the system says can be used now without swapping: Linux's MemAvailable, else free memory. */
+export function availableMemory() {
+    try {
+        const line = /^MemAvailable:\s+(\d+) kB$/m.exec(readFileSync("/proc/meminfo", "utf8"));
+        if (line?.[1])
+            return Math.floor(Number(line[1]) / 1024);
+    }
+    catch {
+        // Not Linux: fall through.
+    }
+    const free = freemem();
+    return free > 0 ? Math.floor(free / MiB) : null;
+}
+/** A video's decoded size from its imported facts (`video/<name>/info.json`), or null. */
+export function videoSize(text) {
+    try {
+        const value = JSON.parse(text);
+        if (!isRecord(value))
+            return null;
+        const { width, height } = value;
+        return typeof width === "number" && typeof height === "number" && width > 0 && height > 0 ? { width, height } : null;
+    }
+    catch {
+        return null;
+    }
+}
+/** Logical processors (two per physical core on most machines). */
+export const logicalCores = () => availableParallelism();

package/dist/render/events.d.ts

@@ -7,6 +7,14 @@ export interface RenderStart {
     readonly raster: RasterName;
     readonly encoder: string;
     readonly workers: number;
+    /** Why that many workers: as asked, one per physical core, as many as the memory available
+     *  holds, or as many as there are segments; and the memory each is expected to use at its
+     *  peak, against what was available when the render started (MiB; null where unknown). */
+    readonly budget: {
+        readonly limit: "requested" | "cores" | "memory" | "segments";
+        readonly perWorker: number;
+        readonly available: number | null;
+    };
 }
 /** One sound processor's run on one stem (or on the whole mix, "master"). */
 export interface SoundReport {
@@ -23,6 +31,33 @@ export interface SoundReport {
     /** What the processor said about its work, if anything. */
     readonly report: Json | null;
 }
+/** The stages of drawing a frame in a render worker, in order: waiting for its video frames, building
+ *  its display list (evaluating the scenes), handing images, glyphs and video frames to the raster,
+ *  drawing (and converting to YUV), handing the frame to the encoder (waiting when it is behind), and
+ *  waiting at the end for the encoders to finish. */
+export type Stage = "decode" | "compose" | "upload" | "raster" | "encode" | "finish";
+export declare const STAGES: ReadonlyArray<Stage>;
+/** Where a render's time went. */
+export interface RenderProfile {
+    /** Milliseconds by stage, summed over every worker and frame (workers run side by side, so these
+     *  add up to more than the render took). */
+    readonly stages: Readonly<Record<Stage, number>>;
+    /** Video decoders started (once per video per seek), and frames decoded only to reach another. */
+    readonly decoders: {
+        readonly starts: number;
+        readonly skipped: number;
+    };
+    /** Pieces encoded and joined: one per contiguous stretch a worker drew. */
+    readonly pieces: number;
+    /** Seconds of the render's own time: getting ready (checks, choosing backends), drawing (until the
+     *  last worker is done), then waiting for the soundtrack, then joining the pieces. */
+    readonly wall: {
+        readonly setup: number;
+        readonly drawing: number;
+        readonly soundtrack: number;
+        readonly join: number;
+    };
+}
 export interface VideoResult {
     readonly out: string;
     readonly width: number;
@@ -38,6 +73,7 @@ export interface VideoResult {
     readonly fps: number;
     /** The sound processing that ran, in order (empty without any). */
     readonly sound: ReadonlyArray<SoundReport>;
+    readonly profile: RenderProfile;
 }
 export type RenderEvent = 
 /** The request is checked and the raster and encoder are chosen; nothing is drawn yet. */

package/dist/render/events.js

@@ -1 +1 @@
-export {};
+export const STAGES = ["decode", "compose", "upload", "raster", "encode", "finish"];

package/dist/render/frame-pipe.d.ts

@@ -0,0 +1,18 @@
+export declare class FramePipe {
+    private readonly handle;
+    private readonly path;
+    private constructor();
+    /** Makes a FIFO at `path` for `start` to write to, starts it, and opens the reading end. A child
+     *  that exits without ever opening the FIFO (it couldn't read its input) would leave the open
+     *  waiting forever, so then we open the writing end ourselves and close it: the open completes
+     *  and reads end at once, and the child's exit says what went wrong. */
+    static connect<P extends {
+        readonly exited: Promise<number>;
+    }>(path: string, start: (fifo: string) => P): Promise<{
+        readonly pipe: FramePipe;
+        readonly proc: P;
+    }>;
+    /** Fills `into` with the next bytes; false when the writer has gone before it was full. */
+    read(into: Uint8Array): Promise<boolean>;
+    close(): Promise<void>;
+}

package/dist/render/frame-pipe.js

@@ -0,0 +1,46 @@
+import { rmSync } from "node:fs";
+import { open } from "node:fs/promises";
+// Raw frames from a child process, through a FIFO: the child writes, we read exact frames into
+// buffers we own. The kernel's pipe is the only buffer between us, so a child that gets ahead simply
+// waits, and nothing piles up in this process however long it goes without reading. (Bun's
+// subprocess streams don't hold back like this: under 1.3.14 they read on into memory, gigabytes a
+// minute; under 1.4.4 they can stop waking the reader with data waiting. See
+// docs/render-performance.md.) Plain TypeScript: reads run on Bun's I/O threads.
+export class FramePipe {
+    handle;
+    path;
+    constructor(handle, path) {
+        this.handle = handle;
+        this.path = path;
+    }
+    /** Makes a FIFO at `path` for `start` to write to, starts it, and opens the reading end. A child
+     *  that exits without ever opening the FIFO (it couldn't read its input) would leave the open
+     *  waiting forever, so then we open the writing end ourselves and close it: the open completes
+     *  and reads end at once, and the child's exit says what went wrong. */
+    static async connect(path, start) {
+        const made = Bun.spawnSync(["mkfifo", "-m", "600", path], { stdout: "ignore", stderr: "pipe" });
+        if (made.exitCode !== 0)
+            throw new Error(`Couldn't make a FIFO at ${path}: ${made.stderr.toString().trim()}`);
+        const proc = start(path);
+        const opening = open(path, "r");
+        const first = await Promise.race([opening.then(() => "opened"), proc.exited.then(() => "exited")]);
+        if (first === "exited")
+            await (await open(path, "w")).close();
+        return { pipe: new FramePipe(await opening, path), proc };
+    }
+    /** Fills `into` with the next bytes; false when the writer has gone before it was full. */
+    async read(into) {
+        let filled = 0;
+        while (filled < into.length) {
+            const { bytesRead } = await this.handle.read(into, filled, into.length - filled, null);
+            if (bytesRead === 0)
+                return false;
+            filled += bytesRead;
+        }
+        return true;
+    }
+    async close() {
+        await this.handle.close().catch(() => undefined);
+        rmSync(this.path, { force: true });
+    }
+}

package/dist/render/frame.d.ts

@@ -1,4 +1,5 @@
 import type { DrawnFrame, FilmRenderer } from "@tongue-core/engine";
+import type { Stopwatch } from "./stopwatch.js";
 import type { FfmpegVideoFrames } from "./video-frames.js";
 /** A video frame a film frame shows couldn't be decoded. The message names the video, its file
  *  and why (ffmpeg's error, or where the file ends). */
@@ -10,4 +11,4 @@ export declare class VideoFrameError extends Error {
  *  Every video frame it shows must have been decoded: a render never lets the engine's poster
  *  stand in for footage (that is the player's fallback while frames arrive), so a frame that is
  *  still missing throws VideoFrameError. */
-export declare function frameWithVideo(renderer: FilmRenderer, frame: number, scale: number, video: FfmpegVideoFrames): Promise<DrawnFrame>;
+export declare function frameWithVideo(renderer: FilmRenderer, frame: number, scale: number, video: FfmpegVideoFrames, clock?: Stopwatch): Promise<DrawnFrame>;

package/dist/render/frame.js

@@ -8,13 +8,26 @@ export class VideoFrameError extends Error {
  *  Every video frame it shows must have been decoded: a render never lets the engine's poster
  *  stand in for footage (that is the player's fallback while frames arrive), so a frame that is
  *  still missing throws VideoFrameError. */
-export async function frameWithVideo(renderer, frame, scale, video) {
+export async function frameWithVideo(renderer, frame, scale, video, clock) {
+    let at = performance.now();
+    const lap = (stage) => {
+        if (clock)
+            at = clock.lap(stage, at);
+    };
     await video.prefetch(renderer.videoFramesAt(frame));
-    const first = renderer.displayList(frame, { scale, video });
-    const drawn = missingIn(first, video).length === 0 ? first : await video.prefetch(missingIn(first, video)).then(() => renderer.displayList(frame, { scale, video }));
+    lap("decode");
+    let drawn = renderer.displayList(frame, { scale, video });
+    lap("compose");
+    if (missingIn(drawn, video).length > 0) {
+        await video.prefetch(missingIn(drawn, video));
+        lap("decode");
+        drawn = renderer.displayList(frame, { scale, video });
+        lap("compose");
+    }
     const [missing] = missingIn(drawn, video);
     if (missing)
         throw new VideoFrameError(video.explain(missing.asset, missing.frame));
+    await video.tick();
     return drawn;
 }
 const missingIn = (drawn, video) => drawn.videoFrames.filter((r) => video.frame(r.asset, r.frame) === null);

package/dist/render/inputs.js

@@ -18,8 +18,9 @@ export const checkVideoInputs = Effect.fn("checkVideoInputs")(function* (film) {
         return inputs;
     const listed = missing.map((m) => `"${m.name}" at ${m.path}`).join(", ");
     const example = missing.map((m) => `"${m.name}": "/path/to/${m.name}.mp4"`).join(", ");
+    const flags = missing.map((m) => `--source ${m.name}=/path/to/${m.name}.mp4`).join(" ");
     return yield* new AssetFailure({
         asset: missing.map((m) => `video:${m.name}`).join(", "),
-        message: `A render decodes each video from its original file, and ${missing.length === 1 ? `video ${listed} isn't there` : `these aren't there: ${listed}`}. Put ${missing.length === 1 ? "it" : "them"} there, or say where ${missing.length === 1 ? "it is" : "they are"} with openFilm's sources option ({ ${example} }).`,
+        message: `A render decodes each video from its original file, and ${missing.length === 1 ? `video ${listed} isn't there` : `these aren't there: ${listed}`}. Put ${missing.length === 1 ? "it" : "them"} there, or say where ${missing.length === 1 ? "it is" : "they are"}: openFilm's sources option ({ ${example} }), or ${flags} with the tongue command.`,
     });
 });

package/dist/render/pipeline.d.ts

@@ -5,8 +5,8 @@ import { BackendUnavailable, InvalidRequest, RenderFailure } from "../errors.js"
 import { type RasterName } from "../raster/rasters.js";
 import { type OpenFilm } from "../project.js";
 import { type SoundRule } from "../sound/Sound.js";
-import type { RenderEvent, RenderStart, VideoResult } from "./events.js";
-export type { RenderEvent, RenderStart, SoundReport, VideoResult } from "./events.js";
+import { type RenderEvent, type RenderStart, type VideoResult } from "./events.js";
+export type { RenderEvent, RenderProfile, RenderStart, SoundReport, Stage, VideoResult } from "./events.js";
 export interface VideoOptions {
     /** The MP4 to write (default: out/<name>.mp4 under the film's root, see defaultVideoOut). */
     readonly out?: string;
@@ -20,7 +20,8 @@ export interface VideoOptions {
     readonly settings?: Partial<EncoderSettings>;
     /** Render processes (default: one per physical core). */
     readonly workers?: number;
-    /** Segment length bounds in frames (default one to two seconds). */
+    /** Segment length bounds in frames: how much a worker is given to draw at a time (default a
+     *  quarter to half a second). Smaller segments share the work out more evenly at the end. */
     readonly segment?: {
         readonly min: number;
         readonly max: number;
@@ -37,7 +38,8 @@ export interface VideoOptions {
         readonly fps: number;
     }) => void;
 }
-/** One worker per physical core: each draws on one thread while its encoder uses the other. */
+/** One worker per physical core: each draws on one thread while its encoder uses the other. (A
+ *  render runs fewer when the machine hasn't the memory for them; see budget.ts.) */
 export declare const defaultWorkers: () => number;
 /** Renders a film (or a range of it) to an MP4 with its sound. `onStart` and `onProgress` hear how
  *  it goes; renderVideoEvents streams everything that happens instead. */

package/dist/render/pipeline.js

@@ -1,4 +1,3 @@
-import { availableParallelism } from "node:os";
 import { dirname, join } from "node:path";
 import { sizeAtScale } from "@tongue-core/engine";
 import { Cause, Deferred, Effect, Fiber, FileSystem, Queue, Stream } from "effect";
@@ -11,10 +10,28 @@ import { plan as planSound } from "../sound/Sound.js";
 import { renderSoundtrack } from "../sound/soundtrack.js";
 import { ffmpegPath, ffprobePath, runTool } from "../tools.js";
 import { entry } from "./entries.js";
+import { availableMemory, budget, encoderFrames, logicalCores, videoSize } from "./budget.js";
+import { STAGES } from "./events.js";
 import { checkVideoInputs } from "./inputs.js";
 import { parseReport, planSegments } from "./protocol.js";
-/** One worker per physical core: each draws on one thread while its encoder uses the other. */
-export const defaultWorkers = () => Math.max(1, Math.round(availableParallelism() / 2));
+import { Runs } from "./runs.js";
+import { noTimes } from "./stopwatch.js";
+/** One worker per physical core: each draws on one thread while its encoder uses the other. (A
+ *  render runs fewer when the machine hasn't the memory for them; see budget.ts.) */
+export const defaultWorkers = () => Math.max(1, Math.round(logicalCores() / 2));
+/** Each video's decoded size, from its imported facts; its longest side square where they can't
+ *  be read (an overestimate, which is what a memory budget wants). */
+const videoSizes = Effect.fn("videoSizes")(function* (film) {
+    const fs = yield* FileSystem.FileSystem;
+    const sizes = [];
+    for (const asset of film.assets) {
+        if (asset.kind !== "video")
+            continue;
+        const text = yield* fs.readFileString(join(film.project.assetsDir, "video", asset.name, "info.json")).pipe(Effect.orElseSucceed(() => ""));
+        sizes.push(videoSize(text) ?? { width: asset.maxSize, height: asset.maxSize });
+    }
+    return sizes;
+});
 const resolveRaster = (choice) => {
     if (choice === "auto")
         return Effect.succeed(autoRaster());
@@ -32,8 +49,8 @@ const resolveEncoder = Effect.fn("resolveEncoder")(function* (choice, settings,
     }
     return yield* new BackendUnavailable({ backend: "encoder", message: "No video encoder works on this machine: is ffmpeg installed with libx264?" });
 });
-/** Runs one worker process: it pulls segments from `queue` until the queue is done. */
-const runWorker = Effect.fn("runWorker")(function* (index, job, cwd, queue, onFrames, segments) {
+/** Runs one worker process: it draws the segments `runs` gives it until there are none left. */
+const runWorker = Effect.fn("runWorker")(function* (index, job, cwd, runs, hooks, segments) {
     const spawner = yield* ChildProcessSpawner.ChildProcessSpawner;
     const mine = yield* Queue.unbounded();
     const fail = (message) => new RenderFailure({ message: `Render worker ${index + 1}: ${message}` });
@@ -43,11 +60,9 @@ const runWorker = Effect.fn("runWorker")(function* (index, job, cwd, queue, onFr
         forceKillAfter: "2 seconds",
     })).pipe(Effect.mapError((e) => fail(e.message)));
     const stderr = yield* handle.stderr.pipe(Stream.decodeText(), Stream.mkString, Effect.forkScoped);
-    const next = Effect.gen(function* () {
-        const segment = yield* Queue.take(queue).pipe(Effect.option);
-        if (segment._tag === "None")
-            return yield* Queue.end(mine);
-        return yield* Queue.offer(mine, segment.value);
+    const next = Effect.suspend(() => {
+        const segment = runs.next(index);
+        return segment ? Queue.offer(mine, segment) : Queue.end(mine);
     });
     const ready = yield* Deferred.make();
     const reported = { last: null };
@@ -59,10 +74,14 @@ const runWorker = Effect.fn("runWorker")(function* (index, job, cwd, queue, onFr
             return Effect.sync(() => { reported.last = report; });
         if (report.kind === "ready")
             return Deferred.succeed(ready, undefined).pipe(Effect.andThen(next));
-        const segment = segments[report.id];
+        if (report.kind === "stats")
+            return Effect.sync(() => hooks.stats(report.stats));
+        if (report.kind === "piece")
+            return Effect.sync(() => hooks.piece(report));
         if (report.kind === "progress")
-            return onFrames(report.id, report.frames);
-        return onFrames(report.id, segment ? segment.to - segment.from : 0).pipe(Effect.andThen(next));
+            return hooks.frames(report.id, report.frames);
+        const segment = segments[report.id];
+        return hooks.frames(report.id, segment ? segment.to - segment.from : 0).pipe(Effect.andThen(next));
     }), Effect.mapError((e) => fail(e.message)));
     const code = yield* handle.exitCode.pipe(Effect.mapError((e) => fail(e.message)));
     if (code !== 0) {
@@ -74,6 +93,19 @@ const runWorker = Effect.fn("runWorker")(function* (index, job, cwd, queue, onFr
         return yield* fail(`exited with ${code}: ${errors.trim().split("\n").slice(-12).join("\n")}`);
     }
 }, Effect.scoped);
+/** The pieces in order, if they cover [from, to) exactly; a gap or an overlap is a worker's bug. */
+const inOrder = (pieces, from, to) => {
+    const ordered = [...pieces].sort((a, b) => a.from - b.from);
+    let at = from;
+    for (const piece of ordered) {
+        if (piece.from !== at)
+            return Effect.fail(new RenderFailure({ message: `The encoded pieces don't fit together: one starts at frame ${piece.from}, where frame ${at} was expected.` }));
+        at = piece.to;
+    }
+    if (at !== to)
+        return Effect.fail(new RenderFailure({ message: `The encoded pieces end at frame ${at}, not ${to}.` }));
+    return Effect.succeed(ordered);
+};
 /** Renders a film to video, telling `emit` how it goes. */
 const render = Effect.fn("renderVideo")(function* (film, options, emit) {
     const fs = yield* FileSystem.FileSystem;
@@ -108,13 +140,14 @@ const render = Effect.fn("renderVideo")(function* (film, options, emit) {
     yield* fs.makeDirectory(renders, { recursive: true });
     const work = yield* fs.makeTempDirectoryScoped({ directory: renders, prefix: "render-" });
     yield* fs.makeDirectory(dirname(out), { recursive: true });
-    const wanted = Math.max(1, Math.min(options.workers ?? defaultWorkers(), raster.maxWorkers ?? Infinity));
-    const segmentSize = options.segment ?? { min: Math.round(plan.fps), max: Math.round(plan.fps * 2) };
-    const segments = planSegments(from, to, wanted, segmentSize).map((s, id) => ({ id, ...s, out: join(work, `seg-${String(id).padStart(4, "0")}.mp4`) }));
-    const workers = Math.min(wanted, segments.length);
-    yield* emit({ _tag: "Started", start: { frames: to - from, width: size.width, height: size.height, raster: raster.name, encoder: encoder.spec.name, workers } });
+    const most = raster.maxWorkers ?? Infinity;
+    const segmentSize = options.segment ?? { min: Math.max(1, Math.round(plan.fps / 4)), max: Math.max(1, Math.round(plan.fps / 2)) };
+    const segments = planSegments(from, to, Math.min(options.workers ?? defaultWorkers(), most), segmentSize).map((s, id) => ({ id, ...s }));
+    const planned = budget({ requested: options.workers, most, segments: segments.length, frame: size, videos: yield* videoSizes(film), encoder: encoderFrames(encoder.name, settings.preset), available: availableMemory(), cores: logicalCores() });
+    const workers = planned.workers;
+    yield* emit({ _tag: "Started", start: { frames: to - from, width: size.width, height: size.height, raster: raster.name, encoder: encoder.spec.name, workers, budget: { limit: planned.limit, perWorker: planned.perWorker, available: planned.available } } });
     const job = {
-        source: film.source, videos, assetsDir: film.project.assetsDir, scale,
+        source: film.source, work, videos, assetsDir: film.project.assetsDir, scale,
         raster: raster.name, encoder: encoder.name, settings, ffmpeg, ffprobe,
     };
     // The soundtrack, alongside the video (and its sound processing, if any).
@@ -133,16 +166,32 @@ const render = Effect.fn("renderVideo")(function* (film, options, emit) {
         lastReport = now;
         return emit(progress(frames()));
     });
-    const queue = yield* Queue.unbounded();
-    yield* Queue.offerAll(queue, segments);
-    yield* Queue.end(queue);
-    yield* Effect.forEach(Array.from({ length: workers }, (_, i) => i), (i) => runWorker(i, job, film.project.root, queue, onFrames, segments), { concurrency: "unbounded" });
+    const runs = new Runs(segments, workers);
+    // What every worker reported, added up: the render's profile, and the pieces to join.
+    const stages = noTimes();
+    const decoders = { starts: 0, skipped: 0 };
+    const pieces = [];
+    const hooks = {
+        frames: onFrames,
+        piece: (piece) => pieces.push(piece),
+        stats: (stats) => {
+            for (const stage of STAGES)
+                stages[stage] += stats.times[stage];
+            decoders.starts += stats.decoderStarts;
+            decoders.skipped += stats.framesSkipped;
+        },
+    };
+    const drawingFrom = performance.now();
+    yield* Effect.forEach(Array.from({ length: workers }, (_, i) => i), (i) => runWorker(i, job, film.project.root, runs, hooks, segments), { concurrency: "unbounded" });
     yield* emit(progress(total));
+    const drawn = performance.now();
     const { audio, sound: processed } = yield* Fiber.join(soundtrack);
+    const soundReady = performance.now();
     // Join without re-encoding, then check what came out.
     yield* emit({ _tag: "Joining" });
-    const list = join(work, "segments.txt");
-    yield* fs.writeFileString(list, segments.map((s) => `file '${s.out}'`).join("\n") + "\n");
+    const ordered = yield* inOrder(pieces, from, to);
+    const list = join(work, "pieces.txt");
+    yield* fs.writeFileString(list, ordered.map((p) => `file '${p.out}'`).join("\n") + "\n");
     yield* runTool(ffmpeg, ["-y", "-v", "error", "-f", "concat", "-safe", "0", "-i", list, "-i", audio, "-map", "0:v", "-map", "1:a", "-c", "copy", "-movflags", "+faststart", out]);
     const { stdout } = yield* runTool(ffprobe, ["-v", "error", "-count_packets", "-select_streams", "v", "-show_entries", "stream=width,height,nb_read_packets", "-of", "csv=p=0", out]);
     const [width = 0, height = 0, count = 0] = stdout.trim().split(",").map(Number);
@@ -151,7 +200,11 @@ const render = Effect.fn("renderVideo")(function* (film, options, emit) {
     if (width !== size.width || height !== size.height)
         return yield* new RenderFailure({ message: `${out} came out ${width}×${height}, not ${size.width}×${size.height} (the ${encoder.spec.name} encoder).` });
     const seconds = (performance.now() - started) / 1000;
-    const result = { out, width, height, frames: count, duration: total / plan.fps, raster: raster.name, encoder: encoder.spec.name, workers, seconds, fps: total / seconds, sound: processed };
+    const profile = {
+        stages, decoders, pieces: ordered.length,
+        wall: { setup: (drawingFrom - started) / 1000, drawing: (drawn - drawingFrom) / 1000, soundtrack: (soundReady - drawn) / 1000, join: (performance.now() - soundReady) / 1000 },
+    };
+    const result = { out, width, height, frames: count, duration: total / plan.fps, raster: raster.name, encoder: encoder.spec.name, workers, seconds, fps: total / seconds, sound: processed, profile };
     return result;
 }, Effect.scoped);
 /** Renders a film (or a range of it) to an MP4 with its sound. `onStart` and `onProgress` hear how