@tongue-core/sdk 0.1.0-rc.4 → 0.1.0-rc.6

package/README.md

@@ -7,9 +7,12 @@ here, with the same results. Built with Effect v4 (orchestration only: the per-f
 TypeScript); a Promise facade is included for applications that don't use Effect.
 
 ```bash
-bun add @tongue-core/sdk@next      # Bun ≥ 1.3; ffmpeg and ffprobe on the PATH
+bun add @tongue-core/sdk@next effect@4.0.6      # Bun ≥ 1.3; ffmpeg and ffprobe on the PATH
 ```
 
+`effect` is a peer dependency: your application and the SDK share one copy of it, so the SDK's
+services, errors and Layers are the same values as yours.
+
 ## Promise API
 
 ```ts
@@ -31,27 +34,32 @@ Failures reject with the SDK's tagged errors (below).
 ## Effect API
 
 ```ts
-import { BunServices } from "@effect/platform-bun";
-import { openFilm, renderVideo } from "@tongue-core/sdk";
+import { openFilm, renderVideo, TongueServices } from "@tongue-core/sdk";
 import { Effect } from "effect";
 
 const program = Effect.gen(function* () {
   const film = yield* openFilm("src/film.tsx");          // load, import assets, plan
   return yield* renderVideo(film, { out: "out/film.mp4", encoder: "auto", raster: "auto" });
 });
-await Effect.runPromise(program.pipe(Effect.provide(BunServices.layer)));
+await Effect.runPromise(program.pipe(Effect.provide(TongueServices.layer)));
 ```
 
+`TongueServices.layer` is what every SDK function runs on (the file system, paths and child
+processes, from Bun): provide it once, where your application runs its Effects.
+
 | Function | Does |
 |---|---|
 | `initProject(dir, { template, name })` | a film project in a new or empty folder, from a built-in template or a template folder |
 | `listTemplates()` / `readTemplate(nameOrPath)` | the built-in templates, or one by name or path |
 | `buildFilm(project, { out })` | builds a film project into `out` (default `dist/`): build.json, film.json, a bundle per scene |
-| `openFilm(path, { scene })` | a film ready to render: a project is built (into `.tongue/builds/<id>`, named by its content) and planned from film.json, a film module is imported and planned; assets imported (cached by size, mtime and options) either way. `scene: { id, props }` opens one scene of a project on its own |
+| `openFilm(path, { scene, sources })` | a film ready to render: a project is built (into `.tongue/builds/<id>`, named by its content) and planned from film.json, a build folder (what `buildFilm` writes) is opened as it is, a film module is imported and planned; assets imported (cached by size, mtime and options) either way. `scene: { id, props }` opens one scene of a project on its own; `sources` says where videos are decoded from ([Footage](#footage)) |
 | `describeFilm(film)` | the plan as text (for agents to read) and as a JSON summary |
 | `checkFilm(path)` | problems and warnings as data |
 | `renderStills(film, { frames, scale, outDir })` / `renderSheet(…)` | PNGs drawn by the wasm raster (the player's), or tiled into one labelled sheet |
-| `renderVideo(film, options)` | an MP4 with sound: the whole film or `[from, to)`, at any scale; `onStart` once the request is checked and the raster and encoder are chosen, `onProgress` while it draws. A frame that throws fails the render with `Frame <n>: <message>` |
+| `renderVideo(film, options)` | an MP4 with sound: the whole film or `[from, to)`, at any scale (`out` defaults to `out/<name>.mp4`); `sound` runs [sound processing](#sound-processing-extensions); `onStart` once the request is checked and the raster and encoder are chosen, `onProgress` while it draws. A frame that throws fails the render with `Frame <n>: <message>` |
+| `renderVideoEvents(film, options)` | the same render as a `Stream` of `RenderEvent`s: `Started`, `Frames`, `Sound` (each processor's progress), `Joining`, then `Done` with the result |
+| `parsePoint`, `parseFrame`, `parseRange`, `clipRange`, `spreadFrames`, `everyFrames` | points in a film as people write them (`90`, `3s`, `50%`, `end`, `site`, `site+30`), for a command line or form of your own; bad ones fail with `InvalidRequest` saying how to write them |
+| `videoInputs(film)`, `checkVideoInputs(film)` | where each video is decoded from, and a check that every one is there |
 | `listBackends()` | every raster and encoder, each tried for real |
 | `studioServer(path, { port, remote })` | the studio on a project (rebuilt and reloaded on every edit) or on a build served anywhere |
 
@@ -81,12 +89,121 @@ the SDK loads it, plus the media its clips play; `<Film assets={[…]}>` replace
 Importing turns them into what the engine reads: TrueType fonts, QOI images, WAV audio, video facts
 and a poster frame.
 
+## Footage
+
+A render decodes every video from its **original file**: a build holds only a video's facts, a
+poster and a preview-quality proxy (what the studio plays). Before a frame is drawn, `renderVideo`
+and `renderStills` check that each video's file is there and fail with `AssetFailure` naming the
+ones that aren't; a frame ffmpeg can't decode, or a file shorter than its import counted, fails the
+render with `Frame <n>: Video "<name>": …` and ffmpeg's reason. A poster never stands in for
+footage in a render (that is the player's fallback while frames arrive).
+
+A video is decoded from its source path under the film's root, unless `sources` says otherwise:
+
+```ts
+const film = yield* openFilm("renders/1234/dist", {   // a build, without the project around it
+  sources: { screen: "/data/media/screen.mp4", camera: "/data/media/camera.mp4" },
+});
+```
+
+A build inside a project (`my-film/dist`) finds its footage in the project; a build on its own
+needs `sources` for every video that isn't beside it.
+
+## Sound processing (extensions)
+
+A render can put part of its soundtrack through processing of your own (voice enhancement, noise
+removal, levelling, effects) and still produce one MP4, in sync. The SDK defines the contract and
+runs it; what a processor does and what it needs (a model, a native tool, a service) is yours, so
+nobody who doesn't use it installs it.
+
+```ts
+import { AudioProcessor, openFilm, renderVideo, Sound } from "@tongue-core/sdk";
+
+const enhance = yield* enhanceVoice({ peakDb: -3 });        // an AudioProcessor (below)
+yield* renderVideo(film, {
+  sound: [
+    Sound.track("voiceover", enhance),   // the clips of the track with id "voiceover"
+    Sound.track("voiceover", level),     // the same selection again: runs after enhance
+    Sound.master(loudness),              // then the whole mix
+  ],
+});
+```
+
+**What a rule takes.** `Sound.track(id, p)`; `Sound.stem({ clips: ["a", "b"] }, p)`;
+`Sound.stem({ voice: true }, p)` (every narration clip); `Sound.master(p)` (the whole mix, after
+the stems, before the limiter). A selection is data (`Sound.AudioSelection` decodes it from flags or
+JSON). The selected clips are mixed into a **stem** of their own (their trims, speed, gain and fades
+applied) across the whole film, processed, and added back; everything else sounds as it always has.
+A track or clip the film doesn't have, or a visual track, is an `InvalidRequest` listing what it
+does have; a clip goes into one stem, so overlapping selections are refused unless they are the same
+(then their processors run in order).
+
+**The contract.** A processor gets a `Stem`: `samples` per channel (the whole film at 48 kHz, from
+its first frame), `channels` (2), the stem as a float WAV `file` and as decoded `pcm`, the `clips` in
+it and the `plan`, a `workDir`, and `progress(0..1)`. It returns `{ pcm }` or `{ file }`, plus an
+optional `report` (any JSON). What it returns must have exactly the stem's rate, channels and
+length, and only finite samples, or the render fails with `ProcessorFailure`: that is what keeps
+the sound in sync, so a processor with a delay (a model's lookahead) takes it out itself. Any
+failure of the processor, or a defect in it, is a `ProcessorFailure` naming it.
+
+**Two ways to make one.** `AudioProcessor.fromPcm({ key, process: (pcm, stem) => pcm })` for
+processing that needs only the samples. `AudioProcessor.make({ key, process: (stem) => Effect })`
+for anything else: the services `process` uses are taken from where `make` runs and travel with the
+processor, so the render itself never needs them:
+
+```ts
+// voice-enhance: its own package, depending on @tongue-core/sdk and on what it needs.
+interface DenoiserOperations {
+  readonly name: string;
+  readonly denoise: (channel: Float32Array, sampleRate: number) => Effect.Effect<Float32Array>;
+}
+export class Denoiser extends Context.Service<Denoiser, DenoiserOperations>()("voice-enhance/Denoiser") {
+  static readonly layer = Layer.effect(Denoiser, loadModel);   // the heavy part, paid once, only by those who provide it
+}
+
+export const enhanceVoice = (settings: EnhanceSettings) =>
+  AudioProcessor.make({
+    key: `voice-enhance/v1 peak ${settings.peakDb} dB`,
+    process: (stem) => Effect.gen(function* () {
+      const denoiser = yield* Denoiser;
+      const pcm = yield* stem.pcm;
+      const channels = yield* Effect.forEach(pcm.channels, (c) => denoiser.denoise(c, pcm.sampleRate));
+      yield* stem.progress(1);
+      return { pcm: { sampleRate: pcm.sampleRate, channels }, report: { denoiser: denoiser.name } };
+    }),
+  });   // Effect<AudioProcessor, never, Denoiser>
+
+// In your command line: make the processor where its Layer is provided.
+const render = Command.make("render", { project: Argument.String("project"), enhance: Flag.String("enhance").pipe(Flag.atLeast(0)) },
+  Effect.fn(function* (flags) {
+    const film = yield* openFilm(flags.project);
+    const enhance = yield* enhanceVoice({ peakDb: -3 });
+    return yield* renderVideo(film, { sound: flags.enhance.map((track) => Sound.track(track, enhance)) });
+  }),
+).pipe(Command.provide(Denoiser.layer));
+```
+
+**The key and the cache.** A processor's `key` names it, its version and its settings. A result is
+cached by the key and the exact audio it was given (`.tongue/cache/sound`, the newest 32 kept): render
+again after changing anything that doesn't change that audio (the picture, another track, the
+range, the scale) and the processor doesn't run; the result says `cached: true`. Change the key
+whenever the output would change.
+
+**What happened.** `VideoResult.sound` lists each processor run: the stem, its clips, the
+processor's key, whether it was cached, its seconds and its report. `renderVideoEvents` streams
+`Sound` events (`started`, `progress`, `done`, `cached`) while it runs.
+
+**Where it runs.** Processors run in your process, alongside the render workers, while the frames
+are drawn (the soundtrack always has); keep heavy work off its main thread (`effect/workers`, a
+child process), or it slows the hand-off of frames to the workers.
+
 ## How a video renders
 
 ```
 coordinator ──segments──▶ worker 1: frames ─▶ raster ─▶ encoder ─▶ seg-0000.mp4
             ◀─progress──  worker 2 …                              seg-0001.mp4
             soundtrack:   sounds + audio clips ─▶ mix ─▶ AAC        audio.m4a
+              with sound processing: stems ─▶ processors (here, cached) ─▶ sum ─▶ master ─▶ limiter ─▶ AAC
             join:         concat + audio ─(stream copy)─▶ film.mp4, frame count probed
 ```
 
@@ -95,8 +212,11 @@ coordinator ──segments──▶ worker 1: frames ─▶ raster ─▶ encode
   encoder, so the JIT stays warm and glyphs and images stay uploaded.
 - **Segments** are one to two seconds of frames, shrinking towards the end so the last ones finish
   together. Each is a closed-GOP encode; the join copies streams without re-encoding.
-- **Video clips** are decoded by ffmpeg in order, frame after frame, and restarted on jumps.
-- **The soundtrack** is mixed in its own process while the workers draw (`soundtrack-process.ts`).
+- **Video clips** are decoded by ffmpeg from their original files, in order, frame after frame,
+  and restarted on jumps; a frame that can't be decoded fails the render ([Footage](#footage)).
+- **The soundtrack** is mixed in its own process while the workers draw (`soundtrack-process.ts`);
+  with sound processing, it writes the stems, the processors run in the coordinator, and it adds
+  them back, limits and encodes (`sound/soundtrack.ts`).
 
 Rasters (who draws) and encoders (who encodes) are chosen by name, because the choice is made here
 but runs inside the workers:
@@ -122,11 +242,12 @@ gives the CLI's exit code.
 | Error | When | Exit |
 |---|---|---|
 | `FilmFailure` | the module is missing, throws, isn't a `<Film>`, or its timeline doesn't plan | 2 |
-| `AssetFailure` | an asset's file is missing or unreadable | 2 |
+| `AssetFailure` | an asset's file is missing or unreadable, or a video's original file isn't there to render from | 2 |
 | `InvalidRequest` | a frame outside the film, a bad range or scale | 2 |
 | `ToolFailure` | ffmpeg or ffprobe is missing or failed | 3 |
 | `BackendUnavailable` | the raster or encoder asked for doesn't work here | 3 |
-| `RenderFailure` | a worker died or a frame failed | 3 |
+| `RenderFailure` | a worker died, a frame failed, or a video frame couldn't be decoded | 3 |
+| `ProcessorFailure` | a sound processor failed, or returned audio that would break sync (rate, channels, length) or isn't audio | 3 |
 | `LayoutError`, `CompileError`, `LoadError`, `TimelineError`, `DocError`, `TemplateError` | a film project doesn't build (from `@tongue-core/project`: each names its file, line or path) | 2 |
 
 ## Environment
@@ -147,10 +268,14 @@ src/
   init.ts                 init from templates
   check.ts, backends.ts   checkFilm, listBackends
   import/                 asset import: fonts, images, audio, video (ffprobe, posters)
-  render/                 stills, the video pipeline, its worker and soundtrack processes, video decoding
+  render/                 stills, the video pipeline and its events, its worker and soundtrack processes, video decoding and inputs
+  sound/                  sound processing: Sound (rules), AudioProcessor (the contract), the soundtrack around them
+  points.ts               points and ranges in a film, as people write them
+  TongueServices.ts       the layer the SDK runs on
   encode/encoders.ts      the ffmpeg encoders and how each is tried
   raster/                 the native and GPU raster bindings (index.ts) and the registry (rasters.ts)
   studio/server.ts        the studio's server: the app bundled, a project's build served and rebuilt
-test/                     unit tests, end-to-end ones on fixtures/tiny (a film with text, video and sound), and
+test/                     unit tests, end-to-end ones on fixtures/tiny (a film with text, video and sound; sound
+                          processing and footage that can't be decoded on copies of it), builds rendered with `sources`, and
                           projects from every template (project.test.ts: film.json draws the same frames as film.tsx)
 ```

package/dist/TongueServices.d.ts

@@ -0,0 +1,5 @@
+import { BunServices } from "@effect/platform-bun";
+import type { Layer } from "effect";
+/** The services every SDK function needs. */
+export type TongueServices = BunServices.BunServices;
+export declare const layer: Layer.Layer<TongueServices>;

package/dist/TongueServices.js

@@ -0,0 +1,2 @@
+import { BunServices } from "@effect/platform-bun";
+export const layer = BunServices.layer;

package/dist/client.js

@@ -1,4 +1,3 @@
-import { BunServices } from "@effect/platform-bun";
 import { Cause, Effect, Exit } from "effect";
 import { listBackends } from "./backends.js";
 import { checkFilm } from "./check.js";
@@ -6,10 +5,11 @@ import { initProject, listTemplates } from "./init.js";
 import { buildFilm, describeFilm, openFilm } from "./project.js";
 import { renderVideo } from "./render/pipeline.js";
 import { renderSheet, renderStills } from "./render/stills.js";
+import * as TongueServices from "./TongueServices.js";
 // The SDK as plain Promises, for applications that don't use Effect. Each call runs the same Effect
 // program the CLI runs; a failure rejects with the SDK's tagged error (FilmFailure, ToolFailure…).
 const run = async (effect) => {
-    const exit = await Effect.runPromiseExit(effect.pipe(Effect.provide(BunServices.layer)));
+    const exit = await Effect.runPromiseExit(effect.pipe(Effect.provide(TongueServices.layer)));
     if (Exit.isSuccess(exit))
         return exit.value;
     throw Cause.squash(exit.cause);

package/dist/errors.d.ts

@@ -43,9 +43,20 @@ declare const RenderFailure_base: Schema.Class<RenderFailure, Schema.TaggedStruc
 /** A frame failed to render, or a render worker died. */
 export declare class RenderFailure extends RenderFailure_base {
 }
-export declare const SdkError: Schema.Union<readonly [typeof FilmFailure, typeof AssetFailure, typeof InvalidRequest, typeof ToolFailure, typeof BackendUnavailable, typeof RenderFailure, typeof LayoutError, typeof CompileError, typeof LoadError, typeof TimelineError, typeof DocError, typeof TemplateError]>;
+declare const ProcessorFailure_base: Schema.Class<ProcessorFailure, Schema.TaggedStruct<"ProcessorFailure", {
+    /** The processor's key. */
+    readonly processor: Schema.String;
+    /** What it was processing: "track voiceover", "master"… */
+    readonly stem: Schema.String;
+    readonly message: Schema.String;
+}>, import("effect/Cause").YieldableError>;
+/** A sound processor failed, or gave back audio that would break the film's sync (another rate, other
+ *  channels, another length) or isn't audio (a sample that isn't a finite number). */
+export declare class ProcessorFailure extends ProcessorFailure_base {
+}
+export declare const SdkError: Schema.Union<readonly [typeof FilmFailure, typeof AssetFailure, typeof InvalidRequest, typeof ToolFailure, typeof BackendUnavailable, typeof RenderFailure, typeof ProcessorFailure, typeof LayoutError, typeof CompileError, typeof LoadError, typeof TimelineError, typeof DocError, typeof TemplateError]>;
 export type SdkError = typeof SdkError.Type;
-export declare const isSdkError: <I>(input: I) => input is I & (FilmFailure | AssetFailure | InvalidRequest | ToolFailure | BackendUnavailable | RenderFailure | LayoutError | CompileError | LoadError | TimelineError | DocError | TemplateError);
+export declare const isSdkError: <I>(input: I) => input is I & (FilmFailure | AssetFailure | InvalidRequest | ToolFailure | BackendUnavailable | RenderFailure | ProcessorFailure | LayoutError | CompileError | LoadError | TimelineError | DocError | TemplateError);
 /** Exhaustive: a new error without a case is a compile error. */
 export declare function exitCodeOf(error: SdkError): number;
 /** The message of anything thrown. */

package/dist/errors.js

@@ -41,8 +41,18 @@ export class RenderFailure extends Schema.TaggedError()("RenderFailure", {
     message: Schema.String,
 }) {
 }
+/** A sound processor failed, or gave back audio that would break the film's sync (another rate, other
+ *  channels, another length) or isn't audio (a sample that isn't a finite number). */
+export class ProcessorFailure extends Schema.TaggedError()("ProcessorFailure", {
+    /** The processor's key. */
+    processor: Schema.String,
+    /** What it was processing: "track voiceover", "master"… */
+    stem: Schema.String,
+    message: Schema.String,
+}) {
+}
 export const SdkError = Schema.Union([
-    FilmFailure, AssetFailure, InvalidRequest, ToolFailure, BackendUnavailable, RenderFailure,
+    FilmFailure, AssetFailure, InvalidRequest, ToolFailure, BackendUnavailable, RenderFailure, ProcessorFailure,
     LayoutError, CompileError, LoadError, TimelineError, DocError, TemplateError,
 ]);
 export const isSdkError = Schema.is(SdkError);
@@ -62,6 +72,7 @@ export function exitCodeOf(error) {
         case "ToolFailure":
         case "BackendUnavailable":
         case "RenderFailure":
+        case "ProcessorFailure":
             return 3;
     }
 }

package/dist/import/importer.js

@@ -71,7 +71,9 @@ export const importAssets = Effect.fn("importAssets")(function* (project, assets
                 const names = (yield* fs.readDirectory(proxyDir)).filter((n) => n.endsWith(".mp4")).sort();
                 const segments = yield* Effect.forEach(names, (n) => fs.readFile(path.join(proxyDir, n)));
                 const proxy = yield* proxyInfo(asset.name, segments);
-                yield* write(importedPath(asset), new TextEncoder().encode(JSON.stringify({ ...video.info, source, proxy })));
+                // The facts and the proxy only: never the source's absolute path, which names this
+                // machine and is wrong on any other (renders find the source themselves, see videoInputs).
+                yield* write(importedPath(asset), new TextEncoder().encode(JSON.stringify({ ...video.info, proxy })));
                 yield* write(videoPosterPath(asset), video.poster);
                 break;
             }

package/dist/index.d.ts

@@ -1,16 +1,23 @@
 export { createTongue, type Tongue } from "./client.js";
-export { openFilm, openProject, buildFilm, describeFilm, type OpenFilm, type OpenOptions, type Project } from "./project.js";
+export * as TongueServices from "./TongueServices.js";
+export { openFilm, openProject, buildFilm, describeFilm, videoInputs, outputName, defaultVideoOut, defaultStillsDir, type OpenFilm, type OpenOptions, type Project, } from "./project.js";
 export { buildFilmProject, readProjectFiles, fileModuleHost, linkFilmTypes, type BuiltProject } from "./build/build.js";
 export { initProject, listTemplates, readTemplate, type InitOptions, type InitReport } from "./init.js";
 export type { FilmSource } from "./render/protocol.js";
 export { importAssets, type ImportReport } from "./import/importer.js";
 export { checkFilm, type CheckReport, type CheckProblem } from "./check.js";
 export { renderStills, renderSheet, type StillsOptions, type Still } from "./render/stills.js";
-export { renderVideo, defaultWorkers, type RenderStart, type VideoOptions, type VideoResult } from "./render/pipeline.js";
+export { renderVideo, renderVideoEvents, defaultWorkers, type VideoOptions } from "./render/pipeline.js";
+export type { RenderEvent, RenderStart, SoundReport, VideoResult } from "./render/events.js";
+export { checkVideoInputs } from "./render/inputs.js";
+export { parsePoint, parseFrame, parseRange, clipRange, spreadFrames, everyFrames, type Planned } from "./points.js";
 export { listBackends, type BackendsReport } from "./backends.js";
 export { studioServer, type StudioOptions } from "./studio/server.js";
 export { ENCODER_NAMES, AUTO_ENCODERS, DEFAULT_SETTINGS, isEncoderName, type EncoderName, type EncoderSettings } from "./encode/encoders.js";
 export { RASTER_NAMES, isRasterName, type RasterName } from "./raster/rasters.js";
 export { NativeRaster, nativeRasterAvailable, GpuRaster, gpuRasterAvailable, type NativeRasterOptions } from "./raster/index.js";
-export { FilmFailure, AssetFailure, InvalidRequest, ToolFailure, BackendUnavailable, RenderFailure, SdkError, isSdkError, exitCodeOf } from "./errors.js";
-export type { FilmPlan, PlanSummary } from "@tongue-core/film";
+export { FilmFailure, AssetFailure, InvalidRequest, ToolFailure, BackendUnavailable, RenderFailure, ProcessorFailure, SdkError, isSdkError, exitCodeOf } from "./errors.js";
+export * as Sound from "./sound/Sound.js";
+export * as AudioProcessor from "./sound/AudioProcessor.js";
+export type { FilmPlan, PlanSummary, ClipPlan, TrackPlan, Json } from "@tongue-core/film";
+export type { PcmAudio } from "@tongue-core/engine";

package/dist/index.js

@@ -3,17 +3,23 @@
 // of truth (the CLI is built on it); `createTongue()` offers the same as Promises.
 export { createTongue } from "./client.js";
 // Effect API
-export { openFilm, openProject, buildFilm, describeFilm } from "./project.js";
+export * as TongueServices from "./TongueServices.js";
+export { openFilm, openProject, buildFilm, describeFilm, videoInputs, outputName, defaultVideoOut, defaultStillsDir, } from "./project.js";
 export { buildFilmProject, readProjectFiles, fileModuleHost, linkFilmTypes } from "./build/build.js";
 export { initProject, listTemplates, readTemplate } from "./init.js";
 export { importAssets } from "./import/importer.js";
 export { checkFilm } from "./check.js";
 export { renderStills, renderSheet } from "./render/stills.js";
-export { renderVideo, defaultWorkers } from "./render/pipeline.js";
+export { renderVideo, renderVideoEvents, defaultWorkers } from "./render/pipeline.js";
+export { checkVideoInputs } from "./render/inputs.js";
+export { parsePoint, parseFrame, parseRange, clipRange, spreadFrames, everyFrames } from "./points.js";
 export { listBackends } from "./backends.js";
 export { studioServer } from "./studio/server.js";
 export { ENCODER_NAMES, AUTO_ENCODERS, DEFAULT_SETTINGS, isEncoderName } from "./encode/encoders.js";
 export { RASTER_NAMES, isRasterName } from "./raster/rasters.js";
 // The server rasters, for applications that drive a raster themselves (FilmRenderer from @tongue-core/engine)
 export { NativeRaster, nativeRasterAvailable, GpuRaster, gpuRasterAvailable } from "./raster/index.js";
-export { FilmFailure, AssetFailure, InvalidRequest, ToolFailure, BackendUnavailable, RenderFailure, SdkError, isSdkError, exitCodeOf } from "./errors.js";
+export { FilmFailure, AssetFailure, InvalidRequest, ToolFailure, BackendUnavailable, RenderFailure, ProcessorFailure, SdkError, isSdkError, exitCodeOf } from "./errors.js";
+// Sound processing: what extensions plug into a render with
+export * as Sound from "./sound/Sound.js";
+export * as AudioProcessor from "./sound/AudioProcessor.js";

package/dist/points.d.ts

@@ -0,0 +1,28 @@
+import type { FilmPlan } from "@tongue-core/film";
+import { Effect } from "effect";
+import { InvalidRequest } from "./errors.js";
+/** What points are read against: a planned film's rate, length and clips. */
+export type Planned = Pick<FilmPlan, "fps" | "length" | "tracks">;
+/** A clip's frames: [first, end), the frames on which it is on screen. */
+export declare const clipRange: (plan: Planned, id: string) => Effect.Effect<{
+    readonly from: number;
+    readonly to: number;
+}, InvalidRequest, never>;
+/** A point in the film, as a frame: where a range starts or ends ("end" is the film's length). */
+export declare const parsePoint: (plan: Planned, text: string) => Effect.Effect<number, InvalidRequest, never>;
+/** A frame to draw: a point, where the film's end ("end", "100%") is its last frame. */
+export declare const parseFrame: (plan: Planned, text: string) => Effect.Effect<number, InvalidRequest, never>;
+/** The frames [from, to) a render covers: a clip's when `clip` is given, else the whole film, with
+ *  `from` and `to` (points) moving either end. */
+export declare const parseRange: (plan: Planned, request: {
+    readonly from?: string;
+    readonly to?: string;
+    readonly clip?: string;
+}) => Effect.Effect<{
+    readonly from: number;
+    readonly to: number;
+}, InvalidRequest, never>;
+/** `count` frames spread evenly over a film of `length` frames, first frame to last. */
+export declare function spreadFrames(length: number, count: number): ReadonlyArray<number>;
+/** Every `step` frames from the first, plus the last. */
+export declare function everyFrames(length: number, step: number): ReadonlyArray<number>;

package/dist/points.js

@@ -0,0 +1,76 @@
+import { Effect } from "effect";
+import { InvalidRequest } from "./errors.js";
+const refuse = (message) => new InvalidRequest({ message });
+/** A clip's frames: [first, end), the frames on which it is on screen. */
+export const clipRange = Effect.fnUntraced(function* (plan, id) {
+    const clip = plan.tracks.flatMap((t) => t.clips).find((c) => c.id === id);
+    if (!clip)
+        return yield* refuse(`There is no clip "${id}" in the film (clips: ${plan.tracks.flatMap((t) => t.clips.map((c) => c.id)).join(", ")}).`);
+    const range = { from: Math.max(0, Math.ceil(clip.start - 1e-6)), to: Math.ceil(clip.end - 1e-6) };
+    return range;
+});
+/** "90", "3s" or "50%" (of `whole` frames) as frames; null when `text` is none of those. */
+const amount = Effect.fnUntraced(function* (text, plan, whole) {
+    const match = /^(\d+(?:\.\d+)?)(s|%)?$/.exec(text);
+    if (!match)
+        return null;
+    const n = Number(match[1]);
+    const unit = match[2];
+    if (unit === "s")
+        return Math.round(n * plan.fps);
+    if (unit === "%")
+        return Math.round((n / 100) * whole);
+    if (!Number.isInteger(n))
+        return yield* refuse(`"${text}" isn't a whole frame; write seconds as "${text}s".`);
+    return n;
+});
+/** A point in the film, as a frame: where a range starts or ends ("end" is the film's length). */
+export const parsePoint = Effect.fnUntraced(function* (plan, text) {
+    const t = text.trim();
+    if (t === "end")
+        return plan.length;
+    const plain = yield* amount(t, plan, plan.length);
+    if (plain !== null)
+        return plain;
+    const inClip = /^([A-Za-z_][\w.-]*?)(?:\+(.+))?$/.exec(t);
+    if (inClip) {
+        const range = yield* clipRange(plan, inClip[1] ?? "");
+        const offset = inClip[2] === undefined ? 0 : yield* amount(inClip[2], plan, range.to - range.from);
+        if (offset === null)
+            return yield* refuse(`"${text}": after "+" write frames (30), seconds (1.5s) or a share of the clip (50%).`);
+        return range.from + offset;
+    }
+    return yield* refuse(`"${text}" isn't a point in the film: write a frame (90), seconds (3s), a share (50%) or a clip (site, site+30).`);
+});
+/** A frame to draw: a point, where the film's end ("end", "100%") is its last frame. */
+export const parseFrame = Effect.fnUntraced(function* (plan, text) {
+    const point = yield* parsePoint(plan, text);
+    if (point === plan.length)
+        return Math.max(0, plan.length - 1);
+    if (point > plan.length) {
+        const what = text.trim() === String(point) ? `Frame ${point} is` : `"${text}" is frame ${point},`;
+        return yield* refuse(`${what} past the film's last frame (${plan.length - 1}).`);
+    }
+    return point;
+});
+/** The frames [from, to) a render covers: a clip's when `clip` is given, else the whole film, with
+ *  `from` and `to` (points) moving either end. */
+export const parseRange = Effect.fnUntraced(function* (plan, request) {
+    const base = request.clip === undefined ? { from: 0, to: plan.length } : yield* clipRange(plan, request.clip);
+    const from = request.from === undefined ? base.from : yield* parsePoint(plan, request.from);
+    const to = request.to === undefined ? base.to : yield* parsePoint(plan, request.to);
+    const range = { from, to };
+    return range;
+});
+/** `count` frames spread evenly over a film of `length` frames, first frame to last. */
+export function spreadFrames(length, count) {
+    if (count <= 1 || length <= 1)
+        return [0];
+    const n = Math.min(count, length);
+    return Array.from({ length: n }, (_, i) => Math.round((i * (length - 1)) / (n - 1)));
+}
+/** Every `step` frames from the first, plus the last. */
+export function everyFrames(length, step) {
+    const out = Array.from({ length: Math.ceil(length / step) }, (_, i) => i * step);
+    return out.at(-1) === length - 1 ? out : [...out, length - 1];
+}

package/dist/project.d.ts

@@ -4,17 +4,21 @@ import { FilmFailure, InvalidRequest } from "./errors.js";
 import { type ImportReport } from "./import/importer.js";
 import type { FilmSource } from "./render/protocol.js";
 export interface Project {
-    /** "project": a film project (tongue.json). "module": one film module. */
-    readonly kind: "project" | "module";
+    /** "project": a film project (tongue.json). "build": a build folder (build.json). "module": one
+     *  film module. */
+    readonly kind: "project" | "build" | "module";
+    /** What asset sources are relative to: the project, the module's folder, or for a build, the
+     *  project it sits in (dist/ of a project) or else the build folder itself. */
     readonly root: string;
-    /** The film module (absolute): a project's film.tsx. */
+    /** The film (absolute): a project's film.tsx, a module, or a build's build.json. */
     readonly film: string;
-    /** Generated files: imported assets, builds, render work. */
+    /** Generated files: imported assets, builds, render work, caches. */
     readonly workDir: string;
     readonly assetsDir: string;
     readonly outDir: string;
 }
-/** The project `path` is: a project folder, or a film module (inside a project, or not). */
+/** The project `path` is: a project folder, a build folder, or a film module (inside a project, or
+ *  not). */
 export declare const openProject: (path: string) => Effect.Effect<Project, FilmFailure, FileSystem.FileSystem | Path.Path>;
 /** A film ready to render: its project, its planned timeline, and its imported assets. */
 export interface OpenFilm {
@@ -24,6 +28,11 @@ export interface OpenFilm {
     readonly imported: ImportReport;
     /** How render processes load the same film. */
     readonly source: FilmSource;
+    /** Where videos are decoded from when the film renders, by asset name (absolute paths). A
+     *  video not named here is decoded from its source path under the root. */
+    readonly sources?: Readonly<Record<string, string>>;
+    /** The scene opened on its own, when one was. */
+    readonly scene?: string;
 }
 export interface OpenOptions {
     /** One scene of a film project on its own, with these props (default: those of the first clip
@@ -32,9 +41,24 @@ export interface OpenOptions {
         readonly id: string;
         readonly props?: Json;
     };
+    /** Where videos are decoded from when the film renders, by asset name: for footage that isn't
+     *  at its source path under the root, such as a build rendered on a machine where the original
+     *  files live somewhere else. Relative paths are from the working directory. A render decodes
+     *  every video from its original file, and fails when one isn't there. */
+    readonly sources?: Readonly<Record<string, string>>;
 }
-/** Opens a film: a project folder (or a file in one), or a film module. */
+/** Opens a film: a project folder (or a file in one), a build folder, or a film module. */
 export declare const openFilm: (path: string, options?: OpenOptions | undefined) => Effect.Effect<OpenFilm, FilmFailure | import("./errors.js").AssetFailure | InvalidRequest | import("./errors.js").ToolFailure | import("@tongue-core/project").LayoutError | import("@tongue-core/project").CompileError | import("@tongue-core/project").LoadError | import("@tongue-core/project").TimelineError | import("@tongue-core/project").DocError | import("effect/Config").ConfigError | import("effect/PlatformError").PlatformError, import("effect/process/ChildProcessSpawner").ChildProcessSpawner | FileSystem.FileSystem | Path.Path>;
+/** Where each of the film's videos is decoded from when it renders, by asset name: its `sources`
+ *  entry, else its source path under the root. */
+export declare const videoInputs: (film: OpenFilm) => Readonly<Record<string, string>>;
+/** What a film's outputs are called: the project's folder (or the module's file) name, and the
+ *  scene's id when one scene was opened on its own. */
+export declare const outputName: (film: OpenFilm) => string;
+/** Where a render goes when no `out` is given: out/<name>.mp4 under the root. */
+export declare const defaultVideoOut: (film: OpenFilm) => string;
+/** Where stills go when no `outDir` is given: out/stills (out/stills-<scene> for one scene). */
+export declare const defaultStillsDir: (film: OpenFilm) => string;
 /** Builds a film project into `out` (default: its dist folder): build.json, film.json and a bundle
  *  per scene. */
 export declare const buildFilm: (path: string, options?: {