@tongue-core/sdk 0.1.0-rc.1 → 0.1.0-rc.10

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Backpine
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -7,9 +7,12 @@ here, with the same results. Built with Effect v4 (orchestration only: the per-f
 TypeScript); a Promise facade is included for applications that don't use Effect.
 
 ```bash
-bun add @tongue-core/sdk@next      # Bun ≥ 1.3; ffmpeg and ffprobe on the PATH
+bun add @tongue-core/sdk@next effect@4.0.6      # Bun ≥ 1.3; ffmpeg and ffprobe on the PATH
 ```
 
+`effect` is a peer dependency: your application and the SDK share one copy of it, so the SDK's
+services, errors and Layers are the same values as yours.
+
 ## Promise API
 
 ```ts
@@ -31,27 +34,32 @@ Failures reject with the SDK's tagged errors (below).
 ## Effect API
 
 ```ts
-import { BunServices } from "@effect/platform-bun";
-import { openFilm, renderVideo } from "@tongue-core/sdk";
+import { openFilm, renderVideo, TongueServices } from "@tongue-core/sdk";
 import { Effect } from "effect";
 
 const program = Effect.gen(function* () {
   const film = yield* openFilm("src/film.tsx");          // load, import assets, plan
   return yield* renderVideo(film, { out: "out/film.mp4", encoder: "auto", raster: "auto" });
 });
-await Effect.runPromise(program.pipe(Effect.provide(BunServices.layer)));
+await Effect.runPromise(program.pipe(Effect.provide(TongueServices.layer)));
 ```
 
+`TongueServices.layer` is what every SDK function runs on (the file system, paths and child
+processes, from Bun): provide it once, where your application runs its Effects.
+
 | Function | Does |
 |---|---|
 | `initProject(dir, { template, name })` | a film project in a new or empty folder, from a built-in template or a template folder |
 | `listTemplates()` / `readTemplate(nameOrPath)` | the built-in templates, or one by name or path |
-| `buildFilm(project, { out })` | builds a film project into `out` (default `dist/`): build.json, film.json, a bundle per scene |
-| `openFilm(path, { scene })` | a film ready to render: a project is built (into `.tongue/builds/<id>`, named by its content) and planned from film.json, a film module is imported and planned; assets imported (cached by size, mtime and options) either way. `scene: { id, props }` opens one scene of a project on its own |
+| `buildFilm(project, { out, media })` | builds a film project into `out` (default `dist/`): build.json, film.json, a bundle per scene; `media` configures acquisition and hosted previews |
+| `openFilm(path, { scene, sources, media })` | a film ready to render: a project is built (into `.tongue/builds/<id>`, named by its content) and planned from film.json, a build folder (what `buildFilm` writes) is opened as it is, a film module is imported and planned; assets imported either way. `scene: { id, props }` opens one scene of a project on its own; `sources` overrides video originals; `media` configures hosted-media acquisition and caching ([Footage](#footage)) |
 | `describeFilm(film)` | the plan as text (for agents to read) and as a JSON summary |
 | `checkFilm(path)` | problems and warnings as data |
 | `renderStills(film, { frames, scale, outDir })` / `renderSheet(…)` | PNGs drawn by the wasm raster (the player's), or tiled into one labelled sheet |
-| `renderVideo(film, options)` | an MP4 with sound: the whole film or `[from, to)`, at any scale |
+| `renderVideo(film, options)` | an MP4 with sound: the whole film or `[from, to)`, at any scale (`out` defaults to `out/<name>.mp4`); `sound` runs [sound processing](#sound-processing-extensions); `onStart` once the request is checked and the raster, encoder and workers are chosen (`budget` says why that many workers: one per physical core, fewer when memory is short), `onProgress` while it draws. A frame that throws fails the render with `Frame <n>: <message>`. The result's `profile` says where the time went: milliseconds per stage of drawing, decoder starts, pieces encoded, and the wall time of each phase |
+| `renderVideoEvents(film, options)` | the same render as a `Stream` of `RenderEvent`s: `Started`, `Frames`, `Sound` (each processor's progress), `Joining`, then `Done` with the result |
+| `parsePoint`, `parseFrame`, `parseRange`, `clipRange`, `spreadFrames`, `everyFrames` | points in a film as people write them (`90`, `3s`, `50%`, `end`, `site`, `site+30`), for a command line or form of your own; bad ones fail with `InvalidRequest` saying how to write them |
+| `videoInputs(film)`, `checkVideoInputs(film)` | where each video is decoded from, and a check that every one is there |
 | `listBackends()` | every raster and encoder, each tried for real |
 | `studioServer(path, { port, remote })` | the studio on a project (rebuilt and reloaded on every edit) or on a build served anywhere |
 
@@ -67,33 +75,211 @@ browser fetches it). `.tongue/types/film` links to the SDK's `@tongue-core/film`
 project's `tsconfig.json` points editors at.
 
 A **film module** (the way films were written before projects) is a `film.tsx` in a folder with a
-`package.json`, which installs `@tongue-core/film` and compiles JSX for it
-(`"jsx": "react-jsx", "jsxImportSource": "@tongue-core/film"`). It is imported and planned directly.
+`package.json`. The SDK bundles it with Bun (into `.tongue/modules/`, JSX compiled for Tongue) against
+its own `@tongue-core/film`, the copy the engine reads, as it links a project's bundles, then plans
+it. The folder may install `@tongue-core/film` too, for its editor's types
+(`"jsx": "react-jsx", "jsxImportSource": "@tongue-core/film"` in its tsconfig); that copy isn't the one
+that runs.
 
-Either way, asset paths are relative to the root, Tongue writes `.tongue/` there (imported assets,
-builds, render work) and renders to `out/`.
+Local asset paths are relative to the root. Tongue writes `.tongue/` there for builds and render
+work, and renders to `out/`. Local converted assets use the project's cache; hosted-media
+downloads and preparation use the shared cache outside the project.
 
 A film's assets are what its modules declare (`font()`, `image()`, `audio()`, `video()`) while
 the SDK loads it, plus the media its clips play; `<Film assets={[…]}>` replaces the declared ones.
 Importing turns them into what the engine reads: TrueType fonts, QOI images, WAV audio, video facts
 and a poster frame.
 
+## Footage
+
+Hosted URLs are the normal source for application-owned media. Declare them with `video`, `audio`,
+`image` or `font`, with an explicit name and a SHA-256 when the application knows it. The SDK
+streams downloads into a shared cache outside the project, verifies content, and atomically
+promotes completed files. Concurrent requests for the same pinned content share a download.
+Unpinned URLs are revalidated with ETag/Last-Modified when available. Temporary network failures,
+408, 429 and server errors retry twice by default, with bounded backoff and Retry-After support;
+missing files, denied access and integrity failures are reported as `MediaFailure`.
+
+```ts
+const tongue = createTongue({
+  media: {
+    cacheDir: "/var/cache/tongue/media",
+    store: { directory: "/srv/media", baseUrl: "https://media.example/prepared/" },
+  },
+});
+await tongue.build("my-film");
+await tongue.render("my-film/dist", { out: "out/film.mp4" });
+```
+
+A portable build contains manifests and scene bundles, with media locations and hashes in
+`build.json` version 2. Originals and generated remote media are never copied into it. Local
+assets can still be packaged, and version 1 builds remain readable. The host must expose the
+configured store at its `baseUrl` (with CORS for browser clients). The CLI reads the equivalent
+`TONGUE_MEDIA_DIR`, `TONGUE_MEDIA_URL` and `TONGUE_MEDIA_CACHE` environment variables. The default
+cache is `$XDG_CACHE_HOME/tongue/media` or `~/.cache/tongue/media`. `tongue studio` supplies a
+local media store and serves it automatically.
+
+For object storage, provide `media.publish(upload)`, an Effect returning the uploaded file's
+HTTP(S) URL. `upload` holds the local `file`, `hash`, `bytes` and `contentType`. Storage belongs
+to the application. A publish callback must finish storing the bytes before returning its URL.
+Use the hash as an idempotent storage key and return a stable URL, so projects share uploaded
+bytes. The built-in directory store does this automatically. Invalid publisher locations fail
+before a portable build is written.
+
+Applications can prepare media at ingestion: `prepareMedia(asset, mediaOptions)` returns a
+`tongue.media` manifest with a canonical asset name, import recipe, source hash, duration and
+prepared files. Host that JSON and pass its URL as the declaration's `prepared` option. The same
+preparation is shared across projects and asset names. Preparation recipes include sizing and
+format versions, so incompatible results are rebuilt. Audio previews are sample-indexed WAV
+segments; video previews use the existing segmented H.264 format. Full-quality decoded audio
+remains available to final renders.
+
+`media.request(url, signal, headers)` can add authentication or refresh a signed URL without
+serializing credentials into the build. Preserve the supplied signal and conditional headers.
+`retries`, `retryDelayMs`, `timeoutMs`, `concurrency` and `onProgress` control acquisition. Original
+downloads default to a 30-minute timeout; metadata/preview requests default to two minutes.
+Cache contents are shared and host-managed; changing the cache directory isolates a job.
+
+Final rendering acquires and verifies its video originals before starting any frame worker.
+Converted assets are also acquired before use. Workers decode local files, with no networking
+in the per-frame path. A fully cached pinned build can render with networking disabled. Build
+snapshots retain immutable prepared-file bindings when a later version changes its assets.
+
+A render decodes every video from its **original file**: a build holds only a video's facts, a
+poster and a preview-quality proxy (what the studio plays). Before a frame is drawn, `renderVideo`
+and `renderStills` check that each video's file is there and fail with `AssetFailure` naming the
+ones that aren't; a frame ffmpeg can't decode, or a file shorter than its import counted, fails the
+render with `Frame <n>: Video "<name>": …` and ffmpeg's reason. A poster never stands in for
+footage in a render (that is the player's fallback while frames arrive).
+
+A local video is decoded from its source path under the film's root, unless `sources` says otherwise:
+
+```ts
+const film = yield* openFilm("renders/1234/dist", {   // a build, without the project around it
+  sources: { screen: "/data/media/screen.mp4", camera: "/data/media/camera.mp4" },
+});
+```
+
+A build inside a project (`my-film/dist`) finds local footage in the project; a build on its own
+needs `sources` for local videos that aren't beside it. Remote originals are resolved automatically
+from the pinned registry in the manifest.
+
+## Sound processing (extensions)
+
+A render can put part of its soundtrack through processing of your own (voice enhancement, noise
+removal, levelling, effects) and still produce one MP4, in sync. The SDK defines the contract and
+runs it; what a processor does and what it needs (a model, a native tool, a service) is yours, so
+nobody who doesn't use it installs it.
+
+```ts
+import { AudioProcessor, openFilm, renderVideo, Sound } from "@tongue-core/sdk";
+
+const enhance = yield* enhanceVoice({ peakDb: -3 });        // an AudioProcessor (below)
+yield* renderVideo(film, {
+  sound: [
+    Sound.track("voiceover", enhance),   // the clips of the track with id "voiceover"
+    Sound.track("voiceover", level),     // the same selection again: runs after enhance
+    Sound.master(loudness),              // then the whole mix
+  ],
+});
+```
+
+**What a rule takes.** `Sound.track(id, p)`; `Sound.stem({ clips: ["a", "b"] }, p)`;
+`Sound.stem({ voice: true }, p)` (every narration clip); `Sound.master(p)` (the whole mix, after
+the stems, before the limiter). A selection is data (`Sound.AudioSelection` decodes it from flags or
+JSON). The selected clips are mixed into a **stem** of their own (their trims, speed, gain and fades
+applied) across the whole film, processed, and added back; everything else sounds as it always has.
+A track or clip the film doesn't have, or a visual track, is an `InvalidRequest` listing what it
+does have; a clip goes into one stem, so overlapping selections are refused unless they are the same
+(then their processors run in order).
+
+**The contract.** A processor gets a `Stem`: `samples` per channel (the whole film at 48 kHz, from
+its first frame), `channels` (2), the stem as a float WAV `file` and as decoded `pcm`, the `clips` in
+it and the `plan`, a `workDir`, and `progress(0..1)`. It returns `{ pcm }` or `{ file }`, plus an
+optional `report` (any JSON). What it returns must have exactly the stem's rate, channels and
+length, and only finite samples, or the render fails with `ProcessorFailure`: that is what keeps
+the sound in sync, so a processor with a delay (a model's lookahead) takes it out itself. Any
+failure of the processor, or a defect in it, is a `ProcessorFailure` naming it.
+
+**Two ways to make one.** `AudioProcessor.fromPcm({ key, process: (pcm, stem) => pcm })` for
+processing that needs only the samples. `AudioProcessor.make({ key, process: (stem) => Effect })`
+for anything else: the services `process` uses are taken from where `make` runs and travel with the
+processor, so the render itself never needs them:
+
+```ts
+// voice-enhance: its own package, depending on @tongue-core/sdk and on what it needs.
+interface DenoiserOperations {
+  readonly name: string;
+  readonly denoise: (channel: Float32Array, sampleRate: number) => Effect.Effect<Float32Array>;
+}
+export class Denoiser extends Context.Service<Denoiser, DenoiserOperations>()("voice-enhance/Denoiser") {
+  static readonly layer = Layer.effect(Denoiser, loadModel);   // the heavy part, paid once, only by those who provide it
+}
+
+export const enhanceVoice = (settings: EnhanceSettings) =>
+  AudioProcessor.make({
+    key: `voice-enhance/v1 peak ${settings.peakDb} dB`,
+    process: (stem) => Effect.gen(function* () {
+      const denoiser = yield* Denoiser;
+      const pcm = yield* stem.pcm;
+      const channels = yield* Effect.forEach(pcm.channels, (c) => denoiser.denoise(c, pcm.sampleRate));
+      yield* stem.progress(1);
+      return { pcm: { sampleRate: pcm.sampleRate, channels }, report: { denoiser: denoiser.name } };
+    }),
+  });   // Effect<AudioProcessor, never, Denoiser>
+
+// In your command line: make the processor where its Layer is provided.
+const render = Command.make("render", { project: Argument.String("project"), enhance: Flag.String("enhance").pipe(Flag.atLeast(0)) },
+  Effect.fn(function* (flags) {
+    const film = yield* openFilm(flags.project);
+    const enhance = yield* enhanceVoice({ peakDb: -3 });
+    return yield* renderVideo(film, { sound: flags.enhance.map((track) => Sound.track(track, enhance)) });
+  }),
+).pipe(Command.provide(Denoiser.layer));
+```
+
+**The key and the cache.** A processor's `key` names it, its version and its settings. A result is
+cached by the key and the exact audio it was given (`.tongue/cache/sound`, the newest 32 kept): render
+again after changing anything that doesn't change that audio (the picture, another track, the
+range, the scale) and the processor doesn't run; the result says `cached: true`. Change the key
+whenever the output would change.
+
+**What happened.** `VideoResult.sound` lists each processor run: the stem, its clips, the
+processor's key, whether it was cached, its seconds and its report. `renderVideoEvents` streams
+`Sound` events (`started`, `progress`, `done`, `cached`) while it runs.
+
+**Where it runs.** Processors run in your process, alongside the render workers, while the frames
+are drawn (the soundtrack always has); keep heavy work off its main thread (`effect/workers`, a
+child process), or it slows the hand-off of frames to the workers.
+
 ## How a video renders
 
 ```
-coordinator ──segments──▶ worker 1: frames ─▶ raster ─▶ encoder ─▶ seg-0000.mp4
-            ◀─progress──  worker 2 …                              seg-0001.mp4
+coordinator ──segments──▶ worker 1: frames ─▶ raster ─▶ encoder ─▶ piece-000000.mp4
+            ◀─progress──  worker 2 …                              piece-000280.mp4
             soundtrack:   sounds + audio clips ─▶ mix ─▶ AAC        audio.m4a
+              with sound processing: stems ─▶ processors (here, cached) ─▶ sum ─▶ master ─▶ limiter ─▶ AAC
             join:         concat + audio ─(stream copy)─▶ film.mp4, frame count probed
 ```
 
-- **Workers** are processes (`src/render/worker.ts`), one per physical core by default. Each loads
-  the film and its raster once and renders every segment it is sent into that segment's own
-  encoder, so the JIT stays warm and glyphs and images stay uploaded.
-- **Segments** are one to two seconds of frames, shrinking towards the end so the last ones finish
-  together. Each is a closed-GOP encode; the join copies streams without re-encoding.
-- **Video clips** are decoded by ffmpeg in order, frame after frame, and restarted on jumps.
-- **The soundtrack** is mixed in its own process while the workers draw (`soundtrack-process.ts`).
+- **Workers** are processes (`src/render/worker.ts`), one per physical core by default, fewer when
+  the memory available wouldn't hold that many (`src/render/budget.ts`). Each loads
+  the film and its raster once and draws every segment it is sent, so the JIT stays warm and glyphs
+  and images stay uploaded.
+- **Segments** are a quarter to half a second of frames: what a worker is given at a time. Each
+  worker starts on its own contiguous run of them; one that finishes early takes over the later
+  half of the longest run left (`src/render/runs.ts`). So every worker draws long contiguous
+  stretches, and its video decoders read straight through instead of seeking.
+- **Pieces** are what is encoded: each contiguous stretch a worker draws goes to one encoder, as one
+  closed-GOP file; the join copies streams without re-encoding. Which pieces a render makes depends
+  on how the work was shared out; the frames don't.
+- **Video clips** are decoded by ffmpeg from their original files, in order, frame after frame, a
+  few frames ahead, through a FIFO that holds ffmpeg back when it gets ahead; restarted on jumps,
+  a window at a time for clips playing backwards, and closed when unused. A frame that can't be
+  decoded fails the render ([Footage](#footage)).
+- **The soundtrack** is mixed in its own process while the workers draw (`soundtrack-process.ts`);
+  with sound processing, it writes the stems, the processors run in the coordinator, and it adds
+  them back, limits and encodes (`sound/soundtrack.ts`).
 
 Rasters (who draws) and encoders (who encodes) are chosen by name, because the choice is made here
 but runs inside the workers:
@@ -119,11 +305,12 @@ gives the CLI's exit code.
 | Error | When | Exit |
 |---|---|---|
 | `FilmFailure` | the module is missing, throws, isn't a `<Film>`, or its timeline doesn't plan | 2 |
-| `AssetFailure` | an asset's file is missing or unreadable | 2 |
+| `AssetFailure` | an asset's file is missing or unreadable, or a video's original file isn't there to render from | 2 |
 | `InvalidRequest` | a frame outside the film, a bad range or scale | 2 |
 | `ToolFailure` | ffmpeg or ffprobe is missing or failed | 3 |
 | `BackendUnavailable` | the raster or encoder asked for doesn't work here | 3 |
-| `RenderFailure` | a worker died or a frame failed | 3 |
+| `RenderFailure` | a worker died, a frame failed, or a video frame couldn't be decoded | 3 |
+| `ProcessorFailure` | a sound processor failed, or returned audio that would break sync (rate, channels, length) or isn't audio | 3 |
 | `LayoutError`, `CompileError`, `LoadError`, `TimelineError`, `DocError`, `TemplateError` | a film project doesn't build (from `@tongue-core/project`: each names its file, line or path) | 2 |
 
 ## Environment
@@ -144,10 +331,14 @@ src/
   init.ts                 init from templates
   check.ts, backends.ts   checkFilm, listBackends
   import/                 asset import: fonts, images, audio, video (ffprobe, posters)
-  render/                 stills, the video pipeline, its worker and soundtrack processes, video decoding
+  render/                 stills, the video pipeline and its events, its worker and soundtrack processes, video decoding and inputs
+  sound/                  sound processing: Sound (rules), AudioProcessor (the contract), the soundtrack around them
+  points.ts               points and ranges in a film, as people write them
+  TongueServices.ts       the layer the SDK runs on
   encode/encoders.ts      the ffmpeg encoders and how each is tried
   raster/                 the native and GPU raster bindings (index.ts) and the registry (rasters.ts)
   studio/server.ts        the studio's server: the app bundled, a project's build served and rebuilt
-test/                     unit tests, end-to-end ones on fixtures/tiny (a film with text, video and sound), and
+test/                     unit tests, end-to-end ones on fixtures/tiny (a film with text, video and sound; sound
+                          processing and footage that can't be decoded on copies of it), builds rendered with `sources`, and
                           projects from every template (project.test.ts: film.json draws the same frames as film.tsx)
 ```

package/dist/TongueServices.d.ts

@@ -0,0 +1,5 @@
+import { BunServices } from "@effect/platform-bun";
+import type { Layer } from "effect";
+/** The services every SDK function needs. */
+export type TongueServices = BunServices.BunServices;
+export declare const layer: Layer.Layer<TongueServices>;

package/dist/TongueServices.js

@@ -0,0 +1,2 @@
+import { BunServices } from "@effect/platform-bun";
+export const layer = BunServices.layer;

package/dist/build/build.d.ts

@@ -3,6 +3,7 @@ import { type Build, type ModuleHost, type ProjectFiles } from "@tongue-core/pro
 import { Effect, FileSystem, type PlatformError } from "effect";
 import { type ImportReport } from "../import/importer.js";
 import type { Project } from "../project.js";
+import { type MediaOptions } from "../media/cache.js";
 /** A project's code and data: every .ts, .tsx and .json file outside hidden folders and the ones
  *  tools write. */
 export declare const readProjectFiles: (root: string) => Effect.Effect<ProjectFiles, PlatformError.PlatformError, FileSystem.FileSystem>;
@@ -11,6 +12,7 @@ export declare const fileModuleHost: (dir: string) => ModuleHost;
 /** Points the project's `.tongue/types/film` at this SDK's @tongue-core/film (tsconfig.json reads it). */
 export declare const linkFilmTypes: (root: string) => Effect.Effect<void, PlatformError.PlatformError, FileSystem.FileSystem>;
 export interface BuiltProject {
+    readonly assetsDir: string;
     readonly build: Build;
     /** Where the build was written. */
     readonly dir: string;
@@ -25,6 +27,7 @@ export declare const importedFiles: (project: Project, assets: readonly Asset[])
     readonly path: string;
     readonly bytes: number;
     readonly hash: string;
+    readonly url?: string | undefined;
 }[], PlatformError.PlatformError, FileSystem.FileSystem>;
 /** How a written build reaches its assets: copies of the files (a build to publish), or a link to
  *  the project's imported assets (a build for Tongue's own use). */
@@ -43,6 +46,6 @@ export declare const writeBuildFiles: (dir: string, build: Build, assets: AssetP
 export declare const keepBuild: (project: Project, build: Build) => Effect.Effect<string, PlatformError.PlatformError, FileSystem.FileSystem>;
 /** Builds a film project into `dir` (what `tongue build` does: dist/ by default), replacing what
  *  was there, with copies of its imported assets: a folder any static host can serve. */
-export declare const buildFilmProject: (project: Project, dir: string) => Effect.Effect<BuiltProject, import("../errors.js").AssetFailure | import("../errors.js").ToolFailure | import("@tongue-core/project").LayoutError | import("@tongue-core/project").CompileError | import("@tongue-core/project").LoadError | import("@tongue-core/project").TimelineError | import("@tongue-core/project").DocError | import("effect/Config").ConfigError | PlatformError.PlatformError, import("effect/process/ChildProcessSpawner").ChildProcessSpawner | FileSystem.FileSystem | import("effect/Path").Path>;
+export declare const buildFilmProject: (project: Project, dir: string, options?: MediaOptions | undefined) => Effect.Effect<BuiltProject, import("../errors.js").AssetFailure | import("../errors.js").ToolFailure | import("@tongue-core/project/media").MediaFailure | import("@tongue-core/project").LayoutError | import("@tongue-core/project").CompileError | import("@tongue-core/project").LoadError | import("@tongue-core/project").TimelineError | import("@tongue-core/project").DocError | import("effect/Config").ConfigError | PlatformError.PlatformError | import("effect/Schema").SchemaError, import("effect/process/ChildProcessSpawner").ChildProcessSpawner | FileSystem.FileSystem | import("effect/Path").Path>;
 /** Builds a film project for Tongue's own use (stills, renders, dev): kept in .tongue/builds. */
-export declare const buildForWork: (project: Project) => Effect.Effect<BuiltProject, import("../errors.js").AssetFailure | import("../errors.js").ToolFailure | import("@tongue-core/project").LayoutError | import("@tongue-core/project").CompileError | import("@tongue-core/project").LoadError | import("@tongue-core/project").TimelineError | import("@tongue-core/project").DocError | import("effect/Config").ConfigError | PlatformError.PlatformError, import("effect/process/ChildProcessSpawner").ChildProcessSpawner | FileSystem.FileSystem | import("effect/Path").Path>;
+export declare const buildForWork: (project: Project, options?: MediaOptions | undefined) => Effect.Effect<BuiltProject, import("../errors.js").AssetFailure | import("../errors.js").ToolFailure | import("@tongue-core/project/media").MediaFailure | import("@tongue-core/project").LayoutError | import("@tongue-core/project").CompileError | import("@tongue-core/project").LoadError | import("@tongue-core/project").TimelineError | import("@tongue-core/project").DocError | import("effect/Config").ConfigError | PlatformError.PlatformError | import("effect/Schema").SchemaError, import("effect/process/ChildProcessSpawner").ChildProcessSpawner | FileSystem.FileSystem | import("effect/Path").Path>;

package/dist/build/build.js

@@ -14,6 +14,9 @@ import { compileProject, extractFilm, IGNORED_DIRS, loadProject, sha256, TYPES_L
 import { Effect, FileSystem } from "effect";
 import { importAssets } from "../import/importer.js";
 import { sdkVersion } from "../version.js";
+import { isRemoteSource } from "@tongue-core/project/media";
+import { mediaCacheDir, mediaLock } from "../media/cache.js";
+import { snapshotAssets } from "../media/snapshot.js";
 // Building a film project on this machine: read its code, compile and bundle it, run the bundles,
 // import the assets they declare (media lengths go into film.json), extract film.json and write
 // the build, with the imported asset files listed in build.json (and copied beside it, or linked,
@@ -99,6 +102,8 @@ export const writeBuildFiles = Effect.fn("writeBuildFiles")(function* (dir, buil
         return;
     }
     for (const file of build.manifest.assets.files) {
+        if (file.url !== undefined)
+            continue;
         const path = join(assetsDir, file.path);
         yield* fs.makeDirectory(dirname(path), { recursive: true });
         yield* fs.copyFile(join(assets.from, file.path), path);
@@ -135,28 +140,36 @@ const pruneBuilds = Effect.fn("pruneBuilds")(function* (builds, keep) {
 });
 /** A film project built in memory: compiled, its bundles run, its assets imported (their lengths
  *  go into film.json), film.json extracted. */
-const buildInMemory = Effect.fn("buildInMemory")(function* (project) {
-    const files = yield* readProjectFiles(project.root);
-    const compiled = yield* compileProject(files);
-    const loaded = yield* loadProject(compiled, fileModuleHost(join(project.workDir, "modules")));
-    const imported = yield* importAssets(project, loaded.assets);
-    const media = { duration: (asset) => imported.durations.get(`${asset.kind}:${asset.name}`) ?? null };
-    const doc = yield* extractFilm(loaded, { media });
-    const build = writeBuild(compiled, doc, { tongue: yield* sdkVersion, assets: yield* importedFiles(project, loaded.assets) });
-    yield* linkFilmTypes(project.root);
-    return { build, imported, scenes: new Map(loaded.scenes.map((s) => [s.id, s.scene])), media };
+const buildInMemory = Effect.fn("buildInMemory")(function* (input, options = {}) {
+    return yield* mediaLock(`project-build:${input.root}`, Effect.gen(function* () {
+        let project = input;
+        const files = yield* readProjectFiles(project.root);
+        const compiled = yield* compileProject(files);
+        const loaded = yield* loadProject(compiled, fileModuleHost(join(project.workDir, "modules")));
+        if (loaded.assets.some((asset) => isRemoteSource(asset.source)))
+            project = { ...project, assetsDir: join(mediaCacheDir(options), "projects", createHash("sha256").update(project.root).digest("hex"), "assets") };
+        const imported = yield* importAssets(project, loaded.assets, options);
+        const media = { duration: (asset) => imported.durations.get(`${asset.kind}:${asset.name}`) ?? null };
+        const doc = yield* extractFilm(loaded, { media });
+        const local = yield* importedFiles(project, loaded.assets.filter((asset) => !isRemoteSource(asset.source)));
+        const build = writeBuild(compiled, doc, { tongue: yield* sdkVersion, assets: [...local, ...(imported.remoteFiles ?? [])], originals: imported.originals });
+        if ((imported.originals?.length ?? 0) > 0 && !options.portable)
+            project = { ...project, assetsDir: yield* snapshotAssets(project.assetsDir, build.manifest.assets.files, imported.remoteFiles ?? [], options) };
+        yield* linkFilmTypes(project.root);
+        return { build, imported, scenes: new Map(loaded.scenes.map((s) => [s.id, s.scene])), media, assetsDir: project.assetsDir };
+    }));
 });
 /** Builds a film project into `dir` (what `tongue build` does: dist/ by default), replacing what
  *  was there, with copies of its imported assets: a folder any static host can serve. */
-export const buildFilmProject = Effect.fn("buildFilmProject")(function* (project, dir) {
-    const built = yield* buildInMemory(project);
-    yield* writeBuildFiles(dir, built.build, { kind: "copy", from: project.assetsDir });
+export const buildFilmProject = Effect.fn("buildFilmProject")(function* (project, dir, options = {}) {
+    const built = yield* buildInMemory(project, { ...options, portable: true });
+    yield* writeBuildFiles(dir, built.build, { kind: "copy", from: built.assetsDir });
     const out = { ...built, dir };
     return out;
 });
 /** Builds a film project for Tongue's own use (stills, renders, dev): kept in .tongue/builds. */
-export const buildForWork = Effect.fn("buildForWork")(function* (project) {
-    const built = yield* buildInMemory(project);
-    const out = { ...built, dir: yield* keepBuild(project, built.build) };
+export const buildForWork = Effect.fn("buildForWork")(function* (project, options = {}) {
+    const built = yield* buildInMemory(project, options);
+    const out = { ...built, dir: yield* keepBuild({ ...project, assetsDir: built.assetsDir }, built.build) };
     return out;
 });

package/dist/check.d.ts

@@ -1,5 +1,6 @@
 import type { PlanSummary } from "@tongue-core/film";
 import { Effect } from "effect";
+import { type OpenOptions } from "./project.js";
 export interface CheckProblem {
     /** layout, compile, load, timeline, doc (film.json), asset. */
     readonly kind: string;
@@ -15,4 +16,4 @@ export interface CheckReport {
     readonly summary: PlanSummary | null;
     readonly text: string | null;
 }
-export declare const checkFilm: (path: string) => Effect.Effect<CheckReport, import("./errors.js").InvalidRequest | import("./errors.js").ToolFailure | import("effect/Config").ConfigError | import("effect/PlatformError").PlatformError, import("effect/process/ChildProcessSpawner").ChildProcessSpawner | import("effect/FileSystem").FileSystem | import("effect/Path").Path>;
+export declare const checkFilm: (path: string, options?: OpenOptions | undefined) => Effect.Effect<CheckReport, import("./errors.js").InvalidRequest | import("./errors.js").ToolFailure | import("effect/Config").ConfigError | import("effect/PlatformError").PlatformError | import("effect/Schema").SchemaError, import("effect/process/ChildProcessSpawner").ChildProcessSpawner | import("effect/FileSystem").FileSystem | import("effect/Path").Path>;

package/dist/check.js

@@ -1,13 +1,14 @@
 import { Effect } from "effect";
 import { describeFilm, openFilm } from "./project.js";
 const failed = (problems) => ({ ok: false, problems, warnings: [], summary: null, text: null });
-export const checkFilm = Effect.fn("checkFilm")(function* (path) {
-    return yield* openFilm(path).pipe(Effect.map((film) => {
+export const checkFilm = Effect.fn("checkFilm")(function* (path, options = {}) {
+    return yield* openFilm(path, options).pipe(Effect.map((film) => {
         const { text, summary } = describeFilm(film);
         return { ok: true, problems: [], warnings: film.plan.warnings, summary, text };
     }), Effect.catchTags({
         FilmFailure: (e) => Effect.succeed(failed([{ kind: e.phase === "load" ? "load" : "timeline", file: e.film, message: e.message }])),
         AssetFailure: (e) => Effect.succeed(failed([{ kind: "asset", file: null, message: e.message }])),
+        MediaFailure: (e) => Effect.succeed(failed([{ kind: "asset", file: null, message: e.message }])),
         LayoutError: (e) => Effect.succeed(failed(e.problems.map((p) => ({ kind: "layout", file: p.file, message: p.message })))),
         CompileError: (e) => Effect.succeed(failed([{ kind: "compile", file: e.line === null ? e.file : `${e.file}:${e.line}`, message: e.message }])),
         LoadError: (e) => Effect.succeed(failed([{ kind: "load", file: e.file, message: e.message }])),

package/dist/client.d.ts

@@ -2,7 +2,10 @@ import { type InitOptions } from "./init.js";
 import { type OpenOptions } from "./project.js";
 import { type VideoOptions } from "./render/pipeline.js";
 import { type StillsOptions } from "./render/stills.js";
-export declare function createTongue(): {
+import type { MediaOptions } from "./media/cache.js";
+export declare function createTongue(configuration?: {
+    readonly media?: MediaOptions;
+}): {
     /** Makes a film project in a new or empty folder from a template. */
     init: (dir: string, options?: InitOptions) => Promise<import("./init.js").InitReport>;
     /** The templates that come with Tongue. */
@@ -18,6 +21,7 @@ export declare function createTongue(): {
     /** Builds a film project: build.json, film.json and a bundle per scene, in its dist folder. */
     build: (project: string, options?: {
         readonly out?: string;
+        readonly media?: MediaOptions;
     }) => Promise<import("./index.js").BuiltProject>;
     /** Imports the film's assets and plans it; resolves to its timeline as text and JSON. */
     info: (film: string) => Promise<{

package/dist/client.js

@@ -1,4 +1,3 @@
-import { BunServices } from "@effect/platform-bun";
 import { Cause, Effect, Exit } from "effect";
 import { listBackends } from "./backends.js";
 import { checkFilm } from "./check.js";
@@ -6,32 +5,33 @@ import { initProject, listTemplates } from "./init.js";
 import { buildFilm, describeFilm, openFilm } from "./project.js";
 import { renderVideo } from "./render/pipeline.js";
 import { renderSheet, renderStills } from "./render/stills.js";
+import * as TongueServices from "./TongueServices.js";
 // The SDK as plain Promises, for applications that don't use Effect. Each call runs the same Effect
 // program the CLI runs; a failure rejects with the SDK's tagged error (FilmFailure, ToolFailure…).
 const run = async (effect) => {
-    const exit = await Effect.runPromiseExit(effect.pipe(Effect.provide(BunServices.layer)));
+    const exit = await Effect.runPromiseExit(effect.pipe(Effect.provide(TongueServices.layer)));
     if (Exit.isSuccess(exit))
         return exit.value;
     throw Cause.squash(exit.cause);
 };
-export function createTongue() {
+export function createTongue(configuration = {}) {
     return {
         /** Makes a film project in a new or empty folder from a template. */
         init: (dir, options) => run(initProject(dir, options)),
         /** The templates that come with Tongue. */
         templates: () => run(listTemplates()),
         /** Builds a film project: build.json, film.json and a bundle per scene, in its dist folder. */
-        build: (project, options) => run(buildFilm(project, options)),
+        build: (project, options) => run(buildFilm(project, { media: configuration.media, ...options })),
         /** Imports the film's assets and plans it; resolves to its timeline as text and JSON. */
-        info: (film) => run(openFilm(film).pipe(Effect.map(describeFilm))),
+        info: (film) => run(openFilm(film, configuration).pipe(Effect.map(describeFilm))),
         /** Problems and warnings as data: never rejects for a broken film. */
-        check: (film) => run(checkFilm(film)),
+        check: (film) => run(checkFilm(film, configuration)),
         /** Frames as PNG files. */
-        stills: (film, options) => run(openFilm(film, options).pipe(Effect.flatMap((f) => renderStills(f, options)))),
+        stills: (film, options) => run(openFilm(film, { ...configuration, ...options }).pipe(Effect.flatMap((f) => renderStills(f, options)))),
         /** Frames tiled into one labelled image. */
-        sheet: (film, options) => run(openFilm(film, options).pipe(Effect.flatMap((f) => renderSheet(f, options)))),
+        sheet: (film, options) => run(openFilm(film, { ...configuration, ...options }).pipe(Effect.flatMap((f) => renderSheet(f, options)))),
         /** The film (or a range of it, or one scene of a project) as an MP4. */
-        render: (film, options) => run(openFilm(film, options).pipe(Effect.flatMap((f) => renderVideo(f, options)))),
+        render: (film, options) => run(openFilm(film, { ...configuration, ...options }).pipe(Effect.flatMap((f) => renderVideo(f, options)))),
         /** The rasters and encoders this machine can use. */
         backends: () => run(listBackends()),
     };

package/dist/encode/encoders.d.ts

@@ -23,7 +23,8 @@ export interface EncoderSpec {
     /** The filter from the raw input to what the encoder takes (uploads to the GPU). */
     readonly filter: string | null;
     readonly encode: ReadonlyArray<string>;
-    /** AMD's HEVC encoder writes no conformance window, so 1080 rows decode as 1088. */
+    /** AMD's HEVC encoder pads the picture and writes no conformance window, or a wrong one, so 1080
+     *  rows decode as 1088 and 540 columns as 572: the window is set again when muxing. */
     readonly fixCrop: boolean;
 }
 /** Where ffmpeg and ffprobe are. */

package/dist/encode/encoders.js

@@ -2,7 +2,7 @@
 // (VAAPI on AMD and Intel under Linux, NVENC on NVIDIA, Quick Sync on Intel) differ only in the
 // arguments around the input. `available()` runs a short encode through the exact same arguments,
 // so it proves the ffmpeg build, the device and the driver all work. Plain TypeScript: encoders run
-// in the render loop. The settings are the ones measured in docs/render-pipeline.md.
+// in the render loop. The settings are the ones measured in docs/render-performance.md.
 export class EncoderError extends Error {
     name = "EncoderError";
 }
@@ -23,10 +23,12 @@ async function run(cmd) {
 /** Muxes a raw HEVC stream into the MP4, setting the conformance window (no re-encode) when the
  *  driver padded the picture. Done on the raw stream: an MP4 already carries the unpadded size. */
 async function muxHevc(tools, raw, target) {
-    const probe = await run([tools.ffprobe, "-v", "error", "-select_streams", "v", "-show_frames", "-read_intervals", "%+#1", "-show_entries", "frame=width,height", "-of", "csv=p=0", raw]);
+    // From the coded size, not the decoded one: hevc_metadata replaces the stream's window, and the
+    // driver sometimes writes part of one (540 across is coded as 576 and windowed to 572).
+    const probe = await run([tools.ffprobe, "-v", "error", "-select_streams", "v", "-show_entries", "stream=coded_width,coded_height", "-of", "csv=p=0", raw]);
     const [width = 0, height = 0] = (probe.stdout.trim().split("\n")[0] ?? "").split(",").map(Number);
     if (width < target.width || height < target.height)
-        throw new EncoderError(`${raw} decodes at ${width}×${height}, smaller than ${target.width}×${target.height}.`);
+        throw new EncoderError(`${raw} is coded at ${width}×${height}, smaller than ${target.width}×${target.height}.`);
     const crop = width !== target.width || height !== target.height ? ["-bsf:v", `hevc_metadata=crop_right=${width - target.width}:crop_bottom=${height - target.height}`] : [];
     const mux = await run([tools.ffmpeg, "-hide_banner", "-v", "error", "-y", "-framerate", String(target.fps), "-i", raw, "-c", "copy", ...crop, "-tag:v", "hvc1", target.out]);
     if (mux.code !== 0)
@@ -92,7 +94,7 @@ export async function encoderWorks(ffmpeg, spec, size = { width: 320, height: 24
 }
 export const DEFAULT_SETTINGS = { preset: "veryfast", crf: 14, threads: 2, qp: null, gop: null, x264Params: "" };
 export const X264_PRESETS = ["ultrafast", "superfast", "veryfast", "faster", "fast", "medium", "slow", "slower", "veryslow"];
-/** Constant QP that clears the master-quality bar on our films (see docs/render-pipeline.md). */
+/** Constant QP that clears the master-quality bar on our films. */
 const HARDWARE_QP = { h264: 16, hevc: 18 };
 const profile = (codec) => (codec === "hevc" ? "main" : "high");
 const vaapiDevice = () => process.env["TONGUE_VAAPI_DEVICE"] ?? "/dev/dri/renderD128";