@hraness/slopcamera 3.3.2 → 3.3.4

package/README.md

@@ -9,10 +9,21 @@ from retained sources.**
 Describe a finished result to Codex, Claude, or another coding agent. Slopcamera gives
 it a Bun CLI, TypeScript SDK, and version-matched Agent Skill to inspect sources,
 direct cameras, edit a composition, and render the result. A separate MCP server
-exposes a fixed diagram and image toolset. There is no Slopcamera account or hosted
+exposes fixed tools for diagrams, images, and scene inspection and planning. There is no Slopcamera account or hosted
 project database.
 
-[Install](#install-slopcamera) · [Make a first diagram](#make-your-first-diagram) · [Capabilities](#what-slopcamera-does) · [Documentation](docs/README.md) · [slopcamera.com](https://slopcamera.com)
+[Install](#install-slopcamera) · [Make a first diagram](#make-your-first-diagram) · [Capabilities](#what-slopcamera-does) · [Documentation](https://slopcamera.com/docs) · [slopcamera.com](https://slopcamera.com)
+
+## Made with Slopcamera
+
+[![An original brass optical instrument rendered in Blender](apps/web/media/native-product-poster-51b1658c46d3.webp)](https://slopcamera.com/docs/tutorials/first-native-film#inspect-the-finished-example)
+
+Watch the [six-second product film](https://slopcamera.com/docs/tutorials/first-native-film#inspect-the-finished-example),
+then change its [geometry, materials and camera](examples/showcase/native/product).
+For 2D motion, try the [editable editorial animation](https://slopcamera.com/docs/tutorials/first-animation).
+The [portrait geometry lesson](https://slopcamera.com/docs/how-to/educational-video#inspect-the-finished-example)
+adds mathematical typesetting, a presenter and authored captions. Each guide pairs
+the rendered output with source, runtime requirements and reproduction steps.
 
 ## Why Slopcamera
 
@@ -33,14 +44,14 @@ project database.
 
 ## Install Slopcamera
 
-Slopcamera v3.3.1 adds [parametric architectural designs](docs/how-to/parametric-design.md): editable dimensions, constraints, four starters, and galleries of retained 3D scenes. The Three renderer fits shadows to the casting geometry.
+Slopcamera v3.3.3 corrects wall openings, arched crowns, and Boolean surfaces, and documents [film integration](docs/how-to/parametric-design.md#integrate-a-design-into-an-existing-film).
 
 Install [Bun 1.3.14 or newer](https://bun.sh), then install the verified
-[Slopcamera v3.3.1 release](https://github.com/hraness/slopcamera/releases/tag/v3.3.1)
+[Slopcamera v3.3.3 release](https://github.com/hraness/slopcamera/releases/tag/v3.3.3)
 from its canonical archive:
 
 ```sh
-bun add --global https://github.com/hraness/slopcamera/releases/download/v3.3.1/hraness-slopcamera-3.3.1.tgz
+bun add --global https://github.com/hraness/slopcamera/releases/download/v3.3.3/hraness-slopcamera-3.3.3.tgz
 slopcamera skill install --target agents
 ```
 
@@ -98,7 +109,7 @@ change an existing release.
 
 ## Make your first diagram
 
-After the source build above, this local task needs no model account. In a new directory,
+After installing Slopcamera, this local task needs no model account. In a new directory,
 create the included diagram, check it, and render it:
 
 ```sh
@@ -115,10 +126,11 @@ You now have `example-flow.tldr`, `example-flow.light.svg`,
 editable, and the `.tldr` file is editable tldraw interchange. Rendering again
 replaces those five derived files.
 
-Follow [Your first diagram](docs/tutorials/first-diagram.md) to change a label and
-see the result. For a moving 3D subject, use [Directed scenes](docs/spatial-scenes.md).
-For detailed native 3D from the current source build, follow
-[Your first native film](docs/tutorials/first-native-film.md).
+Follow [Your first diagram](https://slopcamera.com/docs/tutorials/first-diagram) to change a label and
+see the result. To animate type and SVG, [make your first animation](https://slopcamera.com/docs/tutorials/first-animation). For a moving 3D subject, use [Directed scenes](docs/spatial-scenes.md).
+For detailed native 3D, follow
+[Your first native film](docs/tutorials/first-native-film.md). To generate editable
+architecture from dimensions, [build a parametric design](https://slopcamera.com/docs/how-to/parametric-design).
 
 ## What Slopcamera does
 
@@ -134,12 +146,12 @@ GPU path. Spark admits saved splat worlds for local camera direction.
 > mount the product diagram on a screen in the scene, and show me contact frames
 > before rendering the video.
 
-The portable GLB profile has a defined geometry and material subset; the
-additive rigged profile admits bounded skins and morph targets. Saved splats
+The portable GLB profiles have defined geometry and material subsets. The
+rigged/morph profile in v3.3.1 admits bounded skins and morph targets. Saved splats
 capture appearance; they do not establish collision geometry or editable native
 meshes. See [Directed scenes](docs/spatial-scenes.md).
 
-Current source adds the cinematic planning loop: a `slopcamera.spatial-direction`
+Slopcamera v3.3.1 includes the cinematic planning loop: a `slopcamera.spatial-direction`
 document describes beats, actions, camera coverage, and look intents, compiles
 into proposed performance/camera/cinema/material-lighting/shot documents, and
 fans out into bounded galleries per axis. Declared effects bind into renders,
@@ -160,7 +172,7 @@ See [Direct a cinematic world](docs/how-to/direct-cinematic-worlds.md) and the
 
 ### Film native worlds and educational animation
 
-The current source build can direct Blender for detailed sets, materials,
+Slopcamera can direct Blender for detailed sets, materials,
 lighting, skinned characters, cloth and liquid caches; CadQuery for parametric
 solids and STEP; and Manim Community for mathematical animation. Seven editable
 starters include a product, character, shaded street, cloth, liquid, CAD bracket,
@@ -181,11 +193,19 @@ and [Make an educational video](docs/how-to/educational-video.md).
 ### Build diagrams and motion graphics
 
 Create editable diagrams with tldraw, SVG, and PNG outputs, or turn raster artwork
-into SVG locally with VTracer. Animate graphic layers with HTML, SVG, Motion,
+into SVG locally with VTracer. [Compare a color trace and duotone](https://slopcamera.com/docs/how-to/vectorize-images)
+from the same original raster. Animate graphic layers with HTML, SVG, Motion,
 p5, Two, Paper Shaders, or Three.js. Outputs can stand alone or join a video
 project. The optional vgpu example renders programmable WebGPU passes into
 retained raster frames for use on a world-space screen.
 
+[Render motion graphics from HTML](https://slopcamera.com/docs/how-to/render-motion-graphics)
+connects original examples across all seven profiles to editable source and exact
+render requests. Try the [editorial title](examples/showcase/html/editorial.json),
+[kinetic typography](examples/showcase/html/kinetic-title.json), or
+[Three.js island with original music](examples/showcase/html/island-pulse.json).
+These source examples live in the current repository; `html render` ships in v3.2.8.
+
 Use `slopcamera html catalog` to inspect the admitted local creative tools. The
 [creative toolkit reference](docs/html-overlay-creative-toolkit.md) distinguishes
 available profiles from upstream possibilities. vgpu does not enable shared GPU
@@ -194,11 +214,12 @@ textures or a Three WebGPU renderer inside the current WebGL2/Spark profile.
 ### Generate and direct media
 
 Discover image, video, speech, and transcription models through your own Vercel
-AI Gateway access. Generate images from text and references, add a voiceover,
+AI Gateway access. Set `AI_GATEWAY_API_KEY` in the local process environment;
+the Vercel CLI is optional. Generate images from text and references, add a voiceover,
 transcribe sound, or create video shots using the selected model's supported
 inputs. Availability and pricing come from the live catalog.
 
-The current source build also provides `direct` shot recipes: retain a film
+The released CLI also provides `direct` shot recipes: retain a film
 budget across attempts, review each take before accepting it, and use an accepted
 clip's last decoded frame as the next shot's reference. Changed predecessors
 invalidate affected continuations while earlier paid results remain retained.
@@ -220,8 +241,11 @@ from the same edit.
 > Edit my product demo: cut the pauses, zoom into each important click, keep the
 > speaker framed, add captions and `logo.svg`, and show a preview before export.
 
-Recording bundles can carry clicks, cursor movement, key activity, and
-focused-input information. Read [`PRIVACY.md`](PRIVACY.md) before editing
+The released and current CLIs edit existing recording bundles; they do not
+capture new recordings. Bundles can carry clicks, cursor movement, key activity,
+and focused-input information. Automatic screen-action zooms need this metadata.
+Standalone media imports need an existing project; `html render`, `studio assemble`,
+or `direct assemble` can create one. Read [`PRIVACY.md`](PRIVACY.md) before editing
 sensitive material.
 
 Start with [Edit a video](docs/how-to/edit-video.md) or inspect a reusable recipe:
@@ -236,7 +260,7 @@ slopcamera workflows show social-variants --json
 Keep the source that owns each creative decision. A native scene owns a rig or
 simulation; a portable scene owns supported geometry, cameras, and media surfaces;
 a diagram owns its objects and labels; a video project owns cuts and delivery.
-SLOPCAMERA connects these through explicit assets and rendered derivatives.
+Slopcamera connects these through explicit assets and rendered derivatives.
 
 1. **Prepare the sources.** Import footage and assets or author a scene, diagram,
    or native program. Inspect available tools before choosing an engine.
@@ -274,9 +298,9 @@ declarative graphs, approvals, and resuming work.
 - **Trusted code is not sandboxed.** Native Python and caller-authored Bun
   workflows run with the current user's access. Hashes and receipts identify
   observed inputs and outputs; they do not make arbitrary code hermetic.
-- **MCP is a subset.** Its fixed tools check and render diagrams, plan and audit
-  scenes, and run the bounded portable operations. It does not expose every
-  local CLI operation and never mutates project state.
+- **MCP is a subset.** Its 17 fixed tools check and render diagrams, plan and audit
+  scenes, and run six bounded portable operation codes in v3.3.1. It does not
+  expose every local CLI operation and never mutates project state.
 
 ## Design and trust
 
@@ -295,7 +319,7 @@ Native tools, providers, codecs, and GPU drivers can affect results, so retained
 source identity alone does not promise identical pixels on another machine.
 
 See [Architecture](docs/architecture.md), [`SECURITY.md`](SECURITY.md),
-[`PRIVACY.md`](PRIVACY.md), and [`NOTICE`](NOTICE) for the detailed boundaries.
+[`PRIVACY.md`](PRIVACY.md), and [`NOTICE.md`](NOTICE.md) for the detailed boundaries.
 
 ## Documentation
 
@@ -347,7 +371,7 @@ changing a package or runtime boundary. Report vulnerabilities through
 
 ## License
 
-[MIT](LICENSE), with third-party notices in [`NOTICE`](NOTICE).
+[MIT](LICENSE), with third-party notices in [`NOTICE.md`](NOTICE.md).
 
 ## Optional support
 

package/apps/desktop/application/directing-contract.ts

@@ -20,6 +20,10 @@ const MoneySchema = z.number().int().safe().positive();
 const TextSchema = z.string().min(1).max(100_000).refine(value => value.trim().length > 0 && !/[\u0000-\u0008\u000b\u000c\u000e-\u001f]/u.test(value), "Text must be nonempty and contain no disallowed control characters.");
 const ImageSchema = GatewayMediaSourceReferenceSchema.refine(source => source.mediaType.startsWith("image/") && source.bytes <= 50 * 1024 * 1024, "A directing frame must be an image of at most 50 MiB.");
 const VideoSourceSchema = GatewayMediaSourceReferenceSchema.refine(source => source.mediaType.startsWith("video/") && source.facts?.durationSeconds !== undefined, "Endpoint video sources require a measured duration.");
+const ReferenceSchema = GatewayMediaSourceReferenceSchema.refine(source =>
+  (source.mediaType.startsWith("image/") && source.bytes <= 50 * 1024 * 1024)
+  || (source.mediaType.startsWith("video/") && source.bytes <= 256 * 1024 * 1024), "A directing reference must be an image of at most 50 MiB or a video of at most 256 MiB.");
+const DIRECTING_REFERENCE_LIMITS = Object.freeze({ count: 8, totalBytes: 512 * 1024 * 1024 });
 
 function capture(input: unknown, name: string, maximumBytes: number): unknown {
   return createBoundedJsonValueSnapshot(input, maximumBytes, name, { maximumDepth: 32, maximumValues: 1_000_000 }).value;
@@ -39,8 +43,13 @@ export const DirectingShotSchema = z.strictObject({
     z.strictObject({ kind: z.literal("shot-end"), shotId: SlugSchema }),
   ]).optional(),
   lastFrame: ImageSchema.optional(),
+  references: z.array(ReferenceSchema).min(1).max(DIRECTING_REFERENCE_LIMITS.count).optional(),
 }).superRefine((shot, context) => {
   if (shot.lastFrame !== undefined && shot.firstFrame === undefined) context.addIssue({ code: "custom", message: "A last frame requires an authored first-frame reference." });
+  const references = shot.references ?? [];
+  if (references.length > 0 && (shot.firstFrame !== undefined || shot.lastFrame !== undefined)) context.addIssue({ code: "custom", message: "Directing references are mutually exclusive with frame conditioning." });
+  if (references.reduce((total, reference) => total + reference.bytes, 0) > DIRECTING_REFERENCE_LIMITS.totalBytes) context.addIssue({ code: "custom", message: "Directing references exceed the 512 MiB aggregate bound." });
+  if (new Set(references.map(reference => `${reference.path}${reference.sha256}`)).size !== references.length) context.addIssue({ code: "custom", message: "Directing references must be distinct retained media." });
 });
 
 export const DirectingRecipeSchema = z.preprocess(input => capture(input, "directing recipe", 8 * 1024 * 1024), z.strictObject({
@@ -101,7 +110,7 @@ const ReviewSchema = z.strictObject({ decision: z.enum(["accepted", "rejected"])
 const FailureSchema = z.string().regex(/^[a-z][a-z0-9-]{0,63}$/u);
 const VideoRequestSchema = z.strictObject({ operation: z.literal("video"), request: GatewayVideoOperationInputSchema }).superRefine((input, context) => {
   const request = input.request;
-  if (request.providerOptions !== undefined || request.references !== undefined || request.generateAudio !== undefined || (request.n ?? 1) !== 1 || (request.maxVideosPerCall ?? 1) !== 1) context.addIssue({ code: "custom", message: "Directing uses exactly one video with authored frame conditioning and no provider options." });
+  if (request.providerOptions !== undefined || request.generateAudio !== undefined || (request.n ?? 1) !== 1 || (request.maxVideosPerCall ?? 1) !== 1) context.addIssue({ code: "custom", message: "Directing uses exactly one video with authored conditioning and no provider options." });
 });
 const AttemptBase = {
   id: AttemptIdSchema,
@@ -149,7 +158,7 @@ export function directingRequestId(input: {
   return `gateway_${canonicalJsonSha256({ domain: "slopcamera.directing-request/v1", id, attemptId, recipeSha256, shotSha256, request })}`;
 }
 
-/** Verify the submitted request against the authored recipe and retained frames. */
+/** Verify the submitted request against the authored recipe and retained conditioning. */
 export function directingRequestMatchesShot(attempt: DirectingAttempt, shot: DirectingShot, firstFrame: GatewayMediaSourceReference | undefined): boolean {
   const request = attempt.request.request;
   for (const key of ["prompt", "model", "durationSeconds", "resolution", "aspectRatio", "fps", "seed"] as const) if (request[key] !== shot[key]) return false;
@@ -157,7 +166,10 @@ export function directingRequestMatchesShot(attempt: DirectingAttempt, shot: Dir
   const actualFirst = request.promptImage ?? frames.find(frame => frame.frameType === "first_frame")?.source;
   const actualLast = frames.find(frame => frame.frameType === "last_frame")?.source;
   const equal = (left: GatewayMediaSourceReference | undefined, right: GatewayMediaSourceReference | undefined) => left === undefined || right === undefined ? left === right : sameDirectingSource(left, right);
-  return equal(firstFrame, actualFirst) && equal(shot.lastFrame, actualLast);
+  const actualReferences = request.references ?? [];
+  return equal(firstFrame, actualFirst) && equal(shot.lastFrame, actualLast)
+    && actualReferences.length === (shot.references ?? []).length
+    && (shot.references ?? []).every((reference, index) => sameDirectingSource(reference, actualReferences[index]!));
 }
 
 export const DirectingStateSchema = z.preprocess(input => capture(input, "directing state", DIRECTING_LIMITS.stateBytes), z.strictObject({

package/apps/desktop/application/directing-plan.ts

@@ -28,6 +28,7 @@ export interface ResolvedDirectingShot {
   readonly dependencies: DirectingDependency[];
   readonly firstFrame?: GatewayMediaSourceReference;
   readonly lastFrame?: GatewayMediaSourceReference;
+  readonly references?: readonly GatewayMediaSourceReference[];
 }
 
 function resolveShot(shot: DirectingShot, accepted: ReadonlyMap<string, CompletedDirectingAttempt>): ResolvedDirectingShot | undefined {
@@ -43,7 +44,7 @@ function resolveShot(shot: DirectingShot, accepted: ReadonlyMap<string, Complete
     boundDependencies.push({ ...dependency, shotSha256: predecessor.shotSha256 });
     firstFrame = predecessor.endpoint.image;
   }
-  return { shot, shotSha256: directingBoundShotSha256(shot, boundDependencies), dependencies, ...(firstFrame === undefined ? {} : { firstFrame }), ...(shot.lastFrame === undefined ? {} : { lastFrame: shot.lastFrame }) };
+  return { shot, shotSha256: directingBoundShotSha256(shot, boundDependencies), dependencies, ...(firstFrame === undefined ? {} : { firstFrame }), ...(shot.lastFrame === undefined ? {} : { lastFrame: shot.lastFrame }), ...(shot.references === undefined ? {} : { references: shot.references }) };
 }
 
 function acceptedSelections(state: DirectingState): Map<string, CompletedDirectingAttempt> {

package/apps/desktop/application/operation.ts

@@ -4,7 +4,7 @@ import type { z } from "zod";
 
 import type { ApplicationContext } from "./context";
 
-export const SLOPCAMERA_APPLICATION_TOOL_VERSION = "slopcamera-3.3.2" as const;
+export const SLOPCAMERA_APPLICATION_TOOL_VERSION = "slopcamera-3.3.4" as const;
 
 export const OPERATION_KINDS = [
   "scene.audit",

package/apps/desktop/cli/args.ts

@@ -3231,7 +3231,7 @@ function parseDirectingArgs(argv: readonly string[]): DirectingCommand {
   const required = (name: string): string => optionString(parsed, name) ?? fail(`direct ${action} requires ${name}.`);
   const common = { kind: "directing" as const, json: optionFlag(parsed, "--json") };
   if (action === "anchor") {
-    exactPositionals(parsed, 0, "slopcamera direct anchor --input <image>");
+    exactPositionals(parsed, 0, "slopcamera direct anchor --input <media>");
     return { ...common, action, input: required("--input") };
   }
   const [value] = exactPositionals(parsed, 1, `slopcamera direct ${action} <${["init", "plan", "start"].includes(action) ? "recipe.json" : "direct-id"}>`);

package/apps/desktop/cli/cinema-renderer.ts

@@ -142,11 +142,11 @@ function sliceChain(slice: CinemaMediaSlice, inputIndex: ReadonlyMap<string, num
   const inputDurationUs = slice.fileRange.endUs - slice.fileRange.startUs;
   const outputDurationUs = slice.outputRange.endUs - slice.outputRange.startUs;
   return [
-    `[${inputSpecifier(input, slice.streamIndex)}]`,
-    `trim=start=${seconds(slice.fileRange.startUs)}:end=${seconds(slice.fileRange.endUs)}`,
+    `[${inputSpecifier(input, slice.streamIndex)}]trim=start=${seconds(slice.fileRange.startUs)}:end=${seconds(slice.fileRange.endUs)}`,
     "settb=AVTB",
-    // Source clocks may be coarse; cinema offsets stay in integer microseconds.
-    `setpts=(PTS-STARTPTS)*${decimal(outputDurationUs / inputDurationUs)}+${seconds(slice.outputRange.startUs)}/TB`,
+    // Concat requires each member to start at zero. Position the complete
+    // window once after joining, so internal slice boundaries stay contiguous.
+    `setpts=(PTS-STARTPTS)*${decimal(outputDurationUs / inputDurationUs)}`,
     "format=rgba",
   ].join(",");
 }
@@ -173,7 +173,7 @@ function windowChain(
     : `scale=w=${outputWidth}:h=${outputHeight}:force_original_aspect_ratio=decrease,pad=${outputWidth}:${outputHeight}:(ow-iw)/2:(oh-ih)/2:color=black@0`;
   const joined = label(context, prefix);
   parts.push(
-    `${members.map(member => `[${member}]`).join("")}concat=n=${members.length}:v=1:a=0,${geometry},format=rgba${gradeFilter === undefined ? "" : `,${gradeFilter}`}[${joined}]`,
+    `${members.map(member => `[${member}]`).join("")}concat=n=${members.length}:v=1:a=0,setpts=PTS+${seconds(slices[0]!.outputRange.startUs)}/TB,${geometry},format=rgba${gradeFilter === undefined ? "" : `,${gradeFilter}`}[${joined}]`,
   );
   context.filters.push(...parts);
   return joined;
@@ -255,8 +255,11 @@ function transitionWindowChain(
       stream = windowChain(context, slices, shot.fit, grade, outputWidth, outputHeight, prefix);
     }
     const zeroed = label(context, `${prefix}_zero`);
+    // A dip/flash reveals only the tail of the incoming handle. Its hidden
+    // prefix still consumes source time, so the tail must meet the shot body
+    // without skipping the last part of the handle at the transition boundary.
     context.filters.push(
-      `[${stream}]setpts=PTS-STARTPTS,fps=${context.rate}${limitUs === undefined ? "" : `,trim=end=${seconds(limitUs)},setpts=PTS-STARTPTS`}[${zeroed}]`,
+      `[${stream}]setpts=PTS-STARTPTS${limitUs === undefined ? "" : `,trim=start=${seconds(durationUs - limitUs)}:end=${seconds(durationUs)},setpts=PTS-STARTPTS`},fps=${context.rate}[${zeroed}]`,
     );
     return zeroed;
   };

package/apps/desktop/cli/commands.ts

@@ -355,7 +355,7 @@ import {
   workflowRunStore,
 } from "./workflow-runs";
 
-export const SLOPCAMERA_VERSION = "3.3.2";
+export const SLOPCAMERA_VERSION = "3.3.4";
 
 // Legacy direct renders predate per-target output contracts. Keep them
 // bounded generously enough for long-form production while preventing one
@@ -3897,6 +3897,11 @@ async function inspectStagedGatewayMediaFile(
   ffprobe: string,
   signal?: AbortSignal,
 ): Promise<LoadedGatewayPhysicalFile> {
+  // On-disk JPEG files probe through ffmpeg's image2 sequence demuxer, which the
+  // whitelist deliberately omits; pin the single-image pipe demuxer instead.
+  const forcedInputFormat = file.mediaType === "image/jpeg"
+    ? ["-f", "jpeg_pipe"]
+    : [];
   const probe = await context.runner.run([
     ffprobe,
     "-hide_banner",
@@ -3906,6 +3911,7 @@ async function inspectStagedGatewayMediaFile(
     "-show_entries",
     "format=duration:stream=codec_type,width,height,duration,channels,sample_rate,avg_frame_rate,r_frame_rate",
     "-of", "json",
+    ...forcedInputFormat,
     inspectionPath,
   ], {
     ...(signal === undefined ? {} : { abortSignal: signal }),
@@ -4078,6 +4084,7 @@ async function inspectStagedGatewayMediaFile(
     "-xerror",
     ...SELF_CONTAINED_MEDIA_INPUT_ARGUMENTS,
     "-threads", "1",
+    ...forcedInputFormat,
     "-i", inspectionPath,
     "-map", "0:V?",
     "-map", "0:a?",

package/apps/desktop/cli/directing-blob.ts

@@ -18,7 +18,7 @@ import { resolveVerifiedProjectMedia } from "./project-media-integrity";
 // put.ts, del.ts, and signed-token.ts in https://github.com/vercel/storage.
 // The SDK's network retry/environment overrides are not part of this adapter.
 const API = "https://vercel.com/api/blob";
-const LIMITS = { imageBytes: 30 * 1024 * 1024, entries: 2, receiptBytes: 64 * 1024, responseBytes: 32 * 1024, requestMs: 60_000, cleanupMs: 120_000, accessMs: 15 * 60_000 } as const;
+const LIMITS = { imageBytes: 30 * 1024 * 1024, videoBytes: 256 * 1024 * 1024, entries: 8, receiptBytes: 64 * 1024, responseBytes: 32 * 1024, requestMs: 60_000, uploadMs: 15 * 60_000, cleanupMs: 120_000, accessMs: 15 * 60_000 } as const;
 const RECEIPT = "receipt.json";
 const NOTICE = "Private reference hosting uses Vercel Blob operations, storage, and transfer billed separately from the Gateway generation budget. Expiring model access does not delete the stored object.";
 const IdSchema = z.string().regex(/^direct_[a-z][a-z0-9_-]{0,63}$/u);
@@ -26,10 +26,15 @@ const TakeSchema = z.string().regex(/^take_[a-z][a-z0-9_-]{0,63}$/u);
 const StoreSchema = z.string().regex(/^[A-Za-z0-9]{1,64}$/u);
 const TimeSchema = z.number().int().safe().positive();
 const EtagSchema = z.string().min(1).max(256).regex(/^[\x21-\x7e]+$/u);
-const SourceSchema = GatewayMediaSourceReferenceSchema.refine(source => ["image/png", "image/jpeg", "image/webp"].includes(source.mediaType) && source.bytes <= LIMITS.imageBytes, "Hosted directing references must be PNG, JPEG, or WebP images at most 30 MiB.");
+const SourceSchema = GatewayMediaSourceReferenceSchema.refine(source =>
+  (["image/png", "image/jpeg", "image/webp"].includes(source.mediaType) && source.bytes <= LIMITS.imageBytes)
+  || (["video/mp4", "video/quicktime"].includes(source.mediaType) && source.bytes <= LIMITS.videoBytes), "Hosted directing references must be PNG, JPEG, or WebP images at most 30 MiB, or MP4/QuickTime video at most 256 MiB.");
+function sourceExtension(mediaType: string): string {
+  return mediaType === "image/jpeg" ? "jpg" : mediaType === "video/quicktime" ? "mov" : mediaType.slice(6);
+}
 const EntrySchema = z.strictObject({
   source: SourceSchema,
-  pathname: z.string().regex(/^slopcamera\/directing\/[a-f0-9]{32}\/[01]-[a-f0-9]{64}\.(?:png|jpg|webp)$/u),
+  pathname: z.string().regex(/^slopcamera\/directing\/[a-f0-9]{32}\/[0-7]-[a-f0-9]{64}\.(?:png|jpg|webp|mp4|mov)$/u),
   putStartedAt: TimeSchema,
   putCompletedAt: TimeSchema.optional(),
   etag: EtagSchema.optional(),
@@ -52,7 +57,7 @@ export const DirectingBlobReceiptSchema = z.strictObject({
   entries: z.array(EntrySchema).max(LIMITS.entries),
 }).superRefine((receipt, context) => {
   for (const [index, entry] of receipt.entries.entries()) {
-    const extension = entry.source.mediaType === "image/jpeg" ? "jpg" : entry.source.mediaType.slice(6);
+    const extension = sourceExtension(entry.source.mediaType);
     if (entry.pathname !== `slopcamera/directing/${receipt.namespace}/${index}-${entry.source.sha256}.${extension}`) context.addIssue({ code: "custom", message: "A hosted object must bind this session's exact private namespace and source bytes." });
   }
 });
@@ -161,12 +166,12 @@ export async function createDirectingBlobSession(options: DirectingBlobOptions):
       const save = async () => { await fs.writeTextAtomicGuarded!(RECEIPT, `${canonicalJson(DirectingBlobReceiptSchema.parse(receipt))}\n`, lease.assertOwned); };
       return await execute(receipt, save, lease.assertOwned);
     });
-  const api = async (receipt: DirectingBlobReceipt, pathname: string, method: "GET" | "PUT" | "POST", body: string | Uint8Array | undefined, headers: Record<string, string>, signal: AbortSignal, assertOwned: () => Promise<void>): Promise<{ status: number; data: Uint8Array }> => {
+  const api = async (receipt: DirectingBlobReceipt, pathname: string, method: "GET" | "PUT" | "POST", body: string | Uint8Array | undefined, headers: Record<string, string>, signal: AbortSignal, assertOwned: () => Promise<void>, timeoutMs: number = LIMITS.requestMs): Promise<{ status: number; data: Uint8Array }> => {
     const auth = credentials(options.environment);
     if (auth.storeId !== receipt.storeId) failure("Blob credentials don't match the retained store.", "authorization-required");
     await assertOwned();
     signal.throwIfAborted();
-    const bounded = AbortSignal.any([signal, AbortSignal.timeout(LIMITS.requestMs)]);
+    const bounded = AbortSignal.any([signal, AbortSignal.timeout(timeoutMs)]);
     const response = await request(`${API}${pathname}`, { method, redirect: "error", signal: bounded,
       headers: { authorization: `Bearer ${auth.token}`, "x-vercel-blob-store-id": auth.storeId, "x-api-version": "12", "x-api-blob-request-id": `${auth.storeId}:${randomUUID()}`, "x-api-blob-request-attempt": "0", ...headers },
       ...(body === undefined ? {} : { body: typeof body === "string" ? body : Buffer.from(body) }),
@@ -189,7 +194,7 @@ export async function createDirectingBlobSession(options: DirectingBlobOptions):
   };
   const verifyGet = async (receipt: DirectingBlobReceipt, entry: Entry, url: string, signal: AbortSignal): Promise<{ found: false } | { found: true; etag: string }> => {
     exactPrivateUrl(url, receipt.storeId, entry.pathname, true);
-    const bounded = AbortSignal.any([signal, AbortSignal.timeout(LIMITS.requestMs)]);
+    const bounded = AbortSignal.any([signal, AbortSignal.timeout(LIMITS.uploadMs)]);
     const response = await request(url, { method: "GET", redirect: "error", signal: bounded, headers: { "cache-control": "no-cache" } });
     if (response.status === 404) { await response.body?.cancel(); return { found: false }; }
     if (response.status !== 200) { await response.body?.cancel(); failure("Private hosted reference could not be verified."); }
@@ -212,9 +217,9 @@ export async function createDirectingBlobSession(options: DirectingBlobOptions):
     inspect: read,
     resolveSourceUrl: async (input, data, signal) => {
       const source = SourceSchema.parse(input);
-      if (!(data instanceof Uint8Array) || data.length !== source.bytes) failure("Reference hosting requires the exact retained image bytes.", "invalid-data");
+      if (!(data instanceof Uint8Array) || data.length !== source.bytes) failure("Reference hosting requires the exact retained reference bytes.", "invalid-data");
       const uploadBytes = new Uint8Array(data);
-      if (createHash("sha256").update(uploadBytes).digest("hex") !== source.sha256 || !gatewayMediaBytesMatchType(uploadBytes, source.mediaType)) failure("Reference hosting requires the exact retained image bytes.", "invalid-data");
+      if (createHash("sha256").update(uploadBytes).digest("hex") !== source.sha256 || !gatewayMediaBytesMatchType(uploadBytes, source.mediaType)) failure("Reference hosting requires the exact retained reference bytes.", "invalid-data");
       await resolveVerifiedProjectMedia({ repositoryRoot: application.paths.repositoryRoot, path: source.path, expected: source, label: "Directing hosted reference" });
       const activeSignal = AbortSignal.any([options.signal, signal]);
       return await mutate(async (receipt, save, assertOwned) => {
@@ -222,14 +227,14 @@ export async function createDirectingBlobSession(options: DirectingBlobOptions):
         if (receipt.closedAt !== undefined) failure("This reference-hosting session is closed; use a new take for another generation.", "conflict");
         let entry = receipt.entries.find(value => sourceIdentity(value.source) === sourceIdentity(source));
         if (entry === undefined) {
-          if (receipt.entries.length >= LIMITS.entries) failure("One directing take may host at most two exact image references.", "invalid-data");
-          const extension = source.mediaType === "image/jpeg" ? "jpg" : source.mediaType.slice(6);
+          if (receipt.entries.length >= LIMITS.entries) failure("One directing take may host at most eight exact media references.", "invalid-data");
+          const extension = sourceExtension(source.mediaType);
           entry = { source, pathname: `slopcamera/directing/${receipt.namespace}/${receipt.entries.length}-${source.sha256}.${extension}`, putStartedAt: now(), cleanup: "pending", failures: [] };
           receipt.entries.push(entry);
           await save(); // Durable exact upload intent precedes the only PUT.
           try {
             const uploaded = await api(receipt, `/?${new URLSearchParams({ pathname: entry.pathname })}`, "PUT", uploadBytes,
-              { "content-type": source.mediaType, "x-content-type": source.mediaType, "x-vercel-blob-access": "private", "x-add-random-suffix": "0", "x-allow-overwrite": "0", "x-cache-control-max-age": "60" }, activeSignal, assertOwned);
+              { "content-type": source.mediaType, "x-content-type": source.mediaType, "x-vercel-blob-access": "private", "x-add-random-suffix": "0", "x-allow-overwrite": "0", "x-cache-control-max-age": "60" }, activeSignal, assertOwned, LIMITS.uploadMs);
             if (uploaded.status < 200 || uploaded.status > 299) failure("Private reference upload did not complete.");
             const result = z.object({ url: z.string().max(2048), pathname: z.string().max(512), contentType: z.string().max(128), etag: EtagSchema }).parse(JSON.parse(new TextDecoder().decode(uploaded.data)) as unknown);
             exactPrivateUrl(result.url, receipt.storeId, entry.pathname, false);

package/apps/desktop/cli/directing-media.ts

@@ -20,7 +20,7 @@ import { withMutationLock } from "./mutation-lock";
 import { ensurePhysicalPrivateDirectoryWithin } from "./paths";
 import { resolveVerifiedProjectMedia } from "./project-media-integrity";
 
-const LIMITS = { videoBytes: 512 * 1024 * 1024, imageBytes: 30 * 1024 * 1024, probeBytes: 8 * 1024 * 1024, dimension: 4096, frames: 4000, durationUs: 60_000_000 } as const;
+const LIMITS = { videoBytes: 512 * 1024 * 1024, imageBytes: 30 * 1024 * 1024, referenceBytes: 256 * 1024 * 1024, probeBytes: 8 * 1024 * 1024, dimension: 4096, frames: 4000, durationUs: 60_000_000 } as const;
 const INPUT_ARGUMENTS = ["-protocol_whitelist", "file", "-format_whitelist", "mov", "-enable_drefs", "0", "-use_absolute_path", "0", "-max_pixels", String(LIMITS.dimension ** 2), "-threads", "2"] as const;
 const artifactSchema = z.strictObject({ path: RepositoryRelativePathSchema, bytes: z.number().int().safe().positive(), sha256: Sha256Schema });
 const endpointSchema = DirectingEndpointSchema.refine(endpoint => endpoint.frameIndex < LIMITS.frames, "Directing endpoint exceeds the 4000-frame media bound.");
@@ -226,15 +226,25 @@ async function retainVideo(application: ApplicationContext, input: GatewayMediaS
   return GatewayMediaSourceReferenceSchema.parse({ ...artifact, mediaType: source.mediaType });
 }
 
-/** Imports exactly the caller-named still; never uploads it or reads neighboring files. */
+/** Imports exactly the caller-named still or clip; never uploads it or reads neighboring files. */
 export async function importDirectingAnchor(application: ApplicationContext, path: string, signal: AbortSignal): Promise<GatewayMediaSourceReference> {
   await active(application, signal);
   const resolved = resolve(application.paths.repositoryRoot, path);
   if (await realpath(dirname(resolved)) !== dirname(resolved)) throw new CliError("unsafe-path", "Directing anchors require physical parent directories.");
-  const bytes = await fileBytes(resolved, LIMITS.imageBytes, signal);
+  const bytes = await fileBytes(resolved, LIMITS.referenceBytes, signal);
+  const brand = bytes.length >= 12 && Buffer.from(bytes.subarray(4, 8)).toString("ascii") === "ftyp" ? Buffer.from(bytes.subarray(8, 12)).toString("ascii") : undefined;
+  if (brand === "qt  " || brand !== undefined && /^(?:isom|iso2|iso5|iso6|mp41|mp42|avc1|dash|mmp4|M4V )$/u.test(brand)) {
+    const mediaType = brand === "qt  " ? "video/quicktime" : "video/mp4";
+    const artifact = await publishBytes(application, bytes, mediaType === "video/quicktime" ? "mov" : "mp4", signal);
+    const provisional = GatewayMediaSourceReferenceSchema.parse({ ...artifact, mediaType });
+    const ffprobe = await capability(application, "ffprobe");
+    const probed = await probeVideo(application, provisional, ffprobe.command, signal);
+    return GatewayMediaSourceReferenceSchema.parse({ ...provisional, facts: { durationSeconds: probed.media.durationUs / 1_000_000, width: probed.stream.width, height: probed.stream.height } });
+  }
+  if (bytes.length > LIMITS.imageBytes) fail("Directing anchors require a single unrotated PNG, JPEG, or WebP image no larger than 30 MiB.");
   const metadata = await sharp(bytes, { limitInputPixels: LIMITS.dimension ** 2, failOn: "error", animated: true }).metadata();
   const extension = metadata.format === "jpeg" ? "jpg" : metadata.format;
-  if (!extension || !["png", "jpg", "webp"].includes(extension) || metadata.pages !== undefined && metadata.pages !== 1 || !metadata.width || !metadata.height || metadata.width > LIMITS.dimension || metadata.height > LIMITS.dimension || metadata.orientation !== undefined && metadata.orientation !== 1) fail("Directing anchors require a single unrotated PNG, JPEG, or WebP image no larger than 4096 pixels per side.");
+  if (!extension || !["png", "jpg", "webp"].includes(extension) || metadata.pages !== undefined && metadata.pages !== 1 || !metadata.width || !metadata.height || metadata.width > LIMITS.dimension || metadata.height > LIMITS.dimension || metadata.orientation !== undefined && metadata.orientation !== 1) fail("Directing anchors require a single unrotated PNG, JPEG, or WebP image no larger than 4096 pixels per side, or self-contained MP4/QuickTime video.");
   await sharp(bytes, { limitInputPixels: LIMITS.dimension ** 2, failOn: "error" }).raw().toBuffer();
   const artifact = await publishBytes(application, bytes, extension, signal);
   return GatewayMediaSourceReferenceSchema.parse({ ...artifact, mediaType: metadata.format === "jpeg" ? "image/jpeg" : `image/${metadata.format}`, facts: { width: metadata.width, height: metadata.height } });

package/apps/desktop/cli/directing-quote.ts

@@ -1,5 +1,5 @@
 import type { DirectingQuote, DirectingShot } from "../application/directing-contract";
-import type { GatewayMediaCatalogView, GatewayJsonValue } from "./gateway-media-catalog";
+import { providerVideoResolution, type GatewayMediaCatalogView, type GatewayJsonValue } from "./gateway-media-catalog";
 import { CliError } from "./errors";
 
 /** Integer microdollars avoid floating point drift across small paid attempts. */
@@ -42,9 +42,13 @@ export function quoteDirectingShot(
   const model = catalog.snapshot.models.find(value => value.id === shot.model);
   if (model?.kind !== "video" || model.executionMode !== "video-model") unsupported("the selected model is not a batch video model.");
   const capability = model.capabilities;
-  const operation = shot.lastFrame !== undefined ? "first-last-frame" : shot.firstFrame !== undefined ? "image-to-video" : "text-to-video";
+  const hasVideoReference = (shot.references ?? []).some(reference => reference.mediaType.startsWith("video/"));
+  const operations = (shot.references?.length ?? 0) > 0
+    ? ["reference-to-video", ...(hasVideoReference ? ["video-editing", "motion-control", "extend-video"] : [])]
+    : [shot.lastFrame !== undefined ? "first-last-frame" : shot.firstFrame !== undefined ? "image-to-video" : "text-to-video"];
+  const supportedOperations = capability?.["supported_operations"];
+  if (!Array.isArray(supportedOperations) || !operations.some(operation => supportedOperations.includes(operation))) unsupported(`the catalog does not confirm any of the shot's operations (${operations.join(", ")}).`);
   for (const [key, value] of [
-    ["supported_operations", operation],
     ["supported_resolutions", shot.resolution],
     ["supported_aspect_ratios", shot.aspectRatio],
     ["supported_durations_seconds", shot.durationSeconds],
@@ -53,6 +57,9 @@ export function quoteDirectingShot(
     const supported = capability?.[key];
     if (!Array.isArray(supported) || !supported.includes(value)) unsupported(`the catalog does not confirm ${key}=${value}.`);
   }
+  if (providerVideoResolution(model.id, shot.resolution, shot.aspectRatio) === null) {
+    unsupported(`the provider cannot express resolution ${shot.resolution} at aspect ${shot.aspectRatio}.`);
+  }
   if (model.pricing.varies_by_provider === true) unsupported("provider-dependent pricing cannot establish a bounded reservation.");
   const rows = model.pricing.video_duration_pricing;
   if (!Array.isArray(rows) || rows.length === 0 || rows.length > 128) unsupported("duration pricing is unavailable.");
@@ -76,14 +83,25 @@ export function quoteDirectingShot(
 
 /** Determine transport from the same catalog used to price the shot. */
 export function directingInputTransport(catalog: GatewayMediaCatalogView, shot: DirectingShot): "none" | "inline" | "url" {
-  if (shot.firstFrame === undefined && shot.lastFrame === undefined) return "none";
+  const kinds = new Set((shot.references ?? []).map(reference => reference.mediaType.split("/")[0]!));
+  if (shot.firstFrame !== undefined || shot.lastFrame !== undefined) kinds.add("image");
+  if (kinds.size === 0) return "none";
   const model = catalog.snapshot.models.find(value => value.id === shot.model);
   const inputLimits = record(model?.capabilities?.input_limits);
-  const images = record(inputLimits?.image);
-  const sources = images?.supported_sources;
-  if (sources === undefined || sources === null) return "inline";
-  if (!Array.isArray(sources) || sources.some(value => typeof value !== "string")) unsupported("invalid image transport capabilities.");
-  if (sources.includes("base64") || sources.includes("buffer")) return "inline";
-  if (sources.includes("url")) return "url";
-  return unsupported("the model has no supported image transport.");
+  let needsUrl = false;
+  for (const kind of kinds) {
+    const sources = record(inputLimits?.[kind])?.supported_sources;
+    if (sources === undefined || sources === null) continue;
+    if (!Array.isArray(sources) || sources.some(value => typeof value !== "string")) unsupported("invalid input transport capabilities.");
+    if (sources.includes("base64") || sources.includes("buffer")) continue;
+    if (sources.includes("url")) { needsUrl = true; continue; }
+    unsupported(`the model has no supported ${kind} input transport.`);
+  }
+  if (needsUrl) {
+    for (const kind of kinds) {
+      const sources = record(inputLimits?.[kind])?.supported_sources;
+      if (Array.isArray(sources) && !sources.includes("url")) unsupported(`the model cannot receive hosted ${kind} references.`);
+    }
+  }
+  return needsUrl ? "url" : "inline";
 }

package/apps/desktop/cli/directing-service.ts

@@ -56,6 +56,8 @@ function snapshot(state: DirectingState) {
     budget: { maximumMicroUsd: state.budgetMicroUsd, reservedMicroUsd, remainingMicroUsd: state.budgetMicroUsd - reservedMicroUsd, notice: BUDGET_NOTICE },
     shots: currentRecipe(state).shots.map(shot => ({
       id: shot.id, prompt: shot.prompt, model: shot.model,
+      conditioning: shot.references === undefined ? shot.firstFrame === undefined ? "none" : "frames" : "references",
+      referenceCount: shot.references?.length ?? 0,
       selectedAttempt: Object.hasOwn(state.selections, shot.id) ? state.selections[shot.id] : null,
       selectionStatus: Object.hasOwn(active, shot.id) ? "accepted" : Object.hasOwn(state.selections, shot.id) ? "stale" : "unselected",
     })),
@@ -189,11 +191,12 @@ async function executeDirectingCommandBody(
       const dependencies = resolveDirectingShot(state, command.shot).dependencies;
       for (const dependency of dependencies) await finishEndpoint(dependency.attemptId);
       const resolved = resolveDirectingShot(state, command.shot), shot = resolved.shot;
-      if ((resolved.firstFrame !== undefined || resolved.lastFrame !== undefined) && !command.allowCloudUpload) throw new CliError("authorization-required", "This take uploads its exact retained image references. Supply --allow-cloud-upload for this request.");
+      const hasLocalInputs = resolved.firstFrame !== undefined || resolved.lastFrame !== undefined || (shot.references?.length ?? 0) > 0;
+      if (hasLocalInputs && !command.allowCloudUpload) throw new CliError("authorization-required", "This take uploads its exact retained media references. Supply --allow-cloud-upload for this request.");
       const catalog = await adapters.catalog.get({ forceRefresh: true, freshness: "require-fresh", signal });
       const quote = quoteDirectingShot(catalog, shot, application.clock.now());
       const needsHosting = directingInputTransport(catalog, shot) === "url";
-      if (needsHosting && (!command.allowReferenceHosting || adapters.createReferenceHosting === undefined)) throw new CliError("authorization-required", "This model requires image URLs. Configure a private Vercel Blob store and supply --allow-reference-hosting for expiring access to these exact reference frames; --allow-cloud-upload is also required.");
+      if (needsHosting && (!command.allowReferenceHosting || adapters.createReferenceHosting === undefined)) throw new CliError("authorization-required", "This model requires media URLs. Configure a private Vercel Blob store and supply --allow-reference-hosting for expiring access to these exact retained references; --allow-cloud-upload is also required.");
       if (quote.costMicroUsd > state.budgetMicroUsd - totalDirectingReservedMicroUsd(state)) throw new CliError("authorization-required", "This take exceeds the remaining directing budget. No paid request was dispatched.");
       const request = await prepareGatewayOperation(application, {
         signal,
@@ -207,6 +210,7 @@ async function executeDirectingCommandBody(
             { frameType: "first_frame" as const, source: resolved.firstFrame },
             ...(resolved.lastFrame === undefined ? [] : [{ frameType: "last_frame" as const, source: resolved.lastFrame }]),
           ] }),
+          ...(resolved.references === undefined ? {} : { references: resolved.references.map(reference => ({ ...reference })) }),
         } },
       });
       const requestId = directingRequestId({ id: state.id, attemptId: command.attempt, recipeSha256: state.activeRecipeSha256, shotSha256: resolved.shotSha256, request });