stillmade-sdk 0.1.42 → 0.1.43

package/docs/build/data-types.md

@@ -0,0 +1,83 @@
+# Data types: files, documents, tables, links, dates and colors
+
+Ports can carry general files and structured data, not only media and text.
+Each type has a checked value shape, a generated control and a result layout,
+so a Block that declares them needs no custom view.
+
+| Type | Value | Generated control | Default result layout |
+| --- | --- | --- | --- |
+| `file` | `{kind: "file", assetId, versionId, mimeType, url?, name?, bytes?}`; `mimeType` is `application/pdf`, `text/plain`, `text/csv`, `text/markdown`, `application/json` or `application/zip` | `file` (upload) | `download` |
+| `document` | `{schemaVersion: 1, format: "markdown" \| "plain", text, title?}`, up to 1,000,000 characters | `richtext` | `document` |
+| `table` | `{columns: [name], rows: [{name: text \| number \| boolean \| null}]}`, up to 500 columns and 100,000 rows | `grid` | `table` (with CSV download) |
+| `url` | An absolute `http` or `https` address without credentials | `url` | a link |
+| `date` | ISO 8601: `2026-10-09` or `2026-10-09T14:30` (seconds and an offset optional) | `date` (`time: true` for date and time) | a formatted date |
+| `color` | `#rrggbb` or `#rrggbbaa` | `color` | a swatch |
+
+Lists of these types (`file[]`, `url[]`, `date[]`, `color[]`) work like other
+lists; the `list` control edits `text[]`, `number[]`, `integer[]`, `url[]`,
+`date[]` and `color[]` item by item.
+
+Output presentations in `manifest.ui` choose a layout for an output:
+`{control: "table", port}`, `{control: "document", port}` (a `document` or
+Markdown `text`), `{control: "chart", port, chart: "bar" | "line", x, y}` (a
+`table` with optional column names, or a `number[]`), `{control: "code", port,
+language}` (`text`, `json` or `object`), `{control: "player", port}` (`audio` or
+`video`) and `{control: "download", port}` (`file`, `file[]` or `file-bundle`).
+Each control is checked against its port's type when the manifest is
+validated. Layouts show Block data as text: links in a document are displayed,
+not followed, and downloads go through the host's export policy.
+
+### Files
+
+People upload files to a `file` input from the Block's controls. The host
+checks the extension, the declared type, the size (50 MB) and the bytes (a PDF
+or ZIP signature, valid UTF-8 text, parseable JSON) before the Block receives a
+saved reference. Module Blocks read the bytes with
+`stillmade.files.read(input.script)` and save new files with
+`stillmade.files.write(bytes, {mediaType: "text/csv", name: "shots.csv"})`,
+which returns a `file` value. Files a person uploaded to a Block, or a Block
+saved, are indexed in project context as `files` (`file[]`, permission
+`context.files.read`), apart from media so media lists stay media.
+
+### Helpers
+
+`@stillmade/block-sdk/data` (also exported from the main entry) has the checks
+and builders: `isFileValue`, `isDocumentValue`, `isTableValue`, `isUrlValue`,
+`isDateValue`, `isColorValue`, `createTable(columns, rows)`,
+`createDocument(text, {format, title})`, `tableToCsv(table)`,
+`fileFormatFor(name)`, `checkFileSignature(bytes, mimeType)`, and the
+`DATA_VALUE_LIMITS`, `FILE_FORMATS`, `FILE_MEDIA_TYPES` and `DOCUMENT_FORMATS`
+tables. `IMPORT_FORMATS` lists every upload the host stores (media and files),
+and `inspectMediaImportFile(file, {kinds: ["file"]})` checks a chosen file.
+
+### Conversions
+
+Connections convert between these types with host-owned conversions, version 1:
+
+| Conversion | From | To |
+| --- | --- | --- |
+| `text-to-url`, `url-to-text` | `text` | `url` (checked), and back |
+| `text-to-table`, `table-to-text` | CSV `text` | `table`, and back to CSV |
+| `object-to-table`, `table-to-object` | an `object` shaped as a table (for example from `csv-to-table`) | `table`, and back |
+| `text-to-document`, `document-to-text` | `text` | a plain `document`, and back |
+| `shots-to-shot-plan` | `shot[]` | the Shots workspace's version 1 `shot-plan` |
+
+A shot list reaches the Shots workspace when the output declares the
+workspace's meaning: `{"type": "shot[]", "semantic": "production_shot_plan"}`.
+
+Shots 2.2.0 and Image Panels 2.4.0 take narration from an earlier Step. A
+script whose output declares another meaning (or none), script-named `text`, or
+a transcript reaches their `narrationScript` and `narrationTiming` inputs
+through `script-as-narration`, `text-as-narration` and
+`transcript-as-narration-timing` (version 1, words unchanged; listed in
+`NARRATION_ADAPTERS`, not `ADAPTERS`). These are never
+chosen by matching: StillMade pins one on the connection when such a Step is
+added or replaced, or when the person picks "Use <Step>'s script". Declare
+`"semantic": "narration_script"` on a narration output to connect without one.
+
+### Example: PDF to shot list
+
+`examples/module-pdf-shot-list` is a module Block that takes an uploaded PDF,
+reads its text with pdf.js (bundled, its worker code imported directly so
+nothing is fetched), plans the shots with one `text.generate` call that has a
+JSON schema, and returns a `table` and a `shot[]` the Shots workspace accepts.

package/docs/build/frame-views.md

@@ -0,0 +1,259 @@
+# Frame views: any interface code
+
+# Frame views: any interface code
+
+A frame view (`"runtime": "frame"` in the view) is an interface written with
+ordinary web code: React, Svelte, Vue or Preact bundles, SVG, canvas and
+WebGL, pointer capture and drag, timers, loops, recursion, classes and Web
+Workers. Use it when the reviewed subset of standard views is too small for
+the interface you want.
+
+## Folder layout
+
+```
+my-block/
+  stillmade.block.json
+  src/run.js             or another runtime (recipe, module, capability)
+  src/view.html          markup, for example <div id="root"></div>
+  src/view.css           styles; use StillMade's theme variables
+  src/view.js            ONE classic script: bundle with --format=iife
+  src/view.config.json   {"runtime": "frame"}
+  tests/fixtures.json
+```
+
+`src/view.config.json` may also set `"appearance"` and `"files"` (assets, see
+below). Bundle npm dependencies into `src/view.js` as an IIFE, for example:
+
+```sh
+esbuild src/view.jsx --bundle --format=iife --minify --jsx=automatic \
+  --define:process.env.NODE_ENV='"production"' --outfile=src/view.js
+```
+
+Limits: markup and each stylesheet up to 256 KiB, `view.js` up to 4 MiB,
+and up to 32 asset files with 8 MiB in total.
+
+## The `StillMade` object
+
+Frame views get the same `StillMade` object as standard views: `input` and
+`onInput`, `run`, `useOutput`, `getShared`, `onShared`, `updateShared`,
+`bindShared`, `connectDocument`, `previewImage`, `previewMedia`, `render`
+and the rest of the interface bridge. In addition:
+
+| Member | What it does |
+| --- | --- |
+| `StillMade.asset(path)` | A `blob:` URL for one of the view's asset files, for `<img>`, `new FontFace()` or `<video>`. |
+| `StillMade.assetBytes(path)` | A copy of an asset file's bytes as an `ArrayBuffer`. |
+
+`StillMade.live` gives any view the camera and microphone as they happen, after the
+person presses Start (see Live camera and microphone in
+[Desktop capabilities](DESKTOP_CAPABILITIES.md)). A frame view can hand the frames to a
+Worker it starts from a `blob:` URL.
+
+Shared state works as in every view: keep each independently edited value in
+its own field (for example `point0`, `point1`) so two people editing
+different values never conflict.
+
+## Adding files to an input
+
+A one-page Block shows its input's files on its own screen as tiles and adds
+more there. Never draw an upload box or an `<input type="file">` of your own:
+the + tile asks StillMade, which opens its own file picker and saves the
+files exactly as the Step's own tiles do (working copies, files kept on the
+person's device, the limits and the error messages).
+
+```html
+<ul class="sm-gallery" data-size="small" id="photos">
+  <!-- one <li> per photo, drawn from onInput -->
+  <li><button type="button" class="sm-add" data-stillmade-add="listingPhotos"
+      aria-label="Add photos" title="Add photos"></button></li>
+</ul>
+```
+
+`data-stillmade-add` names a declared `image`, `video`, `audio`, `asset` or
+`file` input, or a list of one of them (`image[]`). It needs no code: a click
+or key press on the tile opens StillMade's picker. A list gets every chosen
+file added; a single input takes the first. The saved values reach the view
+the way every input does, so draw the tiles from `onInput`:
+
+```js
+StillMade.onInput(async input => {
+  for (const photo of input.listingPhotos || []) {
+    const preview = await StillMade.previewImage(photo); // display copy
+    // ...add an <li><img src="preview.url"></li> before the + tile
+  }
+});
+```
+
+From your own button, call `await StillMade.addFiles('listingPhotos')` inside
+its click handler. It resolves to `{added}` (0 when the person closes the
+picker) and rejects with a message for an undeclared input, an input that is
+not a file input, an input an earlier Step fills, a person who can only view
+the project, or a call that does not come from a click or key press in the
+view. The view never receives a `File` or a `blob:` URL, only the saved
+values. StillMade hides `[data-stillmade-add]` tiles from viewers and for
+inputs an earlier Step fills. In the SDK preview, the tile opens your
+computer's file chooser and the preview keeps the files like your own picks.
+
+### Removing and reordering files
+
+A list input's files are removed and put in order on the same screen, so a
+person drops a bad photo or sets the tour's order without leaving the Block.
+Put the buttons in each tile, with `data-index` (from 0) naming its place in
+the list `onInput` gave you:
+
+```html
+<li>
+  <img src="..." alt="Photo 3">
+  <span class="sm-tile-tools">
+    <button type="button" data-stillmade-move="listingPhotos" data-index="2" data-to="1"
+        aria-label="Move earlier" title="Move earlier"><span class="sm-glyph" data-glyph="chevron-left" aria-hidden="true"></span></button>
+    <button type="button" data-stillmade-move="listingPhotos" data-index="2" data-to="3"
+        aria-label="Move later" title="Move later"><span class="sm-glyph" data-glyph="chevron-right" aria-hidden="true"></span></button>
+    <button type="button" data-stillmade-remove="listingPhotos" data-index="2"
+        aria-label="Remove" title="Remove"><span class="sm-glyph" data-glyph="close" aria-hidden="true"></span></button>
+  </span>
+</li>
+```
+
+`data-stillmade-remove` removes the file at `data-index`; `data-stillmade-move`
+moves it to the place `data-to`. A `data-to` past either end of the list does
+nothing, so the first tile's Move earlier is harmless (disable it if you like).
+From your own code, call `await StillMade.removeFile('listingPhotos', 2)` or
+`await StillMade.moveFile('listingPhotos', 2, 1)` inside a click handler; each
+resolves to `{files}`, how many the list now holds. StillMade changes the
+Step's list as its own tile buttons do, keeping each saved file's upload record
+with it, and the new list reaches the view through `onInput`: redraw the tiles
+from it, never from your own copy.
+
+Both work only on list inputs (`image[]`, `video[]`, `audio[]`, `asset[]`,
+`file[]`); `addFiles` replaces a single input's file. They are refused, as
+`addFiles` is, for an undeclared input, an input an earlier Step fills, a
+viewer, Chat and a call that does not come from a click or key press in the
+view, and also when the file at that place changed after the view was given it
+(another person edited the list): the view gets the current list and the person
+tries again. StillMade hides these buttons wherever it hides the + tile. The SDK
+preview changes its example input the same way.
+
+## Theming
+
+The theme variables from StillMade's design system are always defined:
+`var(--bg)`, `var(--bg-elev-1)`, `var(--fg)`, `var(--fg-muted)`,
+`var(--accent)`, `var(--border-tok)`, `var(--divider)`, `var(--r-3)` and the
+rest. Colors written with them follow Light, Dark and White automatically.
+Frame views are not required to use them; the appearance check reports
+hard-coded colors as advice instead of refusing the view.
+
+A color the check takes is a theme variable, `transparent` or `currentColor`,
+or a gradient of them (`linear-gradient(to top, var(--bg), transparent)`).
+`opacity` is not a color and is always allowed. `color-mix()`, `rgba()`,
+`#rrggbbaa` and other written colors are what it reports. For words over a
+photo or video, use the kit's shade: `var(--scrim)` is the page's background,
+partly see-through, in every theme, and `class="sm-scrim"` lays it over the
+bottom of a positioned box (an `sm-stage`) under the words inside it:
+
+```html
+<figure class="sm-stage" data-fill>
+  <img src="..." alt="">
+  <figcaption class="sm-scrim">12 Elm Street · 3 bed</figcaption>
+</figure>
+```
+
+`--scrim` and `sm-scrim` are the only shade a Block uses; StillMade's own
+`--scrim-top` and `--scrim-bottom` (theme.css) are for its pages and the check
+reports them.
+
+## Layout
+
+StillMade zooms a view to fill the space its Step gives it (up to 150%), and
+Ctrl/Cmd +, - and 0 size the view while it has focus. It never centers a frame
+view; a standard view narrower than its Step is centered by its margins.
+
+An app that lays itself out (a full-size canvas or WebGL, its own scroll, pan
+or zoom, its own Ctrl/Cmd +/- keys, as in a CAD or drawing tool) puts
+`data-stillmade-layout="own"` on its outermost element:
+
+```html
+<div id="root" data-stillmade-layout="own"></div>
+```
+
+StillMade then never centers or zooms it and leaves Ctrl/Cmd +, - and 0 to the
+app. Without the attribute StillMade also stays out of the way when the view
+looks like an app: no page padding, a clipped page, a root as tall as the frame,
+fixed over half of it or scrolling over half of it, a canvas, SVG or
+`touch-action: none` surface over 40% of it, or a canvas or SVG of 300x200 or
+more with `touch-action: none` (as Three.js's controls set it). Mark the root
+anyway, in `view.html` rather than from code: StillMade reads it before the view
+starts, so the app never shows StillMade's size control or loses a key to it,
+even while it is still loading.
+
+A `<video controls>` plays in StillMade's player, the same one as everywhere in
+StillMade: play, timeline, time, sound and full screen along the bottom, shown
+while the pointer is on the video, with keys on the focused video (Space or K,
+the arrows, M, F) and touch. Full screen goes through StillMade, right after a
+click or key in the view; the frame itself has no `allow="fullscreen"`. A
+`<video>` without `controls` is the Block's own and StillMade never touches it,
+so a Block that drives playback (its own timeline, an editor or CAD viewer)
+leaves `controls` out, and `data-stillmade-player="native"` keeps the browser's
+controls. Result pictures from `StillMade.previewImage` in `sm-stage` or
+`sm-gallery`, and any element marked `data-stillmade-expand`, get a full-screen
+button in the bottom-right corner while the pointer is on them; pictures with
+the same value (`data-stillmade-expand="shots"`) open as one set, and
+`data-stillmade-expand="off"` removes the button. The keyboard reaches them
+too: such a picture without a `tabindex` of its own joins the tab order,
+keyboard focus on it shows the button, Enter or Space opens the viewer, and
+closing the viewer gives the focus back to the picture.
+
+## A job on one page: a check, then the run
+
+A job the person fills in and gets a result from is one page: the inputs on the
+left and the result's place on the right on a wide screen; on a phone they
+stack in the page's order, the inputs, the button in normal flow (never a sticky
+bar), then the result's place under it. When the expensive run needs a cheap
+check first (the script and a voice sample before the video), keep it on the
+same page: call `StillMade.run({make: 'check'})`, show what comes back, then
+`StillMade.run({make: 'video', script})` with what the person checked. Each run
+is priced and approved by StillMade on its own, so the view shows no price. The
+module side and what the person sees are in MODULES.md, "A check, then the
+expensive run, on one page".
+
+## Assets
+
+Images, fonts, JSON, audio and video the view needs ship as files named by
+SHA-256, like module files:
+
+```json
+{"runtime": "frame", "files": {"schemaVersion": 1, "files": {
+  "fonts/inter.woff2": {"sha256": "…", "bytes": 48256, "mediaType": "font/woff2"}
+}}}
+```
+
+`stillmade-block pack` fills in the digests and sizes. StillMade checks every
+file against its SHA-256 before the view starts.
+
+## What the frame does not allow
+
+The view runs in an opaque-origin frame with its own policy:
+
+- no network (`fetch`, WebSocket and remote images, fonts or scripts fail);
+- no code from strings (`eval`, `new Function`, inline event attributes) and
+  no extra script elements;
+- no navigation, popups, downloads, forms that submit, or dialogs;
+- no WebAssembly on the page (run heavy computation in the Block's runtime,
+  for example a module Block, and call it with `StillMade.run`);
+- no storage or access to StillMade's page.
+
+Workers created from `blob:` URLs are allowed and run off the page's thread.
+
+## Run limits
+
+StillMade compiles the view so every function and loop checks a time budget.
+One task may run for 250 ms; past that it is stopped with an error, and a view
+that keeps the page busy most of the time, stops answering, or navigates its
+frame is closed with a message. A single built-in operation that blocks for a
+second or more (a huge sort, or a regular expression that backtracks) also closes
+the view where the browser reports it. Move long computation into a Worker or the
+Block's runtime.
+
+The host enforces every permission itself: a view's code can message the host
+directly, so a read-only viewer cannot run the Block or save outputs from any
+view.

package/docs/build/generate-media.md

@@ -0,0 +1,59 @@
+# Generate media from JavaScript
+
+A JavaScript Block can generate images and videos with any model in the shared
+catalog (the same models the app's own Canvas and Image Panels use), speech in
+any catalog voice, music and sound effects by returning a declared
+`media.generate` action. Declare it in the manifest:
+
+```json
+"permissions": {"project": [], "actions": ["media.generate"], "network": [], "filesystem": [], "secrets": []},
+"outputs": {"request": {"type": "json"}, "media": {"type": "asset[]"}}
+```
+
+Return the action from your code. Each item is an `image`, `video`, `speech`,
+`music` or `sfx` item with a prompt and optional `settings`. Image and video
+items may also set `references` (images), `firstFrame`/`lastFrame` (video),
+`negativePrompt` and `seed`. Audio items take only `settings`:
+
+- `speech`: the prompt is the text to speak (up to 3,800 characters);
+  `settings: {voice, speed}` with a catalog voice id and a speed from 0.25 to 4.
+- `music`: `settings: {durationSec}`, 5 to 240 whole seconds (default 30).
+- `sfx`: `settings: {durationSec}`, 0.5 to 22 seconds in tenths (default 4).
+
+Video items take `settings: {model, duration, resolution, aspectRatio, generateAudio}`.
+With no `model`, an `aspectRatio` the default model (Vidu Q3 Turbo, which takes none)
+cannot make picks the cheapest catalog model that makes it with the frames and
+`duration` given, so `{aspectRatio: "9:16"}` alone asks for a vertical clip.
+
+```js
+return {request: {
+  schemaVersion: 1, kind: "stillmade.media-action", operation: "media.generate",
+  resultPort: "media",
+  request: {items: [
+    {key: "hero", operation: "image", prompt: input.prompt,
+     settings: {model: "nano-banana-pro", aspectRatio: "16:9", resolution: "2K"},
+     references: input.styleImages},
+    {key: "clip", operation: "video", prompt: "Slow push in on the harbor",
+     settings: {model: "kling-v3-turbo", duration: 5, aspectRatio: "16:9"}},
+    {key: "intro", operation: "speech", prompt: "Welcome back to the show.",
+     settings: {voice: "adam", speed: 1}},
+    {key: "bed", operation: "music", prompt: "Warm lo-fi piano", settings: {durationSec: 30}},
+    {key: "whoosh", operation: "sfx", prompt: "A fast whoosh", settings: {durationSec: 1.5}}
+  ]}
+}};
+```
+
+StillMade turns every item into one exact generation request and asks the server
+for one maximum price for the whole batch, using StillMade's own prices. The user
+sees that maximum and approves it once; credits are reserved before any provider
+call, each call is settled from provider evidence, and unused credits are
+returned when the batch closes. The approval is bound to this Block's exact
+content digest, so changed code needs a new approval. A Block cannot name a
+price, a provider, a key or another Block. References and frames must be images
+saved in the user's own library. Up to 50 items per action.
+
+When it finishes, the declared `resultPort` receives an array of
+`{key, kind, assetId, versionId, url, duration?}` in item order (`kind` is `image`,
+`video` or `audio`; `duration`, in seconds, is the length a video, music or sound
+effect item asked for; speech has none, so decode it with `audio.decode` to time it); failed items are listed
+separately and are not charged beyond what the provider actually received.