stillmade-sdk 0.1.35 → 0.1.36

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/AGENTS.md +8 -4
  2. package/DESKTOP_CAPABILITIES.md +81 -6
  3. package/FRAME_VIEWS.md +5 -0
  4. package/MODULES.md +104 -11
  5. package/PASTE_TO_LLM.md +14 -6
  6. package/SDK.md +245 -50
  7. package/blocks/connected-app/versions/1.6.0/view.js +27 -0
  8. package/blocks/gmail/versions/1.1.0/workspace.js +15 -0
  9. package/blocks/native/sm.when/versions/1.1.0/package.js +216 -0
  10. package/package.json +1 -1
  11. package/packages/block-cli/cli.js +27 -32
  12. package/packages/block-cli/dev-host/assets/host.js +38 -37
  13. package/packages/block-cli/dev-host/assets/index.css +1 -1
  14. package/packages/block-cli/dev-host/index.html +1 -0
  15. package/packages/block-cli/format.js +8 -0
  16. package/packages/block-platform/catalog.js +17 -1
  17. package/packages/block-platform/segment-runner.js +10 -4
  18. package/packages/block-platform/steps.js +6 -0
  19. package/packages/block-sdk/admission.js +11 -4
  20. package/packages/block-sdk/builder-contract.js +10 -5
  21. package/packages/block-sdk/capabilities.js +3 -0
  22. package/packages/block-sdk/connected-app.js +1 -1
  23. package/packages/block-sdk/contracts.js +1 -0
  24. package/packages/block-sdk/desktop-permissions.js +1 -1
  25. package/packages/block-sdk/first-party-identities.js +1 -0
  26. package/packages/block-sdk/first-party-packages.js +2 -1
  27. package/packages/block-sdk/folder-package.js +75 -0
  28. package/packages/block-sdk/index.d.ts +32 -6
  29. package/packages/block-sdk/index.js +4 -0
  30. package/packages/block-sdk/local-admission.js +4 -2
  31. package/packages/block-sdk/media-transform.js +42 -2
  32. package/packages/block-sdk/mock-host.js +3 -16
  33. package/packages/block-sdk/mocks.js +20 -0
  34. package/packages/block-sdk/module-host-calls.js +23 -0
  35. package/packages/block-sdk/module-hosted.js +22 -1
  36. package/packages/block-sdk/module-node.js +15 -20
  37. package/packages/block-sdk/module-runtime.js +7 -4
  38. package/packages/block-sdk/module-sockets.js +66 -0
  39. package/packages/block-sdk/module-worker.js +38 -1
  40. package/packages/block-sdk/network.js +20 -0
  41. package/packages/block-sdk/permission-summary.js +97 -0
  42. package/packages/block-sdk/permission-usage.js +72 -0
  43. package/packages/block-sdk/port-types.js +1 -1
  44. package/packages/block-sdk/schema-fields.js +11 -4
  45. package/packages/block-sdk/testing.js +20 -7
  46. package/packages/block-sdk/view-bridge.js +2 -1
  47. package/blocks/connected-app/versions/1.5.0/view.js +0 -18
  48. package/blocks/gmail/versions/1.0.0/workspace.js +0 -6
package/AGENTS.md CHANGED
@@ -6,7 +6,7 @@ This folder is the StillMade Block SDK. Read this file before writing code; it i
6
6
 
7
7
  Do not tell the user a Block is ready, done or working until `node packages/block-cli/cli.js gate .` passes after your last edit, and report the exact status it prints. In this folder Claude Code (`.claude/settings.json`) and Cursor (`.cursor/hooks.json`) run the gate when you finish a turn and send any failures back to you, but only in a session started in this folder: if you downloaded the SDK during the current session, run the gate yourself. Other agents must also run it themselves. A module Block runs code, so the gate a hook starts does not run it and says so: run `gate` or `ready` on it yourself. If you cannot fix a failure, say plainly that the Block is not ready and list the failing checks.
8
8
 
9
- Contract `stillmade.block-builder@1.7.1`. Humans, coding agents, imported source, GitHub repositories and generated code all adapt to the Block platform; anything that becomes a StillMade Block obeys this contract.
9
+ Contract `stillmade.block-builder@1.10.0`. Humans, coding agents, imported source, GitHub repositories and generated code all adapt to the Block platform; anything that becomes a StillMade Block obeys this contract.
10
10
 
11
11
  ## Source of truth
12
12
 
@@ -20,6 +20,10 @@ Before building, check what already exists (`find_existing_blocks` over MCP, or
20
20
 
21
21
  Blocks come in three sizes and the manifest decides which (packages/block-sdk/block-tier.js): an app is a workspace a person works in (Editor, Canvas, Shots), placed in a Type and never wired automatically; a job is one deliverable a person would name (the narration, the captioned clip), built from the hosted operations it needs (`permissions.capabilities`) and reviewed in one place, the unit Chat picks for a Step; an ingredient is one hosted operation with nothing of its own (a declarative capability Block), what a job is made of. When two ingredients would stand in a row to make one deliverable (writing the words, then voicing them), build or pick the job that does both and keep the review point inside it. A job declares a second output when another Block consumes the intermediate (the script beside the voice); split into two jobs only when the intermediate has a life of its own or the halves have different owners or payment.
22
22
 
23
+ ## Choose the runtime
24
+
25
+ Pick the runtime from the job. No hosted AI: a recipe or isolated `javascript`, both able to run unattended. One text, script or brief prompt returning text, script or JSON: a `capability` Block. Hosted operations on other values (a transcript, media, structured data), several hosted operations in a row, or a result shaped in code: one `module` Block, `module/main.js` calling `stillmade.hosted.run(operation, request)` for each operation in `permissions.capabilities` (MODULES.md), never one Block per operation; it runs in the person's browser while StillMade is open. Build a new Block in a folder with `stillmade-block create`, `test`, `ready` and `pack`, and import the file `pack` writes (a module Block writes `<id>-<version>.stillmade-module.json`) through Create, then Import Block. A `.stillmade-block` archive is only for adapting a GitHub repository.
26
+
23
27
  ## One job per Step
24
28
 
25
29
  A Step is one job: the work between two moments where a person or the outside world must see the result. Start a new Step only at one of three points; everything else is a part of the neighboring Step.
@@ -32,7 +36,7 @@ In a production the Steps are the workspaces a person opens: work feeding the ne
32
36
 
33
37
  A part that only runs in the desktop app cannot share a Step with cloud parts. Before adding a Step, name which of the three reasons makes it separate; if none applies, make it a part of a neighbor. A Step nobody would ever open must not exist on its own. Prefer an app task's own filter (a Gmail query such as `has:attachment filename:png`) to an AI check, and use AI decide only for judgement.
34
38
 
35
- Inside one job, each capability stays its own part: a capability Block performs exactly one hosted operation and a connector one app task, so every part keeps its own permission, approval and credit rules, and StillMade groups the parts of one job into one Step (`groups` in the Project Type). Never invent a combined runtime. When one Block can do the whole job with documented primitives (a JavaScript Block doing several local transforms, a workspace owning its whole document), build it as one Block.
39
+ Build one Block for a job one Block can do: a module Block calls every hosted operation the job needs (it transcribes a sermon, then finds the verses said in it), a JavaScript Block does several local transforms, a workspace owns its whole document. A Block being built never takes an ingredient's result only so the ingredient can be its own part or Step (Transcribe in front of a verse finder): it calls that operation itself. Parts in one Step (`groups` in the Project Type) are for jobs made of Blocks that already exist and app tasks, each keeping its own permission, approval and credit rules. Never invent a combined runtime.
36
40
 
37
41
  Examples: "every new email with an image becomes a generated image" is two Steps, "Gmail · New emails with images" (When, fetch with an image filter, only new) and "Image from email" (AI write, Generate image); listing When, Fetch, Only new, AI decide, Continue only if, AI write and Generate image as seven Steps splits one job into its parts. "A new Meta lead goes to HubSpot" is three Steps: "Meta Ads · New leads", "Qualify lead" and "HubSpot · Add contact".
38
42
 
@@ -70,7 +74,7 @@ Projects are multiplayer. A custom view subscribes through `StillMade.onShared(c
70
74
 
71
75
  ## External services
72
76
 
73
- Reach external services only through StillMade's hosted operations, the connections service (`connection.execute`), or, in a module Block, `stillmade.net.fetch` to origins declared in `permissions.network` (with the key each needs) and `stillmade.connections.call` to adapters or catalog apps in `permissions.connections`; never put keys in code, inputs or headers: StillMade adds each person's own key on its servers. The host owns the model and provider choice, BYOK keys, StillMade credits, spending consent, authorization, token refresh, rate limits, retries and provider errors; a Block never receives a secret and never names a provider unless it must. Treat provider unavailability, timeout, cancellation and partial failure as typed outcomes the Block reports, not crashes. Generated images, video, voices, music and sound effects come from hosted operations or the `media.generate` action, paid with the user's StillMade credits at StillMade's prices; for structured model output declare a `json` output with `capability.schema` instead of parsing prose. When a Block needs npm libraries, WebAssembly or heavy compute, use `runtime: "module"` and reach hosted operations through `stillmade.hosted.run` with the operations listed in `permissions.capabilities`. Decode, cut, join, re-encode, extract or normalise audio and video with `stillmade.media.transform` on the device; never ask for a server render. For a rich interactive interface (React or another framework, SVG, canvas, drag), use a frame view: `src/view.config.json` with `{"runtime":"frame"}` and `src/view.js` bundled as one IIFE, colored with StillMade's theme variables. For files and structured data use the `file`, `document`, `table`, `url`, `date` and `color` types and their generated controls and presentations, not JSON text.
77
+ Reach external services only through StillMade's hosted operations, the connections service (`connection.execute`), or, in a module Block, `stillmade.net.fetch` to origins declared in `permissions.network` (with the key each needs), `stillmade.net.connect` for a live WebSocket to one of them, and `stillmade.connections.call` to adapters or catalog apps in `permissions.connections`; never put keys in code, inputs or headers: StillMade adds each person's own key on its servers. The host owns the model and provider choice, BYOK keys, StillMade credits, spending consent, authorization, token refresh, rate limits, retries and provider errors; a Block never receives a secret and never names a provider unless it must. Treat provider unavailability, timeout, cancellation and partial failure as typed outcomes the Block reports, not crashes. Generated images, video, voices, music and sound effects come from hosted operations or the `media.generate` action, paid with the user's StillMade credits at StillMade's prices; for structured model output declare a `json` output with `capability.schema` instead of parsing prose. When a Block needs npm libraries, WebAssembly (threaded: `"threads": true` in module.json) or heavy compute, use `runtime: "module"` and reach hosted operations through `stillmade.hosted.run` with the operations listed in `permissions.capabilities`. Decode, cut, join, re-encode, extract or normalise audio and video with `stillmade.media.transform` on the device; never ask for a server render. For a rich interactive interface (React or another framework, SVG, canvas, drag), use a frame view: `src/view.config.json` with `{"runtime":"frame"}` and `src/view.js` bundled as one IIFE, colored with StillMade's theme variables. For files and structured data use the `file`, `document`, `table`, `url`, `date` and `color` types and their generated controls and presentations, not JSON text.
74
78
 
75
79
  ## Dependencies
76
80
 
@@ -78,7 +82,7 @@ A Block must declare everything it requires and must not assume anything is inst
78
82
 
79
83
  ## Security
80
84
 
81
- Treat Block code, external data, files, repositories, model output and network responses as untrusted. Request the minimum permissions and capabilities. Do not weaken the sandbox (QuickJS has no fetch, DOM, timers, modules or Node APIs). Validate and bound every value crossing a boundary. Never expose secrets through logs, outputs, project context, UI, error messages or serialized state.
85
+ Treat Block code, external data, files, repositories, model output and network responses as untrusted. Request the minimum permissions and capabilities; `stillmade-block permissions --fix` declares exactly what the source uses. Do not weaken the sandbox (QuickJS has no fetch, DOM, timers, modules or Node APIs). Validate and bound every value crossing a boundary. Never expose secrets through logs, outputs, project context, UI, error messages or serialized state.
82
86
 
83
87
  ## Execution lifecycle
84
88
 
@@ -1,8 +1,15 @@
1
1
  # Desktop capabilities for Blocks
2
2
 
3
- Recording runs on both platforms: in a browser the trusted web host records with the browser's own screen, camera or microphone picker and keeps the finished file on the device (a `sm-media://device/...` reference); in StillMade Desktop the native host gives consent and picks the source. Every other scope below needs StillMade Desktop.
3
+ Recording and folders run on both platforms: in a browser the trusted web host records with the browser's own screen, camera or microphone picker and keeps the finished file on the device (a `sm-media://device/...` reference), and shares folders through the browser's folder picker (Chrome and Edge); in StillMade Desktop the native host does both. Every other scope below needs StillMade Desktop.
4
4
 
5
- The desktop host implements these capabilities on demand. Importing a Block, installing StillMade, or opening a preview does not request device permissions. The first operation displays StillMade's native consent dialog identifying the Block ID and version. An OS prompt may follow. Grants last for the mounted Block workspace, up to one hour; closing/reloading it or revoking access ends that grant. Imported UI receives neither native IPC nor grant tokens.
5
+ **Consent is the person's own action.** A Block reaches only what it declares, and StillMade refuses everything else, so people are never asked to judge a Block. Picking a folder or files in the system dialog, or pressing StillMade's Start button for a recording or live feed, is the consent: there is no second "Allow?" question. Notifications and keeping the computer awake need no question. StillMade asks once only about what happens without such an action: reading the clipboard or the cursor, replacing the clipboard, and running a desktop program. An OS prompt may follow the first use of the camera or microphone. Importing a Block, installing StillMade or opening a preview asks nothing. Imported UI receives neither native IPC nor grant tokens.
6
+
7
+ **What a person allows is remembered for the exact Block where it is used**: their account, the project Step, and the exact package (its digest). Picked folders and answered questions carry over to the next session and the next launch, in StillMade Desktop and in the browser. Any other package starts with nothing, including a new version of the Block, another Block in the same Step, or a Block from another author that reuses the same id. Remembered access ends when:
8
+ - the Step's Block changes: an update, or another Block put in its place (it picks or answers again, once);
9
+ - StillMade quarantines the Block;
10
+ - the person forgets it: **Stop sharing** under the Block, or **Forget** in Desktop access.
11
+
12
+ Under every Block that uses device features, **What this Block did** lists the folders it uses and what it read, wrote, copied or ran on this device. The Block cannot read or hide it.
6
13
 
7
14
  Declare the exact required scopes in `manifest.permissions.desktop`. Other permission families remain unchanged. Pure recipe/JavaScript execution stays sandboxed; the **Block UI bridge** requests desktop operations through the trusted host.
8
15
 
@@ -10,14 +17,16 @@ Declare the exact required scopes in `manifest.permissions.desktop`. Other permi
10
17
  | --- | --- | --- |
11
18
  | `screen.capture` | `record.start`, `{kind:"screen"}` | Choose a screen or window, then record it. Browser and desktop. |
12
19
  | `cursor.track` | `cursor.track`, or `{cursor:true}` on `record.start` | Global cursor coordinates and display geometry; recordings sample at 10 Hz. Does not monitor clicks or keyboard input. |
13
- | `camera.capture` | `record.start`, `{kind:"camera"}` | Camera recording after device/OS approval. Browser and desktop. |
14
- | `microphone.capture` | `record.start`, `{kind:"microphone"}`, or `{kind:"screen", microphone:true}` | Microphone recording after device/OS approval, or the microphone mixed into a screen recording. Browser and desktop. |
20
+ | `camera.capture` | `record.start`, `{kind:"camera"}`; `live.start`, `{camera:true}` | Camera recording, or the live camera feed, after device/OS approval. Browser and desktop. |
21
+ | `microphone.capture` | `record.start`, `{kind:"microphone"}`, or `{kind:"screen", microphone:true}`; `live.start`, `{microphone:true}` | Microphone recording after device/OS approval, the microphone mixed into a screen recording, or the live microphone feed. Browser and desktop. |
15
22
  | `clipboard.read` | `clipboard.read` | Returns `{text}`; bounded to 100,000 characters. |
16
23
  | `clipboard.write` | `clipboard.write`, `{text}` | Explicit permission to replace clipboard text. |
17
24
  | `media.pick` | `media.pick` | Native picker returns `{assets}` for up to 20 selected media files; no arbitrary path access. |
18
25
  | `notifications.show` | `notifications.show`, `{body}` | StillMade/Block-branded desktop notification; maximum one every ten seconds. |
19
26
  | `power.keep-awake` | `power.keep-awake`, `{enabled:true}` | Prevent app suspension until disabled, revoked, closed, or expired. |
20
27
  | `native.tools` | `tool.status` and declared `tool.*` operations | Use one exact versioned adapter for an approved installed desktop application. |
28
+ | `folder.read` | `folder.pick`, `folder.shared`, `folder.list`, `folder.read`, `folder.forget` | List and read files in a folder the person picks. The Block gets an opaque folder id and names files by relative paths. Desktop, and Chrome and Edge. |
29
+ | `folder.write` | `folder.pick`, `{write:true}`; `folder.write` | Create and replace files in a folder the person shared for changing. Declare `folder.read` too. Desktop, and Chrome and Edge. |
21
30
 
22
31
  The host always checks declarations. Declaring a scope does not grant it. For anything but recording, a browser receives a clear desktop-required error; there is no silent paid/cloud fallback. A Block whose only desktop scopes are recording scopes is not marked as requiring StillMade Desktop.
23
32
 
@@ -37,6 +46,72 @@ Natron adapter version 1 currently exposes `launch`, `project.import`, `project.
37
46
 
38
47
  The mapped Natron Block uses `video` as both its primary typed input and output, with semantic `video`. This allows an explicit Project Type chain such as video producer → Natron → review/export Block. Its custom interface submits the connected original input and render settings to `StillMade.run`; the approved desktop executor invokes the renderer and returns the newly registered artifact with a bound native receipt. Source fixtures and JavaScript passthrough do not prove native execution. See [Native desktop execution](DESKTOP_EXECUTION.md) for `src/desktop.json`, offline admission, host callbacks and retained-result requirements.
39
48
 
49
+ ## Folders you choose
50
+
51
+ A Block that works on a person's files (a photo culler, a subtitle batch, a project exporter) asks for one folder at a time. The person picks it in the system folder dialog (StillMade Desktop) or the browser's folder picker (Chrome and Edge); the Block never sees a path and reaches nothing outside that folder. The folder is remembered for the Block where it is used, so start with the folders it already has:
52
+
53
+ ```js
54
+ let [folder] = (await StillMade.desktop('folder.shared')).folders;
55
+ if (!folder) ({folder} = await StillMade.desktop('folder.pick', {write: true}));
56
+ const {entries} = await StillMade.desktop('folder.list', {folder: folder.id, path: 'raw'});
57
+ const {text} = await StillMade.desktop('folder.read', {folder: folder.id, path: 'notes.txt'});
58
+ const {asset} = await StillMade.desktop('folder.read', {folder: folder.id, path: 'raw/a.png', as: 'media'});
59
+ await StillMade.desktop('folder.write', {folder: folder.id, path: 'edits/notes.txt', text: 'Take two'});
60
+ await StillMade.desktop('folder.write', {folder: folder.id, path: 'cover.png', asset, replace: true});
61
+ ```
62
+
63
+ - `folder.pick` returns `{folder: {id, name, write}}`. `{write:true}` asks to share it for changing, and needs `folder.write`; the picker says which. Picking a folder already shared keeps its id. In a browser, call it from a click in the Block: Chrome opens its picker only after one, and otherwise StillMade shows its own Choose folder button first.
64
+ - `folder.shared` returns `{folders: [{id, name, write}]}`: the folders this Block already has here, kept from earlier sessions. `folder.forget` with `{folder}` stops using one.
65
+ - `folder.list` returns `{entries: [{name, kind, size, modified}], truncated}`, folders first, at most 1,000 entries. Links are left out.
66
+ - `folder.read` returns `{text, size}` for a text file up to 1 MB, or `{asset, size}` with `as: "media"` for an image, video or audio file: on Desktop the file becomes a saved desktop asset; in a browser it stays on the device, linked where it sits (`sm-media://device/...`). Either way the Block can show it, pass it to `StillMade.run` or return it.
67
+ - `folder.write` takes `text` (up to 1 MB) or `asset`, a media value this Block saved, read or received in this session. Missing folders on the path are created. An existing file is replaced only with `replace: true`. The file is written beside its destination first and moved into place, so a failed write leaves no partial file.
68
+ - Paths are relative, separated by `/`, at most 32 levels and 1,024 characters. `..`, `.`, absolute paths, drive letters, backslashes, names with leading or trailing spaces or a trailing dot, reserved Windows names (`CON`, `NUL`) and Windows short names (`GIT~1`) are refused. On Desktop, hidden-file checks also run on the resolved path, a symbolic link anywhere on the path is refused rather than followed, and every resolved path is checked to be inside the folder. A browser does not show links or the folders above the picked one; Chrome itself refuses system folders and the home folder.
69
+ - Hidden files and folders (names starting with `.`, such as `.git`, `.env`, `.vscode` or a shell profile) are left out of lists and cannot be read or written.
70
+ - `folder.write` creates and replaces only media, subtitle, plain-text and table files: images (`png`, `jpg`, `webp`, `tiff`, `exr`, `psd` and similar), video (`mp4`, `mov`, `webm`, `mkv`, `mxf` and similar), audio (`mp3`, `wav`, `m4a`, `flac` and similar), `txt`, `srt`, `vtt`, `ass`, `csv`, `tsv`, `edl`, `otio`, `cube` and `pdf`. Structured data goes in a file named `<name>.stillmade.json`. Other `.json` and `.md` files are not written, because tools and coding agents read names like `package.json` or `CLAUDE.md` as settings or instructions; nor are programs, scripts, shortcuts, disk images, macro documents, build files such as `CMakeLists.txt`, or anything inside a macOS bundle such as `Name.app`. This is an allowlist: a type it does not name, including a lookalike written with other characters, is refused. Written files are never executable.
71
+ - The person cannot pick a whole drive, their home folder or a folder above it. A folder shared for changing also cannot be hidden, inside application settings (`AppData`, `Library`) or inside a code repository (a folder with `.git` at or above it on Desktop, inside it in a browser); those can still be shared for reading. Every write checks again, so a repository created inside the folder later stops writes into it.
72
+ - A Block holds at most four folders. They are remembered for the exact Block where it is used until the person stops sharing them (under the Block, or in Desktop access), the Step's Block changes, or StillMade quarantines it. On Desktop each remembered folder is checked again before use: one that was moved, replaced by a link or turned into a repository is dropped. In a browser, Chrome may ask once to allow a remembered folder again after it restarts, or keep it for good with "Allow on every visit".
73
+ - Nothing deletes or renames files. In a read-only workspace a folder can be picked for reading, listed and read; picking one for changing and `folder.write` need edit access.
74
+
75
+ ## Live camera and microphone
76
+
77
+ A view can use the camera, the microphone or both as they happen instead of a
78
+ finished recording: a live filter, a tuner, a level meter, a pose or gesture
79
+ check, live captions. It declares the same scopes as recording and uses
80
+ `StillMade.live` (standard and frame views, browsers and StillMade Desktop):
81
+
82
+ ```js
83
+ const canvas = document.getElementById('picture'), context = canvas.getContext('2d');
84
+ StillMade.live.onFrame((frame, info) => { // an ImageBitmap, info.width x info.height
85
+ canvas.width = info.width; canvas.height = info.height;
86
+ context.drawImage(frame, 0, 0); // draw or copy it now: it is closed afterwards
87
+ });
88
+ StillMade.live.onAudio((samples, info) => { // mono Float32Array, info.sampleRate
89
+ meter.value = Math.max(...samples.map(Math.abs));
90
+ });
91
+ StillMade.live.onEnd(event => { status.textContent = 'Camera off'; }); // event.reason
92
+ document.getElementById('start').addEventListener('click', () =>
93
+ StillMade.live.start({ camera: true, microphone: true, fps: 15, width: 640, sampleRate: 16000 }));
94
+ ```
95
+
96
+ - `live.start` resolves with `{live, camera, microphone, fps, width, height, sampleRate}`
97
+ once the person presses Start in StillMade, in a browser and in StillMade
98
+ Desktop. `fps` is 1 to 30 (default 15), `width` 160 to 1280
99
+ pixels (default 640, never wider than the camera), `sampleRate` 8000 to 48000
100
+ (default 16000; a browser that cannot resample reports its own rate in each chunk).
101
+ - The view never holds the camera or microphone. StillMade keeps the stream,
102
+ shows that the device is on with a Turn off button, and posts transferred
103
+ copies: one `ImageBitmap` per frame and one 4,096-sample `Float32Array` per
104
+ audio chunk. At most two frames are unanswered at a time, so a slow view
105
+ receives fewer frames, never a backlog.
106
+ - `live.stop()` (or Turn off, closing the Block, or revoking access)
107
+ ends the feed; `onEnd` receives the reason (`stopped`, `person`, `closed`,
108
+ `ended` when the device stops, `time`). Nothing is recorded or saved.
109
+ - A recording and a live feed do not run at the same time. `live.stop` and
110
+ `live.status` work in a read-only workspace; `live.start` needs edit access.
111
+ - For heavy processing of frames (a WebAssembly model), copy what you need
112
+ (`getImageData`, `createImageBitmap(frame)`) and hand it to a Worker in a frame view,
113
+ or to the Block's runtime with `StillMade.run`.
114
+
40
115
  ## Ready-to-import recording Blocks
41
116
 
42
117
  Download the complete [Screen recorder](/block-sdk/examples/screen-recorder.stillmade.json), [Camera recorder](/block-sdk/examples/camera-recorder.stillmade.json), or [Microphone recorder](/block-sdk/examples/microphone-recorder.stillmade.json). These packages also ship in the SDK archive. Import one through StillMade’s normal review flow, then use it in a browser or in the desktop app. Each includes a themed interface and pure runtime fixtures that preserve the media reference. The fixtures use synthetic references; they do not exercise your device.
@@ -76,7 +151,7 @@ stop.addEventListener('click', async () => {
76
151
  });
77
152
  ```
78
153
 
79
- Every recording in a browser, and a screen recording on desktop, begins from StillMade's own Start button outside the Block interface (labelled for the screen, camera or microphone): a view never reaches a device by itself, and the site's standing camera or microphone permission is not consent for a Block. On desktop, camera and microphone recordings begin after StillMade's consent dialog naming the Block.
154
+ Every recording, in a browser and in StillMade Desktop, begins from StillMade's own Start button outside the Block interface (labelled for the screen, camera or microphone): pressing it is the consent. A view never reaches a device by itself, and the site's standing camera or microphone permission is not consent for a Block.
80
155
 
81
156
  `record.stop` returns `{asset, duration, width, height, bytes, mimeType, stoppedBy, message, cursor, displays, coordinates}`. `asset` is a canonical media reference with `assetId`, `versionId`, `kind` and a device-local `sm-media:` URL (`sm-media://asset/...` from the desktop media library, `sm-media://device/...` for a file a browser keeps on the device, which `previewMedia`, module `files.read` and the next Step read without uploading). `duration` is seconds measured by the host and written into the WebM header, so the recording seeks and cuts correctly. `stoppedBy` is present when the host ended the recording before `record.stop`: `size`, `time` or `sharing` (the person ended the share); the recording up to that point is kept and `message` says so. Cursor samples contain `{x,y,time}`: global device-independent screen coordinates and milliseconds relative to recording start. Multiple monitors can have negative coordinates. These are not automatically converted into cropped-window video coordinates. Share/upload media through the existing host media flow before teammates on other devices can use it. By default that flow uploads a 1080p working copy of large video (the account's media setting); the original stays in the desktop media library.
82
157
 
@@ -86,7 +161,7 @@ Every recording in a browser, and a screen recording on desktop, begins from Sti
86
161
 
87
162
  `record.cancel` discards an active capture. `session.close` revokes all access and cancels capture. StillMade also renders a host-owned cancel/revoke control outside the Block iframe. Screen source selection happens for every recording. No preview can bypass consent or access the native bridge directly.
88
163
 
89
- Current limits: one recording per Block workspace. In StillMade Desktop the recording streams to disk as it is captured (each chunk crosses to the desktop process as it arrives; the header is completed with the measured duration at the end), so the caps are 4 hours and free disk space; a desktop app without streamed recordings keeps 30 minutes and 128 MiB. In a browser the caps are 30 minutes, 2 GB and the browser's storage quota (`device.info` says how many minutes fit). Hitting a limit, the disk running low, or the person ending the share stops the recording and keeps what was captured; `record.stop` then saves it and reports `stoppedBy` (`size`, `time`, `disk` or `sharing`). `record.status` reports `limits` with `bytes: null` where only disk space applies. Output is WebM where the browser records it (H.264 or VP9 inside WebM in Chromium) and MP4 where only that is available (Safari). Camera recording is video-only; microphone recording is audio-only. Optional `{systemAudio:true}` is implemented through Windows loopback; other platforms reject it explicitly. Microphone mixing, global click/keyboard hooks, remote-control automation, arbitrary subprocesses, and folder watching are not provided by these scopes.
164
+ Current limits: one recording per Block workspace. In StillMade Desktop the recording streams to disk as it is captured (each chunk crosses to the desktop process as it arrives; the header is completed with the measured duration at the end), so the caps are 4 hours and free disk space; a desktop app without streamed recordings keeps 30 minutes and 128 MiB. In a browser the caps are 30 minutes, 2 GB and the browser's storage quota (`device.info` says how many minutes fit). Hitting a limit, the disk running low, or the person ending the share stops the recording and keeps what was captured; `record.stop` then saves it and reports `stoppedBy` (`size`, `time`, `disk` or `sharing`). `record.status` reports `limits` with `bytes: null` where only disk space applies. Output is WebM where the browser records it (H.264 or VP9 inside WebM in Chromium) and MP4 where only that is available (Safari). Camera recording is video-only; microphone recording is audio-only. Optional `{systemAudio:true}` is implemented through Windows loopback; other platforms reject it explicitly. Microphone mixing, global click/keyboard hooks, remote-control automation, arbitrary subprocesses, and folder watching are not provided by these scopes (folder access is `folder.read` and `folder.write`, above).
90
165
 
91
166
  ## Testing and installation
92
167
 
package/FRAME_VIEWS.md CHANGED
@@ -42,6 +42,11 @@ and the rest of the interface bridge. In addition:
42
42
  | `StillMade.asset(path)` | A `blob:` URL for one of the view's asset files, for `<img>`, `new FontFace()` or `<video>`. |
43
43
  | `StillMade.assetBytes(path)` | A copy of an asset file's bytes as an `ArrayBuffer`. |
44
44
 
45
+ `StillMade.live` gives any view the camera and microphone as they happen, after the
46
+ person presses Start (see Live camera and microphone in
47
+ [Desktop capabilities](DESKTOP_CAPABILITIES.md)). A frame view can hand the frames to a
48
+ Worker it starts from a `blob:` URL.
49
+
45
50
  Shared state works as in every view: keep each independently edited value in
46
51
  its own field (for example `point0`, `point1`) so two people editing
47
52
  different values never conflict.
package/MODULES.md CHANGED
@@ -4,8 +4,10 @@ A module Block (`"runtime": "module"`) is an ordinary ES module. It can use
4
4
  everything a modern browser Worker offers: `async`/`await`, timers, typed
5
5
  arrays, `WebAssembly`, `OffscreenCanvas`, `createImageBitmap`, WebCodecs, and
6
6
  any npm package you bundle into it, including C and C++ libraries compiled to
7
- WebAssembly (OpenCV, libvips builds, codecs, solvers). Use it when a recipe or
8
- the QuickJS `javascript` runtime is too small for the job.
7
+ WebAssembly (OpenCV, libvips builds, codecs, solvers), threaded WebAssembly
8
+ (see Threads) and live WebSocket connections to declared origins (see Live
9
+ connections). Use it when a recipe or the QuickJS `javascript` runtime is too
10
+ small for the job.
9
11
 
10
12
  ## Folder layout
11
13
 
@@ -15,7 +17,7 @@ my-block/
15
17
  module/ everything the Block runs
16
18
  main.js the entry (bundle npm dependencies into it)
17
19
  filters.wasm optional WebAssembly and data files
18
- module.json optional: {"entry": "other.js"}
20
+ module.json optional: {"entry": "other.js", "threads": true}
19
21
  tests/fixtures.json [{name, input, expected}]
20
22
  ```
21
23
 
@@ -54,6 +56,8 @@ against the manifest's port types.
54
56
  | `media.transform(request)` | On-device media work done by StillMade: decode audio to samples, extract or loudness-normalise audio, cut, join or re-encode video, and take thumbnails. Results are saved like `files.write`. See [Media processing](#media-processing). |
55
57
  | `actions.run(action)` | Runs a declared portable action. `media.generate` asks the person to approve one StillMade credit maximum for the batch; `stock.search` runs directly. |
56
58
  | `hosted.run(operation, request)` | Calls a hosted operation the manifest lists in `permissions.capabilities` (for example `text.generate`, `image.generate`, `audio.speech`, `web.research`). Each call is quoted and approved like a capability Block and paid with StillMade credits. |
59
+ | `net.fetch(url, {method, headers, body})` | Calls one of the origins declared in `permissions.network` through StillMade, which adds the person's own key or OAuth token. Returns a standard `Response`. |
60
+ | `net.connect(url, {protocols})` | Opens a live WebSocket to the `wss://` form of a declared origin and resolves with the open socket. No key is added. See [Live connections](#live-connections). |
57
61
 
58
62
  Hosted calls take the operation's request fields, for example:
59
63
 
@@ -90,6 +94,19 @@ to the next transform.
90
94
  | `{operation: 'video.concat', sources, settings?}` | A saved MP4 of 2 to 50 videos in order. |
91
95
  | `{operation: 'video.resize', source, settings}` | A saved MP4 re-encoded with new settings. |
92
96
  | `{operation: 'image.thumbnail', source, at?, width?}` | A saved PNG of one frame (default the first frame, 1280 px wide). |
97
+ | `{operation: 'timeline.render', clips, kind?, aspect?, audio?, captions?, settings?}` | A saved MP4 (or, with `kind: 'audio'`, an MP3) of your own timeline, rendered the way the Editor exports. |
98
+
99
+ `timeline.render` is how a Block exports: there is no separate Export, Captions,
100
+ Transition or Color grade Block to call. Each clip is `{source, start, duration,
101
+ from?, transition?: {type, duration?}, look?, mute?}` (a saved image or video; a
102
+ video keeps its own sound unless `mute`), each audio item `{source, start,
103
+ duration, from?, volume?}`, and each caption `{text, start, end, words?,
104
+ position?, highlight?, animation?}`, where `words` (`[{text, start, end}]`)
105
+ highlights each word as it is spoken. Times are seconds on the timeline.
106
+ `aspect` is `16:9`, `9:16`, `1:1` or `4:5`. Looks, transitions and caption
107
+ animations use the Editor's names (`TIMELINE_LOOKS`, `TIMELINE_TRANSITIONS`,
108
+ `CAPTION_ANIMATIONS`). The render follows the person's export plan, like any
109
+ export from StillMade.
93
110
 
94
111
  `source`/`sources` are saved references (`{kind, assetId, versionId, url}`).
95
112
  Render `settings` are `{fps: 24|30|60, resolution: 720|1080|2160, quality:
@@ -122,9 +139,10 @@ WebAssembly codec. Images arrive as saved references: read the bytes with
122
139
  ## Sandbox and limits
123
140
 
124
141
  - The module runs in one dedicated Worker inside a response-sandboxed frame
125
- with an opaque origin. It has no network (`fetch`, WebSocket and remote
126
- imports fail), no storage of the app, no page or DOM access, and cannot read
127
- another Block's files.
142
+ with an opaque origin. It has no network of its own (`fetch`, WebSocket and
143
+ remote imports fail; `net.fetch` and `net.connect` go through StillMade), no
144
+ storage of the app, no page or DOM access, and cannot read another Block's
145
+ files.
128
146
  - Every file is fetched from StillMade's store and checked against its SHA-256
129
147
  before anything runs.
130
148
  - A run may take 2 minutes; each progress report allows 60 more seconds. Time
@@ -135,11 +153,63 @@ WebAssembly codec. Images arrive as saved references: read the bytes with
135
153
  - Generating code at run time (`new Function`, used by Emscripten bindings) is
136
154
  allowed inside the sandbox.
137
155
 
138
- Module Blocks run in StillMade in the person's browser, only while StillMade is
139
- open. They cannot run on a schedule, from an event or on new media: StillMade has
140
- no server sandbox for them. A workflow's background run stops before a module Step
141
- and the person runs it in the project. Marketplace listings show module Blocks as
142
- **Browser only**. For work that must run unattended, use a JavaScript Block.
156
+ Module Blocks run in StillMade on the person's device: in the browser while
157
+ StillMade is open, or in StillMade Desktop. StillMade has no server sandbox for
158
+ them. A schedule or event that hands browser-only Steps to the owner's StillMade
159
+ Desktop ("Run ... on my StillMade Desktop") pauses at a module Step until the
160
+ owner's desktop app, open with "Run my automations on this computer" on, runs it
161
+ with the Step's exact inputs; the workflow then continues from its result. Module
162
+ Steps that call paid hosted operations, generate media or need a device grant
163
+ still wait for the owner. Without that choice a background run stops before a
164
+ module Step and the person runs it in the project. Marketplace listings show
165
+ module Blocks as **Browser only**.
166
+
167
+ ## Live connections
168
+
169
+ `stillmade.net.connect(url, {protocols})` opens a WebSocket to the `wss://` form
170
+ of an origin in `permissions.network` (declare `https://stream.example.org` and
171
+ connect to `wss://stream.example.org/...`) and resolves once it is open:
172
+
173
+ ```js
174
+ const socket = await stillmade.net.connect('wss://stream.example.org/v1/live', { protocols: ['json.v1'] });
175
+ await socket.send(JSON.stringify({ subscribe: input.channel }));
176
+ for await (const data of socket) { // text arrives as strings, binary as ArrayBuffers
177
+ stillmade.progress(0.5, 'Listening'); // each progress report extends the run limit
178
+ if (JSON.parse(data).done) break; // leaving the loop closes the connection (code 1000)
179
+ }
180
+ ```
181
+
182
+ The socket also has `onmessage`, `onclose`, `addEventListener`, `readyState`,
183
+ `protocol`, `close(code, reason)` (1000 or 3000 to 4999) and `closed`, a promise of
184
+ `{code, reason, wasClean}`. `for await` yields every message not yet read, also ones
185
+ that arrived before the loop began (up to 1,024 messages or 16 MiB; older ones are
186
+ dropped). The module's own frame still has no network: StillMade
187
+ checks each URL (`wss://`, the standard port, a declared origin, valid subprotocol
188
+ names) and opens the connection in a separate relay frame that runs no Block code,
189
+ then relays the messages. A run keeps at most 4 connections, sends or receives at
190
+ most 1 MiB a message and 256 MiB in all, and every connection closes when the run
191
+ ends. The service sees an opaque `Origin` (`null`). StillMade adds no key to a live
192
+ connection: for a service that needs one, get a short-lived token with `net.fetch`
193
+ first and pass it as the service documents.
194
+
195
+ ## Threads (SharedArrayBuffer)
196
+
197
+ A module that runs its own Workers with shared memory (Emscripten pthreads,
198
+ threaded WebAssembly builds) declares it in `module.json`:
199
+
200
+ ```json
201
+ { "entry": "main.js", "threads": true }
202
+ ```
203
+
204
+ It then runs in a Worker that can share `SharedArrayBuffer` and
205
+ `WebAssembly.Memory({shared: true})` with the Workers it starts from `blob:` URLs
206
+ (module Workers and remote scripts stay unavailable, so bundle the thread code).
207
+ That Worker is not a secure context, so it has no WebCodecs, WebGPU,
208
+ `crypto.subtle` or `crypto.randomUUID`; a module without `threads` keeps those and
209
+ cannot share memory between Workers. Threads work in Chrome and Edge (version 137
210
+ or later) and in StillMade Desktop; in other browsers `self.crossOriginIsolated` is
211
+ false and `SharedArrayBuffer` is missing, so check it and fall back to one thread.
212
+ `files.write` and `storage.putFile` accept bytes in shared memory (they are copied).
143
213
 
144
214
  ## Test, pack and import
145
215
 
@@ -154,6 +224,29 @@ with the same host API and run limits (not a security sandbox; it is your own
154
224
  code). `files.read`, `files.write`, `media.transform`, actions and hosted calls need StillMade, or
155
225
  a host you pass to `createNodeModuleExecutor` in your own tests.
156
226
 
227
+ A fixture can record what the host answers, so a test of a Block that calls
228
+ hosted operations costs nothing and gives the same result every run:
229
+
230
+ ```json
231
+ {
232
+ "name": "Finds John 3:16",
233
+ "input": { "sermon": { "kind": "audio", "assetId": "a1", "versionId": "v1", "url": "https://example.com/sermon.wav" } },
234
+ "mocks": {
235
+ "hosted": {
236
+ "audio.transcribe": { "schemaVersion": 1, "text": "Turn to John 3:16", "words": [{ "text": "John", "start": 0.6, "end": 0.9 }] },
237
+ "text.generate": { "verses": [{ "reference": "John 3:16", "word": 0 }] }
238
+ }
239
+ },
240
+ "expected": { "verses": [{ "reference": "John 3:16", "start": 0.6 }] }
241
+ }
242
+ ```
243
+
244
+ `mocks` has the shape of `tests/mocks.json` (`hosted`, `transforms`, `actions`,
245
+ `files`, `storage`, `project`, `connections`, `network`); a list answers
246
+ successive calls in order. A fixture's own `mocks` take the place of
247
+ `tests/mocks.json` for that fixture. StillMade's own builder writes module
248
+ Blocks the same way and runs their tests before anyone sees them.
249
+
157
250
  Import the `.stillmade-module.json` in StillMade (Create, then Import).
158
251
  StillMade verifies every file digest, uploads each file to its
159
252
  content-addressed store, and saves the Block. Fixtures then run in your
package/PASTE_TO_LLM.md CHANGED
@@ -2,9 +2,9 @@
2
2
 
3
3
  <!-- builder-contract:start (generated by npm run build:sdk; do not edit) -->
4
4
 
5
- ## Block builder contract (stillmade.block-builder@1.7.1)
5
+ ## Block builder contract (stillmade.block-builder@1.10.0)
6
6
 
7
- Contract `stillmade.block-builder@1.7.1`. Humans, coding agents, imported source, GitHub repositories and generated code all adapt to the Block platform; anything that becomes a StillMade Block obeys this contract.
7
+ Contract `stillmade.block-builder@1.10.0`. Humans, coding agents, imported source, GitHub repositories and generated code all adapt to the Block platform; anything that becomes a StillMade Block obeys this contract.
8
8
 
9
9
  ### Source of truth
10
10
 
@@ -18,6 +18,10 @@ Before building, check what already exists (`find_existing_blocks` over MCP, or
18
18
 
19
19
  Blocks come in three sizes and the manifest decides which (packages/block-sdk/block-tier.js): an app is a workspace a person works in (Editor, Canvas, Shots), placed in a Type and never wired automatically; a job is one deliverable a person would name (the narration, the captioned clip), built from the hosted operations it needs (`permissions.capabilities`) and reviewed in one place, the unit Chat picks for a Step; an ingredient is one hosted operation with nothing of its own (a declarative capability Block), what a job is made of. When two ingredients would stand in a row to make one deliverable (writing the words, then voicing them), build or pick the job that does both and keep the review point inside it. A job declares a second output when another Block consumes the intermediate (the script beside the voice); split into two jobs only when the intermediate has a life of its own or the halves have different owners or payment.
20
20
 
21
+ ### Choose the runtime
22
+
23
+ Pick the runtime from the job. No hosted AI: a recipe or isolated `javascript`, both able to run unattended. One text, script or brief prompt returning text, script or JSON: a `capability` Block. Hosted operations on other values (a transcript, media, structured data), several hosted operations in a row, or a result shaped in code: one `module` Block, `module/main.js` calling `stillmade.hosted.run(operation, request)` for each operation in `permissions.capabilities` (MODULES.md), never one Block per operation; it runs in the person's browser while StillMade is open. Build a new Block in a folder with `stillmade-block create`, `test`, `ready` and `pack`, and import the file `pack` writes (a module Block writes `<id>-<version>.stillmade-module.json`) through Create, then Import Block. A `.stillmade-block` archive is only for adapting a GitHub repository.
24
+
21
25
  ### One job per Step
22
26
 
23
27
  A Step is one job: the work between two moments where a person or the outside world must see the result. Start a new Step only at one of three points; everything else is a part of the neighboring Step.
@@ -30,7 +34,7 @@ In a production the Steps are the workspaces a person opens: work feeding the ne
30
34
 
31
35
  A part that only runs in the desktop app cannot share a Step with cloud parts. Before adding a Step, name which of the three reasons makes it separate; if none applies, make it a part of a neighbor. A Step nobody would ever open must not exist on its own. Prefer an app task's own filter (a Gmail query such as `has:attachment filename:png`) to an AI check, and use AI decide only for judgement.
32
36
 
33
- Inside one job, each capability stays its own part: a capability Block performs exactly one hosted operation and a connector one app task, so every part keeps its own permission, approval and credit rules, and StillMade groups the parts of one job into one Step (`groups` in the Project Type). Never invent a combined runtime. When one Block can do the whole job with documented primitives (a JavaScript Block doing several local transforms, a workspace owning its whole document), build it as one Block.
37
+ Build one Block for a job one Block can do: a module Block calls every hosted operation the job needs (it transcribes a sermon, then finds the verses said in it), a JavaScript Block does several local transforms, a workspace owns its whole document. A Block being built never takes an ingredient's result only so the ingredient can be its own part or Step (Transcribe in front of a verse finder): it calls that operation itself. Parts in one Step (`groups` in the Project Type) are for jobs made of Blocks that already exist and app tasks, each keeping its own permission, approval and credit rules. Never invent a combined runtime.
34
38
 
35
39
  Examples: "every new email with an image becomes a generated image" is two Steps, "Gmail · New emails with images" (When, fetch with an image filter, only new) and "Image from email" (AI write, Generate image); listing When, Fetch, Only new, AI decide, Continue only if, AI write and Generate image as seven Steps splits one job into its parts. "A new Meta lead goes to HubSpot" is three Steps: "Meta Ads · New leads", "Qualify lead" and "HubSpot · Add contact".
36
40
 
@@ -68,7 +72,7 @@ Projects are multiplayer. A custom view subscribes through `StillMade.onShared(c
68
72
 
69
73
  ### External services
70
74
 
71
- Reach external services only through StillMade's hosted operations, the connections service (`connection.execute`), or, in a module Block, `stillmade.net.fetch` to origins declared in `permissions.network` (with the key each needs) and `stillmade.connections.call` to adapters or catalog apps in `permissions.connections`; never put keys in code, inputs or headers: StillMade adds each person's own key on its servers. The host owns the model and provider choice, BYOK keys, StillMade credits, spending consent, authorization, token refresh, rate limits, retries and provider errors; a Block never receives a secret and never names a provider unless it must. Treat provider unavailability, timeout, cancellation and partial failure as typed outcomes the Block reports, not crashes. Generated images, video, voices, music and sound effects come from hosted operations or the `media.generate` action, paid with the user's StillMade credits at StillMade's prices; for structured model output declare a `json` output with `capability.schema` instead of parsing prose. When a Block needs npm libraries, WebAssembly or heavy compute, use `runtime: "module"` and reach hosted operations through `stillmade.hosted.run` with the operations listed in `permissions.capabilities`. Decode, cut, join, re-encode, extract or normalise audio and video with `stillmade.media.transform` on the device; never ask for a server render. For a rich interactive interface (React or another framework, SVG, canvas, drag), use a frame view: `src/view.config.json` with `{"runtime":"frame"}` and `src/view.js` bundled as one IIFE, colored with StillMade's theme variables. For files and structured data use the `file`, `document`, `table`, `url`, `date` and `color` types and their generated controls and presentations, not JSON text.
75
+ Reach external services only through StillMade's hosted operations, the connections service (`connection.execute`), or, in a module Block, `stillmade.net.fetch` to origins declared in `permissions.network` (with the key each needs), `stillmade.net.connect` for a live WebSocket to one of them, and `stillmade.connections.call` to adapters or catalog apps in `permissions.connections`; never put keys in code, inputs or headers: StillMade adds each person's own key on its servers. The host owns the model and provider choice, BYOK keys, StillMade credits, spending consent, authorization, token refresh, rate limits, retries and provider errors; a Block never receives a secret and never names a provider unless it must. Treat provider unavailability, timeout, cancellation and partial failure as typed outcomes the Block reports, not crashes. Generated images, video, voices, music and sound effects come from hosted operations or the `media.generate` action, paid with the user's StillMade credits at StillMade's prices; for structured model output declare a `json` output with `capability.schema` instead of parsing prose. When a Block needs npm libraries, WebAssembly (threaded: `"threads": true` in module.json) or heavy compute, use `runtime: "module"` and reach hosted operations through `stillmade.hosted.run` with the operations listed in `permissions.capabilities`. Decode, cut, join, re-encode, extract or normalise audio and video with `stillmade.media.transform` on the device; never ask for a server render. For a rich interactive interface (React or another framework, SVG, canvas, drag), use a frame view: `src/view.config.json` with `{"runtime":"frame"}` and `src/view.js` bundled as one IIFE, colored with StillMade's theme variables. For files and structured data use the `file`, `document`, `table`, `url`, `date` and `color` types and their generated controls and presentations, not JSON text.
72
76
 
73
77
  ### Dependencies
74
78
 
@@ -76,7 +80,7 @@ A Block must declare everything it requires and must not assume anything is inst
76
80
 
77
81
  ### Security
78
82
 
79
- Treat Block code, external data, files, repositories, model output and network responses as untrusted. Request the minimum permissions and capabilities. Do not weaken the sandbox (QuickJS has no fetch, DOM, timers, modules or Node APIs). Validate and bound every value crossing a boundary. Never expose secrets through logs, outputs, project context, UI, error messages or serialized state.
83
+ Treat Block code, external data, files, repositories, model output and network responses as untrusted. Request the minimum permissions and capabilities; `stillmade-block permissions --fix` declares exactly what the source uses. Do not weaken the sandbox (QuickJS has no fetch, DOM, timers, modules or Node APIs). Validate and bound every value crossing a boundary. Never expose secrets through logs, outputs, project context, UI, error messages or serialized state.
80
84
 
81
85
  ### Execution lifecycle
82
86
 
@@ -178,7 +182,11 @@ For image generation, use runtime:"capability", operation:"image.generate" and p
178
182
  For image analysis, use runtime:"capability", operation:"image.describe" and permissions.capabilities:["image.describe"]. Declare one primary image input (or one image[] input of one to four images to compare together) and one primary text output. capability.image references the input; capability.instruction contains bounded package-owned analysis instructions; capability.maxTokens is 1–4096; capability.output names the text output. The host loads only owned media and controls provider, model, payment and approval. Tests use bounded RGBA pixels and nondeterministic text expectations, never invented live descriptions. Shipped example: {"manifest":{"schemaVersion":1,"sdkVersion":"0.1.0","id":"example.describe-image","version":"1.0.0","name":"Describe an image","description":"Turn one owned image into concise production-ready visual notes.","kind":"task","runtime":"capability","entry":"src/capability.json","license":"MIT","provenance":{"notice":"MIT License\n\nCopyright (c) 2026 StillMade SDK contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of this software and \nassociated documentation files (the \"Software\"), to deal in the Software without restriction, including \nwithout limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell \ncopies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the \nfollowing conditions:\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial \nportions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT \nLIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO \nEVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER \nIN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE \nUSE OR OTHER DEALINGS IN THE SOFTWARE.\n"},"inputs":{"image":{"type":"image","primary":true,"semantic":"source_image","required":true,"description":"Owned image to inspect for visible production details."}},"outputs":{"description":{"type":"text","primary":true,"role":"visual_description","semantic":"visual_description","description":"Concise production notes based only on visible image details."}},"permissions":{"project":[],"capabilities":["image.describe"],"network":[],"filesystem":[],"secrets":[]},"ui":[{"control":"asset","port":"image"}]},"capability":{"schemaVersion":1,"operation":"image.describe","image":{"$input":"image"},"instruction":"Describe only visibly supported subject matter, composition, lighting, color, texture, camera feel and visual style. Return concise production-ready plain text.","maxTokens":700,"output":"description"},"tests":[{"name":"Blue image","input":{"image":{"width":1,"height":1,"data":[20,80,200,255]}},"expectations":{"description":{"kind":"text","minLength":20,"maxLength":32768,"includes":["blue"]}}}]}.
179
183
  For AI-directed editing, use runtime:"capability", operation:"timeline.propose" and permissions.capabilities:["timeline.propose"]. Declare one current timeline input, up to three additional typed context inputs, and one timeline-edit output. Bind every input exactly once through capability.context; declare context.timeline.read and timeline.propose. StillMade controls the provider, strict JSON validation, stale-before checks, preview and Apply. Tests declare timeline-edit command bounds, never fabricated provider results. Shipped example: {"manifest":{"schemaVersion":1,"sdkVersion":"0.1.0","id":"example.timeline-polish","version":"1.0.0","name":"Timeline polish","description":"Turn a timeline and editing direction into a bounded proposal that remains reviewable before Apply.","kind":"task","runtime":"capability","entry":"src/capability.json","license":"MIT","provenance":{"notice":"MIT License\n\nCopyright (c) 2026 StillMade SDK contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of this software and \nassociated documentation files (the \"Software\"), to deal in the Software without restriction, including \nwithout limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell \ncopies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the \nfollowing conditions:\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial \nportions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT \nLIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO \nEVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER \nIN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE \nUSE OR OTHER DEALINGS IN THE SOFTWARE.\n"},"inputs":{"timeline":{"type":"timeline","context":"timeline","primary":true},"direction":{"type":"text","default":"Polish this cut conservatively."}},"outputs":{"edit":{"type":"timeline-edit","primary":true}},"permissions":{"project":["context.timeline.read","timeline.propose"],"capabilities":["timeline.propose"],"network":[],"filesystem":[],"secrets":[]},"ui":[{"control":"text","port":"direction"}]},"capability":{"schemaVersion":1,"operation":"timeline.propose","context":{"timeline":{"$input":"timeline"},"direction":{"$input":"direction"}},"instruction":"Use only supported timeline-edit commands, exact before snapshots, and the current activeTimelineId. Preserve media and locked items. Return strict JSON only.","maxTokens":2000,"output":"edit"},"tests":[{"name":"one restrained edit","input":{"timeline":{"schemaVersion":1,"activeTimelineId":"timeline-1","tracks":[{"id":"video"}],"clips":[{"id":"clip-1","trackId":"video","start":0,"duration":4}],"audioItems":[],"captions":[]},"direction":"Add a soft transition."},"expectations":{"edit":{"kind":"timeline-edit","minCommands":1,"maxCommands":10}}}]}.
180
184
  For video generation use runtime:"capability", operation:"video.generate", permissions.capabilities:["video.generate"], one primary text input and one primary video output. capability.prompt references the input and capability.output names the output. Prompt limit 32000. Tests declare MP4 duration, dimensions and byte bounds; never fabricate video URLs. The enabled host selects its supported model, duration, resolution, payment and approved quote; packages cannot choose provider settings or dispatch jobs. Shipped example: {"manifest":{"schemaVersion":1,"id":"example.generate-video","name":"Generate video","version":"1.0.0","sdkVersion":"0.1.0","description":"Turn a production prompt into a short video using an approved host provider.","kind":"task","runtime":"capability","entry":"src/capability.json","license":"MIT","provenance":{"notice":"MIT License\n\nCopyright (c) 2026 StillMade SDK contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of this software and \nassociated documentation files (the \"Software\"), to deal in the Software without restriction, including \nwithout limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell \ncopies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the \nfollowing conditions:\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial \nportions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT \nLIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO \nEVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER \nIN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE \nUSE OR OTHER DEALINGS IN THE SOFTWARE.\n"},"inputs":{"prompt":{"type":"text","primary":true,"required":true}},"outputs":{"video":{"type":"video","primary":true}},"permissions":{"project":[],"capabilities":["video.generate"],"network":[],"filesystem":[],"secrets":[]},"ui":[{"port":"prompt","control":"text"}]},"capability":{"schemaVersion":1,"operation":"video.generate","prompt":{"$input":"prompt"},"output":"video"},"tests":[{"name":"Produces a bounded MP4 sample","input":{"prompt":"A paper boat floating across a calm blue pond. Gentle camera movement."},"expectations":{"video":{"kind":"video","format":"mp4","minDuration":1,"maxDuration":60,"minBytes":32,"maxBytes":67108864,"minWidth":1,"maxWidth":3840,"minHeight":1,"maxHeight":3840}}}]}.
181
- Each capability package performs one supported operation. A job that needs several operations is built from parts: return each operation as its own part Block, and StillMade groups the parts of one job into one Step (One job per Step); never invent a combined runtime. Hosted image, speech and video source can be statically validated while authoring, but real fixture/sample generation requires the user's subsequent model/payment selection and quote confirmation. Do not claim those calls ran. Deterministic image transformations should remain sandboxed recipe or JavaScript code when supported; do not generate a new image as a substitute for processing an existing one.
185
+ A capability package performs exactly one supported operation. A job that needs several (transcribe a recording, then find what is said in it; write words, then voice them), or code between two operations, is one module Block that calls each with stillmade.hosted.run (Module Blocks); never split one job into part Blocks, and never invent a combined capability runtime. Hosted image, speech and video source can be statically validated while authoring, but real fixture/sample generation requires the user's subsequent model/payment selection and quote confirmation. Do not claim those calls ran. Deterministic image transformations should remain sandboxed recipe or JavaScript code when supported; do not generate a new image as a substitute for processing an existing one.
186
+
187
+ ### Module Blocks
188
+
189
+ A job that needs several hosted operations, or code around one (reading a transcript's words and times, shaping a list), is a module Block: {manifest,moduleSource,tests}, runtime:"module", moduleSource:{"main.js":"<the code>"}. StillMade names main.js by its digest and adds module; give no other files. main.js is plain JavaScript that imports nothing and uses no browser APIs (no fetch, TextEncoder, timers or WebCodecs): export default async function run(input, stillmade) returns the declared outputs. It calls each hosted operation with await stillmade.hosted.run(operation, request), listed in manifest.permissions.capabilities: text.generate, image.generate, video.generate, audio.speech, audio.music, audio.sfx, audio.transcribe, image.describe, media.analyze, web.fetch, web.research. Requests, and what each call returns (a test's mocks record exactly that value): text.generate {prompt, system, maxTokens} returns the text as a string, and with schema (a JSON schema object) returns that object, so ask for JSON with schema; audio.transcribe {audio: the input's saved reference, language:"auto"} returns {schemaVersion:1,text,words:[{text,start,end}]}; image.generate {prompt} and audio.speech {text} return a saved image or audio reference; image.describe {image, instruction} and web.fetch {url, instruction} return a string; media.analyze {media, question} returns {items:[...]}; web.research {topic} returns {summary,sources}. A call that fails throws: let it fail the run rather than catching it and returning an empty result. Media work on the person's device (cut, join or resize video, decode or extract audio, thumbnails) is await stillmade.media.transform(request); exporting is too: {operation:"timeline.render",kind:"video" or "audio",aspect:"9:16",clips:[{source,start,duration,from,transition:{type:"Fade"},look:"Filmic"}],audio:[{source,start,duration}],captions:[{text,start,end,words:[{text,start,end}],highlight:"#ffd400"}]} renders the Block's own timeline with the Editor's transitions, looks and word-by-word captions under the person's export plan, so never ask for a separate export, captions or color Block; stillmade.log and stillmade.progress(fraction, label) report. One press of the Step's button runs every call at the price shown beside it. A module Block runs in the person's browser while StillMade is open, never on a schedule or event. Tests are {name,input,expected,mocks}: mocks records what each call answers, as {hosted:{"audio.transcribe":{schemaVersion:1,text:"...",words:[...]},"text.generate":{...}},transforms:{...}} (a list answers successive calls in order), and expected is the exact output main.js returns for those answers. StillMade runs every test before anyone sees the Block, without spending credits. With the SDK instead, the same code is module/main.js in a Block folder and the same tests are tests/fixtures.json; stillmade-block pack writes the file to import (MODULES.md).
182
190
 
183
191
  ### ComfyUI workflows
184
192