@cyberart-io/engine 0.0.5 → 0.0.7

package/README.md

@@ -1,6 +1,6 @@
 # `@cyberart-io/engine`
 
-An engine for building animated, interactive, generative programs for the web. You write a **cart** — a small object with `getDefaultState` (initial state), `update` (advance that state each tick), and `render` (draw it). That is a state machine in a loop that paints to the screen. The engine mounts it into a DOM element you own, runs the loop, and gives you pause, snapshot, events, and save/load.
+An engine for building animated, interactive, generative programs for the web. You write a **cart** — a small object with `getDefaultState` (initial state), `update` (advance that state each tick), and optional `render` (draw it). A render cart is a state machine in a loop that paints to the screen. A calculation cart runs the same state machine without a canvas, and talks to a sibling that does paint. The engine mounts it into a DOM element you own, runs the loop, and gives you pause, snapshot, events, and save/load.
 
 ## License
 
@@ -94,11 +94,17 @@ Full API for the event router, deterministic replay, and CI harness (so agents c
 - [Asset resolver](docs/asset-resolver.md) — host-pluggable images/audio/fonts/spritesheets, cache, preload, fallbacks
 - [Presentation cue](docs/presentation-cue.md) — deterministic `createPresentationTimeline`, duplicate policy, reduced-motion, lifecycle events
 - [Capability manifest](docs/capability-manifest.md) — versioned JSON for runtime features, phases, managers, assets, events, permissions, integrations
+- [Executable modules](docs/executable-modules.md) — trusted versioned factories, host allowlists, isolation, per-module failures
 - [Normalized geometry](docs/normalized-geometry.md) — coordinate spaces, contain/cover/crop layout, landmarks, hit regions, debug overlay
 - [Runtime group](docs/runtime-group.md) — `createRuntimeGroup`, shared router attach, lockstep clock; `createHeadlessMultiCartHarness` from `@cyberart-io/engine/headless`
+- [Calculation carts](docs/calculation-carts.md) — `kind: 'calculation'`, update + events without paint; sibling render carts still paint
 - [Compositor](docs/compositor.md) — `createCompositor`, transparent stacking, `screen` blend, `writeComposedFrame`
+- [Visual layers](docs/visual-layers.md) — `createVisualLayerController`, versioned overlay `v1`/`v2`, deterministic swaps, `hostState` JSON
 - [Browser harness](docs/browser-harness.md) — `createBrowserHarness`, DOM clicks, viewport/DPR, composed screenshots
 - [Replay inspector](docs/replay-inspector.md) — `createReplayInspector`, causation trees, redacted export/import, headless replay
+- [MIDI](docs/midi.md) — host-owned `MidiManager`, note/CC/pitch in and out, `inject` / fake port, structured `requestAccess`
+- [Snapshots](docs/snapshots.md) — versioned envelope, `defineSnapshot` / `applySnapshotMigrations`, host-owned `hostState`
+- [Audio](docs/audio.md) — `createAudioBroker`, `createAudioCueTimeline`, headless cue traces (PCM is not deterministic)
 
 ## Write a cart
 
@@ -108,18 +114,20 @@ A cart is an `AnimationCart`. Required:
 |---|---|
 | `getDefaultState` | Build the initial `state`. Use `R` (`Random`) for anything that should follow the seed. |
 | `update` | Advance `state` each tick. Return the next state. |
-| `render` | Draw into the 2D context (and optional `ImageData`). |
 | `metadata.id` / `name` / `frameRate` | Identity and loop rate. |
 
+`render` draws into the 2D context (and optional `ImageData`). Omit it on calculation carts (`createRuntime({ kind: 'calculation' })` or a group participant with `kind: 'calculation'`). The runtime does not call `render` and does not create a canvas for that kind. Render carts (the default) still paint as before.
+
 Useful optionals:
 
 | Piece | Role |
 |---|---|
+| `render` | Draw into the 2D context. Omit on calculation carts; the runtime will not call a no-op either. |
 | `teardown` | Dispose long-lived resources (Tone nodes, listeners) when the cart unloads. |
 | `metadata.audio` | `'tone'` if the piece needs Web Audio. Omit for a silent cart. |
 | `metadata.generative` | `true` when output is a function of the token hash. Saves then only load on the same seed. |
 
-Callbacks also receive `DimensionContext` (`width` / `height` and derived size fields), `KeyboardManager`, `PointerManager`, and an optional `HostChannel`. You do not have to use them. Types live on the package: `AnimationCart`, `Random`, `DimensionContext`, `KeyboardManager`, `PointerManager`, `HostChannel`.
+Callbacks also receive `DimensionContext` (`width` / `height` and derived size fields), `KeyboardManager`, `PointerManager`, and an optional `HostChannel`. You do not have to use them. Types live on the package: `AnimationCart`, `CartKind`, `Random`, `DimensionContext`, `KeyboardManager`, `PointerManager`, `HostChannel`.
 
 `Random` is seed-stable: `R.dec(min, max)`, `R.int(min, max)`, `R.bool()`, `R.choose(list)`.
 
@@ -153,6 +161,9 @@ const cart = runtime.mount(artProject, {
 | `audio` | none | Libraries to unlock if `unlockAudio()` runs before `mount`. After mount, the cart’s `metadata.audio` wins. |
 | `deterministic` | off | Host-controlled clock, `step`/`advance`, and scripted input/assets. Leave unset for live kaleidoscope / Art Blocks. |
 | `assets` | off | Host `AssetResolver` plus engine cache/preload. Carts keep logical refs. Leave unset when the piece has no media. |
+| `audioBroker` | off | Shared `createAudioBroker` instance. Optional. Omit when the cart only uses `metadata.audio: 'tone'`. |
+| `audioParticipantId` | none | Group participant id to authorize / teardown on this runtime. |
+| `kind` | `'render'` | `'calculation'` skips canvas construction and paint. `getDefaultState`, `update`, and host-channel events still run. |
 
 `CartHandle` (what `mount` returns):
 
@@ -165,8 +176,10 @@ const cart = runtime.mount(artProject, {
 | `step(frames)` / `advance(ms)` | Deterministic ticks only. Throw if `deterministic` was not set. |
 | `schedule(action)` | Queue a pointer/key/host-event/asset for a future frame. |
 | `getClock()` / `getRandomState()` / `getReplayMetadata()` | Replay inspection. |
-| `exportState()` / `exportStateJSON()` | Pause-safe serializable bundle. |
-| `importState(bundle)` | Restore a bundle (or JSON string). |
+| `exportState()` / `exportStateJSON()` | Pause-safe serializable `CartStateBundle` (schema 1 blob). |
+| `importState(bundle)` | Restore a bundle, JSON string, or snapshot envelope (migrates then loads `engineState`). |
+| `exportSnapshot()` / `exportSnapshotJSON()` | Schema 2 envelope wrapping the bundle plus clock, rng, optional `hostState`. |
+| `importSnapshot(envelope)` | Migrate to the current schema and restore `engineState`. Hosts reapply `hostState`. |
 | `destroy()` | Unload this cart. Idempotent. |
 | `needsAudio` | True when the cart declared audio libraries. Use this to show a click overlay. |
 | `paused` / `isPrepared` / `isLoopRunning` | Loop flags. |
@@ -174,9 +187,9 @@ const cart = runtime.mount(artProject, {
 | `tokenData` | Live hash and token id. |
 | `getCartState()` | Live object for debug UI. Not JSON-safe — use export/import for saves. |
 
-`runtime.destroy()` tears down the runtime. `runtime.unlockAudio()` is for an early click before `mount`. `runtime.onError` receives frame errors (`phase`, `consecutive`, `stopped`).
+`runtime.destroy()` tears down the runtime. `runtime.unlockAudio()` is for an early click before `mount`. `runtime.onError` receives frame errors (`phase`, `consecutive`, `stopped`). `runtime.kind` is `'render'` (default) or `'calculation'`.
 
-One cart per runtime. A second `mount` unloads the first. Several pieces on a page means several `createRuntime()` calls, or one `createRuntimeGroup()` that attaches each mailbox to a shared router and locksteps a deterministic clock.
+One cart per runtime. A second `mount` unloads the first. Several pieces on a page means several `createRuntime()` calls, or one `createRuntimeGroup()` that attaches each mailbox to a shared router and locksteps a deterministic clock. Calculation participants (`kind: 'calculation'`) share that group without a canvas; see [calculation carts](docs/calculation-carts.md).
 
 ## Audio
 
@@ -199,6 +212,14 @@ Install the peer: `npm i tone@^14.8.15`. Call `cart.start()` **inside a click or
 
 If a click can happen before `mount` finishes, pass the same `audio` into `createRuntime` and/or call `runtime.unlockAudio()` from that click. After mount, the cart’s metadata wins.
 
+Several carts on one page should share **one** `createAudioBroker` (pass it to `createRuntime` / `createRuntimeGroup`). One `unlock()` covers authorized participants. Channel gain, mute, duck, and priority live on the broker; cue timing is `createAudioCueTimeline` / `scheduleAudioCue` (presentation-frame clock). Headless CI records cue events with `createHeadlessAudioAdapter` from `@cyberart-io/engine/headless` — the event trace is deterministic, PCM is not. Full API: [audio](docs/audio.md).
+
+## MIDI
+
+Host-owned `MidiManager` — carts and hosts construct it. The runtime does not pass it into `getDefaultState`. Subscribe to note / CC / pitch, `inject` for tests without hardware, and `send` note-on/off, CC, or raw bytes through an injectable port. `requestAccess()` wraps `navigator.requestMIDIAccess` when present; a missing API or denied permission is a structured result, so silent carts keep running. Headless/jsdom has no Web MIDI; inject and send still work via a test port.
+
+Full API and the reproduce command: [MIDI](docs/midi.md). Capability manifests already list `midi` as an integration.
+
 ## Seed
 
 Pass `seed` when you want a deterministic output (for example from `?hash=`):
@@ -358,6 +379,40 @@ validateCapabilityManifest(defined.manifest, {
 
 Fields, host allowlists, and diagnostic codes: [capability manifest](docs/capability-manifest.md).
 
+## Executable modules
+
+Trusted factories keyed by exact `id` + `version`. The host allowlists which refs may load. There is no `eval` / `new Function` path for cart-supplied strings.
+
+```ts
+import { createExecutableModuleHost } from '@cyberart-io/engine';
+
+const host = createExecutableModuleHost({
+  allowlist: [
+    { id: 'overlay-fx', version: '1.0.0' },
+    { id: 'host.module', version: '1.0.0' },
+  ],
+  limits: { maxInvokeMs: 16, maxInvokesPerTurn: 4 },
+  modules: [
+    {
+      id: 'overlay-fx',
+      version: '1.0.0',
+      create: (capabilities) => ({
+        invoke: (input, { signal }) => {
+          if (signal.aborted) return;
+          return { input, seed: capabilities.seed };
+        },
+      }),
+    },
+  ],
+});
+
+const allowed = await host.invoke({ id: 'overlay-fx', version: '1.0.0' });
+await host.invoke({ id: 'overlay-fx', version: '2.0.0' }); // version-mismatch, not invoked
+host.destroy();
+```
+
+Unknown ids, wrong versions, and refs missing from the allowlist fail closed. A throwing module returns `{ ok: false, error }` and the next allowlisted invoke still runs. Isolation is in-process (trusted factories), not a worker or iframe. Carts may declare required refs as optional `modules.refs` on a [capability manifest](docs/capability-manifest.md). Full API and the reproduce command: [executable modules](docs/executable-modules.md).
+
 ## Runtime group
 
 Several production carts, one router, one lockstep clock. Use this instead of intercepting each `onEvent` and republishing by hand.
@@ -368,17 +423,57 @@ import { createRuntimeGroup } from '@cyberart-io/engine';
 const group = createRuntimeGroup({
   origin: 0,
   participants: [
-    { id: 'effects', cart: effectsCart, emit: ['ambience.intent.*'], subscribe: ['host.state.*'] },
-    { id: 'ambience', cart: ambienceCart, subscribe: ['ambience.intent.*'] },
+    { id: 'world', cart: worldCart, kind: 'calculation', emit: ['host.intent.*'], subscribe: ['host.state.*'] },
+    { id: 'overlay', cart: overlayCart, subscribe: ['host.intent.*'] },
   ],
 });
-group.publish({ type: 'host.state.accepted', kind: 'state', payload: { id: 'north' } });
+group.publish({ type: 'host.state.scene-changed', kind: 'state', payload: { sceneId: 'alpha' } });
 await group.step(2);
 const { trace } = await group.inspect();
 group.destroy();
 ```
 
-Headless / CI: `createHeadlessMultiCartHarness` from `@cyberart-io/engine/headless` is the same handle after `installHeadlessCanvas()`. Full options: [runtime group](docs/runtime-group.md). Causation trees, redaction, and tape replay: [replay inspector](docs/replay-inspector.md).
+Headless / CI: `createHeadlessMultiCartHarness` from `@cyberart-io/engine/headless` is the same handle after `installHeadlessCanvas()`. Full options: [runtime group](docs/runtime-group.md). Calculation carts (no canvas / paint): [calculation carts](docs/calculation-carts.md). Causation trees, redaction, and tape replay: [replay inspector](docs/replay-inspector.md).
+
+## Visual layers
+
+Durable overlay / mask / sprite versions (`v1`, `v2`) with deterministic show, hide, replace, and crossfade. An accepted `host.state.accepted` (`sceneId: 'alpha'`) runs a presentation cue; the compositor reveals a **preloaded** alternate source in one `setLayer`. Failed assets keep the prior valid frame and emit `visual.layer.failed`. Persist `controller.snapshot()` in envelope `hostState` — do not put pixels in the envelope.
+
+```ts
+import {
+  ASSET_READY_EVENT,
+  HOST_STATE_ACCEPTED_EVENT,
+  createVisualLayerController,
+} from '@cyberart-io/engine';
+
+const layers = createVisualLayerController({
+  compositor,
+  sceneId: 'alpha',
+  layers: [
+    {
+      id: 'overlay',
+      kind: 'layer',
+      initialVersion: 'v1',
+      versions: [
+        { id: 'v1', assetId: 'overlay.v1' },
+        { id: 'v2', assetId: 'overlay.v2' },
+      ],
+    },
+  ],
+  onAccepted: [{ layerId: 'overlay', toVersion: 'v2', kind: 'replace', durationFrames: 2 }],
+});
+layers.registerSource('overlay.v1', imageV1);
+layers.registerSource('overlay.v2', imageV2);
+layers.handleHostEvent({ type: ASSET_READY_EVENT, kind: 'state', payload: { id: 'overlay.v2' } });
+layers.handleHostEvent({
+  type: HOST_STATE_ACCEPTED_EVENT,
+  kind: 'state',
+  payload: { sceneId: 'alpha' },
+});
+layers.step(2);
+```
+
+Headless capture: `captureVisualLayers(layers)` from `@cyberart-io/engine` or `@cyberart-io/engine/headless`. Full API: [visual layers](docs/visual-layers.md).
 
 ## Normalized geometry
 
@@ -476,14 +571,22 @@ The same declarations produce fixture URLs locally and CDN URLs (plus a typed CO
 
 ## Save and load
 
-Live `state` is not JSON-safe (audio nodes, managers, typed arrays, possible cycles). `snapshot()` is PNG + seed. Use export/import when you want the simulation itself:
+Live `state` is not JSON-safe (audio nodes, managers, typed arrays, possible cycles). `snapshot()` is PNG + seed. Use export/import when you want the simulation itself.
+
+`exportState` still returns the engine-owned `CartStateBundle` (`CART_STATE_BUNDLE_VERSION = 1`) so existing carts keep working. The versioned envelope (`SNAPSHOT_SCHEMA_VERSION = 2`) wraps that blob in `engineState` and keeps host-owned JSON in `hostState` (or a `hostStateRef`). Hosts persist the envelope; Cyberart is not the database. Full field list, migration registry, and diagnostic codes: [versioned snapshots](docs/snapshots.md).
 
 ```ts
 const bundle = await cart.exportState();
 const json = await cart.exportStateJSON();
+const envelope = await cart.exportSnapshot({
+  cartVersion: '3',
+  hostState: { sceneId: 'alpha' },
+});
 
 await cart.importState(bundle);
 await cart.importState(json);
+await cart.importState(envelope);
+await cart.importSnapshot(envelope);
 ```
 
 Both pause, wait until any in-flight `update` finishes, then restore the previous pause flag. Import tears down, runs `getDefaultState` again (fresh managers, empty audio graph), overlays the save onto that scaffold unless `getDefaultState` already returned the revived `customState`, and restores `framesElapsed`. Works on a prepared cart before `start()`. Throws if nothing is prepared, or if the bundle is incompatible (`IncompatibleCartStateError`).
@@ -536,7 +639,7 @@ The engine prefers, in order:
 2. The container’s first `<canvas>`
 3. A canvas it creates (no `id="canvas"`)
 
-A canvas the host adopted is left in place on destroy; a canvas the engine created is removed. Both `cart.destroy()` and `runtime.destroy()` are idempotent.
+A canvas the host adopted is left in place on destroy; a canvas the engine created is removed. Both `cart.destroy()` and `runtime.destroy()` are idempotent. `createRuntime({ kind: 'calculation' })` never creates or adopts a canvas (`cart.canvas` is `undefined`).
 
 ## Limits
 
@@ -547,7 +650,7 @@ A canvas the host adopted is left in place on destroy; a canvas the engine creat
 
 ## Publishing this package (maintainers)
 
-Not part of writing a cart. Engine source lives in `packages/engine/src/` (not mixed into the site). The npm tarball is built from `packages/engine/src/index.ts` and `packages/engine/src/headless.ts` and contains minified `dist/index.js` + `dist/headless.js`, rolled-up `.d.ts` for both, `LICENSE`, `README.md`, `docs/` (including compositor, browser harness, and replay inspector), and `package.json`. Site code that still imports `src/ui/lib/...` hits thin re-export shims so those paths keep working.
+Not part of writing a cart. Engine source lives in `packages/engine/src/` (not mixed into the site). The npm tarball is built from `packages/engine/src/index.ts` and `packages/engine/src/headless.ts` and contains minified `dist/index.js` + `dist/headless.js`, rolled-up `.d.ts` for both, `LICENSE`, `README.md`, `docs/` (including compositor, visual layers, browser harness, replay inspector, MIDI, executable modules, snapshots, audio, and calculation carts), and `package.json`. Site code that still imports `src/ui/lib/...` hits thin re-export shims so those paths keep working.
 
 ```bash
 pnpm run pack:engine