@cyberart-io/engine 0.0.5 → 0.0.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +62 -4
- package/dist/headless.d.ts +283 -2
- package/dist/headless.js +1 -1
- package/dist/index.d.ts +675 -3
- package/dist/index.js +1 -1
- package/docs/audio.md +152 -0
- package/docs/executable-modules.md +112 -0
- package/docs/midi.md +128 -0
- package/docs/snapshots.md +102 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -94,11 +94,15 @@ Full API for the event router, deterministic replay, and CI harness (so agents c
|
|
|
94
94
|
- [Asset resolver](docs/asset-resolver.md) — host-pluggable images/audio/fonts/spritesheets, cache, preload, fallbacks
|
|
95
95
|
- [Presentation cue](docs/presentation-cue.md) — deterministic `createPresentationTimeline`, duplicate policy, reduced-motion, lifecycle events
|
|
96
96
|
- [Capability manifest](docs/capability-manifest.md) — versioned JSON for runtime features, phases, managers, assets, events, permissions, integrations
|
|
97
|
+
- [Executable modules](docs/executable-modules.md) — trusted versioned factories, host allowlists, isolation, per-module failures
|
|
97
98
|
- [Normalized geometry](docs/normalized-geometry.md) — coordinate spaces, contain/cover/crop layout, landmarks, hit regions, debug overlay
|
|
98
99
|
- [Runtime group](docs/runtime-group.md) — `createRuntimeGroup`, shared router attach, lockstep clock; `createHeadlessMultiCartHarness` from `@cyberart-io/engine/headless`
|
|
99
100
|
- [Compositor](docs/compositor.md) — `createCompositor`, transparent stacking, `screen` blend, `writeComposedFrame`
|
|
100
101
|
- [Browser harness](docs/browser-harness.md) — `createBrowserHarness`, DOM clicks, viewport/DPR, composed screenshots
|
|
101
102
|
- [Replay inspector](docs/replay-inspector.md) — `createReplayInspector`, causation trees, redacted export/import, headless replay
|
|
103
|
+
- [MIDI](docs/midi.md) — host-owned `MidiManager`, note/CC/pitch in and out, `inject` / fake port, structured `requestAccess`
|
|
104
|
+
- [Snapshots](docs/snapshots.md) — versioned envelope, `defineSnapshot` / `applySnapshotMigrations`, host-owned `hostState`
|
|
105
|
+
- [Audio](docs/audio.md) — `createAudioBroker`, `createAudioCueTimeline`, headless cue traces (PCM is not deterministic)
|
|
102
106
|
|
|
103
107
|
## Write a cart
|
|
104
108
|
|
|
@@ -153,6 +157,8 @@ const cart = runtime.mount(artProject, {
|
|
|
153
157
|
| `audio` | none | Libraries to unlock if `unlockAudio()` runs before `mount`. After mount, the cart’s `metadata.audio` wins. |
|
|
154
158
|
| `deterministic` | off | Host-controlled clock, `step`/`advance`, and scripted input/assets. Leave unset for live kaleidoscope / Art Blocks. |
|
|
155
159
|
| `assets` | off | Host `AssetResolver` plus engine cache/preload. Carts keep logical refs. Leave unset when the piece has no media. |
|
|
160
|
+
| `audioBroker` | off | Shared `createAudioBroker` instance. Optional. Omit when the cart only uses `metadata.audio: 'tone'`. |
|
|
161
|
+
| `audioParticipantId` | none | Group participant id to authorize / teardown on this runtime. |
|
|
156
162
|
|
|
157
163
|
`CartHandle` (what `mount` returns):
|
|
158
164
|
|
|
@@ -165,8 +171,10 @@ const cart = runtime.mount(artProject, {
|
|
|
165
171
|
| `step(frames)` / `advance(ms)` | Deterministic ticks only. Throw if `deterministic` was not set. |
|
|
166
172
|
| `schedule(action)` | Queue a pointer/key/host-event/asset for a future frame. |
|
|
167
173
|
| `getClock()` / `getRandomState()` / `getReplayMetadata()` | Replay inspection. |
|
|
168
|
-
| `exportState()` / `exportStateJSON()` | Pause-safe serializable
|
|
169
|
-
| `importState(bundle)` | Restore a bundle
|
|
174
|
+
| `exportState()` / `exportStateJSON()` | Pause-safe serializable `CartStateBundle` (schema 1 blob). |
|
|
175
|
+
| `importState(bundle)` | Restore a bundle, JSON string, or snapshot envelope (migrates then loads `engineState`). |
|
|
176
|
+
| `exportSnapshot()` / `exportSnapshotJSON()` | Schema 2 envelope wrapping the bundle plus clock, rng, optional `hostState`. |
|
|
177
|
+
| `importSnapshot(envelope)` | Migrate to the current schema and restore `engineState`. Hosts reapply `hostState`. |
|
|
170
178
|
| `destroy()` | Unload this cart. Idempotent. |
|
|
171
179
|
| `needsAudio` | True when the cart declared audio libraries. Use this to show a click overlay. |
|
|
172
180
|
| `paused` / `isPrepared` / `isLoopRunning` | Loop flags. |
|
|
@@ -199,6 +207,14 @@ Install the peer: `npm i tone@^14.8.15`. Call `cart.start()` **inside a click or
|
|
|
199
207
|
|
|
200
208
|
If a click can happen before `mount` finishes, pass the same `audio` into `createRuntime` and/or call `runtime.unlockAudio()` from that click. After mount, the cart’s metadata wins.
|
|
201
209
|
|
|
210
|
+
Several carts on one page should share **one** `createAudioBroker` (pass it to `createRuntime` / `createRuntimeGroup`). One `unlock()` covers authorized participants. Channel gain, mute, duck, and priority live on the broker; cue timing is `createAudioCueTimeline` / `scheduleAudioCue` (presentation-frame clock). Headless CI records cue events with `createHeadlessAudioAdapter` from `@cyberart-io/engine/headless` — the event trace is deterministic, PCM is not. Full API: [audio](docs/audio.md).
|
|
211
|
+
|
|
212
|
+
## MIDI
|
|
213
|
+
|
|
214
|
+
Host-owned `MidiManager` — carts and hosts construct it. The runtime does not pass it into `getDefaultState`. Subscribe to note / CC / pitch, `inject` for tests without hardware, and `send` note-on/off, CC, or raw bytes through an injectable port. `requestAccess()` wraps `navigator.requestMIDIAccess` when present; a missing API or denied permission is a structured result, so silent carts keep running. Headless/jsdom has no Web MIDI; inject and send still work via a test port.
|
|
215
|
+
|
|
216
|
+
Full API and the reproduce command: [MIDI](docs/midi.md). Capability manifests already list `midi` as an integration.
|
|
217
|
+
|
|
202
218
|
## Seed
|
|
203
219
|
|
|
204
220
|
Pass `seed` when you want a deterministic output (for example from `?hash=`):
|
|
@@ -358,6 +374,40 @@ validateCapabilityManifest(defined.manifest, {
|
|
|
358
374
|
|
|
359
375
|
Fields, host allowlists, and diagnostic codes: [capability manifest](docs/capability-manifest.md).
|
|
360
376
|
|
|
377
|
+
## Executable modules
|
|
378
|
+
|
|
379
|
+
Trusted factories keyed by exact `id` + `version`. The host allowlists which refs may load. There is no `eval` / `new Function` path for cart-supplied strings.
|
|
380
|
+
|
|
381
|
+
```ts
|
|
382
|
+
import { createExecutableModuleHost } from '@cyberart-io/engine';
|
|
383
|
+
|
|
384
|
+
const host = createExecutableModuleHost({
|
|
385
|
+
allowlist: [
|
|
386
|
+
{ id: 'overlay-fx', version: '1.0.0' },
|
|
387
|
+
{ id: 'host.module', version: '1.0.0' },
|
|
388
|
+
],
|
|
389
|
+
limits: { maxInvokeMs: 16, maxInvokesPerTurn: 4 },
|
|
390
|
+
modules: [
|
|
391
|
+
{
|
|
392
|
+
id: 'overlay-fx',
|
|
393
|
+
version: '1.0.0',
|
|
394
|
+
create: (capabilities) => ({
|
|
395
|
+
invoke: (input, { signal }) => {
|
|
396
|
+
if (signal.aborted) return;
|
|
397
|
+
return { input, seed: capabilities.seed };
|
|
398
|
+
},
|
|
399
|
+
}),
|
|
400
|
+
},
|
|
401
|
+
],
|
|
402
|
+
});
|
|
403
|
+
|
|
404
|
+
const allowed = await host.invoke({ id: 'overlay-fx', version: '1.0.0' });
|
|
405
|
+
await host.invoke({ id: 'overlay-fx', version: '2.0.0' }); // version-mismatch, not invoked
|
|
406
|
+
host.destroy();
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
Unknown ids, wrong versions, and refs missing from the allowlist fail closed. A throwing module returns `{ ok: false, error }` and the next allowlisted invoke still runs. Isolation is in-process (trusted factories), not a worker or iframe. Carts may declare required refs as optional `modules.refs` on a [capability manifest](docs/capability-manifest.md). Full API and the reproduce command: [executable modules](docs/executable-modules.md).
|
|
410
|
+
|
|
361
411
|
## Runtime group
|
|
362
412
|
|
|
363
413
|
Several production carts, one router, one lockstep clock. Use this instead of intercepting each `onEvent` and republishing by hand.
|
|
@@ -476,14 +526,22 @@ The same declarations produce fixture URLs locally and CDN URLs (plus a typed CO
|
|
|
476
526
|
|
|
477
527
|
## Save and load
|
|
478
528
|
|
|
479
|
-
Live `state` is not JSON-safe (audio nodes, managers, typed arrays, possible cycles). `snapshot()` is PNG + seed. Use export/import when you want the simulation itself
|
|
529
|
+
Live `state` is not JSON-safe (audio nodes, managers, typed arrays, possible cycles). `snapshot()` is PNG + seed. Use export/import when you want the simulation itself.
|
|
530
|
+
|
|
531
|
+
`exportState` still returns the engine-owned `CartStateBundle` (`CART_STATE_BUNDLE_VERSION = 1`) so existing carts keep working. The versioned envelope (`SNAPSHOT_SCHEMA_VERSION = 2`) wraps that blob in `engineState` and keeps host-owned JSON in `hostState` (or a `hostStateRef`). Hosts persist the envelope; Cyberart is not the database. Full field list, migration registry, and diagnostic codes: [versioned snapshots](docs/snapshots.md).
|
|
480
532
|
|
|
481
533
|
```ts
|
|
482
534
|
const bundle = await cart.exportState();
|
|
483
535
|
const json = await cart.exportStateJSON();
|
|
536
|
+
const envelope = await cart.exportSnapshot({
|
|
537
|
+
cartVersion: '3',
|
|
538
|
+
hostState: { sceneId: 'alpha' },
|
|
539
|
+
});
|
|
484
540
|
|
|
485
541
|
await cart.importState(bundle);
|
|
486
542
|
await cart.importState(json);
|
|
543
|
+
await cart.importState(envelope);
|
|
544
|
+
await cart.importSnapshot(envelope);
|
|
487
545
|
```
|
|
488
546
|
|
|
489
547
|
Both pause, wait until any in-flight `update` finishes, then restore the previous pause flag. Import tears down, runs `getDefaultState` again (fresh managers, empty audio graph), overlays the save onto that scaffold unless `getDefaultState` already returned the revived `customState`, and restores `framesElapsed`. Works on a prepared cart before `start()`. Throws if nothing is prepared, or if the bundle is incompatible (`IncompatibleCartStateError`).
|
|
@@ -547,7 +605,7 @@ A canvas the host adopted is left in place on destroy; a canvas the engine creat
|
|
|
547
605
|
|
|
548
606
|
## Publishing this package (maintainers)
|
|
549
607
|
|
|
550
|
-
Not part of writing a cart. Engine source lives in `packages/engine/src/` (not mixed into the site). The npm tarball is built from `packages/engine/src/index.ts` and `packages/engine/src/headless.ts` and contains minified `dist/index.js` + `dist/headless.js`, rolled-up `.d.ts` for both, `LICENSE`, `README.md`, `docs/` (including compositor, browser harness,
|
|
608
|
+
Not part of writing a cart. Engine source lives in `packages/engine/src/` (not mixed into the site). The npm tarball is built from `packages/engine/src/index.ts` and `packages/engine/src/headless.ts` and contains minified `dist/index.js` + `dist/headless.js`, rolled-up `.d.ts` for both, `LICENSE`, `README.md`, `docs/` (including compositor, browser harness, replay inspector, MIDI, executable modules, snapshots, and audio), and `package.json`. Site code that still imports `src/ui/lib/...` hits thin re-export shims so those paths keep working.
|
|
551
609
|
|
|
552
610
|
```bash
|
|
553
611
|
pnpm run pack:engine
|
package/dist/headless.d.ts
CHANGED
|
@@ -328,6 +328,76 @@ type CartStateBundle = {
|
|
|
328
328
|
state: unknown;
|
|
329
329
|
};
|
|
330
330
|
|
|
331
|
+
/**
|
|
332
|
+
* Copyright (c) 2026 Aaron Boyarsky
|
|
333
|
+
* SPDX-License-Identifier: LicenseRef-CyberArt-Engine
|
|
334
|
+
* See packages/engine/LICENSE
|
|
335
|
+
*
|
|
336
|
+
* Versioned snapshot envelope. Schema 1 is the previous engine-owned
|
|
337
|
+
* `CartStateBundle` (`CART_STATE_BUNDLE_VERSION = 1`). Schema 2 wraps that
|
|
338
|
+
* blob in `engineState` and separates host-owned payload. Hosts persist
|
|
339
|
+
* the JSON; this module is not a database.
|
|
340
|
+
*/
|
|
341
|
+
|
|
342
|
+
/**
|
|
343
|
+
* Current envelope schema. Independent of `CART_STATE_BUNDLE_VERSION` (still
|
|
344
|
+
* 1 inside `engineState`) and of the npm package version.
|
|
345
|
+
*/
|
|
346
|
+
declare const SNAPSHOT_SCHEMA_VERSION: 2;
|
|
347
|
+
type SnapshotCartRef = {
|
|
348
|
+
id: string;
|
|
349
|
+
version: string;
|
|
350
|
+
generative?: boolean;
|
|
351
|
+
};
|
|
352
|
+
type SnapshotModuleRef = {
|
|
353
|
+
id: string;
|
|
354
|
+
version: string;
|
|
355
|
+
};
|
|
356
|
+
type SnapshotClock = {
|
|
357
|
+
framesElapsed: number;
|
|
358
|
+
elapsedSinceStart?: number;
|
|
359
|
+
now?: number;
|
|
360
|
+
frameRate?: number;
|
|
361
|
+
};
|
|
362
|
+
type SnapshotAssetRef = {
|
|
363
|
+
id: string;
|
|
364
|
+
version?: string;
|
|
365
|
+
ref?: string;
|
|
366
|
+
};
|
|
367
|
+
type SnapshotIntegrity = {
|
|
368
|
+
alg: string;
|
|
369
|
+
hash: string;
|
|
370
|
+
};
|
|
371
|
+
type SnapshotProvenance = {
|
|
372
|
+
source?: string;
|
|
373
|
+
integrity?: SnapshotIntegrity;
|
|
374
|
+
};
|
|
375
|
+
type SnapshotEnvelope = {
|
|
376
|
+
schemaVersion: typeof SNAPSHOT_SCHEMA_VERSION;
|
|
377
|
+
runtimeVersion: string;
|
|
378
|
+
cart: SnapshotCartRef;
|
|
379
|
+
seed: string;
|
|
380
|
+
clock: SnapshotClock;
|
|
381
|
+
engineState: CartStateBundle;
|
|
382
|
+
createdAt: string;
|
|
383
|
+
modules?: SnapshotModuleRef[];
|
|
384
|
+
rng?: RandomState;
|
|
385
|
+
hostState?: unknown;
|
|
386
|
+
hostStateRef?: string;
|
|
387
|
+
assets?: SnapshotAssetRef[];
|
|
388
|
+
provenance?: SnapshotProvenance;
|
|
389
|
+
};
|
|
390
|
+
type ExportSnapshotOptions = {
|
|
391
|
+
cartVersion?: string;
|
|
392
|
+
modules?: SnapshotModuleRef[];
|
|
393
|
+
hostState?: unknown;
|
|
394
|
+
hostStateRef?: string;
|
|
395
|
+
assets?: SnapshotAssetRef[];
|
|
396
|
+
createdAt?: string;
|
|
397
|
+
provenance?: SnapshotProvenance;
|
|
398
|
+
runtimeVersion?: string;
|
|
399
|
+
};
|
|
400
|
+
|
|
331
401
|
/**
|
|
332
402
|
* Copyright (c) 2026 Aaron Boyarsky
|
|
333
403
|
* SPDX-License-Identifier: LicenseRef-CyberArt-Engine
|
|
@@ -475,6 +545,75 @@ type AssetPreloader = {
|
|
|
475
545
|
dispose(): void;
|
|
476
546
|
};
|
|
477
547
|
|
|
548
|
+
type AudioUnlockState = 'locked' | 'unlocking' | 'unlocked' | 'failed';
|
|
549
|
+
type AudioUnlockStatus = {
|
|
550
|
+
state: AudioUnlockState;
|
|
551
|
+
error?: string;
|
|
552
|
+
};
|
|
553
|
+
type AudioAssetStatus = 'ready' | 'failed';
|
|
554
|
+
type AudioChannelInspect = {
|
|
555
|
+
id: string;
|
|
556
|
+
participantId?: string;
|
|
557
|
+
gain: number;
|
|
558
|
+
muted: boolean;
|
|
559
|
+
duckGain: number;
|
|
560
|
+
priority: number;
|
|
561
|
+
effectiveGain: number;
|
|
562
|
+
};
|
|
563
|
+
type AudioBrokerInspect = {
|
|
564
|
+
status: AudioUnlockStatus;
|
|
565
|
+
reducedSensory: boolean;
|
|
566
|
+
muted: boolean;
|
|
567
|
+
authorized: string[];
|
|
568
|
+
channels: AudioChannelInspect[];
|
|
569
|
+
assets: Record<string, AudioAssetStatus>;
|
|
570
|
+
};
|
|
571
|
+
type AudioBrokerNotice = {
|
|
572
|
+
type: 'asset';
|
|
573
|
+
id: string;
|
|
574
|
+
status: AudioAssetStatus;
|
|
575
|
+
} | {
|
|
576
|
+
type: 'teardown';
|
|
577
|
+
participantId: string;
|
|
578
|
+
} | {
|
|
579
|
+
type: 'mute';
|
|
580
|
+
} | {
|
|
581
|
+
type: 'destroy';
|
|
582
|
+
};
|
|
583
|
+
type AudioBrokerListener = (notice: AudioBrokerNotice) => void;
|
|
584
|
+
type ActiveAudioCue = {
|
|
585
|
+
idempotencyKey: string;
|
|
586
|
+
channelId: string;
|
|
587
|
+
participantId?: string;
|
|
588
|
+
priority: number;
|
|
589
|
+
};
|
|
590
|
+
type AudioBroker = {
|
|
591
|
+
unlock(): Promise<AudioUnlockStatus>;
|
|
592
|
+
status(): AudioUnlockStatus;
|
|
593
|
+
authorize(participantId: string): void;
|
|
594
|
+
revoke(participantId: string): void;
|
|
595
|
+
isAuthorized(participantId: string | undefined): boolean;
|
|
596
|
+
setChannelGain(channelId: string, gain: number, participantId?: string): void;
|
|
597
|
+
setPriority(channelId: string, priority: number, participantId?: string): void;
|
|
598
|
+
mute(): void;
|
|
599
|
+
unmute(): void;
|
|
600
|
+
muteChannel(channelId: string, participantId?: string): void;
|
|
601
|
+
unmuteChannel(channelId: string): void;
|
|
602
|
+
duck(channelId: string, gain?: number): void;
|
|
603
|
+
unduck(channelId: string): void;
|
|
604
|
+
effectiveGain(channelId: string): number;
|
|
605
|
+
handleHostEvent(event: HostEvent): void;
|
|
606
|
+
assetStatus(id: string): AudioAssetStatus | undefined;
|
|
607
|
+
noteCueStarted(cue: ActiveAudioCue): void;
|
|
608
|
+
noteCueEnded(idempotencyKey: string): void;
|
|
609
|
+
teardown(participantId: string): void;
|
|
610
|
+
onNotice(listener: AudioBrokerListener): () => void;
|
|
611
|
+
inspect(): AudioBrokerInspect;
|
|
612
|
+
destroy(): void;
|
|
613
|
+
readonly reducedSensory: boolean;
|
|
614
|
+
readonly muted: boolean;
|
|
615
|
+
};
|
|
616
|
+
|
|
478
617
|
/**
|
|
479
618
|
* Copyright (c) 2026 Aaron Boyarsky
|
|
480
619
|
* SPDX-License-Identifier: LicenseRef-CyberArt-Engine
|
|
@@ -519,6 +658,17 @@ type CreateRuntimeOptions = {
|
|
|
519
658
|
* scripted `{ type: 'asset' }` actions own delivery timing.
|
|
520
659
|
*/
|
|
521
660
|
assets?: AssetRuntimeOptions;
|
|
661
|
+
/**
|
|
662
|
+
* Shared page-level unlock broker. Optional. Carts that only set
|
|
663
|
+
* `metadata.audio: 'tone'` keep the existing `start()` / `unlockAudio()`
|
|
664
|
+
* path when this is omitted.
|
|
665
|
+
*/
|
|
666
|
+
audioBroker?: AudioBroker;
|
|
667
|
+
/**
|
|
668
|
+
* Runtime-group participant id to authorize on this runtime. Teardown of
|
|
669
|
+
* this id does not close Tone for remaining carts.
|
|
670
|
+
*/
|
|
671
|
+
audioParticipantId?: string;
|
|
522
672
|
};
|
|
523
673
|
type MountOptions<T = unknown> = {
|
|
524
674
|
/** Boot overrides passed as `customState` into `getDefaultState`. Not a live-state replay. */
|
|
@@ -549,7 +699,12 @@ type CartHandle = {
|
|
|
549
699
|
getCartState(): unknown;
|
|
550
700
|
exportState(): Promise<CartStateBundle>;
|
|
551
701
|
exportStateJSON(): Promise<string>;
|
|
552
|
-
importState(bundle: CartStateBundle | string, extras?: {
|
|
702
|
+
importState(bundle: CartStateBundle | SnapshotEnvelope | string, extras?: {
|
|
703
|
+
framebuffer?: ImageData | null;
|
|
704
|
+
}): Promise<void>;
|
|
705
|
+
exportSnapshot(options?: ExportSnapshotOptions): Promise<SnapshotEnvelope>;
|
|
706
|
+
exportSnapshotJSON(options?: ExportSnapshotOptions): Promise<string>;
|
|
707
|
+
importSnapshot(input: SnapshotEnvelope | CartStateBundle | string, extras?: {
|
|
553
708
|
framebuffer?: ImageData | null;
|
|
554
709
|
}): Promise<void>;
|
|
555
710
|
peekExportedFramebuffer(): ImageData | null;
|
|
@@ -590,6 +745,8 @@ type CyberArtRuntime = {
|
|
|
590
745
|
* Survives cart remount; `destroy()` disposes it.
|
|
591
746
|
*/
|
|
592
747
|
readonly assets: AssetPreloader | undefined;
|
|
748
|
+
/** Shared broker when `createRuntime({ audioBroker })` was set. */
|
|
749
|
+
readonly audioBroker: AudioBroker | undefined;
|
|
593
750
|
onError?: (error: unknown, info: FrameErrorInfo) => void;
|
|
594
751
|
};
|
|
595
752
|
|
|
@@ -726,6 +883,8 @@ type CreateRuntimeGroupOptions = {
|
|
|
726
883
|
router?: EventRouterOptions;
|
|
727
884
|
/** Bound on the accepted-event trace (oldest dropped). Default 1024. */
|
|
728
885
|
maxTrace?: number;
|
|
886
|
+
/** Shared page-level audio unlock broker. Optional. */
|
|
887
|
+
audioBroker?: AudioBroker;
|
|
729
888
|
};
|
|
730
889
|
type RuntimeGroupParticipantInspect = {
|
|
731
890
|
state: unknown;
|
|
@@ -761,6 +920,7 @@ type RuntimeGroup = {
|
|
|
761
920
|
readonly router: EventRouter;
|
|
762
921
|
readonly origin: number;
|
|
763
922
|
readonly paused: boolean;
|
|
923
|
+
readonly audioBroker: AudioBroker | undefined;
|
|
764
924
|
participant(id: string): RuntimeGroupParticipantHandle;
|
|
765
925
|
step(frames?: number): Promise<void>;
|
|
766
926
|
pause(): void;
|
|
@@ -1213,6 +1373,127 @@ declare function replayExportedTrace(exported: ReplayInspectorExport, group: Run
|
|
|
1213
1373
|
report: ReplayInspectorReport;
|
|
1214
1374
|
}>;
|
|
1215
1375
|
|
|
1376
|
+
type CueEasing = 'linear' | 'ease-out';
|
|
1377
|
+
type CueDuplicatePolicy = 'ignore' | 'replace' | 'reject';
|
|
1378
|
+
type CueRepeatPolicy = {
|
|
1379
|
+
count: number;
|
|
1380
|
+
} | {
|
|
1381
|
+
forever: true;
|
|
1382
|
+
};
|
|
1383
|
+
type CueReducedMotionPolicy = 'skip' | 'complete' | {
|
|
1384
|
+
durationFrames: number;
|
|
1385
|
+
};
|
|
1386
|
+
type CueSpec = {
|
|
1387
|
+
name: string;
|
|
1388
|
+
idempotencyKey: string;
|
|
1389
|
+
/** Frame when the cue is eligible to start (before delay). Default: play frame. */
|
|
1390
|
+
startFrame?: number;
|
|
1391
|
+
durationFrames: number;
|
|
1392
|
+
delayFrames?: number;
|
|
1393
|
+
easing?: CueEasing;
|
|
1394
|
+
repeat?: CueRepeatPolicy;
|
|
1395
|
+
/**
|
|
1396
|
+
* When the timeline is in reduced-motion mode: skip (complete immediately),
|
|
1397
|
+
* complete (same), or a shorter duration. Default `complete`.
|
|
1398
|
+
*/
|
|
1399
|
+
reducedMotion?: CueReducedMotionPolicy;
|
|
1400
|
+
onDuplicate?: CueDuplicatePolicy;
|
|
1401
|
+
};
|
|
1402
|
+
type CuePhase = 'scheduled' | 'active' | 'completed' | 'cancelled';
|
|
1403
|
+
type CueView = {
|
|
1404
|
+
name: string;
|
|
1405
|
+
idempotencyKey: string;
|
|
1406
|
+
phase: CuePhase;
|
|
1407
|
+
startFrame: number;
|
|
1408
|
+
durationFrames: number;
|
|
1409
|
+
delayFrames: number;
|
|
1410
|
+
easing: CueEasing;
|
|
1411
|
+
progress: number;
|
|
1412
|
+
repeatIndex: number;
|
|
1413
|
+
};
|
|
1414
|
+
|
|
1415
|
+
/**
|
|
1416
|
+
* Copyright (c) 2026 Aaron Boyarsky
|
|
1417
|
+
* SPDX-License-Identifier: LicenseRef-CyberArt-Engine
|
|
1418
|
+
* See packages/engine/LICENSE
|
|
1419
|
+
*
|
|
1420
|
+
* Deterministic audio-cue timeline. Scheduling is frame-stepped via
|
|
1421
|
+
* `createPresentationTimeline`; PCM output is not part of the event trace.
|
|
1422
|
+
*/
|
|
1423
|
+
|
|
1424
|
+
declare const AUDIO_CUE_EVENTS: readonly ["audio.cue.scheduled", "audio.cue.started", "audio.cue.skipped", "audio.cue.failed"];
|
|
1425
|
+
type AudioCueEventType = (typeof AUDIO_CUE_EVENTS)[number];
|
|
1426
|
+
type AudioCueSkipReason = 'reduced-sensory' | 'muted' | 'unauthorized';
|
|
1427
|
+
type AudioCueFailReason = 'asset-failed' | 'torn-down' | 'invalid';
|
|
1428
|
+
type AudioCueReason = AudioCueSkipReason | AudioCueFailReason;
|
|
1429
|
+
type AudioCueSpec = CueSpec & {
|
|
1430
|
+
assetId: string;
|
|
1431
|
+
channelId?: string;
|
|
1432
|
+
participantId?: string;
|
|
1433
|
+
priority?: number;
|
|
1434
|
+
gain?: number;
|
|
1435
|
+
};
|
|
1436
|
+
type AudioCueView = CueView & {
|
|
1437
|
+
assetId: string;
|
|
1438
|
+
channelId: string;
|
|
1439
|
+
participantId?: string;
|
|
1440
|
+
priority: number;
|
|
1441
|
+
audioPhase: 'scheduled' | 'started' | 'skipped' | 'failed' | 'completed';
|
|
1442
|
+
};
|
|
1443
|
+
type AudioCueEvent = {
|
|
1444
|
+
type: AudioCueEventType;
|
|
1445
|
+
atFrame: number;
|
|
1446
|
+
name: string;
|
|
1447
|
+
idempotencyKey: string;
|
|
1448
|
+
assetId: string;
|
|
1449
|
+
channelId: string;
|
|
1450
|
+
participantId?: string;
|
|
1451
|
+
reason?: AudioCueReason;
|
|
1452
|
+
progress: number;
|
|
1453
|
+
};
|
|
1454
|
+
type AudioCueTimelineSnapshot = {
|
|
1455
|
+
frame: number;
|
|
1456
|
+
reducedSensory: boolean;
|
|
1457
|
+
cues: AudioCueView[];
|
|
1458
|
+
events: AudioCueEvent[];
|
|
1459
|
+
};
|
|
1460
|
+
type PlayAudioCueResult = {
|
|
1461
|
+
ok: true;
|
|
1462
|
+
cue: AudioCueView;
|
|
1463
|
+
} | {
|
|
1464
|
+
ok: false;
|
|
1465
|
+
reason: 'duplicate' | 'invalid';
|
|
1466
|
+
detail: string;
|
|
1467
|
+
};
|
|
1468
|
+
type AudioCueTimeline = {
|
|
1469
|
+
play(spec: AudioCueSpec): PlayAudioCueResult;
|
|
1470
|
+
step(frames?: number): AudioCueEvent[];
|
|
1471
|
+
cancel(idempotencyKey: string): boolean;
|
|
1472
|
+
reset(): void;
|
|
1473
|
+
snapshot(): AudioCueTimelineSnapshot;
|
|
1474
|
+
get(idempotencyKey: string): AudioCueView | undefined;
|
|
1475
|
+
dispose(): void;
|
|
1476
|
+
readonly frame: number;
|
|
1477
|
+
readonly reducedSensory: boolean;
|
|
1478
|
+
};
|
|
1479
|
+
type HeadlessAudioAdapter = {
|
|
1480
|
+
readonly broker: AudioBroker;
|
|
1481
|
+
readonly timeline: AudioCueTimeline;
|
|
1482
|
+
unlock(): Promise<AudioUnlockStatus>;
|
|
1483
|
+
play(spec: AudioCueSpec): PlayAudioCueResult;
|
|
1484
|
+
step(frames?: number): AudioCueEvent[];
|
|
1485
|
+
handleHostEvent(event: HostEvent): void;
|
|
1486
|
+
snapshot(): AudioCueTimelineSnapshot & {
|
|
1487
|
+
unlock: AudioUnlockStatus;
|
|
1488
|
+
muted: boolean;
|
|
1489
|
+
};
|
|
1490
|
+
destroy(): void;
|
|
1491
|
+
};
|
|
1492
|
+
declare function createHeadlessAudioAdapter(options?: {
|
|
1493
|
+
reducedSensory?: boolean;
|
|
1494
|
+
originFrame?: number;
|
|
1495
|
+
}): HeadlessAudioAdapter;
|
|
1496
|
+
|
|
1216
1497
|
/**
|
|
1217
1498
|
* Copyright (c) 2026 Aaron Boyarsky
|
|
1218
1499
|
* SPDX-License-Identifier: LicenseRef-CyberArt-Engine
|
|
@@ -1232,4 +1513,4 @@ type WriteComposedFrameResult = ComposedFrame & {
|
|
|
1232
1513
|
*/
|
|
1233
1514
|
declare function writeComposedFrame(compositor: Compositor, path?: string): Promise<WriteComposedFrameResult>;
|
|
1234
1515
|
|
|
1235
|
-
export { type BoundReplaySession, type CausationTreeNode, type CreateHeadlessHarnessOptions, type CreateHeadlessMultiCartHarnessOptions, type CreateReplayInspectorOptions, DEFAULT_HEADLESS_HEIGHT, DEFAULT_HEADLESS_WIDTH, type GlyphAtlas, HEADLESS_PNG_DATA_URL, type HeadlessCanvas2DSettings, type HeadlessFrameError, type HeadlessHarness, type HeadlessImageFixture, type HeadlessInspect, type HeadlessMultiCartHarness, HeadlessUnsupportedOperationError, type InspectorRecord, type InstallHeadlessCanvasOptions, type ReplayCompareResult, type ReplayInspector, type ReplayInspectorExport, type ReplayInspectorReport, type ReplayParticipantSummary, type ReplayTapeAction, type ReplayTraceFilter, UPDATE_GOLDEN_ENV, type VisualArtifactPaths, type VisualArtifacts, type VisualCompareOptions, type VisualCompareResult, type WriteComposedFrameResult, assertPixelsEqual, assertPngDataUrlsEqual, attachHeadlessCanvas2D, compareImageData, compareReplayTraces, createDefaultGlyphAtlas, createHeadlessHarness, createHeadlessMultiCartHarness, createImageFixture, createReplayInspector, decodePng, encodePng, encodePngDataUrl, getHeadlessSurface, imageDataFromPngDataUrl, installHeadlessCanvas, makeImageData, replayExportedTrace, setDefaultGlyphAtlas, shouldUpdateGolden, writeComposedFrame, writeVisualArtifacts };
|
|
1516
|
+
export { type BoundReplaySession, type CausationTreeNode, type CreateHeadlessHarnessOptions, type CreateHeadlessMultiCartHarnessOptions, type CreateReplayInspectorOptions, DEFAULT_HEADLESS_HEIGHT, DEFAULT_HEADLESS_WIDTH, type GlyphAtlas, HEADLESS_PNG_DATA_URL, type HeadlessAudioAdapter, type HeadlessCanvas2DSettings, type HeadlessFrameError, type HeadlessHarness, type HeadlessImageFixture, type HeadlessInspect, type HeadlessMultiCartHarness, HeadlessUnsupportedOperationError, type InspectorRecord, type InstallHeadlessCanvasOptions, type ReplayCompareResult, type ReplayInspector, type ReplayInspectorExport, type ReplayInspectorReport, type ReplayParticipantSummary, type ReplayTapeAction, type ReplayTraceFilter, UPDATE_GOLDEN_ENV, type VisualArtifactPaths, type VisualArtifacts, type VisualCompareOptions, type VisualCompareResult, type WriteComposedFrameResult, assertPixelsEqual, assertPngDataUrlsEqual, attachHeadlessCanvas2D, compareImageData, compareReplayTraces, createDefaultGlyphAtlas, createHeadlessAudioAdapter, createHeadlessHarness, createHeadlessMultiCartHarness, createImageFixture, createReplayInspector, decodePng, encodePng, encodePngDataUrl, getHeadlessSurface, imageDataFromPngDataUrl, installHeadlessCanvas, makeImageData, replayExportedTrace, setDefaultGlyphAtlas, shouldUpdateGolden, writeComposedFrame, writeVisualArtifacts };
|