stillmade-sdk 0.1.42 → 0.1.43
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.stillmade/sdk-baseline.json +6 -8
- package/AGENTS.md +15 -10
- package/FRAME_VIEWS.md +113 -0
- package/INTERFACE_DESIGN.md +62 -22
- package/MOBILE_UI.md +4 -2
- package/MODULES.md +152 -6
- package/PASTE_TO_LLM.md +35 -25
- package/README.md +3 -2
- package/SDK.md +994 -550
- package/UI_AESTHETIC.md +22 -2
- package/blocks/generate/package.js +1 -0
- package/blocks/generate/versions/1.0.0/package.js +463 -0
- package/docs/build/audio-to-editor.md +36 -0
- package/docs/build/block-builder-contract.md +150 -0
- package/docs/build/comfyui.md +227 -0
- package/docs/build/concepts.md +40 -0
- package/docs/build/context.md +159 -0
- package/docs/build/cookbook.md +57 -0
- package/docs/build/custom-interface.md +617 -0
- package/docs/build/data-types.md +83 -0
- package/docs/build/frame-views.md +259 -0
- package/docs/build/generate-media.md +59 -0
- package/docs/build/github-applications.md +314 -0
- package/docs/build/hosted-images.md +214 -0
- package/docs/build/hosted-music.md +39 -0
- package/docs/build/hosted-speech.md +157 -0
- package/docs/build/hosted-text.md +246 -0
- package/docs/build/image-description.md +49 -0
- package/docs/build/interface-design.md +343 -0
- package/docs/build/javascript.md +63 -0
- package/docs/build/local-preview.md +61 -0
- package/docs/build/media-analysis.md +27 -0
- package/docs/build/media.md +400 -0
- package/docs/build/mobile-ui.md +106 -0
- package/docs/build/modules.md +487 -0
- package/docs/build/original-blocks.md +70 -0
- package/docs/build/outside-apis.md +128 -0
- package/docs/build/production-context-templates.md +12 -0
- package/docs/build/project-storage.md +94 -0
- package/docs/build/quickstart.md +48 -0
- package/docs/build/recipes.md +103 -0
- package/docs/build/script-handoff.md +45 -0
- package/docs/build/script-writer.md +34 -0
- package/docs/build/state.md +223 -0
- package/docs/build/testing.md +112 -0
- package/docs/build/timeline-edits.md +116 -0
- package/docs/build/timeline-proposals.md +76 -0
- package/docs/build/transcription.md +59 -0
- package/docs/build/triggers.md +59 -0
- package/docs/build/tutorial.md +52 -0
- package/docs/build/typescript.md +34 -0
- package/docs/build/video-generation.md +61 -0
- package/docs/build/web-reading.md +37 -0
- package/docs/build/with-your-ai.md +27 -0
- package/docs/changelog.md +9 -0
- package/docs/creators.md +21 -0
- package/docs/distribute/discovery.md +43 -0
- package/docs/distribute/imports.md +58 -0
- package/docs/distribute/publishing.md +29 -0
- package/docs/distribute/sdk-versions.md +36 -0
- package/docs/distribute/selling.md +14 -0
- package/docs/distribute/teams.md +60 -0
- package/docs/distribute/versioning.md +28 -0
- package/docs/examples.md +23 -0
- package/docs/gallery.md +26 -0
- package/docs/index.md +50 -0
- package/docs/native-mcp.md +307 -0
- package/docs/project-types/automation.md +43 -0
- package/docs/project-types/batches.md +50 -0
- package/docs/project-types/build.md +42 -0
- package/docs/project-types/completion.md +15 -0
- package/docs/project-types/conditions.md +21 -0
- package/docs/project-types/hosted-review.md +115 -0
- package/docs/project-types/onboarding.md +75 -0
- package/docs/project-types/running.md +309 -0
- package/docs/project-types/testing.md +72 -0
- package/docs/reference/application-artifacts.md +87 -0
- package/docs/reference/authoring-guide.md +134 -0
- package/docs/reference/browser-applications.md +267 -0
- package/docs/reference/capabilities.md +45 -0
- package/docs/reference/cli.md +71 -0
- package/docs/reference/connection-api.md +59 -0
- package/docs/reference/cooperative-jobs.md +13 -0
- package/docs/reference/desktop-capabilities.md +172 -0
- package/docs/reference/desktop-execution.md +77 -0
- package/docs/reference/errors.md +233 -0
- package/docs/reference/execution.md +800 -0
- package/docs/reference/first-party-blocks.md +40 -0
- package/docs/reference/generation-models.md +59 -0
- package/docs/reference/limits.md +306 -0
- package/docs/reference/manifest-kinds-controls.md +36 -0
- package/docs/reference/manifest.md +181 -0
- package/docs/reference/operations.md +36 -0
- package/docs/reference/package-jobs.md +35 -0
- package/docs/reference/parity.md +55 -0
- package/docs/reference/permissions.md +86 -0
- package/docs/reference/ports.md +26 -0
- package/docs/reference/types.md +268 -0
- package/docs/reference/universal-resolver.md +303 -0
- package/docs/reference/view-bridge.md +60 -0
- package/docs/reference/view-rules.md +15 -0
- package/docs/reference/workspace-sync.md +25 -0
- package/docs/sdk.md +34 -0
- package/docs/tools/block-remix.md +211 -0
- package/docs/tools/llm.md +2213 -0
- package/docs/tools/repository-adaptation.md +613 -0
- package/examples/check-clip.stillmade.json +4 -1
- package/examples/generate-video.stillmade.json +6 -2
- package/examples/research-topic.stillmade.json +5 -1
- package/examples/text-workflow.stillmade.json +3 -32
- package/examples/timeline-polish.stillmade.json +11 -3
- package/examples/transcribe-audio.stillmade.json +11 -3
- package/package.json +1 -1
- package/packages/block-cli/cli.js +13 -6
- package/packages/block-cli/dev-host/assets/host.js +76 -38
- package/packages/block-cli/dev-host/assets/index.css +1 -1
- package/packages/block-cli/format.js +1 -1
- package/packages/block-cli/templates.js +31 -13
- package/packages/block-cli/type-packages.js +119 -0
- package/packages/block-platform/block-readiness.js +5 -0
- package/packages/block-platform/canvas.js +2 -1
- package/packages/block-platform/catalog.js +27 -48
- package/packages/block-platform/native-host-inventory.js +0 -226
- package/packages/block-platform/plain-descriptions.js +1 -25
- package/packages/block-platform/project-type-starters.js +13 -53
- package/packages/block-platform/script-handoff.js +2 -4
- package/packages/block-platform/type-workspace.js +2 -0
- package/packages/block-sdk/appearance.js +3 -1
- package/packages/block-sdk/block-tier.js +1 -1
- package/packages/block-sdk/builder-contract.js +10 -8
- package/packages/block-sdk/capabilities.js +1 -1
- package/packages/block-sdk/contracts.js +6 -2
- package/packages/block-sdk/first-party-identities.js +1 -84
- package/packages/block-sdk/first-party-packages.js +3 -85
- package/packages/block-sdk/generation-catalog.js +15 -3
- package/packages/block-sdk/index.d.ts +2 -1
- package/packages/block-sdk/media-analysis-example.js +1 -1
- package/packages/block-sdk/mock-host.js +7 -1
- package/packages/block-sdk/module-host-calls.js +35 -10
- package/packages/block-sdk/module-hosted.js +15 -2
- package/packages/block-sdk/module-media-access.js +71 -0
- package/packages/block-sdk/module-node.js +1 -1
- package/packages/block-sdk/portable-media-actions.js +1 -1
- package/packages/block-sdk/runtime.js +1 -1
- package/packages/block-sdk/timeline-proposal-example.js +1 -1
- package/packages/block-sdk/transcription-example.js +1 -1
- package/packages/block-sdk/video-generation-example.js +1 -1
- package/packages/block-sdk/view-bridge.js +4 -1
- package/packages/block-sdk/view-design.js +17 -14
- package/packages/block-sdk/web-example.js +1 -1
- package/packages/design-system/block-kit.css +48 -12
- package/packages/design-system/block-kit.html +3 -2
- package/blocks/mcp-motion-planner/versions/1.1.0/package.js +0 -97
- package/blocks/motion-designer/versions/1.0.1/package.js +0 -173318
- package/blocks/motion-designer/versions/1.0.2/package.js +0 -173318
- package/blocks/motion-frame-review/versions/1.1.1/package.js +0 -465
- package/blocks/motion-frame-review/versions/1.1.2/package.snapshot.js +0 -572
- package/blocks/narration/package.js +0 -1
- package/blocks/narration/versions/1.0.0/package.js +0 -221
- package/blocks/narration/versions/1.0.1/package.js +0 -221
- package/blocks/native/sm.animated-scene/versions/2.0.0/package.js +0 -475
- package/blocks/native/sm.animated-scene/versions/2.1.0/package.js +0 -498
- package/blocks/native/sm.animated-scene/versions/2.2.0/package.js +0 -524
- package/blocks/native/sm.animated-scene/versions/2.3.0/package.js +0 -618
- package/blocks/native/sm.animated-scene/versions/2.3.1/package.js +0 -618
- package/blocks/native/sm.animated-scene/versions/2.3.2/package.js +0 -723
- package/blocks/native/sm.animated-scene/versions/2.3.3/package.js +0 -723
- package/blocks/native/sm.animated-scene/versions/2.3.4/package.js +0 -676
- package/blocks/native/sm.animated-scene/versions/2.3.5/package.js +0 -693
- package/blocks/native/sm.animated-scene/versions/2.3.6/package.js +0 -700
- package/blocks/native/sm.animated-scene/versions/2.3.7/package.js +0 -747
- package/blocks/native/sm.animated-scene/versions/2.3.8/package.js +0 -769
- package/blocks/native/sm.animated-scene/versions/2.3.9/package.js +0 -769
- package/blocks/native/sm.animated-scene-assistant/versions/1.0.0/package.js +0 -183
- package/blocks/native/sm.blueprint/versions/2.0.0/package.js +0 -358
- package/blocks/native/sm.blueprint/versions/2.1.0/package.js +0 -358
- package/blocks/native/sm.blueprint/versions/2.1.1/package.js +0 -358
- package/blocks/native/sm.blueprint/versions/2.1.2/package.js +0 -358
- package/blocks/native/sm.blueprint/versions/2.1.3/package.js +0 -381
- package/blocks/native/sm.blueprint/versions/2.1.4/package.js +0 -381
- package/blocks/native/sm.blueprint/versions/2.1.5/package.js +0 -339
- package/blocks/native/sm.blueprint/versions/2.1.6/package.js +0 -356
- package/blocks/native/sm.blueprint/versions/2.1.7/package.js +0 -363
- package/blocks/native/sm.blueprint/versions/2.1.8/package.js +0 -410
- package/blocks/native/sm.blueprint/versions/2.1.9/package.js +0 -422
- package/blocks/native/sm.blueprint-assistant/versions/1.0.0/package.js +0 -172
- package/blocks/native/sm.chat/versions/2.0.0/package.js +0 -144
- package/blocks/native/sm.chat/versions/2.0.1/package.js +0 -97
- package/blocks/native/sm.chat/versions/2.0.2/package.js +0 -100
- package/blocks/native/sm.chat/versions/2.0.3/package.js +0 -100
- package/blocks/native/sm.continue-if/versions/1.0.0/package.js +0 -168
- package/blocks/native/sm.describe-media/versions/2.0.0/package.js +0 -153
- package/blocks/native/sm.describe-media/versions/2.0.1/package.js +0 -156
- package/blocks/native/sm.describe-media/versions/2.1.0/package.js +0 -140
- package/blocks/native/sm.image-editor/versions/2.0.0/package.js +0 -183
- package/blocks/native/sm.image-editor/versions/2.0.1/package.js +0 -190
- package/blocks/native/sm.new-items/versions/1.0.0/package.js +0 -191
- package/blocks/native/sm.pick-photos/versions/1.0.0/package.js +0 -334
- package/blocks/native/sm.quality-audit/versions/1.0.0/package.js +0 -192
- package/blocks/native/sm.quality-audit/versions/1.0.1/package.js +0 -192
- package/blocks/native/sm.quality-audit/versions/1.0.2/package.js +0 -195
- package/blocks/native/sm.research/versions/2.0.0/package.js +0 -175
- package/blocks/native/sm.research/versions/2.0.1/package.js +0 -184
- package/blocks/native/sm.research/versions/2.0.2/package.js +0 -187
- package/blocks/native/sm.script/versions/2.0.0/package.js +0 -144
- package/blocks/native/sm.script/versions/2.1.0/package.js +0 -144
- package/blocks/native/sm.script/versions/2.1.1/package.js +0 -144
- package/blocks/native/sm.script/versions/2.1.2/package.js +0 -105
- package/blocks/native/sm.script/versions/2.1.3/package.js +0 -152
- package/blocks/native/sm.script/versions/2.1.4/package.js +0 -156
- package/blocks/native/sm.script/versions/2.1.5/package.js +0 -156
- package/blocks/native/sm.shot-plan/versions/2.0.0/package.js +0 -301
- package/blocks/native/sm.shot-plan/versions/2.1.0/package.js +0 -528
- package/blocks/native/sm.shot-plan/versions/2.1.1/package.js +0 -528
- package/blocks/native/sm.shot-plan/versions/2.1.2/package.js +0 -528
- package/blocks/native/sm.shot-plan/versions/2.1.3/package.js +0 -536
- package/blocks/native/sm.shot-plan/versions/2.2.0/package.js +0 -800
- package/blocks/native/sm.shot-plan/versions/2.2.1/package.js +0 -920
- package/blocks/native/sm.shot-plan-assistant/versions/1.0.0/package.js +0 -173
- package/blocks/native/sm.stock-search/versions/2.0.0/package.js +0 -174
- package/blocks/native/sm.thumbnail/versions/2.0.0/package.js +0 -137
- package/blocks/native/sm.thumbnail/versions/2.0.1/package.js +0 -140
- package/blocks/native/sm.transcribe/versions/2.0.0/package.js +0 -163
- package/blocks/native/sm.transcribe/versions/2.0.1/package.js +0 -168
- package/blocks/native/sm.whiteboard-assistant/versions/1.0.0/package.js +0 -175
- package/blocks/native/sm.youtube/versions/2.0.0/package.js +0 -217
- package/blocks/occasion-story/package.js +0 -1
- package/blocks/occasion-story/versions/1.0.0/package.js +0 -1658
- package/blocks/occasion-story/versions/1.0.1/package.js +0 -1699
- package/blocks/occasion-story/versions/1.0.2/package.js +0 -1682
- package/blocks/photo-reel-composer/versions/1.0.0/package.js +0 -3043
- package/blocks/photo-reel-composer/versions/1.0.1/package.js +0 -3026
- package/blocks/photo-reel-composer/versions/1.0.2/package.js +0 -4149
- package/blocks/product-ad-variants/versions/1.0.0/package.js +0 -2911
- package/blocks/product-launch-director/versions/1.0.0/package.js +0 -103
- package/blocks/release-visuals/package.js +0 -1
- package/blocks/release-visuals/versions/1.0.0/package.js +0 -1492
- package/blocks/release-visuals/versions/1.0.1/package.js +0 -1517
- package/blocks/release-visuals/versions/1.0.2/package.js +0 -1500
- package/blocks/screen-recorder/package.js +0 -1
- package/blocks/screen-recorder/versions/1.0.0/package.js +0 -223
- package/blocks/screen-recorder/versions/1.0.1/package.js +0 -223
- package/blocks/short-form-finish/package.js +0 -1
- package/blocks/short-form-finish/versions/1.0.0/package.js +0 -1043
- package/blocks/short-form-finish/versions/1.0.1/package.js +0 -1043
- package/blocks/short-form-finish/versions/1.0.2/package.js +0 -1043
- package/blocks/short-form-finish/versions/1.0.3/package.js +0 -1026
- package/examples/saas-product-launch-reviewed-source.stillmade.json +0 -383
- package/examples/saas-product-launch.stillmade.json +0 -304
- package/packages/block-platform/dealer-inventory-video-project-type.js +0 -38
- package/packages/block-platform/first-revenue-project-types.js +0 -46
- package/packages/block-platform/fixture-starters.js +0 -13
- package/packages/block-platform/launch-project-source.js +0 -10
- package/packages/block-platform/launch-project.js +0 -30
- package/packages/block-platform/motion-project-types.js +0 -55
- package/packages/block-platform/motion-starters.js +0 -14
- package/packages/block-platform/product-ad-variants-project-type.js +0 -30
- package/packages/block-platform/product-launch-updated.js +0 -90
- package/packages/block-platform/revenue-review-project-types.js +0 -36
- package/packages/block-platform/revenue-review-starters.js +0 -13
- package/packages/block-platform/script-writer.js +0 -29
- package/packages/block-platform/website-source-block.js +0 -14
|
@@ -0,0 +1,223 @@
|
|
|
1
|
+
# Remember data between runs
|
|
2
|
+
|
|
3
|
+
## When to use memory
|
|
4
|
+
Use saved memory for small counters or preferences that belong to one placement. Use [project context](context.md) for scripts, scenes, shots, assets, and production history. Never put provider keys or media bytes into Block memory.
|
|
5
|
+
|
|
6
|
+
## Try the complete example
|
|
7
|
+
[Download Numbered takes](https://stillmade.shop/block-sdk/examples/numbered-takes.stillmade.json). Import it, review its tests and preview, then confirm the import. Add it to a Project Type and run it twice: the labels advance from Take 1 to Take 2. A second placement has its own counter.
|
|
8
|
+
|
|
9
|
+
```json
|
|
10
|
+
{
|
|
11
|
+
"manifest": {
|
|
12
|
+
"schemaVersion": 1,
|
|
13
|
+
"sdkVersion": "0.1.0",
|
|
14
|
+
"id": "example.numbered-takes",
|
|
15
|
+
"action": {
|
|
16
|
+
"label": "Number takes",
|
|
17
|
+
"running": "Numbering takes…"
|
|
18
|
+
},
|
|
19
|
+
"version": "1.0.0",
|
|
20
|
+
"name": "Numbered takes",
|
|
21
|
+
"description": "Label successive creative takes with an independent counter for each production placement.",
|
|
22
|
+
"kind": "task",
|
|
23
|
+
"runtime": "javascript",
|
|
24
|
+
"license": "MIT",
|
|
25
|
+
"provenance": {
|
|
26
|
+
"notice": "MIT License\n\nCopyright (c) 2026 StillMade SDK contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of this software and \nassociated documentation files (the \"Software\"), to deal in the Software without restriction, including \nwithout limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell \ncopies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the \nfollowing conditions:\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial \nportions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT \nLIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO \nEVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER \nIN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE \nUSE OR OTHER DEALINGS IN THE SOFTWARE.\n"
|
|
27
|
+
},
|
|
28
|
+
"inputs": {
|
|
29
|
+
"text": {
|
|
30
|
+
"type": "text",
|
|
31
|
+
"primary": true,
|
|
32
|
+
"semantic": "draft_text",
|
|
33
|
+
"required": true,
|
|
34
|
+
"description": "Creative take text to label with the next number."
|
|
35
|
+
}
|
|
36
|
+
},
|
|
37
|
+
"outputs": {
|
|
38
|
+
"text": {
|
|
39
|
+
"type": "text",
|
|
40
|
+
"primary": true,
|
|
41
|
+
"semantic": "numbered_take",
|
|
42
|
+
"description": "The supplied take prefixed with its saved sequence number."
|
|
43
|
+
}
|
|
44
|
+
},
|
|
45
|
+
"permissions": {
|
|
46
|
+
"project": [
|
|
47
|
+
"state.step"
|
|
48
|
+
]
|
|
49
|
+
},
|
|
50
|
+
"state": {
|
|
51
|
+
"scope": "step",
|
|
52
|
+
"version": 1,
|
|
53
|
+
"initial": {
|
|
54
|
+
"count": 0
|
|
55
|
+
}
|
|
56
|
+
},
|
|
57
|
+
"ui": [
|
|
58
|
+
{
|
|
59
|
+
"port": "text",
|
|
60
|
+
"control": "text"
|
|
61
|
+
}
|
|
62
|
+
]
|
|
63
|
+
},
|
|
64
|
+
"code": "state.count += 1; return {text: \"Take \" + state.count + \": \" + input.text.trim()};",
|
|
65
|
+
"tests": [
|
|
66
|
+
{
|
|
67
|
+
"name": "First take",
|
|
68
|
+
"input": {
|
|
69
|
+
"text": " Opening scene "
|
|
70
|
+
},
|
|
71
|
+
"expected": {
|
|
72
|
+
"text": "Take 1: Opening scene"
|
|
73
|
+
},
|
|
74
|
+
"expectedState": {
|
|
75
|
+
"version": 1,
|
|
76
|
+
"value": {
|
|
77
|
+
"count": 1
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
},
|
|
81
|
+
{
|
|
82
|
+
"name": "Continues saved numbering",
|
|
83
|
+
"input": {
|
|
84
|
+
"text": "Alternate ending"
|
|
85
|
+
},
|
|
86
|
+
"state": {
|
|
87
|
+
"version": 1,
|
|
88
|
+
"value": {
|
|
89
|
+
"count": 4
|
|
90
|
+
}
|
|
91
|
+
},
|
|
92
|
+
"expected": {
|
|
93
|
+
"text": "Take 5: Alternate ending"
|
|
94
|
+
},
|
|
95
|
+
"expectedState": {
|
|
96
|
+
"version": 1,
|
|
97
|
+
"value": {
|
|
98
|
+
"count": 5
|
|
99
|
+
}
|
|
100
|
+
}
|
|
101
|
+
}
|
|
102
|
+
]
|
|
103
|
+
}
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
## Run from the command line
|
|
107
|
+
Create a stateful starter, run its fixtures, and save two separate memory checkpoints:
|
|
108
|
+
|
|
109
|
+
```sh
|
|
110
|
+
node packages/block-cli/cli.js create ./takes stateful
|
|
111
|
+
node packages/block-cli/cli.js test ./takes
|
|
112
|
+
node packages/block-cli/cli.js preview ./takes --save-state take-1.json
|
|
113
|
+
node packages/block-cli/cli.js preview ./takes --state take-1.json --save-state take-2.json
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Without --state, each preview starts fresh. --save-state writes a new file and refuses to overwrite an existing checkpoint. These files contain your Block memory, are pinned to the exact package digest, and should stay private if their contents are private. The CLI never writes state unless you request it.
|
|
117
|
+
|
|
118
|
+
## Run with the offline SDK
|
|
119
|
+
Extract the SDK and save this script alongside its packages folder. The host chooses when to save returned state. Here both runs use a disposable sandbox while the caller holds the accepted state.
|
|
120
|
+
|
|
121
|
+
```javascript
|
|
122
|
+
import {readFile} from 'node:fs/promises';
|
|
123
|
+
import {initialBlockState,runPackageAsync} from './packages/block-sdk/index.js';
|
|
124
|
+
const pkg=JSON.parse(await readFile('./examples/numbered-takes.stillmade.json','utf8'));
|
|
125
|
+
let memory=initialBlockState(pkg.manifest);
|
|
126
|
+
const first=await runPackageAsync(pkg,{text:'Opening'},{state:memory});
|
|
127
|
+
memory=first.state; // accept only after your host validates current permissions and inputs
|
|
128
|
+
const second=await runPackageAsync(pkg,{text:'Alternate opening'},{state:memory});
|
|
129
|
+
console.log(first.outputs.text,second.outputs.text);
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
## State contract and limits
|
|
133
|
+
Stateful JavaScript Blocks can opt into a small JSON state object. Production
|
|
134
|
+
placements save accepted state with their outputs in the existing project document.
|
|
135
|
+
Each placement has independent memory pinned to the exact package digest. Connected
|
|
136
|
+
runs retain proposed memory until the user accepts their results; changed memory
|
|
137
|
+
invalidates an older result. A failed or cancelled run does not advance memory.
|
|
138
|
+
|
|
139
|
+
Block previews keep temporary memory across runs; Reset or closing the preview
|
|
140
|
+
discards it. Import samples and each Project Type rehearsal placement start fresh.
|
|
141
|
+
Fixtures never become project memory. External SDK hosts must supply state explicitly
|
|
142
|
+
or execution fails with `STATE_REQUIRED`.
|
|
143
|
+
|
|
144
|
+
Changing a placement to another build starts fresh memory and archives the previous
|
|
145
|
+
placement through workflow history; restoring that exact placement restores its
|
|
146
|
+
previous memory. Duplicating a workflow placement starts independent fresh memory.
|
|
147
|
+
Stateful single-Block batches process the selection in order using provisional
|
|
148
|
+
memory. Choosing a batch result accepts memory through that result; later
|
|
149
|
+
previews cannot overwrite it. Changes to inputs, selected source versions, or
|
|
150
|
+
starting memory require a new batch. Connected multi-Block batches also carry
|
|
151
|
+
independent provisional memory for every placement in selection order. Use completed
|
|
152
|
+
results validates the original source selection and each memory transition before
|
|
153
|
+
committing the completed prefix together. A failed later transform preserves earlier
|
|
154
|
+
results for review without advancing saved memory automatically. Single-Block background automation supports saved memory when shared Canvas editing is enabled. Each completed job commits its outputs and memory in one transaction; a changed workspace or memory causes a retry. Failed, paused or stale workers do not advance memory. Selecting a completed result does not advance memory again. Connected background workflows pin all starting states and commit every placement’s memory together when the complete workflow succeeds. Failed prefixes retain their results without advancing shared memory; resuming requires the same starting memory. Hosted APIs and ComfyUI still use their explicit execution flow. State is bounded JSON, not a media-storage API. The production workspace’s
|
|
155
|
+
Block memory controls let users reset a placement and undo that reset until
|
|
156
|
+
the next accepted run. Reset keeps previous outputs visible but marks them stale.
|
|
157
|
+
|
|
158
|
+
Declare this alongside the usual manifest fields:
|
|
159
|
+
|
|
160
|
+
```json
|
|
161
|
+
{
|
|
162
|
+
"runtime": "javascript",
|
|
163
|
+
"permissions": {"project": ["state.step"]},
|
|
164
|
+
"state": {"scope": "step", "version": 1, "initial": {"count": 0}}
|
|
165
|
+
}
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
Your JavaScript body receives `input` and, only for stateful Blocks, a mutable
|
|
169
|
+
`state` object. Update its properties and return your ordinary output ports:
|
|
170
|
+
|
|
171
|
+
```js
|
|
172
|
+
state.count += input.amount;
|
|
173
|
+
return {count: state.count};
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
Do not reassign the `state` parameter. Keep it plain JSON. Each invocation uses a
|
|
177
|
+
fresh sandbox and a copy of the supplied state. No storage, network, native API,
|
|
178
|
+
or another Block's memory is exposed. State is limited to 64 KiB and a positive
|
|
179
|
+
schema version. Only JavaScript currently supports this contract.
|
|
180
|
+
|
|
181
|
+
A trusted host calls:
|
|
182
|
+
|
|
183
|
+
```js
|
|
184
|
+
const previous = initialBlockState(pkg.manifest); // first run only
|
|
185
|
+
const result = await runPackageAsync(pkg, {amount: 2}, {state: previous, signal});
|
|
186
|
+
// result.outputs is the normal output map.
|
|
187
|
+
// result.state is {version: 1, value: {count: 2}}.
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
The SDK never writes persistence itself. The host must store state under the
|
|
191
|
+
account, project, and Step instance; recheck the source, previous state, and write
|
|
192
|
+
permission; and save the returned state atomically with accepted results. A failed
|
|
193
|
+
or cancelled run must not overwrite prior state. Never share one state object
|
|
194
|
+
between placements, projects, accounts, or preview sessions. A mismatched state
|
|
195
|
+
version fails with `STATE_VERSION`; a host must not silently reinterpret it.
|
|
196
|
+
|
|
197
|
+
Every stateful fixture requires `expectedState` in addition to ordinary expected
|
|
198
|
+
outputs. Optional `state` supplies a saved starting value; omission uses a fresh
|
|
199
|
+
copy of `manifest.state.initial` for that fixture:
|
|
200
|
+
|
|
201
|
+
```json
|
|
202
|
+
{
|
|
203
|
+
"name": "Continue a saved count",
|
|
204
|
+
"input": {"amount": 3},
|
|
205
|
+
"state": {"version": 1, "value": {"count": 4}},
|
|
206
|
+
"expected": {"count": 7},
|
|
207
|
+
"expectedState": {"version": 1, "value": {"count": 7}}
|
|
208
|
+
}
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
`initialBlockState`, `validateBlockState`, and `STATE_LIMIT_BYTES` are exported by
|
|
212
|
+
the SDK. Both browser workers and Node's disposable QuickJS workers carry this
|
|
213
|
+
contract. Fixture success verifies state transformation, not host persistence.
|
|
214
|
+
|
|
215
|
+
## Troubleshooting
|
|
216
|
+
- STATE_REQUIRED: your external host did not supply an initial or accepted state envelope.
|
|
217
|
+
- STATE_VERSION: the saved version does not match the manifest; restore the previous build or explicitly reset memory. There is no automatic state migration.
|
|
218
|
+
- STATE_LIMIT: keep the JSON state value within 64 KiB.
|
|
219
|
+
- STATE_BEHAVIOR: output assertions passed but the fixture's expected state did not match.
|
|
220
|
+
- STATE_CHANGED: another edit or accepted run changed the memory; run again against current state.
|
|
221
|
+
|
|
222
|
+
## Test the actual behavior
|
|
223
|
+
Each fixture needs expectedState, even when the expected memory is unchanged. Include a fresh-state case and a continuation case. Fixture state is synthetic and never installed as project memory. Run details are available in the production workspace and preview without copying your content into diagnostic downloads.
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
# Test and preview a Block
|
|
2
|
+
|
|
3
|
+
## Fixtures are part of the package
|
|
4
|
+
Include 1–50 fixtures. Each names an input object and the complete expected output object. Defaults and optional ports still follow the declared contract.
|
|
5
|
+
```json
|
|
6
|
+
[
|
|
7
|
+
{
|
|
8
|
+
"name": "Counts words",
|
|
9
|
+
"input": {
|
|
10
|
+
"text": " Hello StillMade "
|
|
11
|
+
},
|
|
12
|
+
"expected": {
|
|
13
|
+
"text": "Hello StillMade",
|
|
14
|
+
"words": 2
|
|
15
|
+
}
|
|
16
|
+
},
|
|
17
|
+
{
|
|
18
|
+
"name": "Empty text",
|
|
19
|
+
"input": {
|
|
20
|
+
"text": " "
|
|
21
|
+
},
|
|
22
|
+
"expected": {
|
|
23
|
+
"text": "",
|
|
24
|
+
"words": 0
|
|
25
|
+
}
|
|
26
|
+
}
|
|
27
|
+
]
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Run the fixture suite
|
|
31
|
+
```sh
|
|
32
|
+
node packages/block-cli/cli.js test ./my-block
|
|
33
|
+
```
|
|
34
|
+
A test report includes `passed`, `digest`, `sdkVersion`, and individual results. Fixtures execute the real package runtime. Matching JSON output is required; approximate image similarity is not used.
|
|
35
|
+
|
|
36
|
+
## What to cover
|
|
37
|
+
- Ordinary input with a useful, observable result.
|
|
38
|
+
- Empty text or minimum-size images, when the contract allows them.
|
|
39
|
+
- Boundary values for controls.
|
|
40
|
+
- Behavior that preserves unrelated data, colors, or alpha.
|
|
41
|
+
- Values that the next Block will actually consume.
|
|
42
|
+
|
|
43
|
+
## Preview with your own sample
|
|
44
|
+
Save a JSON input object as `sample.json` and run:
|
|
45
|
+
```sh
|
|
46
|
+
node packages/block-cli/cli.js preview ./my-block ./sample.json
|
|
47
|
+
```
|
|
48
|
+
Use synthetic input in distributed packages. Private project material belongs in local samples, not published fixtures.
|
|
49
|
+
|
|
50
|
+
## Test offline with stand-ins
|
|
51
|
+
StillMade hosts part of what a Block does: hosted operations, saved files,
|
|
52
|
+
outside APIs and connections, cloud storage, project reads and assets. On your
|
|
53
|
+
own machine, `stillmade-block test`, `run` and `dev` answer these calls with
|
|
54
|
+
stand-ins:
|
|
55
|
+
|
|
56
|
+
- `--mock` turns them on. A folder with `tests/mocks.json` uses them every time,
|
|
57
|
+
including for `pack`, which runs the fixtures first.
|
|
58
|
+
- Every hosted operation has a recorded stand-in with the exact shape StillMade
|
|
59
|
+
returns, checked by the same validators the app uses. Text replies read
|
|
60
|
+
`Stand-in reply to: <prompt>`; images are 1024-pixel PNGs (or the requested
|
|
61
|
+
aspect ratio); speech, music and sound effects are short WAV tones; video is a
|
|
62
|
+
one-second MP4; transcripts, research, media analysis, timeline proposals and
|
|
63
|
+
JSON replies (sampled from the Block's schema) are small fixed results. Files
|
|
64
|
+
a stand-in or `files.write` makes can be read back with `files.read`.
|
|
65
|
+
- `tests/mocks.json` records your own answers. Every section is optional, and a
|
|
66
|
+
list answers successive calls in order (the last one repeats):
|
|
67
|
+
|
|
68
|
+
```json
|
|
69
|
+
{
|
|
70
|
+
"hosted": {"text.generate": ["First reply", "Second reply"]},
|
|
71
|
+
"actions": {"stock.search": {"results": []}},
|
|
72
|
+
"network": [{"method": "GET", "url": "https://api.github.com/repos/octocat/hello-world/issues*", "status": 200, "json": []}],
|
|
73
|
+
"connections": {"app:openapi:notion/retrievePage": {"properties": {}}},
|
|
74
|
+
"files": {"clip-1": "fixtures/clip.wav"},
|
|
75
|
+
"storage": {"settings": {"runs": 3}},
|
|
76
|
+
"project": {"schemaVersion": 1, "metadata": {"name": "Launch"}},
|
|
77
|
+
"transforms": {"video.cut": {"kind": "video", "assetId": "cut-1", "versionId": "v1", "url": "/api/media/stillmade-mock/cut.mp4", "mimeType": "video/mp4", "width": 320, "height": 180, "duration": 1, "bytes": 2048}}
|
|
78
|
+
}
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
| Section | Answers | Value |
|
|
82
|
+
| --- | --- | --- |
|
|
83
|
+
| `hosted` | `stillmade.hosted.run` and capability Blocks, by operation | The output value the call returns |
|
|
84
|
+
| `actions` | `stillmade.actions.run`, by operation | The action result (`{value, failures}` for `media.generate`) |
|
|
85
|
+
| `network` | `stillmade.net.fetch` | Recorded responses: method, url (a trailing `*` matches the rest), status, headers and one of `json`, `text` or `base64` |
|
|
86
|
+
| `connections` | `stillmade.connections.call`, by `appId/operation` | The JSON result |
|
|
87
|
+
| `files` | `files.read`, by asset ID or URL | A path inside the Block folder, or `{"text": "..."}` with a text file's contents |
|
|
88
|
+
| `storage` | `stillmade.storage` | Values stored before the run |
|
|
89
|
+
| `project` | `stillmade.project.read` | A project snapshot |
|
|
90
|
+
| `transforms` | `stillmade.media.transform`, by operation | The transform result (`audio.decode` on a WAV file decodes it for real) |
|
|
91
|
+
|
|
92
|
+
For a capability Block, fixture checks on the model's words (`includes`,
|
|
93
|
+
lengths, word counts) are listed as not checked rather than run against a
|
|
94
|
+
stand-in; StillMade runs them with a real model when you import the Block.
|
|
95
|
+
Media checks (sizes, durations, formats) still run. A run with stand-ins is
|
|
96
|
+
never a live verification: reports say `liveVerified: false` and list every
|
|
97
|
+
stand-in that answered. Nothing contacts a provider or spends credits.
|
|
98
|
+
|
|
99
|
+
The same stand-ins are available to your own tests from
|
|
100
|
+
`@stillmade/block-sdk/mock-host`: `createMockHost({mocks, readLocalFile,
|
|
101
|
+
manifest})` returns `host` (pass it to `createNodeModuleExecutor({files,
|
|
102
|
+
host})`), `capability` (pass it to `runPackageAsync` or `testPackage` for a
|
|
103
|
+
capability Block), `calls` (what answered each call) and `file(url)` (the bytes
|
|
104
|
+
of a stand-in or written file). `standInFixtures(pkg)` returns a capability
|
|
105
|
+
Block's fixtures with the checks on the model's words removed and lists them;
|
|
106
|
+
`validateMocks(value)` checks a `tests/mocks.json` value;
|
|
107
|
+
`sampleJsonSchema(schema)` makes a value that satisfies a JSON schema; and
|
|
108
|
+
`mockPng`, `mockWav` and `mockMp4` make the stand-in media. `MOCKS_FILE` is
|
|
109
|
+
`tests/mocks.json` and `MOCK_MEDIA_PREFIX` is the URL prefix of stand-in media.
|
|
110
|
+
|
|
111
|
+
## Admission and workflow tests
|
|
112
|
+
The app also scans source and permissions, repeats contract tests in its sandbox, runs a sample, validates outputs, and requires review before installation. After insertion, [rehearse the connected workflow](../project-types/testing.md); passing one Block’s fixtures alone does not establish workflow compatibility.
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
# Propose timeline edits
|
|
2
|
+
|
|
3
|
+
### Interactive timeline mixer
|
|
4
|
+
|
|
5
|
+
[Download Timeline mixer](https://stillmade.shop/block-sdk/examples/timeline-mixer.stillmade.json), or
|
|
6
|
+
choose **Timeline mixer template** in the Block builder. Its complete interface
|
|
7
|
+
lists video and audio clips with individual level sliders, live percentages,
|
|
8
|
+
reset, empty-state guidance, and a Review mix action. It uses `StillMade.render`
|
|
9
|
+
and `StillMade.onAction`; the isolated production function returns a typed edit
|
|
10
|
+
proposal. Existing Editor permissions, conflict checks, review, and undo still
|
|
11
|
+
apply. No account APIs or provider keys are exposed to its interface.
|
|
12
|
+
|
|
13
|
+
Import the package, run its fixture, and change a slider in Preview. To use real
|
|
14
|
+
clips, add it to a Project Type after Editor and open its Step in a project with
|
|
15
|
+
a timeline. Review the proposed levels before applying them. It edits up to 100
|
|
16
|
+
clips per invocation; the interface does not play or render the timeline.
|
|
17
|
+
Download its source and SDK from the builder to modify the HTML, CSS, interface
|
|
18
|
+
JavaScript, production function, and fixtures independently, then reimport.
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
A Block can return a `timeline-edit` output to propose changes to existing Editor
|
|
22
|
+
clips. Declare `permissions.project: ["context.timeline.read", "timeline.propose"]`
|
|
23
|
+
and an input `{type:"timeline",context:"timeline",primary:true}`. The output is
|
|
24
|
+
`{schemaVersion:1,title,timelineId,commands}`. Copy `timeline.activeTimelineId` into
|
|
25
|
+
`timelineId`. Each command is `{collection,operation,before,values}`: collection is
|
|
26
|
+
`clips` or `audioItems`, and `before` is the exact unchanged target object from the
|
|
27
|
+
input snapshot. Use at most 100 commands, with one command per target.
|
|
28
|
+
|
|
29
|
+
| Operation | Values | Behavior |
|
|
30
|
+
| --- | --- | --- |
|
|
31
|
+
| move | `{start: seconds}` | Move an existing item; no silent ripple |
|
|
32
|
+
| trim | `{trimStart: sourceSeconds, duration: timelineSeconds}` | Trim within known source duration; variable-speed, frozen and looping clips defer to Editor |
|
|
33
|
+
| split | `{at: timelineSeconds, newId: "unique-clip-id"}` | Split into two source-aligned clips, at least 0.05 seconds from either edge |
|
|
34
|
+
| volume | `{volume: 0..200}` | Set the existing Editor volume percentage |
|
|
35
|
+
| mute | `{muted: boolean}` | Mute or unmute an existing item |
|
|
36
|
+
| remove | `{}` | Remove the timeline instance, preserving source media |
|
|
37
|
+
| grade | `{lut, color}` | Grade clips only; retain original media and timing |
|
|
38
|
+
|
|
39
|
+
Run the Block in a project, accept its result, and open the Editor's media panel.
|
|
40
|
+
Timeline edits from Blocks displays a visual before/after strip and each target's values and applies
|
|
41
|
+
all commands together using ordinary Editor undo history. Desktop and mobile use
|
|
42
|
+
the same proposal contract. A changed target, different timeline version, locked
|
|
43
|
+
track, source overrun, or new clip overlap rejects the proposal. The Block cannot
|
|
44
|
+
set arbitrary fields, insert external URLs, run code inside playback, or silently
|
|
45
|
+
write the timeline. Re-run it against current context after a conflict.
|
|
46
|
+
|
|
47
|
+
[Download Quieter clips](https://stillmade.shop/block-sdk/examples/quieter-clips.stillmade.json) for a
|
|
48
|
+
complete isolated JavaScript example with a realistic timeline fixture. The
|
|
49
|
+
proposal type is a sink for Editor review; it is not interchangeable with a
|
|
50
|
+
`timeline` snapshot.
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
### Caption proposals
|
|
54
|
+
|
|
55
|
+
Use `collection:"captions"` with these commands:
|
|
56
|
+
|
|
57
|
+
| Operation | Before | Values |
|
|
58
|
+
| --- | --- | --- |
|
|
59
|
+
| caption.add | null | `{id,trackId,text,start,end}` |
|
|
60
|
+
| caption.update | Exact existing caption object | `{text,start,end}` |
|
|
61
|
+
| remove | Exact existing caption object | `{}` |
|
|
62
|
+
|
|
63
|
+
Caption times are seconds with end greater than start; text must be nonempty and
|
|
64
|
+
at most 5,000 characters. Adding uses an unused item ID and an unlocked existing
|
|
65
|
+
caption track. The standard `captions` track is created if absent. New captions
|
|
66
|
+
use StillMade's existing caption defaults; updating text or timing preserves the
|
|
67
|
+
caption's styling. Reapplying an insertion rejects its duplicate ID. The host
|
|
68
|
+
compares existing captions against their captured snapshots before update/removal.
|
|
69
|
+
|
|
70
|
+
[Download Caption from text](https://stillmade.shop/block-sdk/examples/caption-from-text.stillmade.json).
|
|
71
|
+
This complete Block takes text plus a scoped timeline, chooses an unused caption
|
|
72
|
+
ID, and proposes insertion without modifying the project. Its output connects to
|
|
73
|
+
the Editor's proposal review. Add or run it after opening a timeline, then review
|
|
74
|
+
its result in the Editor's media panel.
|
|
75
|
+
|
|
76
|
+
## Timed captions and caption styling
|
|
77
|
+
[Download Captions 2.0.0](https://stillmade.shop/block-sdk/examples/captions.stillmade-block). This
|
|
78
|
+
standalone, remixable Block converts a transcript into editable timeline captions.
|
|
79
|
+
It owns phrase grouping, word timing and caption appearance, runs in the JavaScript
|
|
80
|
+
sandbox, and returns ordinary `timeline-edit` proposals for review and Apply.
|
|
81
|
+
It does not transcribe audio or spend execution credits.
|
|
82
|
+
|
|
83
|
+
The optional context binding `{ "type": "transcript", "context": "transcript",
|
|
84
|
+
"required": false }` requires `context.transcript.read`. At whole-project scope
|
|
85
|
+
it supplies a copied `{schemaVersion:1, words:[{text,start,end}], text, audioUrl?}`
|
|
86
|
+
from the saved voiceover when word timings exist. It is omitted when absent and
|
|
87
|
+
for scene, shot or asset selections. URL metadata grants no media access.
|
|
88
|
+
Explicitly connected transcript inputs remain separate from this fallback.
|
|
89
|
+
|
|
90
|
+
Captions uses a connected transcript first, then entered text, then the project's
|
|
91
|
+
saved transcript. Text without word timings uses estimated timing, identified in
|
|
92
|
+
the proposal title. Start offsets word times into the timeline. Duration zero
|
|
93
|
+
keeps transcript timing; a positive duration clips the result to that interval.
|
|
94
|
+
A proposal supports at most 100 lines and rejects longer results without silently
|
|
95
|
+
truncating them. Existing captions stay intact and locked tracks reject Apply.
|
|
96
|
+
|
|
97
|
+
`caption.add` retains its required `{id,trackId,text,start,end}` values and also
|
|
98
|
+
accepts these optional fields. Unknown fields and values outside these limits
|
|
99
|
+
are rejected by the same SDK validator and host apply path:
|
|
100
|
+
|
|
101
|
+
| Field | Accepted values |
|
|
102
|
+
| --- | --- |
|
|
103
|
+
| `words` | Up to 1,000 `{text,start,end}` entries, ordered by start, contained in the caption interval; nonempty text up to 5,000 characters, end at or after start |
|
|
104
|
+
| `font` | `Hanken Grotesk` |
|
|
105
|
+
| `size` | 8–200 |
|
|
106
|
+
| `color`, `highlightColor` | Three- or six-digit hex color |
|
|
107
|
+
| `background` | `none` or three- or six-digit hex color |
|
|
108
|
+
| `backgroundOpacity`, `backgroundPadding`, `backgroundRadius` | 0–100 |
|
|
109
|
+
| `position` | `top`, `middle`, or `bottom` followed by `left`, `center`, or `right`, separated by one space |
|
|
110
|
+
| `animation` | `none` or `fade` |
|
|
111
|
+
|
|
112
|
+
The host fills omitted appearance values with its existing defaults, preserves
|
|
113
|
+
provided styling and clones word timings before saving through ordinary edit
|
|
114
|
+
history. Caption updates retain their existing `{text,start,end}` contract.
|
|
115
|
+
Package source, MIT notices, lineage and three fixtures are retained in the
|
|
116
|
+
canonical archive; validate, test and remix it using the public CLI.
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
# Hosted timeline proposals
|
|
2
|
+
|
|
3
|
+
# Hosted timeline proposals
|
|
4
|
+
|
|
5
|
+
Use `timeline.propose` when a Block needs an LLM to choose reviewable edits from the current timeline. The Block declares the editing policy and typed context. StillMade owns the provider connection, model selection, payment review, exact execution receipt, output validation, preview, and Apply action.
|
|
6
|
+
|
|
7
|
+
This capability never gives package code access to project storage, provider credentials, or an unrestricted editor API. Its only accepted output is the public `timeline-edit` schema. The host compares each command's `before` value with the current project before applying it through ordinary permissions and undo history.
|
|
8
|
+
|
|
9
|
+
## Manifest
|
|
10
|
+
|
|
11
|
+
The manifest must use runtime `capability`, entry `src/capability.json`, exactly one `timeline-edit` output, and one to four inputs. At least one input must be a `timeline`. A project-bound timeline input declares `context: "timeline"`. The Block must declare both `context.timeline.read` and `timeline.propose`, plus the single capability permission `timeline.propose`.
|
|
12
|
+
|
|
13
|
+
```json
|
|
14
|
+
{
|
|
15
|
+
"runtime": "capability",
|
|
16
|
+
"entry": "src/capability.json",
|
|
17
|
+
"inputs": {
|
|
18
|
+
"timeline": {"type":"timeline","context":"timeline","primary":true},
|
|
19
|
+
"direction": {"type":"text","default":"Polish this cut conservatively."}
|
|
20
|
+
},
|
|
21
|
+
"outputs": {"edit":{"type":"timeline-edit","primary":true}},
|
|
22
|
+
"permissions": {
|
|
23
|
+
"project":["context.timeline.read","timeline.propose"],
|
|
24
|
+
"capabilities":["timeline.propose"],
|
|
25
|
+
"network":[],"filesystem":[],"secrets":[]
|
|
26
|
+
}
|
|
27
|
+
}
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Capability descriptor
|
|
31
|
+
|
|
32
|
+
`src/capability.json` contains exactly `schemaVersion`, `operation`, `context`, `instruction`, `maxTokens`, and `output`.
|
|
33
|
+
|
|
34
|
+
```json
|
|
35
|
+
{
|
|
36
|
+
"schemaVersion": 1,
|
|
37
|
+
"operation": "timeline.propose",
|
|
38
|
+
"context": {
|
|
39
|
+
"timeline": {"$input":"timeline"},
|
|
40
|
+
"direction": {"$input":"direction"}
|
|
41
|
+
},
|
|
42
|
+
"instruction": "Use exact before snapshots, preserve locked items, and return strict JSON only.",
|
|
43
|
+
"maxTokens": 2000,
|
|
44
|
+
"output": "edit"
|
|
45
|
+
}
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
`context` must bind every declared input exactly once. Instructions contain 1–6,000 characters. `maxTokens` is 1–4,096. Bound context is inert JSON and must stay under 24,000 encoded bytes so the exact reviewed request remains bounded.
|
|
49
|
+
|
|
50
|
+
The provider must return one strict JSON object with the public timeline-edit shape:
|
|
51
|
+
|
|
52
|
+
```json
|
|
53
|
+
{
|
|
54
|
+
"schemaVersion": 1,
|
|
55
|
+
"title": "Polish the cut",
|
|
56
|
+
"timelineId": "timeline-1",
|
|
57
|
+
"commands": [
|
|
58
|
+
{
|
|
59
|
+
"collection": "clips",
|
|
60
|
+
"operation": "transition",
|
|
61
|
+
"before": {"id":"clip-1","trackId":"video","start":0,"duration":4},
|
|
62
|
+
"values": {"type":"Cross dissolve","duration":0.35,"soundOn":false,"soundId":""}
|
|
63
|
+
}
|
|
64
|
+
]
|
|
65
|
+
}
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
Supported commands and their exact value fields are defined by `packages/block-sdk/timeline-edits.js`. Outputs with prose, code fences, unknown fields, unsupported commands, duplicate targets, more than 100 commands, invalid timing, or malformed `before` values fail before preview or Apply.
|
|
69
|
+
|
|
70
|
+
Fixtures use nondeterministic expectations:
|
|
71
|
+
|
|
72
|
+
```json
|
|
73
|
+
{"edit":{"kind":"timeline-edit","minCommands":1,"maxCommands":10}}
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Offline validation checks the contract, source, permissions, interface, fixtures, and remix readiness. It reports live execution as review-required. In StillMade, import and execution require the normal model/payment review. A generated proposal is shown to the user and never changes the timeline until Apply.
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# Hosted audio transcription
|
|
2
|
+
|
|
3
|
+
# Hosted audio transcription
|
|
4
|
+
|
|
5
|
+
Use `runtime: "capability"` and `operation: "audio.transcribe"` to turn one persistent StillMade audio asset into reviewed text with word-level timings. The Block declares the work; the trusted host reads the selected owner-scoped file and performs the provider request. Package code and custom interfaces never receive an API key, filesystem path, network access, or provider response outside the typed result.
|
|
6
|
+
|
|
7
|
+
## Complete contract
|
|
8
|
+
|
|
9
|
+
The manifest must declare exactly one `audio` input, one `text` language input, and one `transcript` output. Include an asset control so the Block can be used directly, and give language a default such as `auto`.
|
|
10
|
+
|
|
11
|
+
```json
|
|
12
|
+
{
|
|
13
|
+
"manifest": {
|
|
14
|
+
"schemaVersion":1,"sdkVersion":"0.1.0",
|
|
15
|
+
"id":"example.transcribe-audio","version":"1.0.0",
|
|
16
|
+
"name":"Transcribe audio","description":"Turn one stored audio file into reviewed text with word timings.",
|
|
17
|
+
"kind":"task","runtime":"capability","entry":"src/capability.json","license":"MIT",
|
|
18
|
+
"provenance":{"notice":"Include the complete applicable license notice."},
|
|
19
|
+
"inputs":{"audio":{"type":"audio","primary":true},"language":{"type":"text","default":"auto"}},
|
|
20
|
+
"outputs":{"transcript":{"type":"transcript","primary":true}},
|
|
21
|
+
"permissions":{"project":[],"capabilities":["audio.transcribe"],"network":[],"filesystem":[],"secrets":[]},
|
|
22
|
+
"ui":[{"control":"asset","port":"audio"},{"control":"select","port":"language","options":["auto","en","es","fr","de","it","pt","ja","ko","zh"]}]
|
|
23
|
+
},
|
|
24
|
+
"capability":{"schemaVersion":1,"operation":"audio.transcribe","audio":{"$input":"audio"},"language":{"$input":"language"},"output":"transcript"},
|
|
25
|
+
"tests":[{"name":"English speech","input":{"audio":{"kind":"audio","assetId":"fixture-audio","versionId":"version-1","url":"/api/media/fixture/audio.mp3","mimeType":"audio/mpeg"},"language":"en"},"expectations":{"transcript":{"kind":"transcript","minWords":1,"maxWords":5000}}}]
|
|
26
|
+
}
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
The capability descriptor has exactly `schemaVersion`, `operation`, `audio`, `language`, and `output`. Both inputs use exact `$input` bindings. The output names the single declared transcript port. Language is `auto` or a lowercase two-letter ISO 639-1 code. The audio value must be a persistent typed reference with `kind:"audio"`, nonblank `assetId`, `versionId`, and `url`; an in-memory blob or arbitrary remote URL is not sufficient.
|
|
30
|
+
|
|
31
|
+
The result is exactly:
|
|
32
|
+
|
|
33
|
+
```json
|
|
34
|
+
{"schemaVersion":1,"text":"Hello world.","words":[{"text":"Hello","start":0,"end":0.4},{"text":"world.","start":0.4,"end":0.9}]}
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Times are finite seconds in ascending word order, from 0 through 86,400. Results allow at most 100,000 words and one million transcript characters. Fixture expectations contain exactly `kind:"transcript"`, `minWords`, and `maxWords`; they test useful bounds rather than invented provider text.
|
|
38
|
+
|
|
39
|
+
## Host execution and review
|
|
40
|
+
|
|
41
|
+
The host offers two selections. **StillMade transcription** runs the app's own transcription (Groq Whisper Large v3 Turbo, with fal Whisper fallbacks) and is paid with StillMade credits at the app's upload-transcription price, quoted per fixture and sample before anything runs. **OpenAI Whisper** (`whisper-1`) uses the account owner's connected OpenAI key, spends zero StillMade credits, and OpenAI bills the connected account directly. Both accept supported audio up to 25 MB, request word timestamps, validate the response size and shape, and return only the normalized transcript. Uploading, validating, quoting, or previewing the package does not send audio.
|
|
42
|
+
|
|
43
|
+
Before dispatch, StillMade binds the package source and version, exact inputs, owner, project, provider, model, payment method, and zero-credit quote. It checks project access and key availability, then checks account suspension again immediately before reading media and before provider dispatch. It reads only an owner-scoped StillMade media key. An uncertain provider submission is not automatically repeated, avoiding duplicate external charges.
|
|
44
|
+
|
|
45
|
+
Offline validation does not need a key and cannot claim a live transcription. Import is Upload → validation and security/license checks → fixture/sample review → Install. In a project, the user chooses an audio asset and language, reviews the request, runs it, and accepts the timed transcript. The output can connect to any compatible `transcript` input.
|
|
46
|
+
|
|
47
|
+
## Package and validate
|
|
48
|
+
|
|
49
|
+
The downloaded SDK includes `packages/block-sdk/transcription-example.js` and `examples/transcribe-audio.stillmade.json`. Use your own namespace and include the real license and provenance for adapted source.
|
|
50
|
+
|
|
51
|
+
```sh
|
|
52
|
+
node packages/block-cli/cli.js validate examples/transcribe-audio.stillmade.json
|
|
53
|
+
node packages/block-cli/cli.js pack examples/transcribe-audio.stillmade.json
|
|
54
|
+
node packages/block-cli/cli.js validate example.transcribe-audio-1.0.0.stillmade.json
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
`pack` writes `<id>-<version>.stillmade.json`, the file to import in StillMade (Create, then Import Block).
|
|
58
|
+
|
|
59
|
+
Before returning an artifact, confirm that it has one exact operation, usable generated controls, realistic fixtures, complete attribution, no embedded credentials or provider choices, no ambient permissions, and passes the same packaged validator used by StillMade.
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# Triggers and unattended runs
|
|
2
|
+
|
|
3
|
+
A Block can declare events it runs on by itself. Declaring a trigger runs
|
|
4
|
+
nothing: when the Block is placed in a project, the project owner reviews each
|
|
5
|
+
trigger and turns it on for that Step, with the same reviews as the automation
|
|
6
|
+
panels. Results wait for the owner to review before they are used.
|
|
7
|
+
|
|
8
|
+
```json
|
|
9
|
+
{"triggers": [
|
|
10
|
+
{"event": "media.saved", "input": "video", "kinds": ["video"], "label": "When a new video is saved"},
|
|
11
|
+
{"event": "schedule", "everySeconds": 86400},
|
|
12
|
+
{"event": "webhook", "eventKind": "order.created"}
|
|
13
|
+
]}
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
| Event | Fields | Runs when |
|
|
17
|
+
| --- | --- | --- |
|
|
18
|
+
| `media.saved` | `input` (the primary image, video, audio or asset input), optional `kinds` | New media is saved to the project (checked about every minute). Existing media is not processed. |
|
|
19
|
+
| `schedule` | `everySeconds` (60 to 31,536,000) | On the interval, starting a minute after the owner turns it on. Missed runs are skipped. |
|
|
20
|
+
| `webhook` | optional `eventKind` | Another service sends `POST /api/workflow-trigger-events/<id>` with the key the owner receives once (`Authorization: Bearer <key>`) and a JSON event `{id, kind, occurredAt, data}` up to 64 KB. Repeated event IDs run once. |
|
|
21
|
+
| `project.completed` | none | Another project the owner chooses is marked complete. |
|
|
22
|
+
| `provider` | `appId`, `eventKind` | A connected app reports the event; the owner connects the account. |
|
|
23
|
+
|
|
24
|
+
Up to 4 triggers per Block. A webhook, schedule or provider Block receives the
|
|
25
|
+
event through an input with `"semantic": "workflow_event"` and
|
|
26
|
+
`"sources": ["project"]`.
|
|
27
|
+
|
|
28
|
+
Triggers work for recipe, JavaScript and hosted (`runtime: "capability"`)
|
|
29
|
+
Blocks. Module Blocks run only on the person's device, so they cannot declare
|
|
30
|
+
triggers of their own; a schedule or event can hand their Steps to the owner's
|
|
31
|
+
StillMade Desktop (see Module Blocks). Sandboxed code has 25 seconds per run.
|
|
32
|
+
|
|
33
|
+
### Hosted Blocks run inside an approved budget
|
|
34
|
+
|
|
35
|
+
A hosted Block (generation, analysis, speech, transcription, web reading) runs
|
|
36
|
+
unattended only after the owner approves it for that Step:
|
|
37
|
+
the model and payment, credits per run, total credits, number of runs and how
|
|
38
|
+
many days the approval lasts. The approval reuses the owner's earlier review of
|
|
39
|
+
the Block with that model, so run the Step once yourself first. Each automatic
|
|
40
|
+
run:
|
|
41
|
+
|
|
42
|
+
- claims its credits on the approval before anything is sent, so the total is
|
|
43
|
+
never exceeded, and stops and waits for the owner if the price rises above the
|
|
44
|
+
per-run ceiling or the approval is used up, revoked or expired;
|
|
45
|
+
- runs as one job with one workflow budget, and is never paid twice: a resumed
|
|
46
|
+
or retried job polls the same hosted request;
|
|
47
|
+
- can take longer than the sandbox's 25 seconds. A long call (for example video
|
|
48
|
+
generation) puts the job in a timed wait and StillMade polls it until it
|
|
49
|
+
finishes. Cancelling the job cancels the hosted call.
|
|
50
|
+
|
|
51
|
+
Timeline and workspace proposals never run unattended; they always wait for a
|
|
52
|
+
person.
|
|
53
|
+
|
|
54
|
+
`autoCaptions` and `orderNote` in `@stillmade/block-sdk` are complete examples:
|
|
55
|
+
transcription of each new video into timed captions (`media.saved`), and a
|
|
56
|
+
JavaScript Block that turns a shop's webhook into a production note. The SDK
|
|
57
|
+
helpers `BLOCK_TRIGGER_EVENTS`, `BLOCK_TRIGGER_LIMITS`, `UNATTENDED_RUNTIMES`,
|
|
58
|
+
`triggerProblems`, `blockTriggers`, `describeTrigger` and `triggerReview`
|
|
59
|
+
(what the owner reviews for each trigger) check and describe declarations.
|