@nodetool-ai/agents 0.8.4-a24.2 → 0.8.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/.nodetool-build-stamp +1 -1
- package/dist/_skills/LICENSE-iart-motion-design-skills +21 -0
- package/dist/_skills/README.md +211 -0
- package/dist/_skills/api-agents/SKILL.md +58 -0
- package/dist/_skills/api-apps/SKILL.md +54 -0
- package/dist/_skills/api-assets/SKILL.md +66 -0
- package/dist/_skills/api-collections/SKILL.md +33 -0
- package/dist/_skills/api-games/SKILL.md +81 -0
- package/dist/_skills/api-media/SKILL.md +131 -0
- package/dist/_skills/api-memory/SKILL.md +72 -0
- package/dist/_skills/api-models/SKILL.md +78 -0
- package/dist/_skills/api-scripts/SKILL.md +79 -0
- package/dist/_skills/api-settings/SKILL.md +49 -0
- package/dist/_skills/api-sketches/SKILL.md +61 -0
- package/dist/_skills/api-storyboards/SKILL.md +94 -0
- package/dist/_skills/api-timelines/SKILL.md +151 -0
- package/dist/_skills/api-web/SKILL.md +57 -0
- package/dist/_skills/api-workflows/SKILL.md +123 -0
- package/dist/_skills/beat-sync-editing/SKILL.md +208 -0
- package/dist/_skills/caption-titles/SKILL.md +94 -0
- package/dist/_skills/color-motion/SKILL.md +185 -0
- package/dist/_skills/commercial-beat-sheet/SKILL.md +338 -0
- package/dist/_skills/elevenlabs-audio-prompting/SKILL.md +163 -0
- package/dist/_skills/explainer-storyboard/SKILL.md +63 -0
- package/dist/_skills/flux-2-klein-prompting/SKILL.md +107 -0
- package/dist/_skills/frame-composition/SKILL.md +291 -0
- package/dist/_skills/game-direction/SKILL.md +99 -0
- package/dist/_skills/gpt-image-2-prompting/SKILL.md +161 -0
- package/dist/_skills/hailuo-prompting/SKILL.md +107 -0
- package/dist/_skills/kling-video-prompting/SKILL.md +188 -0
- package/dist/_skills/launch-commercial/SKILL.md +267 -0
- package/dist/_skills/launch-kit/SKILL.md +120 -0
- package/dist/_skills/logo-reveal/SKILL.md +225 -0
- package/dist/_skills/manimgx/SKILL.md +123 -0
- package/dist/_skills/minimax-h3-prompting/SKILL.md +134 -0
- package/dist/_skills/motion-ad/SKILL.md +281 -0
- package/dist/_skills/motion-background/SKILL.md +145 -0
- package/dist/_skills/motion-curves/SKILL.md +302 -0
- package/dist/_skills/motion-direction/SKILL.md +259 -0
- package/dist/_skills/motion-graphics/SKILL.md +217 -0
- package/dist/_skills/motion-principles/SKILL.md +255 -0
- package/dist/_skills/music-video-treatment/SKILL.md +40 -0
- package/dist/_skills/nano-banana-pro-prompting/SKILL.md +133 -0
- package/dist/_skills/native-game/SKILL.md +310 -0
- package/dist/_skills/native-game-reference/SKILL.md +879 -0
- package/dist/_skills/nodetool-3d-scene/SKILL.md +110 -0
- package/dist/_skills/nodetool-api-reference/SKILL.md +338 -0
- package/dist/_skills/nodetool-app-builder/SKILL.md +291 -0
- package/dist/_skills/nodetool-browser-agent/SKILL.md +119 -0
- package/dist/_skills/nodetool-chat-cli/SKILL.md +184 -0
- package/dist/_skills/nodetool-custom-node-developer/SKILL.md +294 -0
- package/dist/_skills/nodetool-deployment/SKILL.md +195 -0
- package/dist/_skills/nodetool-js-scripting/SKILL.md +223 -0
- package/dist/_skills/nodetool-model-provider-config/SKILL.md +348 -0
- package/dist/_skills/nodetool-rag-indexing/SKILL.md +153 -0
- package/dist/_skills/nodetool-sketch/SKILL.md +138 -0
- package/dist/_skills/nodetool-skill-author/SKILL.md +125 -0
- package/dist/_skills/nodetool-troubleshooter/SKILL.md +370 -0
- package/dist/_skills/nodetool-video-post/SKILL.md +149 -0
- package/dist/_skills/nodetool-workflow-builder/SKILL.md +295 -0
- package/dist/_skills/product-commercial/SKILL.md +132 -0
- package/dist/_skills/qwen-image-prompting/SKILL.md +122 -0
- package/dist/_skills/script-video/SKILL.md +106 -0
- package/dist/_skills/seedance-2-prompting/SKILL.md +194 -0
- package/dist/_skills/seedream-prompting/SKILL.md +117 -0
- package/dist/_skills/short-film/SKILL.md +123 -0
- package/dist/_skills/stable-audio-prompting/SKILL.md +130 -0
- package/dist/_skills/storyboard-core/SKILL.md +716 -0
- package/dist/_skills/timeline-edit-ops/SKILL.md +736 -0
- package/dist/_skills/trailer-template/SKILL.md +291 -0
- package/dist/_skills/ugc-video/SKILL.md +186 -0
- package/dist/_skills/veo-3-prompting/SKILL.md +140 -0
- package/dist/_skills/video-audio-continuity/SKILL.md +90 -0
- package/dist/_skills/video-clone/SKILL.md +96 -0
- package/dist/_skills/video-workflow/SKILL.md +119 -0
- package/dist/_skills/wan-2-6-prompting/SKILL.md +107 -0
- package/dist/app-build/bridge.d.ts.map +1 -1
- package/dist/app-build/bridge.js +44 -17
- package/dist/app-build/bridge.js.map +1 -1
- package/dist/capabilities/apps.d.ts.map +1 -1
- package/dist/capabilities/apps.js +55 -13
- package/dist/capabilities/apps.js.map +1 -1
- package/dist/capabilities/apps.specs.d.ts.map +1 -1
- package/dist/capabilities/apps.specs.js +5 -3
- package/dist/capabilities/apps.specs.js.map +1 -1
- package/dist/capabilities/game-asset-browser.d.ts.map +1 -1
- package/dist/capabilities/game-asset-browser.js +6 -1
- package/dist/capabilities/game-asset-browser.js.map +1 -1
- package/dist/capabilities/game.d.ts +1 -1
- package/dist/capabilities/game.d.ts.map +1 -1
- package/dist/capabilities/game.js +71 -56
- package/dist/capabilities/game.js.map +1 -1
- package/dist/capabilities/game.specs.js +1 -1
- package/dist/capabilities/game.specs.js.map +1 -1
- package/dist/capabilities/game3d.d.ts.map +1 -1
- package/dist/capabilities/game3d.js +14 -19
- package/dist/capabilities/game3d.js.map +1 -1
- package/dist/capabilities/gameAssertions.d.ts +52 -0
- package/dist/capabilities/gameAssertions.d.ts.map +1 -0
- package/dist/capabilities/gameAssertions.js +32 -0
- package/dist/capabilities/gameAssertions.js.map +1 -0
- package/dist/capabilities/models.d.ts.map +1 -1
- package/dist/capabilities/models.js +9 -0
- package/dist/capabilities/models.js.map +1 -1
- package/dist/capabilities/storyboards.d.ts.map +1 -1
- package/dist/capabilities/storyboards.js +36 -11
- package/dist/capabilities/storyboards.js.map +1 -1
- package/dist/capabilities/storyboards.specs.d.ts.map +1 -1
- package/dist/capabilities/storyboards.specs.js +11 -1
- package/dist/capabilities/storyboards.specs.js.map +1 -1
- package/dist/capabilities/workflows.d.ts.map +1 -1
- package/dist/capabilities/workflows.js +1 -0
- package/dist/capabilities/workflows.js.map +1 -1
- package/dist/capabilities/workflows.specs.d.ts.map +1 -1
- package/dist/capabilities/workflows.specs.js +4 -3
- package/dist/capabilities/workflows.specs.js.map +1 -1
- package/dist/evals/surfaces/game.d.ts.map +1 -1
- package/dist/evals/surfaces/game.js +29 -1
- package/dist/evals/surfaces/game.js.map +1 -1
- package/dist/game-authoring.d.ts +120 -0
- package/dist/game-authoring.d.ts.map +1 -1
- package/dist/game-build-integrity.d.ts +12 -0
- package/dist/game-build-integrity.d.ts.map +1 -0
- package/dist/game-build-integrity.js +78 -0
- package/dist/game-build-integrity.js.map +1 -0
- package/dist/game-examples.d.ts +44 -3
- package/dist/game-examples.d.ts.map +1 -1
- package/dist/game-examples.js +60 -10
- package/dist/game-examples.js.map +1 -1
- package/dist/system-skills.d.ts +1 -1
- package/dist/system-skills.js +1 -1
- package/dist/tools/builtin-tools.d.ts.map +1 -1
- package/dist/tools/builtin-tools.js +2 -0
- package/dist/tools/builtin-tools.js.map +1 -1
- package/package.json +19 -19
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: api-timelines
|
|
3
|
+
description: "Call nodetool.timelines from a code action: create, read, edit or write a whole timeline, study the shipped example timelines, validate at the showcase tier, preview composited frames and contact sheets, compare versions, render the cut, manage versions and insert reusable compositions. Load before the first call into nodetool.timelines."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# nodetool.timelines
|
|
7
|
+
|
|
8
|
+
A timeline is a sequence: tracks of clips — media, text, shapes, groups, 3D
|
|
9
|
+
models and MIDI — with animations, transitions, masks, mattes, effects and
|
|
10
|
+
markers. This skill is the call reference. The `edit` op contract, the
|
|
11
|
+
animation roles and the preset catalog are `timeline-edit-ops`. The build
|
|
12
|
+
loop and the choice between the code pack and edit ops are `motion-graphics`.
|
|
13
|
+
The craft is in `motion-direction`, `motion-principles`, `motion-curves`,
|
|
14
|
+
`frame-composition`, `color-motion` and `beat-sync-editing`.
|
|
15
|
+
|
|
16
|
+
## Calls
|
|
17
|
+
|
|
18
|
+
| Call | Does | Answers |
|
|
19
|
+
| :--- | :--- | :--- |
|
|
20
|
+
| `list({query, limit})` | Lists sequences, newest first. | `{timelines}` |
|
|
21
|
+
| `create(name, {fps, width, height, project_id})` | Creates an empty sequence. The defaults are 30 fps and 1920×1080. Vertical is 1080×1920. | `{timeline_id, …}` |
|
|
22
|
+
| `get(id)` | Reads the stored document: fps, size, duration, tracks, clips, markers. | The document with its metadata |
|
|
23
|
+
| `edit(id, ops)` | Applies up to 60 ops in order and saves. | `{applied, failed, ops, tracks, clips}`. Each entry of `ops` has its own `ok`. |
|
|
24
|
+
| `setDocument(id, document, {fps, width, height, expected_updated_at, snapshot_name})` | Writes a whole document. It is validated first and snapshotted before the write. | The validation of what landed |
|
|
25
|
+
| `validate(idOrDocument, {tier, normalize, fps, width, height})` | Checks the structure without a render. | `{ok, errors, warnings, summary}` |
|
|
26
|
+
| `preview(idOrDocument, opts)` | Composites real frames. | Frames with image handles and layer reports, or one contact sheet |
|
|
27
|
+
| `compare(a, b, {times_ms, range, width})` | Differences the pixels of two sides — each an id, `{timeline_id, version}` or a document; never a shipped-example slug. | A difference from 0 to 1 per frame, and a side-by-side sheet |
|
|
28
|
+
| `render(id, opts)` | Renders the cut to a video file as a job. | `{job_id}`, or the asset with `wait: true` |
|
|
29
|
+
| `versions(id, {save_type, limit})`, `getVersion(id, n)` | Lists and reads snapshots. | Version rows, newest first |
|
|
30
|
+
| `snapshot(id, {name})` (also `createVersion`) | Saves a manual version. Manual versions are never pruned. | The version number |
|
|
31
|
+
| `restore(id, n)` | Rolls back. The state it replaces is snapshotted first, and the result is validated. | The findings |
|
|
32
|
+
| `deleteVersion(id, n)` | Deletes one snapshot. This cannot be undone. | — |
|
|
33
|
+
| `examples.list({query})` | Lists the shipped example timelines. | Slugs, stats, poster and video locators |
|
|
34
|
+
| `examples.get(slug, {scene_id, clip_offset, clip_limit})` | Reads one example scene. | Metadata, a scene catalog and a bounded excerpt |
|
|
35
|
+
| `examples.source(slug)` | Reads the builder script that made an example. | Full JavaScript, imports included |
|
|
36
|
+
| `compositions.list({source, query, limit})`, `compositions.get(id)` | Lists and reads templates: title card, lower third, caption bar, callout, CTA end card, logo sting, and the user's own. | Rows with their parameters |
|
|
37
|
+
| `compositions.save(timelineId, groupTarget, name, params, {description})` | Saves a group as a template. | The composition id |
|
|
38
|
+
| `compositions.remove(id)` | Deletes a saved template. | — |
|
|
39
|
+
| `code.get(id)` | Reads the timeline's authoring code, when it has any. | `{code, baked_at, scenes: [{name, group_id, edited}]}` — `code` is `null` for a timeline with none |
|
|
40
|
+
| `code.set(id, code, {force, require_match})` | Bakes `code` and merges it into the document, scene by scene: an untouched scene is replaced, a hand-edited one is kept and reported in `conflicts` unless `force` overwrites it (`true` for every scene, or an array of names). The bake runs no capability, so `code` must build the timeline from values alone. `require_match: true` attaches the code only if it bakes to exactly the saved document, which is what `v.save()` does. | `{timeline_id, errors, warnings, conflicts, scenes}`, plus `embedded` with `require_match` |
|
|
41
|
+
| `code.edit(id, edits, {force})` | Exact-match string replacements against the stored code (each `old` must match exactly once), then bakes and merges the same way as `set`. The small-diff path for one change. | Same as `set` |
|
|
42
|
+
| `code.rebake(id, {force})` | Reruns the stored code unchanged and merges again — for picking up a shipped example's own current script, or re-applying `force` to a scene left conflicted. | Same as `set` |
|
|
43
|
+
| `code.detach(id, scenes)` | Stops tracking scenes (`scenes` is `"all"` or an array of names) — a later rebake never touches them again. | `{timeline_id, scenes}` |
|
|
44
|
+
|
|
45
|
+
### Changing a code-backed timeline
|
|
46
|
+
|
|
47
|
+
For any timeline that already has `code` (`code.get(id).code` is not null),
|
|
48
|
+
the path is: `code.get(id)` → `code.edit(id, edits)` with a few small exact
|
|
49
|
+
replacements → read the `conflicts`/`warnings` it answers → `preview(id,
|
|
50
|
+
{sheet: true})` and look. `edit_timeline` ops still apply, for a timeline with
|
|
51
|
+
no code and for a fine hand tweak on a code-backed one — but a hand edit
|
|
52
|
+
inside a scene the code tracks makes that scene "edited": the next
|
|
53
|
+
`code.set`/`code.edit`/`code.rebake` reports it in `conflicts` and keeps the
|
|
54
|
+
hand-edited version rather than silently overwriting it. Pass `force` (`true`,
|
|
55
|
+
or the scene's name in an array) to overwrite it with the code's own version
|
|
56
|
+
instead of keeping the edit. A new timeline: build it and call `save()` — it
|
|
57
|
+
attaches the retained program, the scene code with every research or
|
|
58
|
+
generation result as a literal, so the timeline is already code-backed and a
|
|
59
|
+
rebake repeats no call. Read `saved.code.warnings` when `embedded` is false.
|
|
60
|
+
Otherwise, or to attach code after the fact, `create`, then `code.set`.
|
|
61
|
+
|
|
62
|
+
## Coordinates and units
|
|
63
|
+
|
|
64
|
+
- `transform.position.x/y` is in sequence pixels from the frame **centre**.
|
|
65
|
+
Positive x moves right and positive y moves down. This holds for every
|
|
66
|
+
clip, text and shapes included.
|
|
67
|
+
- Shape geometry `shape.x/y/width/height` is in 0..1 fractions of the frame
|
|
68
|
+
from the **top-left**. It is not the transform position.
|
|
69
|
+
- Masks use the 0..1 space of the clip itself.
|
|
70
|
+
- Track order is z-order: index 0 draws on top. `add_track` appends to the
|
|
71
|
+
bottom, so add the overlays last or fix the order with `move_track`.
|
|
72
|
+
- Times are in milliseconds. Custom-curve keyframes use `t` from 0 to 1 over
|
|
73
|
+
the window of the animation.
|
|
74
|
+
|
|
75
|
+
## Three ways to build
|
|
76
|
+
|
|
77
|
+
1. **In code, for motion graphics.** Import `@nodetool-ai/sandbox-timeline`:
|
|
78
|
+
`const v = video({...})` builds scenes of shapes, text and images,
|
|
79
|
+
keyframed in frames, and `v.save(nodetool.timelines)` saves them in one
|
|
80
|
+
call. Read `nodetool.packs.docs("@nodetool-ai/sandbox-timeline")`
|
|
81
|
+
first. This is the path for a dense, layered piece.
|
|
82
|
+
2. **`setDocument`**, for a whole document you already hold. It replaces the
|
|
83
|
+
stored document field for field, so read `get(id)` and send it back
|
|
84
|
+
changed. Missing bookkeeping is filled in: track `index`, `visible`,
|
|
85
|
+
`locked`; clip `sourceType`, `status`, `locked`, `versions`; animation and
|
|
86
|
+
effect ids; `markers`. Pass `expected_updated_at` to refuse the write when
|
|
87
|
+
someone changed the timeline after you read it.
|
|
88
|
+
3. **`edit(id, ops)`**, for targeted changes. Start with `{op: "get_state"}` to
|
|
89
|
+
read the ids. Check `failed` and the `ok` of each entry in `ops` before a
|
|
90
|
+
dependent edit. A batch
|
|
91
|
+
can save its good ops even when a later one fails.
|
|
92
|
+
|
|
93
|
+
```js
|
|
94
|
+
await nodetool.timelines.edit(id, [
|
|
95
|
+
{ op: "add_track", type: "audio", name: "Music" },
|
|
96
|
+
{ op: "add_text_clip", text: "Hi", startMs: 0, durationMs: 2000 },
|
|
97
|
+
{ op: "animate_clip", target: "Hi", animations: [{ role: "in", preset: "fade" }] },
|
|
98
|
+
{ op: "set_effects", target: "Hi", effects: [{ type: "blur", radius: 8 }] }
|
|
99
|
+
]);
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Existing media goes on with `{op: "add_media_clip", asset: "asset://<id>.mp4"}`,
|
|
103
|
+
which appends after the last clip on the track. One call for each asset lays
|
|
104
|
+
them end to end. `set_effects` replaces the whole chain, and an unknown effect
|
|
105
|
+
parameter is refused.
|
|
106
|
+
|
|
107
|
+
## The loop: validate, look, fix
|
|
108
|
+
|
|
109
|
+
1. **Study an example** before a showcase piece. `examples.list()`, then
|
|
110
|
+
`examples.get(slug)` for the scene catalog, then
|
|
111
|
+
`examples.get(slug, {scene_id})` for one scene. Page long scenes with
|
|
112
|
+
`clip_offset: next_clip_offset` until `truncated` is false. Return only
|
|
113
|
+
the fields you study; the excerpts are large.
|
|
114
|
+
2. **Validate.** `validate(id, {tier: "showcase"})` adds warnings about the
|
|
115
|
+
count of scene groups, custom animations per visible second, effect
|
|
116
|
+
variety, clip density, fast motion without blur, the finish, camera depth
|
|
117
|
+
and concurrent layers. Each warning names an example. Warnings do not
|
|
118
|
+
change `ok`. For an inline document, pass `normalize: true` to fill in the
|
|
119
|
+
bookkeeping as `setDocument` does.
|
|
120
|
+
3. **Look.** Validation cannot judge the picture. Preview real frames:
|
|
121
|
+
- `times_ms` (up to 8) or `count` (default 3) for single frames.
|
|
122
|
+
- `range: {from_ms, to_ms, count}` (up to 24) to sweep a move, with
|
|
123
|
+
`sheet: true` for one labelled contact sheet.
|
|
124
|
+
- `width` (default 640, max 1280) and `motion_blur_samples` with
|
|
125
|
+
`shutter_angle`.
|
|
126
|
+
- The sheet comes back as `sheet.image`. Pass its `asset_id` to
|
|
127
|
+
`view_image`. Without a sheet, each frame has its own `image`.
|
|
128
|
+
- Sample the middle of an animation, not its ends.
|
|
129
|
+
- A frame's `degraded` list names where the preview draws differently from
|
|
130
|
+
the export, and `effects_not_applied` names effects it cannot draw.
|
|
131
|
+
4. **Fix and compare.** After a change nobody asked for, run
|
|
132
|
+
`compare({timeline_id: id, version: n}, id)` to prove which frames it
|
|
133
|
+
touched. A difference of 0 everywhere means the edit was cosmetic.
|
|
134
|
+
5. **Render** before you call the cut done. `render(id, {preview_scale: 0.5})`
|
|
135
|
+
is a fast draft. `wait: true` blocks up to `timeout_ms` (default 180000,
|
|
136
|
+
clamped to 300000). On a timeout the job keeps going; wait on it with
|
|
137
|
+
`nodetool.jobs.wait(job_id)`.
|
|
138
|
+
|
|
139
|
+
## Render options
|
|
140
|
+
|
|
141
|
+
`format` is `mp4` (default), `webm`, `mov` or `png_sequence`. Only `webm`,
|
|
142
|
+
`mov` and `png_sequence` carry `alpha`. Also `video_codec`, `bitrate` (a
|
|
143
|
+
number or `"8M"`), `motion_blur_samples` with `shutter_angle`,
|
|
144
|
+
`include_audio` (default true) and `preview_scale`.
|
|
145
|
+
|
|
146
|
+
## Documents from other surfaces
|
|
147
|
+
|
|
148
|
+
A storyboard or a script assembles into a timeline of its own media
|
|
149
|
+
(`nodetool.storyboards.assembleTimeline`, `nodetool.scripts.assembleTimeline`).
|
|
150
|
+
For anything else, create your own timeline instead of editing one the user
|
|
151
|
+
has open.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: api-web
|
|
3
|
+
description: "Call nodetool.web or nodetool.email from a code action: search the web, news or images, read a page's text, make an HTTP request, download a file into the workspace, screenshot a page, or search, archive and label Gmail messages. Load before the first call into these namespaces."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# nodetool.web and nodetool.email
|
|
7
|
+
|
|
8
|
+
## nodetool.web
|
|
9
|
+
|
|
10
|
+
| Call | Does | Answers |
|
|
11
|
+
| :--- | :--- | :--- |
|
|
12
|
+
| `search(query, opts)` | Web search. | Pages with a title, URL and snippet |
|
|
13
|
+
| `news(query, opts)` | News search. | Articles with dates and sources |
|
|
14
|
+
| `images(query, opts)` | Image search. `num_results` defaults to 20. | Title, page link, original image URL and thumbnail URL for each result |
|
|
15
|
+
| `browse(url)` | Fetches a page and strips the HTML. | The readable text |
|
|
16
|
+
| `fetch(url, {method, headers, body})` | One raw HTTP request. `method` defaults to GET. | The response body as text |
|
|
17
|
+
| `download(url, outputFile)` | Saves the bytes of a URL into the workspace. `url` may also be an `asset://` URI, a `/api/storage/` key or a `data:` URI. | The workspace path, an `asset_url`, and for media a `display_markdown` snippet |
|
|
18
|
+
| `screenshot(url, outputFile, {full_page})` | Renders the page in headless Chrome and saves a PNG. `outputFile` defaults to `screenshot.png`. `full_page: true` captures the whole scrollable page. | The workspace path |
|
|
19
|
+
|
|
20
|
+
Options for `search`, `news` and `images`:
|
|
21
|
+
|
|
22
|
+
- `provider` pins one backend: `"serpapi"`, `"dataforseo"`, `"brave"`,
|
|
23
|
+
`"apify"`, `"openai"` or `"google"` (Gemini grounded search, or SerpAPI for
|
|
24
|
+
`news`). Without it the call uses the first configured backend.
|
|
25
|
+
- `images` cannot use `"openai"` or `"google"`, because they answer with prose
|
|
26
|
+
instead of results.
|
|
27
|
+
- `allowed_domains` and `blocked_domains` scope the results.
|
|
28
|
+
- `num_results` sets how many come back (10 by default for web and news).
|
|
29
|
+
|
|
30
|
+
Rules:
|
|
31
|
+
|
|
32
|
+
- Search result pages are blocked in `browse`. Use `search` to search, and
|
|
33
|
+
`browse` to read a page it found.
|
|
34
|
+
- `download` is how an asset reaches the workspace, for ffmpeg or a document
|
|
35
|
+
conversion. To keep a downloaded file in the library, pass its `asset_url`
|
|
36
|
+
to `nodetool.assets.save(name, {source})`.
|
|
37
|
+
- When you show a downloaded image to the user, paste `display_markdown` as it
|
|
38
|
+
is. Do not build a link from the workspace path.
|
|
39
|
+
- Fan out independent lookups with `Promise.all`. Ten searches take one round
|
|
40
|
+
trip:
|
|
41
|
+
|
|
42
|
+
```js
|
|
43
|
+
const hits = await Promise.all(topics.map((t) => nodetool.web.search(t)));
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
## nodetool.email
|
|
47
|
+
|
|
48
|
+
Gmail, when the account is connected.
|
|
49
|
+
|
|
50
|
+
| Call | Does |
|
|
51
|
+
| :--- | :--- |
|
|
52
|
+
| `search({subject, text, since_hours_ago, max_results})` | Finds messages. `since_hours_ago` defaults to 6 and `max_results` to 50. Each result has `message_id`, `subject`, `sender` and `body`. |
|
|
53
|
+
| `archive(messageIds)` | Archives one id or an array of ids. |
|
|
54
|
+
| `label(messageId, label)` | Adds a label to one message. |
|
|
55
|
+
|
|
56
|
+
Archiving and labelling change the user's mailbox. Declare the action as
|
|
57
|
+
high risk unless the user asked for exactly that change.
|
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: api-workflows
|
|
3
|
+
description: "Call nodetool.workflows, nodetool.jobs or nodetool.nodes from a code action: list, validate, save, run, debug and version workflow graphs, wait on background jobs, and look up or probe node types. Load before the first call into these namespaces."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# nodetool.workflows, nodetool.jobs, nodetool.nodes
|
|
7
|
+
|
|
8
|
+
These three namespaces cover saved graphs, the runs they start, and the node
|
|
9
|
+
catalog a graph is built from. How to design a graph is
|
|
10
|
+
`nodetool-workflow-builder`. This skill is the call reference.
|
|
11
|
+
|
|
12
|
+
Every method takes an id as the full id or its 12-character prefix, and every
|
|
13
|
+
method throws when its backing tool is not on this belt.
|
|
14
|
+
|
|
15
|
+
## nodetool.workflows
|
|
16
|
+
|
|
17
|
+
| Call | Does | Answers |
|
|
18
|
+
| :--- | :--- | :--- |
|
|
19
|
+
| `list({workflow_type, query, limit})` | Lists workflows. `workflow_type` is `"user"` (default), `"example"` or `"all"`. | `{workflows: [{id, name, description, tags}]}` — an envelope, not an array |
|
|
20
|
+
| `get(id)` | Reads one workflow with its graph. | The workflow record |
|
|
21
|
+
| `create(name, graph, {description, tags, access, project_id})` | Saves a new workflow. `graph` is a DSL result, `{nodes, edges}`, or a workflow record. `create({name, graph})` also works. | The saved workflow with its `id` |
|
|
22
|
+
| `validate(idOrGraph)` | Checks a graph against the node registry without a run. | `{ok, issues}`. Throws when the graph has errors, so a broken graph cannot reach a paid run. |
|
|
23
|
+
| `run(id, params, {interactive, node_ids, reuse_results})` | Runs and blocks until the run settles. | The run result. With `interactive: true`, a failed node returns `status: "escalated"`. |
|
|
24
|
+
| `start(id, params, {node_ids, reuse_results})` | Starts a run and returns at once. | A receipt `{job_id, id, status: "running"}` |
|
|
25
|
+
| `debug(id, params, {interactive, node_ids, reuse_results, include_graph, log_limit})` | Runs and reports. | `{workflow_id, run, job, workflow}`. An escalated debug answers `{workflow_id, run}` with the escalation under `run`. |
|
|
26
|
+
| `resolve(sessionId, escalationId, action, {outputs, reason, apply_to})` | Answers an escalation. | The next escalation, or the final report |
|
|
27
|
+
| `example("<package>/<name>")` | Loads a shipped example with its graph. A bare name reads from `nodetool-base`. | The example workflow |
|
|
28
|
+
| `versions(id, {limit})`, `getVersion(id, n)` | Lists and reads snapshots. | Version rows, newest first |
|
|
29
|
+
| `snapshot(id, {name, description})` | Saves the current graph as a manual version. | The new version number |
|
|
30
|
+
| `restore(id, n)` | Rolls the graph back. The graph it replaces is snapshotted first, so a restore can be undone. | — |
|
|
31
|
+
| `deleteVersion(id, n)` | Deletes one snapshot. This cannot be undone. | — |
|
|
32
|
+
| `open(id?)` | Opens the editable object model of a graph that is open in the editor. Throws when no editor tools are on the belt. | A workflow object: `addNode`, `connect`, `commit()` |
|
|
33
|
+
|
|
34
|
+
`node_ids` runs only those nodes and what they need upstream. Unrelated
|
|
35
|
+
branches and downstream nodes are not executed or billed. An upstream
|
|
36
|
+
generator that already produced output in this workflow is fed that output
|
|
37
|
+
instead of running again, unless `reuse_results: false`. This is the cheapest
|
|
38
|
+
way to re-run one repaired node.
|
|
39
|
+
|
|
40
|
+
### The debug report
|
|
41
|
+
|
|
42
|
+
`debug` answers `{workflow_id, run, job, workflow}`:
|
|
43
|
+
|
|
44
|
+
- `run.status` is the verdict of the run, and `run.error` names the failure.
|
|
45
|
+
- `run.outputs` is keyed by output name. Each name holds an **array** of the
|
|
46
|
+
values it emitted, so read `run.outputs.summary[0]`, not `run.outputs.summary`.
|
|
47
|
+
- `run.verdict.issues` lists what went wrong, node by node.
|
|
48
|
+
- `job` carries the status, the cost and the logs.
|
|
49
|
+
|
|
50
|
+
### Escalations
|
|
51
|
+
|
|
52
|
+
An interactive run parks on a failed node and answers `status: "escalated"`
|
|
53
|
+
with a `session_id`, an `escalation_id` and `escalation.allowedActions`. The actions
|
|
54
|
+
are `retry`, `substitute` (with `{outputs}`, only when the escalation has a
|
|
55
|
+
`candidateOutput`), `skip`, `end_stream` and `fail` (with `{reason}`).
|
|
56
|
+
`apply_to: "signature"` also answers later failures with the same signature.
|
|
57
|
+
`run` answers the escalation at the top level. `debug` nests it under `run`,
|
|
58
|
+
so drive the loop with `run`. `resolve` answers the next escalation or the
|
|
59
|
+
final report, both at the top level. Write the whole loop in one action:
|
|
60
|
+
|
|
61
|
+
```js
|
|
62
|
+
let report = await nodetool.workflows.run(id, params, { interactive: true });
|
|
63
|
+
while (report.status === "escalated") {
|
|
64
|
+
report = await nodetool.workflows.resolve(report.session_id, report.escalation_id, "skip");
|
|
65
|
+
}
|
|
66
|
+
return { status: report.status, error: report.error, issues: report.verdict?.issues };
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
An unanswered escalation fails the run when its decision timeout expires.
|
|
70
|
+
|
|
71
|
+
### Rules
|
|
72
|
+
|
|
73
|
+
- A saved graph is not a run. When the user asked for results, call `run`.
|
|
74
|
+
- Validate before `create` and before a paid run. `validate` throws on errors,
|
|
75
|
+
and its message lists the errors only.
|
|
76
|
+
- A `*_model` property left unset fails validation. Assign
|
|
77
|
+
`(await nodetool.models.pick(capability)).ref`.
|
|
78
|
+
- `create` checks model properties. An unknown provider or model id is refused
|
|
79
|
+
and not saved.
|
|
80
|
+
|
|
81
|
+
## nodetool.jobs
|
|
82
|
+
|
|
83
|
+
A job is one execution of a workflow, and a render of a timeline is a job too.
|
|
84
|
+
|
|
85
|
+
| Call | Answers |
|
|
86
|
+
| :--- | :--- |
|
|
87
|
+
| `list({workflow_id, limit})` | Job rows with status, timing, error and the names of the outputs. The values are not in the list. |
|
|
88
|
+
| `get(idOrReceipt)` | One job, with the outputs once it has settled |
|
|
89
|
+
| `logs(idOrReceipt, {limit})` | The log tail. It answers for a failed job too, with the failure under `job_error`. |
|
|
90
|
+
| `wait(idOrReceipt, {timeoutMs, pollMs})` | Polls until the status is `completed`, `failed`, `cancelled` or `error`, then answers the job. On a timeout it throws with the last status. |
|
|
91
|
+
|
|
92
|
+
Every call takes the id string or the receipt that `start()` answered.
|
|
93
|
+
Start, do other work, then wait, all in one action:
|
|
94
|
+
|
|
95
|
+
```js
|
|
96
|
+
const receipt = await nodetool.workflows.start(id, { prompt: "a fox" });
|
|
97
|
+
// … other work in the same action …
|
|
98
|
+
const settled = await nodetool.jobs.wait(receipt, { timeoutMs: 300000 });
|
|
99
|
+
return { status: settled.status, outputs: settled.outputs };
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Do not write one `get()` per action. `wait` is the polling loop. A code
|
|
103
|
+
action stops after 600 s, so keep `timeoutMs` well under 600000 and call
|
|
104
|
+
`wait` again in the next action when the job is still running.
|
|
105
|
+
|
|
106
|
+
## nodetool.nodes
|
|
107
|
+
|
|
108
|
+
| Call | Does | Answers |
|
|
109
|
+
| :--- | :--- | :--- |
|
|
110
|
+
| `search(query, {n_results, namespace, input_type, output_type, include_provider_nodes})` | Searches the catalog. `query` is a string or an array of terms. | `{total, results}`. The node type of a result is on `type`, not `node_type`. |
|
|
111
|
+
| `info(type)` | Reads one node type: properties, inputs, outputs, defaults. | The node metadata |
|
|
112
|
+
| `list({namespace, limit})` | Browses a namespace. An unknown namespace answers with the namespaces that exist. | Node rows |
|
|
113
|
+
| `run(type, inputs)` | Runs ONE node with a property bag, with no workflow. | The outputs, keyed by slot |
|
|
114
|
+
|
|
115
|
+
- Never guess a node type. Search, then read `info` for the exact property
|
|
116
|
+
names before you wire the node.
|
|
117
|
+
- Provider nodes (`openai.*`, `fal.*` and others) are hidden unless you pass
|
|
118
|
+
`include_provider_nodes: true`. Pass it only when the user named a provider.
|
|
119
|
+
- `run` is the cheapest probe of what a node does. Try one node before you
|
|
120
|
+
put it in a graph.
|
|
121
|
+
- To call nodes as functions inside an action, import them from
|
|
122
|
+
`@nodetool-ai/sandbox-flow`. To author a graph, import them from
|
|
123
|
+
`@nodetool-ai/sandbox-dsl`. `nodetool-js-scripting` covers both packs.
|
|
@@ -0,0 +1,208 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: beat-sync-editing
|
|
3
|
+
description: Cut a NodeTool timeline to music and shape its pacing — detect the beat grid, place cuts on phrases, pick a cut type, build speed ramps with time remap, and give the piece an arc. Use when clips should land on the music, when pacing drags or feels choppy, when a hit needs impact, or when transitions feel arbitrary. Not for what each shot contains.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Beat-Sync Editing → cuts, rhythm and retiming
|
|
7
|
+
|
|
8
|
+
Where a cut lands matters as much as what is on screen. This turns a track and a
|
|
9
|
+
brief into millisecond cut points on the timeline.
|
|
10
|
+
|
|
11
|
+
`motion-direction` sets the energy; this turns it into a grid and an arc.
|
|
12
|
+
`timeline-edit-ops` carries the op contract for the calls named here.
|
|
13
|
+
`video-audio-continuity` decides where the bed comes from before there is a
|
|
14
|
+
grid to read: a track you add yourself, or one generated clip carrying every
|
|
15
|
+
scene.
|
|
16
|
+
|
|
17
|
+
## Get the grid from the audio, not from arithmetic
|
|
18
|
+
|
|
19
|
+
When the bed does not exist yet, generate it with the grid in mind: a Stable
|
|
20
|
+
Audio prompt states its BPM (`stable-audio-prompting`), and an ElevenLabs
|
|
21
|
+
`composition_plan` pins each section's length in milliseconds
|
|
22
|
+
(`elevenlabs-audio-prompting`), so the drop lands where the plan says instead
|
|
23
|
+
of where the model felt like putting it. Then measure it anyway — the
|
|
24
|
+
generated file, not the prompt, is the grid.
|
|
25
|
+
|
|
26
|
+
`detect_audio_events` on the music clip reports onsets and a tempo.
|
|
27
|
+
|
|
28
|
+
- `onsets.times` are in **seconds**. Every timeline op takes milliseconds, so
|
|
29
|
+
multiply by 1000. This is the single most common mistake here.
|
|
30
|
+
- Read `tempo.reliable` before you build anything off `bpm`. Speech and room
|
|
31
|
+
tone produce a confident-looking number from nothing. Unreliable means use the
|
|
32
|
+
onsets directly.
|
|
33
|
+
- The first onset is the grid's offset. A grid anchored at 0 instead of beat one
|
|
34
|
+
puts every cut a few frames early — the edit that feels "almost right".
|
|
35
|
+
|
|
36
|
+
With a reliable tempo:
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
ms_per_beat = 60000 / bpm
|
|
40
|
+
frames_per_beat = (60 / bpm) × fps # fps comes from get_timeline
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
At 120 BPM: 500ms a beat, 15 frames at 30fps, a 2000ms bar. Round each
|
|
44
|
+
**cumulative** beat, never the step, or the grid drifts over a long edit.
|
|
45
|
+
|
|
46
|
+
Lay the grid down so you can see it, then pull edges onto it:
|
|
47
|
+
|
|
48
|
+
```json
|
|
49
|
+
{"op": "set_markers_from_beats", "bpm": 120, "offset_ms": 0, "count": 32}
|
|
50
|
+
{"op": "snap_to_beats", "targets": "all", "tolerance_ms": 60,
|
|
51
|
+
"mode": "start", "action": "move"}
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Either takes `onsets_ms`, or a `bpm` with an `offset_ms`. With `bpm`,
|
|
55
|
+
`set_markers_from_beats` also needs `count`, the number of beats to mark. An
|
|
56
|
+
`onsets_ms` grid lays one marker per onset and ignores `count`. The 60ms default
|
|
57
|
+
tolerance is about two frames at 30fps — close enough to read as on the beat,
|
|
58
|
+
tight enough not to drag an edge onto a beat it was never near. `snap_to_beats`
|
|
59
|
+
reports every target including the ones nothing was in reach of, with the reason.
|
|
60
|
+
`action: "move"` keeps the clip's length and shifts what follows; `action:
|
|
61
|
+
"trim"` changes the length and leaves the neighbours alone. Move a title that has
|
|
62
|
+
to land on a hit; trim a picture clip whose out point has to meet the next shot.
|
|
63
|
+
|
|
64
|
+
For an animation that should follow tempo changes, store a `beat` anchor on the
|
|
65
|
+
animation: `{index: 3, scope: "clip", offsetMs: -20}` counts from the clip's
|
|
66
|
+
start; `scope: "sequence"` counts from the document tempo's `offsetMs`.
|
|
67
|
+
`index` is one-based, and `delayMs` adds another offset. This is distinct from
|
|
68
|
+
snapping a clip edge or baking a measured audio curve. `animate_clip` does not
|
|
69
|
+
accept `beat` in its tool input; write the animation's `beat` through the full
|
|
70
|
+
document with `set_timeline_document`. A stored animation needs its `id`,
|
|
71
|
+
`role`, `preset`, and `durationMs`; keep those and the rest of the animation
|
|
72
|
+
when adding `beat`. Keep the document's tempo, setup, media tracks, template
|
|
73
|
+
identity, clips, tracks, and markers in the write.
|
|
74
|
+
|
|
75
|
+
## Cut on phrases, not on beats
|
|
76
|
+
|
|
77
|
+
A cut on every beat is relentless and the viewer fatigues by beat eight. Cut on
|
|
78
|
+
2, 4 or 8 beats, and accent on the eighth grid sparingly.
|
|
79
|
+
|
|
80
|
+
| Personality | Cut every | Transition family | Retime |
|
|
81
|
+
|---|---|---|---|
|
|
82
|
+
| Playful | 4 beats, syncopated accents | Hard cut, occasional `push` | Light ramps |
|
|
83
|
+
| Premium | 8–16 beats, long holds | Hard cut, occasional `crossfade` | Slow, smooth |
|
|
84
|
+
| Corporate | 8 beats, steady | Hard cut only | None |
|
|
85
|
+
| Energetic | 1–2 beats at the peak | Hard cut, `zoom`, `push` | Aggressive, on the beat |
|
|
86
|
+
|
|
87
|
+
Put the single biggest visual on the drop. One piece, one peak.
|
|
88
|
+
|
|
89
|
+
## Shape an arc
|
|
90
|
+
|
|
91
|
+
Uniform pacing reads as flat regardless of content.
|
|
92
|
+
|
|
93
|
+
| Phase | Cut every | Shot length | Note |
|
|
94
|
+
|---|---|---|---|
|
|
95
|
+
| Establish | 8–16 beats | Longest | Set the place, let it breathe |
|
|
96
|
+
| Develop | 8 → 4 beats | Shortening | Accelerate to build |
|
|
97
|
+
| Climax | 1–2 beats | Shortest | The drop, the biggest visual, the ramp |
|
|
98
|
+
| Resolve | One long hold | Longest | The end card breathes after the peak |
|
|
99
|
+
|
|
100
|
+
## Hide the seam
|
|
101
|
+
|
|
102
|
+
A cut is invisible when motion carries the eye across it. Cut during a movement,
|
|
103
|
+
and match its direction and speed across the cut. For graphics, cut at a
|
|
104
|
+
transform peak — mid-travel, not at rest.
|
|
105
|
+
|
|
106
|
+
Default to the hard cut: two clips on the same track, meeting exactly. Reach for
|
|
107
|
+
anything else only when it is motivated.
|
|
108
|
+
|
|
109
|
+
| Cut | How, on the timeline | Reach for it when |
|
|
110
|
+
|---|---|---|
|
|
111
|
+
| Hard cut | Adjacent clips, no overlap, no `set_transition` | Nine cuts in ten |
|
|
112
|
+
| Match cut | Hard cut where both frames share a shape, position or motion vector | Two shots rhyme; the strongest move in motion design |
|
|
113
|
+
| Crossfade | `set_transition` `crossfade` on the incoming clip, clips overlapped by at least its `durationMs` | Two shots of the same scene |
|
|
114
|
+
| Dip | `dipToColor` with your own `color` | A chapter break |
|
|
115
|
+
| Wipe / push / slide | Those types with a `direction` naming the edge the incoming clip arrives from | A graphic feel, or lateral energy |
|
|
116
|
+
| Zoom | `zoom` | Pushing into the next beat |
|
|
117
|
+
| J-cut | The next clip's audio clip starts 4–12 frames before its picture, on its own audio track | Dialogue, reveals: it pulls the viewer forward |
|
|
118
|
+
| L-cut | The current clip's audio runs 12–24 frames under the next picture | Scene changes, ambience |
|
|
119
|
+
| Whip | A short `offsetX` move plus a `blur` effect either side of a hard cut | Hype, a scene jump |
|
|
120
|
+
|
|
121
|
+
Two things the engine does whether you asked or not:
|
|
122
|
+
|
|
123
|
+
- Two clips on one track whose times overlap dissolve across the overlap with no
|
|
124
|
+
call at all. **An accidental overlap is an accidental dissolve.** When a cut
|
|
125
|
+
looks soft where you wanted it hard, check `startMs` and `durationMs` first.
|
|
126
|
+
- A transition plays over the incoming clip's head against whatever is beneath
|
|
127
|
+
it. With no overlap there is nothing under it and it reads as a fade from
|
|
128
|
+
black.
|
|
129
|
+
|
|
130
|
+
J and L cuts are built by splitting picture and sound onto separate clips, not
|
|
131
|
+
by a transition: the shot's audio twin already sits on its own audio track after
|
|
132
|
+
an assemble, so move that clip rather than the picture.
|
|
133
|
+
|
|
134
|
+
## Speed ramps
|
|
135
|
+
|
|
136
|
+
`set_time_remap` maps timeline time onto source time. `t` runs 0..1 over the
|
|
137
|
+
clip's own window, must start at 0, end at 1 and ascend; `sourceMs` says which
|
|
138
|
+
millisecond of the source plays there. Descending `sourceMs` is a reverse, a flat
|
|
139
|
+
pair is a freeze.
|
|
140
|
+
|
|
141
|
+
Ramp into a hit and snap out, with the slowest frame on the beat. On a 2000ms
|
|
142
|
+
clip whose impact sits at 1200ms:
|
|
143
|
+
|
|
144
|
+
```json
|
|
145
|
+
{"op": "set_time_remap", "target": "Shot 3", "timeRemap": {"keyframes": [
|
|
146
|
+
{"t": 0, "sourceMs": 0},
|
|
147
|
+
{"t": 0.5, "sourceMs": 900, "easing": "easeIn"},
|
|
148
|
+
{"t": 0.6, "sourceMs": 1000, "easing": "easeOut"},
|
|
149
|
+
{"t": 1, "sourceMs": 2400}]}}
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Between 0.5 and 0.6 the source advances 100ms across 200ms of timeline: half
|
|
153
|
+
speed on the hit. `t` that repeats or goes backwards is
|
|
154
|
+
`time_remap_not_monotonic`, an error. `split_clip` and `trim_clip` refuse a
|
|
155
|
+
remapped clip, because re-deriving the curve silently is how a ramp becomes a
|
|
156
|
+
different move — clear the remap, cut, re-apply.
|
|
157
|
+
|
|
158
|
+
`speedMultiplier` on `set_clip_params` is the flat alternative: one rate for the
|
|
159
|
+
whole clip, no curve. Use it when nothing needs to ramp.
|
|
160
|
+
|
|
161
|
+
For rhythmic texture, `steppedTime: {fps}` quantizes one clip's sampling
|
|
162
|
+
clock. `temporalEcho: {copies, intervalMs, opacityDecay}` draws delayed copies,
|
|
163
|
+
which can trail a hit without duplicating clips by hand. A clip's `motionBlur`
|
|
164
|
+
sets `samplesPerFrame` and `shutterAngle` for its own shutter window. Values
|
|
165
|
+
above one request a minimum scene sample count; the scene uses the highest
|
|
166
|
+
request, capped at 32, and samples each layer evenly over its own shutter.
|
|
167
|
+
Set these fields in the full timeline document. `set_clip_params` refuses them.
|
|
168
|
+
|
|
169
|
+
If motion must follow the actual audio envelope or detected onsets, use
|
|
170
|
+
`bake_audio_animation`. It writes a source-anchored curve with `bakedFrom`
|
|
171
|
+
provenance through `set_baked_animation`; re-baking replaces that measured
|
|
172
|
+
curve. A beat anchor instead recomputes timing from the document tempo, while
|
|
173
|
+
markers and `snap_to_beats` move edit points.
|
|
174
|
+
|
|
175
|
+
## The plan, before the calls
|
|
176
|
+
|
|
177
|
+
Write the cut down before editing, in the units the ops take:
|
|
178
|
+
|
|
179
|
+
- Track spec: `bpm`, `fps`, the first onset in ms, the drop in ms, ms per beat.
|
|
180
|
+
- The four phases with their ms ranges and target shot length.
|
|
181
|
+
- Every cut as an absolute ms, its clip, and its cut type.
|
|
182
|
+
- Audio offsets in ms for each J and L cut.
|
|
183
|
+
- Each ramp as its `t` / `sourceMs` keyframes.
|
|
184
|
+
- The marker list you will pass to `set_markers_from_beats`.
|
|
185
|
+
|
|
186
|
+
## Check it
|
|
187
|
+
|
|
188
|
+
`validate_timeline` after every batch: it catches overlapping clips, a
|
|
189
|
+
transition longer than the clip carrying it, and clips shorter than a frame.
|
|
190
|
+
|
|
191
|
+
Then look and listen. `preview_timeline_frame` at the ms either side of a cut
|
|
192
|
+
says whether the seam is where you think it is; `analyze_audio` on the music
|
|
193
|
+
confirms the grid you built matches the file. Before calling the cut done,
|
|
194
|
+
`render_timeline` — a preview answers what one timecode looks like, a render
|
|
195
|
+
answers whether it plays.
|
|
196
|
+
|
|
197
|
+
## Common mistakes
|
|
198
|
+
|
|
199
|
+
| Symptom | Cause | Fix |
|
|
200
|
+
|---|---|---|
|
|
201
|
+
| Frantic | A cut on every beat | Cut on 2, 4 or 8 beat phrases |
|
|
202
|
+
| Almost on beat | Grid anchored at 0, ignoring the first onset | Anchor to the onset, in ms |
|
|
203
|
+
| Drift over a long edit | Rounding each step | Round each cumulative beat |
|
|
204
|
+
| Jarring cut | Placed at rest, between motions | Cut mid-motion, match direction |
|
|
205
|
+
| Soft where it should be hard | Overlapping clips on one track | Fix `startMs` and `durationMs` |
|
|
206
|
+
| Transition fades from black | Nothing beneath the incoming head | Overlap the clips by the transition's `durationMs` |
|
|
207
|
+
| Flat | Uniform pacing | Establish, develop, climax, resolve |
|
|
208
|
+
| Nothing lands | No single peak | One biggest visual, on the drop, then a hold |
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: caption-titles
|
|
3
|
+
description: Add and animate a consistent text layer on an existing NodeTool timeline. Use for titles, lower-thirds, captions, callouts, end cards, number tickers, deterministic scrambles, and text on a path.
|
|
4
|
+
featured: true
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Caption & Titles → Timeline Agent
|
|
8
|
+
|
|
9
|
+
Work directly with timeline capabilities and open-timeline tools. Do not build a workflow or fabricate a cut.
|
|
10
|
+
|
|
11
|
+
`timeline-edit-ops` carries the op contract for every call named here. `motion-principles` gives the entry and exit durations and easing behind each tier's presets, `frame-composition` fixes the safe inset and where a tier sits, `color-motion` the scrim and contrast fixes when type does not read, and `logo-reveal` owns the brand mark at the head or tail.
|
|
12
|
+
|
|
13
|
+
## Type is post, never a render prompt
|
|
14
|
+
|
|
15
|
+
Every board skill and every video prompting skill sends its on-screen copy here. A title card, a super, a lyric or a CTA is a text clip on the timeline, not words in a `generate_video` or `generate_image` prompt: generators letter unreliably, and copy baked into pixels cannot be corrected, translated or re-timed. When a board's shot `action` carries `Text: …`, that is the caption map's input, and the render prompt should not have asked for it.
|
|
16
|
+
|
|
17
|
+
## Hard gate
|
|
18
|
+
|
|
19
|
+
List and read the selected timeline first. When the job is to caption or title footage, the timeline must contain picture clips. If it is missing or has none, stop and ask for a real cut; route users to storyboard or script-to-timeline work instead. A typographic piece built from type and shapes alone, such as a title sequence, kinetic text or a standalone end card, needs no picture clips: build it through `motion-graphics` and use the tiers below for its type. Snapshot the timeline before editing with `create_timeline_version`.
|
|
20
|
+
|
|
21
|
+
Read `fps`, `width` and `height` off `get_timeline` before setting a size: `fontSizePx` is authored against the sequence resolution, and the tiers below are fractions of `height`.
|
|
22
|
+
|
|
23
|
+
## Lock one type system before adding text
|
|
24
|
+
|
|
25
|
+
Resolve type family and tier weights, five size tiers as fractions of frame height, text/accent/scrim palette, safe inset and maximum width, and per-tier motion presets. Prefer a supplied brand kit, then existing text on the timeline, then a platform-safe system. Keep these tiers consistent: T1 title/end card, T2 lower-third, T3 captions, T4 callouts, T5 CTA.
|
|
26
|
+
|
|
27
|
+
The family is one NodeTool ships — `Inter`, `Space Grotesk`, `Bebas Neue`, `Playfair Display`, `Lora`, `JetBrains Mono` — or every host resolves it against its own installed fonts, the editor and the render disagree on line widths, and `validate_timeline` reports `font_not_portable`. Sizes below 2.5% of frame height report `text_illegible`; body copy sits at 2.5–4%. `motion-direction` holds the same rule for the whole piece, so when a motion language already exists, take its type family rather than picking a second one.
|
|
28
|
+
|
|
29
|
+
Motion per tier follows `motion-principles`: `fade` or `slide` in at 300–500ms, out at 200–300ms, role-default easing. Stagger only on a T1 or a kinetic T3 line of five words or fewer, and work out the span (`durationMs + offsetMs × (words − 1)`) against the clip — a span that does not fit is compressed silently and only `stagger_compressed` says so.
|
|
30
|
+
|
|
31
|
+
Each tier has a shipped composition. Insert it with `{"op": "insert_composition", "composition_id": "<id>", ...}` and override its `params` rather than building the rig from bare clips — the timing and motion inside it are already balanced.
|
|
32
|
+
|
|
33
|
+
| Tier | Composition id |
|
|
34
|
+
|---|---|
|
|
35
|
+
| T1 title | `title-card` |
|
|
36
|
+
| T2 lower-third | `lower-third` |
|
|
37
|
+
| T3 captions | `caption-bar` |
|
|
38
|
+
| T4 callouts | `callout` |
|
|
39
|
+
| T5 CTA / end card | `cta-end-card` |
|
|
40
|
+
|
|
41
|
+
A brand mark at the head or tail is `logo-sting`.
|
|
42
|
+
|
|
43
|
+
Every live-picture T2–T4 element gets a matching scrim unless the plate is demonstrably controlled.
|
|
44
|
+
|
|
45
|
+
## Caption map
|
|
46
|
+
|
|
47
|
+
Write the timed map before editing: tier, exact text, start/end, position, and scrim. Caption duration is at least `max(1200ms, words × 350ms)`. Titles/end cards stay on screen at least 2000ms, which leaves the read hold `motion-principles` § Holds sets between their in and out. Lower-thirds arrive about 500ms after a person appears and hold 2500–4000ms. One text idea per frame. Unknown facts use `[INPUT NEEDED: …]`.
|
|
48
|
+
|
|
49
|
+
## Animate the letters and their content
|
|
50
|
+
|
|
51
|
+
For a number that changes during a text clip, store `textAnimator` on an
|
|
52
|
+
animation with `kind: "ticker"`, `from`, and `to`. It also accepts `decimals`,
|
|
53
|
+
`padTo`, `prefix`, and `suffix`. Set `groupSeparator` to `","` to show
|
|
54
|
+
thousands as `2,847`; omitting it keeps existing tickers ungrouped. For a
|
|
55
|
+
deterministic decode into the clip's
|
|
56
|
+
authored text, use `kind: "scramble"` with a fixed `seed` and optional
|
|
57
|
+
`charset`. These operate over the animation window, so a held final frame
|
|
58
|
+
shows the finished value.
|
|
59
|
+
|
|
60
|
+
```json
|
|
61
|
+
{"id": "count-in", "role": "in", "preset": "fade", "durationMs": 900,
|
|
62
|
+
"textAnimator": {"kind": "ticker", "from": 1, "to": 7,
|
|
63
|
+
"padTo": 2, "suffix": " / 07"}}
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
For unit-by-unit motion, set `stagger.unit` to `character`, `word`, or `line`
|
|
67
|
+
and give it `offsetMs`. With a text stagger, `styleTracks` can target
|
|
68
|
+
`glyph.color`, `glyph.blurPx`, or `glyph.trackingPx`. The colour uses CSS
|
|
69
|
+
colour values; blur and tracking use sequence pixels. Use `text.color` or
|
|
70
|
+
`text.fill.stops.0.color` for a whole line. A text fill gradient also exposes
|
|
71
|
+
`text.fill.angle` and stop offsets. The target must refer to an existing
|
|
72
|
+
style field; `text.color` is hidden while a gradient fill is set.
|
|
73
|
+
|
|
74
|
+
To lay a title along a curve, set `textStyle.path` to an SVG path in
|
|
75
|
+
normalized canvas coordinates. This places the baseline on the path. It does
|
|
76
|
+
not make each character travel along it. Shape and mask paths can morph through
|
|
77
|
+
`shape.d` and `mask.d` style tracks when both paths have matching commands and
|
|
78
|
+
point counts. Use `followPath` when the whole clip should move along a path.
|
|
79
|
+
|
|
80
|
+
`edit_timeline`'s `animate_clip` input accepts presets and custom numeric
|
|
81
|
+
curves, but does not accept `textAnimator` or `styleTracks`; those fields are
|
|
82
|
+
dropped from that call. To author them, read the current document and use
|
|
83
|
+
`set_timeline_document` with the animation fields in the clip. That call
|
|
84
|
+
validates the complete document and snapshots the previous state. Preserve
|
|
85
|
+
the other tracks and clips from the document. `timeline-edit-ops` owns the full
|
|
86
|
+
tool contract.
|
|
87
|
+
|
|
88
|
+
## Build and verify
|
|
89
|
+
|
|
90
|
+
Add separate overlay and subtitle tracks. For each map item, add scrim before text, apply the locked tier style, then animate it with that tier’s entry and exit preset. No hard cut-on. Use stagger only for tone-appropriate kinetic captions. Keep all text within safe margins and clear of faces and product moments.
|
|
91
|
+
|
|
92
|
+
Two facts about stacking decide whether the text is seen at all. The lowest track `index` draws on top, so the scrim's track index sits between the picture's and the text's — picture, scrim, text, in descending index. And word-level captions generated from a voiceover clip composite above every real track regardless of where that clip sits, so no title can cover them: time a T1 around the narration rather than over it.
|
|
93
|
+
|
|
94
|
+
Verify in two passes, and they answer different questions. `validate_timeline` reads the document: `text_illegible` (size, or under 3:1 contrast against the text's own `background` plate or a full-frame shape behind it), `font_not_portable`, `stagger_compressed`, `animation_exceeds_clip`, overlaps, and clips shorter than a frame. Its contrast check refuses to guess — a translucent scrim, gradient type or a plate it cannot prove is behind the text produces no finding — so a silent pass is not a pass. Then `preview_timeline_frame` at each item's entry midpoint, a held moment, and its exit: that is where text over a busy plate, a scrim covering a face, or a caption still finishing when its clip ends shows up. Re-read the map against the frames: timing, reading floors, scrim ordering, one CTA at the end. Then render with `render_timeline` before calling the layer done, as `timeline-edit-ops` § The render loop describes: the frames show one timecode, the render shows the timing at speed.
|