@kolbo/mcp 1.89.3 → 1.90.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -0
- package/package.json +1 -1
- package/skill/GENERATED.md +1 -1
- package/skill/SKILL.md +19 -1
- package/skill/references/models/gpt-image.md +3 -2
- package/skill/references/models/html-presentation.md +1 -1
- package/skill/references/models/landing-page.md +1 -1
- package/skill/references/models/seedance25.md +1 -1
- package/skill/references/models/visual-code.md +1 -1
- package/skill/references/workflows/cost-and-validation.md +4 -7
- package/skill/references/workflows/filmmaking.md +1 -1
- package/skill/references/workflows/troubleshooting.md +2 -2
- package/skill/references/workflows/video-editor.md +28 -0
- package/src/toolAnnotations.js +5 -1
- package/src/tools/editor.js +20 -1
- package/src/tools/generate.js +90 -0
package/README.md
CHANGED
|
@@ -314,3 +314,14 @@ Both are optional — the local install logs in via the browser on first use.
|
|
|
314
314
|
### Chat thinking level
|
|
315
315
|
|
|
316
316
|
`chat_send_message` accepts optional `thinking_level`, using an ID from `list_models` with `type: "text"`. The server validates it against the resolved model; omitted or invalid values use `thinkingDefault`. Existing safeguards and legacy `deep_think` take precedence. Discover allowed levels through `thinkingLevels`; no package update is required when the server changes a model capability. For the Auto model, optional `routing_mode` accepts `fast`, `balanced`, or `smart`; omitted uses the server default of `balanced`.
|
|
317
|
+
|
|
318
|
+
## Agentic Video Editor
|
|
319
|
+
|
|
320
|
+
| Tool | Purpose |
|
|
321
|
+
| --- | --- |
|
|
322
|
+
| `get_video_editor_schema` | Complete writable settings and operation contract |
|
|
323
|
+
| `list_video_editor_sessions` | Paginated project timelines |
|
|
324
|
+
| `get_video_editor_session` | Full saved timeline and revision |
|
|
325
|
+
| `update_video_editor_session` | Atomic rename, settings, tracks, clips, trims, speed, captions and effects |
|
|
326
|
+
|
|
327
|
+
`create_video_editor_session` also accepts optional advanced `session_data` instead of clips/audio/texts. Read the schema and saved revision before editing. Changes affect saved state; reload an already-open editor before manual editing. Existing create/export names and arguments remain supported.
|
package/package.json
CHANGED
package/skill/GENERATED.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# AUTO-GENERATED — do not edit
|
|
2
2
|
|
|
3
|
-
This tree is mirrored from kolbo-code@
|
|
3
|
+
This tree is mirrored from kolbo-code@b77c8b4, the single source of truth.
|
|
4
4
|
Canonical source: packages/opencode/skills/kolbo/
|
|
5
5
|
Distribution: .github/workflows/sync-skill-to-plugin.yml
|
|
6
6
|
|
package/skill/SKILL.md
CHANGED
|
@@ -102,11 +102,14 @@ For multi-scene / batch work this pairs with `generate_creative_director` (see b
|
|
|
102
102
|
| Confirm **cost** or validate **resolution / aspect / duration** against model caps | `references/workflows/cost-and-validation.md` |
|
|
103
103
|
| Hit an **auth / MCP / 429** issue | `references/workflows/troubleshooting.md` |
|
|
104
104
|
| Inspect or change a connected **Blender** scene, render, import Kolbo media, or run approved Blender Python | `references/workflows/blender.md` |
|
|
105
|
+
| Create or edit a saved **Video Editor** timeline, clips, trims, speed or captions | `references/workflows/video-editor.md` |
|
|
105
106
|
|
|
106
107
|
Each `references/models/*.md` mirrors the matching skill prompt in `kolbo-api/src/config/systemPrompt.js` — same battle-tuned rules that power Kolbo's web-app help widget. Keep parity (see `packages/opencode/CLAUDE.md` "MCP & Skill Sync Rule").
|
|
107
108
|
|
|
108
109
|
## Available MCP Tools
|
|
109
110
|
|
|
111
|
+
For editable video timelines, read `references/workflows/video-editor.md`. Use `get_video_editor_schema`, `list_video_editor_sessions`, `get_video_editor_session`, `create_video_editor_session`, `update_video_editor_session`, and `export_video_editor_session`. Edit existing sessions in place using their saved revision; do not recreate them to rename or change clips.
|
|
112
|
+
|
|
110
113
|
For personal font uploads, font selection, or reuse, read `references/workflows/personal-fonts.md`. Use the dedicated My Fonts tools; never the media-upload path or agent-rendered specimens.
|
|
111
114
|
|
|
112
115
|
Font tools (when exposed by the installed MCP): `list_fonts`, `get_font`, `upload_font`, `get_font_upload_status`, `create_font_upload_ticket`, `font_upload_widget`, `rename_font`, `delete_font`. Image creation/editing and image-mode Creative Director accept `font_ids` only on models with `supports_custom_fonts`.
|
|
@@ -294,7 +297,7 @@ A user-named tool — in any language — overrides every other rule. Recognized
|
|
|
294
297
|
4. **Pick the model**:
|
|
295
298
|
- User named one → that name is a **family lock**, not a single catalog row. Use it. Identifiers resolve leniently — `"z-image"` / `"nano banana 2"` / `"grok imagine"` auto-resolve, including to the sibling for the tool you are calling (`grok-imagine-text-to-video` on `generate_video_from_image` becomes `grok-imagine-image-to-video`). `list_models` is still authoritative for constraints, caps, and pricing — not for swapping brands.
|
|
296
299
|
- **Never cheapest-swap a named family.** After compaction, "animate those images" is still Grok if the user said Grok. Seedance / Kling / Veo are not a "best balance" substitute. If the named family has no variant for this modality, ASK — do not silently switch.
|
|
297
|
-
- Auto-select → **only when no model was named on this task**. Then pick from "Auto-selectable" (models with a `summary`).
|
|
300
|
+
- Auto-select → **only when no model was named on this task**. Then pick from "Auto-selectable" (models with a `summary`). Follow Media selection preferences below: ordinary production uses the eligible general workhorse; select specialists for matching tasks and budget models for explicit cheap tests/drafts. Cost breaks ties between suitable choices.
|
|
298
301
|
- Never auto-select from "Named-only" section.
|
|
299
302
|
5. **Validate inputs** against model caps — see `references/workflows/cost-and-validation.md`.
|
|
300
303
|
6. **Fire the call(s)** — then follow "⚠️ Generation lifecycle" below for waiting, status, and failure handling.
|
|
@@ -440,3 +443,18 @@ HTML/SVG/Mermaid artifacts have a **Share** button in the preview toolbar that u
|
|
|
440
443
|
---
|
|
441
444
|
|
|
442
445
|
If at this point you still don't know which `references/` file to load, default to `references/models/prompt-copilot.md` for generation prompts or `references/workflows/cost-and-validation.md` for cost/validation questions, or just keep going with this core file's rules.
|
|
446
|
+
|
|
447
|
+
## Media selection preferences
|
|
448
|
+
Honor explicit models, presets, budget and inputs. Choose only eligible catalog candidates with all required capabilities. Use requested presets; otherwise use fitting presets when useful. For video generation, editing and lip-sync, when the user has not explicitly selected an output resolution, use the cheapest supported output resolution from the live catalog and pass it explicitly; do not inherit an expensive provider default. Preserve explicit user-selected resolution/settings. Finish fully, cinematic, professional, final, production and available credits are NOT permission to increase resolution. Never infer output resolution from reference media or export settings. A budget is a ceiling, not a spending target. Do not upscale or regenerate at a higher tier without explicit user authorization. If pricing or supported resolutions cannot be verified, inspect the catalog before dispatch; never invent a tier. Models with fixed output resolution use their native output. Never treat a policy refusal as a technical failure or route around safeguards.
|
|
449
|
+
Default images and edits: GPT Image 2.5 Flare/Sunburst; medium for value, high for ordinary maximum quality. Reserve xhigh/max for exceptional dense or difficult multilingual text after medium/high prove insufficient; do not automatically spend on retries. Nano Banana 2 is secondary. Seedream 5.0 Pro favors cinematic aesthetics over complex instruction fidelity; Wan 2.7 Pro is another creative alternative. Z Image/P Image for cheap tests. Midjourney for artistic concepts only, never editing. Soul V2 for realistic people/UGC concepts; derive character sheets before registering finished Visual DNA. Mirage Film 2 for environments and cinematic inspiration.
|
|
450
|
+
Default video: Seedance 2.5. Kling specializes in controlled single-image and first/last-frame shots. Wan 3.0 specializes in motion graphics and Hebrew/dialogue work. MiniMax H3 offers higher resolution; H3 Max favors speed at lower resolution. Gemini Omni Flash is a secondary Hebrew option (up to 10 seconds); Grok Imagine 1.5 and Seedance 2.0 are alternatives. P Video/Draft for cheap fast tests. Use base, edit or extend variants only with their required inputs.
|
|
451
|
+
Existing-video lip-sync: Sync 3 for active-speaker handling; PixVerse for cartoons/2D and economical faster work. Portrait lip-sync: Veed Fabric or HeyGen Avatar; P Avatar for budget work. LTX Audio to Video for camera/environment motion with audio-driven performance.
|
|
452
|
+
Default music: Suno v6. ElevenLabs Music is an alternative, especially for duration-directed scoring. Both accept custom duration requests; validate the selected tool schema and inspect actual output duration.
|
|
453
|
+
|
|
454
|
+
## Capability and result honesty
|
|
455
|
+
- Your current tools and mode define what you can do. In Act, use available authorized tools to generate/edit media, manage project assets and Visual DNA, import originals, or build/export an editable video timeline. In Ask or a prompt-only/final-answer stage without those tools, explain, write prompts or navigate; never pretend to execute. Do not claim access to every Kolbo feature, the user's browser/computer, private accounts or unrelated projects.
|
|
456
|
+
- Match the live model and tool schema to the task: inputs, reference slots, duration, resolution, quality, native audio and output format. A model preference is not a capability guarantee. Reuse recent catalog evidence; refresh only when missing or contradicted. If a requested operation is unavailable, explain the specific limitation and a supported alternative without silently changing the brief.
|
|
457
|
+
- Website media discovery reads public page markup and embedded data; it is not interactive browsing and does not execute page scripts. Discovery is not import. Use verified original assets, retain attribution and report blocked/missing assets honestly. When an existing logo is requested, never invent or substitute a generated logo for the original. Original logo design is a separate supported creative task when requested.
|
|
458
|
+
- Distinguish planned, submitted, running, generated, saved, exported and inspected. A playable render is not an editable timeline; only a successful editor-session result proves a saved edit exists. Tool success proves execution, not visual fidelity, exact logos, readable text, lip-sync or absence of black frames. Inspect with available tools before claiming quality; otherwise mark it unverified.
|
|
459
|
+
- A timeout is not proof of failure. Reconcile existing job IDs before retrying. Submit independent work together; describe queued versus running work truthfully. Never promise unlimited concurrency, automatic refunds, exact credit costs or approval outcomes. Use current receipts/pricing and preserve authorized budgets.
|
|
460
|
+
- Continue within granted autonomy and the current approved scope. Ask only for essential missing input or required approval. Content-policy refusals do not authorize switching providers to bypass safeguards. Support may review a restriction, but never promise an exception or entitlement.
|
|
@@ -32,9 +32,10 @@ Load this file when the user wants a **GPT Image 2 or GPT Image 2.5** image (Ope
|
|
|
32
32
|
|
|
33
33
|
## Latency vs Fidelity (recommend `quality` param)
|
|
34
34
|
|
|
35
|
-
- **low**:
|
|
36
|
-
- **medium**:
|
|
35
|
+
- **low**: only when the user prioritizes minimum cost or latency; medium remains the ordinary GPT Image 2.5 default.
|
|
36
|
+
- **medium**: default best price/quality for ordinary generations, edits and exploration.
|
|
37
37
|
- **high**: final assets, small/dense text, multi-font layouts, close-up portraits, identity-sensitive edits, infographics, diagrams, posters, UI with labels, scientific visuals, slides with charts/footnotes.
|
|
38
|
+
- **xhigh/max** (GPT Image 2.5 only, when listed): exceptional dense text or difficult multilingual/Hebrew typography after medium/high are insufficient. Do not auto-run retries or raise spending without authorization.
|
|
38
39
|
|
|
39
40
|
## Use Cases (text → image)
|
|
40
41
|
|
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
Load this file when the user wants to **build / create / generate an HTML presentation, slide deck, or pitch deck**. For landing pages see `models/landing-page.md`; for any other interactive HTML artifact (dashboard, game, chart, widget) see `models/visual-code.md`.
|
|
8
8
|
|
|
9
|
-
**
|
|
9
|
+
**Kobi Code routing:** write the artifact as a single HTML block in your reply. The Kobi Code panel renders it as a previewable artifact card. Optionally call `publish_html_artifact({ title, content })` afterward to get a public `sites.kolbo.ai` URL.
|
|
10
10
|
|
|
11
11
|
## 🚨 NON-NEGOTIABLE: Viewport Fitting
|
|
12
12
|
|
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
Load this file when the user wants to **build / create a landing page, marketing site, one-pager, product page, app launch page, SaaS sign-up page, or event page**. For slide decks see `models/html-presentation.md`; for dashboards / games / charts / widgets see `models/visual-code.md`.
|
|
8
8
|
|
|
9
|
-
**
|
|
9
|
+
**Kobi Code routing:** write the artifact as a single HTML block in your reply. Kobi Code's panel renders it as a previewable artifact card. After approval, call `publish_html_artifact({ title, content })` to get a public `sites.kolbo.ai` URL.
|
|
10
10
|
|
|
11
11
|
## 🎯 Design Thinking — Commit Before You Code
|
|
12
12
|
|
|
@@ -13,7 +13,7 @@ Load this file when the user wants a **Seedance 2.5** video (they said "2.5" / "
|
|
|
13
13
|
|
|
14
14
|
**Dialogue language: English.** Other languages are not reliably performed, and Hebrew does not work — it returns accented gibberish or English-shaped mouth movement. Never offer a user "Hebrew dialogue directly". This restriction applies to rendered dialogue/prose only, never to binding identifiers: preserve an exact stored Hebrew Visual DNA or moodboard tag such as `@אביב` / `#ישראל` literally. See `models/seedance.md` for the three honest alternatives.
|
|
15
15
|
|
|
16
|
-
**
|
|
16
|
+
**Use the cheapest supported tier unless the user selected an output resolution.** Resolution is a credit MULTIPLIER, not a flat rate. Relative to 720p: 480p ×0.44, 1080p ×2.25. A 30s pass costs ~540cr at 480p against ~1230cr at 720p and ~2770cr at 1080p. When no output resolution was selected and 480p is the cheapest supported tier, block the film at 480p, get the user's sign-off on staging, performance and timing, then re-run only the approved cut at a higher delivery resolution if the user explicitly authorizes that resolution increase. Approval of the creative cut alone does not authorize a more expensive resolution. If no output resolution was selected, use the cheapest supported tier from the live catalog even for final work; pass it explicitly.
|
|
17
17
|
|
|
18
18
|
## What's NEW in 2.5 (verified — never hedge)
|
|
19
19
|
|
|
@@ -8,7 +8,7 @@ Load this file when the user wants to **build an interactive HTML artifact where
|
|
|
8
8
|
|
|
9
9
|
If the user asks for a **presentation** → see `models/html-presentation.md`. If they ask for a **landing page** → see `models/landing-page.md`. Everything else visual-and-interactive is here.
|
|
10
10
|
|
|
11
|
-
**
|
|
11
|
+
**Kobi Code routing:** write the artifact as a single HTML block in your reply. Kobi Code's panel renders it as a previewable artifact card. Call `publish_html_artifact({ title, content })` to publish to `sites.kolbo.ai` after approval.
|
|
12
12
|
|
|
13
13
|
## What This Skill Is For
|
|
14
14
|
|
|
@@ -102,18 +102,15 @@ Normal cost formula: `final_cost = credit × output_seconds × resolution_multip
|
|
|
102
102
|
- ✅ Show them the supported set in one line and ask:
|
|
103
103
|
> "Seedance 2 elements supports `[720p, 1080p, 1440p, 2160p]` — 480p isn't available. Closest cheap option is 720p (~+0 credits over your intent). Want 720p, or pick another?"
|
|
104
104
|
- Only fire after they reply.
|
|
105
|
-
2. **
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
- final / production / hero → highest the user's budget allows (3K-4K / 1440p-2160p)
|
|
109
|
-
3. **No quality signal AND cost difference >2×** OR total batch ≥4 outputs → **ask the user once** with a one-line cost comparison, then default to standard if they don't reply.
|
|
110
|
-
4. **No quality signal AND cost difference ≤1.5×** → quietly use the cheapest supported, no need to interrupt.
|
|
105
|
+
2. **No explicit video output resolution**: choose the cheapest supported tier using current catalog pricing and pass it explicitly. This applies to drafts, normal work and final delivery alike. Do not default to 720p/1080p when a cheaper supported tier exists. Fixed-resolution models use their native output.
|
|
106
|
+
3. **Creative intent is not spending authorization**: "finish fully", "cinematic", "professional", "final", "production", "hero" and "don't ask me" do not authorize higher resolution, upscaling or a second high-resolution generation. A budget is a ceiling, not a target. Reference-video resolution and export resolution do not authorize matching generation resolution.
|
|
107
|
+
4. Preserve explicit user-selected settings. Otherwise proceed economically without a resolution approval loop. Inspect missing pricing/capabilities before dispatch. Upgrade only when the user explicitly selects a higher output tier or authorizes the resolution increase; never treat silence as approval. Image quality follows the image-model guidance (GPT Image 2.5 medium by default), not a generic final-work maximum.
|
|
111
108
|
5. **Sound on a video model with `sound_credit_multiplier > 1`** → if user didn't ask for sound, leave it off. If user said "with sound" / "with music", enable it.
|
|
112
109
|
|
|
113
110
|
## Defaults When Nothing Is Specified
|
|
114
111
|
|
|
115
112
|
- **Image**: `1K` (or the cheapest in `supported_resolutions`).
|
|
116
|
-
- **Video**:
|
|
113
|
+
- **Video**: cheapest supported output resolution from current catalog pricing, passed explicitly. Use the duration required by the user/task; do not lengthen clips to spend the available budget.
|
|
117
114
|
- **Sound**: respect `sound_enabled_by_default`; if false, leave off.
|
|
118
115
|
|
|
119
116
|
## Log Approved Resolution / Duration / Sound Choices
|
|
@@ -145,4 +145,4 @@ Fix errors before delivery. Report warnings that represent genuine creative trad
|
|
|
145
145
|
|
|
146
146
|
Read [workflows.md](references/filmmaking/workflows.md) for single shots, dialogue scenes, music performance, connected sequences, impossible shots, and feature workflows.
|
|
147
147
|
|
|
148
|
-
This workflow is part of the canonical Kolbo skill. The
|
|
148
|
+
This workflow is part of the canonical Kolbo skill. The Kobi Code sync pipeline mirrors it to MCP and plugin consumers; product surfaces may compile the same filmmaking truth through their own model adapters.
|
|
@@ -73,9 +73,9 @@ Branch on `failure.category` / `failure.retryable`:
|
|
|
73
73
|
- `retryable === true` (transient: network, rate limit, provider 5xx) → retry once with the same payload after a short pause. If it fails again, surface to user.
|
|
74
74
|
- `retryable === false` and unknown category → surface the raw `message` to the user, don't retry.
|
|
75
75
|
|
|
76
|
-
##
|
|
76
|
+
## Kobi Code Documentation
|
|
77
77
|
|
|
78
|
-
Full public documentation for
|
|
78
|
+
Full public documentation for Kobi Code (the CLI you are running inside) lives at **[docs.kolbo.ai/docs/kolbo-code](https://docs.kolbo.ai/docs/kolbo-code)**. If the user asks about installation, authentication, voice input, supported languages, commands, or how to uninstall, point them to the matching page below rather than guessing:
|
|
79
79
|
|
|
80
80
|
| Topic | Path |
|
|
81
81
|
|-------|------|
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Agentic Video Editor
|
|
2
|
+
|
|
3
|
+
Use these tools for a saved timeline the user can reopen and edit in Kolbo.
|
|
4
|
+
|
|
5
|
+
1. Call `list_video_editor_sessions` with the project ID, or use a session ID supplied by the user.
|
|
6
|
+
2. Call `get_video_editor_schema` for the current writable fields and operation schemas.
|
|
7
|
+
3. Call `get_video_editor_session`. Preserve its revision and stable track/item IDs.
|
|
8
|
+
4. Call `update_video_editor_session` with that revision as expected_revision and a bounded batch of operations. On REVISION_CONFLICT, read again and reapply the user's intent. Never blindly retry an old full timeline.
|
|
9
|
+
5. Read back the saved result. Export with `export_video_editor_session` only when a rendered deliverable is requested. Reuse a pending export's job_id.
|
|
10
|
+
|
|
11
|
+
For creation, `create_video_editor_session` accepts the original clips/audio/texts builder or advanced session_data, never both. Advanced data supports blank, text-only and caption-only timelines. Supply the name and authorized project ID. Import external media into Kolbo before adding its URL.
|
|
12
|
+
|
|
13
|
+
## Editing semantics
|
|
14
|
+
|
|
15
|
+
- session_flags changes isPinned, isArchived or isLocked using the same revision. A locked session requires an explicit isLocked=false request before content changes.
|
|
16
|
+
|
|
17
|
+
- session_data patches name, format, dimensions, fps, duration, background, and mediaLibrary. Supplying tracks replaces the full track array, so prefer operations for existing edits.
|
|
18
|
+
- Operations add/update/remove tracks and items, move items between tracks, order tracks, and order clips sequentially. Read the schema for exact argument names.
|
|
19
|
+
- Source trimStart and trimEnd are milliseconds removed from the source head and tail. originalDuration remains the source length. Changing trims or playbackRate recalculates timeline duration unless explicitly supplied. Fractional milliseconds are retained.
|
|
20
|
+
- reorder_items lays every item in one track end-to-end from the requested start. It does not move other tracks. move_item preserves duration. Track order is bottom-to-top.
|
|
21
|
+
- Nested settings replace the previous value. Use unset to clear an optional setting; do not merge grading presets.
|
|
22
|
+
- Session duration expands for new content but does not automatically shrink. Set duration explicitly to remove trailing background.
|
|
23
|
+
- Caption items support content, words, typography, RTL, background, stroke, shadow and captionStyle. Word startTime/endTime use absolute timeline milliseconds and must fit inside the caption item. Moving/reordering captions moves their words too. To derive captions from speech, use `transcribe_audio` first; edits themselves do not transcribe or generate media.
|
|
24
|
+
- Settings include transforms, opacity, visibility, audio volume/gain/fades, playback rate, grading, motion and the available text/caption effects. Consult the schema rather than inventing fields.
|
|
25
|
+
|
|
26
|
+
The tools edit saved state. Updated browser editors receive each saved change automatically and preserve the playhead and view. Unsaved manual edits pause autosave and show a conflict choice. Browser and agent writes both use revision checks. This requires the updated API and browser; it does not stream an agent's unsaved intermediate operations.
|
|
27
|
+
|
|
28
|
+
Saved settings and successful exports do not prove visual parity for every renderer effect. Inspect the output before claiming visual quality. Respect the user's deployment, publishing and generation-spend boundaries.
|
package/src/toolAnnotations.js
CHANGED
|
@@ -9,6 +9,7 @@
|
|
|
9
9
|
*/
|
|
10
10
|
|
|
11
11
|
const READ_ONLY = [
|
|
12
|
+
'get_video_editor_schema', 'list_video_editor_sessions', 'get_video_editor_session',
|
|
12
13
|
'list_fonts', 'get_font', 'get_font_upload_status',
|
|
13
14
|
'get_creative_director_status', 'get_generation_status', 'list_models',
|
|
14
15
|
'check_credits', 'show_plans', 'get_session_usage', 'list_voices',
|
|
@@ -67,6 +68,8 @@ const PRIVATE_WRITE = [
|
|
|
67
68
|
];
|
|
68
69
|
|
|
69
70
|
const DESTRUCTIVE_WRITE = [
|
|
71
|
+
'extend_music', 'cover_music',
|
|
72
|
+
'update_video_editor_session',
|
|
70
73
|
'delete_font',
|
|
71
74
|
'export_video_editor_session',
|
|
72
75
|
// These actions spend credits, enqueue irreversible work, or cancel it.
|
|
@@ -96,7 +99,8 @@ const DESTRUCTIVE_WRITE = [
|
|
|
96
99
|
'edit_review_comment', 'delete_review_comment',
|
|
97
100
|
];
|
|
98
101
|
|
|
99
|
-
const OPEN_WORLD_WRITE = [
|
|
102
|
+
const OPEN_WORLD_WRITE = [
|
|
103
|
+
'import_music_audio',
|
|
100
104
|
'publish_html_artifact', 'create_review_share_link', 'blender_capture_viewport',
|
|
101
105
|
];
|
|
102
106
|
|
package/src/tools/editor.js
CHANGED
|
@@ -2,12 +2,31 @@
|
|
|
2
2
|
const { z } = require('zod');
|
|
3
3
|
|
|
4
4
|
function registerEditorTools(server, client) {
|
|
5
|
+
const output = data => ({ content: [{ type: 'text', text: JSON.stringify(data) }] });
|
|
6
|
+
server.tool('get_video_editor_schema',
|
|
7
|
+
'Read the full writable Video Editor schema before advanced edits. Documents every clip, track, caption and session setting, timing units, revision checks and operation semantics.', {},
|
|
8
|
+
async () => output(await client.get('/v1/editor/schema')));
|
|
9
|
+
server.tool('list_video_editor_sessions',
|
|
10
|
+
'List saved Video Editor sessions in an accessible project. Paginated; use get_video_editor_session for full timeline data.',
|
|
11
|
+
{ project_id: z.string().min(1), limit: z.number().int().min(1).max(100).optional(), offset: z.number().int().min(0).max(100000).optional() },
|
|
12
|
+
async args => output(await client.get(`/v1/editor/sessions?${new URLSearchParams(Object.entries(args).filter(([, value]) => value !== undefined))}`)));
|
|
13
|
+
server.tool('get_video_editor_session',
|
|
14
|
+
'Read an existing Video Editor timeline: stable track/item IDs, all settings, clips, captions and revision. Always read before updating.',
|
|
15
|
+
{ session_id: z.string().min(1).max(100) },
|
|
16
|
+
async ({ session_id }) => output(await client.get(`/v1/editor/sessions/${encodeURIComponent(session_id)}`)));
|
|
17
|
+
server.tool('update_video_editor_session',
|
|
18
|
+
'Edit or rename a saved Video Editor session. First read get_video_editor_schema and get_video_editor_session. session_data patches settings (name, format, duration, fps, dimensions, background) or replaces tracks. operations support add/update/remove/move items and tracks, reorder_items and reorder_tracks. Full clip settings include source trims, speed, transforms, grading, audio, text and word-timed captions. Nested values replace rather than merge. All times are milliseconds. Requires the revision from the read; conflicts require re-reading. Changes save atomically without generation credits. Reload an already-open editor before manual edits.',
|
|
19
|
+
{ session_id: z.string().min(1).max(100), expected_revision: z.string().min(1).max(100),
|
|
20
|
+
session_flags: z.object({ isPinned: z.boolean().optional(), isArchived: z.boolean().optional(), isLocked: z.boolean().optional() }).strict().optional(),
|
|
21
|
+
session_data: z.record(z.unknown()).optional(), operations: z.array(z.record(z.unknown())).min(1).max(200).optional() },
|
|
22
|
+
async args => output(await client.patch('/v1/editor/sessions', args)));
|
|
5
23
|
server.tool('create_video_editor_session',
|
|
6
24
|
'Create an editable timeline from existing Kolbo-hosted images/video, audio layers and text. No new media generation. Requires project_id with edit access. Returns session_id and editor_url. Preserve the returned ID; creation is not a polling operation.',
|
|
7
25
|
{
|
|
26
|
+
session_data: z.record(z.unknown()).optional().describe('Advanced initial timeline settings and tracks; read get_video_editor_schema. Use instead of clips/audio/texts. Supports blank or caption-only sessions.'),
|
|
8
27
|
project_id: z.string().min(1), name: z.string().min(1).max(120),
|
|
9
28
|
format: z.enum(['16:9', '9:16', '1:1', '4:5', '21:9']).optional(),
|
|
10
|
-
clips: z.array(z.object({ url: z.string().url(), name: z.string().optional(), duration_ms: z.number().finite().min(0).max(1800000).optional(), trim_start_ms: z.number().finite().min(0).max(1800000).optional(), trim_end_ms: z.number().finite().min(0).max(1800000).optional(), volume: z.number().finite().min(0).max(2).optional(), muted: z.boolean().optional() })).min(1).max(60),
|
|
29
|
+
clips: z.array(z.object({ url: z.string().url(), name: z.string().optional(), duration_ms: z.number().finite().min(0).max(1800000).optional(), trim_start_ms: z.number().finite().min(0).max(1800000).optional(), trim_end_ms: z.number().finite().min(0).max(1800000).optional(), volume: z.number().finite().min(0).max(2).optional(), muted: z.boolean().optional() })).min(1).max(60).optional(),
|
|
11
30
|
audio: z.array(z.object({ url: z.string().url(), name: z.string().optional(), start_ms: z.number().finite().min(0).max(1800000).optional(), duration_ms: z.number().finite().min(0).max(1800000).optional(), volume: z.number().finite().min(0).max(2).optional(), fade_in_ms: z.number().finite().min(0).max(1800000).optional(), fade_out_ms: z.number().finite().min(0).max(1800000).optional() })).max(16).optional(),
|
|
12
31
|
texts: z.array(z.object({ content: z.string().min(1).max(2000), start_ms: z.number().finite().min(0).max(1800000).optional(), duration_ms: z.number().finite().min(0).max(1800000).optional(), font_size: z.number().finite().min(8).max(512).optional(), color: z.string().optional(), vertical: z.enum(['top', 'middle', 'bottom']).optional() })).max(120).optional(),
|
|
13
32
|
},
|
package/src/tools/generate.js
CHANGED
|
@@ -852,6 +852,96 @@ function registerGenerateTools(server, client, options = {}) {
|
|
|
852
852
|
}
|
|
853
853
|
);
|
|
854
854
|
|
|
855
|
+
// ─── music import / extend / cover ─────────────────────────
|
|
856
|
+
// Three tools over the SDK's /v1/generate/music/{import,extend,cover}. The web app has
|
|
857
|
+
// had upload-extend and upload-cover for a long time; agents could not reach either
|
|
858
|
+
// until those routes existed.
|
|
859
|
+
const musicSourceFields = {
|
|
860
|
+
audio_url: z.string().optional().describe('PUBLIC URL of the source audio. Either this (with rights_confirmed) or upload_id is required. Local file? Call upload_media first and pass the returned https:// URL.'),
|
|
861
|
+
upload_id: z.string().optional().describe('Upload id from import_music_audio. Use this to reuse one imported file across several calls instead of re-importing it.'),
|
|
862
|
+
rights_confirmed: z.boolean().optional().describe('REQUIRED with audio_url: confirms the caller holds the rights to the source audio. Not needed when passing upload_id (it was confirmed at import).'),
|
|
863
|
+
prompt: z.string().optional().describe('What the new material should sound like.'),
|
|
864
|
+
style: z.string().optional().describe('Music style / genre for the new material.'),
|
|
865
|
+
title: z.string().optional().describe('Title for the result. Generated if omitted.'),
|
|
866
|
+
instrumental: z.boolean().optional().describe('Produce instrumental only, no vocals. Default: false'),
|
|
867
|
+
lyrics: z.string().optional().describe('Custom lyrics. Omit to have them generated, or set instrumental.'),
|
|
868
|
+
model: z.string().optional().describe('Model identifier. Omit for the Suno default.'),
|
|
869
|
+
project_id: projectIdField,
|
|
870
|
+
session_id: sessionIdField
|
|
871
|
+
};
|
|
872
|
+
|
|
873
|
+
server.tool(
|
|
874
|
+
'import_music_audio',
|
|
875
|
+
'Import a PUBLIC audio URL into Kolbo as a reusable music source, returning an upload_id for extend_music / cover_music. Use it when the same track feeds several calls — both of those tools also accept audio_url directly for a one-shot. LOCAL FILE? Call upload_media first and pass the returned https:// URL; this tool cannot read the caller\'s disk. You must set rights_confirmed: true — Kolbo requires the caller to hold the rights to audio they upload.',
|
|
876
|
+
{
|
|
877
|
+
audio_url: z.string().describe('PUBLIC URL of the audio file to import.'),
|
|
878
|
+
upload_type: z.string().optional().describe('What the import is for: "extend" or "cover". Must match the tool you later call with it. Default: "extend".'),
|
|
879
|
+
rights_confirmed: z.boolean().describe('REQUIRED true: confirms the caller holds the rights to this audio.'),
|
|
880
|
+
confirmation_text: z.string().optional().describe('Optional free-text rights confirmation recorded with the upload.'),
|
|
881
|
+
project_id: projectIdField,
|
|
882
|
+
session_id: sessionIdField
|
|
883
|
+
},
|
|
884
|
+
async ({ audio_url, upload_type, rights_confirmed, confirmation_text, project_id, session_id }) => {
|
|
885
|
+
const r = await client.post('/v1/generate/music/import', {
|
|
886
|
+
audio_url, upload_type: upload_type || 'extend', rights_confirmed, confirmation_text, project_id, session_id
|
|
887
|
+
});
|
|
888
|
+
return result(r);
|
|
889
|
+
}
|
|
890
|
+
);
|
|
891
|
+
|
|
892
|
+
// Both generation tools share one runner — the only difference is the route and the label.
|
|
893
|
+
const runMusicSourceOp = async (route, toolName, args) => {
|
|
894
|
+
const { model: rawModel, continue_at, ...rest } = args;
|
|
895
|
+
const model = await canonicalModelId(client, rawModel, 'music_gen');
|
|
896
|
+
const gen = await client.post(route, { ...rest, model, ...(continue_at !== undefined ? { continue_at } : {}) });
|
|
897
|
+
|
|
898
|
+
if (returnsImmediately()) return submittedResult({
|
|
899
|
+
tool: toolName, kind: 'audio', gen, client, model: model || 'Suno', prompt: rest.prompt,
|
|
900
|
+
settings: { mode: rest.instrumental ? 'instrumental' : (rest.style || undefined) },
|
|
901
|
+
});
|
|
902
|
+
|
|
903
|
+
const poll = await pollOrTimedOut(client, gen.generation_id, {
|
|
904
|
+
interval: (gen.poll_interval_hint || 8) * 1000,
|
|
905
|
+
timeout: 150000
|
|
906
|
+
});
|
|
907
|
+
if (poll.timedOut) return poll.timedOut;
|
|
908
|
+
const res = poll.result;
|
|
909
|
+
|
|
910
|
+
return uiCompleted({
|
|
911
|
+
tool: toolName, kind: 'audio', gen, client, model: model || 'Suno', prompt: rest.prompt,
|
|
912
|
+
settings: { mode: rest.instrumental ? 'instrumental' : (rest.style || undefined) },
|
|
913
|
+
urls: res.result.urls,
|
|
914
|
+
playback_urls: res.result.playback_urls,
|
|
915
|
+
tracks: res.result.tracks,
|
|
916
|
+
title: res.result.title,
|
|
917
|
+
duration: res.result.duration,
|
|
918
|
+
credits_used: creditFields(res).credits_used,
|
|
919
|
+
}, JSON.stringify({
|
|
920
|
+
...creditFields(res),
|
|
921
|
+
session_id: gen.session_id,
|
|
922
|
+
urls: res.result.urls,
|
|
923
|
+
tracks: res.result.tracks,
|
|
924
|
+
title: res.result.title,
|
|
925
|
+
duration: res.result.duration,
|
|
926
|
+
lyrics: res.result.lyrics
|
|
927
|
+
}, null, 2));
|
|
928
|
+
};
|
|
929
|
+
|
|
930
|
+
server.tool(
|
|
931
|
+
'extend_music',
|
|
932
|
+
'Continue an existing track — Kolbo generates new music that carries on from a point in the source audio. Pass audio_url (with rights_confirmed) or an upload_id from import_music_audio. There is NO target-length control here: the length follows the source and continue_at, which is why duration_seconds is not accepted. Returns the final audio URL when complete.',
|
|
933
|
+
{ ...musicSourceFields, continue_at: z.number().optional().describe('Seconds into the source audio to continue FROM. Omit to continue from the end.') },
|
|
934
|
+
async (args) => runMusicSourceOp('/v1/generate/music/extend', 'extend_music', args)
|
|
935
|
+
);
|
|
936
|
+
|
|
937
|
+
server.tool(
|
|
938
|
+
'cover_music',
|
|
939
|
+
'Re-record an existing track in a new style, keeping its musical identity — a lo-fi cover of a rock song, an acoustic take on an electronic track. Pass audio_url (with rights_confirmed) or an upload_id from import_music_audio. There is NO target-length control here: a cover follows the length of its source, which is why duration_seconds is not accepted. Returns the final audio URL when complete.',
|
|
940
|
+
{ ...musicSourceFields },
|
|
941
|
+
async (args) => runMusicSourceOp('/v1/generate/music/cover', 'cover_music', args)
|
|
942
|
+
);
|
|
943
|
+
|
|
944
|
+
|
|
855
945
|
// ─── generate_speech ───────────────────────────────────────
|
|
856
946
|
server.tool(
|
|
857
947
|
'generate_speech',
|