aitoearn-dsh-plugin-test 0.0.2 → 0.0.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/lib/client.js +34 -36
  2. package/lib/client.js.map +2 -2
  3. package/package.json +57 -40
  4. package/presets/aitoearn/lib/mcp.js +85 -21
  5. package/presets/aitoearn/lib/mcp.js.map +2 -2
  6. package/presets/aitoearn/package.json +14 -13
  7. package/skills/cloudedit/SKILL.md +113 -0
  8. package/skills/cloudedit/references/amk-scenarios/edit-asr.md +160 -0
  9. package/skills/cloudedit/references/edit-position.md +64 -0
  10. package/skills/cloudedit/references/edit-project.md +192 -0
  11. package/skills/cloudedit/references/edit-quality-gate.md +136 -0
  12. package/skills/cloudedit/references/edit-shader.md +114 -0
  13. package/skills/cloudedit/references/edit-text.md +227 -0
  14. package/skills/cloudedit/references/edit-timeline.md +29 -0
  15. package/skills/cloudedit-planner/SKILL.md +102 -0
  16. package/skills/cloudedit-planner/evals/evals.json +80 -0
  17. package/skills/cloudedit-planner/references/editing-plan.md +63 -0
  18. package/skills/cloudedit-planner/references/executable-handoff.md +25 -0
  19. package/skills/cloudedit-planner/references/media-evidence-models.md +32 -0
  20. package/skills/cloudedit-planner/references/media-understanding.md +25 -0
  21. package/skills/cloudedit-planner/references/processing-and-lineage.md +22 -0
  22. package/skills/cloudedit-planner/references/subtitle-timing.md +50 -0
  23. package/src/dsh/client/AitoearnCard.js +2 -2
  24. package/src/dsh/client/AitoearnCard.js.map +1 -1
  25. package/src/dsh/client/card-store.d.ts +2 -2
  26. package/src/dsh/client/card-store.js +18 -9
  27. package/src/dsh/client/card-store.js.map +1 -1
  28. package/src/dsh/client/index.js +15 -13
  29. package/src/dsh/client/index.js.map +1 -1
  30. package/src/dsh/index.d.ts +2 -1
  31. package/src/dsh/index.js +4 -6
  32. package/src/dsh/index.js.map +1 -1
  33. package/src/dsh/mcp.js +90 -21
  34. package/src/dsh/mcp.js.map +1 -1
  35. package/src/dsh/settings-schema.d.ts +16 -12
  36. package/src/dsh/settings-schema.js +8 -10
  37. package/src/dsh/settings-schema.js.map +1 -1
@@ -0,0 +1,192 @@
1
+ # MediaKit Edit Project-Editing Fundamentals
2
+
3
+ ## Purpose
4
+
5
+ This skill provides the general MediaKit project model, editing-operation context, project-access workflow, and editor usage rules. It does not provide detailed tool parameters, complete task manuals, or generation-prompt recipes. Load the matching MediaKit Edit skill and use the tool schema to implement the workflow for a specific task.
6
+
7
+ ## Role
8
+
9
+ When working with a MediaKit Edit project, act as a professional video-editing assistant. Users think in clips, edits, stories, and visible results rather than data structures. Apply video-editing judgment to clarify requirements, recommend a concrete strategy, and execute the required edits.
10
+
11
+ Before creative or strategic work that will shape the output, align on the requirements and concrete strategy: video purpose, content format, output format, source-material strategy, creative direction, or editing approach. Mechanical operations such as renaming, small property changes, obvious reversals, or edits to items explicitly identified by the user can be executed directly.
12
+
13
+ ## Environment
14
+
15
+ MediaKit Editor is a MediaKit-based multitrack nonlinear video editor. A project contains one timeline with its own Canvas (FPS, width, and height), video tracks, audio tracks, text tracks, effect tracks, a shared material pool, and a shared asset library.
16
+
17
+ Every CloudEdit tool in this agent takes `project_id`. Pass the id returned by `create_project` or `list_projects`.
18
+
19
+ MCP tools connect to the MediaKit Editor server and map directly to writes, so previews and changes are real and visible. Do not write directly to the database or guess hidden IDs; read them from matching project, track, element, or material results.
20
+
21
+ You and the user work from project data, tool results, transcripts, materials, and composed-timeline evidence. Do not assume project state, the material pool, timeline layout, or element state remains unchanged over time because the user may have edited the project manually. **The latest `preview_timeline`, `list_materials`, `inspect_elements`, `inspect_materials`, and `read_project` results are your sole basis for editing.**
22
+
23
+ ## Collaboration
24
+
25
+ You normally collaborate with another editor, and either party may change the timeline at any time. The work is generally concurrent rather than parallel. If the data model is missing, inconsistent, or misaligned, refresh your knowledge promptly through the MCP tools.
26
+
27
+ ## Data Model
28
+
29
+ ### Project
30
+
31
+ A project is the top-level container. It owns a shared material pool, shared asset library, and one timeline. The timeline defines its canvas and contains tracks, items, and local timeline structure.
32
+
33
+ ### Timeline
34
+
35
+ A timeline contains multiple tracks that arrange elements to form a complete video edit. `preview_timeline` provides the latest snapshot of the timeline layout.
36
+
37
+ ### Canvas
38
+
39
+ The Canvas is the render area for visible elements and includes width, height, background color, and export frame rate. A timeline has one Canvas. Treat its dimensions as the visible area of the finished result when editing element position and size.
40
+
41
+ ### Material
42
+
43
+ A material is source media in the project material pool. One material can be used by multiple tracks on the timeline.
44
+
45
+ Before `sync_materials`, call `get_media_metadata` for each HTTP(S) URL. Video and audio require integer `duration_ms`, set to `round(duration × 1000)` from the tool's duration in seconds. Video and images require `width` and `height`. Do not guess these numbers. Use `probe_video_metadata`, `probe_audio_metadata`, or `probe_image_metadata` only for a `vod://` or `tos://` address, or when an image needs average hue or BlurHash. Video and audio probes return completed metadata (`format_meta` and the stream meta). Use that result before syncing. Image probe returns immediately.
46
+
47
+ Agent-facing material types include video, audio, images, GIFs, motion graphics, and SVG. Source media, file name, remote readiness, properties, and other content-level attributes belong to the material.
48
+
49
+ When a user asks to use, edit, place, replace, caption, trim, inspect, or otherwise process a material that is not explicitly present in context, do not immediately treat it as missing. The user can upload materials directly in MediaKit Editor, so first call `list_materials` and match by file name, type, visible content, transcription status, or other available metadata.
50
+
51
+ Ask the user to upload or provide a material only after checking the project and confirming that it is absent, not ready, inaccessible, or ambiguous.
52
+
53
+ ### Track
54
+
55
+ A track is a visual or audio channel on the timeline.
56
+
57
+ Video tracks render in layers. A higher video track renders above a lower one: its elements cover lower video during their duration, while an empty upper track reveals lower video. If audio continues with no visible video element, the rendered Canvas may be black.
58
+
59
+ Audio tracks are mixed in parallel. They do not mask one another; multiple audio tracks playing at the same time are all audible.
60
+
61
+ Elements on the same track must not overlap.
62
+
63
+ Place consecutive clips on one track in increasing time order. Layered visual elements such as filters, effects, and text belong on higher text/effect tracks above the content they cover.
64
+
65
+ ### Element
66
+
67
+ An element is an instance of content on the timeline. It references media material (video, audio, image, GIF, or SVG), text, or an asset (effect or filter), and has positional and temporal properties.
68
+
69
+ Editing an element can change when or where its content appears: timeline start, duration, track, position, dimensions, opacity, fades, source offset, or playback speed. Editing a material can change reusable source content or transition properties; editing text can change its style.
70
+
71
+ Timeline positions and durations use milliseconds natively. User-facing summaries may use seconds or milliseconds.
72
+
73
+ The user may add or delete assets in real time.
74
+
75
+ ### Asset
76
+
77
+ An asset is a built-in MediaKit Edit capability for visual or audio enhancement, including animation, filters, and effects.
78
+
79
+ Assets are normally fixed and users do not modify existing ones. When an asset is missing from context, `list_resources` is the sole source for verifying whether it exists.
80
+
81
+ If it does not exist, tell the user that no built-in asset matches the requested ID, name, or type.
82
+
83
+ ### Editing Operations: Gap, Ripple, and Absorb
84
+
85
+ Timeline editing preserves gaps by default.
86
+
87
+ - Deleting an element does not move later elements to close the gap unless Ripple is explicitly enabled.
88
+ - Adding an element does not move later elements to make room unless Ripple is explicitly enabled.
89
+ - Shortening an element leaves a gap, while lengthening it may run out of room. Move later items deliberately when a gap should close or space must be created. An insertion into an occupied interval on the same track is rejected if the edit has not made room.
90
+ - Absorb applies only when shortening or lengthening an element. When enabled, it automatically moves later elements to close a gap or make room.
91
+
92
+ Ripple, Absorb, and transitions affect only the same track. **After Ripple, Absorb, or another structural edit, dependent tracks such as subtitles and music may no longer align with the edited dialogue or video and must be checked.**
93
+
94
+ For overlap conflicts, first decide whether the content is sequential or layered. Sequential content belongs on one track in time order; layered content belongs on a higher video track. Account for masking relationships when layering.
95
+
96
+ ### Editing Operation: Automatic Element Placement
97
+
98
+ If no track ID is supplied, the tool automatically searches upward from the bottom for a suitable available track. This is useful for quickly adding an element to an existing track, but not for placing an element beneath another to create masking.
99
+
100
+ Automatic placement can be combined with `ripple` and `absorb`, and prefers a track that can satisfy those behaviors.
101
+
102
+ ## Element Trimming and Playback Speed
103
+
104
+ Audio and video elements have both source-media time and timeline time. `source_range` identifies the selected interval in the source material; `from_time_ms` is the element's start position on the timeline, not a source-media timestamp; and `duration_ms` is the element's displayed duration on the timeline.
105
+
106
+ Choose between the following update modes according to the user's intent when changing playback speed:
107
+
108
+ - To change only the playback speed while preserving the complete current source range, pass only `playback_rate`. The MCP keeps the current source range unchanged and automatically recalculates the timeline duration as `timelineDuration = ceil(sourceRangeDuration / playback_rate)`.
109
+ - To declare the final displayed duration, pass both `duration_ms` and `playback_rate`. In this case, `duration_ms` is only an assertion about the speed-change result and must exactly equal the automatically recalculated duration. A mismatch rejects the entire batch and does not extend or shorten the source range.
110
+
111
+ When an existing element has no explicit `source_range`, the MCP derives and persists a source range starting at 0 using `ceil(current timeline duration × previous playback rate)`, then recalculates the timeline using the new playback rate. An update cannot pass `source_start_ms` and `playback_rate` together. To trim and change speed, first trim in a separate operation and read back the timeline, then set the playback rate separately. Adds may still combine `source_start_ms`, `duration_ms`, and `playback_rate` to create a specified source range.
112
+
113
+ Changing playback speed may alter the element's end time and either leave or consume timeline space. Pass `from_time_ms` only when the user asks to move the element; do not treat it as a speed-change parameter. When space on the same track must be closed or created, use `absorb` according to the tool constraints and read back the result with a fresh `preview_timeline`. Do not treat manual calculations as a verified timeline.
114
+
115
+ ## Alignment and Execution
116
+
117
+ ### Aligning Before Action
118
+
119
+ Understanding the intended result is foundational to creative editing. Clarify before making creative or strategic choices.
120
+
121
+ Match the amount of alignment to what the user already provided:
122
+
123
+ - “Cut this interview into a one-minute YouTube version” provides platform and duration but may still require deciding what to retain.
124
+ - “Turn this podcast into a highlight clip” is vague; align on target platform, duration, and what counts as a highlight.
125
+ - For “make a promotional video for our app,” if a product URL is provided without brand assets, align on logo/materials, platform, aspect ratio, duration, production approach, and tone.
126
+ - “Add English subtitles” is clear and narrow; execute directly.
127
+ - After prior alignment, “make it shorter” or “continue” normally needs no new alignment round.
128
+ - Without a product type, “make a promotional video for my product” is ambiguous; clarify the product type, target platform, and use case before choosing what to show.
129
+
130
+ ### When to Align
131
+
132
+ Align first when a request involves a new project, vague creative intent, paid or time-consuming generation without creative detail, multicamera or multimaterial consistency, or major choices such as narration versus instrumental music, cinematic versus casual tone, retained content, or emphasized features.
133
+
134
+ For important dependent steps, confirm the foundation before building downstream content whenever practical. Selection uses what you already saw and heard. Subtitles additionally need `asr_subtitles` and [edit-asr](./amk-scenarios/edit-asr.md). If a cut still has no source time, follow the planner [timestamp fallback](../../cloudedit-planner/references/media-understanding.md). Do not start another content-understanding pass during execution.
135
+
136
+ ### When to Skip Alignment
137
+
138
+ Do not start another alignment round when the user has already specified the goal, style, and constraints; the task is mechanical and reversible; the user says to continue; the user gives a follow-up correction; or the user explicitly requests end-to-end execution.
139
+
140
+ ### How to Align Well
141
+
142
+ Ask only load-bearing questions. Do not use a fixed checklist or ask for information that project state, materials, transcripts, or visual evidence can establish. The user should answer only preferences, requirements, or missing-material questions that genuinely require their decision.
143
+
144
+ ## Verification Before Changes
145
+
146
+ Before changing timeline elements, tracks, materials, or assets, refresh only the necessary discovery stage if its information may be stale or unknown. The user may have edited the project manually in the browser since the previous round.
147
+
148
+ Do not rely on stale item IDs, track layouts, material readiness, transcription status, or earlier timeline positions for project-scoped edits.
149
+
150
+ `read_project` returns only project mapping, Canvas, source-material counts from the Materials API, and the timeline directory (tracks and element counts). Omitted details are unknown, not empty. Use `preview_timeline` for timeline tracks, paginated elements, and gaps. Narrow timeline reads with `track_type`, `tracks`, `elements`, `from_time`, or `to_time`.
151
+
152
+ Use `inspect_elements` with `element_ids` to retrieve full details for placed elements in batches, `list_materials` for the source-material pool, and `inspect_materials` for source-material details. Follow `nextOffset` when the required element or material is not on the current page. Do not call multiple discovery stages in parallel or rebuild the entire topology by default.
153
+
154
+ ## Execute Only What Was Requested
155
+
156
+ Stop after completing the user’s request. Do not silently add unrequested music, subtitles, transitions, filters, color grading, or other enhancements. You may suggest additions when useful, but do not execute them without user intent.
157
+
158
+ At editing checkpoints, use `preview_timeline` as the structural review. Do not export merely because the timeline changed. Export only when the user asks for a file, a render, a download, or final delivery.
159
+
160
+ `preview_timeline` returns structure rather than a viewer. When the user asks for a file, call `export_project`. The finished result includes `result.video_url`. Do not claim the file exists before that result arrives. A broad request such as “edit this video” still delivers the editable project, not an MP4.
161
+
162
+ Do not infer export intent from broad requests such as “edit this video,” “make it shorter,” “clean it up,” or “make a version.” By default, a MediaKit Edit request delivers a reviewable editable timeline, not a downloadable MP4. Inferred verification is not user approval. After verification, keep the live project available and let the user choose whether to continue editing or export.
163
+
164
+ When reporting a reviewable edit, pair a concise result summary with a natural next step based on the visible interface. If the editor is open or available, naturally tell the user that they can click Play in the editor to view the result. Match the wording to the conversation and context rather than using a fixed approval script.
165
+
166
+ At a MediaKit Edit review checkpoint, “project,” “version,” “edit,” or “put it in MediaKit Edit” means an editable MediaKit Edit timeline unless the user explicitly requests a standalone finished file. Do not satisfy a MediaKit Edit editing request by placing only a locally rendered, flattened MP4 on the timeline.
167
+
168
+ ## How to Think About Editing
169
+
170
+ Start from project context: which materials exist, where they are on the timeline, what they say, and what the audience sees and hears. Go deeper only when needed.
171
+
172
+ Editing has a natural order: establish structure, refine pacing, then add polish. Reversing that order creates rework because subtitles, video, audio, and music depend on the final structure.
173
+
174
+ Think from the audience’s visual and auditory experience rather than from individual tracks alone. Account for how track order and masking relationships shape what is seen and heard.
175
+
176
+ Before reporting completion, run the currently applicable gate checks. Do not run a general persistence check for every add, update, or delete, and do not check the Canvas. An explicit gap on one track is not a black frame; a global visual-coverage hole is only a weak warning. Distinguish an `in_progress` intermediate state from a `review_ready` state according to the downstream editing plan. Missing visuals in an intermediate state or intentional black can proceed after recording the reason. A `preview_timeline` projection cannot prove that the raw document has no overlaps or that layers look correct. After an important structural edit, read back the timing and parameters of known dependent subtitles, videos, images, and music.
177
+
178
+ After completing a logical editing batch, execute the `timeline_review` in the [editing quality gate](./edit-quality-gate.md). Do not repeat it between consecutive tool writes in the same batch. If structural changes affect downstream subtitles, audio, effects, or packaging, add one `structure_checkpoint` before entering the dependent stage. An automatic gate pass means only that the applicable hard checks for this batch passed; it may coexist with non-blocking visual-coverage warnings. Visual appearance, perceived audio quality, and the final file must be reported as manual-review items, never as automatically verified.
179
+
180
+ ### Handling Visual Gaps
181
+
182
+ When a video contains a visual gap, delete the blank interval and close the timeline gap so the surrounding visuals remain continuous, unless a specific editing technique requires it—such as an intentional black frame or rhythmic pause—or the user explicitly requires it to remain. Judge visual gaps from the final composited output across all visible tracks; do not mistake a gap on a single track for a blank frame. Temporary visual gaps may remain while an edit is in progress, but they must be resolved before delivering a review-ready version. When retaining a gap, record the editing intent or user requirement that justifies it.
183
+
184
+ When closing a visual gap, move all affected audio, subtitles, effects, and other dependent elements together, then read back and verify their timing and synchronization. Do not close only the video-track gap and leave the audio or subtitles misaligned.
185
+
186
+ ## Design-Style Consistency
187
+
188
+ Design style is the project’s visual identity: its colors, fonts, styling, and the visual logic among elements. It primarily shapes how elements are assembled and also governs visual elements such as subtitles, filters, and effects.
189
+
190
+ When work spans multiple related visual outputs, align on or follow a coherent design style before batch production so the project reads as one series. Do not fix a design style based on unconfirmed assumptions.
191
+
192
+ Unless the user requests it, skip design-style work for a one-off quick fix. A single corner badge or small overlay does not automatically create a project-wide design-style decision.
@@ -0,0 +1,136 @@
1
+ # MediaKit Edit Editing Quality Gate
2
+
3
+ ## Purpose and Current Boundaries
4
+
5
+ After a logical editing batch completes, this gate reads back the real Editor state and checks the verifiable project structure and parameters affected by the batch.
6
+
7
+ The current Editing API provides access only to projects, project materials, preset assets, and one Editor document. An Editor `PUT` fully replaces the document with an optional revision. Current MCP write tools first read the Editor, then atomically write back the full document using the retrieved revision while preserving unknown fields.
8
+
9
+ `preview_timeline` does not provide a viewer or a rendered composite. `export_project` can produce a file when the user asks for one, but that file is outside this gate. This gate can automatically pass only the **Editor project structure**, not final composite pixels, perceived audio quality, or encoded-file quality.
10
+
11
+ There is currently no general “write result” check: the gate does not separately verify that every add, update, or delete in the batch was persisted. It also does not check Canvas dimensions or background color. Other applicable checks may still read back the existing element state they require.
12
+
13
+ ## Available Tools and Evidence
14
+
15
+ | Tool | How the gate uses it | What it cannot establish |
16
+ | --- | --- | --- |
17
+ | `read_project` | Obtain the revision, total project duration, source-material counts, and track directory | `assets` contains only total and per-type counts from the Materials API, not material entries or statuses; contains no viewer; this gate does not check the Canvas |
18
+ | `preview_timeline` | Read elements, explicit gaps, revision, and pagination by track and time window | Returns a read-only projection; ordinary same-track overlaps are projected at their midpoint, so it cannot prove that the raw document has no overlaps; does not return material sources, a viewer, or transcripts |
19
+ | `inspect_elements` | Read back time, track, material, transform, audio parameters, text style, grading, and transitions in batches using current element IDs | Does not return a revision or prove actual rendered appearance |
20
+ | `list_materials` | Obtain material IDs from the Materials API, optionally filtered by processing state or timeline usage | List items do not include the `status` value itself; call `inspect_materials` to confirm one material |
21
+ | `inspect_materials` | Confirm material existence, `status`, duration, dimensions, and allowlisted technical properties | Does not return source imagery, transcript, or rights status |
22
+ | `edit_element` / `split_element` / `edit_track` | Repair deterministic local issues; use `validate_only=true` before writing when layout is uncertain | `validate_only` proves only that a request can be submitted, not that it was written; IDs created during validation cannot be reused |
23
+
24
+ The Studio API may return HTTP 200 for both successful requests and business errors on recognized routes. Through MCP, the client checks the mutually exclusive `data` / `error` envelope and converts business errors into tool failures. Do not bypass the MCP to perform a partial `PUT` directly against the Editor.
25
+
26
+ ## Trigger Timing
27
+
28
+ | Profile | Trigger | Scope |
29
+ | --- | --- | --- |
30
+ | `structure_checkpoint` | A batch of structural writes that changes downstream timing has completed, and subtitles, audio, effects, or packaging will be generated next | Affected tracks, from the earliest changed point through the end of the dependent range, plus known dependent elements |
31
+ | `timeline_review` | The current logical editing batch is complete, before reporting the result to the user | A final readback proportionate to the risk of the changes |
32
+
33
+ - Trigger the gate by logical editing batch, not MCP call count. Combine consecutive writes for the same user intent and affected range into one batch.
34
+ - Determine editing state from the current editing plan: use `in_progress` while planned video, images, subtitles, or packaging within the range are still missing; use `review_ready` when the agent is ready to deliver the range as the current completed result and no known visual work remains.
35
+ - For a change only to one element’s text, volume, style, or position, read back that element and only the necessary adjacent window; do not scan the entire timeline.
36
+ - For a fixed-position add, delete, move, or trim, check the affected track and neighboring elements before and after the change point.
37
+ - For `ripple`, `absorb`, `tightenTrack`, track reordering, or batch speed changes, check from the earliest change point to the end of that track. Check dependent tracks only when cross-track dependent elements are known.
38
+ - Fully paginate every relevant track only for full-video assembly, whole-track tightening, or an explicit user request to inspect the entire video.
39
+ - Do not trigger the gate when a write fails and project state remains unchanged. If the user explicitly requests step-by-step previews, run it at their requested checkpoints.
40
+
41
+ ## Execution Procedure
42
+
43
+ 1. From the batch editing plan, record affected tracks, time ranges, known dependent elements, and applicable checks. Do not create a general persistence checklist of adds, updates, and deletes.
44
+ 2. Call `read_project` and record `gate_revision` and the track directory.
45
+ 3. Call `preview_timeline` for the scope determined above. Follow `pagination.next_offset` until the target range is complete. Its revision must equal `gate_revision`.
46
+ 4. Call `inspect_elements` only for existing target elements and known dependent IDs required by applicable checks, with at most 100 per batch; split larger sets into batches.
47
+ 5. Call `inspect_materials` for every material ID newly referenced or replaced in the batch. When video or audio has a trim, also verify material duration.
48
+ 6. Call `read_project` again. If its revision differs from `gate_revision`, the project changed during gate readback. Rerun the gate once from the latest revision. If it changes again, return `blocked`, report concurrent editing, and do not combine evidence from different revisions.
49
+ 7. Run each applicable check. If a hard check fails, repair only the relevant element, track, or time window, then rerun the gate for the invalidated scope. Do not automatically repair visual-coverage warnings.
50
+
51
+ ## Automatically Executable Checks
52
+
53
+ | Code | Pass condition | Tool evidence | Failure handling |
54
+ | --- | --- | --- | --- |
55
+ | `REVISION_UNSTABLE` | Both `read_project.revision` values and `preview_timeline.revision` are equal | `read_project`, `preview_timeline` | Retry once from the latest revision; if it changes again, stop and report concurrent editing |
56
+ | `TRACK_STATE_INVALID` | The main video track, track order, and track types satisfy current structural constraints, with no unexpected change to virtual empty tracks | `read_project.tracks`; `edit_track(listTracks)` if needed | Restore the relevant track action; do not infer a general write result from element counts |
57
+ | `AFFECTED_WINDOW_INVALID` | The target window is fully readable, contains no `PREVIEW_OVERLAP_UNRESOLVABLE`, and orders adjacent elements according to plan | Target `preview_timeline` window and pagination | Repair ordering or abnormal nested overlap; do not treat the read-only projection as proof that the raw document has no overlaps |
58
+ | `VISUAL_COVERAGE_WARNING` | Emit a weak warning for a global visual-coverage hole in the affected visual window, distinguishing `in_progress` from `review_ready` | Fully paginated `preview_timeline` without `tracks` or `elements` filters; current batch editing plan | Do not block or auto-repair. If the agent confirms the hole is an intermediate area awaiting content or intentional black/blank space, mark it `acknowledged` and continue |
59
+ | `MATERIAL_UNRESOLVED` | Every newly or newly-referenced material can be read exactly, has `status=complete`, and matches the element type | `inspect_materials`; material IDs from `list_materials` | Wait for completion, use a valid material, or stop and explain the gap |
60
+ | `SOURCE_RANGE_INVALID` | For trimmed video/audio, `source_start_ms >= 0` and the source range end is greater than `source_start_ms` without exceeding `inspect_materials.duration_ms`; display `duration_ms` and `playback_rate` for all changed media are valid positive values | `inspect_elements`, `inspect_materials` | Adjust source in-point, duration, or speed and check again |
61
+ | `DEPENDENT_TIME_STATE_INVALID` | Known dependent elements still satisfy planned time relationships, such as subtitle ranges, audio boundaries, or effect coverage windows | `inspect_elements`; dependency IDs and relationships from the batch plan | Reorder only identified dependent elements; the MCP cannot discover semantic dependencies automatically |
62
+ | `TEXT_STATE_INVALID` | Actual values for changed text—including content, time, font, size, color, bold, alignment, animation, and transform—match the request | Root `content`, `offset_x`, and `offset_y`; `inspect_elements.text_style`; and transform fields | Correct divergent fields and read them back again |
63
+ | `AUDIO_STATE_INVALID` | Time, `volume`, `muted`, fades, and `playback_rate` of changed audio/video elements match the request; if the plan defines a video end, known audio does not extend beyond it | `inspect_elements`, `preview_timeline` | Correct time or audio parameters; do not infer loudness or perceived quality |
64
+ | `TRANSITION_STATE_INVALID` | Planned adjacent main-track boundaries have the correct outgoing/incoming IDs, asset ID, and duration; boundaries that need no transition have none | `preview_timeline.has_transition`, `inspect_elements.transition` | Repair only the relevant main-track boundary |
65
+
66
+ Execute only checks relevant to the current batch. The gate does not require discovering or changing unrelated elements merely to fill a checklist.
67
+
68
+ ### Determining Timeline Gaps
69
+
70
+ `preview_timeline` returns gaps per track. An empty interval on one track does not mean other visual tracks have no image, so never report a single-track gap as a “black frame” or “possible black frame.”
71
+
72
+ Run `VISUAL_COVERAGE_WARNING` when the batch changes visual structure and a visual review window can be identified:
73
+
74
+ 1. Call `preview_timeline` for the window without `tracks` or `elements` filters, and follow `next_offset` through every page.
75
+ 2. Collect `timeline_range` for every item on every track where `kind=item` and `type != audio`, then take the union of those intervals. Do not directly merge per-track gap records.
76
+ 3. Subtract that union from the review window. Only the remaining intervals are “global visual-coverage holes.” Do not infer the review window automatically from project `duration_ms`, because audio or effects may outlast the main visual content.
77
+ 4. Determine whether the range is `in_progress` or `review_ready` from the current editing plan. Do not assume editing is finished merely because the gate is running.
78
+ 5. `in_progress`: if a later step will add imagery, subtitles, or packaging, the agent may confirm the hole as an intermediate editing state, record why it remains, mark it `acknowledged`, and continue editing.
79
+ 6. `review_ready`: record `warning` by default. The agent may still confirm intentional black or blank space from an explicit user request, an approved editing plan, or a black/blank interval deliberately arranged by the agent in this batch; record the reason and mark it `acknowledged`.
80
+
81
+ The agent’s judgment must come from current user intent or an established editing plan. Never claim “intentional black” without evidence merely to pass the gate. Neither `warning` nor `acknowledged` blocks the gate, and neither should trigger automatic filling.
82
+
83
+ A visual-coverage warning never changes overall status to `needs_repair` or `blocked`. It proves only that the interval structurally contains no non-audio element. It does not inspect the Canvas and is not pixel evidence; it cannot detect black frames inside a visual element, decoding failure, transparent elements, or elements outside the Canvas.
84
+
85
+ ## Items Currently Requiring Editor Review
86
+
87
+ The existing MCP and Editing API cannot automatically verify the following, so do not mark them automatically as `passed`:
88
+
89
+ - final composite pixels, black frames within visual elements, subject integrity after cropping, actual layer occlusion, or subtitle safe areas;
90
+ - cut-off actions, jump-cut quality, visual repetition, actual transition appearance, or scene continuity;
91
+ - dialogue clarity, BGM masking speech, clipping, background noise, unintended silence, loudness, or actual audio/video synchronization;
92
+ - copy accuracy, material copyright, or reference-footage usage rights unless the current task provides explicit external evidence;
93
+ - final MP4 decoding, resolution, frame rate, audio/video stream coverage, or file integrity. A finished `export_project` result can show that the file was produced; it does not prove those perceptual or decode checks. This gate does not mark them `passed`.
94
+
95
+ When a batch affects visuals, text, audio, or transitions, include the corresponding `manual_review` item in the result and ask the user to play the edit in the editor. Do not present `manual_review` as automated evidence that already ran.
96
+
97
+ ## Status and Output
98
+
99
+ - `passed`: all applicable hard checks for the batch passed. Non-blocking `warning` or `acknowledged` items may also be present. This means only that the Editor project gate passed.
100
+ - `needs_repair`: at least one automated check failed and can be repaired locally.
101
+ - `blocked`: materials, permissions, or a stable revision are unavailable, so consistent evidence cannot be formed.
102
+
103
+ ```json
104
+ {
105
+ "scope": "editor_structure",
106
+ "profile": "timeline_review",
107
+ "editState": "in_progress",
108
+ "projectRevision": 12,
109
+ "status": "passed",
110
+ "checkedRange": {
111
+ "trackIds": ["video-track-id"],
112
+ "from_time_ms": 4000,
113
+ "toTimeMs": 9000
114
+ },
115
+ "checks": [
116
+ {
117
+ "code": "AFFECTED_WINDOW_INVALID",
118
+ "status": "passed"
119
+ }
120
+ ],
121
+ "warnings": [
122
+ {
123
+ "code": "VISUAL_COVERAGE_WARNING",
124
+ "status": "acknowledged",
125
+ "from_time_ms": 6200,
126
+ "toTimeMs": 6500,
127
+ "reason": "A title card will be added during the later packaging stage"
128
+ }
129
+ ],
130
+ "manualReview": [
131
+ "Play 4–9 seconds in the editor and confirm the actual cut and transition appearance"
132
+ ]
133
+ }
134
+ ```
135
+
136
+ When reporting to the user, state the actual checked scope, any failures or repairs, and the visual or audio qualities that still require confirmation in the editor. Do not describe an `editor_structure` pass as passing final-video quality.
@@ -0,0 +1,114 @@
1
+ # MediaKit Edit Shaders
2
+
3
+ Use `list_resources` to obtain the exact `resource_id` of a Shader already available to the project.
4
+
5
+ `list_resources` is the sole source of truth for built-in effects, built-in transitions, and project effect/transition assets. Built-in assets use stable global asset IDs rather than project-level database assets, so they do not appear in `list_materials`.
6
+
7
+ Recommended catalog-browsing patterns:
8
+
9
+ ```text
10
+ list_resources(resource_types: ["effect"])
11
+ list_resources(resource_types: ["effect"], offset: 10, limit: 10)
12
+ list_resources(resource_types: ["lut"])
13
+ ```
14
+
15
+ Resource categories use the same names as element types: `effect` for effects and `lut` for filters. After querying filters, copy the returned resource ID unchanged into a `type: "lut"` element in `edit_element`:
16
+
17
+ ```text
18
+ list_resources(resource_types: ["lut"])
19
+ edit_element(adds: [{"type": "lut", "resource_id": "<lut-resource-id>", ...}])
20
+ ```
21
+
22
+ ## Effect Elements and Tracks
23
+
24
+ Effect elements include effects and filters. Effects provide visual changes such as split screens and inversion; filters provide complete color-grading presets. Effects and filters can be placed only on effect tracks, and every element has an explicit time range.
25
+
26
+ An effect element applies to all visual tracks during its own time range and does not follow the upper/lower masking relationship of ordinary video tracks.
27
+
28
+ ## Transitions and Tracks
29
+
30
+ All examples in this section use the midpoint timeline returned by `preview_timeline`. `*` means the preceding element has an outgoing transition; `->` means an ordinary hard cut:
31
+
32
+ ```text
33
+ A(0,200) * B(200,300) -> C(300,400)
34
+ ```
35
+
36
+ Adding a transition between elements overlaps the tail of the outgoing time with the head of the incoming time.
37
+
38
+ A transition can connect only two chronologically adjacent `video` or `image` elements on the main video track. Adjacent means no other element lies between them. Adding a transition requires `outgoing_element_id`, `incoming_element_id`, and a positive integer `duration_ms`; both clips’ timeline durations must be at least as long as the transition.
39
+
40
+ Do not construct the overlap manually. `edit_element` adjusts the incoming element and its successor chain to produce an overlap of the requested duration. Adding a transition to an ordinary hard cut shortens the total preview duration by one transition duration. The tool validates only the clips’ current timeline durations; it does not check for unused source headroom or tailroom and does not extend `trim` automatically.
41
+
42
+ After adding, replacing, or deleting a transition, call `preview_timeline` again. Never calculate the new timeline yourself.
43
+
44
+ **The time basis returned by `preview_timeline` is your sole operational foundation. You do not need to account for the transition overlap itself; the tool returns the most appropriate timeline and supports operations against that returned timeline.**
45
+
46
+ ### Transition Read Semantics
47
+
48
+ The project stores overlapping time intervals. `preview_timeline` projects those overlaps into an end-to-end midpoint timeline and projects `source_range` consistently with `playback_rate`. **Base all later movement, trimming, and speed changes on this midpoint timeline.**
49
+
50
+ `inspect_elements` returns the stored overlapping time, full source range, and transition parameters. Do not use its stored intervals directly as midpoint trim times.
51
+
52
+ ### Preserving Transitions After Editing
53
+
54
+ During a structural edit, `edit_element` first converts all existing transitions into midpoint hard cuts, performs the batch edit, and then restores transitions against the final timeline.
55
+
56
+ An existing transition is preserved or rebound only when all of the following are true:
57
+
58
+ - The original outgoing element still exists and remains on the main video track.
59
+ - The original outgoing element and its final successor are both `video` or `image`.
60
+ - The final midpoint timeline strictly satisfies `outgoing.end == incoming.start`.
61
+ - The original visible duration of both clips is at least the original transition duration.
62
+ - The batch did not explicitly delete or replace the transition on that outgoing element.
63
+
64
+ The incoming element may change. As long as the original outgoing element ends with an eligible adjacent successor, its original transition binds to that successor:
65
+
66
+ ```text
67
+ Original timeline: A(0,200) * B(200,300) -> C(300,400)
68
+
69
+ 1. Shorten A by 100ms and close successors with absorb; transition is preserved
70
+ A(0,100) * B(100,200) -> C(200,300)
71
+
72
+ 2. Shorten A by 100ms without closing the gap; transition disappears implicitly
73
+ A(0,100) -> Gap(100,200) -> B(200,300) -> C(300,400)
74
+
75
+ 3. Swap B and C; original transition rebinds between A and C
76
+ A(0,200) * C(200,300) -> B(300,400)
77
+
78
+ 4. Insert D between A and B and reorder successors; original transition rebinds between A and D
79
+ A(0,200) * D(200,300) -> B(300,400) -> C(400,500)
80
+ ```
81
+
82
+ If the elements remain strictly adjacent but either clip is shorter than the original transition duration, the entire batch fails and rolls back; the transition is not silently deleted.
83
+
84
+ ### Implicit Disappearance and Explicit Deletion
85
+
86
+ When movement or trimming creates a Gap, the transition disappears implicitly. The midpoint hard-cut timing remains, and the full overlap time previously occupied by the transition is not restored.
87
+
88
+ Explicitly delete a transition with:
89
+
90
+ ```text
91
+ edit_element(deletes: [{"id": "<outgoing-element-id>", "transition": true}])
92
+ ```
93
+
94
+ Explicit deletion removes the transition from the outgoing element and shifts the incoming element and its successor chain right by the original overlap duration. Implicit disappearance and explicit deletion produce different timelines, so read `preview_timeline` again after either operation.
95
+
96
+ Simple trimming or speed changes can be edited directly, using `absorb` as needed to preserve midpoint adjacency. For complex rearrangement, use the sequence “explicitly delete transition → read timeline again → edit elements → read timeline again → add transition.”
97
+
98
+ `split_element` does not preserve transitions before or after the split element. To restore them, record the transition parameters first, complete the split, and add them again using the new element IDs.
99
+
100
+ ### Reserving Source-Material Headroom
101
+
102
+ By default, adding a transition consumes already displayed content on both sides and shortens the timeline. If the user wants to preserve the original cut points and content, first confirm that both materials have enough headroom. Then make room from the back of the timeline toward the front and extend the outgoing tail and incoming head. Each side normally needs half the transition duration. For example, if A plays for 300ms and B for 400ms, a 100ms transition requires 50ms of extra tail on A and 50ms of extra head on B. In special cases, reserve more or less on either side as needed to satisfy the 100ms transition and the user’s intent.
103
+
104
+ Throughout the process, operate only on the logical timeline returned by `preview_timeline` and keep same-track elements from overlapping. The MCP creates the actual overlap required by the transition. An existing transition is preserved automatically when the two edited segments remain adjacent and long enough; add a transition again only when creating or replacing it.
105
+
106
+ Run validation before committing the edit, then read the timeline again after completion. If there is insufficient source headroom, shorten the transition or keep the hard cut.
107
+
108
+ ### Default Decision
109
+
110
+ When the user gives no explicit direction, close gaps and preserve transitions during speed changes, trimming, or movement.
111
+
112
+ ## Transitions and Alignment
113
+
114
+ Transitions and their structural edits adjust only their main track. Subtitles, music, and other tracks do not follow automatically, so check cross-track alignment after every structural edit.