@pixerate/editor 0.83.0 → 0.85.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@pixerate/editor",
-  "version": "0.83.0",
+  "version": "0.85.0",
   "description": "UI-agnostic rich text and media editing core library",
   "author": "Pixerate",
   "license": "MIT",

package/skills/pixerate-video-editor/SKILL.md

@@ -38,12 +38,17 @@ For errors, namespaced tool names and undo conventions, see `pixerate-agent-tool
    `slip_video_clip` (change what a shot shows), `slide_video_clip` (move a shot between its
    neighbours); for a whole stretch, `split_video_at`, `lift_video_range` or
    `extract_video_range`. Animate opacity, volume or position with `set_video_keyframes`.
-6. **Check.** `check_video_project` and fix every error and warning; `find_video_gaps` for
+6. **Cut by text** when the media has speech: `set_video_transcript` for each spoken asset (or
+   check `transcriptWords` in `get_video_project`), `get_video_transcript` to read the edit, then
+   `find_video_fillers`, `find_video_pauses` or `find_video_transcript_text` and pass the ranges
+   you want gone to `remove_video_ranges` in one call. Re-read the transcript afterwards: every
+   time after a cut has moved.
+7. **Check.** `check_video_project` and fix every error and warning; `find_video_gaps` for
    unintended black or silence.
-7. **Look and listen.** `get_video_frame` at cuts and transitions for layers and audio gains as
+8. **Look and listen.** `get_video_frame` at cuts and transitions for layers and audio gains as
    numbers; `render_video_frame` with `times` for a contact sheet of the sequence. Render
    after a batch, not after every edit.
-8. **Export** when asked: `export_video_ffmpeg` (a command to run, not a render) or
+9. **Export** when asked: `export_video_ffmpeg` (a command to run, not a render) or
    `export_video_edl` (one track) for finishing elsewhere. Read the returned notes. For a
    delivery loudness, have the host run `export_video_loudness_measure` first and pass what it
    prints as `loudness.measured`; one pass (`loudness` without `measured`) is fine for a draft.
@@ -62,8 +67,9 @@ For errors, namespaced tool names and undo conventions, see `pixerate-agent-tool
   dialogue and scene changes.
 - **Transitions sparingly.** A straight cut is the default. Use `crossfade` for passage of time
   or a soft scene change, `dip` (to black, or a `color`) for a section break, `wipe` rarely.
-  Keep them short (300-1000 ms). Crossfades and wipes need handles: half the duration of
-  unused media past the cut on both clips; `dip` needs none.
+  Keep them short (300-1000 ms). `push` and `slide` suit graphics, montages and social edits
+  more than drama; `iris` is a stylised period or comic effect. Every type but `dip` needs
+  handles: half the duration of unused media past the cut on both clips; `dip` needs none.
 - **Insert vs overwrite.** `add_video_clip` and `move_video_clip` take `mode`: `refuse`
   (default; the error names the next free place), `overwrite` (replace what is underneath, like
   an overwrite edit) or `insert` (push later material along). Use `ripple: true` on trim and
@@ -78,6 +84,10 @@ For errors, namespaced tool names and undo conventions, see `pixerate-agent-tool
 - **Keyframes sparingly.** Use `set_video_keyframes` for a push-in (`scale` 1 to 1.1 over the
   shot), a duck under dialogue (`volume`), or a fade that is not at an edge; ease (`ease-in-out`)
   rather than linear for motion. Simple fades at the clip edges are `fadeIn` / `fadeOut`.
+- **Tighten dialogue by text first.** With a transcript, remove fillers and pauses with the find
+  tools and one `remove_video_ranges`; review the fillers list first (a "you know" can be
+  meant). Keep 150-300 ms around words (`keep`) so speech breathes. Without a transcript, use the
+  sound itself, below.
 - **Tighten dialogue by ear, in numbers.** `find_video_silences` on an interview or voiceover clip
   lists the pauses in timeline time; extract the long ones (keep 150-300 ms either side so speech
   does not sound clipped: shrink each range before `extract_video_range`, and extract from the
@@ -92,12 +102,24 @@ For errors, namespaced tool names and undo conventions, see `pixerate-agent-tool
   `volume` multiply. Give every audio clip that starts or ends mid-sound a short `fadeIn` /
   `fadeOut` (10-50 ms) to avoid clicks; fade music beds over 1-3 s. Shape sound fades with
   `fadeInCurve` / `fadeOutCurve`: `equal-power` where two sounds overlap (it keeps loudness
-  steady, where linear dips in the middle), `exponential` to fade music out to silence
+  steady, where linear dips in the middle; transitions' `audioCurve` already defaults to it for
+  crossfades and wipes), `exponential` to fade music out to silence
   naturally, `s-curve` for a gentle start and finish. Linear is the default.
 - **Delivery loudness.** Normalise the final mix rather than pushing clip volumes: -14 LUFS for
   web and streaming, -16 for podcasts, -23 (EBU R128) or -24 (ATSC) for broadcast, with true
   peaks at or under -1 dBTP. Two passes keep the dynamics; one pass rides the level.
-- **Titles.** Keep text inside the title-safe area (about 90% of the frame), hold it at least as
+- **Grade in two passes: correct, then look.** First balance each shot so a scene matches
+  (`update_video_clip` `grade`: fix exposure and white balance against the best shot, then
+  `copy_video_grade` from it to the rest of the scene), then add a look with
+  `apply_video_grade_preset` at a modest `amount` (0.3-0.6). Check with `render_video_frame` at a
+  few times side by side. Skin tones are the reference: avoid strong `hue` or `tint` on people.
+- **Blend overlays, don't key them.** Light leaks, flares, film burns and glows shot on black go on
+  a higher video track with `blend: "screen"` (or `add` for a hotter look), so the black drops
+  out; textures, paper and dust on white use `multiply`. Lower the clip's `opacity` to taste.
+  A blend affects everything beneath the clip on lower tracks.
+- **Titles.** Bring titles in and out with `animateIn` / `animateOut` rather than hand-made
+  keyframes: `rise` or `fade` for lower thirds, `pop` for a playful callout, `zoom` for a slow
+  reveal; 300-600 ms in, a little quicker out. Keep text inside the title-safe area (about 90% of the frame), hold it at least as
   long as it takes to read twice, and use `background` for contrast over busy footage.
 - **Pacing.** Vary shot length with content; trim heads and tails of every clip to the
   essential action rather than using whole source files.
@@ -112,6 +134,10 @@ For errors, namespaced tool names and undo conventions, see `pixerate-agent-tool
 | --- | --- |
 | `get_video_project` | Start here. Returns frame size, fps, length (ms), assets (id, kind, src, duration), tracks top to bottom with each clip in time order (id, type, start, duration, end, source in/out, fades, transition), markers, playhead, selection, issues and whether it can be edited. |
 | `get_video_frame` | Inspect a `time` (ms) as data: pictures back to front (clip, track, asset, source time, opacity, placement rect, rotation, wipe) and sounds (clip, source time, gain after volumes, fades and transitions). Use to verify a cut, layer order or J/L cut without images. |
+| `get_video_transcript` | Read what the timeline says on one `track` (default: the first audio track with transcribed media): the text, and each word with its timeline `start` / `end` ms and `clip`. `from` / `to` narrow it; `limit` (default 1000) with `more`. Words cut out of the edit are not listed. |
+| `find_video_transcript_text` | Find every place a word or phrase (`query`, case and punctuation ignored) is said, as `{ from, to, text }` timeline ranges for `remove_video_ranges`, `split_video_at` or `seek_video`. |
+| `find_video_fillers` | Find filler words (um, uh, erm, you know, I mean…; or your own `fillers`) as timeline ranges, longest phrase first, with their `total` ms. Review before cutting. |
+| `find_video_pauses` | Find gaps between words longer than `min` (default 700 ms), less `keep` (default 150 ms) next to each word, snapped inward to frames, as timeline ranges with their `total`. |
 | `find_video_gaps` | List empty stretches between clips (and before the first) as `{ track, start, end }` ms, on every track or one `track`. Then `close_video_gaps` or ripple. |
 | `check_video_project` | After every batch and before export. Lists issues errors first with a code (`missing-asset`, `wrong-track`, `overlap`, `source-overrun`, `transition-without-cut`, `transition-too-long`, `transition-handles`, `fades-too-long`, `unknown-duration`, `unused-asset`, `marker-past-end`, `empty`), what it is about and how to fix it. |
 | `render_video_frame` | Look at the picture at `time`, or at 2-16 `times` as one labelled contact sheet (`columns` 1-8). SVG by default; `format: "png"` only when the host can rasterize. Video without a `poster` shows as a labelled box with its source timecode. |
@@ -126,6 +152,8 @@ For errors, namespaced tool names and undo conventions, see `pixerate-agent-tool
 | `get_video_audio_levels` | Measure a media `clip`'s source sound (timeline times) or an `asset`'s (source times) between `from` and `to`: peak dBFS in each of `buckets` (default 50, up to 500) equal stretches, and the loudest peak. Levels are the recording before clip volume and fades. Use it to find where speech or a beat starts or to judge loudness. The first call for an asset decodes its file. |
 | `find_video_silences` | List stretches of a `clip` or `asset` that stay below `thresholdDb` (default -40) for at least `minDuration` ms (default 300). For a clip the `{ start, end }` are timeline times that go straight into `extract_video_range` (cut a pause, closing up) or `split_video_at`. |
 | `seek_video` | Show the person a moment: `time`, a `clip`'s start or a `marker`, with `play` true or false. Does not change the project. Returns the playhead and clips under it. |
+| `set_video_transcript` | Attach what is said in a video or audio `asset`: `text` (a Whisper JSON, .srt or .vtt file's contents) or `words` (`[{ word, start, end }]` ms of the source). Captions give approximate word timings. Returns the word count and first and last words. Clear it with `update_video_asset` `remove: ["transcript"]`. |
+| `remove_video_ranges` | Cut `ranges` (`[{ from, to }]` timeline ms, as the find tools return them) out and close them up on every sync-locked track, as one undo step: merged where they touch, extracted last first. Returns how many stretches and how much time went, and the new length. |
 | `set_video_settings` | Set `name`, `width` / `height` (pixels, 16-8192), `fps` (e.g. 24, 25, 30, 60) or `background` colour. One undo step. |
 | `add_video_asset` | Register a file: `kind` (video, audio, image), `src` (URL, path or data URL), `duration` ms (required in practice for video and audio), optional `id`, `name`, `width`, `height`, `hasAudio` (default true for video), `poster`. Returns the asset id. |
 | `update_video_asset` | Relink or correct an asset via `set` (src, name, duration, size, hasAudio, poster) and `remove`. Refused if clips would then run past its end. |
@@ -145,10 +173,12 @@ For errors, namespaced tool names and undo conventions, see `pixerate-agent-tool
 | `lift_video_range` | Empty `from`..`to` ms on all unlocked tracks or the given `tracks`, leaving a gap: clips inside are deleted, clips across an edge trimmed, a clip across both cut in two. Returns what was removed, trimmed and created. |
 | `extract_video_range` | Remove `from`..`to` ms and close it up: as lift, then later material moves earlier on those and every sync-locked track. Refused (naming the clip) when a sync-locked track outside `tracks` has material there. Use to cut a section out of a finished sequence. |
 | `delete_video_clips` | Remove `clips`; `ripple: true` pulls later clips up to close the gaps (a ripple delete). |
-| `update_video_clip` | Change properties via `set`: opacity, transform (`x`, `y` px from centre, `scale`, `rotation`), `fit` (contain default for media, cover, fill, none), `fadeIn` / `fadeOut`, name; media `volume` (0-4), `muted`, `fadeInCurve` / `fadeOutCurve` (the sound's fade shape: linear, equal-power, s-curve, exponential, logarithmic), `speed` (0.05-20, keeps the source range so the length changes), `asset`; title text and style; colour. `remove` resets to defaults. Not for timing. |
+| `update_video_clip` | Change properties via `set`: opacity, transform (`x`, `y` px from centre, `scale`, `rotation`), `fit` (contain default for media, cover, fill, none), `animateIn` / `animateOut` (`{ effect: fade, rise, drop, slide-left, slide-right, zoom or pop, duration ms default 500 }`), `grade` (`exposure` stops -3-3, `contrast`, `saturation`, `temperature`, `tint` -1-1, `hue` degrees; replaces the whole grade), `blend` (normal, multiply, screen, overlay, darken, lighten, color-dodge, color-burn, hard-light, soft-light, difference, exclusion, add), `fadeIn` / `fadeOut`, name; media `volume` (0-4), `muted`, `fadeInCurve` / `fadeOutCurve` (the sound's fade shape: linear, equal-power, s-curve, exponential, logarithmic), `speed` (0.05-20, keeps the source range so the length changes), `asset`; title text and style; colour. `remove` resets to defaults. Not for timing. |
+| `apply_video_grade_preset` | Give `clips` (on video tracks) a named look at an `amount` 0-1 (default 1), replacing their grades: warm, cool, punchy, faded, black-and-white, bleach-bypass, day-for-night. One undo step; returns each clip's grade to fine-tune with `update_video_clip`. |
+| `copy_video_grade` | Match shots: give the `to` clips the grade of the `from` clip (one undo step; a source with no grade clears theirs). Returns the grade copied. |
 | `set_video_keyframes` | Animate one `property` of a `clip` (`opacity` 0-1, `volume` 0-4 for media, `x` / `y` px from centre, `scale`, `rotation` degrees) with `keys` `[{ t, v, ease? }]`, replacing its keys. `t` is clip time (ms from the clip's start), so keys travel with the clip. `ease`: linear (default), hold, ease-in, ease-out, ease-in-out or cubic-bezier points. Returns the sorted keys. |
 | `remove_video_keyframes` | Remove all keys of a clip `property`, leaving the value it had at the clip's start. |
-| `set_video_transition` | Set the transition into a `clip` from the clip ending where it starts on the same track: `{ type: crossfade, dip or wipe, duration ms centred on the cut, color (dip, default black), direction (wipe, default right) }`, or `null` to remove. When handles are short the error gives the longest that fits. |
+| `set_video_transition` | Set the transition into a `clip` from the clip ending where it starts on the same track: `{ type: crossfade, dip, wipe, push, slide or iris, duration ms centred on the cut, color (dip, default black), direction (wipe, push, slide; default right), audioCurve (the sound's fades: equal-power by default for crossfade and wipe, linear for dip) }`, or `null` to remove. When handles are short the error gives the longest that fits. |
 | `close_video_gaps` | Slide clips on a `track` earlier so no gaps remain, from time `from` (default the start). Returns the clips that moved. |
 | `add_video_marker` | Add a marker at `time` with `label`, `color` and optional `id`, for beats, sections or review notes. Returns its id. |
 | `remove_video_marker` | Remove a `marker` by id. |
@@ -162,11 +192,16 @@ For errors, namespaced tool names and undo conventions, see `pixerate-agent-tool
   impossible transition is refused whole (unless you asked for `mode: "overwrite"` or
   `"insert"`); read the message (it names the next free place or the longest trim /
   transition that fits) and retry with those numbers.
+- **Animations write keyframes for you.** `animateIn` / `animateOut` move opacity, x, y and scale;
+  if the clip keys one of those itself, the animation leaves it alone (`animation-keyed`), and
+  entrance plus exit longer than the clip are shortened (`animations-too-long`).
 - **Numbers replace keyframes.** Setting `opacity` or `volume` as a plain
   number (in `add_video_clip` or `update_video_clip`) drops that property's keys; animate with
   `set_video_keyframes` after the static settings, not before.
 - **Slip, slide and roll clamp.** They move as far as the media allows and say how far in
   `delta`; check it instead of assuming the full amount was applied.
+- **Times move after a cut.** Ranges from the find tools are valid for the timeline as it was;
+  pass them all to one `remove_video_ranges` (it cuts last first), or find again after each cut.
 - **`edit_video` has no result references.** Unlike `edit_animation`, ops cannot say `$0.id`;
   set clip, track and marker ids yourself when later ops in the batch need them.
 - **`edit_video` op shapes differ slightly** from the single tools: `add_asset` wraps the asset