aivee 0.0.0-stage → 0.1.0-dev.7025438

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (73) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +117 -2
  3. package/docs/architecture.md +401 -0
  4. package/docs/aspect-ratio-presets.md +72 -0
  5. package/docs/releasing.md +19 -0
  6. package/docs/usage.md +1193 -0
  7. package/package.json +50 -4
  8. package/requirements-otio.txt +1 -0
  9. package/src/capabilities.mjs +307 -0
  10. package/src/cli.mjs +646 -0
  11. package/src/doctor.mjs +133 -0
  12. package/src/edits/analysis-to-edl.mjs +43 -0
  13. package/src/edits/detect-silence.mjs +51 -0
  14. package/src/edits/find-mistake-candidates.mjs +42 -0
  15. package/src/edits/plan-shot-placements.mjs +105 -0
  16. package/src/edits/plan-silence-cuts.mjs +49 -0
  17. package/src/edits/retime-transcript.mjs +105 -0
  18. package/src/files.mjs +80 -0
  19. package/src/index.mjs +44 -0
  20. package/src/interchange/otio.mjs +204 -0
  21. package/src/interchange/otio_bridge.py +521 -0
  22. package/src/media/analysis-provider.mjs +75 -0
  23. package/src/media/compatibility.mjs +45 -0
  24. package/src/media/ffmpeg-capabilities.mjs +129 -0
  25. package/src/media/frame-rate.mjs +58 -0
  26. package/src/media/frame-time.mjs +22 -0
  27. package/src/media/preview-assets.mjs +350 -0
  28. package/src/media/probe.mjs +149 -0
  29. package/src/media/render-cuts.mjs +90 -0
  30. package/src/media/scene-detection.mjs +107 -0
  31. package/src/media/scene-review.mjs +205 -0
  32. package/src/project/assets.mjs +1194 -0
  33. package/src/project/create.mjs +100 -0
  34. package/src/project/presets.mjs +44 -0
  35. package/src/render/checkpoints.mjs +192 -0
  36. package/src/render/formats.mjs +163 -0
  37. package/src/render/hyperframes.mjs +590 -0
  38. package/src/render/process.mjs +111 -0
  39. package/src/render/progress.mjs +46 -0
  40. package/src/render/timeline-control.mjs +53 -0
  41. package/src/timeline/assemble.mjs +119 -0
  42. package/src/timeline/audio-effects.mjs +75 -0
  43. package/src/timeline/audio-mastering.mjs +236 -0
  44. package/src/timeline/audio-mix.mjs +105 -0
  45. package/src/timeline/audio-track-effects.mjs +139 -0
  46. package/src/timeline/audio-volume.mjs +63 -0
  47. package/src/timeline/background.mjs +8 -0
  48. package/src/timeline/blend-mode.mjs +34 -0
  49. package/src/timeline/caption-layouts.mjs +61 -0
  50. package/src/timeline/captions.mjs +80 -0
  51. package/src/timeline/edit.mjs +965 -0
  52. package/src/timeline/fingerprint.mjs +6 -0
  53. package/src/timeline/from-edl.mjs +31 -0
  54. package/src/timeline/history-store.mjs +281 -0
  55. package/src/timeline/history.mjs +69 -0
  56. package/src/timeline/inspect.mjs +117 -0
  57. package/src/timeline/keyframes.mjs +158 -0
  58. package/src/timeline/layers.mjs +18 -0
  59. package/src/timeline/mask.mjs +53 -0
  60. package/src/timeline/project.mjs +52 -0
  61. package/src/timeline/render.mjs +759 -0
  62. package/src/timeline/sidechain-render.mjs +209 -0
  63. package/src/timeline/source-fit.mjs +15 -0
  64. package/src/timeline/speech-bubble.mjs +26 -0
  65. package/src/timeline/speed-ramp.mjs +188 -0
  66. package/src/timeline/track-state.mjs +22 -0
  67. package/src/timeline/transition.mjs +34 -0
  68. package/src/timeline/validate.mjs +322 -0
  69. package/src/transcript/corrections.mjs +21 -0
  70. package/src/transcript/normalize.mjs +17 -0
  71. package/src/transcript/provider.mjs +42 -0
  72. package/src/transcript/subtitles.mjs +57 -0
  73. package/src/transcript/validate.mjs +39 -0
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Waren Gonzaga
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,118 @@
1
- # Temporary Holding Version
1
+ # AIVEE
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ <p align="center">
4
+ <img src="https://ghrb.waren.build/render/banner?header=AIVEE&amp;subheader=AI%20Video%20Editing%20Engine%20for%20AI%20workflows&amp;bg=0D1117-1B2B4B&amp;color=F0F6FC&amp;subheadercolor=C7D2E5&amp;headerfont=Inter&amp;subheaderfont=Inter" alt="AIVEE — AI Video Editing Engine for AI workflows">
5
+ </p>
6
+
7
+ <p align="center">
8
+ <strong>A local-first video editing CLI and API for AI workflows.</strong><br>
9
+ Build timelines, review edits, compose layers, and render with Node.js.
10
+ </p>
11
+
12
+ ## ✨ Features
13
+
14
+ - **Generic timeline:** editable video, audio, and overlay tracks with validated clips, transitions, keyframes, masks, speech-bubble captions, and other overlays.
15
+ - **Media workflow:** register and verify assets, create proxies and previews, inspect scenes, and plan reviewable B-roll edits.
16
+ - **Audio and transcripts:** retime word-level transcripts, mix and master audio, apply fades and ducking, and export subtitle files.
17
+ - **Rendering:** render MP4, WebM, MOV, GIF, or PNG sequences with progress, cancellation, output checks, and resumable capture checkpoints.
18
+ - **Interchange and animation:** optional OpenTimelineIO support, generic timed image layers, and keyframes. Avatar art and speech-driven character animation are owned by consuming studios such as [Explainer Studio](https://github.com/warengonzaga/explainer-studio).
19
+ - **AI-ready interfaces:** use the same versioned timeline through the CLI or JavaScript API; proposed content cuts remain reviewable before application.
20
+
21
+ ## 🖥️ Platform Support
22
+
23
+ Windows and Linux are supported. Windows has a local test and render-smoke
24
+ baseline. Linux passed all 137 tests, the full render smoke, and the production
25
+ dependency audit in a Debian Bookworm Docker container with Node.js 22, FFmpeg,
26
+ and the HyperFrames headless browser. Container rendering used software
27
+ graphics.
28
+
29
+ macOS is not officially supported yet. The repository has a manual macOS
30
+ workflow option, but native Mac validation is still required. Follow
31
+ [issue #2](https://github.com/warengonzaga/ai-video-editing-engine/issues/2) for
32
+ that work.
33
+
34
+ ## 🚀 Getting Started
35
+
36
+ ### Requirements
37
+
38
+ - Node.js 22 or newer
39
+ - FFmpeg and ffprobe
40
+ - HyperFrames browser runtime for rendering
41
+ - Python and OpenTimelineIO only for timeline interchange
42
+
43
+ ### Install and check the local toolchain
44
+
45
+ ```sh
46
+ npm install
47
+ npm run doctor
48
+ node src/cli.mjs capabilities
49
+ ```
50
+
51
+ Create a project and list the supported canvas presets:
52
+
53
+ ```sh
54
+ npm run new-video -- vertical-video 1080x1920 30 15
55
+ node src/cli.mjs presets list
56
+ ```
57
+
58
+ After the first release, install the CLI from npm with:
59
+
60
+ ```sh
61
+ npm install --global aivee
62
+ aivee doctor
63
+ ```
64
+
65
+ The CLI command is `aivee`. The release workflow publishes the unscoped npm
66
+ package `aivee` and the owner-scoped GitHub Packages package
67
+ `@warengonzaga/aivee`. Neither package has been published yet. The first release
68
+ requires a repository `NPM_TOKEN` with npm publish access and GitHub Actions
69
+ package-write permission.
70
+ See the [release guide](docs/releasing.md) for the one-time setup.
71
+
72
+ ## 📖 Usage
73
+
74
+ Start with the [Video Editing Usage Guide](docs/usage.md) for project creation,
75
+ media registration, editing operations, rendering, transcripts, and
76
+ OpenTimelineIO interchange.
77
+
78
+ Related references:
79
+
80
+ - [Aspect ratio presets](docs/aspect-ratio-presets.md)
81
+ - [Architecture and API boundaries](docs/architecture.md)
82
+
83
+ ## 🧪 Development and Validation
84
+
85
+ Run the automated checks from the repository root:
86
+
87
+ ```sh
88
+ npm test
89
+ npm run test:render-smoke
90
+ npm audit --omit=dev
91
+ ```
92
+
93
+ GitHub Actions are manual-trigger only. Use the platform selector for Linux,
94
+ Windows, macOS, or the full matrix. A native macOS run is still required before
95
+ claiming Mac support.
96
+
97
+ ## 🤝 Contributing
98
+
99
+ Use short-lived feature branches from `dev` and open pull requests targeting
100
+ `dev`. Follow the repository's Clean Commit format and keep changes focused. Do
101
+ not create a `dev`-to-`main` pull request without a release decision.
102
+
103
+ ## 🐛 Issues & Support
104
+
105
+ Report defects, request capabilities, or track platform validation through
106
+ [GitHub Issues](https://github.com/warengonzaga/ai-video-editing-engine/issues).
107
+ See [issue #2](https://github.com/warengonzaga/ai-video-editing-engine/issues/2)
108
+ for native macOS support validation.
109
+
110
+ ## 📄 License
111
+
112
+ AIVEE (AI Video Editing Engine) is licensed under the [MIT License](LICENSE). Third-party
113
+ dependencies retain their respective licenses.
114
+
115
+ ## 👥 Authors & Contributors
116
+
117
+ Created and maintained by [Waren Gonzaga](https://github.com/warengonzaga) and
118
+ contributors.
@@ -0,0 +1,401 @@
1
+ # Architecture
2
+
3
+ ## What to carry forward
4
+
5
+ Herk Edit Kit demonstrates a useful editing sequence: preserve source media,
6
+ transcribe into word-level timestamps, propose cuts, review decisions, retime
7
+ the transcript, compose visuals, then lint, preview, and inspect a draft before
8
+ rendering the final output. Its local HyperFrames and GSAP workflow also gives
9
+ this project a concrete renderer to build around.
10
+
11
+ We will implement the workflow independently. No source files, skills, card
12
+ library, brand assets, footage, or examples are copied into this repository.
13
+ If a future change ports or adapts upstream material, review its license and
14
+ third-party notices first and preserve the required attribution.
15
+
16
+ ## Improvement targets
17
+
18
+ 1. **Provider-neutral transcription:** one normalized transcript contract, with
19
+ provider-specific adapters and a local option.
20
+ 2. **Traceable edits:** every cut keeps original source times and a reason; all
21
+ derived media and transcript timings identify the EDL used to produce them.
22
+ 3. **Safer defaults:** never overwrite source media; require an explicit review
23
+ artifact before applying proposed content cuts; keep credentials and personal
24
+ media out of version control.
25
+ 4. **Smaller reusable core:** separate media operations and validation from
26
+ assistant instructions, creative styles, and project examples.
27
+ 5. **Verifiable output:** make machine checks catch structural and timing errors,
28
+ while keeping rendered audio and visual review as a human step.
29
+ 6. **Agent-readable state:** expose a deterministic read-only summary of the
30
+ project, media, tracks, and clip timing before an editing agent proposes
31
+ changes.
32
+
33
+ These targets define this repository's engine scope. Consumer products, plugins,
34
+ image editing, asset-generation marketplaces, and large bundled style libraries
35
+ belong elsewhere.
36
+
37
+ New projects include a validated empty timeline initialized from project
38
+ dimensions, frame rate, duration, source-fit policy, and registered asset
39
+ metadata. Existing projects can create the same timeline through the CLI or
40
+ `createProjectTimeline` API. The default track scaffold contains video, audio,
41
+ and overlay tracks; consumers can add or remove tracks through normal edit
42
+ operations.
43
+
44
+ ## OpenTimelineIO interchange boundary
45
+
46
+ The optional `src/interchange/otio.mjs` module invokes a small Python bridge
47
+ that uses the maintained OpenTimelineIO library and its native `otio_json`
48
+ adapter. This keeps OTIO parsing and serialization out of hand-written Node
49
+ code and keeps Python/OpenTimelineIO optional for consumers that do not use
50
+ interchange. The pinned optional dependency is in `requirements-otio.txt`.
51
+
52
+ The interchange profile maps external media references, video/audio tracks,
53
+ nested track stacks and nested compositions inside tracks (flattened into
54
+ ordered Engine tracks), clips, gaps, root-stack review markers, track enabled
55
+ state, clip enabled state, and project duration including trailing gaps. Import
56
+ expands standard `SMPTE_Dissolve` transitions into overlapping
57
+ Engine clips with video opacity fades or audio crossfades. It supports
58
+ constant-rate `LinearTimeWarp` retiming and trimmed track/stack ranges when
59
+ their boundaries preserve complete transitions. It rejects variable speed
60
+ ramps, retimed crossfades, nested or combined stack trims, custom transition
61
+ types, other overlaps, effects, nested-composition trims combined with parent
62
+ track trims, non-file remote references, and missing media instead of silently
63
+ changing their meaning. Export converts explicitly faded two-clip overlaps to
64
+ standard `SMPTE_Dissolve` transitions. Unmarked or
65
+ chained overlaps, standalone fades, clip-edge transitions, and variable speed
66
+ ramps are rejected instead of silently changing their meaning. Track-level
67
+ audio effects and volume automation are also refused on export because the profile does not
68
+ map them to standard OTIO effects. The full Engine timeline is carried in
69
+ namespaced OTIO metadata so Engine-specific overlays and settings can survive
70
+ an Engine-to-OTIO-to-Engine round trip. Other editors may discard unknown
71
+ metadata, so this is not a cross-editor preservation guarantee. OTIO references
72
+ media paths and does not package the media files.
73
+
74
+ ## Phase 1 capability audit (2026-11-05)
75
+
76
+ The table separates what the local checkout implements from what reference
77
+ project documentation says it does. Reference claims are not local verification.
78
+
79
+ | Capability | Engine local checkout | Herk Edit Kit reference | Free247Editor reference |
80
+ | --- | --- | --- | --- |
81
+ | Media inspection | Implemented with ffprobe | Uses FFmpeg-based workflow | Requires FFmpeg; setup checks tools |
82
+ | Transcript timing | Normalize/import and HyperFrames transcription wrapper | Word timing and transcript-driven workflow | Groq or faster-whisper routes advertised |
83
+ | Edit decisions | Silence-gap proposals, repeated-word candidates, validated EDL and retiming | Silence/mistake cutting and reviewed EDL workflow | Describes dead air, breaths, false starts, and retake cuts |
84
+ | Cut/render pipeline | FFmpeg cut helper plus EDL-to-timeline MP4 orchestration | Integrated agent workflow | README says it installs a pinned Herk Edit Kit under `kit/` |
85
+ | Composition/motion | HyperFrames project render wrapper; GSAP starter | HyperFrames, GSAP, reusable motion cards/templates | Attributes motion graphics to HyperFrames + GSAP through Herk |
86
+ | Captions/B-roll/audio design | EDL-to-MP4, timed text, still/video layers, gain/fades/clip-timing ducking; no automatic B-roll selection | Workflow includes captions, B-roll, music/SFX, review | README advertises word-level captions, B-roll, SFX/music, camera reframing |
87
+ | Consumer interface | Local Node ESM API and private CLI binary | Agent skills orchestrate kit tools | Agent skill orchestrates bundled/pinned Herk kit |
88
+ | Avatar speech | Consumer-owned; studios can generate avatar layers as timed image clips on generic video tracks | Talking Explainer is a separate reference, not a Herk core requirement | Not a core editor capability |
89
+ | Platform evidence | Windows checkout only; do not infer universal support | README documents Windows/macOS/Linux requirements; not independently tested here | README reports Windows 11 test; macOS/Linux are stated but untested |
90
+
91
+ **Audit conclusion:** The Engine now provides a shared timeline schema,
92
+ validation, CLI/API, media registration, EDL conversion, transcript-to-caption
93
+ assembly, a HyperFrames renderer, and basic audio mix automation. Visual
94
+ effects breadth, cross-platform coverage, and full audiovisual quality review
95
+ remain open. Skills, personal style/rules,
96
+ explainer storyboards, and owned avatar art remain consumer concerns. Studio
97
+ can call reusable Engine operations while retaining its chat approvals and
98
+ creative direction.
99
+
100
+ **License/provenance note:** the Herk repository identifies its project
101
+ license as MIT and lists separate third-party notices; each included
102
+ dependency/asset retains its own terms. Free247Editor identifies itself as MIT
103
+ and describes downloading a pinned Herk kit rather than copying it into its
104
+ own engine source. This Engine will implement its own contracts and code; do
105
+ not copy either project's skills, assets, templates, or code during the audit
106
+ or implementation without a provenance and license review.
107
+
108
+ ## Proposed generic project and timeline contract
109
+
110
+ The first contract is intentionally small and renderer-neutral. JSON examples
111
+ will be checked into docs only after the validator and CLI/API land.
112
+
113
+ - `schemaVersion`: integer for explicit compatibility and migrations.
114
+ - `project`: stable id, width, height, frame rate, duration in seconds, and an
115
+ optional validated background color. New projects default to `#10141c`,
116
+ while legacy timelines without a color retain the renderer's black
117
+ background; `null` means transparent for overlay workflows.
118
+ - `assets`: stable ids and project-relative media paths; probe metadata is
119
+ derived and records streams, duration, dimensions, average and nominal frame
120
+ rates, per-stream timing, and codecs. A/V records preserve container aliases,
121
+ metadata for every video stream, and metadata for every audio stream. When
122
+ the container duration is absent, probing falls back to the longest valid
123
+ stream duration, deriving seconds from `duration_ts × time_base` when needed;
124
+ media with no usable duration remains invalid. Asset verification compares SHA-256
125
+ identities, while relinking replaces media under the same stable id/path and
126
+ refreshes its probe record without modifying the replacement source.
127
+ Optional video-only H.264 proxies retain their own dimensions, stream timing,
128
+ settings, and source/output SHA-256 identities. Draft renders can explicitly
129
+ select those variants; audio and default final renders continue using source
130
+ assets.
131
+ - `tracks`: ordered tracks with stable ids and kinds (`video`, `audio`,
132
+ `overlay`), plus explicit z-order, optional video/overlay visibility, audio
133
+ mute/solo state, and a lock that prevents edits to track contents/settings.
134
+ - `clips`: stable id, asset reference, source in/out, timeline start, and
135
+ optional placement/fit, constant `playbackRate`, source-relative `speedRamp`,
136
+ or reverse playback on video and audio clips,
137
+ volume, and composition reference. A speed ramp has linear or named eased
138
+ `playbackRate` interpolation between `{ sourceOffset, playbackRate, ease? }`
139
+ points covering the full clip source span; the optional ease controls the
140
+ outgoing segment. Its timeline duration is the integral of source time
141
+ divided by playback rate. Eased curves are sampled into bounded linear
142
+ segments for rendering. Constant-rate clips use source span = clip duration
143
+ × playback rate. All times are seconds; renderers quantize to project frames
144
+ in one documented step.
145
+ - `keyframes`: optional clip-local visual states on video and overlay clips.
146
+ Normalized x/y/width/height, opacity, rotation, brightness, contrast,
147
+ object-position, 1×–4× video zoom, saturation, blur, and hue interpolate
148
+ through GSAP. Zoom scales media inside its fixed, clipped viewport;
149
+ allowlisted easing controls each segment. Per-property easing ranges preserve
150
+ the curve when a clip is trimmed or split inside a tween. The first key must
151
+ be at time zero, geometry must remain within the canvas, and opacity keyframes
152
+ cannot overlap the separate clip-fade controls.
153
+ - `mask`: optional clip-path mask on video and overlay clips. Version 1 supports
154
+ normalized `circle`, `ellipse`, and `rect` shapes; `maskProgress` keyframes
155
+ animate the configured shape between closed and full bounds. Shape fields are
156
+ validated and compiled to CSS; raw CSS strings are not accepted.
157
+ - `blendMode`: optional allowlisted CSS compositing mode on video and overlay
158
+ clips. Modes blend the clip against lower visual layers within an isolated
159
+ project canvas; arbitrary CSS and audio-track use are rejected.
160
+ - `transitionIn` and `transitionOut`: optional version-1 `fade` or directional
161
+ `wipe` transitions on video and overlay clips. Their durations cannot overlap;
162
+ wipes cannot share a static mask, and fades cannot share opacity animation.
163
+ - `audioMix`: optional clip-timing ducking and signal-aware sidechain rules
164
+ between audio tracks. Timing ducking combines with track/clip gain and fades
165
+ in one volume envelope. Signal-aware rules build temporary detector stems
166
+ from source track clips and create temporary compressed target audio clips;
167
+ all generated media is removed after render.
168
+ - `audioEffects`: optional ordered per-clip equalizer, high-pass, low-pass,
169
+ compressor, stereo-balance, and noise-reduction effects on audio tracks.
170
+ Audio tracks can also define an ordered track-level chain. Clip effects run
171
+ first; track clips are then mixed to a temporary stem and the track effects
172
+ run across the full stem. Required FFmpeg filters for clip/track effects,
173
+ sidechain, and mastering are included in capability discovery and checked
174
+ before rendering.
175
+ - `volumeKeyframes`: optional, clip-relative audio gain automation with linear
176
+ interpolation and a held final value. Time 0 is required; trim, split, and
177
+ synchronized timeline operations rebase and preserve the curve.
178
+ - Audio tracks can also define `volumeKeyframes` in project time. The curve is
179
+ multiplied with static track/clip gain and clip automation; gap insertion and
180
+ ripple deletion preserve its shape relative to edited timeline content.
181
+ - `audioMastering`: optional final-mix integrated loudness, true-peak ceiling,
182
+ and loudness-range targets. FFmpeg `loudnorm` uses measured two-pass mode
183
+ when possible, dynamic single-pass mode for unmeasurable non-silent input,
184
+ and preserves valid silent tracks without a loudness filter. Optional
185
+ parametric equalizer bands run before analysis/normalization, and an optional
186
+ sample-peak limiter runs at the end of the final mix chain. Consumers can
187
+ request mono or stereo output and the encoded channel count is verified.
188
+ - `overlays`: timed composition references with validated duration and layer
189
+ order. Text/caption semantics live in composition data, not in a Studio-only
190
+ field.
191
+ - `markers`: optional project-time review/chapter points with stable ids,
192
+ labels, and optional display colors. Markers are non-rendering and are retimed
193
+ by synchronized gap insertion and ripple deletion. OpenTimelineIO interchange
194
+ maps them to root-stack markers; other formats may drop them.
195
+ - `lineage`: optional transcript/EDL references and source identities so
196
+ transcript edits can compile into the same timeline.
197
+
198
+ Validation rejects missing assets, paths escaping the project when assets are
199
+ project-managed, non-finite or negative times, invalid source ranges, timeline
200
+ clips beyond project duration, source ranges beyond known registered media
201
+ duration, mismatches between source span and constant-rate or integrated
202
+ speed-ramp duration,
203
+ unknown track kinds, and duplicate ids. A non-default media playback rate is
204
+ rendered through a temporary FFmpeg proxy that is removed after render; video
205
+ proxy encoding and pitch-preserving audio `atempo` are explicit costs. Eased
206
+ audio speed ramps use FFmpeg's optional `rubberband` filter to preserve pitch;
207
+ rendering such a ramp fails with an actionable error when the filter is absent.
208
+ Reverse video and audio playback use an FFmpeg temporary proxy for the selected
209
+ source range. They can combine with a constant playback rate, but not a speed
210
+ ramp; image assets cannot be reversed. FFmpeg's reverse filters buffer the
211
+ selected clip, so memory use grows with clip duration and resolution. Trim and
212
+ split operations preserve the reverse source direction. Detaching audio from a
213
+ reversed video clip copies the reverse setting to the audio clip. OTIO export
214
+ rejects reverse playback rather than silently dropping it.
215
+ Capability discovery reports the `reverse` and `areverse` filters, and render
216
+ preflight checks the specific filter required by a reversed track before the
217
+ renderer creates temporary media.
218
+ If a linked video's audio stream has a material start or
219
+ duration offset, the renderer creates a temporary FFmpeg-normalized audio file
220
+ for the composition and removes it afterward; source media stays unchanged. Overlap
221
+ is allowed within a track only when explicitly represented; tracks define
222
+ compositing order. Existing schemaVersion 1 transcript and EDL contracts stay
223
+ supported and can be compiled into the timeline model.
224
+
225
+ Consumers use the same operations through CLI and JavaScript exports. Timeline
226
+ operations include validation, EDL assembly, track/clip add and remove, track
227
+ locks, trim, slip, split, move, synchronized gap insertion and clip/range ripple
228
+ deletion, crossfade, registered-asset replacement, reviewable speed changes,
229
+ versioned fade/wipe transitions, reviewable visual framing and keyframe changes,
230
+ track visibility/mute/solo/lock/volume changes, caption generation, and rendering.
231
+ Reviewable audio clip gain/fade/stream selection, video stream selection, and
232
+ overlay content/style changes join this operation surface. Video stream
233
+ selection uses a zero-based video ordinal and compiles to a temporary clip
234
+ proxy, preserving the original asset. `set-audio-mix` and
235
+ `set-audio-mastering` expose validated reviewable changes for timeline-level
236
+ mix rules and final-mix settings. `set-project-state` can change timeline
237
+ duration and project-wide source fitting while keeping width, height, and frame
238
+ rate consistent with Engine project metadata.
239
+ Consumers can wrap immutable edits in bounded in-memory undo/redo history that returns
240
+ isolated snapshots and drops redo entries after a new branch of edits. CLI agents
241
+ can also use persistent timeline-file history across process invocations. It
242
+ stores bounded snapshots in an atomic sidecar journal, recovers an interrupted
243
+ timeline write, and refuses to overwrite a timeline modified outside the
244
+ history session until the user resets that session. A lock serializes history
245
+ updates and stale locks from exited processes are cleared. Crossfades configure GSAP opacity or
246
+ audio-volume fades on a validated, caller-positioned clip overlap. Video and
247
+ overlay clips can animate normalized placement, size, opacity, rotation, and
248
+ CSS visual filters with clip-local keyframes. Rendering compiles video clips, audio
249
+ tracks, text overlays, and timed compositions into a HyperFrames project before
250
+ draft/standard/high rendering. MP4, WebM, MOV, and GIF file exports are accepted
251
+ by extension; PNG sequences are requested explicitly. GIF and PNG sequences are
252
+ rejected for timelines with audio. PNG sequences are written as validated
253
+ full-resolution RGBA frame directories.
254
+
255
+ Injected transcription and media-analysis providers are generic JavaScript
256
+ adapters. Transcript output is normalized to the source duration. Media
257
+ findings are returned as time-validated suggestions for review and are not
258
+ automatically converted into timeline edits. Provider credentials remain in
259
+ consumer code and are not stored in project metadata.
260
+ Rational project frame rates are passed to
261
+ the renderer and checked in the output. MP4, MOV, and WebM renders can override
262
+ the quality preset with validated codec, CRF, or target video bitrate options.
263
+ Explicit codec overrides are applied by a final FFmpeg encode and the output
264
+ codec is checked before publishing the file. Render options can bound capture
265
+ workers from 1 to 64, select HyperFrames low-memory mode, and set a bounded
266
+ render timeout that also applies to the explicit-codec FFmpeg stage. The async
267
+ render APIs accept an `AbortSignal`; the CLI maps Ctrl+C to process-tree
268
+ cancellation. `renderTimelineAsync` applies one deadline to FFmpeg/ffprobe
269
+ preprocessing, HyperFrames, final encoding, output probing, and audio mastering.
270
+ Timeout and cancellation remove Engine and HyperFrames temporary outputs and
271
+ roll back timeline metadata. The async timeline renderer uses cancellable
272
+ FFmpeg/ffprobe child processes for media preparation and mastering, so timeout
273
+ and external abort signals terminate the active process tree.
274
+ Color profiles can request Rec.709, Rec.2020 SDR, or HDR10. HDR10 export is an
275
+ SDR-to-HDR conversion for MP4/MOV: it maps the Rec.709 render to 10-bit
276
+ BT.2020/PQ at a 203-nit reference white and requires FFmpeg `zscale` plus
277
+ `libx265` Main 10. This changes the transfer function and encoding precision;
278
+ it cannot recreate highlight information absent from the SDR render.
279
+ The async renderer can preserve a completed HyperFrames capture for retrying
280
+ the final stage. Checkpoints are content-fingerprinted, SHA-256 verified,
281
+ bounded by per-capture size and expiration, and removed after success. An
282
+ interrupted browser capture starts over; this is stage-level recovery, not
283
+ frame-level continuation. PNG sequences and synchronous rendering are excluded.
284
+ The core validator does not require FFmpeg or a
285
+ browser. Schema changes require a version increment or an explicit
286
+ backward-compatible migration.
287
+
288
+ ## Current implementation
289
+
290
+ ```text
291
+ src/
292
+ transcript/ normalized word timing and validation
293
+ edits/ silence proposals, mistake candidates, EDL retiming
294
+ media/ FFmpeg/ffprobe process helpers
295
+ project/ isolated project generation
296
+ render/ HyperFrames lint, preview, transcription, and render
297
+ cli.mjs command-line interface
298
+ video-projects/ generated per-video workspaces
299
+ ```
300
+
301
+ The engine creates projects at caller-selected dimensions, registers and
302
+ fingerprints media, imports or generates transcripts, proposes editable cuts,
303
+ applies approved EDLs, retimes transcripts, generates captions, and
304
+ lints/previews/renders HyperFrames compositions. The generated starter
305
+ composition is intentionally small; editing intelligence and user approval
306
+ stay outside the renderer.
307
+
308
+ Named social-video canvas presets are 9:16, 4:5, 1:1, and 16:9, with arbitrary
309
+ dimensions remaining supported. The registry is exposed through the JavaScript
310
+ API and CLI; project metadata records the selected preset id and resolved canvas
311
+ dimensions. Their platform mapping and default pixel dimensions are documented
312
+ in [`aspect-ratio-presets.md`](aspect-ratio-presets.md). A preset selects canvas
313
+ dimensions only; source fit/crop behavior stays an explicit rendering choice.
314
+ Source-fit modes are `contain`, `cover` (center crop), and `fill` (stretch),
315
+ with `contain` as the non-cropping default. Project policy may be overridden by
316
+ a video clip's `fit` value.
317
+
318
+ ## Version 1 data contracts
319
+
320
+ Transcript JSON uses `schemaVersion: 1`, `durationSeconds`, and `words` with
321
+ `text`, `start`, and `end` fields. Times are seconds on the original media
322
+ timeline. Words must be ordered by start time and stay within the media duration.
323
+ Provider word intervals may overlap; the engine preserves those source timings
324
+ and their overlap when retiming. It does not infer which provider timestamp is
325
+ more accurate or force neighboring words apart.
326
+ Registered assets retain FFprobe container and per-stream start times and time
327
+ bases for diagnostics and downstream timing decisions. Use `secondsToFrame`,
328
+ `frameToSeconds`, or `snapTimeToFrame` for explicit output-frame rounding.
329
+ Variable-frame-rate source timestamps remain seconds-based and must not be
330
+ snapped to the project frame rate; broader VFR seek/render behavior remains a
331
+ validation item. When linked video/audio start times differ, or a standalone
332
+ audio stream starts after zero, the renderer creates and removes a temporary
333
+ timestamp-aligned audio proxy. Original assets remain untouched.
334
+
335
+ An approved EDL uses `schemaVersion: 1`, the same `sourceDurationSeconds`, and a
336
+ `cuts` array. Each cut has `sourceStart`, `sourceEnd`, and a non-empty `reason`.
337
+ Cuts must be ordered, non-overlapping, positive-duration intervals within the
338
+ source. A cut may fully contain a transcript word, removing it. It may not split
339
+ a word; cuts that touch only the beginning or end of a word are also rejected.
340
+ With overlapping word intervals, a cut that fully contains one word but only
341
+ partly intersects another is rejected because it would clip the latter word.
342
+ These checks prevent the retimed transcript from claiming that clipped speech
343
+ is intact.
344
+
345
+ The retimer verifies both contracts and matching source durations before writing
346
+ anything. It drops fully cut words and shifts retained timestamps by the removed
347
+ duration preceding each word. Source transcript and EDL files are never modified.
348
+
349
+ ## Tool boundary
350
+
351
+ - Node.js owns the CLI, validation, orchestration, and file handling.
352
+ - FFmpeg and ffprobe handle media probing, cuts, and encoding.
353
+ - HyperFrames renders HTML compositions; its supported browser runtime is part
354
+ of the render environment.
355
+ - GSAP adds timeline animation to compositions; static compositions do not need it.
356
+ - Playwright is for browser automation and smoke checks, not required by the
357
+ transcript/EDL core or every render.
358
+ - Transcription providers are optional adapters; credentials never enter project
359
+ metadata, generated HTML, logs, or committed files.
360
+
361
+ ## Runtime requirements by capability
362
+
363
+ | Capability | Requirements |
364
+ | --- | --- |
365
+ | Transcript and EDL validation/retiming | Node.js 22+; current code uses built-ins only |
366
+ | Probe, cut, and encode media | FFmpeg and ffprobe |
367
+ | Render HTML compositions | HyperFrames, FFmpeg, and the browser runtime supported by the pinned HyperFrames version |
368
+ | Animate compositions | GSAP, when the composition uses animation |
369
+ | Automated browser smoke checks | Playwright, when those checks are enabled |
370
+ | Generate a transcript | Optional cloud provider key, or a separately installed local transcription runtime and model |
371
+ | Reproducible container rendering | Docker, if selected for the render workflow |
372
+
373
+ The engine checks the tools needed by the requested operation. For example,
374
+ transcript retiming does not require FFmpeg, a browser, or API keys. `npm run
375
+ doctor` checks Node.js, FFmpeg, ffprobe, HyperFrames, and GSAP, then runs
376
+ HyperFrames' environment doctor for browser/render setup and optional local AI
377
+ runtimes. Consumers can query the versioned `getEngineCapabilities()` API or
378
+ `aivee capabilities` CLI command to discover required tools, common
379
+ FFmpeg demuxers and decoders for common input media, encoders, output formats,
380
+ detected codec/per-format color-profile support, predefined canvas presets,
381
+ canvas background support and its accepted values, and optional render filters
382
+ without relying on doctor output parsing. Input
383
+ `available` reports whether FFmpeg and ffprobe are installed; `inventoryChecked`
384
+ reports whether the decoder and demuxer lists could be read. Container and
385
+ decoder fields are independent checks; having both does not prove that every
386
+ media stream combination will decode. Format availability reflects the relevant local FFmpeg,
387
+ ffprobe, HyperFrames, encoder, and filter inventory. The browser runtime is
388
+ reported as unchecked and must be verified with `npm run doctor`. The report
389
+ marks a format available when its render runtime and at least one compatible
390
+ encoder are present; `defaultCodecAvailable` indicates whether the default
391
+ encoder is installed. It does not guarantee that every input codec or
392
+ media-specific render operation is supported.
393
+ For operation-specific input checks, the JavaScript API exposes
394
+ `listFfmpegDecoders()`, `listFfmpegDemuxers()`, their boolean `has` helpers, and
395
+ their throwing `assert` helpers. `inspectMediaCompatibility(path)` probes a
396
+ source and checks its container plus every reported audio/video stream against
397
+ the local FFmpeg inventory before ingest; it does not decode the full file.
398
+
399
+ Pinned package versions are an initial baseline from the installed Herk Edit Kit
400
+ setup, not a promise of compatibility with future releases. Recheck versions and
401
+ licenses before upgrades or distributing a build.
@@ -0,0 +1,72 @@
1
+ # Social Video Aspect-Ratio Presets
2
+
3
+ This document defines the platform-informed aspect-ratio presets exposed by
4
+ AIVEE. The engine remains general-purpose: presets are
5
+ convenience defaults, and callers can continue to use arbitrary dimensions.
6
+
7
+ ## Initial preset baseline
8
+
9
+ | Preset | Ratio | Default dimensions | Common destinations |
10
+ | --- | --- | --- | --- |
11
+ | `vertical` | 9:16 | 1080 × 1920 | TikTok, Instagram/Facebook Reels and Stories, YouTube Shorts |
12
+ | `portrait-feed` | 4:5 | 1080 × 1350 | Instagram feed, LinkedIn vertical feed video |
13
+ | `square` | 1:1 | 1080 × 1080 | TikTok, Instagram/LinkedIn feed placements |
14
+ | `landscape` | 16:9 | 1920 × 1080 | YouTube standard video, LinkedIn landscape feed video |
15
+
16
+ The ratio is the platform-oriented choice; the pixel dimensions are practical
17
+ HD defaults, not a promise that every platform requires those exact pixels.
18
+ Where a platform publishes a lower or different maximum, export dimensions can
19
+ be lowered while preserving the selected ratio. Keep dimensions configurable.
20
+
21
+ The JavaScript API exports `CANVAS_PRESETS`, `listCanvasPresets()`, and
22
+ `resolveCanvasPreset(id)` from the package entry point. The CLI exposes the same
23
+ catalog with `node src/cli.mjs presets list`; create projects with
24
+ `node src/cli.mjs project create <slug> --preset <id> [fps] [durationSeconds]`.
25
+ The selected identifier is stored as `canvasPreset` beside the resolved
26
+ `width` and `height` in `meta.json`. Existing custom-dimension project creation
27
+ remains supported.
28
+
29
+ ## Selection guidance
30
+
31
+ - Recommend `vertical` for full-screen short-form video. YouTube identifies
32
+ Shorts as vertical; TikTok recommends 9:16 and lists 16:9 and 1:1 as other
33
+ supported ad shapes.
34
+ - Recommend `portrait-feed` for tall feed placements where a full-screen 9:16
35
+ canvas is not wanted. LinkedIn explicitly recommends 4:5 vertical video and
36
+ allows up to 1080 × 1350 pixels.
37
+ - Recommend `square` for flexible feed distribution. TikTok and LinkedIn list
38
+ 1:1 as a supported video shape.
39
+ - Recommend `landscape` for conventional long-form video and landscape feeds.
40
+ YouTube's standard desktop player ratio is 16:9, with 1920 × 1080 listed as
41
+ its 1080p resolution.
42
+ - Keep 4:3 and other ratios available through custom width/height settings;
43
+ this initial preset list does not restrict the renderer.
44
+
45
+ ## Rendering behavior
46
+
47
+ Choosing a preset sets the output canvas only. Source-to-canvas behavior is a
48
+ separate policy with these options:
49
+
50
+ - `contain` scales the full source frame inside the clip bounds, leaving empty
51
+ canvas around it when the aspect ratios differ. This is the default because
52
+ it preserves the full source image.
53
+ - `cover` fills the clip bounds and center-crops any overflow.
54
+ - `fill` stretches the source to the clip bounds and can distort its shape.
55
+
56
+ Set `sourceFit` on project metadata or `timeline.project` for the project-wide
57
+ default; a video clip's `fit` field overrides that policy. The CLI accepts
58
+ `--fit contain|cover|fill` during project creation. The renderer validates these
59
+ values rather than silently substituting a crop mode. Platform safe-area guides
60
+ are separate and are not implied by a ratio preset.
61
+
62
+ ## Sources
63
+
64
+ - [YouTube: Video resolution and aspect ratios](https://support.google.com/youtube/answer/6375112)
65
+ - [YouTube: Shorts upload tips](https://support.google.com/youtube/answer/12921536)
66
+ - [TikTok Business Help: Reservation in-feed ad specifications](https://ads.tiktok.com/resources/help/article/tiktok-reservation-in-feed-ads-reach-frequency)
67
+ - [LinkedIn Help: Video ads advertising specifications](https://www.linkedin.com/help/linkedin/answer/a424737)
68
+ - [Vimeo Help: Instagram video aspect-ratio options](https://help.vimeo.com/hc/en-us/articles/12426476775953-How-to-select-a-ratio-in-the-video-editor-for-Instagram-posts)
69
+
70
+ Platform specifications can change and may differ between organic uploads,
71
+ advertising placements, and publishing tools. Recheck the destination's current
72
+ requirements before adding validation that rejects an export.
@@ -0,0 +1,19 @@
1
+ # Releasing AIVEE
2
+
3
+ The `Build Flow` workflow validates pull requests targeting `dev` or `main`.
4
+ Merging the release pull request to `main` publishes `aivee` to npm and
5
+ `@warengonzaga/aivee` to GitHub Packages, then creates a GitHub Release after
6
+ both registries report success. Development, pull-request, and manual runs do
7
+ not publish packages.
8
+
9
+ ## One-time setup
10
+
11
+ 1. Add an npm automation token with publish access as the repository secret
12
+ `NPM_TOKEN`. The workflow uses it to create the initial npm package and to
13
+ publish later versions.
14
+ 2. Enable GitHub Actions package write access for the repository. The workflow
15
+ uses `GITHUB_TOKEN` to publish the GitHub Packages copy.
16
+
17
+ After setup, release promotion from `dev` to `main` runs the version, package,
18
+ and GitHub Release flow. Check both registries after the first automated release;
19
+ publishing to them is not atomic.