@topy-ai/maggie 0.8.10 → 0.8.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README-zh-TW.md +1 -1
- package/README.md +1 -1
- package/bundled-references/ffmpeg-recipes.md +5 -1
- package/bundled-references/video-timeline-model.md +10 -8
- package/bundled-skills/maggie-video/SKILL.md +7 -4
- package/bundled-tools/clis/maggie_video.py +176 -44
- package/package.json +1 -1
- package/references/ffmpeg-recipes.md +5 -1
- package/references/video-timeline-model.md +10 -8
package/README-zh-TW.md
CHANGED
|
@@ -143,7 +143,7 @@ structure drift 或重複 treatment product 時 fail closed。
|
|
|
143
143
|
deployment、memory、feedback 和 MaggieDash。內容先 draft/review，外部寫入、
|
|
144
144
|
publish 與 production deployment 需要明確確認。
|
|
145
145
|
|
|
146
|
-
目前 npm 版本為 `0.8.
|
|
146
|
+
目前 npm 版本為 `0.8.11`,收尾 N 類影片功能:`maggie_video.py compile` 再補上視頻 clip ken-burns、sidechain ducking、以及非透明度 overlay 效果(move/slide、圖片 zoom),並新增 `autocut` 子命令(silencedetect 去靜音)。先前 0.8.10 新增圖片 ken-burns + 文字透明度關鍵幀與 `captions`(whisper)子命令。先前 0.8.9 新增 `compile`(宣告式 timeline JSON→單一 ffmpeg 命令)。先前 0.8.8 新增 `maggie-video`(製作與處理行銷影片:剪輯、字幕、裁切成各渠道比例、浮水印、轉碼、響度正規化、clip 組裝、程式化動畫),附一個實測可用的 ffmpeg 配方 CLI(`maggie_video.py`),從參考庫的影片專案(moviepy/editly/ffmpeg 命令層、Revideo/Motion Canvas/CCapture 程式化渲染、MLT/OpenShot/Olive 的 timeline 資料形狀)提煉而來;另加 `ffmpeg-recipes`、`programmatic-video`、`video-timeline-model` 三個 reference。先前 0.8.7 把 redesign 稽核併入 `maggie-design`(改編自 MIT 的 `elayadesign/redesign-skill`):掃描 + 診斷 AI 通病/缺失狀態/雜亂程式碼,再經 in-place/verify 流程就地修正,因此「稽核/打磨/升級既有站點」會路由到 maggie-design。先前 0.8.6 補齊兩個設計技能:加入完整的生產 reference,並把設計來源明確為 Figma 非必需(實時 URL、既有程式碼、Figma、或純截圖)。先前 0.8.5 採納兩個設計技能(改編自 MIT 的 `elayadesign/ai-design-skills`):maggie-landing-page(轉化策略 + 視覺系統紀律,具體數值為可被專案 DESIGN.md 覆蓋的預設值;由 maggie-design 生產與驗證)與 maggie-design-motion(靜態設計→像素級動畫 HTML 複刻→匯出 MP4 + GIF)。先前 0.8.4 將技能庫擴展成完整的「一人用 AI 跑一家公司」C-suite,新增八條職能線:maggie-sales(CRM 中立的銷售管線)、maggie-finance(只解釋的 CFO 指標,含確定性 maggie_finance.py 計算器)、maggie-support(面向客戶的客戶成功)、maggie-product(路線圖 + 預註冊實驗)、maggie-fundraising(pitch deck、data room、投資人更新)、maggie-legal(ToS/隱私/DPA/SOC2 readiness)、maggie-monetization(定價/paywall/dunning,金錢操作需人工批准)、maggie-orchestrator(CEO 編排層:將目標拆成帶審批門的 mission 分派到各職能線,含 maggie_orchestrator.py 規劃器)。先前 0.8.3 將 cmo.so/www 開發經驗沉澱進技能庫:新增兩個技能 maggie-skill-authoring(依 catalog 與 MCP 工具白名單撰寫/校驗社區營銷 playbook 技能)與 maggie-feedback-triage(反饋的團隊接收端:決策/定級/分派、PRD + issue 流程、24/7 隊列守護);新增多個 reference(OAuth PKCE 登入、Expo 約定與 Router 骨架、MCP evals 測試框架、conversation cards、WordPress→Firebase 認證遷移手冊);並增補 maggie-assistant-tools(tool≤user 每輪工具過濾、agent loop 安全不變量、preview-only 橋接代理、MCP resources/prompts、注解加固、CLI 與 ChatGPT 插件打包)、maggie-auth-reference(自清理登入測試套件、自簽單次令牌)、maggie-mobile-app(OTA 開機拉取、live==built 部署守衛)、maggie-deployment(pm2 單包發布、live-hash 校驗)與 maggie-ops(template-as-code 郵件同步)。先前 0.8.2 新增 maggie-spirit(為 AI 品牌生成平面像素 spirit/sprite 系統)與 goal-led AI agent 設計參考;0.8.0 為安全優化 roadmap(scan、image-patch、標頭評分、SSVC、GDPR 日誌規則、新 MCP 工具)。先前 0.8.0 為安全優化 roadmap(scan、image-patch、標頭評分、SSVC、GDPR 日誌規則、新 MCP 工具),含 fix-plan 修補風險分級 T1–T4(判斷補丁是否安全套用)與 host-impact 補丁影響面量測(哪些服務需重啟、是否需重開機)、public-check、Maggie Security、batch 31 排程 agent routines、報告收件匣與無人值守 agent 限制、PHP runtime 盤點與主機鎖、pull 網站改寫、query 點子與社群佇列檢查，batch 30 MaggieDash 先裝核心、release privacy gate、feedback 備援、社群連結與圖片檢查、大量頁面產生與 app 檢查，batch 29 預約規則、部署、app QA、助理與社群功能，以及 batch 28 產品 app、assistant tools、mobile、product release、skills catalog 與 bridge host 功能、batch 26、27 feedback 修正與 idempotent `maggie update`與 Booking email-template 合約/CLI validator 同步。本版完成 2026-09-24 booking feedback batch：Calendar grid/free-slot entry 與 host modal focus-restoration evidence。本版新增 Starter marketplace provisioning plan；本版完成 feedback batches 22、23：加入 declarative browser interaction evidence、design markup/style/content-key gate、compiler source anchoring、entity double-escape protection、`update --help` read-only、noindex hypothetical indexability、可過期 baseline acknowledgement、DOM-only link extraction regression，以及 MaggieDash 0.2.10 的 content override、短期 browser session、CDN-safe public-session cache contracts。authentication 和 isolated dev server 仍由 host 負責。`maggie-booking` workflow、傳統 email/password、Stripe、Resend、public booking flow 及既有安全邊界維持不變；provider-bound video generation 與 transactional delivery 仍記錄在 issue #39，前一批延後功能仍在 issue #38，不會直接成為 active memory。0.7.33 記錄並審查一筆更正回饋；確認 0.7.32 已正確處理 batch index，沒有修改已驗證的 shared behavior。0.7.32 加入 feedback 一基索引支援、MaggieDash panel 的 source/freshness/error evidence、Consent Mode 與實際 write/no-op reconciliation、SEO baseline recapture、sitemap origin rebasing、content-diff scope、migration ledger reconciliation、deployment credential preflight、release provenance/runtime preflight、icon release gate、opt-in blog auto-publish gate，以及 least-privilege VPS deployer。也修復 top-level `maggie feedback` dispatcher parity。0.7.31 加入 update 差異摘要、atomic installer copy、subset install manifest merge，以及 inventory drift 時 doctor non-zero。0.7.30 補上 feedback tracking issue closeout gate 與 batch 14/15 audit ledger。0.7.29 修正 npm package landing page 使用英文 README；0.7.28 加入 versioned Gemini model policy、明確 fallback 與 provenance、bundled-first CLI
|
|
147
147
|
dispatch、`maggie --version`、feedback batch review 聚合與重複偵測、marketplace enrichment evidence、booking
|
|
148
148
|
worker/resolver evidence，以及 read-only migration preflight。它也包含 host-owned mobile app surface contract、signed-in
|
|
149
149
|
camera-state QA、直接 Astro route resolution、correlated feedback batch，以及
|
package/README.md
CHANGED
|
@@ -89,7 +89,7 @@ Maggie keeps the existing project foundation and asks for decisions before
|
|
|
89
89
|
shared routes, analytics, or publishing boundaries change. The current
|
|
90
90
|
package ships 37 installable skills and a local-first MaggieDash foundation.
|
|
91
91
|
|
|
92
|
-
This package is `0.8.
|
|
92
|
+
This package is `0.8.11` — `maggie_video.py compile` now also does video-clip ken-burns, sidechain ducking, and non-opacity overlay effects (move/slide, image zoom), and a new `autocut` subcommand removes silent gaps. `0.8.10` added image ken-burns + text opacity keyframes and the `captions` (whisper) subcommand. `0.8.9` added `compile` (declarative timeline JSON -> one ffmpeg command). `0.8.8` added `maggie-video` (produce and process marketing video: cut, caption, resize to channel aspects, watermark, transcode, normalize loudness, assemble clips, render code-drawn animation) with a tested ffmpeg-recipe CLI, distilled from the reference library's video projects; plus references `ffmpeg-recipes`, `programmatic-video`, `video-timeline-model`. `0.8.7` folded a redesign audit into `maggie-design` (from the MIT `elayadesign/redesign-skill`): scan + diagnose generic AI-design patterns/weak states/sloppy code, then fix in place via the in-place/verify flow, so "audit/polish/upgrade an existing site" routes to maggie-design. `0.8.6` completed the two adopted design skills with full production references and made the design source Figma-optional (live URL, existing code, Figma, or a screenshot). `0.8.5` adopted the two design skills (adapted, MIT, from `elayadesign/ai-design-skills`): `maggie-landing-page` (conversion strategy + a visual-system discipline whose values defer to the project's DESIGN.md) and `maggie-design-motion` (static design → pixel-exact animated HTML replica → MP4 + GIF export). `0.8.4` expanded the library into a full "one person runs a company with AI" C-suite with eight new function lanes: `maggie-sales` (CRM-neutral pipeline), `maggie-finance` (explain-only CFO metrics + a deterministic `maggie_finance.py` calculator), `maggie-support` (customer success), `maggie-product` (roadmap + pre-registered experiments), `maggie-fundraising` (deck, data room, investor updates), `maggie-legal` (ToS/privacy/DPA/SOC2 readiness), `maggie-monetization` (pricing/paywalls/dunning; money actions human-approved), and `maggie-orchestrator` (the CEO layer that decomposes a goal into gated missions across the lanes, with a `maggie_orchestrator.py` planner). `0.8.3` folded cmo.so/www development learnings in (maggie-skill-authoring, maggie-feedback-triage, and mobile/MCP/auth references); `0.8.2` added `maggie-spirit`; `0.8.0` was the security optimization roadmap.
|
|
93
93
|
|
|
94
94
|
The 0.7.59 release made `maggie security host-audit` (the deep
|
|
95
95
|
lynis audit over SSH) robust on hosts where a storage test can stall: lynis runs
|
|
@@ -103,9 +103,13 @@ ffmpeg -y -i IN -vf "thumbnail,scale=1280:-1" -frames:v 1 OUT.jpg # most repres
|
|
|
103
103
|
|
|
104
104
|
```bash
|
|
105
105
|
ffmpeg -i IN -af silencedetect=noise=-30dB:d=0.3 -f null - # detect ranges
|
|
106
|
-
# then build a
|
|
106
|
+
# then build a trim+concat of the kept ranges (pad each ~0.1s)
|
|
107
107
|
```
|
|
108
108
|
|
|
109
|
+
`maggie_video.py autocut --input talk.mp4 --output tight.mp4 --run` does both:
|
|
110
|
+
detects the silence with `silencedetect`, computes the kept ranges padded by
|
|
111
|
+
`--margin`, and renders the trimmed video+audio.
|
|
112
|
+
|
|
109
113
|
## Transitions (between two clips)
|
|
110
114
|
|
|
111
115
|
`xfade=transition=fade:duration=0.5:offset=5.0` for video and `acrossfade=d=0.5`
|
|
@@ -97,16 +97,18 @@ shape renders through ffmpeg without any GPL engine.
|
|
|
97
97
|
|
|
98
98
|
`maggie_video.py compile --spec timeline.json` compiles this model into one ffmpeg
|
|
99
99
|
`filter_complex` command (printed; run with `--run`). Supported today: a main video
|
|
100
|
-
track of clips (video/image) fit `cover`/`contain
|
|
101
|
-
|
|
100
|
+
track of clips (video/image) fit `cover`/`contain`, hard-cut `concat` or sequential
|
|
101
|
+
`crossfade` (from `transitions`), **ken-burns on image *and* video clips**
|
|
102
102
|
(`"kenBurns":{"zoom":"in"|"out","amount":0.12}` or a `zoom` effect, via `zoompan`),
|
|
103
103
|
image/text/caption overlays with `position`+`duration` timing and `rect` placement,
|
|
104
|
-
**
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
104
|
+
**overlay keyframes** - opacity (`effects:[{"type":"opacity","keyframes":[{"t","v"}]}]`
|
|
105
|
+
-> drawtext/alpha), **move/slide** (`{"type":"move","x":[{t,v}],"y":[{t,v}]}` ->
|
|
106
|
+
animated x/y), and **zoom on sized image overlays** (`{"type":"zoom","keyframes":...}`
|
|
107
|
+
with `rect` w+h) - and an `audio[]` mix with per-track `position`/`gain`, **sidechain
|
|
108
|
+
ducking** (`"ducking":{"under":"<asset>"}` -> `sidechaincompress`), plus `loudnorm`.
|
|
109
|
+
Output is bounded to the computed video duration and uses the shared web-safe profile.
|
|
110
|
+
Remaining programmatic-engine territory (emitted as `# NOTE:` lines): animated text
|
|
111
|
+
font size and free-form overlay shrink/pop below 1x.
|
|
110
112
|
|
|
111
113
|
```bash
|
|
112
114
|
maggie tool maggie_video.py compile --spec timeline.json # print the command
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
name: maggie-video
|
|
3
3
|
description: Produce and process marketing video - render a composition to MP4/GIF/WebM, assemble clips/images/text/captions/audio on a timeline, cut to channel aspects (reel/square/youtube), burn in captions, normalize loudness, watermark, transcode, and make social cuts. Engine-neutral over the renderer (ffmpeg for assembly/transcode, a programmatic engine for code-drawn animation). Use when the user wants to make, edit, cut, caption, resize, or export a video for marketing.
|
|
4
4
|
metadata:
|
|
5
|
-
version: 0.1.
|
|
5
|
+
version: 0.1.3
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# Maggie Video
|
|
@@ -66,15 +66,18 @@ maggie tool maggie_video.py concat --inputs a.mp4 b.mp4 --output all.mp4 --reenc
|
|
|
66
66
|
maggie tool maggie_video.py thumb --input in.mp4 --output poster.jpg --at 00:00:03
|
|
67
67
|
maggie tool maggie_video.py compile --spec timeline.json # declarative -> one command
|
|
68
68
|
maggie tool maggie_video.py captions --input vo.mp3 --output vo.srt # local whisper -> SRT
|
|
69
|
+
maggie tool maggie_video.py --run autocut --input talk.mp4 --output tight.mp4 # remove silent gaps
|
|
69
70
|
maggie tool maggie_video.py --run encode --input in.mov --output out.mp4 # actually runs
|
|
70
71
|
maggie tool maggie_video.py presets
|
|
71
72
|
```
|
|
72
73
|
|
|
73
74
|
Subcommands: `encode` (web-safe MP4 / VP9 WebM), `gif`, `trim`, `concat`, `social`,
|
|
74
75
|
`overlay`, `subtitle` (burn or soft-mux), `loudnorm`, `thumb`, `compile` (timeline
|
|
75
|
-
JSON -> ffmpeg
|
|
76
|
-
|
|
77
|
-
|
|
76
|
+
JSON -> ffmpeg: ken-burns on image+video clips, concat/crossfade, overlay opacity/
|
|
77
|
+
move/zoom keyframes, audio mix + sidechain ducking + loudnorm), `autocut` (remove
|
|
78
|
+
silent gaps), `captions` (local whisper speech-to-SRT), `presets`. `--run` (before
|
|
79
|
+
the subcommand) executes when the tool is on PATH; otherwise the command is printed
|
|
80
|
+
for review.
|
|
78
81
|
|
|
79
82
|
## Auto-captions workflow (voiceover -> subtitles -> burned-in)
|
|
80
83
|
|
|
@@ -9,7 +9,7 @@ LGPL/GPL use); it links no libav and vendors no GPL code. Output is always force
|
|
|
9
9
|
to a web-safe, platform-correct profile.
|
|
10
10
|
|
|
11
11
|
Subcommands: encode, gif, trim, concat, social, overlay, subtitle, loudnorm,
|
|
12
|
-
thumb, compile, captions, presets.
|
|
12
|
+
thumb, compile, autocut, captions, presets.
|
|
13
13
|
"""
|
|
14
14
|
import argparse
|
|
15
15
|
import json
|
|
@@ -161,28 +161,36 @@ def _clip_dur(clip, default):
|
|
|
161
161
|
return float(default)
|
|
162
162
|
|
|
163
163
|
|
|
164
|
-
def _kenburns(w, h, fps, dur, direction="in", amount=0.12):
|
|
165
|
-
"""A ken-burns (slow zoom) filter chain
|
|
166
|
-
|
|
164
|
+
def _kenburns(w, h, fps, dur, direction="in", amount=0.12, normalize_fps=False):
|
|
165
|
+
"""A ken-burns (slow zoom) filter chain, fit to WxH. Prescales with headroom so
|
|
166
|
+
the zoom stays sharp and animates over the clip (d=1 keeps it smooth on video
|
|
167
|
+
too). normalize_fps prepends an fps filter so a video clip's input frame count
|
|
168
|
+
matches dur*fps (image inputs are already pinned via -framerate)."""
|
|
167
169
|
bw, bh = int(w * 1.6), int(h * 1.6)
|
|
168
170
|
n = max(1, round(dur * fps))
|
|
169
171
|
if direction == "out":
|
|
170
172
|
z = f"(1+{amount})-{amount}*in/{n}"
|
|
171
173
|
else:
|
|
172
174
|
z = f"1+{amount}*in/{n}"
|
|
173
|
-
|
|
175
|
+
lead = f"fps={fps}," if normalize_fps else ""
|
|
176
|
+
return (f"{lead}scale={bw}:{bh}:force_original_aspect_ratio=increase,crop={bw}:{bh},"
|
|
174
177
|
f"zoompan=z='{z}':d=1:x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':"
|
|
175
178
|
f"s={w}x{h}:fps={fps},setsar=1,format=yuv420p")
|
|
176
179
|
|
|
177
180
|
|
|
178
|
-
def
|
|
179
|
-
"""A piecewise-linear ffmpeg expression
|
|
180
|
-
|
|
181
|
-
expr = f"{
|
|
182
|
-
for (ta, va), (tb, vb) in reversed(list(zip(
|
|
181
|
+
def _kf_expr(points):
|
|
182
|
+
"""A piecewise-linear ffmpeg expression in t (seconds) from (t_abs, value) points.
|
|
183
|
+
Holds the first/last value outside the keyframed range."""
|
|
184
|
+
expr = f"{points[-1][1]}"
|
|
185
|
+
for (ta, va), (tb, vb) in reversed(list(zip(points, points[1:]))):
|
|
183
186
|
seg = f"({va}+({vb}-{va})*(t-{ta})/({tb - ta}))"
|
|
184
187
|
expr = f"if(lt(t,{tb}),{seg},{expr})"
|
|
185
|
-
return f"if(lt(t,{
|
|
188
|
+
return f"if(lt(t,{points[0][0]}),{points[0][1]},{expr})"
|
|
189
|
+
|
|
190
|
+
|
|
191
|
+
def _kf_alpha_expr(keyframes, offset):
|
|
192
|
+
"""Opacity (alpha) expression in t from keyframes relative to the clip offset."""
|
|
193
|
+
return _kf_expr([(offset + float(k["t"]), float(k["v"])) for k in keyframes])
|
|
186
194
|
|
|
187
195
|
|
|
188
196
|
def _opacity_kfs(clip):
|
|
@@ -192,6 +200,29 @@ def _opacity_kfs(clip):
|
|
|
192
200
|
return None
|
|
193
201
|
|
|
194
202
|
|
|
203
|
+
def _move_exprs(clip, offset, W, H, base_x, base_y):
|
|
204
|
+
"""x/y ffmpeg expressions (in t) for a move/slide effect, else (None, None).
|
|
205
|
+
Effect: {type: move|slide, x:[{t,v}...], y:[{t,v}...]} with v as %/px/0..1."""
|
|
206
|
+
for e in clip.get("effects", []) or []:
|
|
207
|
+
if e.get("type") in ("move", "slide", "position"):
|
|
208
|
+
xe = ye = None
|
|
209
|
+
if e.get("x"):
|
|
210
|
+
xe = _kf_expr([(offset + float(k["t"]), _px(k["v"], W)) for k in e["x"]])
|
|
211
|
+
if e.get("y"):
|
|
212
|
+
ye = _kf_expr([(offset + float(k["t"]), _px(k["v"], H)) for k in e["y"]])
|
|
213
|
+
return (xe if xe is not None else base_x, ye if ye is not None else base_y)
|
|
214
|
+
return (None, None)
|
|
215
|
+
|
|
216
|
+
|
|
217
|
+
def _overlay_zoom_cfg(clip):
|
|
218
|
+
"""direction, amount for an image-overlay zoom effect, else None."""
|
|
219
|
+
for e in clip.get("effects", []) or []:
|
|
220
|
+
if e.get("type") == "zoom" and e.get("keyframes"):
|
|
221
|
+
vs = [float(k["v"]) for k in e["keyframes"]]
|
|
222
|
+
return ("out" if vs[-1] < vs[0] else "in"), (abs(vs[-1] - vs[0]) or 0.15)
|
|
223
|
+
return None
|
|
224
|
+
|
|
225
|
+
|
|
195
226
|
def _kenburns_cfg(clip):
|
|
196
227
|
kb = clip.get("kenBurns")
|
|
197
228
|
if isinstance(kb, dict):
|
|
@@ -204,6 +235,13 @@ def _kenburns_cfg(clip):
|
|
|
204
235
|
return None
|
|
205
236
|
|
|
206
237
|
|
|
238
|
+
def _color(c):
|
|
239
|
+
"""Normalize a CSS-ish color for ffmpeg: expand #abc -> #aabbcc; pass the rest."""
|
|
240
|
+
if isinstance(c, str) and c.startswith("#") and len(c) == 4:
|
|
241
|
+
return "#" + "".join(ch * 2 for ch in c[1:])
|
|
242
|
+
return c
|
|
243
|
+
|
|
244
|
+
|
|
207
245
|
def _px(val, total):
|
|
208
246
|
"""Resolve '80%' / 0.8 / 120 against a total dimension to an int pixel value."""
|
|
209
247
|
if isinstance(val, str) and val.strip().endswith("%"):
|
|
@@ -260,9 +298,11 @@ def c_compile(a):
|
|
|
260
298
|
idx = add_input([], asset.get("src", ""))
|
|
261
299
|
s = clip.get("start", 0)
|
|
262
300
|
trim = f"trim=start={s}:end={s + dur},setpts=PTS-STARTPTS,"
|
|
263
|
-
filters.append(f"[{idx}:v]{trim}{_fit(W, H, fit, FPS)}[v{i}]")
|
|
264
301
|
if kb:
|
|
265
|
-
|
|
302
|
+
chain = _kenburns(W, H, FPS, dur, kb[0], kb[1], normalize_fps=True)
|
|
303
|
+
else:
|
|
304
|
+
chain = _fit(W, H, fit, FPS)
|
|
305
|
+
filters.append(f"[{idx}:v]{trim}{chain}[v{i}]")
|
|
266
306
|
else:
|
|
267
307
|
notes.append(f"clip {i}: unsupported type (need asset of type image/video)")
|
|
268
308
|
continue
|
|
@@ -300,19 +340,19 @@ def c_compile(a):
|
|
|
300
340
|
dur = _clip_dur(clip, def_dur)
|
|
301
341
|
en = f"enable='between(t,{pos},{pos + dur})'"
|
|
302
342
|
op_kfs = _opacity_kfs(clip)
|
|
303
|
-
if clip.get("effects") and not op_kfs:
|
|
304
|
-
notes.append("overlay effect other than opacity (e.g. zoom) not compiled on this overlay")
|
|
305
343
|
ctype = clip.get("type")
|
|
306
344
|
if ctype == "text":
|
|
307
345
|
st = clip.get("style", {})
|
|
308
346
|
rect = clip.get("rect", {})
|
|
309
|
-
|
|
310
|
-
|
|
347
|
+
bx, by = _px(rect.get("x", "10%"), W), _px(rect.get("y", "80%"), H)
|
|
348
|
+
mx, my = _move_exprs(clip, pos, W, H, bx, by)
|
|
349
|
+
xv = f"'{mx}'" if mx is not None else f"{bx}"
|
|
350
|
+
yv = f"'{my}'" if my is not None else f"{by}"
|
|
311
351
|
txt = str(clip.get("text", "")).replace(":", r"\:").replace("'", "’")
|
|
312
|
-
box = f":box=1:boxcolor={st['box']}:boxborderw=12" if st.get("box") else ""
|
|
352
|
+
box = f":box=1:boxcolor={_color(st['box'])}:boxborderw=12" if st.get("box") else ""
|
|
313
353
|
alpha = f":alpha='{_kf_alpha_expr(op_kfs, pos)}'" if op_kfs else ""
|
|
314
|
-
dt = (f"drawtext=text='{txt}':x={
|
|
315
|
-
f"fontcolor={st.get('color','#FFFFFF')}{box}{alpha}:{en}")
|
|
354
|
+
dt = (f"drawtext=text='{txt}':x={xv}:y={yv}:fontsize={st.get('size',48)}:"
|
|
355
|
+
f"fontcolor={_color(st.get('color','#FFFFFF'))}{box}{alpha}:{en}")
|
|
316
356
|
filters.append(f"{cur}{dt}{nxt}")
|
|
317
357
|
elif ctype == "captions":
|
|
318
358
|
src = clip.get("src", "")
|
|
@@ -321,46 +361,75 @@ def c_compile(a):
|
|
|
321
361
|
asset = assets[clip["asset"]]
|
|
322
362
|
rect = clip.get("rect", {})
|
|
323
363
|
ow = _px(rect.get("w", "15%"), W)
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
364
|
+
bx, by = _px(rect.get("x", "80%"), W), _px(rect.get("y", "5%"), H)
|
|
365
|
+
zoom = _overlay_zoom_cfg(clip)
|
|
366
|
+
if zoom and rect.get("h") not in (None, "auto"):
|
|
367
|
+
oh = _px(rect.get("h"), H)
|
|
368
|
+
idx = add_input(["-loop", "1", "-framerate", f"{FPS}", "-t", f"{dur}"],
|
|
369
|
+
asset.get("src", ""))
|
|
370
|
+
filters.append(f"[{idx}:v]{_kenburns(ow, oh, FPS, dur, zoom[0], zoom[1])}[ov{step}]")
|
|
371
|
+
else:
|
|
372
|
+
if zoom:
|
|
373
|
+
notes.append("image-overlay zoom needs rect w+h; placed at a static size "
|
|
374
|
+
"(use a programmatic engine for free-form overlay scaling)")
|
|
375
|
+
idx = add_input(["-loop", "1"], asset.get("src", ""))
|
|
376
|
+
filters.append(f"[{idx}:v]scale={ow}:-1[ov{step}]")
|
|
377
|
+
mx, my = _move_exprs(clip, pos, W, H, bx, by)
|
|
378
|
+
xv = f"x='{mx}'" if mx is not None else f"x={bx}"
|
|
379
|
+
yv = f"y='{my}'" if my is not None else f"y={by}"
|
|
380
|
+
filters.append(f"{cur}[ov{step}]overlay={xv}:{yv}:{en}:eof_action=pass{nxt}")
|
|
329
381
|
else:
|
|
330
382
|
notes.append(f"overlay clip of type {ctype!r} not compiled")
|
|
331
383
|
continue
|
|
384
|
+
if clip.get("effects") and not (op_kfs or _move_exprs(clip, pos, W, H, 0, 0) != (None, None) or _overlay_zoom_cfg(clip)):
|
|
385
|
+
notes.append("an overlay effect was not recognised (supported: opacity, move/slide, zoom on sized images)")
|
|
332
386
|
cur = nxt
|
|
333
387
|
step += 1
|
|
334
388
|
filters.append(f"{cur}format=yuv420p[vout]")
|
|
335
389
|
|
|
336
|
-
# --- audio: mix audio[] tracks with gain
|
|
337
|
-
aud = spec.get("audio", [])
|
|
338
|
-
|
|
390
|
+
# --- audio: mix audio[] tracks with gain, sidechain ducking, loudnorm ---
|
|
391
|
+
aud = [t for t in spec.get("audio", []) if assets.get(t.get("asset"), {}).get("src")]
|
|
392
|
+
# which assets are used as a ducking key (the VO under which music ducks)
|
|
393
|
+
keys = {t["ducking"]["under"] for t in aud if t.get("ducking") and t["ducking"].get("under")}
|
|
394
|
+
asset_track = {}
|
|
395
|
+
raw = {} # track index -> its raw [araw{i}] label
|
|
339
396
|
for i, tr in enumerate(aud):
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
continue
|
|
343
|
-
idx = add_input([], asset["src"])
|
|
344
|
-
chain = f"[{idx}:a]"
|
|
397
|
+
idx = add_input([], assets[tr["asset"]]["src"])
|
|
398
|
+
parts = []
|
|
345
399
|
pos = float(tr.get("position", 0))
|
|
346
400
|
if pos > 0:
|
|
347
401
|
ms = int(pos * 1000)
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
402
|
+
parts.append(f"adelay={ms}|{ms}")
|
|
403
|
+
if tr.get("gain"):
|
|
404
|
+
parts.append(f"volume={tr['gain']}dB")
|
|
405
|
+
chain = ",".join(parts) if parts else "anull"
|
|
406
|
+
filters.append(f"[{idx}:a]{chain}[araw{i}]")
|
|
407
|
+
raw[i] = f"[araw{i}]"
|
|
408
|
+
asset_track.setdefault(tr["asset"], i)
|
|
409
|
+
# split any track that serves as a ducking key into a mix copy and a key copy
|
|
410
|
+
mix = dict(raw)
|
|
411
|
+
keyl = {}
|
|
412
|
+
for aid in keys:
|
|
413
|
+
j = asset_track.get(aid)
|
|
414
|
+
if j is None:
|
|
415
|
+
continue
|
|
416
|
+
filters.append(f"{raw[j]}asplit=2[amx{j}][akey{j}]")
|
|
417
|
+
mix[j], keyl[aid] = f"[amx{j}]", f"[akey{j}]"
|
|
418
|
+
# apply sidechain ducking: music under its key VO
|
|
419
|
+
for i, tr in enumerate(aud):
|
|
420
|
+
d = tr.get("ducking")
|
|
421
|
+
if d and keyl.get(d.get("under")):
|
|
422
|
+
filters.append(f"{mix[i]}{keyl[d['under']]}sidechaincompress="
|
|
423
|
+
f"threshold=0.03:ratio=8:attack=20:release=300[aduck{i}]")
|
|
424
|
+
mix[i] = f"[aduck{i}]"
|
|
425
|
+
notes.append(f"ducking: '{tr['asset']}' compressed under '{d['under']}' (sidechain)")
|
|
426
|
+
alabels = [mix[i] for i in range(len(aud))]
|
|
357
427
|
have_audio = bool(alabels)
|
|
358
428
|
if have_audio:
|
|
359
429
|
if len(alabels) == 1:
|
|
360
430
|
filters.append(f"{alabels[0]}aresample=async=1[amix]")
|
|
361
431
|
else:
|
|
362
|
-
filters.append(f"{''.join(alabels)}amix=inputs={len(alabels)}:duration=longest[amix]")
|
|
363
|
-
norm = next((t for t in aud if isinstance(t, dict)), {}) # master normalize lives on audio? use output
|
|
432
|
+
filters.append(f"{''.join(alabels)}amix=inputs={len(alabels)}:duration=longest:normalize=0[amix]")
|
|
364
433
|
tgt = (spec.get("audio_normalize") or {}).get("I", -14)
|
|
365
434
|
filters.append(f"[amix]loudnorm=I={tgt}:TP=-1.5:LRA=11[aout]")
|
|
366
435
|
|
|
@@ -374,6 +443,61 @@ def c_compile(a):
|
|
|
374
443
|
return emit(cmd, a.run)
|
|
375
444
|
|
|
376
445
|
|
|
446
|
+
def c_autocut(a):
|
|
447
|
+
"""Remove silent gaps automatically (auto-editor's idea, via ffmpeg
|
|
448
|
+
silencedetect). Prints the detect command; with --run it detects, computes the
|
|
449
|
+
kept ranges (padded by --margin), and renders the trimmed video+audio."""
|
|
450
|
+
import re
|
|
451
|
+
detect = ["ffmpeg", "-hide_banner", "-i", a.input, "-af",
|
|
452
|
+
f"silencedetect=noise={a.noise}:d={a.min_silence}", "-f", "null", "-"]
|
|
453
|
+
print("# step 1 (detect): " + " ".join(shlex.quote(x) for x in detect))
|
|
454
|
+
print("# step 2: kept ranges = complement of detected silence, padded by --margin, then trim+concat.")
|
|
455
|
+
if not a.run:
|
|
456
|
+
print("# re-run with --run to detect and render the cut (needs ffmpeg + ffprobe).")
|
|
457
|
+
return 0
|
|
458
|
+
if not (shutil.which("ffmpeg") and shutil.which("ffprobe")):
|
|
459
|
+
sys.stderr.write("autocut --run needs ffmpeg + ffprobe on PATH.\n")
|
|
460
|
+
return 3
|
|
461
|
+
total = float(subprocess.run(["ffprobe", "-v", "error", "-show_entries",
|
|
462
|
+
"format=duration", "-of", "csv=p=0", a.input],
|
|
463
|
+
capture_output=True, text=True).stdout.strip() or 0)
|
|
464
|
+
err = subprocess.run(detect, capture_output=True, text=True).stderr
|
|
465
|
+
starts = [float(x) for x in re.findall(r"silence_start: ([0-9.]+)", err)]
|
|
466
|
+
ends = [float(x) for x in re.findall(r"silence_end: ([0-9.]+)", err)]
|
|
467
|
+
if len(ends) < len(starts): # trailing silence runs to the end
|
|
468
|
+
ends.append(total)
|
|
469
|
+
sil = list(zip(starts, ends))
|
|
470
|
+
keep, cur = [], 0.0
|
|
471
|
+
for s, e in sil:
|
|
472
|
+
if s > cur:
|
|
473
|
+
keep.append([cur, s])
|
|
474
|
+
cur = max(cur, e)
|
|
475
|
+
if cur < total:
|
|
476
|
+
keep.append([cur, total])
|
|
477
|
+
m = a.margin
|
|
478
|
+
padded = [[max(0.0, s - m), min(total, e + m)] for s, e in keep]
|
|
479
|
+
merged = []
|
|
480
|
+
for seg in padded:
|
|
481
|
+
if merged and seg[0] <= merged[-1][1]:
|
|
482
|
+
merged[-1][1] = max(merged[-1][1], seg[1])
|
|
483
|
+
else:
|
|
484
|
+
merged.append(seg)
|
|
485
|
+
print(f"# detected {len(sil)} silence gap(s); keeping {len(merged)} segment(s), "
|
|
486
|
+
f"{round(sum(e - s for s, e in merged), 2)}s of {round(total, 2)}s")
|
|
487
|
+
if not merged:
|
|
488
|
+
sys.stderr.write("autocut: everything was silence; nothing to keep.\n")
|
|
489
|
+
return 1
|
|
490
|
+
parts, labels = [], []
|
|
491
|
+
for i, (s, e) in enumerate(merged):
|
|
492
|
+
parts.append(f"[0:v]trim=start={s}:end={e},setpts=PTS-STARTPTS[v{i}]")
|
|
493
|
+
parts.append(f"[0:a]atrim=start={s}:end={e},asetpts=PTS-STARTPTS[a{i}]")
|
|
494
|
+
labels.append(f"[v{i}][a{i}]")
|
|
495
|
+
fc = ";".join(parts) + f";{''.join(labels)}concat=n={len(merged)}:v=1:a=1[v][a]"
|
|
496
|
+
cmd = PREFIX + ["-i", a.input, "-filter_complex", fc, "-map", "[v]", "-map", "[a]"] + \
|
|
497
|
+
MP4_OUT + ["-c:a", "aac", "-b:a", "128k", a.output]
|
|
498
|
+
return emit(cmd, True)
|
|
499
|
+
|
|
500
|
+
|
|
377
501
|
def c_captions(a):
|
|
378
502
|
"""Transcribe speech to an SRT with a local, offline engine (no paid API).
|
|
379
503
|
Pairs with `subtitle` (burn in) and `compile` (a captions overlay)."""
|
|
@@ -470,6 +594,14 @@ def main(argv=None):
|
|
|
470
594
|
cp.add_argument("--output", help="override the output path from the spec")
|
|
471
595
|
cp.set_defaults(fn=c_compile)
|
|
472
596
|
|
|
597
|
+
ac = sub.add_parser("autocut", help="remove silent gaps automatically (silencedetect)")
|
|
598
|
+
ac.add_argument("--input", required=True); ac.add_argument("--output", required=True)
|
|
599
|
+
ac.add_argument("--noise", default="-30dB", help="silence threshold, e.g. -30dB")
|
|
600
|
+
ac.add_argument("--min-silence", dest="min_silence", type=float, default=0.5,
|
|
601
|
+
help="minimum silence length to cut (s)")
|
|
602
|
+
ac.add_argument("--margin", type=float, default=0.1, help="padding kept around speech (s)")
|
|
603
|
+
ac.set_defaults(fn=c_autocut)
|
|
604
|
+
|
|
473
605
|
ca = sub.add_parser("captions", help="transcribe speech to SRT (local whisper)")
|
|
474
606
|
ca.add_argument("--input", required=True); ca.add_argument("--output", required=True, help="target .srt")
|
|
475
607
|
ca.add_argument("--engine", choices=["whisper", "whispercpp", "whisperx"], default="whisper")
|
package/package.json
CHANGED
|
@@ -103,9 +103,13 @@ ffmpeg -y -i IN -vf "thumbnail,scale=1280:-1" -frames:v 1 OUT.jpg # most repres
|
|
|
103
103
|
|
|
104
104
|
```bash
|
|
105
105
|
ffmpeg -i IN -af silencedetect=noise=-30dB:d=0.3 -f null - # detect ranges
|
|
106
|
-
# then build a
|
|
106
|
+
# then build a trim+concat of the kept ranges (pad each ~0.1s)
|
|
107
107
|
```
|
|
108
108
|
|
|
109
|
+
`maggie_video.py autocut --input talk.mp4 --output tight.mp4 --run` does both:
|
|
110
|
+
detects the silence with `silencedetect`, computes the kept ranges padded by
|
|
111
|
+
`--margin`, and renders the trimmed video+audio.
|
|
112
|
+
|
|
109
113
|
## Transitions (between two clips)
|
|
110
114
|
|
|
111
115
|
`xfade=transition=fade:duration=0.5:offset=5.0` for video and `acrossfade=d=0.5`
|
|
@@ -97,16 +97,18 @@ shape renders through ffmpeg without any GPL engine.
|
|
|
97
97
|
|
|
98
98
|
`maggie_video.py compile --spec timeline.json` compiles this model into one ffmpeg
|
|
99
99
|
`filter_complex` command (printed; run with `--run`). Supported today: a main video
|
|
100
|
-
track of clips (video/image) fit `cover`/`contain
|
|
101
|
-
|
|
100
|
+
track of clips (video/image) fit `cover`/`contain`, hard-cut `concat` or sequential
|
|
101
|
+
`crossfade` (from `transitions`), **ken-burns on image *and* video clips**
|
|
102
102
|
(`"kenBurns":{"zoom":"in"|"out","amount":0.12}` or a `zoom` effect, via `zoompan`),
|
|
103
103
|
image/text/caption overlays with `position`+`duration` timing and `rect` placement,
|
|
104
|
-
**
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
104
|
+
**overlay keyframes** - opacity (`effects:[{"type":"opacity","keyframes":[{"t","v"}]}]`
|
|
105
|
+
-> drawtext/alpha), **move/slide** (`{"type":"move","x":[{t,v}],"y":[{t,v}]}` ->
|
|
106
|
+
animated x/y), and **zoom on sized image overlays** (`{"type":"zoom","keyframes":...}`
|
|
107
|
+
with `rect` w+h) - and an `audio[]` mix with per-track `position`/`gain`, **sidechain
|
|
108
|
+
ducking** (`"ducking":{"under":"<asset>"}` -> `sidechaincompress`), plus `loudnorm`.
|
|
109
|
+
Output is bounded to the computed video duration and uses the shared web-safe profile.
|
|
110
|
+
Remaining programmatic-engine territory (emitted as `# NOTE:` lines): animated text
|
|
111
|
+
font size and free-form overlay shrink/pop below 1x.
|
|
110
112
|
|
|
111
113
|
```bash
|
|
112
114
|
maggie tool maggie_video.py compile --spec timeline.json # print the command
|