@topy-ai/maggie 0.8.15 → 0.8.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README-zh-TW.md +1 -1
- package/README.md +1 -1
- package/bundled-references/captions-transcription.md +21 -1
- package/bundled-references/design-editor-patterns.md +29 -2
- package/bundled-references/ffmpeg-recipes.md +21 -0
- package/bundled-references/generative-visuals.md +23 -0
- package/bundled-references/image-recipes.md +13 -0
- package/bundled-references/programmatic-video.md +22 -0
- package/bundled-references/video-timeline-model.md +22 -4
- package/bundled-skills/maggie-image/SKILL.md +1 -1
- package/bundled-skills/maggie-video/SKILL.md +1 -1
- package/bundled-tools/clis/maggie_image.py +12 -2
- package/bundled-tools/clis/maggie_video.py +22 -14
- package/package.json +1 -1
- package/references/captions-transcription.md +21 -1
- package/references/design-editor-patterns.md +29 -2
- package/references/ffmpeg-recipes.md +21 -0
- package/references/generative-visuals.md +23 -0
- package/references/image-recipes.md +13 -0
- package/references/programmatic-video.md +22 -0
- package/references/video-timeline-model.md +22 -4
package/README-zh-TW.md
CHANGED
|
@@ -143,7 +143,7 @@ structure drift 或重複 treatment product 時 fail closed。
|
|
|
143
143
|
deployment、memory、feedback 和 MaggieDash。內容先 draft/review,外部寫入、
|
|
144
144
|
publish 與 production deployment 需要明確確認。
|
|
145
145
|
|
|
146
|
-
目前 npm 版本為 `0.8.
|
|
146
|
+
目前 npm 版本為 `0.8.16`,把先前僅 README 的 repo 解稀疏為完整源碼並逐一代碼級研究(驗證所有 reference 並補 net-new:Remotion delayRender gate、opentoonz BSD 可移植的 xsheet 時間線、綠幕/LUT/卡拉OK 濾鏡、whisper turbo + 字幕旗標、Babylon NullEngine 無頭),並新增 maggie_image.py trim/auto-orient 與更豐富的 captions 旗標。先前 0.8.15 補齊除 Q 外所有設計類的覆蓋:maggie_image.py 新增 svg-export(Inkscape)/composite/look,新增 design-editor-patterns reference,並擴充 web-animation/generative-visuals/video-timeline;docs/design-coverage-audit.md 逐 repo 確認。先前 0.8.14 新增 `maggie-ai-media`(去背/放大/arm's-length ComfyUI 的命令生成器,自身不下載模型、不用 GPU),讓 Q 類達到同級;重型執行仍受成本把關。先前 0.8.13 新增 AI 媒體生成規劃 reference(Q 類:自託管模型概覽、程式碼授權≠權重授權規則、商用安全子集 rembg/Real-ESRGAN、成本/BYOK 守則;實際建置待明確許可)。先前 0.8.12 收尾其餘設計類:新增 `maggie-image`(影像處理 CLI:尺寸/OG/favicon/WebP-AVIF/svg/montage)與 web 動效、生成式視覺、圖像配方、字幕四個 reference。先前 0.8.11 收尾 N 類影片功能:`maggie_video.py compile` 再補上視頻 clip ken-burns、sidechain ducking、以及非透明度 overlay 效果(move/slide、圖片 zoom),並新增 `autocut` 子命令(silencedetect 去靜音)。先前 0.8.10 新增圖片 ken-burns + 文字透明度關鍵幀與 `captions`(whisper)子命令。先前 0.8.9 新增 `compile`(宣告式 timeline JSON→單一 ffmpeg 命令)。先前 0.8.8 新增 `maggie-video`(製作與處理行銷影片:剪輯、字幕、裁切成各渠道比例、浮水印、轉碼、響度正規化、clip 組裝、程式化動畫),附一個實測可用的 ffmpeg 配方 CLI(`maggie_video.py`),從參考庫的影片專案(moviepy/editly/ffmpeg 命令層、Revideo/Motion Canvas/CCapture 程式化渲染、MLT/OpenShot/Olive 的 timeline 資料形狀)提煉而來;另加 `ffmpeg-recipes`、`programmatic-video`、`video-timeline-model` 三個 reference。先前 0.8.7 把 redesign 稽核併入 `maggie-design`(改編自 MIT 的 `elayadesign/redesign-skill`):掃描 + 診斷 AI 通病/缺失狀態/雜亂程式碼,再經 in-place/verify 流程就地修正,因此「稽核/打磨/升級既有站點」會路由到 maggie-design。先前 0.8.6 補齊兩個設計技能:加入完整的生產 reference,並把設計來源明確為 Figma 非必需(實時 URL、既有程式碼、Figma、或純截圖)。先前 0.8.5 採納兩個設計技能(改編自 MIT 的 `elayadesign/ai-design-skills`):maggie-landing-page(轉化策略 + 視覺系統紀律,具體數值為可被專案 DESIGN.md 覆蓋的預設值;由 maggie-design 生產與驗證)與 maggie-design-motion(靜態設計→像素級動畫 HTML 複刻→匯出 MP4 + GIF)。先前 0.8.4 將技能庫擴展成完整的「一人用 AI 跑一家公司」C-suite,新增八條職能線:maggie-sales(CRM 中立的銷售管線)、maggie-finance(只解釋的 CFO 指標,含確定性 maggie_finance.py 計算器)、maggie-support(面向客戶的客戶成功)、maggie-product(路線圖 + 預註冊實驗)、maggie-fundraising(pitch deck、data room、投資人更新)、maggie-legal(ToS/隱私/DPA/SOC2 readiness)、maggie-monetization(定價/paywall/dunning,金錢操作需人工批准)、maggie-orchestrator(CEO 編排層:將目標拆成帶審批門的 mission 分派到各職能線,含 maggie_orchestrator.py 規劃器)。先前 0.8.3 將 cmo.so/www 開發經驗沉澱進技能庫:新增兩個技能 maggie-skill-authoring(依 catalog 與 MCP 工具白名單撰寫/校驗社區營銷 playbook 技能)與 maggie-feedback-triage(反饋的團隊接收端:決策/定級/分派、PRD + issue 流程、24/7 隊列守護);新增多個 reference(OAuth PKCE 登入、Expo 約定與 Router 骨架、MCP evals 測試框架、conversation cards、WordPress→Firebase 認證遷移手冊);並增補 maggie-assistant-tools(tool≤user 每輪工具過濾、agent loop 安全不變量、preview-only 橋接代理、MCP resources/prompts、注解加固、CLI 與 ChatGPT 插件打包)、maggie-auth-reference(自清理登入測試套件、自簽單次令牌)、maggie-mobile-app(OTA 開機拉取、live==built 部署守衛)、maggie-deployment(pm2 單包發布、live-hash 校驗)與 maggie-ops(template-as-code 郵件同步)。先前 0.8.2 新增 maggie-spirit(為 AI 品牌生成平面像素 spirit/sprite 系統)與 goal-led AI agent 設計參考;0.8.0 為安全優化 roadmap(scan、image-patch、標頭評分、SSVC、GDPR 日誌規則、新 MCP 工具)。先前 0.8.0 為安全優化 roadmap(scan、image-patch、標頭評分、SSVC、GDPR 日誌規則、新 MCP 工具),含 fix-plan 修補風險分級 T1–T4(判斷補丁是否安全套用)與 host-impact 補丁影響面量測(哪些服務需重啟、是否需重開機)、public-check、Maggie Security、batch 31 排程 agent routines、報告收件匣與無人值守 agent 限制、PHP runtime 盤點與主機鎖、pull 網站改寫、query 點子與社群佇列檢查,batch 30 MaggieDash 先裝核心、release privacy gate、feedback 備援、社群連結與圖片檢查、大量頁面產生與 app 檢查,batch 29 預約規則、部署、app QA、助理與社群功能,以及 batch 28 產品 app、assistant tools、mobile、product release、skills catalog 與 bridge host 功能、batch 26、27 feedback 修正與 idempotent `maggie update`與 Booking email-template 合約/CLI validator 同步。本版完成 2026-09-24 booking feedback batch:Calendar grid/free-slot entry 與 host modal focus-restoration evidence。本版新增 Starter marketplace provisioning plan;本版完成 feedback batches 22、23:加入 declarative browser interaction evidence、design markup/style/content-key gate、compiler source anchoring、entity double-escape protection、`update --help` read-only、noindex hypothetical indexability、可過期 baseline acknowledgement、DOM-only link extraction regression,以及 MaggieDash 0.2.10 的 content override、短期 browser session、CDN-safe public-session cache contracts。authentication 和 isolated dev server 仍由 host 負責。`maggie-booking` workflow、傳統 email/password、Stripe、Resend、public booking flow 及既有安全邊界維持不變;provider-bound video generation 與 transactional delivery 仍記錄在 issue #39,前一批延後功能仍在 issue #38,不會直接成為 active memory。0.7.33 記錄並審查一筆更正回饋;確認 0.7.32 已正確處理 batch index,沒有修改已驗證的 shared behavior。0.7.32 加入 feedback 一基索引支援、MaggieDash panel 的 source/freshness/error evidence、Consent Mode 與實際 write/no-op reconciliation、SEO baseline recapture、sitemap origin rebasing、content-diff scope、migration ledger reconciliation、deployment credential preflight、release provenance/runtime preflight、icon release gate、opt-in blog auto-publish gate,以及 least-privilege VPS deployer。也修復 top-level `maggie feedback` dispatcher parity。0.7.31 加入 update 差異摘要、atomic installer copy、subset install manifest merge,以及 inventory drift 時 doctor non-zero。0.7.30 補上 feedback tracking issue closeout gate 與 batch 14/15 audit ledger。0.7.29 修正 npm package landing page 使用英文 README;0.7.28 加入 versioned Gemini model policy、明確 fallback 與 provenance、bundled-first CLI
|
|
147
147
|
dispatch、`maggie --version`、feedback batch review 聚合與重複偵測、marketplace enrichment evidence、booking
|
|
148
148
|
worker/resolver evidence,以及 read-only migration preflight。它也包含 host-owned mobile app surface contract、signed-in
|
|
149
149
|
camera-state QA、直接 Astro route resolution、correlated feedback batch,以及
|
package/README.md
CHANGED
|
@@ -89,7 +89,7 @@ Maggie keeps the existing project foundation and asks for decisions before
|
|
|
89
89
|
shared routes, analytics, or publishing boundaries change. The current
|
|
90
90
|
package ships 39 installable skills and a local-first MaggieDash foundation.
|
|
91
91
|
|
|
92
|
-
This package is `0.8.
|
|
92
|
+
This package is `0.8.16` — de-sparses the earlier README-only repos to full source and studies them at code level (confirm + net-new: Remotion delayRender gate, opentoonz BSD port target, green-screen/LUT/karaoke filters, whisper turbo + caption flags, Babylon NullEngine), plus maggie_image.py trim/auto-orient and richer captions flags. `0.8.15` closed the non-Q design coverage gap: `maggie_image.py` gains svg-export/composite/look, a new design-editor-patterns reference, and growth in web-animation/generative-visuals/video-timeline references. `0.8.14` adds `maggie-ai-media` (bg removal / upscaling / arm's-length ComfyUI as a command generator; no model downloads or GPU itself). `0.8.13` adds the AI-media-generation planning reference (category Q: licensing + safe subset + cost guardrails; build gated on go-ahead). `0.8.12` adds `maggie-image` (process images via an ImageMagick recipe CLI) + references for web animation, generative visuals, image recipes, and captions. `0.8.11` - `maggie_video.py compile` now also does video-clip ken-burns, sidechain ducking, and non-opacity overlay effects (move/slide, image zoom), and a new `autocut` subcommand removes silent gaps. `0.8.10` added image ken-burns + text opacity keyframes and the `captions` (whisper) subcommand. `0.8.9` added `compile` (declarative timeline JSON -> one ffmpeg command). `0.8.8` added `maggie-video` (produce and process marketing video: cut, caption, resize to channel aspects, watermark, transcode, normalize loudness, assemble clips, render code-drawn animation) with a tested ffmpeg-recipe CLI, distilled from the reference library's video projects; plus references `ffmpeg-recipes`, `programmatic-video`, `video-timeline-model`. `0.8.7` folded a redesign audit into `maggie-design` (from the MIT `elayadesign/redesign-skill`): scan + diagnose generic AI-design patterns/weak states/sloppy code, then fix in place via the in-place/verify flow, so "audit/polish/upgrade an existing site" routes to maggie-design. `0.8.6` completed the two adopted design skills with full production references and made the design source Figma-optional (live URL, existing code, Figma, or a screenshot). `0.8.5` adopted the two design skills (adapted, MIT, from `elayadesign/ai-design-skills`): `maggie-landing-page` (conversion strategy + a visual-system discipline whose values defer to the project's DESIGN.md) and `maggie-design-motion` (static design → pixel-exact animated HTML replica → MP4 + GIF export). `0.8.4` expanded the library into a full "one person runs a company with AI" C-suite with eight new function lanes: `maggie-sales` (CRM-neutral pipeline), `maggie-finance` (explain-only CFO metrics + a deterministic `maggie_finance.py` calculator), `maggie-support` (customer success), `maggie-product` (roadmap + pre-registered experiments), `maggie-fundraising` (deck, data room, investor updates), `maggie-legal` (ToS/privacy/DPA/SOC2 readiness), `maggie-monetization` (pricing/paywalls/dunning; money actions human-approved), and `maggie-orchestrator` (the CEO layer that decomposes a goal into gated missions across the lanes, with a `maggie_orchestrator.py` planner). `0.8.3` folded cmo.so/www development learnings in (maggie-skill-authoring, maggie-feedback-triage, and mobile/MCP/auth references); `0.8.2` added `maggie-spirit`; `0.8.0` was the security optimization roadmap.
|
|
93
93
|
|
|
94
94
|
The 0.7.59 release made `maggie security host-audit` (the deep
|
|
95
95
|
lynis audit over SSH) robust on hosts where a storage test can stall: lynis runs
|
|
@@ -9,7 +9,7 @@ Distilled from the reference library's R-category projects.
|
|
|
9
9
|
|
|
10
10
|
| Engine | What / how | License | Reach for it when |
|
|
11
11
|
|---|---|---|---|
|
|
12
|
-
| openai-whisper |
|
|
12
|
+
| openai-whisper | `whisper IN --model turbo --output_format srt --output_dir DIR` (default model is now **`turbo`** = large-v3-turbo; formats `txt/vtt/srt/tsv/json`) | MIT | the default; simplest install (`pip install openai-whisper`) |
|
|
13
13
|
| whisper.cpp | C/C++ port, fast on CPU, GGML models: `whisper-cli -m model.bin -f IN -osrt -of OUTBASE` | MIT | no GPU / fastest CPU / embedding in a pipeline |
|
|
14
14
|
| whisperX | whisper + forced alignment for **word-level timestamps** and diarization: `whisperx IN --model large-v3 --output_format srt --output_dir DIR` | BSD-4 | word-accurate caption timing or speaker labels |
|
|
15
15
|
|
|
@@ -38,4 +38,24 @@ Model size trades speed for accuracy: `tiny` / `base` (fast, rough) -> `small` /
|
|
|
38
38
|
API is ever wired in instead, the global spend-first rule applies - state the cost.
|
|
39
39
|
- Keep caption text inside the safe area (see `maggie-video` channel presets).
|
|
40
40
|
|
|
41
|
+
## Caption-shaping flags (built into openai-whisper)
|
|
42
|
+
|
|
43
|
+
The core CLI already does what we previously routed to whisperX or post-processing
|
|
44
|
+
(`maggie_video.py captions` exposes these):
|
|
45
|
+
|
|
46
|
+
- `--word_timestamps True` - word-level timings (whisperX still wins on alignment +
|
|
47
|
+
diarization, but core whisper now has them).
|
|
48
|
+
- `--highlight_words True` - **karaoke-style** per-word highlight in SRT/VTT (the
|
|
49
|
+
social-caption effect); render the ASS with ffmpeg `subtitles`.
|
|
50
|
+
- `--max_line_width` / `--max_line_count` / `--max_words_per_line` - native line
|
|
51
|
+
wrapping so captions fit the channel safe-area, no post-processing.
|
|
52
|
+
- `--initial_prompt "<brand, product names>"` - bias the model toward correct brand/
|
|
53
|
+
jargon spelling (the concrete mitigation for the mishears-brand-terms risk; still
|
|
54
|
+
human-review).
|
|
55
|
+
- Anti-hallucination over music/silence: `--hallucination_silence_threshold`,
|
|
56
|
+
`--compression_ratio_threshold`, `--logprob_threshold`, `--no_speech_threshold` -
|
|
57
|
+
worth setting for music-bed marketing clips.
|
|
58
|
+
- `--task translate` (foreign audio -> English subs), `--clip_timestamps` (only given
|
|
59
|
+
ranges).
|
|
60
|
+
|
|
41
61
|
See `maggie-video` for the subcommands and the burn-in / soft-mux / compile options.
|
|
@@ -15,8 +15,12 @@ the image is reconstructed by replaying the stack. Borrow this as a maggie "imag
|
|
|
15
15
|
recipe": a small JSON `[ {op, params}, ... ]` stored next to the original and replayed
|
|
16
16
|
by `maggie_image.py` (each op -> an ImageMagick/sharp command). It is **reproducible,
|
|
17
17
|
auditable, re-editable, and batchable** - apply one "look" recipe to every photo in a
|
|
18
|
-
campaign for consistent brand grading. `darktable-cli <
|
|
19
|
-
from such a stack (
|
|
18
|
+
campaign for consistent brand grading. `darktable-cli <input> [sidecar.xmp]
|
|
19
|
+
<output-dir>` renders from such a stack (XMP optional/auto-discovered, final arg is a
|
|
20
|
+
**directory**; GPL binary, arm's-length). Two source details worth adopting: **params
|
|
21
|
+
versioning** (each module keeps `legacy_params` for forward-compatible migration -
|
|
22
|
+
version your recipe ops) and **"Styles" = saved reusable history** (name a look once,
|
|
23
|
+
apply to many).
|
|
20
24
|
|
|
21
25
|
## LLM-generatable scene / diagram formats
|
|
22
26
|
|
|
@@ -75,3 +79,26 @@ only: penpot (MPL), tldraw (custom), krita/krita-ai-diffusion (GPL), drawio icon
|
|
|
75
79
|
MIT and freely borrowable as ideas (still reimplement, don't bulk-copy): excalidraw,
|
|
76
80
|
Method-Draw, rough.js, Pinta. drawio code Apache-2.0; its diagram output is
|
|
77
81
|
unrestricted.
|
|
82
|
+
|
|
83
|
+
## From full-source study (net-new)
|
|
84
|
+
|
|
85
|
+
- **tldraw** - the LLM mutation surface is `editor.createShapes/updateShapes/
|
|
86
|
+
deleteShapes` (batch partial mutations); the store ships schema **migration
|
|
87
|
+
sequences**, a composable **validation layer**, and a **side-effects manager**
|
|
88
|
+
(before/after create/change/delete hooks). `parentId` can be a page *or* a shape;
|
|
89
|
+
pages carry a fractional `index`. Adopt the batch-mutation + validate + migrate
|
|
90
|
+
shape as the model for an agent-written canvas doc.
|
|
91
|
+
- **penpot** - **design tokens are W3C DTCG** (import/export + aliasing), components
|
|
92
|
+
have **variants**, shapes carry **flex/grid layout**; the agent boundary is an
|
|
93
|
+
**official MCP server talking to a sandboxed in-app plugin over WebSocket** - a
|
|
94
|
+
strong arm's-length model for making a design doc agent-writable (mirrors Maggie's
|
|
95
|
+
own MCP posture).
|
|
96
|
+
- **drawio** - the `mxConstants` STYLE_* key table is a **validated allow-list** for
|
|
97
|
+
LLM-emitted styles; `mxEdgeStyle` routing + hierarchical/tree/organic **auto-layout**
|
|
98
|
+
mean a generated node+edge list can self-position; `.drawio` pages are often
|
|
99
|
+
**deflate+base64** compressed.
|
|
100
|
+
- **krita** - an **adjustment/filter layer holds a serialized filter config** = a
|
|
101
|
+
non-destructive recipe living in the layer stack; ops are **mask/selection-scoped**;
|
|
102
|
+
`.kra` is `zip(manifest XML + per-layer blobs)`; a **Python scripting API** and a
|
|
103
|
+
JSON-manifest plugin boundary let a generative backend register like any filter -
|
|
104
|
+
the "AI as a swappable backend" posture.
|
|
@@ -128,3 +128,24 @@ maggie tool maggie_video.py compile --spec timeline.json # declarative timelin
|
|
|
128
128
|
maggie tool maggie_video.py --run encode --input in.mov --output out.mp4 # actually runs
|
|
129
129
|
maggie tool maggie_video.py presets
|
|
130
130
|
```
|
|
131
|
+
|
|
132
|
+
## More filters worth reaching for (confirmed in libavfilter)
|
|
133
|
+
|
|
134
|
+
- **Green-screen / keying** (UGC, product on a brand background):
|
|
135
|
+
`chromakey=0x00FF00:0.1:0.0` (or `colorkey`), clean fringes with `despill`.
|
|
136
|
+
- **Fade to/from black** (distinct from clip crossfade): `fade=t=in:st=0:d=0.5`,
|
|
137
|
+
`fade=t=out:st=9.5:d=0.5`.
|
|
138
|
+
- **Brand color grade / LUT**: `lut3d=brand.cube`, or `curves`/`eq`/`colorbalance`/`hue`.
|
|
139
|
+
- **Karaoke captions**: `subtitles=in.ass` with a full ASS file (word highlight) -
|
|
140
|
+
pairs with whisper `--highlight_words` ([captions](captions-transcription.md)).
|
|
141
|
+
- **Focus/credits**: `gblur` + `vignette`; `scroll` for scrolling outros.
|
|
142
|
+
- **GPU fast path** when hardware is present: `overlay_cuda`/`overlay_vulkan`,
|
|
143
|
+
`xfade_opencl` for long renders.
|
|
144
|
+
|
|
145
|
+
## Target-filesize social presets (HandBrake idea)
|
|
146
|
+
|
|
147
|
+
Beyond aspect, encode to **fit under a platform's upload cap**: pick a two-pass video
|
|
148
|
+
bitrate from the cap and duration - `bitrate_bits = size_bytes * 8 / duration - audio`
|
|
149
|
+
- e.g. "25 MB / 30 s / 1080p". A named preset bundles `(upload cap, max duration) ->
|
|
150
|
+
resolution + bitrate`. (HandBrake ships exactly these as "Social 25 MB 30 Seconds"
|
|
151
|
+
presets.)
|
|
@@ -119,3 +119,26 @@ nannou - licenses in the table above. Capture pipeline: see
|
|
|
119
119
|
prompting, ControlNet/IP-Adapter conditioning), see
|
|
120
120
|
[AI media generation](ai-media-generation.md) and
|
|
121
121
|
[design editor patterns](design-editor-patterns.md).
|
|
122
|
+
|
|
123
|
+
### Headless capture + 3D export (from full-source study)
|
|
124
|
+
|
|
125
|
+
How to render these engines **without a screen**, and extra output formats worth
|
|
126
|
+
knowing:
|
|
127
|
+
|
|
128
|
+
- **Babylon.js `NullEngine`** (Apache-2.0) - first-class **server-side headless**
|
|
129
|
+
rendering (no GPU/canvas); the cleanest headless story for render-automation jobs,
|
|
130
|
+
plus a built-in **procedural-texture library** (brick/cloud/fire/marble/noise/wood),
|
|
131
|
+
`CreateScreenshotUsingRenderTarget` for 4K+ stills from a small viewport, and
|
|
132
|
+
**NodeMaterial** (shaders serialize to JSON - an LLM-emittable look). This is the
|
|
133
|
+
concrete reason to pick Babylon over three for automation.
|
|
134
|
+
- **three.js** (MIT) - `readRenderTargetPixels()`/`...Async` read pixels from an
|
|
135
|
+
offscreen `WebGLRenderTarget` (true headless high-res, not on-screen canvas);
|
|
136
|
+
**SVGRenderer** emits a 3D scene as editable SVG; **USDZExporter** turns a 3D product
|
|
137
|
+
into a free **iOS AR Quick Look** asset; **GPUComputationRenderer** (ping-pong FBO
|
|
138
|
+
sim: flocking/fluid) = a living, seeded generative hero background.
|
|
139
|
+
- **openFrameworks** (MIT) - `ofFbo` offscreen framebuffer for arbitrary-res stills;
|
|
140
|
+
**ofxEmscripten** compiles a desktop sketch to WebAssembly for the browser;
|
|
141
|
+
**ofxSvg** parses + emits structured SVG.
|
|
142
|
+
- **Processing** - `PSurfaceNone` (headless) and `createGraphics(big)` (render larger
|
|
143
|
+
than the screen -> high-res still). License at feature granularity: core raster is
|
|
144
|
+
LGPL, but **SVG/PDF export is in non-core (GPL)** - reimplement, don't vendor.
|
|
@@ -121,3 +121,16 @@ non-destructive edit recipe): `grayscale` (`-colorspace Gray`), `sepia`
|
|
|
121
121
|
`maggie_image.py composite` and `maggie_image.py look` generate these. For a
|
|
122
122
|
reusable, batchable edit *recipe* (a JSON stack of named ops applied to many photos),
|
|
123
123
|
see [design editor patterns](design-editor-patterns.md).
|
|
124
|
+
|
|
125
|
+
## More ImageMagick ops worth baking in (confirmed in source)
|
|
126
|
+
|
|
127
|
+
- **`-auto-orient`** first in every pipeline - fixes sideways phone photos by EXIF
|
|
128
|
+
(`maggie_image.py` resize/social/trim already prepend it).
|
|
129
|
+
- **`-thumbnail WxH`** - metadata-stripping fast resize; a better default than
|
|
130
|
+
`-resize` for web/OG thumbnails.
|
|
131
|
+
- **`-fuzz N% -trim +repage`** - trim a uniform border or cut a product/logo out of
|
|
132
|
+
its background (`maggie_image.py trim`).
|
|
133
|
+
- **`-clut` / `-hald-clut`** - repeatable LUT brand grading (raster equivalent of
|
|
134
|
+
ffmpeg `lut3d`).
|
|
135
|
+
- **`-distort perspective '<src coords> <dst coords>'`** - place a screenshot into a
|
|
136
|
+
device mockup.
|
|
@@ -78,3 +78,25 @@ apply.
|
|
|
78
78
|
`libx264 -crf 14-20 -preset slow/medium -pix_fmt yuv420p -movflags +faststart`,
|
|
79
79
|
even dimensions, plus 960px / 800x500 GIF deliverables. For transparent output,
|
|
80
80
|
Revideo uses per-format pixel formats (`yuva420p` WebM, `yuva444p10le` ProRes).
|
|
81
|
+
|
|
82
|
+
## Remotion's real renderer (from source) and the ideas to borrow
|
|
83
|
+
|
|
84
|
+
Source study (Remotion License - concept only, no vendoring) shows Remotion is **not**
|
|
85
|
+
the base64-PNG pipe we lump with Revideo/Motion Canvas: it drives headless Chromium
|
|
86
|
+
over CDP, captures frames via **shared memory**, and encodes with a **per-platform
|
|
87
|
+
native Rust sidecar that statically links FFmpeg** (a long-running streaming process),
|
|
88
|
+
plus a bundled `ffmpeg` for the final stitch. Borrowable ideas (improve our Part C):
|
|
89
|
+
|
|
90
|
+
- **A readiness gate, not just a fake clock.** `delayRender()`/`continueRender()`: a
|
|
91
|
+
component declares async work pending and the renderer blocks the screenshot until
|
|
92
|
+
all handles clear, reporting **labeled** open handles on timeout. Add an explicit
|
|
93
|
+
readiness handshake (plus a per-frame `await document.fonts.ready`) to Part C so no
|
|
94
|
+
frame is captured before fonts/data/images are ready - prevents FOUT/half-loaded
|
|
95
|
+
frames.
|
|
96
|
+
- **Tab-pool concurrency**: open N browser pages, each seeking a different frame in
|
|
97
|
+
parallel (frame range / N).
|
|
98
|
+
- **Chunked parallel encode + seamless concat**: split the frame range, encode chunks
|
|
99
|
+
independently, concat without re-encode - throughput for long renders.
|
|
100
|
+
- **Kept-open, LRU-cached source-video decoders**: for embedded video assets, keep the
|
|
101
|
+
decoder open and cache frames so repeated seeks to a timestamp are cheap (our flat
|
|
102
|
+
timeline currently punts this to raw ffmpeg `-ss`).
|
|
@@ -161,10 +161,28 @@ GPL = concept only):
|
|
|
161
161
|
`blendMode` values on overlay clips: Porter-Duff (`Over, Under, In, Out, ATop, Copy,
|
|
162
162
|
Matte`) + blends (`Multiply, Screen, Overlay, SoftLight, HardLight, ColorBurn,
|
|
163
163
|
ColorDodge, Difference, Exclusion, Hue, Saturation, Color, Luminosity`).
|
|
164
|
-
- **Xsheet tabular timing** (opentoonz/tahoma2d, BSD
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
164
|
+
- **Xsheet tabular timing** (opentoonz/tahoma2d, **BSD - code reusable with
|
|
165
|
+
attribution, a direct port target**). An optional rows=frames / columns=assets
|
|
166
|
+
exposure grid (cells = a drawing exposed for N frames; a *hold* is the same cell
|
|
167
|
+
repeated). The source implements this (and more of our model than we first credited)
|
|
168
|
+
in `TXsheet` / `TXshCellColumn` (`std::vector<TXshCell> m_cells`) / `TXshCell`
|
|
169
|
+
(`{level, TFrameId}`) / `TFrameId` (`{frame, letter}`), plus the cell-op vocabulary
|
|
170
|
+
(`reverseCells, swingCells, stepCells, reframeCells, timeStretch`). It also backs, as
|
|
171
|
+
BSD code: the **decomposed transform channels** (`TStageObject::Channel` T_X/T_Y/
|
|
172
|
+
T_Angle/T_ScaleX/... each a keyframe curve, with parenting + `T_Path` follow-spline),
|
|
173
|
+
a **richer easing enum** (`TDoubleKeyframe::Type` = Constant / Linear / SpeedInOut /
|
|
174
|
+
EaseInOut / EaseInOutPercentage / Exponential / Expression / File), **nested
|
|
175
|
+
sub-xsheets** (`TXshChildLevel` = precomp), and a **node-graph fx schematic**
|
|
176
|
+
(`FxDag`/`TFx`). So the easing/transform/nested-clip/effect-graph extensions above
|
|
177
|
+
have a BSD implementation to port, not only GPL concept sources.
|
|
178
|
+
- **Strip/effect model + full blend set** (blender VSE `Strip`, concept). The Video
|
|
179
|
+
Sequence Editor's `Strip` (input1/input2, effect strips CROSS/WIPE/GLOW/SPEED/
|
|
180
|
+
ADJUSTMENT/TEXT, a Photoshop-complete `StripBlendMode`) and **F-curve modifier stack**
|
|
181
|
+
(noise/cycles/limits/stepped) + **drivers** (property-driven channels) are strong
|
|
182
|
+
concept borrows for overlay/effect clips.
|
|
183
|
+
- **OTIO interchange** (kdenlive, concept). Document a mapping to OpenTimelineIO (the
|
|
184
|
+
ASWF-standard timeline interchange): gap=blank, stack=track, externalReference=asset
|
|
185
|
+
src - so the model can round-trip with real NLEs.
|
|
168
186
|
- **Optional node-graph "effect graph"** (Natron, concept) for branched layered effects
|
|
169
187
|
(mask -> blur -> merge) that a linear track can't express - heavy; a documented
|
|
170
188
|
escape hatch, not the default.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
name: maggie-image
|
|
3
3
|
description: Process marketing images - resize and fit to channel/OG sizes, convert to WebP/AVIF, make favicon and OG-image sets, strip metadata, compress, optimize and render SVG, and build contact-sheet montages. A recipe generator over ImageMagick (primary) with dedicated tools (svgo/resvg/cwebp/avifenc/pngquant) used when installed. Use when the user wants to resize, convert, optimize, favicon, watermark, or batch-process images for a site or social post.
|
|
4
4
|
metadata:
|
|
5
|
-
version: 0.1.
|
|
5
|
+
version: 0.1.3
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# Maggie Image
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
name: maggie-video
|
|
3
3
|
description: Produce and process marketing video - render a composition to MP4/GIF/WebM, assemble clips/images/text/captions/audio on a timeline, cut to channel aspects (reel/square/youtube), burn in captions, normalize loudness, watermark, transcode, and make social cuts. Engine-neutral over the renderer (ffmpeg for assembly/transcode, a programmatic engine for code-drawn animation). Use when the user wants to make, edit, cut, caption, resize, or export a video for marketing.
|
|
4
4
|
metadata:
|
|
5
|
-
version: 0.1.
|
|
5
|
+
version: 0.1.5
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
# Maggie Video
|
|
@@ -59,18 +59,25 @@ def _resize_ops(w, h, mode, pad):
|
|
|
59
59
|
|
|
60
60
|
def c_resize(a):
|
|
61
61
|
w, h = a.size.lower().split("x")
|
|
62
|
-
cmd = _im() + [a.input, "-strip"] + _resize_ops(w, h, a.mode, a.pad) + \
|
|
62
|
+
cmd = _im() + [a.input, "-auto-orient", "-strip"] + _resize_ops(w, h, a.mode, a.pad) + \
|
|
63
63
|
["-quality", str(a.quality), a.output]
|
|
64
64
|
return emit(cmd, a.run, _im_tool())
|
|
65
65
|
|
|
66
66
|
|
|
67
67
|
def c_social(a):
|
|
68
68
|
w, h = SOCIAL[a.preset]
|
|
69
|
-
cmd = _im() + [a.input, "-strip"] + _resize_ops(w, h, a.mode, a.pad) + \
|
|
69
|
+
cmd = _im() + [a.input, "-auto-orient", "-strip"] + _resize_ops(w, h, a.mode, a.pad) + \
|
|
70
70
|
["-quality", str(a.quality), a.output]
|
|
71
71
|
return emit(cmd, a.run, _im_tool())
|
|
72
72
|
|
|
73
73
|
|
|
74
|
+
def c_trim(a):
|
|
75
|
+
"""Trim uniform border / cut a product or logo out of its background
|
|
76
|
+
(ImageMagick -fuzz -trim; +repage resets the virtual canvas)."""
|
|
77
|
+
cmd = _im() + [a.input, "-auto-orient", "-fuzz", f"{a.fuzz}%", "-trim", "+repage", a.output]
|
|
78
|
+
return emit(cmd, a.run, _im_tool())
|
|
79
|
+
|
|
80
|
+
|
|
74
81
|
def c_webp(a):
|
|
75
82
|
if shutil.which("cwebp") and not a.force_im:
|
|
76
83
|
return emit(["cwebp", "-q", str(a.quality), "-m", "6", a.input, "-o", a.output], a.run, "cwebp")
|
|
@@ -253,6 +260,9 @@ def main(argv=None):
|
|
|
253
260
|
fv.add_argument("--input", required=True); fv.add_argument("--out-dir", dest="out_dir", required=True)
|
|
254
261
|
fv.set_defaults(fn=c_favicon)
|
|
255
262
|
|
|
263
|
+
tr = sub.add_parser("trim", help="trim border / cut product out of its background")
|
|
264
|
+
io(tr); tr.add_argument("--fuzz", type=int, default=5, help="color tolerance %%"); tr.set_defaults(fn=c_trim)
|
|
265
|
+
|
|
256
266
|
st = sub.add_parser("strip", help="strip metadata (EXIF/GPS/ICC)"); io(st); st.set_defaults(fn=c_strip)
|
|
257
267
|
co = sub.add_parser("compress", help="compress JPEG/PNG"); io(co)
|
|
258
268
|
co.add_argument("--quality", type=int, default=82); co.set_defaults(fn=c_compress)
|
|
@@ -511,17 +511,20 @@ def c_captions(a):
|
|
|
511
511
|
cmd = [tool, "-m", a.model, "-f", a.input, "-osrt", "-of",
|
|
512
512
|
os.path.join(out_dir, base)]
|
|
513
513
|
print(f"# whisper.cpp writes {os.path.join(out_dir, base)}.srt; --model is the .bin path")
|
|
514
|
-
|
|
515
|
-
tool = "whisperx"
|
|
516
|
-
cmd = [tool, a.input, "--model", a.model, "--output_format",
|
|
514
|
+
else: # openai-whisper (default) / whisperx - same flag surface
|
|
515
|
+
tool = "whisperx" if a.engine == "whisperx" else "whisper"
|
|
516
|
+
cmd = [tool, a.input, "--model", a.model, "--output_format", a.format,
|
|
517
517
|
"--output_dir", out_dir] + lang
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
518
|
+
if a.word_timestamps:
|
|
519
|
+
cmd += ["--word_timestamps", "True"]
|
|
520
|
+
if a.highlight_words:
|
|
521
|
+
cmd += ["--highlight_words", "True"] # karaoke-style per-word highlight
|
|
522
|
+
if a.initial_prompt:
|
|
523
|
+
cmd += ["--initial_prompt", a.initial_prompt] # bias brand/product spellings
|
|
524
|
+
print(f"# {tool} writes {os.path.join(out_dir, base)}.{a.format} in --output_dir; "
|
|
525
|
+
f"openai-whisper's default model is now 'turbo' (large-v3-turbo).")
|
|
526
|
+
print("# transcription is local/offline (no paid API); a larger model is slower but better.")
|
|
527
|
+
print("# --highlight-words gives karaoke captions; --initial-prompt fixes brand-term spelling.")
|
|
525
528
|
print("# then: maggie_video.py subtitle --input video.mp4 --srt <that>.srt --output out.mp4 (burn in)")
|
|
526
529
|
return emit(cmd, a.run, tool=tool)
|
|
527
530
|
|
|
@@ -602,11 +605,16 @@ def main(argv=None):
|
|
|
602
605
|
ac.add_argument("--margin", type=float, default=0.1, help="padding kept around speech (s)")
|
|
603
606
|
ac.set_defaults(fn=c_autocut)
|
|
604
607
|
|
|
605
|
-
ca = sub.add_parser("captions", help="transcribe speech to SRT (local whisper)")
|
|
606
|
-
ca.add_argument("--input", required=True); ca.add_argument("--output", required=True, help="target
|
|
608
|
+
ca = sub.add_parser("captions", help="transcribe speech to SRT/VTT (local whisper)")
|
|
609
|
+
ca.add_argument("--input", required=True); ca.add_argument("--output", required=True, help="target caption file")
|
|
607
610
|
ca.add_argument("--engine", choices=["whisper", "whispercpp", "whisperx"], default="whisper")
|
|
608
|
-
ca.add_argument("--model", default="
|
|
609
|
-
ca.add_argument("--
|
|
611
|
+
ca.add_argument("--model", default="turbo", help="model (openai-whisper default 'turbo'); .bin path for whispercpp")
|
|
612
|
+
ca.add_argument("--format", default="srt", help="srt|vtt|txt|tsv|json (openai-whisper/whisperx)")
|
|
613
|
+
ca.add_argument("--lang", default="auto")
|
|
614
|
+
ca.add_argument("--word-timestamps", dest="word_timestamps", action="store_true")
|
|
615
|
+
ca.add_argument("--highlight-words", dest="highlight_words", action="store_true", help="karaoke per-word highlight")
|
|
616
|
+
ca.add_argument("--initial-prompt", dest="initial_prompt", help="bias brand/product term spelling")
|
|
617
|
+
ca.set_defaults(fn=c_captions)
|
|
610
618
|
|
|
611
619
|
sub.add_parser("presets", help="list channel aspect presets").set_defaults(fn=c_presets)
|
|
612
620
|
|
package/package.json
CHANGED
|
@@ -9,7 +9,7 @@ Distilled from the reference library's R-category projects.
|
|
|
9
9
|
|
|
10
10
|
| Engine | What / how | License | Reach for it when |
|
|
11
11
|
|---|---|---|---|
|
|
12
|
-
| openai-whisper |
|
|
12
|
+
| openai-whisper | `whisper IN --model turbo --output_format srt --output_dir DIR` (default model is now **`turbo`** = large-v3-turbo; formats `txt/vtt/srt/tsv/json`) | MIT | the default; simplest install (`pip install openai-whisper`) |
|
|
13
13
|
| whisper.cpp | C/C++ port, fast on CPU, GGML models: `whisper-cli -m model.bin -f IN -osrt -of OUTBASE` | MIT | no GPU / fastest CPU / embedding in a pipeline |
|
|
14
14
|
| whisperX | whisper + forced alignment for **word-level timestamps** and diarization: `whisperx IN --model large-v3 --output_format srt --output_dir DIR` | BSD-4 | word-accurate caption timing or speaker labels |
|
|
15
15
|
|
|
@@ -38,4 +38,24 @@ Model size trades speed for accuracy: `tiny` / `base` (fast, rough) -> `small` /
|
|
|
38
38
|
API is ever wired in instead, the global spend-first rule applies - state the cost.
|
|
39
39
|
- Keep caption text inside the safe area (see `maggie-video` channel presets).
|
|
40
40
|
|
|
41
|
+
## Caption-shaping flags (built into openai-whisper)
|
|
42
|
+
|
|
43
|
+
The core CLI already does what we previously routed to whisperX or post-processing
|
|
44
|
+
(`maggie_video.py captions` exposes these):
|
|
45
|
+
|
|
46
|
+
- `--word_timestamps True` - word-level timings (whisperX still wins on alignment +
|
|
47
|
+
diarization, but core whisper now has them).
|
|
48
|
+
- `--highlight_words True` - **karaoke-style** per-word highlight in SRT/VTT (the
|
|
49
|
+
social-caption effect); render the ASS with ffmpeg `subtitles`.
|
|
50
|
+
- `--max_line_width` / `--max_line_count` / `--max_words_per_line` - native line
|
|
51
|
+
wrapping so captions fit the channel safe-area, no post-processing.
|
|
52
|
+
- `--initial_prompt "<brand, product names>"` - bias the model toward correct brand/
|
|
53
|
+
jargon spelling (the concrete mitigation for the mishears-brand-terms risk; still
|
|
54
|
+
human-review).
|
|
55
|
+
- Anti-hallucination over music/silence: `--hallucination_silence_threshold`,
|
|
56
|
+
`--compression_ratio_threshold`, `--logprob_threshold`, `--no_speech_threshold` -
|
|
57
|
+
worth setting for music-bed marketing clips.
|
|
58
|
+
- `--task translate` (foreign audio -> English subs), `--clip_timestamps` (only given
|
|
59
|
+
ranges).
|
|
60
|
+
|
|
41
61
|
See `maggie-video` for the subcommands and the burn-in / soft-mux / compile options.
|
|
@@ -15,8 +15,12 @@ the image is reconstructed by replaying the stack. Borrow this as a maggie "imag
|
|
|
15
15
|
recipe": a small JSON `[ {op, params}, ... ]` stored next to the original and replayed
|
|
16
16
|
by `maggie_image.py` (each op -> an ImageMagick/sharp command). It is **reproducible,
|
|
17
17
|
auditable, re-editable, and batchable** - apply one "look" recipe to every photo in a
|
|
18
|
-
campaign for consistent brand grading. `darktable-cli <
|
|
19
|
-
from such a stack (
|
|
18
|
+
campaign for consistent brand grading. `darktable-cli <input> [sidecar.xmp]
|
|
19
|
+
<output-dir>` renders from such a stack (XMP optional/auto-discovered, final arg is a
|
|
20
|
+
**directory**; GPL binary, arm's-length). Two source details worth adopting: **params
|
|
21
|
+
versioning** (each module keeps `legacy_params` for forward-compatible migration -
|
|
22
|
+
version your recipe ops) and **"Styles" = saved reusable history** (name a look once,
|
|
23
|
+
apply to many).
|
|
20
24
|
|
|
21
25
|
## LLM-generatable scene / diagram formats
|
|
22
26
|
|
|
@@ -75,3 +79,26 @@ only: penpot (MPL), tldraw (custom), krita/krita-ai-diffusion (GPL), drawio icon
|
|
|
75
79
|
MIT and freely borrowable as ideas (still reimplement, don't bulk-copy): excalidraw,
|
|
76
80
|
Method-Draw, rough.js, Pinta. drawio code Apache-2.0; its diagram output is
|
|
77
81
|
unrestricted.
|
|
82
|
+
|
|
83
|
+
## From full-source study (net-new)
|
|
84
|
+
|
|
85
|
+
- **tldraw** - the LLM mutation surface is `editor.createShapes/updateShapes/
|
|
86
|
+
deleteShapes` (batch partial mutations); the store ships schema **migration
|
|
87
|
+
sequences**, a composable **validation layer**, and a **side-effects manager**
|
|
88
|
+
(before/after create/change/delete hooks). `parentId` can be a page *or* a shape;
|
|
89
|
+
pages carry a fractional `index`. Adopt the batch-mutation + validate + migrate
|
|
90
|
+
shape as the model for an agent-written canvas doc.
|
|
91
|
+
- **penpot** - **design tokens are W3C DTCG** (import/export + aliasing), components
|
|
92
|
+
have **variants**, shapes carry **flex/grid layout**; the agent boundary is an
|
|
93
|
+
**official MCP server talking to a sandboxed in-app plugin over WebSocket** - a
|
|
94
|
+
strong arm's-length model for making a design doc agent-writable (mirrors Maggie's
|
|
95
|
+
own MCP posture).
|
|
96
|
+
- **drawio** - the `mxConstants` STYLE_* key table is a **validated allow-list** for
|
|
97
|
+
LLM-emitted styles; `mxEdgeStyle` routing + hierarchical/tree/organic **auto-layout**
|
|
98
|
+
mean a generated node+edge list can self-position; `.drawio` pages are often
|
|
99
|
+
**deflate+base64** compressed.
|
|
100
|
+
- **krita** - an **adjustment/filter layer holds a serialized filter config** = a
|
|
101
|
+
non-destructive recipe living in the layer stack; ops are **mask/selection-scoped**;
|
|
102
|
+
`.kra` is `zip(manifest XML + per-layer blobs)`; a **Python scripting API** and a
|
|
103
|
+
JSON-manifest plugin boundary let a generative backend register like any filter -
|
|
104
|
+
the "AI as a swappable backend" posture.
|
|
@@ -128,3 +128,24 @@ maggie tool maggie_video.py compile --spec timeline.json # declarative timelin
|
|
|
128
128
|
maggie tool maggie_video.py --run encode --input in.mov --output out.mp4 # actually runs
|
|
129
129
|
maggie tool maggie_video.py presets
|
|
130
130
|
```
|
|
131
|
+
|
|
132
|
+
## More filters worth reaching for (confirmed in libavfilter)
|
|
133
|
+
|
|
134
|
+
- **Green-screen / keying** (UGC, product on a brand background):
|
|
135
|
+
`chromakey=0x00FF00:0.1:0.0` (or `colorkey`), clean fringes with `despill`.
|
|
136
|
+
- **Fade to/from black** (distinct from clip crossfade): `fade=t=in:st=0:d=0.5`,
|
|
137
|
+
`fade=t=out:st=9.5:d=0.5`.
|
|
138
|
+
- **Brand color grade / LUT**: `lut3d=brand.cube`, or `curves`/`eq`/`colorbalance`/`hue`.
|
|
139
|
+
- **Karaoke captions**: `subtitles=in.ass` with a full ASS file (word highlight) -
|
|
140
|
+
pairs with whisper `--highlight_words` ([captions](captions-transcription.md)).
|
|
141
|
+
- **Focus/credits**: `gblur` + `vignette`; `scroll` for scrolling outros.
|
|
142
|
+
- **GPU fast path** when hardware is present: `overlay_cuda`/`overlay_vulkan`,
|
|
143
|
+
`xfade_opencl` for long renders.
|
|
144
|
+
|
|
145
|
+
## Target-filesize social presets (HandBrake idea)
|
|
146
|
+
|
|
147
|
+
Beyond aspect, encode to **fit under a platform's upload cap**: pick a two-pass video
|
|
148
|
+
bitrate from the cap and duration - `bitrate_bits = size_bytes * 8 / duration - audio`
|
|
149
|
+
- e.g. "25 MB / 30 s / 1080p". A named preset bundles `(upload cap, max duration) ->
|
|
150
|
+
resolution + bitrate`. (HandBrake ships exactly these as "Social 25 MB 30 Seconds"
|
|
151
|
+
presets.)
|
|
@@ -119,3 +119,26 @@ nannou - licenses in the table above. Capture pipeline: see
|
|
|
119
119
|
prompting, ControlNet/IP-Adapter conditioning), see
|
|
120
120
|
[AI media generation](ai-media-generation.md) and
|
|
121
121
|
[design editor patterns](design-editor-patterns.md).
|
|
122
|
+
|
|
123
|
+
### Headless capture + 3D export (from full-source study)
|
|
124
|
+
|
|
125
|
+
How to render these engines **without a screen**, and extra output formats worth
|
|
126
|
+
knowing:
|
|
127
|
+
|
|
128
|
+
- **Babylon.js `NullEngine`** (Apache-2.0) - first-class **server-side headless**
|
|
129
|
+
rendering (no GPU/canvas); the cleanest headless story for render-automation jobs,
|
|
130
|
+
plus a built-in **procedural-texture library** (brick/cloud/fire/marble/noise/wood),
|
|
131
|
+
`CreateScreenshotUsingRenderTarget` for 4K+ stills from a small viewport, and
|
|
132
|
+
**NodeMaterial** (shaders serialize to JSON - an LLM-emittable look). This is the
|
|
133
|
+
concrete reason to pick Babylon over three for automation.
|
|
134
|
+
- **three.js** (MIT) - `readRenderTargetPixels()`/`...Async` read pixels from an
|
|
135
|
+
offscreen `WebGLRenderTarget` (true headless high-res, not on-screen canvas);
|
|
136
|
+
**SVGRenderer** emits a 3D scene as editable SVG; **USDZExporter** turns a 3D product
|
|
137
|
+
into a free **iOS AR Quick Look** asset; **GPUComputationRenderer** (ping-pong FBO
|
|
138
|
+
sim: flocking/fluid) = a living, seeded generative hero background.
|
|
139
|
+
- **openFrameworks** (MIT) - `ofFbo` offscreen framebuffer for arbitrary-res stills;
|
|
140
|
+
**ofxEmscripten** compiles a desktop sketch to WebAssembly for the browser;
|
|
141
|
+
**ofxSvg** parses + emits structured SVG.
|
|
142
|
+
- **Processing** - `PSurfaceNone` (headless) and `createGraphics(big)` (render larger
|
|
143
|
+
than the screen -> high-res still). License at feature granularity: core raster is
|
|
144
|
+
LGPL, but **SVG/PDF export is in non-core (GPL)** - reimplement, don't vendor.
|
|
@@ -121,3 +121,16 @@ non-destructive edit recipe): `grayscale` (`-colorspace Gray`), `sepia`
|
|
|
121
121
|
`maggie_image.py composite` and `maggie_image.py look` generate these. For a
|
|
122
122
|
reusable, batchable edit *recipe* (a JSON stack of named ops applied to many photos),
|
|
123
123
|
see [design editor patterns](design-editor-patterns.md).
|
|
124
|
+
|
|
125
|
+
## More ImageMagick ops worth baking in (confirmed in source)
|
|
126
|
+
|
|
127
|
+
- **`-auto-orient`** first in every pipeline - fixes sideways phone photos by EXIF
|
|
128
|
+
(`maggie_image.py` resize/social/trim already prepend it).
|
|
129
|
+
- **`-thumbnail WxH`** - metadata-stripping fast resize; a better default than
|
|
130
|
+
`-resize` for web/OG thumbnails.
|
|
131
|
+
- **`-fuzz N% -trim +repage`** - trim a uniform border or cut a product/logo out of
|
|
132
|
+
its background (`maggie_image.py trim`).
|
|
133
|
+
- **`-clut` / `-hald-clut`** - repeatable LUT brand grading (raster equivalent of
|
|
134
|
+
ffmpeg `lut3d`).
|
|
135
|
+
- **`-distort perspective '<src coords> <dst coords>'`** - place a screenshot into a
|
|
136
|
+
device mockup.
|
|
@@ -78,3 +78,25 @@ apply.
|
|
|
78
78
|
`libx264 -crf 14-20 -preset slow/medium -pix_fmt yuv420p -movflags +faststart`,
|
|
79
79
|
even dimensions, plus 960px / 800x500 GIF deliverables. For transparent output,
|
|
80
80
|
Revideo uses per-format pixel formats (`yuva420p` WebM, `yuva444p10le` ProRes).
|
|
81
|
+
|
|
82
|
+
## Remotion's real renderer (from source) and the ideas to borrow
|
|
83
|
+
|
|
84
|
+
Source study (Remotion License - concept only, no vendoring) shows Remotion is **not**
|
|
85
|
+
the base64-PNG pipe we lump with Revideo/Motion Canvas: it drives headless Chromium
|
|
86
|
+
over CDP, captures frames via **shared memory**, and encodes with a **per-platform
|
|
87
|
+
native Rust sidecar that statically links FFmpeg** (a long-running streaming process),
|
|
88
|
+
plus a bundled `ffmpeg` for the final stitch. Borrowable ideas (improve our Part C):
|
|
89
|
+
|
|
90
|
+
- **A readiness gate, not just a fake clock.** `delayRender()`/`continueRender()`: a
|
|
91
|
+
component declares async work pending and the renderer blocks the screenshot until
|
|
92
|
+
all handles clear, reporting **labeled** open handles on timeout. Add an explicit
|
|
93
|
+
readiness handshake (plus a per-frame `await document.fonts.ready`) to Part C so no
|
|
94
|
+
frame is captured before fonts/data/images are ready - prevents FOUT/half-loaded
|
|
95
|
+
frames.
|
|
96
|
+
- **Tab-pool concurrency**: open N browser pages, each seeking a different frame in
|
|
97
|
+
parallel (frame range / N).
|
|
98
|
+
- **Chunked parallel encode + seamless concat**: split the frame range, encode chunks
|
|
99
|
+
independently, concat without re-encode - throughput for long renders.
|
|
100
|
+
- **Kept-open, LRU-cached source-video decoders**: for embedded video assets, keep the
|
|
101
|
+
decoder open and cache frames so repeated seeks to a timestamp are cheap (our flat
|
|
102
|
+
timeline currently punts this to raw ffmpeg `-ss`).
|
|
@@ -161,10 +161,28 @@ GPL = concept only):
|
|
|
161
161
|
`blendMode` values on overlay clips: Porter-Duff (`Over, Under, In, Out, ATop, Copy,
|
|
162
162
|
Matte`) + blends (`Multiply, Screen, Overlay, SoftLight, HardLight, ColorBurn,
|
|
163
163
|
ColorDodge, Difference, Exclusion, Hue, Saturation, Color, Luminosity`).
|
|
164
|
-
- **Xsheet tabular timing** (opentoonz/tahoma2d, BSD
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
164
|
+
- **Xsheet tabular timing** (opentoonz/tahoma2d, **BSD - code reusable with
|
|
165
|
+
attribution, a direct port target**). An optional rows=frames / columns=assets
|
|
166
|
+
exposure grid (cells = a drawing exposed for N frames; a *hold* is the same cell
|
|
167
|
+
repeated). The source implements this (and more of our model than we first credited)
|
|
168
|
+
in `TXsheet` / `TXshCellColumn` (`std::vector<TXshCell> m_cells`) / `TXshCell`
|
|
169
|
+
(`{level, TFrameId}`) / `TFrameId` (`{frame, letter}`), plus the cell-op vocabulary
|
|
170
|
+
(`reverseCells, swingCells, stepCells, reframeCells, timeStretch`). It also backs, as
|
|
171
|
+
BSD code: the **decomposed transform channels** (`TStageObject::Channel` T_X/T_Y/
|
|
172
|
+
T_Angle/T_ScaleX/... each a keyframe curve, with parenting + `T_Path` follow-spline),
|
|
173
|
+
a **richer easing enum** (`TDoubleKeyframe::Type` = Constant / Linear / SpeedInOut /
|
|
174
|
+
EaseInOut / EaseInOutPercentage / Exponential / Expression / File), **nested
|
|
175
|
+
sub-xsheets** (`TXshChildLevel` = precomp), and a **node-graph fx schematic**
|
|
176
|
+
(`FxDag`/`TFx`). So the easing/transform/nested-clip/effect-graph extensions above
|
|
177
|
+
have a BSD implementation to port, not only GPL concept sources.
|
|
178
|
+
- **Strip/effect model + full blend set** (blender VSE `Strip`, concept). The Video
|
|
179
|
+
Sequence Editor's `Strip` (input1/input2, effect strips CROSS/WIPE/GLOW/SPEED/
|
|
180
|
+
ADJUSTMENT/TEXT, a Photoshop-complete `StripBlendMode`) and **F-curve modifier stack**
|
|
181
|
+
(noise/cycles/limits/stepped) + **drivers** (property-driven channels) are strong
|
|
182
|
+
concept borrows for overlay/effect clips.
|
|
183
|
+
- **OTIO interchange** (kdenlive, concept). Document a mapping to OpenTimelineIO (the
|
|
184
|
+
ASWF-standard timeline interchange): gap=blank, stack=track, externalReference=asset
|
|
185
|
+
src - so the model can round-trip with real NLEs.
|
|
168
186
|
- **Optional node-graph "effect graph"** (Natron, concept) for branched layered effects
|
|
169
187
|
(mask -> blur -> merge) that a linear track can't express - heavy; a documented
|
|
170
188
|
escape hatch, not the default.
|