@lemoncat7/dsh-partner 2.8.1 → 2.8.3

package/resources/skills/video-recap-production/upstream/skills/video-recap/references/audio-routing.md

@@ -0,0 +1,126 @@
+# Recap audio routing
+
+`--edit-mode` selects the picture workflow; `--audio-mode` independently
+selects the final audio workflow:
+
+| Audio mode | Narration validation/review | TTS | Assemble input |
+| --- | --- | --- | --- |
+| `narration` (default) | Runs as before | Runs as before | source or rendered cut |
+| `source-mix` | Not applicable | Not run | current source or rendered cut |
+| `adopted-packet-copy` | Not applicable | Not run | current source or rendered cut |
+
+`full` with either non-narration mode goes directly to assemble and does not
+implicitly run understanding. `cut` still requires the existing `clip_plan`:
+the first run analyzes and pauses for that plan, then later runs the established
+single- or multi-source cut renderer. It does not pause for `narration.json`.
+
+## Local adopted full-sound assembly
+
+The strict local-adoption route accepts one prebuilt picture in `full` mode and
+exactly this all-or-none bundle:
+
+```text
+--tts-meta PATH --narration-adoption PATH --audio-mix-adoption PATH
+```
+
+It requires narration mode, stream 0, a new explicit `--work-dir`, and a delivery path
+that does not already exist. `--output-dir` is optional; when omitted, delivery uses the
+new work directory's parent. The route calls only the video-assemble CLI; it does
+not run understanding, script validation, narration review, voiceover, MiMo QC, cut,
+continuation, editor export, or material-cache reuse. Ambient TTS provider and voice
+configuration are inert. Explicit TTS/voice/review/MiMo/editor flags are rejected rather
+than silently ignored.
+
+`recap_run_manifest.json` records the resolved path of all three local artifacts under
+`audio.local_adoption`, and the picture's path plus `{size, mtime_ns}` as
+`source_video_identity`. Analysis settings remain separate and unchanged. Recap validates
+routing and, after the child run, checks that the assembler's binding records reference
+the same adoption files and picture; video-assemble remains authoritative for the adoption
+schemas, PCM shapes, frame clock, mix, bindings, and final media transaction. This path
+executes caller-adopted assets and does not claim that recap authored or approved their
+story, voice, or mix.
+
+## Keep adopted voice after a cut
+
+The single input above is the **finished picture**, which may contain one or many source
+episodes. Use two existing stages rather than sending an old mix binding into a new cut:
+
+1. Finish the selected cut with `video-cut`: either run `recap.py ... --edit-mode cut`,
+   or call that skill's own `scripts/cut.py` from its installed directory, with an
+   existing clip plan and source manifest:
+
+   ```bash
+   python3 <video-cut>/scripts/cut.py ep1.mp4 --work-dir CUT_WORK \
+     --sources-manifest SOURCES_JSON
+   ```
+
+   A single-source cut omits `--sources-manifest`. For already locked frame decisions,
+   use the picture-plan path instead. Read `clip_plan_validated.json` (or the locked path's
+   `picture_map.json`) before placing sound.
+2. Keep the selected WAVs, `tts_meta.json`, and `narration_adoption.json` when the words,
+   WAV bytes, and selected voice are unchanged. Do not run voiceover just to move a line.
+   In this explicit mix, new `output_start_sample` values own placement; old generation
+   windows in `tts_meta.json` are not the revised output timeline.
+3. Match source dialogue, ambience and music to the new picture. Reuse the prepared bed
+   only if that sound and its complete sample clock remain appropriate; otherwise rebuild
+   it with the existing `video-assemble` source/score producer. Write a **new** mix adoption
+   with the prepared receipt path, total samples and complete WAV placements for the new
+   picture. Reordering shots can require moving dialogue too; a mix adoption written for
+   the old picture cannot simply be pointed at the new one.
+4. Run the local adopted `full` command in `SKILL.md` with `CUT_WORK/edited_source.mp4` and a new
+   assembly work/delivery directory. Inspect the emitted narration placements and subtitles
+   against that mixed output. Rebind any precise subtitle track to the new finished audio.
+
+The same steps handle a later revision: keep unchanged picture/voice assets, rebuild only
+the affected bed and placement decisions, then assemble into a new directory. The local
+assembly command still does not resume an old work directory or decide new placements.
+
+## Identity and resume
+
+`recap_run_manifest.json` binds `{mode, selected_stream_index}` separately from
+understanding/material settings. Changing audio mode or stream requires a new
+work directory; it never reuses narrated TTS accidentally. Legacy manifests
+without the audio field are interpreted as `narration`, stream 0, preserving
+old in-flight narration runs. Continuation commands retain the audio selection.
+
+An unbound work directory containing `narration.json` is ambiguous and cannot
+be adopted by a source mode. Existing run-local QC stages are reset after the
+work-directory policy is validated. Source runs record TTS, narration
+validation, and narration review as `not_applicable`; old `tts_meta.json` and
+narration review files are not read as evidence for the current run.
+
+## Fail-closed combinations
+
+- `dub` cannot be combined with a non-narration audio mode.
+- Explicit TTS/text-policy options are rejected in source modes rather than
+  ignored. Ambient TTS provider/voice configuration is inert because TTS does
+  not run, and continuation commands do not promote it into explicit flags.
+- Cut audio modes support selected stream 0 only. Full mode may pass
+  another stream to assemble, subject to assemble/export support.
+- Source modes do not support advisory MiMo QC; use `off`.
+- Local adopted full-sound assembly consumes a prebuilt picture in `full` mode; use the
+  two-stage workflow above for single- or multi-source cuts. That assembly invocation
+  does not resume an old work directory or export an editor draft.
+- `source-mix` rejects an explicit `subtitle_track.json` before analysis or cut,
+  because the precise-track contract only binds adopted packet audio.
+
+`--preserve-approved-text` remains opt-in and is forwarded unchanged to
+voiceover and continuation only in narration mode.
+
+Narration may explicitly select `--tts-provider index-tts`; `auto` does not
+select Index TTS. Index TTS cannot be combined with MiMo voice selection,
+`--voice-ref`, or `dub`. Its endpoint and voice remain backend configuration,
+not recap defaults.
+
+## Packaging and frozen-audio boundary
+
+Source modes never regenerate `visual_overlays.json`. A work directory already
+bound to narration is rejected, preventing narration-derived overlays from
+silently becoming source-mode author input. Otherwise existing overlays are
+treated as caller-authored packaging and preserved.
+
+`adopted-packet-copy` freezes only the selected audio stream on the actual file
+passed to assemble. In `full`, that is the input video. In `cut`, the cut stage
+has already rendered `edited_source.mp4`; packet-copy verification therefore
+applies to that file's audio, **not** to AAC packets from any original source.
+This routing does not prove listening quality or publication approval.

package/resources/skills/video-recap-production/upstream/skills/video-recap/references/config-playbook.md

@@ -0,0 +1,65 @@
+# Config playbook (override-only)
+
+The bundle runs **zero-config** with sensible defaults. To change behavior, set the
+environment variables below (or pass the noted CLI flags) — they **override** the defaults.
+Nothing here is required; this is documentation only. The only config file any tool reads is an
+optional `recap_project.json` passed with `--project` (see `resource-library.md`): it resolves to the same
+variables below, and a variable you set that disagrees with the project stops the run. The bundle ships no
+root `CLAUDE.md` (so it never collides with your project/global instructions).
+Defaults below are bundle-level defaults unless a note scopes them to a specific stage.
+
+| Concern | Env var / flag | Default | Notes |
+|---|---|---|---|
+| MiMo API key | `MIMO_API_KEY` | — | **required for the default pipeline**; one key drives ASR + VLM + default MiMo TTS. `tp-*` Token-Plan keys auto-route to the cluster base URL; `sk-*` keys support pay-as-you-go without a subscription |
+| Token-Plan cluster | `MIMO_TOKEN_PLAN_CLUSTER` | `cn` | `cn` / `sgp` / `ams` (only for `tp-*` keys) |
+| VLM / chat model | `MIMO_MODEL` | `mimo-v2.5` | frame VLM + reviewer + consolidate |
+| ASR model | `MIMO_ASR_MODEL` | `mimo-v2.5-asr` | speech-to-text |
+| ASR language | `MIMO_ASR_LANGUAGE` | `auto` | `auto` / `zh` / `en` |
+| ASR window | `ASR_SEGMENT_SECONDS` | `15` | smaller → finer dialogue timestamps (stays under MiMo's 10MB base64 cap) |
+| TTS provider | `TTS_PROVIDER` / `--tts-provider {auto,mimo-tts,fish-audio,index-tts}` | `auto` | `auto` prefers configured MiMo, then Fish Audio, and never picks index-tts; explicit selection is recommended for repeatable runs |
+| MiMo TTS model | `MIMO_TTS_MODEL` | `mimo-v2.5-tts` | MiMo provider only |
+| MiMo voice | `MIMO_TTS_VOICE` / `--mimo-tts-voice` | `冰糖` | |
+| Cloned narration voice | `VOICE_REF` / `--voice-ref` | off | MiMo full/cut only; lazily normalize once, then use `mimo-v2.5-tts-voiceclone`; mutually exclusive with `--mimo-tts-voice`; requires authorization and sends the reference to MiMo |
+| Fish Audio key | `FISH_API_KEY` | — | required when Fish Audio is selected; never written to artifacts or cache metadata |
+| Fish Audio model | `FISH_TTS_MODEL` | `s2.1-pro-free` | free model under Fair Use, no SLA; check Fish Audio's current policy |
+| Fish Audio voice | `FISH_TTS_REFERENCE_ID` | `5653cea4ac83480aaf2bf45406556185`（娱乐扒妹） | optional override for the built-in narration voice; part of the per-segment cache settings |
+| Fish Audio endpoint | `FISH_TTS_API_URL` | `https://api.fish.audio/v1/tts` | returns WAV directly to the existing voiceover pipeline |
+| Self-hosted TTS | `INDEX_TTS_ENDPOINT` / `INDEX_TTS_VOICE` | — | explicit `--tts-provider index-tts` only; the endpoint is never written to disk, and segment `emotion`/style is rejected |
+| TTS transport | `TTS_TIMEOUT` / `TTS_WORKERS` / `TTS_RETRIES` | `300` / `4` / `3` | request timeout, parallel segments, and per-segment retries for all providers |
+| Advisory MiMo QC | `MIMO_QC` / `--mimo-qc {off,pre-assemble,post-render,both}` | `off` | optional subjective review at the selected stage(s), one request per stage. Always fail-open: results only point the agent/user to `mimo_qc.json`, never block or auto-repair |
+| MiMo QC refresh/model | `MIMO_QC_REFRESH` / `--mimo-qc-refresh`; `MIMO_QC_MODEL` | cache on / VLM fallback | the reuse check compares evidence file sizes/mtimes, model and prompt, never absolute paths. Post-render temporarily samples at most 6 JPEGs (≤768px); base64 is never persisted. The standalone adapter requires explicit `mimo_qc.py --live` for network access |
+| Narration block coverage | `NARRATION_COVERAGE_TARGET` / `NARRATION_BLOCK_SECONDS` | `0.7` / `9.0` | current block-recap density controls |
+| Narration speed | `NARRATION_SPEED` | `1.15` | global atempo on the voiceover; set `1.0` for long-form/documentary |
+| Narration authored start | `NARRATION_DELAY_SECONDS` | `0` | the renderer uses the Agent-authored `start` exactly. Set a non-zero value only for legacy drafts; hidden delay can move a validated sentence-boundary entry back into source speech |
+| Source sentence boundary detector | `SOURCE_BOUNDARY_NOISE_THRESHOLD` / `SOURCE_BOUNDARY_MIN_PAUSE` / `SOURCE_BOUNDARY_MAX_ALIGNMENT_ERROR` | `-18dB` / `0.12` / `2.1` | aligns ASR terminal punctuation to short acoustic pauses and writes `speech_boundary_anchors.json`; unsafe narration entries and cut in/out points are blocked. There is no intentional-interrupt override |
+| Cut sentence snapping | `SNAP_CLIP_LINE_END` / `CLIP_START_SNAP_MAX_PREPEND` / `CLIP_START_SNAP_MAX_TRIM` / `CLIP_SNAP_MAX_EXTEND` | on / `1.8` / `0.35` / `2.0` | scene-cut cleanup runs first; reliable sentence/quiet snapping runs last. An ASR-owned edge still inside speech produces `unsafe_clip_sentence_boundary` instead of a half sentence |
+| Mask source subs | `MASK_SOURCE_SUBTITLES` / `SOURCE_SUBTITLE_MASK_POLICY` / `SOURCE_SUBTITLE_MASK_RATIO` | off / `off` / `0.14` | masking requires an explicit policy (`opt_in`, `safe`, or `forced`) and burned recap subtitles. Passing measured `--subtitle-y-top/--subtitle-y-bot` opts in automatically. With `--no-burn-subtitles`, the MP4 stays unmasked |
+| Measured subtitle band | `SUBTITLE_Y_TOP` / `SUBTITLE_Y_BOT`; `--subtitle-y-top/--subtitle-y-bot` | off | half-open `[top, bot)` ffmpeg auto-rotated display-frame pixel coordinates; square-pixel video and bottom ASS alignment (1/2/3) only; use `tools/measure_subtitle.py` to generate source-identified previews and suggestions |
+| Subtitle mask look | `SUBTITLE_MASK_OPACITY` / `SOURCE_SUBTITLE_MASK_TIMING` / `SUBTITLE_MASK_PADDING` | `0.6` / `narration` / `4` | generic ratio masks stay translucent by default; an explicitly measured `--subtitle-y-top/--subtitle-y-bot` band defaults to opaque so source glyphs cannot show through. Set `SUBTITLE_MASK_OPACITY` to override; timing `all` restores full-time masking |
+| Original ducking | `IDLE_ORIG_VOLUME` / `SPEECH_DUCKING_VOLUME` | `1.0` / `0.2` | the original returns to full-volume `IDLE` in deliberate gaps/original blocks, and ducks to `SPEECH` under narration. Inter-beat gaps shorter than `DUCK_BRIDGE_SECONDS` stay ducked so a single narration block does not swell between sentences. `DUCKING_ORIG_VOLUME` (`0.3`) is only the fallback when beats carry no placement info |
+| Foreign source audio | `FOREIGN_SOURCE_AUDIO` | off | set when the original audio is in a language the narration is **not** (e.g. a Japanese drama recapped in Chinese). The under-narration original (`SPEECH_DUCKING_VOLUME` / `ZONE_DUCKING_VOLUME`) drops from `0.2`/`0.12` to `0.05` so the foreign speech doesn't bleed under the narration as 怪音; original-audio gap blocks still play full-volume (`IDLE_ORIG_VOLUME`). Explicit `SPEECH_DUCKING_VOLUME`/`ZONE_DUCKING_VOLUME` still override. Pairs with bring-your-own `user_subtitles.*` for the foreign dialogue |
+| Duck fade | `DUCK_FADE_SECONDS` | `0.3` | attack ramp target. Sentence-safe source restore is shortened to fit wholly inside the measured acoustic pause (`pause_start`→anchor), so it never reveals the previous source sentence tail |
+| Duck bridge | `DUCK_BRIDGE_SECONDS` | `1.5` | inter-beat gaps shorter than this stay ducked inside one narration block; gaps >= this are treated as intentional original-audio blocks and return to `IDLE_ORIG_VOLUME` |
+| Background music | `BGM_PATH` / `BGM_VOLUME` / `BGM_DUCKING_VOLUME` | off / `0.18` / `0.10` | optional looped music bed mixed as its own track; point `BGM_PATH` at any audio file. It ducks to `BGM_DUCKING_VOLUME` under narration |
+| Final loudness | `FINAL_LOUDNORM` / `TARGET_LUFS` | `true` / `-14` | end-of-pipeline normalize |
+| Output compression | `OUTPUT_CRF` / `OUTPUT_PRESET` / `OUTPUT_MAX_HEIGHT` | `18` / `veryfast` / `0` | x264 re-encode controls, applied whenever the final mux re-encodes (burning subtitles / masking / scaling / `FORCE_VIDEO_REENCODE`). Higher `OUTPUT_CRF` = smaller file/lower quality (18≈visually lossless, 23–26 much smaller); `slow`/`slower` preset shrinks more at the same CRF; `OUTPUT_MAX_HEIGHT>0` downscales the final height (keeps aspect, even width), e.g. `720` to halve 1080p pixels. Subtitles/mask render at native res then downscale, so they stay crisp |
+| Style | `--style` | `纪录片` | freeform verbatim guidance passed into the agent brief. The agent synthesizes voice/pacing from this exact text plus evidence; it is not a fixed option list, preset, switch, or finite style taxonomy |
+| Edit mode | `EDIT_MODE` / `--edit-mode` | `full` | `full`, `cut`, or `dub`; multi-source input supports `cut` only |
+| Cut target | `TARGET_DURATION` / `--target-duration` | — | e.g. `10m` (cut mode) |
+| Scene threshold | `--scene-threshold` | `0.1` | scene-cut sensitivity |
+| Shot-change-aware cut | `SCENE_CUT_SNAP` / `SCENE_CUT_SNAP_MARGIN` / `SCENE_CUT_DETECT_THRESHOLD` | on / `0.5` / `0.4` | cut mode: nudge each clip boundary off the original footage's hard cuts so the edit point doesn't flash a sliver of the adjacent shot (闪烁). source_start moves forward onto / source_end back onto any shot-change within the margin; boundaries already on a cut, or that would shrink a clip below ~0.5s, are left as-is. Set `SCENE_CUT_SNAP=0` to disable |
+| VLM workers | `VLM_WORKERS` | `8` | lower to 1 if a proxy/WAF rate-limits |
+| Subtitle size | `SUBTITLE_FONT_SIZE` / `SUBTITLE_MARGIN_V` | `42` / `48` | look & placement |
+| Subtitle font | `SUBTITLE_FONT_NAME` / `SUBTITLE_FONT_FILE` | `Arial` / — | with a font file, burned ASS subtitles load fonts from its directory (`fontsdir`) and on-screen text overlays use it (`fontfile`); `SUBTITLE_FONT_NAME` must be that file's family name. A bound `subtitle_style` template sets both |
+| 整理 / index | `--no-consolidate` / `--consolidate-asr` | on | build the understanding index (and optionally clean ASR); use `--no-consolidate` to skip |
+| Advisory / strict narration review | `REVIEW_NARRATION` / `--review-narration` / `--no-review-narration`; strict: `REQUIRE_NARRATION_REVIEW` / `--require-narration-review` | advisory on, strict off | runs the narration review stage after validation and before TTS. Default advisory mode is fail-open; strict mode blocks TTS on review failure, parse error, or error-severity findings. In cut mode the reviewer uses `clip_plan_validated.json` to remap VLM/ASR grounding onto the output timeline |
+| 剪映 export (optional) | `--export-jianying` / `EXPORT_JIANYING` | off | after rendering, also write a 剪映/JianYing draft from `timeline.json`. Decoupled — the core render never needs it |
+| 剪映 draft dir | `JIANYING_DRAFT_DIR` | work_dir | parent folder for the exported draft (point it at 剪映's drafts root to open in-app) |
+| 剪映 bundle media | `JIANYING_BUNDLE_MEDIA` / `--jianying-no-bundle-media` | **on** | copies video/audio/photo into `Resources/local/*`, uses draft-placeholder paths, and writes the type-0 material index in `draft_meta_info.json`. This makes a cloned/moved draft self-contained and is required on sandboxed macOS. Use `--jianying-no-bundle-media` only if 剪映 can reach the original paths |
+| Source video | `--source-video` | — | original video (cut mode) so `timeline.json` / 剪映 export reference the real source clips instead of the concatenated `edited_source.mp4`; direct `video-assemble` runs intentionally ignore ambient `SOURCE_VIDEO` unless `--source-video` is passed |
+
+`video-assemble` always writes `timeline.json` — a backend-neutral multi-track model
+(video / original-audio / narration / BGM / subtitle, with ducking automation). The
+canonical renderer is ffmpeg; the 剪映 exporter is an optional consumer of the same file. Subtitle text in `timeline.json` is display-ready and follows the same terminal-punctuation policy as SRT/ASS.
+
+See each stage skill's SKILL.md for the full per-stage option list.