@oh-story/dsh 0.1.11 → 0.1.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -11
- package/lib/client.js +85 -35
- package/lib/client.js.map +3 -3
- package/lib/drama/manifest.json +41 -36
- package/lib/drama/skills/short-drama/scripts/creator_markdown_check.py +340 -13
- package/lib/drama/skills/short-drama-assets/references/voice-direction.md +4 -2
- package/lib/drama/skills/short-drama-edit/SKILL.md +32 -12
- package/lib/drama/skills/short-drama-edit/references/still-cuts.md +90 -0
- package/lib/drama/skills/short-drama-edit/scripts/edit_tool.py +1035 -145
- package/lib/drama/skills/short-drama-edit/scripts/selftest.py +48 -0
- package/lib/drama/skills/short-drama-produce/SKILL.md +6 -1
- package/lib/drama/skills/short-drama-produce/references/adapter-contract.md +12 -0
- package/lib/drama/skills/short-drama-produce/scripts/production_tool.py +138 -9
- package/lib/drama/skills/short-drama-produce/scripts/provider_adapters.py +6 -0
- package/lib/drama/skills/short-drama-storyboard/SKILL.md +2 -0
- package/lib/drama/skills/short-drama-storyboard/references/production-shot-grammar.md +3 -1
- package/lib/drama/skills/short-drama-video-prompts/SKILL.md +7 -0
- package/lib/drama/skills/short-drama-video-prompts/references/performance-action-timing.md +14 -0
- package/lib/drama/skills/short-drama-video-prompts/references/seedance-2.5.md +19 -6
- package/lib/drama/skills/short-drama-video-prompts/references/stage-contract.md +1 -0
- package/lib/drama/skills/short-drama-video-prompts/references/target-model-profile.md +5 -0
- package/lib/drama/skills/short-drama-video-prompts/references/wan-3.0.md +4 -1
- package/lib/index.js +16 -7
- package/lib/index.js.map +2 -2
- package/lib/novel-to-game/README.md +98 -44
- package/lib/novel-to-game/README_ZH.md +100 -46
- package/lib/novel-to-game/VERSION +1 -1
- package/lib/novel-to-game/examples/jin-ping-mei/build/BUILD_BRIEF.md +27 -202
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/RUN.md +23 -115
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/chamber.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/court_night.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/court_rain.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/garden.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/hall.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/kitchen.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/street.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/street_dawn.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/temple.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/cg/yue/moon.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/angry.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/blush.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/neutral.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/sad.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/smile.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/surprised.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/angry.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/blush.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/neutral.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/sad.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/smile.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/surprised.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/angry.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/blush.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/neutral.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/sad.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/smile.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/surprised.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/angry.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/blush.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/neutral.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/sad.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/smile.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/surprised.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/angry.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/blush.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/neutral.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/sad.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/smile.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/surprised.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/css/style.css +326 -2503
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/index.html +71 -4
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/adv.js +221 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/audio.js +113 -257
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/main.js +511 -3026
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/menus.js +277 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/petals.js +67 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/stage.js +180 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/store.js +70 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/common.js +198 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/common2.js +204 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/common3.js +198 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/dates.js +186 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/index.js +122 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/meng.js +190 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/pan.js +212 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/pinger.js +198 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/xuee.js +199 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/yue.js +214 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/package.json +2 -2
- package/lib/novel-to-game/examples/jin-ping-mei/build/art/generated-art.json +182 -45
- package/lib/novel-to-game/examples/jin-ping-mei/qa/README.md +7 -17
- package/lib/novel-to-game/examples/jin-ping-mei/qa/verification.json +9 -9
- package/lib/novel-to-game/manifest.json +303 -78
- package/lib/novel-to-game/skills/game-art-direction/references/art-direction-method.md +7 -2
- package/lib/novel-to-game/skills/game-concept/references/concept-method.md +2 -0
- package/lib/novel-to-game/skills/game-world-design/references/narrative-design-method.md +2 -0
- package/lib/novel-to-game/skills/game-world-design/references/world-design-method.md +9 -0
- package/lib/types/client/drama-production-view.d.ts +11 -0
- package/lib/types/client/drama-production-view.d.ts.map +1 -1
- package/lib/types/client/file-activity.d.ts +1 -1
- package/lib/types/client/file-activity.d.ts.map +1 -1
- package/lib/types/client/production-prompts.d.ts +7 -1
- package/lib/types/client/production-prompts.d.ts.map +1 -1
- package/lib/types/client/production-runtime.d.ts +7 -2
- package/lib/types/client/production-runtime.d.ts.map +1 -1
- package/lib/types/drama-adapters.d.ts.map +1 -1
- package/lib/types/skill-provider.d.ts.map +1 -1
- package/lib/types/video-project.d.ts.map +1 -1
- package/lib/video-recap/CHANGELOG.md +108 -1
- package/lib/video-recap/README.en.md +18 -14
- package/lib/video-recap/README.md +18 -14
- package/lib/video-recap/manifest.json +297 -321
- package/lib/video-recap/skills/video-assemble/SKILL.md +18 -16
- package/lib/video-recap/skills/video-assemble/references/explicit-audio-mix.md +3 -1
- package/lib/video-recap/skills/video-assemble/references/jianying/SOURCE.md +5 -5
- package/lib/video-recap/skills/video-assemble/references/packaging.md +4 -4
- package/lib/video-recap/skills/video-assemble/references/subtitle-track.md +10 -10
- package/lib/video-recap/skills/video-assemble/scripts/adoption/audio_mix_binding.py +13 -2
- package/lib/video-recap/skills/video-assemble/scripts/adoption/av_clock.py +109 -0
- package/lib/video-recap/skills/video-assemble/scripts/adoption/strict_publish.py +4 -1
- package/lib/video-recap/skills/video-assemble/scripts/artifacts.py +2 -9
- package/lib/video-recap/skills/video-assemble/scripts/assemble.py +188 -51
- package/lib/video-recap/skills/video-assemble/scripts/assemble_constants.py +0 -1
- package/lib/video-recap/skills/video-assemble/scripts/assembly_contract.py +32 -75
- package/lib/video-recap/skills/video-assemble/scripts/assembly_settings.py +8 -15
- package/lib/video-recap/skills/video-assemble/scripts/audio_mix.py +202 -161
- package/lib/video-recap/skills/video-assemble/scripts/codec_peak.py +112 -0
- package/lib/video-recap/skills/video-assemble/scripts/export_jianying.py +10 -101
- package/lib/video-recap/skills/video-assemble/scripts/jianying/builders.py +44 -475
- package/lib/video-recap/skills/video-assemble/scripts/jianying/model.py +1 -22
- package/lib/video-recap/skills/video-assemble/scripts/jianying/templates.py +0 -3
- package/lib/video-recap/skills/video-assemble/scripts/jianying/timeline_contract.py +36 -103
- package/lib/video-recap/skills/video-assemble/scripts/jianying/tracks.py +0 -8
- package/lib/video-recap/skills/video-assemble/scripts/jianying/writer.py +28 -203
- package/lib/video-recap/skills/video-assemble/scripts/lib.py +20 -34
- package/lib/video-recap/skills/video-assemble/scripts/loudness.py +332 -0
- package/lib/video-recap/skills/video-assemble/scripts/media.py +158 -38
- package/lib/video-recap/skills/video-assemble/scripts/narration_audio.py +30 -16
- package/lib/video-recap/skills/video-assemble/scripts/render_preflight.py +49 -8
- package/lib/video-recap/skills/video-assemble/scripts/source_subtitles.py +17 -6
- package/lib/video-recap/skills/video-assemble/scripts/subtitles/core.py +24 -2
- package/lib/video-recap/skills/video-assemble/scripts/subtitles/track.py +2 -5
- package/lib/video-recap/skills/video-assemble/scripts/subtitles/track_binding.py +2 -9
- package/lib/video-recap/skills/video-assemble/scripts/timeline.py +12 -34
- package/lib/video-recap/skills/video-assemble/scripts/timeline_emit.py +3 -4
- package/lib/video-recap/skills/video-assemble/scripts/visual_render.py +28 -2
- package/lib/video-recap/skills/video-cut/SKILL.md +37 -11
- package/lib/video-recap/skills/video-cut/references/shot-review.md +1 -1
- package/lib/video-recap/skills/video-cut/scripts/cut.py +2 -34
- package/lib/video-recap/skills/video-cut/scripts/cut_cli.py +96 -124
- package/lib/video-recap/skills/video-cut/scripts/cut_contract.py +97 -182
- package/lib/video-recap/skills/video-cut/scripts/cut_render.py +177 -166
- package/lib/video-recap/skills/video-cut/scripts/frame_grid.py +249 -0
- package/lib/video-recap/skills/video-cut/scripts/lib.py +0 -3
- package/lib/video-recap/skills/video-cut/scripts/media_geometry.py +173 -7
- package/lib/video-recap/skills/video-cut/scripts/sentence_boundaries.py +234 -151
- package/lib/video-recap/skills/video-cut/scripts/sentence_gate.py +189 -0
- package/lib/video-recap/skills/video-cut/scripts/shot_review.py +2 -6
- package/lib/video-recap/skills/video-recap/SKILL.md +24 -27
- package/lib/video-recap/skills/video-recap/references/audio-routing.md +8 -6
- package/lib/video-recap/skills/video-recap/references/config-playbook.md +8 -11
- package/lib/video-recap/skills/video-recap/references/data-schema.md +131 -74
- package/lib/video-recap/skills/video-recap/references/resource-library.md +17 -1
- package/lib/video-recap/skills/video-recap/scripts/doctor.py +51 -268
- package/lib/video-recap/skills/video-recap/scripts/final_qc.py +89 -254
- package/lib/video-recap/skills/video-recap/scripts/lib.py +81 -70
- package/lib/video-recap/skills/video-recap/scripts/library.py +40 -4
- package/lib/video-recap/skills/video-recap/scripts/materials.py +13 -80
- package/lib/video-recap/skills/video-recap/scripts/recap_cli.py +49 -17
- package/lib/video-recap/skills/video-recap/scripts/recap_inspect.py +1 -3
- package/lib/video-recap/skills/video-recap/scripts/recap_runner.py +86 -116
- package/lib/video-recap/skills/video-recap/scripts/recap_runtime.py +87 -23
- package/lib/video-recap/skills/video-recap/scripts/recap_source.py +0 -39
- package/lib/video-recap/skills/video-recap/scripts/recap_stage_qc.py +37 -135
- package/lib/video-recap/skills/video-recap/scripts/recap_timeline.py +118 -127
- package/lib/video-recap/skills/video-recap/scripts/resources/__init__.py +4 -0
- package/lib/video-recap/skills/video-recap/scripts/{resource_lock.py → resources/lock.py} +1 -1
- package/lib/video-recap/skills/video-recap/scripts/{project_binding.py → resources/project_binding.py} +78 -17
- package/lib/video-recap/skills/video-reference/SKILL.md +108 -0
- package/lib/video-recap/skills/video-reference/references/reference-schema.md +139 -0
- package/lib/video-recap/skills/video-reference/scripts/lib.py +85 -0
- package/lib/video-recap/skills/video-reference/scripts/reference.py +96 -0
- package/lib/video-recap/skills/video-reference/scripts/reference_check.py +553 -0
- package/lib/video-recap/skills/video-reference/scripts/reference_frames.py +122 -0
- package/lib/video-recap/skills/video-reference/scripts/reference_measure.py +253 -0
- package/lib/video-recap/skills/video-reference/scripts/reference_profile.py +271 -0
- package/lib/video-recap/skills/video-script/SKILL.md +9 -7
- package/lib/video-recap/skills/video-script/scripts/agent_text.py +18 -267
- package/lib/video-recap/skills/video-script/scripts/deslop_qc.py +12 -30
- package/lib/video-recap/skills/video-script/scripts/entry_suggestion.py +125 -0
- package/lib/video-recap/skills/video-script/scripts/evidence_bundle.py +2 -26
- package/lib/video-recap/skills/video-script/scripts/lib.py +5 -15
- package/lib/video-recap/skills/video-script/scripts/lint_summary.py +184 -0
- package/lib/video-recap/skills/video-script/scripts/narration_lint.py +120 -100
- package/lib/video-recap/skills/video-script/scripts/review.py +3 -34
- package/lib/video-recap/skills/video-script/scripts/review_grounding.py +20 -46
- package/lib/video-recap/skills/video-script/scripts/review_response.py +120 -258
- package/lib/video-recap/skills/video-script/scripts/review_runner.py +2 -10
- package/lib/video-recap/skills/video-script/scripts/speech_ownership.py +89 -5
- package/lib/video-recap/skills/video-script/scripts/validate.py +26 -34
- package/lib/video-recap/skills/video-understanding/SKILL.md +29 -4
- package/lib/video-recap/skills/video-understanding/references/data-schema.md +5 -22
- package/lib/video-recap/skills/video-understanding/scripts/agent_text.py +24 -121
- package/lib/video-recap/skills/video-understanding/scripts/asr.py +11 -5
- package/lib/video-recap/skills/video-understanding/scripts/asr_timing_evidence.py +15 -24
- package/lib/video-recap/skills/video-understanding/scripts/briefing/builder.py +59 -25
- package/lib/video-recap/skills/video-understanding/scripts/briefing/context.py +39 -3
- package/lib/video-recap/skills/video-understanding/scripts/briefing/inputs.py +15 -27
- package/lib/video-recap/skills/video-understanding/scripts/briefing/timeline.py +136 -37
- package/lib/video-recap/skills/video-understanding/scripts/consolidate.py +121 -37
- package/lib/video-recap/skills/video-understanding/scripts/detect.py +118 -16
- package/lib/video-recap/skills/video-understanding/scripts/index_normalize.py +359 -0
- package/lib/video-recap/skills/video-understanding/scripts/lib.py +62 -21
- package/lib/video-recap/skills/video-understanding/scripts/storyboard.py +181 -51
- package/lib/video-recap/skills/video-understanding/scripts/timeline_fusion.py +0 -26
- package/lib/video-recap/skills/video-understanding/scripts/understanding_brief.py +37 -14
- package/lib/video-recap/skills/video-understanding/scripts/understanding_cache.py +40 -32
- package/lib/video-recap/skills/video-understanding/scripts/understanding_runner.py +100 -127
- package/lib/video-recap/skills/video-understanding/scripts/understanding_storyboard.py +182 -30
- package/lib/video-recap/skills/video-understanding/scripts/vlm.py +30 -27
- package/lib/video-recap/skills/video-voiceover/SKILL.md +19 -12
- package/lib/video-recap/skills/video-voiceover/references/index-tts.md +3 -3
- package/lib/video-recap/skills/video-voiceover/scripts/approved_text_policy.py +3 -1
- package/lib/video-recap/skills/video-voiceover/scripts/dub.py +60 -135
- package/lib/video-recap/skills/video-voiceover/scripts/lib.py +9 -34
- package/lib/video-recap/skills/video-voiceover/scripts/providers/fish_audio.py +6 -1
- package/lib/video-recap/skills/video-voiceover/scripts/tts_audio.py +64 -1
- package/lib/video-recap/skills/video-voiceover/scripts/tts_cache.py +99 -0
- package/lib/video-recap/skills/video-voiceover/scripts/voiceover.py +177 -200
- package/package.json +45 -45
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/cg/group/inner_court_accord_five.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/cg/group/public_day15.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/household/li_jiaoer.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/css/new-guofeng.css +0 -1494
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/assets.js +0 -147
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/data.js +0 -7470
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/engine.js +0 -13066
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/text.js +0 -32
- package/lib/novel-to-game/examples/jin-ping-mei/qa/playable-model.json +0 -99
- package/lib/novel-to-game/examples/jin-ping-mei/qa/verify_playable_model.mjs +0 -152
- package/lib/video-recap/skills/video-assemble/references/foreground-compose.md +0 -118
- package/lib/video-recap/skills/video-assemble/references/jianying/empty_jy_combination_segment.json +0 -84
- package/lib/video-recap/skills/video-assemble/references/jianying/empty_jy_combination_video_material.json +0 -98
- package/lib/video-recap/skills/video-assemble/references/jianying/empty_jy_draft.json +0 -204
- package/lib/video-recap/skills/video-assemble/references/pair-media.md +0 -74
- package/lib/video-recap/skills/video-assemble/scripts/compose_foreground.py +0 -293
- package/lib/video-recap/skills/video-assemble/scripts/pair_media.py +0 -216
- package/lib/video-recap/skills/video-recap/references/shift-left-qc-schema.md +0 -23
- package/lib/video-recap/skills/video-recap/scripts/mimo_qc.py +0 -33
- package/lib/video-recap/skills/video-recap/scripts/qc/__init__.py +0 -1
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_client.py +0 -13
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_contract.py +0 -9
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_evidence.py +0 -287
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_observations.py +0 -175
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_payload.py +0 -206
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_report.py +0 -318
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_runner.py +0 -130
- package/lib/video-recap/skills/video-recap/scripts/qc_contract.py +0 -320
- package/lib/video-recap/skills/video-script/scripts/timeline_fusion.py +0 -108
- package/lib/video-recap/skills/video-understanding/scripts/brief.py +0 -9
- package/lib/video-recap/skills/video-understanding/scripts/deslop_qc.py +0 -211
- package/lib/video-recap/skills/video-understanding/scripts/narration_lint.py +0 -707
- package/lib/video-recap/skills/video-understanding/scripts/speech_ownership.py +0 -143
- /package/lib/video-recap/skills/video-cut/scripts/{narration_mapping.py → cut_qc.py} +0 -0
|
@@ -0,0 +1,189 @@
|
|
|
1
|
+
"""The sentence-boundary gate: may a clip edge cut the source audio here?
|
|
2
|
+
|
|
3
|
+
Every audible clip edge must rest on the source start/end, a sentence or quiet pause, a
|
|
4
|
+
lossless same-source join, or outside detected speech. A blocked edge names the nearest
|
|
5
|
+
safe edge times on either side.
|
|
6
|
+
"""
|
|
7
|
+
|
|
8
|
+
|
|
9
|
+
def _unverified_window(row):
|
|
10
|
+
return row.get("boundary_use") == "unverified"
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
def _same_source(left, right):
|
|
14
|
+
# Single-source normalized plans omit source identity; both sides then read as None.
|
|
15
|
+
return left.get("source_id") == right.get("source_id") and left.get(
|
|
16
|
+
"source_path"
|
|
17
|
+
) == right.get("source_path")
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
def _continuous_source_join(left, right, tolerance=0.05):
|
|
21
|
+
"""`right` plays straight on from `left` with no media removed between them.
|
|
22
|
+
|
|
23
|
+
Plan neighbours only: a per-source sub-plan (multi-source snapping) packs clips that
|
|
24
|
+
another source's clip separates in the real plan, so consecutive clip_ids are required.
|
|
25
|
+
"""
|
|
26
|
+
return (
|
|
27
|
+
right.get("clip_id", 1) - left.get("clip_id", 0) == 1
|
|
28
|
+
and _same_source(left, right)
|
|
29
|
+
and abs(left["source_end"] - right["source_start"]) <= tolerance
|
|
30
|
+
and abs(left["output_end"] - right["output_start"]) <= tolerance
|
|
31
|
+
)
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
def _edge_joined(clips, idx, edge, tolerance=0.05):
|
|
35
|
+
"""Whether clips[idx]'s `edge` is a continuous same-source join with its plan neighbour."""
|
|
36
|
+
if edge == "start":
|
|
37
|
+
return idx > 0 and _continuous_source_join(clips[idx - 1], clips[idx], tolerance)
|
|
38
|
+
return idx + 1 < len(clips) and _continuous_source_join(clips[idx], clips[idx + 1], tolerance)
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
# How far either side of a blocked edge the gate looks for a safe edge to suggest.
|
|
42
|
+
_SAFE_EDGE_SEARCH_SECONDS = 5.0
|
|
43
|
+
# How close to a pause window or speech span an edge counts as on it.
|
|
44
|
+
_GATE_TOLERANCE = 0.05
|
|
45
|
+
# The longest first-video-frame delay still waived as the source start. Typical mp4 offsets
|
|
46
|
+
# are a few ms; a capture whose picture begins later than this has real audio before it.
|
|
47
|
+
_VIDEO_START_WAIVER_SECONDS = 0.2
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
def _edge_classifier(boundary_windows, speech_spans, video_duration, tolerance,
|
|
51
|
+
video_start=0.0):
|
|
52
|
+
"""classify(edge, ts, contiguous) -> (status, reason) under the sentence-boundary gate.
|
|
53
|
+
|
|
54
|
+
`video_start` is the source's first video frame (seconds after the file start): a clip
|
|
55
|
+
cannot start any earlier on the picture, so a start up to it is the source start. Only
|
|
56
|
+
up to `_VIDEO_START_WAIVER_SECONDS`: a later first frame (a TS capture whose picture
|
|
57
|
+
starts 1.5 s into the audio) gets the normal speech rules, so a start moved onto it
|
|
58
|
+
that lands mid-sentence still blocks.
|
|
59
|
+
"""
|
|
60
|
+
verified_windows = [row for row in boundary_windows if not _unverified_window(row)]
|
|
61
|
+
unverified_windows = [row for row in boundary_windows if _unverified_window(row)]
|
|
62
|
+
source_start = video_start if video_start <= _VIDEO_START_WAIVER_SECONDS else 0.0
|
|
63
|
+
|
|
64
|
+
def inside(rows, ts):
|
|
65
|
+
return any(row["start"] - tolerance <= ts <= row["end"] + tolerance for row in rows)
|
|
66
|
+
|
|
67
|
+
def classify(edge, ts, contiguous=False):
|
|
68
|
+
if edge == "start" and ts <= source_start + tolerance:
|
|
69
|
+
return "safe", "source_start"
|
|
70
|
+
if edge == "end" and ts >= video_duration - tolerance:
|
|
71
|
+
return "safe", "source_end"
|
|
72
|
+
if contiguous:
|
|
73
|
+
return "safe", "continuous_source_join"
|
|
74
|
+
if inside(verified_windows, ts):
|
|
75
|
+
return "safe", "sentence_or_quiet_boundary"
|
|
76
|
+
if inside(unverified_windows, ts) and (not speech_spans or inside(speech_spans, ts)):
|
|
77
|
+
# Word-safe pause, but the sentence end is only a coarse-ASR estimate.
|
|
78
|
+
return "safe", "unverified_sentence_boundary"
|
|
79
|
+
if not speech_spans:
|
|
80
|
+
return "unchecked", "speech_timing_unavailable"
|
|
81
|
+
if not inside(speech_spans, ts):
|
|
82
|
+
return "safe", "outside_detected_speech"
|
|
83
|
+
return "blocking", "inside_detected_speech"
|
|
84
|
+
|
|
85
|
+
return classify
|
|
86
|
+
|
|
87
|
+
|
|
88
|
+
def _nearest_safe_edges(classify, edge, clip, boundary_windows, speech_spans, video_duration,
|
|
89
|
+
tolerance, frame_snap=None):
|
|
90
|
+
"""The closest safe edge times before and after a blocked edge, within the search span.
|
|
91
|
+
|
|
92
|
+
Candidates are pause-window bounds, the first instants clear of a speech span (past the
|
|
93
|
+
gate tolerance), and the source start/end; each is re-checked with the gate itself and
|
|
94
|
+
must leave the clip a positive length. With `frame_snap(edge, clip, ts)` (where the
|
|
95
|
+
frame-grid pass would move that edge) each candidate is reported where it lands on the
|
|
96
|
+
source frame grid, and only when that landing is safe too: a candidate between two frames
|
|
97
|
+
(a speech start minus the 60 ms clearance is half a frame at 25 fps) is never handed back
|
|
98
|
+
for the frame pass to re-decide, so sending the suggestion back cannot be blocked again or
|
|
99
|
+
moved onto the other side of a source hard cut.
|
|
100
|
+
"""
|
|
101
|
+
ts = clip["source_start"] if edge == "start" else clip["source_end"]
|
|
102
|
+
|
|
103
|
+
def usable(when):
|
|
104
|
+
if not 0.0 <= when <= video_duration or abs(when - ts) > _SAFE_EDGE_SEARCH_SECONDS:
|
|
105
|
+
return False
|
|
106
|
+
return when < clip["source_end"] if edge == "start" else when > clip["source_start"]
|
|
107
|
+
|
|
108
|
+
margin = tolerance + 0.01
|
|
109
|
+
candidates = {0.0, round(video_duration, 3)}
|
|
110
|
+
for row in boundary_windows:
|
|
111
|
+
candidates.update((row["start"], row["end"]))
|
|
112
|
+
for row in speech_spans:
|
|
113
|
+
candidates.update((row["start"] - margin, row["end"] + margin))
|
|
114
|
+
safe = {}
|
|
115
|
+
for when in sorted(round(c, 3) for c in candidates):
|
|
116
|
+
if not usable(when) or classify(edge, when)[0] != "safe":
|
|
117
|
+
continue
|
|
118
|
+
if frame_snap is not None:
|
|
119
|
+
when = round(frame_snap(edge, clip, when), 3)
|
|
120
|
+
status, reason = classify(edge, when)
|
|
121
|
+
if status == "safe" and usable(when):
|
|
122
|
+
safe.setdefault(when, reason)
|
|
123
|
+
before = [when for when in safe if when < ts]
|
|
124
|
+
after = [when for when in safe if when > ts]
|
|
125
|
+
|
|
126
|
+
def suggestion(when):
|
|
127
|
+
return {"time": when, "reason": safe[when], "delta": round(when - ts, 3)}
|
|
128
|
+
|
|
129
|
+
return {"before": suggestion(max(before)) if before else None,
|
|
130
|
+
"after": suggestion(min(after)) if after else None}
|
|
131
|
+
|
|
132
|
+
|
|
133
|
+
def enforce_clip_sentence_boundaries(
|
|
134
|
+
plan, boundary_windows, speech_spans, video_duration, tolerance=_GATE_TOLERANCE,
|
|
135
|
+
video_start=0.0, frame_snap=None,
|
|
136
|
+
):
|
|
137
|
+
"""Block any audible clip edge that falls inside detected source speech.
|
|
138
|
+
|
|
139
|
+
Safe edges are: source start (up to a prompt first video frame, `video_start`) and end, a
|
|
140
|
+
sentence/quiet pause (`unverified_sentence_boundary` when only a coarse-ASR sentence
|
|
141
|
+
estimate covers it), or a truly contiguous same-source join (no media is removed).
|
|
142
|
+
Missing ASR timing degrades to `unchecked` rather than inventing speech. Once ASR says
|
|
143
|
+
an edge is speech-owned, failure to snap is blocking, and the check names the nearest
|
|
144
|
+
safe edge times on either side (`nearest_safe`, frame-checked through `frame_snap`).
|
|
145
|
+
"""
|
|
146
|
+
clips = plan["clips"]
|
|
147
|
+
checks, new_blockers = [], []
|
|
148
|
+
classify = _edge_classifier(boundary_windows, speech_spans, video_duration, tolerance,
|
|
149
|
+
video_start)
|
|
150
|
+
|
|
151
|
+
for idx, clip in enumerate(clips):
|
|
152
|
+
for edge, ts in (("start", clip["source_start"]), ("end", clip["source_end"])):
|
|
153
|
+
status, reason = classify(edge, ts, _edge_joined(clips, idx, edge, tolerance))
|
|
154
|
+
check = {
|
|
155
|
+
"clip_id": clip["clip_id"],
|
|
156
|
+
"source_id": clip.get("source_id"),
|
|
157
|
+
"edge": edge,
|
|
158
|
+
"time": round(ts, 3),
|
|
159
|
+
"status": status,
|
|
160
|
+
"reason": reason,
|
|
161
|
+
}
|
|
162
|
+
if status == "blocking":
|
|
163
|
+
check["nearest_safe"] = _nearest_safe_edges(
|
|
164
|
+
classify, edge, clip, boundary_windows, speech_spans, video_duration,
|
|
165
|
+
tolerance, frame_snap,
|
|
166
|
+
)
|
|
167
|
+
checks.append(check)
|
|
168
|
+
if status == "blocking":
|
|
169
|
+
new_blockers.append(
|
|
170
|
+
{
|
|
171
|
+
"code": "unsafe_clip_sentence_boundary",
|
|
172
|
+
**check,
|
|
173
|
+
"message": "剪辑边界仍落在原声讲话区间内,必须移动到句末锚点,不能截断原声句子。"
|
|
174
|
+
"nearest_safe 给出前后最近的安全边界时间。",
|
|
175
|
+
}
|
|
176
|
+
)
|
|
177
|
+
|
|
178
|
+
qc = plan.setdefault("qc", {})
|
|
179
|
+
qc.setdefault("boundary_status", {})["sentence_checks"] = checks
|
|
180
|
+
existing = [
|
|
181
|
+
row
|
|
182
|
+
for row in qc.get("blocking", [])
|
|
183
|
+
if row["code"] != "unsafe_clip_sentence_boundary"
|
|
184
|
+
]
|
|
185
|
+
if existing or new_blockers:
|
|
186
|
+
qc["blocking"] = existing + new_blockers
|
|
187
|
+
else:
|
|
188
|
+
qc.pop("blocking", None)
|
|
189
|
+
return plan
|
|
@@ -341,14 +341,10 @@ def main():
|
|
|
341
341
|
parser.add_argument("--threshold", type=float, default=0.35)
|
|
342
342
|
parser.add_argument("--roi", nargs=4, type=int, metavar=("X", "Y", "WIDTH", "HEIGHT"))
|
|
343
343
|
parser.add_argument("--max-short-frames", type=int, default=None,
|
|
344
|
-
help="explicit frame cap; default derives round(fps *
|
|
345
|
-
parser.add_argument("--max-short-seconds", type=float, default=1.0)
|
|
346
|
-
parser.add_argument("--dense-window-seconds", type=float, default=2.0)
|
|
347
|
-
parser.add_argument("--min-dense-cuts", type=int, default=4)
|
|
344
|
+
help="explicit frame cap; default derives round(fps * 1s) from the measured clock")
|
|
348
345
|
args = parser.parse_args()
|
|
349
346
|
report = write_scan(args.video, args.output, threshold=args.threshold, plan_path=args.plan, roi=args.roi,
|
|
350
|
-
max_short_frames=args.max_short_frames
|
|
351
|
-
dense_window_seconds=args.dense_window_seconds, min_dense_cuts=args.min_dense_cuts)
|
|
347
|
+
max_short_frames=args.max_short_frames)
|
|
352
348
|
print(json.dumps({"status": report["status"], "short_spans": len(report["short_spans"]),
|
|
353
349
|
"dense_windows": len(report["dense_windows"]), "report": args.output}))
|
|
354
350
|
|
|
@@ -15,8 +15,13 @@ description: >
|
|
|
15
15
|
video-understanding ─▶ Agent 按 video-script 制定方案并写稿 ─▶ [video-cut] ─▶ video-voiceover ─▶ video-assemble
|
|
16
16
|
```
|
|
17
17
|
|
|
18
|
+
把成片拆成制作参考不是生产路径。用户要求时:对成片跑 video-understanding(建议 `ASR_SEGMENT_SECONDS=5`),
|
|
19
|
+
再用 video-reference 做 measure、标注、check、export,把导出的 `production_reference.json` 复制进下一次运行的
|
|
20
|
+
`work_dir`,或登记成资源库的 `production_reference` 模板、采纳后经 `--project` 绑定。recap 不会自动运行它,也不拿新成片与参考做比对。
|
|
21
|
+
|
|
18
22
|
流程支持断点续跑:写好 `narration.json` 后重复同一条命令即可继续。第二阶段会比对
|
|
19
23
|
`recap_run_manifest.json` 记录的源视频路径、文件大小/修改时间与运行参数,拒绝复用来自其他源视频或其他参数的旧工作目录;视频理解产物也只在来源一致时复用。
|
|
24
|
+
暂停时打印的续跑命令就是原命令:保留原来的写法,视频与路径参数转成绝对路径,补上 `--work-dir` 和来自环境变量的设置,从任何目录都能直接运行;同一份参数写在 manifest 的 `argv`。
|
|
20
25
|
|
|
21
26
|
画面流程 `--edit-mode full|cut|dub` 与声音策略 `--audio-mode` 是两个独立开关;
|
|
22
27
|
`narration` 保留上述解说流程,`source-mix` 不做配音,`adopted-packet-copy` 冻结当前输入的已采用 AAC 音轨。
|
|
@@ -46,7 +51,7 @@ python3 scripts/recap.py picture.mp4 --edit-mode full --work-dir NEW_WORK \
|
|
|
46
51
|
```
|
|
47
52
|
|
|
48
53
|
三个 JSON 参数必须同时出现。该入口只接受单视频、full、narration、音轨 0、新工作目录和未存在的
|
|
49
|
-
交付文件;不运行理解、写稿、解说评审、TTS、cut
|
|
54
|
+
交付文件;不运行理解、写稿、解说评审、TTS、cut 或剪映导出。语义与媒体形状仍由
|
|
50
55
|
video-assemble 严格验证,recap 只核对子技能绑定记录引用的是同一批采用文件与母版路径,不把调用方
|
|
51
56
|
采用的声音或混音声明成自动创作或发布批准。详见 `references/audio-routing.md`。
|
|
52
57
|
|
|
@@ -85,9 +90,8 @@ TTS 供应商由 `--tts-provider mimo-tts|fish-audio|index-tts`(或 `TTS_PROVI
|
|
|
85
90
|
可选能力:
|
|
86
91
|
|
|
87
92
|
- `--mimo-video-overview`:按场景块补充 MiMo 视频理解。
|
|
88
|
-
- `--mimo-qc pre-assemble|post-render|both`:在合成前、成片后或两个阶段给出建议型复核。
|
|
89
93
|
|
|
90
|
-
|
|
94
|
+
可覆盖配置见 `references/config-playbook.md`,`final_qc.json` 的字段见 `references/data-schema.md`。
|
|
91
95
|
|
|
92
96
|
下面的 `scripts/...` 均相对于本技能目录。若执行器从仓库根目录启动,请给脚本路径加上本技能的绝对目录。脚本启动后会自行定位兄弟技能和资源。
|
|
93
97
|
|
|
@@ -97,6 +101,7 @@ MiMo QC 默认关闭;每个选定阶段最多请求一次,写入 `mimo_qc.js
|
|
|
97
101
|
|
|
98
102
|
若能识别影片、剧集或主题,先按 video-understanding 技能的调研指南 `research-guide.md` 调研并写入
|
|
99
103
|
`work_dir/background_research.json`。视频理解会把人物名和剧情背景折入 VLM 上下文,避免只得到“黑衣男子”一类模糊描述。无法识别来源时可跳过。
|
|
104
|
+
多视频运行同样写在项目 `work_dir` 下:recap 在每个来源理解前把它复制到 `sources/<source_id>/`(来源目录里没有、或比项目文件旧时才复制,所以某一集需要单独的调研时,在项目文件之后写入该来源目录即可)。
|
|
100
105
|
|
|
101
106
|
### 4.2 分析并暂停创作
|
|
102
107
|
|
|
@@ -116,7 +121,8 @@ python3 scripts/recap.py <video> --work-dir <work_dir> --context "背景"
|
|
|
116
121
|
时间线有两条不可降级的硬约束:原声只能在可靠句末/静音边界被切入、切出或恢复;旁白必须使用
|
|
117
122
|
完整逐段音频,任何 clip 映射裁段、TTS 裁尾或剪映引用更长的加速前素材都阻断。Agent 收到
|
|
118
123
|
`interrupts_source_sentence` / `unsafe_clip_sentence_boundary` / `no_safe_fit` /
|
|
119
|
-
`timeline_audio_mismatch` 时,应移动边界、缩短整句或删除该块,而不是增加抢断 override
|
|
124
|
+
`timeline_audio_mismatch` 时,应移动边界、缩短整句或删除该块,而不是增加抢断 override;
|
|
125
|
+
`unsafe_clip_sentence_boundary` 附带的 `nearest_safe` 给出前后最近的安全边界时间。
|
|
120
126
|
|
|
121
127
|
### 4.3 多视频与素材库
|
|
122
128
|
|
|
@@ -126,6 +132,8 @@ python3 scripts/recap.py <video> --work-dir <work_dir> --context "背景"
|
|
|
126
132
|
python3 scripts/recap.py ep1.mp4 ep2.mp4 --edit-mode cut --target-duration 10m --work-dir work_dir_multi_ep
|
|
127
133
|
```
|
|
128
134
|
|
|
135
|
+
第二阶段与单视频一样生成剪后故事板 `storyboard/edited_storyboard.*`,取自各来源 `sources/<source_id>/frames/`,brief 顶部列出 `S1`/`S2`… 对应的 `source_id`;素材库恢复的来源没有抽帧,这些片段不出现在故事板里,用 `inspect clip-map` 核对。
|
|
136
|
+
|
|
129
137
|
可选文件系统素材库:
|
|
130
138
|
|
|
131
139
|
```bash
|
|
@@ -135,8 +143,8 @@ python3 scripts/recap.py ep1.mp4 ep2.mp4 --edit-mode cut --material-library-dir
|
|
|
135
143
|
|
|
136
144
|
素材检索只是对 JSON / MD / JSONL 做 grep,例如 `grep -R "keyword" .video-materials`。当前版本不复制原始媒体,也不提供数据库、向量或语义搜索。
|
|
137
145
|
|
|
138
|
-
同一根目录还可以登记可复用的资源(BGM
|
|
139
|
-
格式与 `scripts/library.py check|list|show` 只读工具见 `references/resource-library.md`。用 `--project recap_project.json`
|
|
146
|
+
同一根目录还可以登记可复用的资源(BGM、音效、音色、字体、图片)、带版本与采用记录的模板(字幕样式、包装图层、制作参考)和样片。
|
|
147
|
+
格式与 `scripts/library.py check|list|show` 只读工具见 `references/resource-library.md`。用 `--project recap_project.json` 把已采用的字幕样式、包装、制作参考、音色与 BGM 绑定到这次运行;
|
|
140
148
|
每次 full / cut 合成后 `work_dir/resource_lock.json` 记下实际用到的资源与授权状态。
|
|
141
149
|
|
|
142
150
|
### 4.4 继续生成成片
|
|
@@ -149,15 +157,7 @@ python3 scripts/recap.py <video> --work-dir <work_dir> # 可追加 --edit-mode
|
|
|
149
157
|
|
|
150
158
|
流程会校验当前阶段的硬输入(`clip_plan.json` / `narration.json`);两份创作计划仍是 Agent 与建议型评审使用的工作记录,不是渲染门禁。cut 模式随后生成 `edited_source.mp4`,再合成旁白并输出 `recap_<name>.mp4`。
|
|
151
159
|
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
```bash
|
|
155
|
-
python3 scripts/recap.py <video> --work-dir <work_dir> --mimo-qc both
|
|
156
|
-
```
|
|
157
|
-
|
|
158
|
-
合成前复核会读取脚本、计划和 TTS 元数据;成片后还会读取最多六张临时 JPEG。输入文件的大小/修改时间、模型与提示都未变时直接复用上次报告,`--mimo-qc-refresh` 可强制刷新。帧的 base64 与凭证不会写入磁盘。
|
|
159
|
-
|
|
160
|
-
已有批准解说稿时加 `--preserve-approved-text`:校验与 TTS 原样保留批准稿(只更新 `overlaps_speech`),装不下时间窗即失败,不缩稿、不降级为部分成功。
|
|
160
|
+
校验从不改写解说稿(只更新实测的 `overlaps_speech`);full 模式下文本装不下时间窗会以 `over_budget` error 退回给 Agent。已有批准解说稿时加 `--preserve-approved-text`,TTS 也原样保留批准稿,装不下时间窗即失败,不缩稿、不降级为部分成功。
|
|
161
161
|
|
|
162
162
|
### 4.5 字幕与克隆旁白
|
|
163
163
|
|
|
@@ -184,11 +184,16 @@ python3 tools/measure_subtitle.py <video>
|
|
|
184
184
|
scene score、亮度统计、contact sheet 与自动 QC 只负责定位候选问题;最终判断以真实播放为准。密集切点的来源判断与处理规则按剪辑技能执行。修复失败时回到剪点、声音或文案层,不用更多包装掩盖。
|
|
185
185
|
|
|
186
186
|
full/cut 交付如需让确定性的最终检查影响命令退出状态,显式传
|
|
187
|
-
`--require-final-qc`。只有 `final_qc.json`
|
|
188
|
-
`
|
|
187
|
+
`--require-final-qc`。只有 `final_qc.json` 的摘要为 `ok: true` 且整数
|
|
188
|
+
`blocker_count: 0` 才打印完成并返回成功;缺失、畸形或 blocker
|
|
189
189
|
会保留报告和已渲染诊断媒体,但命令非零退出且不打印完成。默认仍是仅报告、不阻断。
|
|
190
190
|
该参数不支持 `--edit-mode dub`;dub 未传该参数时的准备和渲染行为不变。
|
|
191
191
|
|
|
192
|
+
交付时读 `final_qc.json` 的 `metadata.warnings`,有内容就原样告诉用户,不能只报“完成”。`subtitle_burn_degraded`
|
|
193
|
+
表示 ffmpeg 缺 libass、默认烧录被降级:成片里没有字幕,字幕在成片旁的同名 `.srt`。用户要烧录字幕,就请他装带 libass 的
|
|
194
|
+
ffmpeg 后重跑;显式传 `--burn-subtitles` 时缺 libass 会在开跑前报错。旁白里写了 `visual_overlays` 而 ffmpeg 缺
|
|
195
|
+
drawtext 时,流程在配音前停下,按报错删掉叠加或换 ffmpeg 后续跑。
|
|
196
|
+
|
|
192
197
|
### 4.7 不需要解说的片子
|
|
193
198
|
|
|
194
199
|
```bash
|
|
@@ -211,19 +216,13 @@ python3 scripts/recap.py adopted.mp4 --work-dir packaging_work --audio-mode adop
|
|
|
211
216
|
python3 scripts/recap.py <video> --edit-mode dub --work-dir <work_dir>
|
|
212
217
|
```
|
|
213
218
|
|
|
214
|
-
准备阶段会转写英文、提取一段参考音频,并写出 `dub_brief.md` 与 `dub_transcript.json`。Agent
|
|
219
|
+
准备阶段会转写英文、提取一段参考音频,并写出 `dub_brief.md` 与 `dub_transcript.json`。Agent 按 `dub_brief.md` 里的翻译要求(逐句忠实、时间窗、语速)写 `dub_script.json`:
|
|
215
220
|
|
|
216
221
|
```json
|
|
217
222
|
[{"start": 0.0, "end": 2.0, "zh": "中文译文"}]
|
|
218
223
|
```
|
|
219
224
|
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
- 逐句忠实翻译,不删钩子、不合并、不擅自压缩;原文重复,译文也按时间重复。
|
|
223
|
-
- 每句沿用原声 `[start, end]`,相邻句不重叠。
|
|
224
|
-
- 译文尽量控制在约 5 字/秒,使其能在原时间窗内说完。
|
|
225
|
-
|
|
226
|
-
重复同一命令后输出 `dub_<name>.mp4`。每句单独克隆并贴回原时间线;只有即将覆盖下一句时才局部加速。当前版本只支持单说话者、整轨替换,不分离背景音乐。
|
|
225
|
+
重复同一命令后先做确定性 lint(`dub_lint.json`,空行、重叠、越界即中止),再输出 `dub_<name>.mp4`。每句单独克隆并贴回原时间线;只有即将覆盖下一句时才局部加速。当前版本只支持单说话者、整轨替换,不分离背景音乐。
|
|
227
226
|
|
|
228
227
|
## 6. 自检与只读 dashboard
|
|
229
228
|
|
|
@@ -249,12 +248,10 @@ python3 scripts/dashboard_server.py --root <目录> [--port 0] [--open]
|
|
|
249
248
|
- `subtitles.srt` / `subtitles.ass`:字幕。
|
|
250
249
|
- `work_dir/`:全部中间产物,契约见 `references/data-schema.md`。
|
|
251
250
|
- `work_dir/recap_story_plan.json` / `visual_audio_board.json`:Agent 创作意图与剪辑决定。
|
|
252
|
-
- `work_dir/mimo_qc.json`:可选的建议型复核,不作为发布门禁。
|
|
253
251
|
|
|
254
252
|
完整参数列表以 `python3 scripts/recap.py --help` 为准。`--style` 是原样传给 Agent 的自由文本指导,不是 preset、枚举、开关或有限风格分类。
|
|
255
253
|
|
|
256
254
|
## 8. 能力边界
|
|
257
255
|
|
|
258
256
|
- 语义评审默认建议型、失败开放;只有调用方显式启用严格解说评审时,事实矛盾、残句或评审不可用才会在 TTS 前阻断。确定性校验阶段始终负责硬校验。
|
|
259
|
-
- MiMo QC 不能阻断、自动修复或改变退出状态,只提供定位建议。
|
|
260
257
|
- 宣发标题、花字或外部文案回填见 `video-script` 的 references/promotional-copy.md。
|
|
@@ -26,9 +26,9 @@ exactly this all-or-none bundle:
|
|
|
26
26
|
It requires narration mode, stream 0, a new explicit `--work-dir`, and a delivery path
|
|
27
27
|
that does not already exist. `--output-dir` is optional; when omitted, delivery uses the
|
|
28
28
|
new work directory's parent. The route calls only the video-assemble CLI; it does
|
|
29
|
-
not run understanding, script validation, narration review, voiceover,
|
|
29
|
+
not run understanding, script validation, narration review, voiceover, cut,
|
|
30
30
|
continuation, editor export, or material-cache reuse. Ambient TTS provider and voice
|
|
31
|
-
configuration are inert. Explicit TTS/voice/review/
|
|
31
|
+
configuration are inert. Explicit TTS/voice/review/editor flags are rejected rather
|
|
32
32
|
than silently ignored.
|
|
33
33
|
|
|
34
34
|
`recap_run_manifest.json` records the resolved path of all three local artifacts under
|
|
@@ -54,8 +54,10 @@ episodes. Use two existing stages rather than sending an old mix binding into a
|
|
|
54
54
|
--sources-manifest SOURCES_JSON
|
|
55
55
|
```
|
|
56
56
|
|
|
57
|
-
|
|
58
|
-
|
|
57
|
+
`SOURCES_JSON` must be `{"sources": [{"source_id": ..., "source_path": ...}]}`
|
|
58
|
+
(optional per-source `duration` and `source_work_dir`); recap's
|
|
59
|
+
`multi_source_manifest.json` already has this shape. A single-source cut omits
|
|
60
|
+
`--sources-manifest`. For already locked frame decisions, use the picture-plan path instead. Read `clip_plan_validated.json` (or the locked path's
|
|
59
61
|
`picture_map.json`) before placing sound.
|
|
60
62
|
2. Keep the selected WAVs, `tts_meta.json`, and `narration_adoption.json` when the words,
|
|
61
63
|
WAV bytes, and selected voice are unchanged. Do not run voiceover just to move a line.
|
|
@@ -97,7 +99,6 @@ narration review files are not read as evidence for the current run.
|
|
|
97
99
|
not run, and continuation commands do not promote it into explicit flags.
|
|
98
100
|
- Cut audio modes support selected stream 0 only. Full mode may pass
|
|
99
101
|
another stream to assemble, subject to assemble/export support.
|
|
100
|
-
- Source modes do not support advisory MiMo QC; use `off`.
|
|
101
102
|
- Local adopted full-sound assembly consumes a prebuilt picture in `full` mode; use the
|
|
102
103
|
two-stage workflow above for single- or multi-source cuts. That assembly invocation
|
|
103
104
|
does not resume an old work directory or export an editor draft.
|
|
@@ -105,7 +106,8 @@ narration review files are not read as evidence for the current run.
|
|
|
105
106
|
because the precise-track contract only binds adopted packet audio.
|
|
106
107
|
|
|
107
108
|
`--preserve-approved-text` remains opt-in and is forwarded unchanged to
|
|
108
|
-
voiceover and continuation only in narration mode.
|
|
109
|
+
voiceover and continuation only in narration mode. The narration validator no longer takes
|
|
110
|
+
it: validation never rewrites narration in any mode.
|
|
109
111
|
|
|
110
112
|
Narration may explicitly select `--tts-provider index-tts`; `auto` does not
|
|
111
113
|
select Index TTS. Index TTS cannot be combined with MiMo voice selection,
|
|
@@ -26,23 +26,20 @@ Defaults below are bundle-level defaults unless a note scopes them to a specific
|
|
|
26
26
|
| Fish Audio endpoint | `FISH_TTS_API_URL` | `https://api.fish.audio/v1/tts` | returns WAV directly to the existing voiceover pipeline |
|
|
27
27
|
| Self-hosted TTS | `INDEX_TTS_ENDPOINT` / `INDEX_TTS_VOICE` | — | explicit `--tts-provider index-tts` only; the endpoint is never written to disk, and segment `emotion`/style is rejected |
|
|
28
28
|
| TTS transport | `TTS_TIMEOUT` / `TTS_WORKERS` / `TTS_RETRIES` | `300` / `4` / `3` | request timeout, parallel segments, and per-segment retries for all providers |
|
|
29
|
-
|
|
|
30
|
-
| MiMo QC refresh/model | `MIMO_QC_REFRESH` / `--mimo-qc-refresh`; `MIMO_QC_MODEL` | cache on / VLM fallback | the reuse check compares evidence file sizes/mtimes, model and prompt, never absolute paths. Post-render temporarily samples at most 6 JPEGs (≤768px); base64 is never persisted. The standalone adapter requires explicit `mimo_qc.py --live` for network access |
|
|
31
|
-
| Narration block coverage | `NARRATION_COVERAGE_TARGET` / `NARRATION_BLOCK_SECONDS` | `0.7` / `9.0` | current block-recap density controls |
|
|
29
|
+
| TTS hallucination bound | `TTS_MIN_SPEECH_RATE` | `2.5` | chars/s (each digit counts 1, `%` counts 3, a Latin word 1.5); a narration segment or dub line longer than reading its text at this rate plus pause and edge allowances is treated as invented speech: retried, never cached, failed after `TTS_RETRIES` with the last take kept as `*.rejected.wav`; `0` disables |
|
|
32
30
|
| Narration speed | `NARRATION_SPEED` | `1.15` | global atempo on the voiceover; set `1.0` for long-form/documentary |
|
|
33
|
-
| Narration authored start | `NARRATION_DELAY_SECONDS` | `0` | the renderer uses the Agent-authored `start` exactly. Set a non-zero value only for legacy drafts; hidden delay can move a validated sentence-boundary entry back into source speech |
|
|
34
31
|
| Source sentence boundary detector | `SOURCE_BOUNDARY_NOISE_THRESHOLD` / `SOURCE_BOUNDARY_MIN_PAUSE` / `SOURCE_BOUNDARY_MAX_ALIGNMENT_ERROR` | `-18dB` / `0.12` / `2.1` | aligns ASR terminal punctuation to short acoustic pauses and writes `speech_boundary_anchors.json`; unsafe narration entries and cut in/out points are blocked. There is no intentional-interrupt override |
|
|
35
|
-
| Cut sentence snapping | `SNAP_CLIP_LINE_END` / `CLIP_START_SNAP_MAX_PREPEND` / `CLIP_START_SNAP_MAX_TRIM` / `CLIP_SNAP_MAX_EXTEND` | on / `1.8` / `0.35` / `2.0` | scene-cut cleanup runs first
|
|
36
|
-
| Mask source subs | `MASK_SOURCE_SUBTITLES` / `SOURCE_SUBTITLE_MASK_POLICY` / `SOURCE_SUBTITLE_MASK_RATIO` | off / `off` / `0.14` | masking requires an explicit policy (`opt_in`, `safe`, or `forced`) and burned recap subtitles. Passing measured `--subtitle-y-top/--subtitle-y-bot` opts in automatically. With `--no-burn-subtitles`, the MP4 stays unmasked |
|
|
37
|
-
| Measured subtitle band | `SUBTITLE_Y_TOP` / `SUBTITLE_Y_BOT`; `--subtitle-y-top/--subtitle-y-bot` | off | half-open `[top, bot)`
|
|
32
|
+
| Cut sentence snapping | `SNAP_CLIP_LINE_END` / `CLIP_START_SNAP_MAX_PREPEND` / `CLIP_START_SNAP_MAX_TRIM` / `CLIP_SNAP_MAX_EXTEND` | on / `1.8` / `0.35` / `2.0` | scene-cut cleanup runs first, then reliable sentence/quiet snapping, then a sub-frame frame-grid alignment that keeps the edge in its pause. An ASR-owned edge still inside speech produces `unsafe_clip_sentence_boundary` instead of a half sentence, with `nearest_safe` before/after times within 5 s, already on the source frame grid |
|
|
33
|
+
| Mask source subs | `MASK_SOURCE_SUBTITLES` / `SOURCE_SUBTITLE_MASK_POLICY` / `SOURCE_SUBTITLE_MASK_RATIO` | off / `off` / `0.14` | masking requires an explicit policy (`opt_in`, `safe`, or `forced`) and burned recap subtitles. Passing measured `--subtitle-y-top/--subtitle-y-bot` opts in automatically. With `--no-burn-subtitles`, or when a default burn degrades because ffmpeg lacks libass, the MP4 stays unmasked |
|
|
34
|
+
| Measured subtitle band | `SUBTITLE_Y_TOP` / `SUBTITLE_Y_BOT`; `--subtitle-y-top/--subtitle-y-bot` | off | half-open `[top, bot)` rows of the auto-rotated display canvas (the stored width is scaled by the SAR; a rotated non-square stream's rows are scaled by it too, and the mask converts them back to decoded-frame rows); square or near-square pixels (SAR within 2% of 1:1; unspecified `0:1` counts as square) and bottom ASS alignment (1/2/3) only; use `tools/measure_subtitle.py` to generate source-identified previews and suggestions |
|
|
38
35
|
| Subtitle mask look | `SUBTITLE_MASK_OPACITY` / `SOURCE_SUBTITLE_MASK_TIMING` / `SUBTITLE_MASK_PADDING` | `0.6` / `narration` / `4` | generic ratio masks stay translucent by default; an explicitly measured `--subtitle-y-top/--subtitle-y-bot` band defaults to opaque so source glyphs cannot show through. Set `SUBTITLE_MASK_OPACITY` to override; timing `all` restores full-time masking |
|
|
39
|
-
| Original ducking | `IDLE_ORIG_VOLUME` / `SPEECH_DUCKING_VOLUME` | `1.0` / `0.2` | the original returns to full-volume `IDLE` in deliberate gaps/original blocks, and ducks to `SPEECH` under narration. Inter-beat gaps shorter than `DUCK_BRIDGE_SECONDS` stay ducked so a single narration block does not swell between sentences.
|
|
36
|
+
| Original ducking | `IDLE_ORIG_VOLUME` / `SPEECH_DUCKING_VOLUME` | `1.0` / `0.2` | the original returns to full-volume `IDLE` in deliberate gaps/original blocks, and ducks to `SPEECH` under narration. Inter-beat gaps shorter than `DUCK_BRIDGE_SECONDS` stay ducked so a single narration block does not swell between sentences. A fixed `0.3` is only the fallback when beats carry no placement info |
|
|
40
37
|
| Foreign source audio | `FOREIGN_SOURCE_AUDIO` | off | set when the original audio is in a language the narration is **not** (e.g. a Japanese drama recapped in Chinese). The under-narration original (`SPEECH_DUCKING_VOLUME` / `ZONE_DUCKING_VOLUME`) drops from `0.2`/`0.12` to `0.05` so the foreign speech doesn't bleed under the narration as 怪音; original-audio gap blocks still play full-volume (`IDLE_ORIG_VOLUME`). Explicit `SPEECH_DUCKING_VOLUME`/`ZONE_DUCKING_VOLUME` still override. Pairs with bring-your-own `user_subtitles.*` for the foreign dialogue |
|
|
41
38
|
| Duck fade | `DUCK_FADE_SECONDS` | `0.3` | attack ramp target. Sentence-safe source restore is shortened to fit wholly inside the measured acoustic pause (`pause_start`→anchor), so it never reveals the previous source sentence tail |
|
|
42
39
|
| Duck bridge | `DUCK_BRIDGE_SECONDS` | `1.5` | inter-beat gaps shorter than this stay ducked inside one narration block; gaps >= this are treated as intentional original-audio blocks and return to `IDLE_ORIG_VOLUME` |
|
|
43
40
|
| Background music | `BGM_PATH` / `BGM_VOLUME` / `BGM_DUCKING_VOLUME` | off / `0.18` / `0.10` | optional looped music bed mixed as its own track; point `BGM_PATH` at any audio file. It ducks to `BGM_DUCKING_VOLUME` under narration |
|
|
44
|
-
| Final loudness | `FINAL_LOUDNORM` / `TARGET_LUFS` | `true` / `-14` | end-of-pipeline normalize |
|
|
45
|
-
| Output compression | `OUTPUT_CRF` / `OUTPUT_PRESET` / `OUTPUT_MAX_HEIGHT` | `18` / `veryfast` / `0` | x264 re-encode controls, applied whenever the final mux re-encodes (burning subtitles / masking / scaling / `FORCE_VIDEO_REENCODE`). Higher `OUTPUT_CRF` = smaller file/lower quality (18≈visually lossless, 23–26 much smaller); `slow`/`slower` preset shrinks more at the same CRF; `OUTPUT_MAX_HEIGHT>0` downscales the final height (keeps aspect, even width), e.g. `720` to halve 1080p pixels. Subtitles/mask render at native res then downscale, so they stay crisp |
|
|
41
|
+
| Final loudness | `FINAL_LOUDNORM` / `TARGET_LUFS` / `LOUDNESS_LIMITER_MAX_DB` | `true` / `-14` / `6` | end-of-pipeline two-pass normalize with one constant gain, never dynamically compressed; when the mix's true peak has no room for that gain an oversampled true-peak limiter takes up to `LOUDNESS_LIMITER_MAX_DB` of peak first, and only the gain beyond that lowers the target (`0` disables the limiter). The -1 dBTP ceiling holds for the delivered AAC file: the first render aims 0.5 dB under it, the encoded file is decoded and measured, and an overshoot re-encodes only the audio lower (at most twice; still over blocks with `delivered_true_peak_over_target`). `assembly_qc.json` records the mode ffmpeg actually ran (`loudness_mode`, `loudnorm_final_pass` incl. `peak_limiter` and the measured `delivered` file) |
|
|
42
|
+
| Output compression | `OUTPUT_CRF` / `OUTPUT_PRESET` / `OUTPUT_MAX_HEIGHT` | `18` / `veryfast` / `0` | x264 re-encode controls, applied whenever the final mux re-encodes (burning subtitles / masking / scaling / `FORCE_VIDEO_REENCODE`, or a source picture that is not H.264 8-bit 4:2:0 with even dimensions; the delivery is always H.264 8-bit 4:2:0 with `+faststart`: `yuv420p`, or a full-range `yuvj420p` H.264 source copied unchanged; tagged BT.709 when the source is untagged or BT.709, keeping any other declared colour tags, and an RGB source (colour space `gbr` or an RGB/paletted pixel format) is converted to BT.709 limited range; multi-source cuts whose sources disagree convert every clip's matrix and range to BT.709 limited. Dub mode (`--edit-mode dub`) copies the source picture as is). Higher `OUTPUT_CRF` = smaller file/lower quality (18≈visually lossless, 23–26 much smaller); `slow`/`slower` preset shrinks more at the same CRF; `OUTPUT_MAX_HEIGHT>0` downscales the final height (keeps aspect, even width), e.g. `720` to halve 1080p pixels. Subtitles/mask render at native res then downscale, so they stay crisp |
|
|
46
43
|
| Style | `--style` | `纪录片` | freeform verbatim guidance passed into the agent brief. The agent synthesizes voice/pacing from this exact text plus evidence; it is not a fixed option list, preset, switch, or finite style taxonomy |
|
|
47
44
|
| Edit mode | `EDIT_MODE` / `--edit-mode` | `full` | `full`, `cut`, or `dub`; multi-source input supports `cut` only |
|
|
48
45
|
| Cut target | `TARGET_DURATION` / `--target-duration` | — | e.g. `10m` (cut mode) |
|
|
@@ -56,7 +53,7 @@ Defaults below are bundle-level defaults unless a note scopes them to a specific
|
|
|
56
53
|
| 剪映 export (optional) | `--export-jianying` / `EXPORT_JIANYING` | off | after rendering, also write a 剪映/JianYing draft from `timeline.json`. Decoupled — the core render never needs it |
|
|
57
54
|
| 剪映 draft dir | `JIANYING_DRAFT_DIR` | work_dir | parent folder for the exported draft (point it at 剪映's drafts root to open in-app) |
|
|
58
55
|
| 剪映 bundle media | `JIANYING_BUNDLE_MEDIA` / `--jianying-no-bundle-media` | **on** | copies video/audio/photo into `Resources/local/*`, uses draft-placeholder paths, and writes the type-0 material index in `draft_meta_info.json`. This makes a cloned/moved draft self-contained and is required on sandboxed macOS. Use `--jianying-no-bundle-media` only if 剪映 can reach the original paths |
|
|
59
|
-
| Source video | `--source-video` | — | original video (cut mode) so `timeline.json` / 剪映 export reference the real source clips instead of the concatenated `edited_source.mp4
|
|
56
|
+
| Source video | `--source-video` | — | original video (cut mode) so `timeline.json` / 剪映 export reference the real source clips instead of the concatenated `edited_source.mp4` |
|
|
60
57
|
|
|
61
58
|
`video-assemble` always writes `timeline.json` — a backend-neutral multi-track model
|
|
62
59
|
(video / original-audio / narration / BGM / subtitle, with ducking automation). The
|