@oh-story/dsh 0.1.12 → 0.1.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -11
- package/lib/client.js +85 -35
- package/lib/client.js.map +3 -3
- package/lib/drama/manifest.json +41 -36
- package/lib/drama/skills/short-drama/scripts/creator_markdown_check.py +340 -13
- package/lib/drama/skills/short-drama-assets/references/voice-direction.md +4 -2
- package/lib/drama/skills/short-drama-edit/SKILL.md +32 -12
- package/lib/drama/skills/short-drama-edit/references/still-cuts.md +90 -0
- package/lib/drama/skills/short-drama-edit/scripts/edit_tool.py +1035 -145
- package/lib/drama/skills/short-drama-edit/scripts/selftest.py +48 -0
- package/lib/drama/skills/short-drama-produce/SKILL.md +6 -1
- package/lib/drama/skills/short-drama-produce/references/adapter-contract.md +12 -0
- package/lib/drama/skills/short-drama-produce/scripts/production_tool.py +138 -9
- package/lib/drama/skills/short-drama-produce/scripts/provider_adapters.py +6 -0
- package/lib/drama/skills/short-drama-storyboard/SKILL.md +2 -0
- package/lib/drama/skills/short-drama-storyboard/references/production-shot-grammar.md +3 -1
- package/lib/drama/skills/short-drama-video-prompts/SKILL.md +7 -0
- package/lib/drama/skills/short-drama-video-prompts/references/performance-action-timing.md +14 -0
- package/lib/drama/skills/short-drama-video-prompts/references/seedance-2.5.md +19 -6
- package/lib/drama/skills/short-drama-video-prompts/references/stage-contract.md +1 -0
- package/lib/drama/skills/short-drama-video-prompts/references/target-model-profile.md +5 -0
- package/lib/drama/skills/short-drama-video-prompts/references/wan-3.0.md +4 -1
- package/lib/index.js +16 -7
- package/lib/index.js.map +2 -2
- package/lib/novel-to-game/README.md +98 -44
- package/lib/novel-to-game/README_ZH.md +100 -46
- package/lib/novel-to-game/VERSION +1 -1
- package/lib/novel-to-game/examples/jin-ping-mei/build/BUILD_BRIEF.md +27 -202
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/RUN.md +23 -115
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/chamber.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/court_night.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/court_rain.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/garden.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/hall.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/kitchen.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/street.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/street_dawn.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/bg/temple.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/cg/yue/moon.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/angry.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/blush.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/neutral.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/sad.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/smile.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/meng/surprised.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/angry.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/blush.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/neutral.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/sad.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/smile.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pan/surprised.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/angry.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/blush.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/neutral.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/sad.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/smile.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/pinger/surprised.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/angry.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/blush.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/neutral.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/sad.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/smile.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/xuee/surprised.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/angry.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/blush.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/neutral.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/sad.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/smile.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/sprite/yue/surprised.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/css/style.css +326 -2503
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/index.html +71 -4
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/adv.js +221 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/audio.js +113 -257
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/main.js +511 -3026
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/menus.js +277 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/petals.js +67 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/stage.js +180 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/store.js +70 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/common.js +198 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/common2.js +204 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/common3.js +198 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/dates.js +186 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/index.js +122 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/meng.js +190 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/pan.js +212 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/pinger.js +198 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/xuee.js +199 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/story/yue.js +214 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/package.json +2 -2
- package/lib/novel-to-game/examples/jin-ping-mei/build/art/generated-art.json +182 -45
- package/lib/novel-to-game/examples/jin-ping-mei/qa/README.md +7 -17
- package/lib/novel-to-game/examples/jin-ping-mei/qa/verification.json +9 -9
- package/lib/novel-to-game/manifest.json +303 -78
- package/lib/novel-to-game/skills/game-art-direction/references/art-direction-method.md +7 -2
- package/lib/novel-to-game/skills/game-concept/references/concept-method.md +2 -0
- package/lib/novel-to-game/skills/game-world-design/references/narrative-design-method.md +2 -0
- package/lib/novel-to-game/skills/game-world-design/references/world-design-method.md +9 -0
- package/lib/types/client/drama-production-view.d.ts +11 -0
- package/lib/types/client/drama-production-view.d.ts.map +1 -1
- package/lib/types/client/file-activity.d.ts +1 -1
- package/lib/types/client/file-activity.d.ts.map +1 -1
- package/lib/types/client/production-prompts.d.ts +7 -1
- package/lib/types/client/production-prompts.d.ts.map +1 -1
- package/lib/types/client/production-runtime.d.ts +7 -2
- package/lib/types/client/production-runtime.d.ts.map +1 -1
- package/lib/types/drama-adapters.d.ts.map +1 -1
- package/lib/types/skill-provider.d.ts.map +1 -1
- package/lib/types/video-project.d.ts.map +1 -1
- package/lib/video-recap/CHANGELOG.md +108 -1
- package/lib/video-recap/README.en.md +18 -14
- package/lib/video-recap/README.md +18 -14
- package/lib/video-recap/manifest.json +297 -321
- package/lib/video-recap/skills/video-assemble/SKILL.md +18 -16
- package/lib/video-recap/skills/video-assemble/references/explicit-audio-mix.md +3 -1
- package/lib/video-recap/skills/video-assemble/references/jianying/SOURCE.md +5 -5
- package/lib/video-recap/skills/video-assemble/references/packaging.md +4 -4
- package/lib/video-recap/skills/video-assemble/references/subtitle-track.md +10 -10
- package/lib/video-recap/skills/video-assemble/scripts/adoption/audio_mix_binding.py +13 -2
- package/lib/video-recap/skills/video-assemble/scripts/adoption/av_clock.py +109 -0
- package/lib/video-recap/skills/video-assemble/scripts/adoption/strict_publish.py +4 -1
- package/lib/video-recap/skills/video-assemble/scripts/artifacts.py +2 -9
- package/lib/video-recap/skills/video-assemble/scripts/assemble.py +188 -51
- package/lib/video-recap/skills/video-assemble/scripts/assemble_constants.py +0 -1
- package/lib/video-recap/skills/video-assemble/scripts/assembly_contract.py +32 -75
- package/lib/video-recap/skills/video-assemble/scripts/assembly_settings.py +8 -15
- package/lib/video-recap/skills/video-assemble/scripts/audio_mix.py +202 -161
- package/lib/video-recap/skills/video-assemble/scripts/codec_peak.py +112 -0
- package/lib/video-recap/skills/video-assemble/scripts/export_jianying.py +10 -101
- package/lib/video-recap/skills/video-assemble/scripts/jianying/builders.py +44 -475
- package/lib/video-recap/skills/video-assemble/scripts/jianying/model.py +1 -22
- package/lib/video-recap/skills/video-assemble/scripts/jianying/templates.py +0 -3
- package/lib/video-recap/skills/video-assemble/scripts/jianying/timeline_contract.py +36 -103
- package/lib/video-recap/skills/video-assemble/scripts/jianying/tracks.py +0 -8
- package/lib/video-recap/skills/video-assemble/scripts/jianying/writer.py +28 -203
- package/lib/video-recap/skills/video-assemble/scripts/lib.py +20 -34
- package/lib/video-recap/skills/video-assemble/scripts/loudness.py +332 -0
- package/lib/video-recap/skills/video-assemble/scripts/media.py +158 -38
- package/lib/video-recap/skills/video-assemble/scripts/narration_audio.py +30 -16
- package/lib/video-recap/skills/video-assemble/scripts/render_preflight.py +49 -8
- package/lib/video-recap/skills/video-assemble/scripts/source_subtitles.py +17 -6
- package/lib/video-recap/skills/video-assemble/scripts/subtitles/core.py +24 -2
- package/lib/video-recap/skills/video-assemble/scripts/subtitles/track.py +2 -5
- package/lib/video-recap/skills/video-assemble/scripts/subtitles/track_binding.py +2 -9
- package/lib/video-recap/skills/video-assemble/scripts/timeline.py +12 -34
- package/lib/video-recap/skills/video-assemble/scripts/timeline_emit.py +3 -4
- package/lib/video-recap/skills/video-assemble/scripts/visual_render.py +28 -2
- package/lib/video-recap/skills/video-cut/SKILL.md +37 -11
- package/lib/video-recap/skills/video-cut/references/shot-review.md +1 -1
- package/lib/video-recap/skills/video-cut/scripts/cut.py +2 -34
- package/lib/video-recap/skills/video-cut/scripts/cut_cli.py +96 -124
- package/lib/video-recap/skills/video-cut/scripts/cut_contract.py +97 -182
- package/lib/video-recap/skills/video-cut/scripts/cut_render.py +177 -166
- package/lib/video-recap/skills/video-cut/scripts/frame_grid.py +249 -0
- package/lib/video-recap/skills/video-cut/scripts/lib.py +0 -3
- package/lib/video-recap/skills/video-cut/scripts/media_geometry.py +173 -7
- package/lib/video-recap/skills/video-cut/scripts/sentence_boundaries.py +234 -151
- package/lib/video-recap/skills/video-cut/scripts/sentence_gate.py +189 -0
- package/lib/video-recap/skills/video-cut/scripts/shot_review.py +2 -6
- package/lib/video-recap/skills/video-recap/SKILL.md +24 -27
- package/lib/video-recap/skills/video-recap/references/audio-routing.md +8 -6
- package/lib/video-recap/skills/video-recap/references/config-playbook.md +8 -11
- package/lib/video-recap/skills/video-recap/references/data-schema.md +131 -74
- package/lib/video-recap/skills/video-recap/references/resource-library.md +17 -1
- package/lib/video-recap/skills/video-recap/scripts/doctor.py +51 -268
- package/lib/video-recap/skills/video-recap/scripts/final_qc.py +89 -254
- package/lib/video-recap/skills/video-recap/scripts/lib.py +81 -70
- package/lib/video-recap/skills/video-recap/scripts/library.py +40 -4
- package/lib/video-recap/skills/video-recap/scripts/materials.py +13 -80
- package/lib/video-recap/skills/video-recap/scripts/recap_cli.py +49 -17
- package/lib/video-recap/skills/video-recap/scripts/recap_inspect.py +1 -3
- package/lib/video-recap/skills/video-recap/scripts/recap_runner.py +86 -116
- package/lib/video-recap/skills/video-recap/scripts/recap_runtime.py +87 -23
- package/lib/video-recap/skills/video-recap/scripts/recap_source.py +0 -39
- package/lib/video-recap/skills/video-recap/scripts/recap_stage_qc.py +37 -135
- package/lib/video-recap/skills/video-recap/scripts/recap_timeline.py +118 -127
- package/lib/video-recap/skills/video-recap/scripts/resources/__init__.py +4 -0
- package/lib/video-recap/skills/video-recap/scripts/{resource_lock.py → resources/lock.py} +1 -1
- package/lib/video-recap/skills/video-recap/scripts/{project_binding.py → resources/project_binding.py} +78 -17
- package/lib/video-recap/skills/video-reference/SKILL.md +108 -0
- package/lib/video-recap/skills/video-reference/references/reference-schema.md +139 -0
- package/lib/video-recap/skills/video-reference/scripts/lib.py +85 -0
- package/lib/video-recap/skills/video-reference/scripts/reference.py +96 -0
- package/lib/video-recap/skills/video-reference/scripts/reference_check.py +553 -0
- package/lib/video-recap/skills/video-reference/scripts/reference_frames.py +122 -0
- package/lib/video-recap/skills/video-reference/scripts/reference_measure.py +253 -0
- package/lib/video-recap/skills/video-reference/scripts/reference_profile.py +271 -0
- package/lib/video-recap/skills/video-script/SKILL.md +9 -7
- package/lib/video-recap/skills/video-script/scripts/agent_text.py +18 -267
- package/lib/video-recap/skills/video-script/scripts/deslop_qc.py +12 -30
- package/lib/video-recap/skills/video-script/scripts/entry_suggestion.py +125 -0
- package/lib/video-recap/skills/video-script/scripts/evidence_bundle.py +2 -26
- package/lib/video-recap/skills/video-script/scripts/lib.py +5 -15
- package/lib/video-recap/skills/video-script/scripts/lint_summary.py +184 -0
- package/lib/video-recap/skills/video-script/scripts/narration_lint.py +120 -100
- package/lib/video-recap/skills/video-script/scripts/review.py +3 -34
- package/lib/video-recap/skills/video-script/scripts/review_grounding.py +20 -46
- package/lib/video-recap/skills/video-script/scripts/review_response.py +120 -258
- package/lib/video-recap/skills/video-script/scripts/review_runner.py +2 -10
- package/lib/video-recap/skills/video-script/scripts/speech_ownership.py +89 -5
- package/lib/video-recap/skills/video-script/scripts/validate.py +26 -34
- package/lib/video-recap/skills/video-understanding/SKILL.md +29 -4
- package/lib/video-recap/skills/video-understanding/references/data-schema.md +5 -22
- package/lib/video-recap/skills/video-understanding/scripts/agent_text.py +24 -121
- package/lib/video-recap/skills/video-understanding/scripts/asr.py +11 -5
- package/lib/video-recap/skills/video-understanding/scripts/asr_timing_evidence.py +15 -24
- package/lib/video-recap/skills/video-understanding/scripts/briefing/builder.py +59 -25
- package/lib/video-recap/skills/video-understanding/scripts/briefing/context.py +39 -3
- package/lib/video-recap/skills/video-understanding/scripts/briefing/inputs.py +15 -27
- package/lib/video-recap/skills/video-understanding/scripts/briefing/timeline.py +136 -37
- package/lib/video-recap/skills/video-understanding/scripts/consolidate.py +121 -37
- package/lib/video-recap/skills/video-understanding/scripts/detect.py +118 -16
- package/lib/video-recap/skills/video-understanding/scripts/index_normalize.py +359 -0
- package/lib/video-recap/skills/video-understanding/scripts/lib.py +62 -21
- package/lib/video-recap/skills/video-understanding/scripts/storyboard.py +181 -51
- package/lib/video-recap/skills/video-understanding/scripts/timeline_fusion.py +0 -26
- package/lib/video-recap/skills/video-understanding/scripts/understanding_brief.py +37 -14
- package/lib/video-recap/skills/video-understanding/scripts/understanding_cache.py +40 -32
- package/lib/video-recap/skills/video-understanding/scripts/understanding_runner.py +100 -127
- package/lib/video-recap/skills/video-understanding/scripts/understanding_storyboard.py +182 -30
- package/lib/video-recap/skills/video-understanding/scripts/vlm.py +30 -27
- package/lib/video-recap/skills/video-voiceover/SKILL.md +19 -12
- package/lib/video-recap/skills/video-voiceover/references/index-tts.md +3 -3
- package/lib/video-recap/skills/video-voiceover/scripts/approved_text_policy.py +3 -1
- package/lib/video-recap/skills/video-voiceover/scripts/dub.py +60 -135
- package/lib/video-recap/skills/video-voiceover/scripts/lib.py +9 -34
- package/lib/video-recap/skills/video-voiceover/scripts/providers/fish_audio.py +6 -1
- package/lib/video-recap/skills/video-voiceover/scripts/tts_audio.py +64 -1
- package/lib/video-recap/skills/video-voiceover/scripts/tts_cache.py +99 -0
- package/lib/video-recap/skills/video-voiceover/scripts/voiceover.py +177 -200
- package/package.json +45 -45
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/cg/group/inner_court_accord_five.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/cg/group/public_day15.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/assets/household/li_jiaoer.webp +0 -0
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/css/new-guofeng.css +0 -1494
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/assets.js +0 -147
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/data.js +0 -7470
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/engine.js +0 -13066
- package/lib/novel-to-game/examples/jin-ping-mei/build/app/js/text.js +0 -32
- package/lib/novel-to-game/examples/jin-ping-mei/qa/playable-model.json +0 -99
- package/lib/novel-to-game/examples/jin-ping-mei/qa/verify_playable_model.mjs +0 -152
- package/lib/video-recap/skills/video-assemble/references/foreground-compose.md +0 -118
- package/lib/video-recap/skills/video-assemble/references/jianying/empty_jy_combination_segment.json +0 -84
- package/lib/video-recap/skills/video-assemble/references/jianying/empty_jy_combination_video_material.json +0 -98
- package/lib/video-recap/skills/video-assemble/references/jianying/empty_jy_draft.json +0 -204
- package/lib/video-recap/skills/video-assemble/references/pair-media.md +0 -74
- package/lib/video-recap/skills/video-assemble/scripts/compose_foreground.py +0 -293
- package/lib/video-recap/skills/video-assemble/scripts/pair_media.py +0 -216
- package/lib/video-recap/skills/video-recap/references/shift-left-qc-schema.md +0 -23
- package/lib/video-recap/skills/video-recap/scripts/mimo_qc.py +0 -33
- package/lib/video-recap/skills/video-recap/scripts/qc/__init__.py +0 -1
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_client.py +0 -13
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_contract.py +0 -9
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_evidence.py +0 -287
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_observations.py +0 -175
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_payload.py +0 -206
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_report.py +0 -318
- package/lib/video-recap/skills/video-recap/scripts/qc/mimo_runner.py +0 -130
- package/lib/video-recap/skills/video-recap/scripts/qc_contract.py +0 -320
- package/lib/video-recap/skills/video-script/scripts/timeline_fusion.py +0 -108
- package/lib/video-recap/skills/video-understanding/scripts/brief.py +0 -9
- package/lib/video-recap/skills/video-understanding/scripts/deslop_qc.py +0 -211
- package/lib/video-recap/skills/video-understanding/scripts/narration_lint.py +0 -707
- package/lib/video-recap/skills/video-understanding/scripts/speech_ownership.py +0 -143
- /package/lib/video-recap/skills/video-cut/scripts/{narration_mapping.py → cut_qc.py} +0 -0
|
@@ -1,11 +1,17 @@
|
|
|
1
1
|
"""Generate and cache source and edited storyboard sheets."""
|
|
2
2
|
|
|
3
|
+
import json
|
|
3
4
|
from pathlib import Path
|
|
4
5
|
|
|
5
6
|
from lib import CONFIG, log
|
|
6
7
|
|
|
7
8
|
|
|
8
|
-
from storyboard import
|
|
9
|
+
from storyboard import (
|
|
10
|
+
_frame_index,
|
|
11
|
+
build_edited_storyboard,
|
|
12
|
+
build_source_storyboard,
|
|
13
|
+
work_dir_relative_pages,
|
|
14
|
+
)
|
|
9
15
|
|
|
10
16
|
|
|
11
17
|
from understanding_cache import (
|
|
@@ -25,6 +31,44 @@ def _storyboard_sample_policy():
|
|
|
25
31
|
}
|
|
26
32
|
|
|
27
33
|
|
|
34
|
+
def _relocated(payload):
|
|
35
|
+
"""A cached storyboard whose stored paths still name the work_dir it was written in.
|
|
36
|
+
|
|
37
|
+
Sidecars written before 0.6.1 stored absolute page paths; a copied work_dir hits the
|
|
38
|
+
cache (identities are size + mtime) and would otherwise point the brief at the original
|
|
39
|
+
directory. Paths are re-derived from file names, so the copy names its own pages."""
|
|
40
|
+
if not isinstance(payload, dict):
|
|
41
|
+
return payload
|
|
42
|
+
payload = dict(payload)
|
|
43
|
+
payload["page_images"] = work_dir_relative_pages(payload.get("page_images"))
|
|
44
|
+
if payload.get("edited_video_path"):
|
|
45
|
+
payload["edited_video_path"] = "edited_source.mp4" # the only edited output there is
|
|
46
|
+
return payload
|
|
47
|
+
|
|
48
|
+
|
|
49
|
+
def _reuse_cached_storyboard(json_path, meta, label):
|
|
50
|
+
"""The cached storyboard with work_dir-relative paths, or None when it is unreadable.
|
|
51
|
+
|
|
52
|
+
A legacy sidecar with absolute paths is rewritten (and re-stamped) once, so the JSON on
|
|
53
|
+
disk names this work_dir's pages too. Advisory: never raises."""
|
|
54
|
+
try:
|
|
55
|
+
cached = _load_json(json_path)
|
|
56
|
+
except (OSError, ValueError):
|
|
57
|
+
log(f"storyboard {label} 缓存命中但文件损坏,重建")
|
|
58
|
+
return None
|
|
59
|
+
relocated = _relocated(cached)
|
|
60
|
+
if relocated != cached:
|
|
61
|
+
try:
|
|
62
|
+
json_path.write_text(
|
|
63
|
+
json.dumps(relocated, ensure_ascii=False, indent=2), encoding="utf-8"
|
|
64
|
+
)
|
|
65
|
+
_write_stage_meta(json_path, meta)
|
|
66
|
+
except OSError as exc:
|
|
67
|
+
log(f"storyboard {label} 路径改写失败(忽略): {exc}")
|
|
68
|
+
log(f"storyboard 跳过 {label}(缓存匹配)")
|
|
69
|
+
return relocated
|
|
70
|
+
|
|
71
|
+
|
|
28
72
|
def _edited_storyboard_meta(clip_plan_validated_json, frames_manifest_path):
|
|
29
73
|
"""Cache key for the edited storyboard: clip plan + fps + frame-set all in the key, so an
|
|
30
74
|
fps change OR a re-validated plan invalidates it. font availability is deliberately NOT in
|
|
@@ -41,6 +85,23 @@ def _edited_storyboard_meta(clip_plan_validated_json, frames_manifest_path):
|
|
|
41
85
|
}
|
|
42
86
|
|
|
43
87
|
|
|
88
|
+
def _multi_source_edited_storyboard_meta(clip_plan_validated_json, frame_sets):
|
|
89
|
+
"""Cache key for a multi-source edited storyboard: the plan plus every used source's
|
|
90
|
+
frames-manifest identity and fps."""
|
|
91
|
+
return {
|
|
92
|
+
"schema_version": 1,
|
|
93
|
+
"stage": "edited_storyboard",
|
|
94
|
+
"inputs": {
|
|
95
|
+
"clip_plan_validated": _artifact_identity(clip_plan_validated_json),
|
|
96
|
+
"frames_manifests": {
|
|
97
|
+
sid: _artifact_identity(entry["manifest"]) for sid, entry in frame_sets.items()
|
|
98
|
+
},
|
|
99
|
+
},
|
|
100
|
+
"fps": {sid: entry["fps"] for sid, entry in frame_sets.items()},
|
|
101
|
+
"sample_policy": _storyboard_sample_policy(),
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
|
|
44
105
|
def _generate_source_storyboard(
|
|
45
106
|
work_dir, video_path, scenes, scenes_json, *, force=False
|
|
46
107
|
):
|
|
@@ -69,14 +130,8 @@ def _generate_source_storyboard(
|
|
|
69
130
|
"sample_policy": _storyboard_sample_policy(),
|
|
70
131
|
}
|
|
71
132
|
if not force and _stage_cache_valid(json_path, meta):
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
except (OSError, ValueError):
|
|
75
|
-
log(
|
|
76
|
-
"storyboard source 缓存命中但文件损坏,重建"
|
|
77
|
-
) # advisory: never raise out
|
|
78
|
-
else:
|
|
79
|
-
log("storyboard 跳过 source(缓存匹配)")
|
|
133
|
+
cached = _reuse_cached_storyboard(json_path, meta, "source")
|
|
134
|
+
if cached is not None:
|
|
80
135
|
return cached
|
|
81
136
|
result = build_source_storyboard(work_dir, video_path, scenes, CONFIG["fps"])
|
|
82
137
|
if result is not None and json_path.exists():
|
|
@@ -84,53 +139,140 @@ def _generate_source_storyboard(
|
|
|
84
139
|
return result
|
|
85
140
|
|
|
86
141
|
|
|
142
|
+
def _multi_source_frame_sets(work_dir, plan_sources, clips=None):
|
|
143
|
+
"""{source_id: {paths, numbers, fps, source_path}} for a multi-source validated plan.
|
|
144
|
+
|
|
145
|
+
Each source's frames live in its own work_dir (`sources.<id>.source_work_dir`, relative to
|
|
146
|
+
the project work_dir) at the fps recorded in that source's frames manifest — sources of
|
|
147
|
+
different lengths are extracted at different fps. A source without frames is left out, and
|
|
148
|
+
so is one no clip in `clips` uses (when given): it would not appear on the sheet, yet it
|
|
149
|
+
would enter the cache key.
|
|
150
|
+
"""
|
|
151
|
+
used = None if clips is None else {
|
|
152
|
+
clip.get("source_id") for clip in clips if isinstance(clip, dict)
|
|
153
|
+
}
|
|
154
|
+
frame_sets = {}
|
|
155
|
+
for source_id, source in plan_sources.items():
|
|
156
|
+
if used is not None and source_id not in used:
|
|
157
|
+
continue
|
|
158
|
+
if not isinstance(source, dict):
|
|
159
|
+
log(f"storyboard edited:来源 {source_id} 不是对象,跳过")
|
|
160
|
+
continue
|
|
161
|
+
rel = source.get("source_work_dir")
|
|
162
|
+
if not rel:
|
|
163
|
+
log(f"storyboard edited:来源 {source_id} 未登记 source_work_dir,跳过")
|
|
164
|
+
continue
|
|
165
|
+
source_dir = Path(work_dir) / rel
|
|
166
|
+
manifest_path = _frames_manifest_path(source_dir)
|
|
167
|
+
try:
|
|
168
|
+
fps = float(_load_json(manifest_path)["fps"])
|
|
169
|
+
except (OSError, ValueError, TypeError, KeyError):
|
|
170
|
+
log(f"storyboard edited:来源 {source_id} 缺少可读的 frames_manifest.json,跳过")
|
|
171
|
+
continue
|
|
172
|
+
paths, numbers = _frame_index(source_dir)
|
|
173
|
+
if not paths or fps <= 0:
|
|
174
|
+
log(f"storyboard edited:来源 {source_id} 的 frames/ 缺失,跳过")
|
|
175
|
+
continue
|
|
176
|
+
frame_sets[source_id] = {
|
|
177
|
+
"paths": paths,
|
|
178
|
+
"numbers": numbers,
|
|
179
|
+
"fps": fps,
|
|
180
|
+
"source_path": source.get("source_path"),
|
|
181
|
+
"manifest": manifest_path,
|
|
182
|
+
}
|
|
183
|
+
return frame_sets
|
|
184
|
+
|
|
185
|
+
|
|
87
186
|
def _generate_edited_storyboard(work_dir, source_video_path, *, force=False):
|
|
88
187
|
"""Generate (or reuse cached) the edited storyboard, GATED on clip_plan_validated.json
|
|
89
188
|
file-presence (NOT on edit_mode — recap.py forwards --edit-mode cut in BOTH passes, so the
|
|
90
189
|
validated plan presence is the only reliable pass2 signal). Advisory: returns dict|None.
|
|
190
|
+
|
|
191
|
+
A multi-source plan (one with `sources`) reads every source's frames from its own work_dir
|
|
192
|
+
under the project work_dir; the single-source plan reads work_dir/frames.
|
|
91
193
|
"""
|
|
92
194
|
if not CONFIG["storyboard"]:
|
|
93
195
|
return None
|
|
94
196
|
clip_plan_validated_json = Path(work_dir) / "clip_plan_validated.json"
|
|
95
197
|
if not clip_plan_validated_json.exists():
|
|
96
198
|
return None # pass1 (no validated plan yet) → no edited storyboard
|
|
97
|
-
frames_dir = Path(work_dir) / "frames"
|
|
98
|
-
if not frames_dir.is_dir() or not any(frames_dir.glob("frame_*.jpg")):
|
|
99
|
-
log("storyboard 跳过 edited:frames/ 缺失(缓存命中跳过了帧提取?)")
|
|
100
|
-
return None
|
|
101
|
-
json_path = Path(work_dir) / "storyboard" / "edited_storyboard.json"
|
|
102
|
-
meta = _edited_storyboard_meta(
|
|
103
|
-
clip_plan_validated_json, _frames_manifest_path(work_dir)
|
|
104
|
-
)
|
|
105
|
-
if not force and _stage_cache_valid(json_path, meta):
|
|
106
|
-
try:
|
|
107
|
-
cached = _load_json(json_path)
|
|
108
|
-
except (OSError, ValueError):
|
|
109
|
-
log(
|
|
110
|
-
"storyboard edited 缓存命中但文件损坏,重建"
|
|
111
|
-
) # advisory: never raise out
|
|
112
|
-
else:
|
|
113
|
-
log("storyboard 跳过 edited(缓存匹配)")
|
|
114
|
-
return cached
|
|
115
199
|
try:
|
|
116
200
|
clip_plan_validated = _load_json(clip_plan_validated_json)
|
|
117
201
|
except (OSError, ValueError):
|
|
118
202
|
log("storyboard 跳过 edited:clip_plan_validated.json 无法解析")
|
|
119
203
|
return None
|
|
204
|
+
if not isinstance(clip_plan_validated, dict):
|
|
205
|
+
log("storyboard 跳过 edited:clip_plan_validated.json 不是对象")
|
|
206
|
+
return None
|
|
207
|
+
plan_sources = clip_plan_validated.get("sources")
|
|
208
|
+
if isinstance(plan_sources, dict) and plan_sources:
|
|
209
|
+
clips = clip_plan_validated.get("clips")
|
|
210
|
+
if not isinstance(clips, list) or not clips:
|
|
211
|
+
log("storyboard 跳过 edited:clip_plan_validated 无 clips")
|
|
212
|
+
return None
|
|
213
|
+
frame_sets = _multi_source_frame_sets(work_dir, plan_sources, clips)
|
|
214
|
+
if not frame_sets:
|
|
215
|
+
log("storyboard 跳过 edited:剪辑用到的来源都没有 frames/")
|
|
216
|
+
return None
|
|
217
|
+
meta = _multi_source_edited_storyboard_meta(clip_plan_validated_json, frame_sets)
|
|
218
|
+
else:
|
|
219
|
+
frame_sets = None
|
|
220
|
+
frames_dir = Path(work_dir) / "frames"
|
|
221
|
+
if not frames_dir.is_dir() or not any(frames_dir.glob("frame_*.jpg")):
|
|
222
|
+
log("storyboard 跳过 edited:frames/ 缺失(缓存命中跳过了帧提取?)")
|
|
223
|
+
return None
|
|
224
|
+
meta = _edited_storyboard_meta(
|
|
225
|
+
clip_plan_validated_json, _frames_manifest_path(work_dir)
|
|
226
|
+
)
|
|
227
|
+
json_path = Path(work_dir) / "storyboard" / "edited_storyboard.json"
|
|
228
|
+
if not force and _stage_cache_valid(json_path, meta):
|
|
229
|
+
cached = _reuse_cached_storyboard(json_path, meta, "edited")
|
|
230
|
+
if cached is not None:
|
|
231
|
+
return cached
|
|
120
232
|
result = build_edited_storyboard(
|
|
121
|
-
work_dir, source_video_path, clip_plan_validated, CONFIG["fps"]
|
|
233
|
+
work_dir, source_video_path, clip_plan_validated, CONFIG["fps"],
|
|
234
|
+
source_frames=frame_sets,
|
|
122
235
|
)
|
|
123
236
|
if result is not None and json_path.exists():
|
|
124
237
|
_write_stage_meta(json_path, meta)
|
|
125
238
|
return result
|
|
126
239
|
|
|
127
240
|
|
|
241
|
+
_STORYBOARD_HEADING = "## Storyboard(先看 storyboard 再写;路径相对 work_dir)"
|
|
242
|
+
# Also matches the pre-0.6.1 heading (no "路径相对 work_dir"), so a re-run replaces it too.
|
|
243
|
+
_STORYBOARD_HEADING_PREFIX = "## Storyboard(先看 storyboard 再写"
|
|
244
|
+
|
|
245
|
+
|
|
246
|
+
def _write_edited_storyboard_only(video, work_dir):
|
|
247
|
+
"""--edited-storyboard-only: build the cut-output storyboard and point the brief at it.
|
|
248
|
+
|
|
249
|
+
For callers that write the pass-2 brief themselves (a multi-source orchestrator): no
|
|
250
|
+
extraction, ASR, VLM or API; the existing work_dir/agent_narration_brief.md, if any, gets
|
|
251
|
+
the same storyboard header a --brief-only rebuild would carry. Advisory: a storyboard
|
|
252
|
+
failure only logs.
|
|
253
|
+
"""
|
|
254
|
+
work_dir = Path(work_dir)
|
|
255
|
+
edited = _generate_edited_storyboard(work_dir, video)
|
|
256
|
+
brief_path = work_dir / "agent_narration_brief.md"
|
|
257
|
+
if edited and brief_path.exists():
|
|
258
|
+
_prepend_storyboard_brief_header(brief_path, None, edited, cut_mode=True)
|
|
259
|
+
print(
|
|
260
|
+
json.dumps(
|
|
261
|
+
{
|
|
262
|
+
"status": "edited_storyboard" if edited else "edited_storyboard_skipped",
|
|
263
|
+
"work_dir": str(work_dir),
|
|
264
|
+
"page_images": (edited or {}).get("page_images", []),
|
|
265
|
+
},
|
|
266
|
+
ensure_ascii=False,
|
|
267
|
+
)
|
|
268
|
+
)
|
|
269
|
+
|
|
270
|
+
|
|
128
271
|
def _prepend_storyboard_brief_header(
|
|
129
272
|
brief_path, source_storyboard, edited_storyboard, *, cut_mode
|
|
130
273
|
):
|
|
131
274
|
"""Post-process the RETURNED brief markdown FILE (C1): prepend a short storyboard header.
|
|
132
275
|
|
|
133
|
-
Editing the brief FILE on disk (not brief.py) keeps the brief⇄narration twin byte-identical.
|
|
134
276
|
Branches the edited-storyboard line on clip_plan_validated presence (edited_storyboard truthy)
|
|
135
277
|
so pass1 never prints a not-yet-existing path. If labels_burned:false, point to inspect clip-map.
|
|
136
278
|
"""
|
|
@@ -138,7 +280,7 @@ def _prepend_storyboard_brief_header(
|
|
|
138
280
|
return
|
|
139
281
|
try:
|
|
140
282
|
brief_path = Path(brief_path)
|
|
141
|
-
lines = [
|
|
283
|
+
lines = [_STORYBOARD_HEADING, ""]
|
|
142
284
|
any_labels_missing = False
|
|
143
285
|
if source_storyboard:
|
|
144
286
|
pages = source_storyboard.get("page_images") or []
|
|
@@ -155,6 +297,12 @@ def _prepend_storyboard_brief_header(
|
|
|
155
297
|
)
|
|
156
298
|
if not edited_storyboard.get("labels_burned", False):
|
|
157
299
|
any_labels_missing = True
|
|
300
|
+
sources = edited_storyboard.get("sources") or []
|
|
301
|
+
if sources:
|
|
302
|
+
lines.append(
|
|
303
|
+
"- 来源标签: "
|
|
304
|
+
+ ",".join(f"{s['label']}={s['source_id']}" for s in sources)
|
|
305
|
+
)
|
|
158
306
|
if any_labels_missing:
|
|
159
307
|
lines.append(
|
|
160
308
|
"- 时间戳未烧入 → 用 `inspect clip-map` 查时间(JSON sidecar 仍为权威时间源)"
|
|
@@ -162,6 +310,10 @@ def _prepend_storyboard_brief_header(
|
|
|
162
310
|
lines.append("")
|
|
163
311
|
header = "\n".join(lines) + "\n"
|
|
164
312
|
existing = brief_path.read_text(encoding="utf-8") if brief_path.exists() else ""
|
|
313
|
+
if existing.startswith(_STORYBOARD_HEADING_PREFIX):
|
|
314
|
+
# Re-running on the same brief replaces the earlier header instead of stacking.
|
|
315
|
+
end = existing.find("\n\n", existing.find("\n") + 2)
|
|
316
|
+
existing = existing[end + 2:] if end >= 0 else ""
|
|
165
317
|
brief_path.write_text(header + existing, encoding="utf-8")
|
|
166
318
|
except OSError as exc:
|
|
167
319
|
log(f"storyboard brief 头部写入失败(忽略): {exc}")
|
|
@@ -13,10 +13,25 @@ from extract import (
|
|
|
13
13
|
parse_frame_number,
|
|
14
14
|
)
|
|
15
15
|
from lib import CONFIG
|
|
16
|
-
from lib import
|
|
16
|
+
from lib import (
|
|
17
|
+
log, api_call, load_prompt, mimo_video_api_call, run_cmd, file_identity, is_moderation_refusal,
|
|
18
|
+
is_usable_overview_chunk, offline_ignored_settings, settings_match,
|
|
19
|
+
)
|
|
17
20
|
|
|
18
21
|
# ── Step 4: VLM 视觉分析 ─────────────────────────────────────────────
|
|
19
22
|
|
|
23
|
+
def vlm_prompt_payload():
|
|
24
|
+
"""The exact prompt/context text the VLM stage sends; the VLM cache key compares it by
|
|
25
|
+
equality, so the prompt that is sent and the one that is keyed are built in one place."""
|
|
26
|
+
prompt = load_prompt("VLM_DEPTH_PROMPT")
|
|
27
|
+
if not prompt:
|
|
28
|
+
raise RuntimeError("references/prompt-templates.md 缺少 VLM_DEPTH_PROMPT 模板")
|
|
29
|
+
context = CONFIG.get("context_info", "")
|
|
30
|
+
if context:
|
|
31
|
+
prompt = f"已知信息:{context}\n\n{prompt}"
|
|
32
|
+
return {"prompt_text": prompt, "context_info": context}
|
|
33
|
+
|
|
34
|
+
|
|
20
35
|
def _parse_vlm_depth_response(raw_text):
|
|
21
36
|
"""解析 VLM 深度分析响应,提取【描述】、【帧标签】和【深层分析】"""
|
|
22
37
|
if not raw_text or not raw_text.strip():
|
|
@@ -119,13 +134,7 @@ def analyze_scenes(scenes, frames, work_dir, *, resume=True):
|
|
|
119
134
|
if fps <= 0:
|
|
120
135
|
raise ValueError("CONFIG['fps'] 必须大于 0;请先运行完整 pipeline 或指定 --fps")
|
|
121
136
|
|
|
122
|
-
vlm_prompt =
|
|
123
|
-
if not vlm_prompt:
|
|
124
|
-
vlm_prompt = "仔细观察这些视频帧。分两部分输出:\n【描述】不超过80字,描述画面中正在发生什么。\n【深层分析】不超过120字,分析角色情绪、关系动态、潜台词。"
|
|
125
|
-
|
|
126
|
-
ctx = CONFIG["context_info"]
|
|
127
|
-
if ctx:
|
|
128
|
-
vlm_prompt = f"已知信息:{ctx}\n\n{vlm_prompt}"
|
|
137
|
+
vlm_prompt = vlm_prompt_payload()["prompt_text"]
|
|
129
138
|
|
|
130
139
|
# 构建帧时间映射 (frame_NNNNN.jpg -> time in seconds)。换算规则由 extract.py 独家定义:
|
|
131
140
|
# ffmpeg 首帧在 t=0 而文件编号从 1 起,所以是 (n-1)/fps,不是 n/fps。
|
|
@@ -223,8 +232,14 @@ def analyze_scenes(scenes, frames, work_dir, *, resume=True):
|
|
|
223
232
|
if not raw_response.strip():
|
|
224
233
|
raise RuntimeError("VLM 连续 3 次返回空内容")
|
|
225
234
|
|
|
235
|
+
refused = is_moderation_refusal(raw_response)
|
|
236
|
+
if refused:
|
|
237
|
+
# A moderation refusal is not a scene description; record the scene as unreadable.
|
|
238
|
+
log(f" 场景 {i+1} 被 MiMo 内容审核拒绝,记为无法识别")
|
|
226
239
|
# 解析 【描述】、【帧标签】和【深层分析】
|
|
227
|
-
description, depth_analysis, frame_facts = _parse_vlm_depth_response(
|
|
240
|
+
description, depth_analysis, frame_facts = _parse_vlm_depth_response(
|
|
241
|
+
"" if refused else raw_response
|
|
242
|
+
)
|
|
228
243
|
|
|
229
244
|
result = {
|
|
230
245
|
"scene_id": i,
|
|
@@ -235,6 +250,8 @@ def analyze_scenes(scenes, frames, work_dir, *, resume=True):
|
|
|
235
250
|
}
|
|
236
251
|
if frame_facts:
|
|
237
252
|
result["frame_facts"] = frame_facts
|
|
253
|
+
if refused:
|
|
254
|
+
result["analysis_status"] = "moderation_refused"
|
|
238
255
|
|
|
239
256
|
return i, result
|
|
240
257
|
|
|
@@ -511,11 +528,12 @@ def mimo_video_overview_cache_fresh(overview_path, video_path, scenes):
|
|
|
511
528
|
return False
|
|
512
529
|
if not isinstance(overview, dict) or overview.get("input") != "scene_chunks":
|
|
513
530
|
return False
|
|
514
|
-
|
|
531
|
+
ignored = offline_ignored_settings(CONFIG.get("mimo_video_api_key"), "mimo_video_api_url")
|
|
532
|
+
if not settings_match(overview.get("settings"), mimo_video_settings(), ignored):
|
|
515
533
|
return False
|
|
516
534
|
chunks = overview.get("chunks")
|
|
517
535
|
if not isinstance(chunks, list) or not all(
|
|
518
|
-
isinstance(chunk, dict) and
|
|
536
|
+
isinstance(chunk, dict) and is_usable_overview_chunk(chunk.get("content"))
|
|
519
537
|
for chunk in chunks
|
|
520
538
|
):
|
|
521
539
|
return False # a moderation-rejected chunk is retried, never served from cache
|
|
@@ -566,21 +584,6 @@ def _analyze_mimo_video_chunk(chunk_path, chunk):
|
|
|
566
584
|
}
|
|
567
585
|
|
|
568
586
|
|
|
569
|
-
_MIMO_REJECTION_MARKERS = (
|
|
570
|
-
"request was rejected", "considered high risk", "high risk",
|
|
571
|
-
"content policy", "cannot process", "无法处理", "内容审核", "违规",
|
|
572
|
-
)
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
def _is_mimo_chunk_usable(content):
|
|
576
|
-
"""A chunk is usable only if MiMo returned real analysis (not empty / a moderation refusal)."""
|
|
577
|
-
text = str(content or "").strip()
|
|
578
|
-
if not text:
|
|
579
|
-
return False
|
|
580
|
-
low = text.lower()
|
|
581
|
-
return not any(marker in low for marker in _MIMO_REJECTION_MARKERS)
|
|
582
|
-
|
|
583
|
-
|
|
584
587
|
def analyze_video_overview(video_path, work_dir, scenes=None):
|
|
585
588
|
"""Use MiMo video understanding over local ffmpeg scene chunks."""
|
|
586
589
|
if not CONFIG["mimo_video_overview"]:
|
|
@@ -620,7 +623,7 @@ def analyze_video_overview(video_path, work_dir, scenes=None):
|
|
|
620
623
|
f"{chunk['start']:.1f}-{chunk['end']:.1f}s"
|
|
621
624
|
)
|
|
622
625
|
chunk_result = _analyze_mimo_video_chunk(chunk_path, chunk)
|
|
623
|
-
if
|
|
626
|
+
if is_usable_overview_chunk(chunk_result["content"]):
|
|
624
627
|
chunk_results.append(chunk_result)
|
|
625
628
|
done[cache_key] = chunk_result
|
|
626
629
|
_save_mimo_partial(partial_path, done, video_path, scenes)
|
|
@@ -43,7 +43,7 @@ cut 流程先剪后配:`narration.json` 本身就是按剪后成片的输出
|
|
|
43
43
|
python3 scripts/voiceover.py --work-dir <work_dir> --narration <narration.json> \
|
|
44
44
|
[--tts-provider auto|mimo-tts|fish-audio|index-tts] \
|
|
45
45
|
[--mimo-voice 冰糖 | --voice-ref <reference-audio>] \
|
|
46
|
-
[--preserve-approved-text]
|
|
46
|
+
[--preserve-approved-text] [--allow-partial-tts]
|
|
47
47
|
```
|
|
48
48
|
|
|
49
49
|
单独运行且省略 `--narration` 时,默认读取 `work_dir/narration.json`;`--narration` 只用于指定其他路径的同格式稿件。
|
|
@@ -65,27 +65,34 @@ python3 scripts/voiceover.py --work-dir <work_dir> --narration <narration.json>
|
|
|
65
65
|
|
|
66
66
|
## 6. 运行规则
|
|
67
67
|
|
|
68
|
-
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
`
|
|
68
|
+
- 分段音频按内容缓存在 `tts_segments/cache/`:键是实读文本、实际发给供应商的语气请求(MiMo 是那句自然语言指令,语速只在 ≥+6% 或 ≤-3% 时改变措辞;Fish Audio 是数值 speed;index-tts 没有段级控制)与 TTS 设置,不含段序号和时间窗;因此段位变化让名义语速从 +5% 变成 -2% 时,MiMo 不重新合成;
|
|
69
|
+
缓存 WAV 的 `size`/`mtime_ns` 变了即失效。`narr_NNN.wav` 是指向缓存的硬链接(不支持时为副本),`tts_meta.json`
|
|
70
|
+
照旧引用它。删掉、插入或挪动某段后,只重生成文本或发给供应商的请求变了的段(名义语速随首段、末两段的位置变化);
|
|
71
|
+
旧版的 `narr_NNN.wav.cache.json` 不再读取。
|
|
72
|
+
- 批准稿保护策略属于缓存设置:严格模式往缓存键里加入策略与原稿,与默认策略(`report-over-budget-v2`,不进键)互不命中;旧版逐段缓存(含自动缩稿音频)不再读取;只有同一严格策略下、
|
|
73
|
+
`spoken_text` 完整匹配且 WAV 存在非空的缓存才可离线复用;复用时仍按当前时间窗检查,放不下照样失败。
|
|
72
74
|
- 严格 CLI 在本轮合成前把旧 `tts_meta.json` 按时间戳归档至 `tts_meta.history/`,因此失败时
|
|
73
75
|
当前路径不会继续冒充本轮成功;成功元数据通过同目录临时文件原子替换。
|
|
74
76
|
- `auto` 优先使用已配置的 MiMo,MiMo key 缺失且设置了 `FISH_API_KEY` 时使用 Fish Audio;需要可复现的 provider 选择时显式传 `--tts-provider`。
|
|
75
77
|
- 自托管 index-tts 端点只能由 `--tts-provider index-tts` 或 `TTS_PROVIDER=index-tts` 显式选择,`auto`
|
|
76
78
|
永不兜底选择它。协议、请求体、receipt 语义与缓存失效规则见 `references/index-tts.md`。
|
|
77
|
-
- Fish Audio 直接请求 WAV;默认使用“娱乐扒妹”音色(`5653cea4ac83480aaf2bf45406556185`),`FISH_TTS_REFERENCE_ID` 可覆盖。模型、音色 ID、API URL
|
|
79
|
+
- Fish Audio 直接请求 WAV;默认使用“娱乐扒妹”音色(`5653cea4ac83480aaf2bf45406556185`),`FISH_TTS_REFERENCE_ID` 可覆盖。模型、音色 ID、API URL、归一化设置或按内容计算出的语速变化时会重新生成缓存(Fish 不接收音高和情绪,它们变了不重新生成)。当前免费模型无 SLA,受 Fair Use 和官方免费期限约束。
|
|
78
80
|
- `--voice-ref` 仅用于 full/cut 解说克隆,切换到 `mimo-v2.5-tts-voiceclone`。仅在确需新合成时惰性规范化一次;
|
|
79
|
-
|
|
80
|
-
|
|
81
|
+
参考音频的路径、`size`/`mtime_ns` 或预处理版本变化会使旧缓存失效。仅在获得授权后使用,参考音频会发送到 MiMo。
|
|
82
|
+
- dub voiceclone 原始 WAV 也会按模型、提示、台词和参考音频的 `size`/`mtime_ns` 缓存;匹配重跑不再重复请求或计费,
|
|
83
|
+
`dub_manifest.json` 逐行记录 `tts_cache=hit|miss`。
|
|
84
|
+
- 合成出的段音频比按 `TTS_MIN_SPEECH_RATE`(默认 2.5 字/秒,英文按每词 1.5 字)读完全文、再加停顿与首尾静音的上限还长时,
|
|
85
|
+
视为 TTS 幻读(读完原稿后又编出一段话),按失败重试,不缓存也不交给 assemble;重试用尽则该段失败,报错写明时长与上限,
|
|
86
|
+
最后一次被拒的音频留在 `tts_segments/narr_NNN.rejected.wav` 供试听。数字(半角/全角)逐个计 1 字,`%` 计 3 字(百分之)。
|
|
87
|
+
dub 的 voiceclone 台词走同一道检查与重试(被拒的留在 `dub_tts/line_NNN_raw.rejected.wav`)。
|
|
88
|
+
旧版本缓存下的这类 WAV 在重跑时不再复用,会重新合成。设为 `0` 关闭这道检查。
|
|
81
89
|
- `TTS_WORKERS`、`TTS_TIMEOUT`、`TTS_RETRIES`、`ALLOW_PARTIAL_TTS` 用于调整并发、超时、重试与部分成功策略。
|
|
82
|
-
- dub 模式有独立的确定性门禁:`dub_lint.json
|
|
83
|
-
|
|
84
|
-
`dub.py --stage lint|review` 或 `dub.py --print-schema` 单独调用。
|
|
90
|
+
- dub 模式有独立的确定性门禁:`dub.py --stage render` 在语音克隆前写 `dub_lint.json`,
|
|
91
|
+
空行、重叠或越界译文即中止。`dub.py` 只由编排入口的 `--edit-mode dub` 调用,没有单独的手动阶段。
|
|
85
92
|
|
|
86
93
|
## 7. 能力边界
|
|
87
94
|
|
|
88
|
-
-
|
|
95
|
+
- 超窗时保留原稿并记录日志;assemble 有界提速放不下则在渲染前以 `no_safe_fit` 阻断。批准稿加 `--preserve-approved-text`,超窗即在 TTS 阶段失败。
|
|
89
96
|
- 不混流、不压低原声、不渲染字幕。
|
|
90
97
|
- 不分析视频,也不选择时间点;只为输入稿件中的既定分段配音。
|
|
91
98
|
- Fish Audio 与 IndexTTS 路径都不接受本地 `--voice-ref`;前者用已创建的 `FISH_TTS_REFERENCE_ID` 选择音色。
|
|
@@ -26,6 +26,6 @@ export INDEX_TTS_VOICE=<authorized-voice-name>
|
|
|
26
26
|
每段的 `provider_receipt` 只记录 `provider` 与“请求的 voice”(`requested_voice`)。它说明请求参数,
|
|
27
27
|
不是该声线的声学验证,也不代表已做人耳听审或音色身份验证。endpoint 不写入任何文件。
|
|
28
28
|
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
服务端模型或部署变化不会被自动察觉;更换声线实现后请删除 `tts_segments
|
|
29
|
+
缓存复用条件:`tts_segments/cache/` 中按文本与设置(含 `index_tts_voice`)寻址的条目存在、缓存 WAV 的
|
|
30
|
+
`size`/`mtime_ns` 未变,且 sidecar 的 receipt 对应当前 voice;缓存命中复用 sidecar 中的 receipt,不能现场补造。
|
|
31
|
+
服务端模型或部署变化不会被自动察觉;更换声线实现后请删除 `tts_segments/cache/` 强制重合成。
|
|
@@ -7,7 +7,9 @@ from datetime import datetime, timezone
|
|
|
7
7
|
from pathlib import Path
|
|
8
8
|
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
# Recorded per block in tts_meta.json; not part of the TTS cache key (only the strict policy
|
|
11
|
+
# adds fields there), so renaming it invalidates no cached audio.
|
|
12
|
+
LEGACY_TEXT_POLICY = "report-over-budget-v2"
|
|
11
13
|
PRESERVE_APPROVED_TEXT_POLICY = "preserve-approved-text-v1"
|
|
12
14
|
|
|
13
15
|
|