@hypit/hypit 0.1.4 → 0.1.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -2
- package/dist/public/caption.d.ts +206 -183
- package/dist/public/composition.d.ts +25 -3
- package/dist/public/endpoint-kit.d.ts +24 -2
- package/dist/public/generation.d.ts +5 -1
- package/dist/public/hyperframes.d.ts +19 -1
- package/dist/public/narrative.d.ts +17 -3
- package/dist/public/performance.d.ts +1590 -0
- package/dist/public/program-space.d.ts +179 -32
- package/dist/public/runtime-kit.d.ts +21 -8
- package/dist/public/sound.d.ts +822 -0
- package/dist/public/speech.d.ts +41 -0
- package/dist/public/studio-adapter.d.ts +77 -15
- package/dist/public/temporal-markup.d.ts +9 -44
- package/dist/public/temporal.d.ts +37 -28
- package/dist/public/{semantic-track.d.ts → timeline.d.ts} +62 -49
- package/examples/README.md +18 -10
- package/examples/minimal-author-package/packages/example-component/preview/preview.svml +19 -12
- package/examples/minimal-author-package/packages/example-component/src/component.ts +5 -5
- package/examples/minimal-author-package/packages/example-component/src/fragment.ts +3 -5
- package/examples/minimal-author-package/packages/example-component/src/manifest.ts +8 -10
- package/examples/minimal-author-package/packages/example-component/src/surface.ts +2 -2
- package/examples/provider-package/README.md +78 -0
- package/examples/provider-package/hypit.runtime.json +12 -0
- package/examples/provider-package/packages/provider-images/package.json +11 -0
- package/examples/provider-package/packages/provider-images/src/activation.ts +24 -0
- package/examples/provider-package/packages/provider-images/src/provider.ts +122 -0
- package/examples/provider-package/packages/provider-images/tsconfig.json +9 -0
- package/examples/semantic-composition/README.md +6 -0
- package/examples/semantic-composition/chat.svml +6 -5
- package/examples/semantic-composition/packages/chat-scene/README.md +3 -3
- package/examples/semantic-composition/packages/chat-scene/src/activation.ts +18 -21
- package/examples/semantic-composition/packages/chat-scene/src/render.ts +4 -4
- package/examples/semantic-composition/packages/performance-styles/README.md +42 -0
- package/examples/semantic-composition/packages/performance-styles/package.json +25 -0
- package/examples/semantic-composition/packages/performance-styles/src/activation.ts +57 -0
- package/examples/semantic-composition/packages/performance-styles/src/render.ts +54 -0
- package/examples/semantic-composition/packages/performance-styles/src/studio.ts +28 -0
- package/examples/semantic-composition/packages/performance-styles/tsconfig.json +18 -0
- package/examples/semantic-composition/packages/responsive-explainer/README.md +16 -9
- package/examples/semantic-composition/packages/responsive-explainer/src/activation.ts +12 -12
- package/examples/semantic-composition/packages/responsive-explainer/src/render.ts +5 -5
- package/examples/semantic-composition/packages/sound-styles/README.md +36 -0
- package/examples/semantic-composition/packages/sound-styles/package.json +25 -0
- package/examples/semantic-composition/packages/sound-styles/src/activation.ts +53 -0
- package/examples/semantic-composition/packages/sound-styles/src/render.ts +14 -0
- package/examples/semantic-composition/packages/sound-styles/tsconfig.json +18 -0
- package/package.json +32 -15
- package/packages/audio-track/README.md +20 -13
- package/packages/audio-track/package.json +1 -1
- package/packages/audio-track/src/component.ts +4 -4
- package/packages/audio-track/src/fragment.ts +6 -5
- package/packages/audio-track/src/index.ts +1 -1
- package/packages/audio-track/src/manifest.ts +17 -16
- package/packages/audio-track/src/program.ts +21 -25
- package/packages/audio-track/src/surface.ts +19 -18
- package/packages/audio-track-studio/README.md +23 -0
- package/packages/audio-track-studio/src/index.ts +28 -17
- package/packages/author-kit/README.md +1 -1
- package/packages/background-removal/README.md +4 -7
- package/packages/browser-capture/package.json +1 -1
- package/packages/build-result/README.md +11 -0
- package/packages/build-result/src/decode.ts +2 -0
- package/packages/build-result/src/execution-log.ts +17 -0
- package/packages/build-result/src/index.ts +1 -0
- package/packages/build-result/src/store.ts +14 -1
- package/packages/build-result/src/types.ts +3 -0
- package/packages/build-result-s3/src/repository.ts +4 -0
- package/packages/caption/README.md +61 -73
- package/packages/caption/package.json +3 -2
- package/packages/caption/src/activation.ts +2 -2
- package/packages/caption/src/component.ts +20 -5
- package/packages/caption/src/display.ts +3 -18
- package/packages/caption/src/fragment.ts +3 -7
- package/packages/caption/src/index.ts +6 -5
- package/packages/caption/src/manifest.ts +29 -46
- package/packages/caption/src/style.ts +24 -135
- package/packages/caption/src/surface.ts +12 -92
- package/packages/caption/src/temporalize.ts +18 -53
- package/packages/caption/src/types.ts +8 -35
- package/packages/caption/src/visibility.ts +23 -0
- package/packages/caption-fine/README.md +14 -9
- package/packages/caption-fine/package.json +4 -2
- package/packages/caption-fine/src/component.ts +3 -3
- package/packages/caption-fine/src/fragment.ts +7 -19
- package/packages/caption-fine/src/index.ts +2 -0
- package/packages/caption-fine/src/manifest.ts +26 -17
- package/packages/caption-fine/src/recipe.ts +7 -0
- package/packages/caption-fine/src/render.ts +14 -13
- package/packages/caption-fine/src/schedule.ts +39 -35
- package/packages/caption-fine/src/style.ts +10 -9
- package/packages/caption-fine/src/surface.ts +62 -35
- package/packages/caption-fine/src/types.ts +2 -0
- package/packages/caption-fine-studio/README.md +24 -2
- package/packages/caption-fine-studio/package.json +3 -1
- package/packages/caption-fine-studio/src/activation.ts +3 -3
- package/packages/caption-fine-studio/src/index.ts +54 -42
- package/packages/cli/README.md +57 -4
- package/packages/cli/package.json +1 -0
- package/packages/cli/src/arguments.ts +25 -6
- package/packages/cli/src/build-planning.ts +3 -2
- package/packages/cli/src/command.ts +6 -0
- package/packages/cli/src/commands/environment.ts +28 -18
- package/packages/cli/src/commands/execution.ts +81 -26
- package/packages/cli/src/commands/results.ts +7 -11
- package/packages/cli/src/index.ts +2 -2
- package/packages/cli/src/machine-view.ts +3 -2
- package/packages/cli/src/main.ts +34 -42
- package/packages/cli/src/observation.ts +7 -1
- package/packages/cli/src/output.ts +65 -23
- package/packages/cli/src/view.ts +64 -18
- package/packages/comment-sticker/README.md +3 -3
- package/packages/comment-sticker/package.json +1 -1
- package/packages/comment-sticker/src/component.ts +4 -4
- package/packages/comment-sticker/src/fragment.ts +5 -4
- package/packages/comment-sticker/src/manifest.ts +7 -6
- package/packages/comment-sticker/src/program.ts +9 -9
- package/packages/comment-sticker/src/surface.ts +6 -7
- package/packages/composition/README.md +21 -0
- package/packages/composition/src/audio-presentation.ts +44 -0
- package/packages/composition/src/index.ts +3 -0
- package/packages/composition/src/schema.ts +8 -3
- package/packages/composition/src/track.ts +15 -1
- package/packages/credential-store-env/README.md +1 -1
- package/packages/deck-track/README.md +3 -3
- package/packages/deck-track/package.json +1 -1
- package/packages/deck-track/src/component.ts +5 -5
- package/packages/deck-track/src/fragment.ts +6 -5
- package/packages/deck-track/src/lower.ts +12 -12
- package/packages/deck-track/src/manifest.ts +8 -7
- package/packages/deck-track/src/program.ts +13 -16
- package/packages/deck-track/src/surface.ts +9 -10
- package/packages/driver-node/src/driver.ts +41 -3
- package/packages/elevenlabs-speech/README.md +27 -0
- package/packages/{provider-xiaomi-mimo → elevenlabs-speech}/package.json +5 -8
- package/packages/elevenlabs-speech/src/activation.ts +26 -0
- package/packages/elevenlabs-speech/src/fragment.ts +11 -0
- package/packages/elevenlabs-speech/src/index.ts +111 -0
- package/packages/elevenlabs-speech/src/surface.ts +167 -0
- package/packages/endpoint-kit/README.md +24 -0
- package/packages/endpoint-kit/src/index.ts +7 -1
- package/packages/film/README.md +12 -7
- package/packages/film/package.json +1 -1
- package/packages/film/src/component.ts +4 -4
- package/packages/film/src/fragment.ts +6 -5
- package/packages/film/src/manifest.ts +11 -10
- package/packages/film/src/program.ts +5 -5
- package/packages/film/src/recipe.ts +2 -1
- package/packages/film/src/surface.ts +6 -6
- package/packages/film-studio/package.json +1 -1
- package/packages/film-studio/src/index.ts +2 -2
- package/packages/fishaudio-speech/README.md +33 -0
- package/packages/fishaudio-speech/package.json +21 -0
- package/packages/fishaudio-speech/src/activation.ts +32 -0
- package/packages/fishaudio-speech/src/fragment.ts +11 -0
- package/packages/fishaudio-speech/src/index.ts +156 -0
- package/packages/fishaudio-speech/src/surface.ts +192 -0
- package/packages/generation/README.md +11 -1
- package/packages/generation/src/mapping.ts +28 -30
- package/packages/gpt-image/README.md +1 -2
- package/packages/grok-imagine/README.md +1 -2
- package/packages/hyperframes/README.md +7 -3
- package/packages/hyperframes/src/browser-program.ts +11 -2
- package/packages/hyperframes/src/document.ts +15 -2
- package/packages/interview-emoji-reveal/README.md +4 -4
- package/packages/interview-emoji-reveal/package.json +1 -1
- package/packages/interview-emoji-reveal/src/component.ts +4 -4
- package/packages/interview-emoji-reveal/src/fragment.ts +6 -5
- package/packages/interview-emoji-reveal/src/manifest.ts +9 -8
- package/packages/interview-emoji-reveal/src/program.ts +14 -14
- package/packages/interview-emoji-reveal/src/surface.ts +8 -9
- package/packages/media-execution/README.md +7 -3
- package/packages/media-execution/src/audio-presentation.ts +23 -0
- package/packages/media-execution/src/execute.ts +2 -0
- package/packages/media-pipeline/README.md +7 -3
- package/packages/media-pipeline/src/audio-plan.ts +4 -1
- package/packages/media-pipeline/src/component.ts +16 -2
- package/packages/media-pipeline/src/manifest.ts +3 -1
- package/packages/media-pipeline/src/types.ts +2 -1
- package/packages/media-track/README.md +14 -47
- package/packages/media-track/package.json +1 -1
- package/packages/media-track/preview/preview.svml +5 -5
- package/packages/media-track/src/author.ts +12 -6
- package/packages/media-track/src/component.ts +10 -16
- package/packages/media-track/src/fragment.ts +8 -7
- package/packages/media-track/src/index.ts +3 -1
- package/packages/media-track/src/lower.ts +7 -7
- package/packages/media-track/src/manifest.ts +16 -38
- package/packages/media-track/src/program.ts +62 -67
- package/packages/media-track/src/sampling.ts +6 -6
- package/packages/media-track/src/sequence.ts +8 -8
- package/packages/media-track/src/surface.ts +25 -43
- package/packages/media-track-studio/README.md +9 -0
- package/packages/media-track-studio/src/index.ts +16 -11
- package/packages/mimo-speech/README.md +1 -1
- package/packages/minimax-h3/README.md +1 -1
- package/packages/nano-banana/README.md +1 -1
- package/packages/narrative/README.md +23 -1
- package/packages/narrative/src/identity.ts +11 -0
- package/packages/narrative/src/index.ts +3 -0
- package/packages/narrative/src/schema.ts +28 -27
- package/packages/narrative/src/selection.ts +52 -0
- package/packages/narrative/src/types.ts +2 -0
- package/packages/package-loader-node/README.md +30 -0
- package/packages/package-loader-node/package.json +5 -2
- package/packages/package-loader-node/src/distribution-resolution.ts +10 -3
- package/packages/package-loader-node/src/index.ts +1 -0
- package/packages/package-loader-node/src/loader.ts +12 -15
- package/packages/package-loader-node/src/location.ts +27 -1
- package/packages/package-loader-node/src/module-scope.ts +183 -0
- package/packages/package-loader-node/src/types.ts +4 -0
- package/packages/performance/README.md +83 -0
- package/packages/performance/package.json +30 -0
- package/packages/performance/src/activation.ts +11 -0
- package/packages/performance/src/component.ts +46 -0
- package/packages/performance/src/index.ts +6 -0
- package/packages/performance/src/manifest.ts +53 -0
- package/packages/{media-track/src/performance.ts → performance/src/media.ts} +9 -10
- package/packages/performance/src/program.ts +51 -0
- package/packages/performance/src/style.ts +35 -0
- package/packages/performance/src/surface.ts +99 -0
- package/packages/performance-studio/README.md +25 -0
- package/packages/performance-studio/package.json +21 -0
- package/packages/performance-studio/src/activation.ts +6 -0
- package/packages/performance-studio/src/index.ts +84 -0
- package/packages/program-space/README.md +9 -18
- package/packages/program-space/src/activation.ts +1 -5
- package/packages/program-space/src/index.ts +2 -11
- package/packages/program-space/src/surface.ts +1 -20
- package/packages/project-context-node/README.md +20 -0
- package/packages/project-context-node/package.json +10 -0
- package/packages/project-context-node/src/index.ts +3 -0
- package/packages/{cli → project-context-node}/src/runtime-selection.ts +7 -2
- package/packages/provider-hyperframes-local/README.md +43 -15
- package/packages/provider-hyperframes-local/src/activation.ts +4 -3
- package/packages/provider-hyperframes-local/src/capture-process.ts +26 -4
- package/packages/provider-hyperframes-local/src/capture.ts +96 -26
- package/packages/provider-hyperframes-local/src/concurrency.ts +86 -0
- package/packages/provider-hyperframes-local/src/opaque-capture.ts +92 -0
- package/packages/provider-hyperframes-local/src/options.ts +2 -0
- package/packages/provider-hyperframes-local/src/program.ts +6 -3
- package/packages/provider-hyperframes-local/src/progress.ts +53 -0
- package/packages/provider-hyperframes-local/src/provider.ts +18 -5
- package/packages/provider-hyperframes-local/src/render.ts +48 -7
- package/packages/provider-hypihub/README.md +48 -17
- package/packages/provider-hypihub/package.json +2 -0
- package/packages/provider-hypihub/src/mapping.ts +31 -7
- package/packages/provider-hypihub/src/oauth.ts +6 -0
- package/packages/provider-hypihub/src/provider.ts +25 -13
- package/packages/provider-hypihub/src/routes.ts +1 -2
- package/packages/provider-hypihub/src/upload.ts +2 -0
- package/packages/provider-media-local/README.md +3 -4
- package/packages/provider-whisperx-local/src/program.ts +1 -0
- package/packages/provider-whisperx-local/src/provider.ts +3 -0
- package/packages/ranking/README.md +8 -6
- package/packages/ranking/package.json +1 -1
- package/packages/ranking/src/component.ts +6 -6
- package/packages/ranking/src/fragment.ts +11 -7
- package/packages/ranking/src/manifest.ts +21 -14
- package/packages/ranking/src/render.ts +22 -26
- package/packages/ranking/src/schedule.ts +16 -19
- package/packages/ranking/src/surface.ts +12 -13
- package/packages/ranking-studio/README.md +18 -0
- package/packages/ranking-studio/src/index.ts +61 -9
- package/packages/render-hyperframes/README.md +7 -7
- package/packages/render-hyperframes/package.json +1 -1
- package/packages/render-hyperframes/src/component.ts +11 -1
- package/packages/render-hyperframes/src/fragment.ts +7 -4
- package/packages/render-hyperframes/src/manifest.ts +4 -4
- package/packages/render-hyperframes/src/surface.ts +6 -6
- package/packages/runtime/README.md +8 -1
- package/packages/runtime/src/execution.ts +15 -4
- package/packages/runtime/src/index.ts +2 -1
- package/packages/runtime/src/log.ts +51 -0
- package/packages/runtime/src/types.ts +5 -8
- package/packages/runtime-host-node/README.md +30 -0
- package/packages/runtime-host-node/package.json +1 -0
- package/packages/runtime-host-node/src/index.ts +24 -9
- package/packages/runtime-host-node/src/packages.ts +47 -50
- package/packages/runtime-kit/README.md +5 -0
- package/packages/runtime-kit/src/index.ts +3 -1
- package/packages/runtime-local/README.md +92 -4
- package/packages/runtime-local/src/config.ts +202 -56
- package/packages/runtime-local/src/control.ts +9 -1
- package/packages/runtime-local/src/executor.ts +66 -0
- package/packages/runtime-local/src/host.ts +37 -17
- package/packages/runtime-local/src/log.ts +48 -0
- package/packages/runtime-local/src/programs.ts +2 -1
- package/packages/runtime-local/src/result-writer.ts +2 -0
- package/packages/runtime-local/src/runtime.ts +8 -6
- package/packages/runtime-local/src/supervisor.ts +142 -0
- package/packages/runtime-local/src/types.ts +9 -2
- package/packages/runtime-local/src/worker-process.ts +7 -24
- package/packages/runtime-local/src/worker.ts +55 -83
- package/packages/screen-overlay/README.md +3 -3
- package/packages/screen-overlay/package.json +1 -1
- package/packages/screen-overlay/src/component.ts +3 -3
- package/packages/screen-overlay/src/fragment.ts +5 -4
- package/packages/screen-overlay/src/manifest.ts +7 -6
- package/packages/screen-overlay/src/program.ts +9 -9
- package/packages/screen-overlay/src/surface.ts +6 -7
- package/packages/script/README.md +15 -2
- package/packages/script/src/manifest.ts +2 -2
- package/packages/script/src/narrative.ts +1 -0
- package/packages/script/src/surface.ts +4 -3
- package/packages/script/src/types.ts +1 -1
- package/packages/script-studio/package.json +9 -3
- package/packages/script-studio/src/index.ts +2 -0
- package/packages/script-studio/src/projection.ts +105 -0
- package/packages/seedance/README.md +30 -1
- package/packages/seedance/src/index.ts +18 -7
- package/packages/seedance/src/surface.ts +17 -4
- package/packages/seedance-kits/README.md +7 -1
- package/packages/seedream/README.md +1 -1
- package/packages/sound/README.md +81 -0
- package/packages/sound/package.json +25 -0
- package/packages/sound/src/activation.ts +11 -0
- package/packages/sound/src/component.ts +28 -0
- package/packages/sound/src/index.ts +5 -0
- package/packages/sound/src/manifest.ts +47 -0
- package/packages/sound/src/program.ts +84 -0
- package/packages/sound/src/style.ts +34 -0
- package/packages/sound/src/surface.ts +90 -0
- package/packages/sound-studio/README.md +22 -0
- package/packages/sound-studio/package.json +20 -0
- package/packages/sound-studio/src/activation.ts +6 -0
- package/packages/sound-studio/src/index.ts +74 -0
- package/packages/speech/README.md +2 -2
- package/packages/speech-alignment/src/local.ts +8 -0
- package/packages/speech-alignment/src/locate.ts +1 -1
- package/packages/store-sqlite/README.md +7 -0
- package/packages/store-sqlite/src/store.ts +84 -83
- package/packages/studio/INSPECTOR.md +15 -2
- package/packages/studio/README.md +88 -14
- package/packages/studio/package.json +8 -7
- package/packages/studio/src/companion-assembly.ts +1 -0
- package/packages/studio/src/domain.ts +1 -1
- package/packages/studio/src/fallback-companions.ts +3 -3
- package/packages/studio/src/feedback-server.ts +51 -0
- package/packages/studio/src/feedback-store.ts +66 -0
- package/packages/studio/src/feedback.ts +60 -0
- package/packages/studio/src/parameter-values.ts +43 -1
- package/packages/studio/src/parameters.ts +114 -24
- package/packages/studio/src/preview/render.ts +5 -1
- package/packages/studio/src/preview/runtime-shim.ts +49 -3
- package/packages/studio/src/programme.ts +14 -30
- package/packages/studio/src/server.ts +10 -6
- package/packages/studio/src/shared.ts +4 -2
- package/packages/studio/src/snapshot.ts +21 -132
- package/packages/studio/src/studio-preflight.ts +3 -3
- package/packages/studio/src/studio-registry.ts +50 -6
- package/packages/studio/src/style.css +209 -48
- package/packages/studio/src/ui/comments.ts +233 -0
- package/packages/studio/src/ui/icons.ts +5 -0
- package/packages/studio/src/ui/main.ts +145 -86
- package/packages/studio/src/ui/overlay.ts +12 -8
- package/packages/studio/src/ui/scrub-preview.ts +108 -0
- package/packages/studio/src/ui/stage.ts +67 -4
- package/packages/studio/src/ui/timeline.ts +177 -108
- package/packages/studio/start.ts +4 -2
- package/packages/studio-adapter/README.md +130 -11
- package/packages/studio-adapter/src/index.ts +100 -13
- package/packages/temporal/README.md +6 -6
- package/packages/temporal/package.json +1 -1
- package/packages/temporal/src/component.ts +5 -6
- package/packages/temporal/src/index.ts +8 -6
- package/packages/temporal/src/location.ts +11 -11
- package/packages/temporal/src/projection.ts +19 -19
- package/packages/temporal/src/rational.ts +4 -4
- package/packages/temporal-markup/EDITING.md +2 -2
- package/packages/temporal-markup/README.md +27 -11
- package/packages/temporal-markup/package.json +1 -1
- package/packages/temporal-markup/src/index.ts +20 -25
- package/packages/temporal-markup/src/space.ts +10 -41
- package/packages/timeline/README.md +31 -0
- package/packages/{semantic-track → timeline}/package.json +1 -1
- package/packages/timeline/src/activation.ts +8 -0
- package/packages/timeline/src/component.ts +24 -0
- package/packages/timeline/src/identity.ts +60 -0
- package/packages/timeline/src/index.ts +6 -0
- package/packages/{semantic-track → timeline}/src/location.ts +20 -20
- package/packages/timeline/src/manifest.ts +39 -0
- package/packages/{semantic-track → timeline}/src/projection.ts +14 -14
- package/packages/timeline/src/types.ts +24 -0
- package/packages/timeline-author/README.md +68 -0
- package/packages/{speech-track → timeline-author}/package.json +6 -4
- package/packages/timeline-author/src/activation.ts +22 -0
- package/packages/{speech-track → timeline-author}/src/component.ts +15 -13
- package/packages/{speech-track → timeline-author}/src/fragment.ts +14 -23
- package/packages/timeline-author/src/index.ts +7 -0
- package/packages/timeline-author/src/manifest.ts +78 -0
- package/packages/timeline-author/src/program.ts +107 -0
- package/packages/timeline-author/src/surface.ts +46 -0
- package/packages/timeline-author/src/types.ts +6 -0
- package/packages/typography-track/README.md +4 -4
- package/packages/typography-track/package.json +1 -1
- package/packages/typography-track/src/component.ts +4 -4
- package/packages/typography-track/src/fragment.ts +4 -3
- package/packages/typography-track/src/manifest.ts +11 -9
- package/packages/typography-track/src/program.ts +15 -15
- package/packages/typography-track/src/surface.ts +17 -18
- package/packages/video-cli/README.md +4 -1
- package/packages/video-cli/package.json +2 -1
- package/packages/video-cli/src/creation.ts +14 -4
- package/packages/video-cli/src/index.ts +1 -1
- package/packages/video-cli/src/studio-distribution.ts +2 -1
- package/packages/volcengine-matting/README.md +3 -3
- package/packages/whisperx/README.md +1 -1
- package/services/whisperx/README.md +14 -0
- package/services/whisperx/pyproject.toml +2 -0
- package/services/whisperx/uv.lock +20 -20
- package/packages/provider-hyperframes-aws-lambda/README.md +0 -30
- package/packages/provider-hyperframes-aws-lambda/package.json +0 -30
- package/packages/provider-hyperframes-aws-lambda/src/activation.ts +0 -97
- package/packages/provider-hyperframes-aws-lambda/src/client.ts +0 -114
- package/packages/provider-hyperframes-aws-lambda/src/index.ts +0 -21
- package/packages/provider-hyperframes-aws-lambda/src/provider.ts +0 -522
- package/packages/provider-kie/README.md +0 -134
- package/packages/provider-kie/package.json +0 -43
- package/packages/provider-kie/src/activation.ts +0 -71
- package/packages/provider-kie/src/index.ts +0 -10
- package/packages/provider-kie/src/mapping.ts +0 -182
- package/packages/provider-kie/src/provider.ts +0 -657
- package/packages/provider-kie/src/routes.ts +0 -150
- package/packages/provider-xiaomi-mimo/README.md +0 -32
- package/packages/provider-xiaomi-mimo/src/activation.ts +0 -57
- package/packages/provider-xiaomi-mimo/src/index.ts +0 -5
- package/packages/provider-xiaomi-mimo/src/provider.ts +0 -214
- package/packages/runtime/src/environment.ts +0 -6
- package/packages/semantic-track/README.md +0 -21
- package/packages/semantic-track/src/activation.ts +0 -8
- package/packages/semantic-track/src/component.ts +0 -24
- package/packages/semantic-track/src/identity.ts +0 -58
- package/packages/semantic-track/src/index.ts +0 -6
- package/packages/semantic-track/src/manifest.ts +0 -38
- package/packages/semantic-track/src/types.ts +0 -26
- package/packages/speech-track/README.md +0 -51
- package/packages/speech-track/src/activation.ts +0 -19
- package/packages/speech-track/src/index.ts +0 -7
- package/packages/speech-track/src/manifest.ts +0 -73
- package/packages/speech-track/src/program.ts +0 -64
- package/packages/speech-track/src/surface.ts +0 -35
- package/packages/speech-track/src/types.ts +0 -6
- package/packages/speech-track-studio/README.md +0 -3
- package/packages/speech-track-studio/package.json +0 -14
- package/packages/speech-track-studio/src/activation.ts +0 -7
- package/packages/speech-track-studio/src/index.ts +0 -40
- /package/packages/{cli → project-context-node}/src/project-context.ts +0 -0
- /package/packages/{speech-track → timeline-author}/preview/Track.png +0 -0
|
@@ -1,10 +1,10 @@
|
|
|
1
|
+
import type { Timeline } from "@hypit/timeline";
|
|
1
2
|
import type {
|
|
2
3
|
VisualSamplingRational,
|
|
3
4
|
VisualSamplingSegment,
|
|
4
5
|
VisualTimedSampling,
|
|
5
6
|
} from "@hypit/composition";
|
|
6
7
|
import { assertProgramSpaceIdentity } from "@hypit/program-space";
|
|
7
|
-
import type { ProgramSpace } from "@hypit/program-space";
|
|
8
8
|
|
|
9
9
|
import type { MediaVisualOccupancy, MediaVisualTrim } from "./types.js";
|
|
10
10
|
|
|
@@ -58,21 +58,21 @@ function assertOccupancy(value: MediaVisualOccupancy): void {
|
|
|
58
58
|
* frame. This avoids a second, renderer-specific frame quantizer.
|
|
59
59
|
*/
|
|
60
60
|
export function resolveVisualSampling(input: {
|
|
61
|
-
readonly
|
|
61
|
+
readonly timeline: Timeline;
|
|
62
62
|
readonly sourceFrameRate: { readonly numerator: number; readonly denominator: number };
|
|
63
63
|
readonly sourceFrameCount: number;
|
|
64
64
|
readonly targetFrameCount: number;
|
|
65
65
|
readonly trim?: MediaVisualTrim;
|
|
66
66
|
readonly occupancy: MediaVisualOccupancy;
|
|
67
67
|
}): VisualTimedSampling {
|
|
68
|
-
assertProgramSpaceIdentity(input.
|
|
68
|
+
assertProgramSpaceIdentity(input.timeline);
|
|
69
69
|
assert(Number.isSafeInteger(input.sourceFrameCount) && input.sourceFrameCount > 0,
|
|
70
70
|
"Timed visual source frame count is invalid.");
|
|
71
71
|
assert(Number.isSafeInteger(input.targetFrameCount) && input.targetFrameCount > 0,
|
|
72
72
|
"Timed visual target frame count is invalid.");
|
|
73
|
-
assert(input.sourceFrameRate.numerator === input.
|
|
74
|
-
&& input.sourceFrameRate.denominator === input.
|
|
75
|
-
"Timed visual source must be normalized to
|
|
73
|
+
assert(input.sourceFrameRate.numerator === input.timeline.frameRate.numerator
|
|
74
|
+
&& input.sourceFrameRate.denominator === input.timeline.frameRate.denominator,
|
|
75
|
+
"Timed visual source must be normalized to Timeline frame rate before Media authoring.");
|
|
76
76
|
assertOccupancy(input.occupancy);
|
|
77
77
|
const trim = input.trim ?? { startFrame: 0, endFrameExclusive: input.sourceFrameCount };
|
|
78
78
|
assert(Number.isSafeInteger(trim.startFrame) && Number.isSafeInteger(trim.endFrameExclusive)
|
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import type { Timeline } from "@hypit/timeline";
|
|
1
2
|
import type {
|
|
2
3
|
VisualAnimation,
|
|
3
4
|
VisualEasing,
|
|
@@ -5,7 +6,6 @@ import type {
|
|
|
5
6
|
VisualStyleDeclaration,
|
|
6
7
|
} from "@hypit/composition";
|
|
7
8
|
import { assertProgramSpaceIdentity, programSpaceFrameCount } from "@hypit/program-space";
|
|
8
|
-
import type { ProgramSpace } from "@hypit/program-space";
|
|
9
9
|
import { canonicalize } from "@hypit/protocol";
|
|
10
10
|
import { assertCanvasSpace, assertSpatialFrame } from "@hypit/spatial";
|
|
11
11
|
import type { CanvasSpace } from "@hypit/spatial";
|
|
@@ -109,12 +109,12 @@ function appendMediaSequenceMemberAtFrame(
|
|
|
109
109
|
|
|
110
110
|
export function appendMediaSequenceMember(
|
|
111
111
|
set: MediaSequenceMemberSet,
|
|
112
|
-
|
|
112
|
+
timeline: Timeline,
|
|
113
113
|
layers: MediaLayerSet,
|
|
114
114
|
spec: MediaSequenceMemberSpec,
|
|
115
115
|
activation: TemporalInstant,
|
|
116
116
|
): MediaSequenceMemberSet {
|
|
117
|
-
assertTemporalInstantFor(activation, { subjectId: spec.id, space });
|
|
117
|
+
assertTemporalInstantFor(activation, { subjectId: spec.id, space: timeline });
|
|
118
118
|
return appendMediaSequenceMemberAtFrame(set, layers, spec, activation.frame);
|
|
119
119
|
}
|
|
120
120
|
|
|
@@ -180,7 +180,7 @@ function resolvedHandoff(
|
|
|
180
180
|
|
|
181
181
|
export function resolveMediaSequence(
|
|
182
182
|
header: MediaTrackHeader,
|
|
183
|
-
|
|
183
|
+
timeline: Timeline,
|
|
184
184
|
canvas: CanvasSpace,
|
|
185
185
|
memberSet: MediaSequenceMemberSet,
|
|
186
186
|
frame: MediaSequenceProgram["frame"],
|
|
@@ -188,7 +188,7 @@ export function resolveMediaSequence(
|
|
|
188
188
|
sounds: MediaSoundSet,
|
|
189
189
|
terminalFrame: number,
|
|
190
190
|
): MediaSequenceProgram {
|
|
191
|
-
assertProgramSpaceIdentity(
|
|
191
|
+
assertProgramSpaceIdentity(timeline);
|
|
192
192
|
assertCanvasSpace(canvas);
|
|
193
193
|
assertMediaSequenceMemberSet(memberSet);
|
|
194
194
|
assertMediaSequenceSpec(spec);
|
|
@@ -205,7 +205,7 @@ export function resolveMediaSequence(
|
|
|
205
205
|
terminalFrame,
|
|
206
206
|
triggers: memberSet.members.map((member) => ({ id: member.id, frame: member.activationFrame })),
|
|
207
207
|
});
|
|
208
|
-
assert(terminalFrame <= programSpaceFrameCount(
|
|
208
|
+
assert(terminalFrame <= programSpaceFrameCount(timeline), `Media Sequence ${spec.id} exits Timeline.`);
|
|
209
209
|
const handoffs = spec.handoffs.map((handoff, index) => {
|
|
210
210
|
const from = memberSet.members[index]!;
|
|
211
211
|
const to = memberSet.members[index + 1]!;
|
|
@@ -360,7 +360,7 @@ export function sequenceMemberHandoffAnimation(
|
|
|
360
360
|
|
|
361
361
|
export function lowerMediaSequencePresents(
|
|
362
362
|
sequence: MediaSequenceProgram,
|
|
363
|
-
|
|
363
|
+
timeline: Timeline,
|
|
364
364
|
): readonly VisualPresent[] {
|
|
365
365
|
return sequence.members.map((member, index) => {
|
|
366
366
|
const incoming = sequence.handoffs[index - 1];
|
|
@@ -389,7 +389,7 @@ export function lowerMediaSequencePresents(
|
|
|
389
389
|
subjectId: sequence.id,
|
|
390
390
|
span: { ...item.span },
|
|
391
391
|
stacking: { ...item.stacking },
|
|
392
|
-
elements: lowerMediaItemElements(item,
|
|
392
|
+
elements: lowerMediaItemElements(item, timeline, {
|
|
393
393
|
includeHandoffWrapper: true,
|
|
394
394
|
...(handoffAnimation === undefined ? {} : { handoffAnimation }),
|
|
395
395
|
lifecycleAnimationOverride: lifecycleAnimationWindow(
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import {
|
|
1
|
+
import { resolveTemporalContext } from "@hypit/temporal-markup";
|
|
2
2
|
import {
|
|
3
3
|
assertEmptyElement as empty,
|
|
4
4
|
assertAttributes as allowed,
|
|
@@ -85,7 +85,6 @@ type FragmentLayer =
|
|
|
85
85
|
type FragmentSound = { readonly sourceName: string; readonly specName: string };
|
|
86
86
|
|
|
87
87
|
type FragmentItem = {
|
|
88
|
-
readonly performance?: { readonly fitName: string; readonly sampleName: string };
|
|
89
88
|
readonly suffix: string;
|
|
90
89
|
readonly windowName: string;
|
|
91
90
|
readonly frameName: string;
|
|
@@ -167,11 +166,11 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: s
|
|
|
167
166
|
spec = operation(bindId);
|
|
168
167
|
}
|
|
169
168
|
const common = {
|
|
170
|
-
set: operation(set), header: input("header"),
|
|
169
|
+
set: operation(set), header: input("header"), timeline: input("timeline"), canvas: input("canvas"),
|
|
171
170
|
layers: operation(layers), frame: input(item.frameName), spec, sounds: operation(sounds), window: input(item.windowName),
|
|
172
171
|
};
|
|
173
|
-
operations.push({ id: appendId, producer:
|
|
174
|
-
inputs:
|
|
172
|
+
operations.push({ id: appendId, producer: mediaTrackProducers.appendItem,
|
|
173
|
+
inputs: common, result: { kind: "output", name: "set" } });
|
|
175
174
|
set = appendId;
|
|
176
175
|
}
|
|
177
176
|
for (const sequence of sequences) {
|
|
@@ -184,7 +183,7 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: s
|
|
|
184
183
|
const layers = appendLayers(operations, memberPrefix, member.layers);
|
|
185
184
|
const appendId = `${memberPrefix}:append`;
|
|
186
185
|
operations.push({ id: appendId, producer: mediaTrackProducers.appendMember, inputs: {
|
|
187
|
-
members: operation(members),
|
|
186
|
+
members: operation(members), timeline: input("timeline"), layers: operation(layers), spec: input(member.specName), activation: input(member.instantName),
|
|
188
187
|
}, result: { kind: "output", name: "members" } });
|
|
189
188
|
members = appendId;
|
|
190
189
|
}
|
|
@@ -199,7 +198,7 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: s
|
|
|
199
198
|
spec = operation(bindId);
|
|
200
199
|
}
|
|
201
200
|
const common = {
|
|
202
|
-
set: operation(set), header: input("header"),
|
|
201
|
+
set: operation(set), header: input("header"), timeline: input("timeline"), canvas: input("canvas"),
|
|
203
202
|
members: operation(members), frame: input(sequence.frameName), spec, sounds: operation(sounds),
|
|
204
203
|
};
|
|
205
204
|
operations.push({ id: appendId, producer: mediaTrackProducers.appendSequence,
|
|
@@ -207,10 +206,10 @@ function createMediaTrackSurfaceFragment(inputTypes: readonly { readonly name: s
|
|
|
207
206
|
set = appendId;
|
|
208
207
|
}
|
|
209
208
|
operations.push(
|
|
210
|
-
{ id: "track:finalize", producer: mediaTrackProducers.finalize, inputs: { set: operation(set), header: input("header"),
|
|
211
|
-
{ id: "track:visual", producer: mediaTrackProducers.projectVisual, inputs: {
|
|
209
|
+
{ id: "track:finalize", producer: mediaTrackProducers.finalize, inputs: { set: operation(set), header: input("header"), timeline: input("timeline") }, result: { kind: "output", name: "program" } },
|
|
210
|
+
{ id: "track:visual", producer: mediaTrackProducers.projectVisual, inputs: { timeline: input("timeline"), program: operation("track:finalize") }, result: { kind: "output", name: "track" } },
|
|
212
211
|
);
|
|
213
|
-
if (audio) operations.push({ id: "track:audio", producer: mediaTrackProducers.projectAudio, inputs: {
|
|
212
|
+
if (audio) operations.push({ id: "track:audio", producer: mediaTrackProducers.projectAudio, inputs: { timeline: input("timeline"), program: operation("track:finalize") }, result: { kind: "output", name: "track" } });
|
|
214
213
|
return sealGraphFragment({
|
|
215
214
|
inputs: inputTypes,
|
|
216
215
|
operations,
|
|
@@ -495,39 +494,33 @@ function sound(
|
|
|
495
494
|
}
|
|
496
495
|
|
|
497
496
|
export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, resolveReference }) => {
|
|
498
|
-
allowed(element, ["id", "
|
|
497
|
+
allowed(element, ["id", "timeline", "canvas"]);
|
|
499
498
|
const trackId = text(element, "id");
|
|
500
499
|
const state = builder();
|
|
501
500
|
const context = resolveTemporalContext({ element, resolveReference });
|
|
502
|
-
|
|
503
|
-
state.addReference("space", time.space);
|
|
501
|
+
state.addReference("timeline", context.timeline);
|
|
504
502
|
state.addReference("canvas", reference(element.attributes.canvas, `${element.name}.canvas`, spatialTypes.canvas, resolveReference));
|
|
505
503
|
const headerId = `${trackId}.header`;
|
|
506
504
|
state.addRecord("header", headerId, mediaTrackTypes.header,
|
|
507
505
|
sealMediaTrackHeader({ id: trackId }), element.range);
|
|
508
506
|
const items: FragmentItem[] = [];
|
|
509
507
|
const sequences: FragmentSequence[] = [];
|
|
510
|
-
const temporalComponents: SurfaceComponentDraft[] = [
|
|
511
|
-
const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [
|
|
508
|
+
const temporalComponents: SurfaceComponentDraft[] = [];
|
|
509
|
+
const temporalFragments: ReturnType<typeof createTemporalWindowProjection>["fragments"][number][] = [];
|
|
512
510
|
let itemIndex = 0;
|
|
513
511
|
let sequenceIndex = 0;
|
|
514
512
|
let hasAudio = false;
|
|
515
513
|
for (const child of element.children) {
|
|
516
514
|
if (child.kind === "text") {
|
|
517
|
-
if (child.value.trim().length > 0) throw new Error(`${element.name} accepts only
|
|
515
|
+
if (child.value.trim().length > 0) throw new Error(`${element.name} accepts only Item and Sequence children.`);
|
|
518
516
|
continue;
|
|
519
517
|
}
|
|
520
|
-
|
|
521
|
-
if (isPerformance) {
|
|
522
|
-
if (context.semantic === undefined) throw new Error(`${child.name} requires semantic performance input.`);
|
|
523
|
-
if (state.inputs.semantic === undefined) state.addReference("semantic", context.semantic);
|
|
524
|
-
}
|
|
525
|
-
if (isPerformance || child.name.endsWith(":Item") || child.name === "Item") {
|
|
518
|
+
if (child.name.endsWith(":Item") || child.name === "Item") {
|
|
526
519
|
itemIndex += 1;
|
|
527
520
|
const itemSuffix = suffix(itemIndex);
|
|
528
521
|
allowed(child, [
|
|
529
522
|
"id", "frame", "appearance", "motion",
|
|
530
|
-
|
|
523
|
+
"image", "media", "surface", "extent", "source-audio", "audio-gain",
|
|
531
524
|
"clip", ...temporalWindowAttributeNames,
|
|
532
525
|
]);
|
|
533
526
|
const id = optionalText(child, "id") ?? `${trackId}.item.${itemSuffix}`;
|
|
@@ -539,23 +532,12 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res
|
|
|
539
532
|
}
|
|
540
533
|
const motionRecipe = child.attributes.motion === undefined ? undefined
|
|
541
534
|
: recipe(reference(child.attributes.motion, `${child.name}.motion`, svsRecipeType, resolveReference), `${child.name}.motion`);
|
|
542
|
-
const temporal = createTemporalWindowProjection({ id, element: child, ...context,
|
|
535
|
+
const temporal = createTemporalWindowProjection({ id, element: child, ...context, resolveReference });
|
|
543
536
|
state.records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);
|
|
544
|
-
const
|
|
545
|
-
|
|
546
|
-
} : undefined;
|
|
547
|
-
const directSource = isPerformance ? undefined : declaredVisualSource(child, resolveReference);
|
|
548
|
-
validateUnitChildren(child, isPerformance || directSource !== undefined, true);
|
|
549
|
-
if (performance !== undefined) {
|
|
550
|
-
state.addRecord(performance.fitName, `${id}.fit`, spatialTypes.fit, decodeMediaFit(appearance), child.range);
|
|
551
|
-
if (appearance.properties.playback !== undefined || appearance.properties["trim-start"] !== undefined || appearance.properties["trim-end"] !== undefined) {
|
|
552
|
-
throw new Error(`${child.name} follows the performance's source time. Select its interval with during or start/end.`);
|
|
553
|
-
}
|
|
554
|
-
state.addRecord(performance.sampleName, `${id}.sample`, mediaTrackTypes.sampleLayerSpec,
|
|
555
|
-
decodeMediaSampleSpec(appearance, "content", "timed", decodeSamplingMotion(child, true), true), child.range);
|
|
556
|
-
}
|
|
537
|
+
const directSource = declaredVisualSource(child, resolveReference);
|
|
538
|
+
validateUnitChildren(child, directSource !== undefined, true);
|
|
557
539
|
const layers = unitLayers(state, { trackId, unitSuffix: `item-${itemSuffix}`, element: child, appearance,
|
|
558
|
-
...(directSource === undefined ? {} : { directSource }), allowFramePaint: true, allowEmpty:
|
|
540
|
+
...(directSource === undefined ? {} : { directSource }), allowFramePaint: true, allowEmpty: false }, resolveReference);
|
|
559
541
|
const selectedAudio = sourceAudio(child);
|
|
560
542
|
const specName = `item-${itemSuffix}-spec`;
|
|
561
543
|
const windowName = `item-${itemSuffix}-window`;
|
|
@@ -572,12 +554,12 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res
|
|
|
572
554
|
}
|
|
573
555
|
}
|
|
574
556
|
if (selectedAudio !== undefined || fragmentSounds.length > 0) hasAudio = true;
|
|
575
|
-
items.push({ suffix: itemSuffix, windowName, frameName, specName,
|
|
557
|
+
items.push({ suffix: itemSuffix, windowName, frameName, specName,
|
|
576
558
|
...(clipPathName === undefined ? {} : { clipPathName }), layers, sounds: fragmentSounds });
|
|
577
559
|
continue;
|
|
578
560
|
}
|
|
579
561
|
if (!(child.name.endsWith(":Sequence") || child.name === "Sequence")) {
|
|
580
|
-
throw new Error(`${element.name} accepts only
|
|
562
|
+
throw new Error(`${element.name} accepts only Item and Sequence children.`);
|
|
581
563
|
}
|
|
582
564
|
sequenceIndex += 1;
|
|
583
565
|
const sequenceSuffix = suffix(sequenceIndex);
|
|
@@ -611,7 +593,7 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res
|
|
|
611
593
|
for (const [index, member] of memberElements.entries()) {
|
|
612
594
|
const memberSuffix = suffix(index + 1);
|
|
613
595
|
allowed(member, ["id", "image", "media", "surface", "extent", "appearance", ...temporalInstantAttributeNames, "source-audio", "audio-gain"]);
|
|
614
|
-
const temporal = createTemporalInstantProjection({ id: `${id}.${memberIds[index]}`, subjectId: memberIds[index]!, element: member, ...context,
|
|
596
|
+
const temporal = createTemporalInstantProjection({ id: `${id}.${memberIds[index]}`, subjectId: memberIds[index]!, element: member, ...context, resolveReference });
|
|
615
597
|
state.records.push(...temporal.records); temporalComponents.push(...temporal.components); temporalFragments.push(...temporal.fragments);
|
|
616
598
|
const memberAppearance = member.attributes.appearance === undefined ? appearance
|
|
617
599
|
: recipe(reference(member.attributes.appearance, `${member.name}.appearance`, svsRecipeType, resolveReference), `${member.name}.appearance`);
|
|
@@ -641,7 +623,7 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res
|
|
|
641
623
|
}
|
|
642
624
|
if (sequenceSounds.length > 0) hasAudio = true;
|
|
643
625
|
const terminalTemporal = createTemporalInstantProjection({
|
|
644
|
-
id: `${id}.terminal`, subjectId: id, element: child, ...context,
|
|
626
|
+
id: `${id}.terminal`, subjectId: id, element: child, ...context, resolveReference,
|
|
645
627
|
semanticAttribute: "until", boundaryAttribute: "until-boundary", projectedAttribute: false,
|
|
646
628
|
boundaryFallback: "end",
|
|
647
629
|
});
|
|
@@ -656,7 +638,7 @@ export const decodeMediaTrackSurface: StructuredSurfaceHandler = ({ element, res
|
|
|
656
638
|
sequences.push({ suffix: sequenceSuffix, terminalName, frameName, specName,
|
|
657
639
|
...(clipPathName === undefined ? {} : { clipPathName }), members, sounds: sequenceSounds });
|
|
658
640
|
}
|
|
659
|
-
if (items.length === 0 && sequences.length === 0) throw new Error(`${element.name} requires at least one
|
|
641
|
+
if (items.length === 0 && sequences.length === 0) throw new Error(`${element.name} requires at least one Item or Sequence.`);
|
|
660
642
|
const fragment = createMediaTrackSurfaceFragment(state.inputTypes, items, sequences, hasAudio);
|
|
661
643
|
return {
|
|
662
644
|
records: state.records,
|
|
@@ -1,3 +1,12 @@
|
|
|
1
1
|
# `@hypit/media-track-studio`
|
|
2
2
|
|
|
3
3
|
Hypit Studio Companion for `@hypit/media-track`.
|
|
4
|
+
|
|
5
|
+
Each Item has a title and image or video body. An explicit author `id` supplies
|
|
6
|
+
its title; otherwise the Companion displays its `image`, `media` or `surface`
|
|
7
|
+
reference. Display names do not replace the Item's identity or writeback
|
|
8
|
+
endpoint. Source-audio gain uses the same percentage display and scalar
|
|
9
|
+
writeback as Audio Track.
|
|
10
|
+
|
|
11
|
+
Media and Audio Track share selection and temporal editing behavior. Media
|
|
12
|
+
composes pictures by draw order; Audio Track mixes overlapping sounds.
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { mediaTrackMarkupSurfaces, mediaTrackModuleRef, mediaTrackTypes } from "@hypit/media-track";
|
|
1
|
+
import { mediaAppearanceDefaults, mediaTrackMarkupSurfaces, mediaTrackModuleRef, mediaTrackTypes } from "@hypit/media-track";
|
|
2
2
|
import type { MediaLayerProgram, MediaTrackProgram } from "@hypit/media-track";
|
|
3
3
|
import { compositionTypes } from "@hypit/composition";
|
|
4
4
|
import { sameType } from "@hypit/protocol";
|
|
@@ -10,7 +10,7 @@ import type {
|
|
|
10
10
|
StudioMaterialPreview,
|
|
11
11
|
StudioSourceBindingDeclaration,
|
|
12
12
|
} from "@hypit/studio-adapter";
|
|
13
|
-
import { artifactPreview, childEntities, previewLayer, requiredSurfaceValue, temporalLineageFor, temporalSemanticSource } from "@hypit/studio-adapter";
|
|
13
|
+
import { artifactPreview, authoredItemTitle, childEntities, previewLayer, requiredSurfaceValue, temporalLineageFor, temporalSemanticSource } from "@hypit/studio-adapter";
|
|
14
14
|
|
|
15
15
|
const frameParameters: readonly StudioSourceBindingDeclaration[] = [
|
|
16
16
|
{ name: "within" },
|
|
@@ -70,13 +70,15 @@ const mediaPercentProperties = new Set(["frame-x", "frame-y", "content-x", "cont
|
|
|
70
70
|
const mediaPixelProperties = new Set(["fit-offset-x", "fit-offset-y", "radius", "blur", "border-width"]);
|
|
71
71
|
|
|
72
72
|
function mediaRecipe(name: "appearance" | "motion"): {
|
|
73
|
-
readonly bindings: readonly
|
|
73
|
+
readonly bindings: readonly import("@hypit/studio-adapter").StudioRecipeBindingDeclaration[];
|
|
74
74
|
readonly inspector: readonly StudioInspectorFieldDeclaration[];
|
|
75
75
|
} {
|
|
76
76
|
const attribute = mediaItemAttributes.find((candidate) => candidate.name === name);
|
|
77
77
|
const properties = attribute !== undefined && "recipe" in attribute ? attribute.recipe : [];
|
|
78
78
|
return {
|
|
79
|
-
bindings: properties.map(({ name: property }) => ({ name: property
|
|
79
|
+
bindings: properties.map(({ name: property }) => ({ name: property,
|
|
80
|
+
...(Object.hasOwn(mediaAppearanceDefaults, property) ? { fallback: mediaAppearanceDefaults[property as keyof typeof mediaAppearanceDefaults] } : {}),
|
|
81
|
+
})),
|
|
80
82
|
inspector: properties.map((property) => {
|
|
81
83
|
const placement = mediaPlacement.get(property.name);
|
|
82
84
|
if (placement === undefined) throw new Error(`Media Studio has no explicit Inspector declaration for ${property.name}.`);
|
|
@@ -99,7 +101,8 @@ function mediaRecipe(name: "appearance" | "motion"): {
|
|
|
99
101
|
};
|
|
100
102
|
}
|
|
101
103
|
|
|
102
|
-
const
|
|
104
|
+
export const mediaAppearanceStudioFields = mediaRecipe("appearance");
|
|
105
|
+
const mediaAppearanceRecipe = mediaAppearanceStudioFields;
|
|
103
106
|
const mediaMotionRecipe = mediaRecipe("motion");
|
|
104
107
|
|
|
105
108
|
function materialPreview(
|
|
@@ -153,15 +156,16 @@ function projectMedia(context: StudioTrackCompanionContext): readonly StudioEnti
|
|
|
153
156
|
const semanticSource = temporalSemanticSource(temporal);
|
|
154
157
|
return {
|
|
155
158
|
...entity,
|
|
159
|
+
display: {
|
|
160
|
+
...entity.display,
|
|
161
|
+
title: authoredItemTitle(context, entity.authoredId, item.sourceTypes, ["image", "media", "surface"]),
|
|
162
|
+
...(item.preview === undefined ? {} : {
|
|
163
|
+
layers: [previewLayer(item.preview, facet === "audio" ? "waveform" : item.preview.kind === "video" ? "storyboard" : "repeat-x")],
|
|
164
|
+
}),
|
|
165
|
+
},
|
|
156
166
|
...(semanticSource?.id === undefined
|
|
157
167
|
? {}
|
|
158
168
|
: { markerId: semanticSource.id }),
|
|
159
|
-
...(item.preview === undefined ? {} : {
|
|
160
|
-
display: {
|
|
161
|
-
...entity.display,
|
|
162
|
-
layers: [previewLayer(item.preview, facet === "audio" ? "waveform" : item.preview.kind === "video" ? "storyboard" : "repeat-x")],
|
|
163
|
-
},
|
|
164
|
-
}),
|
|
165
169
|
...(temporal === undefined ? {} : { temporal }),
|
|
166
170
|
};
|
|
167
171
|
});
|
|
@@ -178,6 +182,7 @@ const commonInspector: readonly StudioInspectorFieldDeclaration[] = [
|
|
|
178
182
|
{
|
|
179
183
|
binding: "audio-gain", label: "Audio Gain", domain: "how",
|
|
180
184
|
page: { id: "audio", label: "Audio" }, section: { id: "audio", label: "Audio" }, control: "number",
|
|
185
|
+
unit: "%", number: { scale: 100, minimum: 0, step: 1 },
|
|
181
186
|
},
|
|
182
187
|
{
|
|
183
188
|
binding: "until-boundary", label: "Until Boundary", domain: "when",
|
|
@@ -9,7 +9,7 @@ The package owns two distinct speech operations:
|
|
|
9
9
|
- `mimo-v2.5-tts-voiceclone` uses one accepted audio voice reference to create independent speech.
|
|
10
10
|
|
|
11
11
|
It contains no API URL, credential, retry, queue or Xiaomi wire encoding. Those belong to a Runtime
|
|
12
|
-
Endpoint
|
|
12
|
+
Endpoint implementing the selected speech capability. Both models return the shared `GeneratedAudioSet`;
|
|
13
13
|
the author Surfaces expose its primary member as an ordinary audio Resource.
|
|
14
14
|
|
|
15
15
|
```xml
|
|
@@ -5,7 +5,7 @@ Exact author/compute contracts and package-owned author Surfaces for MiniMax H3
|
|
|
5
5
|
Text, frame-guided and subject-reference modes are separate Surfaces rather than one dynamic
|
|
6
6
|
port mode. Each produces an ordinary video Artifact. This package owns request semantics
|
|
7
7
|
and validation only; Provider calls, credentials, retries and queueing belong to a Runtime Endpoint
|
|
8
|
-
|
|
8
|
+
selected in the Runtime Profile.
|
|
9
9
|
|
|
10
10
|
```xml
|
|
11
11
|
<h3:TextVideo id="idea" prompt={prompt} duration="6" resolution="768P" aspect-ratio="9:16"/>
|
|
@@ -4,7 +4,7 @@ Exact author/compute contracts and package-owned author Surfaces for Nano Banana
|
|
|
4
4
|
|
|
5
5
|
The model variants are separate endpoints with exact request validation. Their Surfaces project the
|
|
6
6
|
primary result to an ordinary image Artifact. The package contains no Provider selection, API key or
|
|
7
|
-
network execution
|
|
7
|
+
network execution. The selected Provider implements its exact capability.
|
|
8
8
|
|
|
9
9
|
Import the model variant you mean and connect prompt and references as ordinary graph edges:
|
|
10
10
|
|
|
@@ -4,4 +4,26 @@ External components use `@hypit/hypit/narrative` from their `@hypit/hypit` devel
|
|
|
4
4
|
owns the types and helpers below; Source imports retain the `@hypit/narrative@1` Module identity.
|
|
5
5
|
|
|
6
6
|
|
|
7
|
-
Public, provider-neutral
|
|
7
|
+
Public, provider-neutral values for complete authored content and its semantic references.
|
|
8
|
+
|
|
9
|
+
A `Narrative` owns speech Tokens, Segments, Turns, semantic Anchors, Selections and Moments, plus
|
|
10
|
+
its `caption: CaptionDocument`. The document supplies display Words, speech/display correspondence
|
|
11
|
+
and Cue breaks. These are authored relationships; frame timing and visual Styles belong downstream.
|
|
12
|
+
Script exports the complete value as `story` and the same document as the narrow `story.caption`
|
|
13
|
+
Record. A third-party author package can produce the same values without importing the Script parser.
|
|
14
|
+
|
|
15
|
+
## Query content through semantic references
|
|
16
|
+
|
|
17
|
+
- `narrativeAnchorTokenBoundary(narrative, anchorId)` resolves a boundary in authored token order.
|
|
18
|
+
- `narrativeSelectionTokenRange(narrative, selection)` returns the half-open authored token range.
|
|
19
|
+
- `narrativeTokensForSelection(narrative, selection)` returns those Tokens in authored order.
|
|
20
|
+
|
|
21
|
+
These pure queries use existing Anchors and Tokens, with no stored secondary index. An exported
|
|
22
|
+
Selection retains its Narrative identity; content lookup checks that identity. Structural anchors
|
|
23
|
+
remain distinct even when they resolve to the same token boundary, and an empty Segment can select
|
|
24
|
+
no Tokens. Authored order is independent of gaps, overlap or placement order in a Timeline.
|
|
25
|
+
|
|
26
|
+
Caption uses this content query and the Narrative's display correspondence to select complete
|
|
27
|
+
subtitle units. Timeline projection uses the same semantic references to locate events in physical
|
|
28
|
+
time. Neither operation substitutes for the other. Token lookup returns Tokens, not reconstructed
|
|
29
|
+
source prose: whitespace, comments and source-preserving edits remain the author language's concern.
|
|
@@ -25,6 +25,17 @@ export function assertNarrativeIdentity(value: Narrative): void {
|
|
|
25
25
|
if (value.segments.length === 0) throw new Error("Narrative must contain at least one Segment.");
|
|
26
26
|
const segmentIds = unique(value.segments.map((item) => item.id), "Narrative Segment id");
|
|
27
27
|
const tokenIds = unique(value.tokens.map((item) => item.id), "Narrative Token id");
|
|
28
|
+
assertCaptionDocumentIdentity(value.caption);
|
|
29
|
+
if (value.caption.narrativeId !== value.id) throw new Error("Narrative CaptionDocument belongs to another Narrative.");
|
|
30
|
+
const turns = new Map(value.turns.map((turn) => [turn.id, turn]));
|
|
31
|
+
const tokens = new Map(value.tokens.map((token) => [token.id, token]));
|
|
32
|
+
for (const unit of value.caption.units) {
|
|
33
|
+
const turn = turns.get(unit.turnId);
|
|
34
|
+
if (turn === undefined || turn.segmentId !== unit.segmentId || turn.role !== unit.role
|
|
35
|
+
|| unit.sourceTokenIds.some((id) => tokens.get(id)?.segmentId !== unit.segmentId)) {
|
|
36
|
+
throw new Error(`Caption unit ${unit.id} disagrees with its authored speech.`);
|
|
37
|
+
}
|
|
38
|
+
}
|
|
28
39
|
unique(value.turns.map((item) => item.id), "Narrative Turn id");
|
|
29
40
|
unique(value.selections.map((item) => item.id), "Narrative Selection id");
|
|
30
41
|
unique(value.moments.map((item) => item.id), "Narrative Moment id");
|
|
@@ -9,3 +9,6 @@ export {
|
|
|
9
9
|
} from "./identity.js";
|
|
10
10
|
export { captionDocumentSchema, narrativeExcerptSchema, narrativeMomentSchema, narrativeSchema, narrativeSelectionSchema } from "./schema.js";
|
|
11
11
|
export type * from "./types.js";
|
|
12
|
+
|
|
13
|
+
export { narrativeAnchorTokenBoundary, narrativeSelectionTokenRange, narrativeTokensForSelection } from "./selection.js";
|
|
14
|
+
export type { NarrativeTokenRange } from "./selection.js";
|
|
@@ -23,33 +23,6 @@ const semanticAnchor: ValueSchema = { kind: "oneOf", variants: [
|
|
|
23
23
|
kind: { schema: { kind: "literal", value: "program-end" } },
|
|
24
24
|
}),
|
|
25
25
|
] };
|
|
26
|
-
export const narrativeSchema: ValueSchema = object({
|
|
27
|
-
id: { schema: string },
|
|
28
|
-
segments: { schema: { kind: "array", minItems: 1, items: object({
|
|
29
|
-
id: { schema: string }, startAnchorId: { schema: string },
|
|
30
|
-
endAnchorId: { schema: string }, tokenStart: { schema: integer }, tokenEndExclusive: { schema: integer },
|
|
31
|
-
}) } },
|
|
32
|
-
tokens: { schema: { kind: "array", items: object({
|
|
33
|
-
id: { schema: string }, segmentId: { schema: string },
|
|
34
|
-
startAnchorId: { schema: string }, endAnchorId: { schema: string },
|
|
35
|
-
text: { schema: string }, normalized: { schema: string },
|
|
36
|
-
}) } },
|
|
37
|
-
turns: { schema: { kind: "array", items: object({
|
|
38
|
-
id: { schema: string }, segmentId: { schema: string }, role: { schema: string, optional: true },
|
|
39
|
-
tokenStart: { schema: integer }, tokenEndExclusive: { schema: integer },
|
|
40
|
-
}) } },
|
|
41
|
-
selections: { schema: { kind: "array", items: object({
|
|
42
|
-
id: { schema: string }, startAnchorId: { schema: string }, endAnchorId: { schema: string },
|
|
43
|
-
}) } },
|
|
44
|
-
moments: { schema: { kind: "array", items: object({
|
|
45
|
-
id: { schema: string }, anchorId: { schema: string },
|
|
46
|
-
}) } },
|
|
47
|
-
semanticIndex: { schema: object({
|
|
48
|
-
|
|
49
|
-
anchors: { schema: { kind: "array", minItems: 2, items: semanticAnchor } },
|
|
50
|
-
}) },
|
|
51
|
-
});
|
|
52
|
-
|
|
53
26
|
export const narrativeExcerptSchema: ValueSchema = object({
|
|
54
27
|
narrativeId: { schema: string },
|
|
55
28
|
kind: { schema: { kind: "literal", value: "segment" } }, id: { schema: string },
|
|
@@ -77,6 +50,34 @@ export const captionDocumentSchema: ValueSchema = object({
|
|
|
77
50
|
words: { schema: { kind: "array", items: captionDisplayWord } },
|
|
78
51
|
cueBreaks: { schema: { kind: "array", items: captionCueBreak } },
|
|
79
52
|
});
|
|
53
|
+
export const narrativeSchema: ValueSchema = object({
|
|
54
|
+
id: { schema: string },
|
|
55
|
+
caption: { schema: captionDocumentSchema },
|
|
56
|
+
segments: { schema: { kind: "array", minItems: 1, items: object({
|
|
57
|
+
id: { schema: string }, startAnchorId: { schema: string },
|
|
58
|
+
endAnchorId: { schema: string }, tokenStart: { schema: integer }, tokenEndExclusive: { schema: integer },
|
|
59
|
+
}) } },
|
|
60
|
+
tokens: { schema: { kind: "array", items: object({
|
|
61
|
+
id: { schema: string }, segmentId: { schema: string },
|
|
62
|
+
startAnchorId: { schema: string }, endAnchorId: { schema: string },
|
|
63
|
+
text: { schema: string }, normalized: { schema: string },
|
|
64
|
+
}) } },
|
|
65
|
+
turns: { schema: { kind: "array", items: object({
|
|
66
|
+
id: { schema: string }, segmentId: { schema: string }, role: { schema: string, optional: true },
|
|
67
|
+
tokenStart: { schema: integer }, tokenEndExclusive: { schema: integer },
|
|
68
|
+
}) } },
|
|
69
|
+
selections: { schema: { kind: "array", items: object({
|
|
70
|
+
id: { schema: string }, startAnchorId: { schema: string }, endAnchorId: { schema: string },
|
|
71
|
+
}) } },
|
|
72
|
+
moments: { schema: { kind: "array", items: object({
|
|
73
|
+
id: { schema: string }, anchorId: { schema: string },
|
|
74
|
+
}) } },
|
|
75
|
+
semanticIndex: { schema: object({
|
|
76
|
+
|
|
77
|
+
anchors: { schema: { kind: "array", minItems: 2, items: semanticAnchor } },
|
|
78
|
+
}) },
|
|
79
|
+
});
|
|
80
|
+
|
|
80
81
|
export const narrativeMomentSchema: ValueSchema = object({
|
|
81
82
|
narrativeId: { schema: string },
|
|
82
83
|
id: { schema: string },
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
import type { Narrative, NarrativeSelection, NarrativeSelectionRef, NarrativeToken } from "./types.js";
|
|
2
|
+
|
|
3
|
+
/** A range in authored token order, independent of material placement and frame timing. */
|
|
4
|
+
export type NarrativeTokenRange = {
|
|
5
|
+
readonly narrativeId: string;
|
|
6
|
+
readonly tokenStart: number;
|
|
7
|
+
readonly tokenEndExclusive: number;
|
|
8
|
+
};
|
|
9
|
+
|
|
10
|
+
/** Resolve a semantic anchor to a boundary in the authored token sequence. */
|
|
11
|
+
export function narrativeAnchorTokenBoundary(narrative: Narrative, anchorId: string): number {
|
|
12
|
+
const anchor = narrative.semanticIndex.anchors.find((item) => item.id === anchorId);
|
|
13
|
+
if (anchor === undefined) throw new Error(`Narrative ${narrative.id} has no anchor ${anchorId}.`);
|
|
14
|
+
if (anchor.kind === "program-start") return 0;
|
|
15
|
+
if (anchor.kind === "program-end") return narrative.tokens.length;
|
|
16
|
+
const segment = narrative.segments.find((item) => item.id === anchor.segmentId);
|
|
17
|
+
if (segment === undefined) throw new Error(`Anchor ${anchorId} has no authored Segment.`);
|
|
18
|
+
if (anchor.kind === "segment-start") return segment.tokenStart;
|
|
19
|
+
if (anchor.kind === "segment-end") return segment.tokenEndExclusive;
|
|
20
|
+
const index = narrative.tokens.findIndex((item) => item.id === anchor.tokenId);
|
|
21
|
+
if (index < 0) throw new Error(`Anchor ${anchorId} has no authored token.`);
|
|
22
|
+
return anchor.kind === "token-start" ? index : index + 1;
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
export function narrativeSelectionTokenRange(
|
|
26
|
+
narrative: Narrative,
|
|
27
|
+
selection: NarrativeSelection | NarrativeSelectionRef,
|
|
28
|
+
): NarrativeTokenRange {
|
|
29
|
+
if ("narrativeId" in selection && selection.narrativeId !== narrative.id) {
|
|
30
|
+
throw new Error(`Selection ${selection.id} belongs to another Narrative.`);
|
|
31
|
+
}
|
|
32
|
+
const anchors = narrative.semanticIndex.anchors;
|
|
33
|
+
const start = anchors.findIndex((item) => item.id === selection.startAnchorId);
|
|
34
|
+
const end = anchors.findIndex((item) => item.id === selection.endAnchorId);
|
|
35
|
+
if (start < 0 || end < 0 || end < start) {
|
|
36
|
+
throw new Error(`Selection ${selection.id} has invalid authored anchor order.`);
|
|
37
|
+
}
|
|
38
|
+
return {
|
|
39
|
+
narrativeId: narrative.id,
|
|
40
|
+
tokenStart: narrativeAnchorTokenBoundary(narrative, selection.startAnchorId),
|
|
41
|
+
tokenEndExclusive: narrativeAnchorTokenBoundary(narrative, selection.endAnchorId),
|
|
42
|
+
};
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
/** Select content without consulting a Timeline or reconstructing prose from token strings. */
|
|
46
|
+
export function narrativeTokensForSelection(
|
|
47
|
+
narrative: Narrative,
|
|
48
|
+
selection: NarrativeSelection | NarrativeSelectionRef,
|
|
49
|
+
): readonly NarrativeToken[] {
|
|
50
|
+
const range = narrativeSelectionTokenRange(narrative, selection);
|
|
51
|
+
return narrative.tokens.slice(range.tokenStart, range.tokenEndExclusive);
|
|
52
|
+
}
|
|
@@ -113,12 +113,14 @@ export type SemanticAnchor =
|
|
|
113
113
|
readonly tokenId?: string;
|
|
114
114
|
};
|
|
115
115
|
|
|
116
|
+
/** Complete authored content; local views are projections of this value. */
|
|
116
117
|
export type Narrative = {
|
|
117
118
|
/** Author-visible `<script id>`; every exported semantic view retains it. */
|
|
118
119
|
readonly id: string;
|
|
119
120
|
readonly segments: readonly NarrativeSegment[];
|
|
120
121
|
readonly tokens: readonly NarrativeToken[];
|
|
121
122
|
readonly turns: readonly NarrativeTurn[];
|
|
123
|
+
readonly caption: CaptionDocument;
|
|
122
124
|
readonly selections: readonly NarrativeSelection[];
|
|
123
125
|
readonly moments: readonly NarrativeMoment[];
|
|
124
126
|
readonly semanticIndex: {
|
|
@@ -21,3 +21,33 @@ It does not use a CommonJS root export as evidence that a package is installed:
|
|
|
21
21
|
import-only and resource-only packages are valid. The active Distribution owns `@hypit/*`;
|
|
22
22
|
Distribution code may use upstream dependencies from the machine npm home, while project packages
|
|
23
23
|
continue to resolve their own dependencies from the project.
|
|
24
|
+
|
|
25
|
+
The machine home stores `<package-name>/<exact-version>/node_modules`, with the importing package's
|
|
26
|
+
ordinary `dependencies` selecting the version. Loading a second Distribution does not replace an
|
|
27
|
+
existing release. ESM, resource and executable lookup use the same selection; project dependencies
|
|
28
|
+
retain ordinary project ownership. There is no inventory or source-content matching.
|
|
29
|
+
|
|
30
|
+
Runtime declaration discovery may use `deferExternalDependencies` to inspect capabilities and
|
|
31
|
+
Managed Programs before their execution tools are prepared. Actual JavaScript imports still follow
|
|
32
|
+
Node resolution and must be available. A selected Profile remains a valid package configuration;
|
|
33
|
+
Endpoint scoping is not permission to execute an invalid or absent Provider implementation.
|
|
34
|
+
|
|
35
|
+
## Execution-owned module scopes
|
|
36
|
+
|
|
37
|
+
`NodePackageLoadOptions.importModule` lets the Host own module lifetime while package selection and
|
|
38
|
+
contribution validation stay here. The default uses ordinary process imports. The local execution
|
|
39
|
+
Host supplies a `NodeModuleScope` for each Build: selected project packages and their transitive
|
|
40
|
+
JavaScript dependencies receive scope-local module identities. The installed `@hypit/*` Distribution
|
|
41
|
+
remains shared process code. Registries and configured adapter instances are still created per Build.
|
|
42
|
+
|
|
43
|
+
`NodeModuleScope` uses Node resolution/loading hooks and transpiles scope-owned TypeScript/TSX. Its CommonJS
|
|
44
|
+
bridge has a scope-owned module cache, Node-style named-export discovery and synchronous evaluation;
|
|
45
|
+
Node's VM dynamic-import support is enabled only for the execution carrier. Public paths such as
|
|
46
|
+
`import.meta.url`, `__filename` and relative asset reads remain real filesystem locations. Module
|
|
47
|
+
identity URLs are internal names; no copied source directory is created. Closing a scope drops its
|
|
48
|
+
live lookup and CommonJS table; the Host retires an empty carrier to release Node's ESM cache.
|
|
49
|
+
|
|
50
|
+
This is trusted execution, not a sandbox. Builtins, native libraries and process-global state
|
|
51
|
+
(including `process.cwd()` and `process.env`) remain shared. Existing module bindings do not change
|
|
52
|
+
under an active Build; a file or dependency first
|
|
53
|
+
read later uses ordinary filesystem semantics. Scoped loading does not make a project tree immutable.
|