omnius 1.0.869 → 1.0.871
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +48 -840
- package/dist/api/py-embed.js +12 -0
- package/dist/index.js +2 -2
- package/dist/library.js +21 -5
- package/dist/tui/native/worker.js +11 -0
- package/dist/update-worker.js +12 -0
- package/docs/.vitepress/config.mts +1 -0
- package/docs/DISCOVERY.json +1095 -24
- package/docs/DISCOVERY.md +30 -2
- package/docs/design/omni-analysis-TODO.md +60 -0
- package/docs/design/single-final-response-TODO.md +93 -0
- package/docs/reference/omni-analysis.md +251 -0
- package/docs/reference/rest-api.md +19 -6
- package/docs/work-orders/evidence/final-reply-browser-20261016.json +1225 -0
- package/docs/work-orders/evidence/final-reply-dark-1440-accepted-final.png +0 -0
- package/docs/work-orders/evidence/final-reply-light-320-accepted-final.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/completion-failed-partial.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-100x30.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-120x36.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-160x48.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-60x20.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-80x24.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-clean.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/completion-live-clean.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/completion-paused-partial.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/completion-restored-clean.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/completion-technical-expanded.png +0 -0
- package/docs/work-orders/evidence/instrument-completion-live-20261016/report.json +743 -0
- package/docs/work-orders/evidence/native-completion-presentation-verification-20261016.json +65 -0
- package/docs/work-orders/evidence/omni-analysis-browser-20261017.json +577 -0
- package/docs/work-orders/evidence/omni-analysis-desktop-controls-20261017.png +0 -0
- package/docs/work-orders/evidence/omni-analysis-desktop-evidence-20261017.png +0 -0
- package/docs/work-orders/evidence/omni-analysis-execution-verification-20261017.json +78 -0
- package/docs/work-orders/evidence/omni-analysis-integration-verification-20261017.json +215 -0
- package/docs/work-orders/evidence/omni-analysis-phone-controls-20261017.png +0 -0
- package/docs/work-orders/evidence/omni-analysis-phone-model-20261017.png +0 -0
- package/docs/work-orders/evidence/tui-response-ownership-verification-20261016.json +79 -0
- package/npm-shrinkwrap.json +2 -2
- package/package.json +2 -2
package/docs/DISCOVERY.md
CHANGED
|
@@ -162,7 +162,8 @@ Daemon equivalents are `GET /v1/discovery/bootstrap`, `GET /v1/discovery?q=<inte
|
|
|
162
162
|
| `api.v1-mcps-name` | /v1/mcps/{name} | Get MCP server details (PT-03) |
|
|
163
163
|
| `api.v1-mcps-name-call` | /v1/mcps/{name}/call | Invoke a tool on an MCP server (PT-03) |
|
|
164
164
|
| `api.v1-media-audio` | /v1/media/audio | Generate sound — body {projectRoot,prompt,request_id?,session_id?,model?,backend?,duration?,seed?} |
|
|
165
|
-
| `api.v1-media-av-analyze` | /v1/media/av/analyze | Analyze
|
|
165
|
+
| `api.v1-media-av-analyze` | /v1/media/av/analyze | Analyze actual audio/video with the configured Omni comprehension runtime |
|
|
166
|
+
| `api.v1-media-av-status` | /v1/media/av/status | Inspect configured Omni adapter availability without loading weights |
|
|
166
167
|
| `api.v1-media-file` | /v1/media/file | Stream an exact artifact — query {projectRoot,kind,name} or {scope:'legacy-global',kind,name}; Validate exact project/archive identity, MIME and ETag before acquisition; Delete a selected project artifact or explicit legacy copy and matching metadata |
|
|
167
168
|
| `api.v1-media-gallery` | /v1/media/gallery | Discover existing artifacts in a registered project, or the explicit legacy global archive |
|
|
168
169
|
| `api.v1-media-generations-cancel` | /v1/media/generations/cancel | Request daemon-owned Stop — JSON {projectRoot,request_id\|generation_id,session_id?} |
|
|
@@ -435,7 +436,9 @@ Daemon equivalents are `GET /v1/discovery/bootstrap`, `GET /v1/discovery?q=<inte
|
|
|
435
436
|
| `guide.design-dashboard-todo-uppercase` | Dashboard reconstruction — durable TODO | Updated: 2026-10-16. Contract: styleguide. Scope and sequence: plan. |
|
|
436
437
|
| `guide.design-media-unification-design` | Unified media: project artifacts, shared models, deliberate presentation | Status: implementation contract, 2026-10-16. Follows the delivered dashboard reconstruction (6cfaebbd). Track verification in media-unification-TODO.md. |
|
|
437
438
|
| `guide.design-media-unification-todo-uppercase` | Next phase: media storage and generation interface unification | Requested 2026-10-16. Active after dashboard reconstruction delivery 6cfaebbd. Contract: media unification design. The preceding dashboard receipt does not count as verification of this phase. |
|
|
439
|
+
| `guide.design-omni-analysis-todo-uppercase` | Real Omni analysis | Acceptance: Audio, Video, and Audio/video Analysis must submit actual media to the Omni comprehension runtime. Missing weights or an unavailable runtime must never yield synthetic scene entities, events, or a successful mock answer. |
|
|
438
440
|
| `guide.design-session-presentation-todo-uppercase` | Session media, code and working-state presentation | Requested 2026-10-16 after media generation/store unification. |
|
|
441
|
+
| `guide.design-single-final-response-todo-uppercase` | One final response per run | Requested after the retained generation panel, completion tool arguments, and final receipt displayed the same answer three times, with the final copy losing Markdown structure. |
|
|
439
442
|
| `guide.design-update-operation-cancellation` | Owned update cancellation and restart | The native update dialog stays present while an operation is running. Escape, Ctrl+C and outside clicks do not dismiss it. Its top-right Cancel control sends an explicit request; it shows cancellation pending until the owned work settles. An error remains in the same dialog with a Close action. Ordinary menu dismissal and native frame animation remain unchan |
|
|
440
443
|
| `guide.design-update-session-restoration-todo-uppercase` | Queued next: update lifecycle and exact session restoration | Requested 2026-10-16. Implemented after media unification; source-only delivery, with no real installer or running daemon changes during verification. |
|
|
441
444
|
| `guide.discovery-agent-map` | Agent Map | Bundled Omnius documentation artifact: docs/discovery/agent-map.json. |
|
|
@@ -494,7 +497,8 @@ Daemon equivalents are `GET /v1/discovery/bootstrap`, `GET /v1/discovery?q=<inte
|
|
|
494
497
|
| `guide.proposals-opencode-modules-runner` | Runner | Bundled Omnius documentation artifact: docs/proposals/opencode-modules/runner.ts. |
|
|
495
498
|
| `guide.reference-auth-map` | Authentication And Authorization Map | Omnius has two separate credential boundaries: |
|
|
496
499
|
| `guide.reference-configuration` | Configuration Reference | Configuration is layered from environment, global user settings, project settings, and active TUI state. |
|
|
497
|
-
| `guide.reference-
|
|
500
|
+
| `guide.reference-omni-analysis` | Omni media analysis | Omnius uses an explicitly configured Omni adapter for genuine audio and video comprehension. A model name containing -omni does not add audio/video execution to stock Ollama: the adapter must decode the media and run the corresponding model/projector weights. An unavailable runtime is an unavailable capability, never a reason to substitute synthetic observat |
|
|
501
|
+
| `guide.reference-rest-api` | REST API Reference | This is the maintained human inventory for the supported Omnius daemon API. The machine contract is generated by packages/cli/src/api/openapi.ts and served at /openapi.json; every operation in that contract must appear here. The root README links to this inventory and stays focused on the system overview. Browser HTML routes and implementation-only compatibi |
|
|
498
502
|
| `guide.reference-slash-commands` | Slash Commands | This file is generated from packages/cli/src/tui/command-registry.ts. |
|
|
499
503
|
| `guide.research-aiwg-long-horizon-feature-workflow` | AIWG research teams and long-horizon feature delivery | Date: 2026-09-05. Status: design research for consolidation; no runtime implementation in this note. |
|
|
500
504
|
| `guide.research-archived-context-window-attention-model` | Context Window Attention Model | Bundled Omnius documentation artifact: docs/research/_archived/context_window_attention_model.py. |
|
|
@@ -559,6 +563,9 @@ Daemon equivalents are `GET /v1/discovery/bootstrap`, `GET /v1/discovery?q=<inte
|
|
|
559
563
|
| `guide.work-orders-evidence-fallback-model-benchmarks-20261012` | Fallback Model Benchmarks 20261012 | Bundled Omnius documentation artifact: docs/work-orders/evidence/fallback-model-benchmarks-20261012.json. |
|
|
560
564
|
| `guide.work-orders-evidence-fallback-quality-installed-20261012` | Fallback Quality Installed 20261012 | Bundled Omnius documentation artifact: docs/work-orders/evidence/fallback-quality-installed-20261012.json. |
|
|
561
565
|
| `guide.work-orders-evidence-fallback-reply-contracts-installed-20261012` | Fallback Reply Contracts Installed 20261012 | Bundled Omnius documentation artifact: docs/work-orders/evidence/fallback-reply-contracts-installed-20261012.json. |
|
|
566
|
+
| `guide.work-orders-evidence-final-reply-browser-20261016` | Final Reply Browser 20261016 | Bundled Omnius documentation artifact: docs/work-orders/evidence/final-reply-browser-20261016.json. |
|
|
567
|
+
| `guide.work-orders-evidence-final-reply-dark-1440-accepted-final` | Final Reply Dark 1440 Accepted Final | Bundled Omnius documentation artifact: docs/work-orders/evidence/final-reply-dark-1440-accepted-final.png. |
|
|
568
|
+
| `guide.work-orders-evidence-final-reply-light-320-accepted-final` | Final Reply Light 320 Accepted Final | Bundled Omnius documentation artifact: docs/work-orders/evidence/final-reply-light-320-accepted-final.png. |
|
|
562
569
|
| `guide.work-orders-evidence-genuine-opencode-access-repair-20261014` | Genuine Opencode Access Repair 20261014 | Bundled Omnius documentation artifact: docs/work-orders/evidence/genuine-opencode-access-repair-20261014.json. |
|
|
563
570
|
| `guide.work-orders-evidence-inference-device-policy-readonly-20261016` | Inference Device Policy Readonly 20261016 | Bundled Omnius documentation artifact: docs/work-orders/evidence/inference-device-policy-readonly-20261016.json. |
|
|
564
571
|
| `guide.work-orders-evidence-installed-fallback-verification-20261012` | Installed Fallback Verification 20261012 | Bundled Omnius documentation artifact: docs/work-orders/evidence/installed-fallback-verification-20261012.json. |
|
|
@@ -569,6 +576,18 @@ Daemon equivalents are `GET /v1/discovery/bootstrap`, `GET /v1/discovery?q=<inte
|
|
|
569
576
|
| `guide.work-orders-evidence-instrument-80x24` | Instrument 80x24 | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-80x24.png. |
|
|
570
577
|
| `guide.work-orders-evidence-instrument-clean-transcript-120x36` | Instrument Clean Transcript 120x36 | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-clean-transcript-120x36.png. |
|
|
571
578
|
| `guide.work-orders-evidence-instrument-collapse-120x36` | Instrument Collapse 120x36 | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-collapse-120x36.png. |
|
|
579
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-completion-failed-partial` | Completion Failed Partial | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/completion-failed-partial.png. |
|
|
580
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-completion-final-100x30` | Completion Final 100x30 | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-100x30.png. |
|
|
581
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-completion-final-120x36` | Completion Final 120x36 | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-120x36.png. |
|
|
582
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-completion-final-160x48` | Completion Final 160x48 | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-160x48.png. |
|
|
583
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-completion-final-60x20` | Completion Final 60x20 | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-60x20.png. |
|
|
584
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-completion-final-80x24` | Completion Final 80x24 | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-80x24.png. |
|
|
585
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-completion-final-clean` | Completion Final Clean | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/completion-final-clean.png. |
|
|
586
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-completion-live-clean` | Completion Live Clean | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/completion-live-clean.png. |
|
|
587
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-completion-paused-partial` | Completion Paused Partial | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/completion-paused-partial.png. |
|
|
588
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-completion-restored-clean` | Completion Restored Clean | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/completion-restored-clean.png. |
|
|
589
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-completion-technical-expanded` | Completion Technical Expanded | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/completion-technical-expanded.png. |
|
|
590
|
+
| `guide.work-orders-evidence-instrument-completion-live-20261016-report` | Report | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-completion-live-20261016/report.json. |
|
|
572
591
|
| `guide.work-orders-evidence-instrument-devices-expanded-120x36` | Instrument Devices Expanded 120x36 | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-devices-expanded-120x36.png. |
|
|
573
592
|
| `guide.work-orders-evidence-instrument-divider-collapsed-160x48` | Instrument Divider Collapsed 160x48 | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-divider-collapsed-160x48.png. |
|
|
574
593
|
| `guide.work-orders-evidence-instrument-divider-resized-160x48` | Instrument Divider Resized 160x48 | Bundled Omnius documentation artifact: docs/work-orders/evidence/instrument-divider-resized-160x48.png. |
|
|
@@ -646,6 +665,7 @@ Daemon equivalents are `GET /v1/discovery/bootstrap`, `GET /v1/discovery?q=<inte
|
|
|
646
665
|
| `guide.work-orders-evidence-kilo-production-compatibility-20261012` | Kilo Production Compatibility 20261012 | Bundled Omnius documentation artifact: docs/work-orders/evidence/kilo-production-compatibility-20261012.json. |
|
|
647
666
|
| `guide.work-orders-evidence-local-admission-provider-regression-20261014` | Local Admission Provider Regression 20261014 | Bundled Omnius documentation artifact: docs/work-orders/evidence/local-admission-provider-regression-20261014.json. |
|
|
648
667
|
| `guide.work-orders-evidence-local-recovery-production-routes-20261014` | Local Recovery Production Routes 20261014 | Bundled Omnius documentation artifact: docs/work-orders/evidence/local-recovery-production-routes-20261014.json. |
|
|
668
|
+
| `guide.work-orders-evidence-native-completion-presentation-verification-20261016` | Native Completion Presentation Verification 20261016 | Bundled Omnius documentation artifact: docs/work-orders/evidence/native-completion-presentation-verification-20261016.json. |
|
|
649
669
|
| `guide.work-orders-evidence-native-conversation-prompt-audit-20261012` | Native Conversation Prompt Audit 20261012 | Bundled Omnius documentation artifact: docs/work-orders/evidence/native-conversation-prompt-audit-20261012.json. |
|
|
650
670
|
| `guide.work-orders-evidence-native-goal-contract-probe-20261012` | Native Goal Contract Probe 20261012 | Bundled Omnius documentation artifact: docs/work-orders/evidence/native-goal-contract-probe-20261012.json. |
|
|
651
671
|
| `guide.work-orders-evidence-native-interface-state-verification-20261016` | Native Interface State Verification 20261016 | Bundled Omnius documentation artifact: docs/work-orders/evidence/native-interface-state-verification-20261016.json. |
|
|
@@ -670,6 +690,13 @@ Daemon equivalents are `GET /v1/discovery/bootstrap`, `GET /v1/discovery?q=<inte
|
|
|
670
690
|
| `guide.work-orders-evidence-native-tui-semantic-components-20261015` | Native Tui Semantic Components 20261015 | Bundled Omnius documentation artifact: docs/work-orders/evidence/native-tui-semantic-components-20261015.json. |
|
|
671
691
|
| `guide.work-orders-evidence-native-tui-targeted-verification-20261015` | Native Tui Targeted Verification 20261015 | Bundled Omnius documentation artifact: docs/work-orders/evidence/native-tui-targeted-verification-20261015.json. |
|
|
672
692
|
| `guide.work-orders-evidence-native-working-state-verification-20261016` | Native Working State Verification 20261016 | Bundled Omnius documentation artifact: docs/work-orders/evidence/native-working-state-verification-20261016.json. |
|
|
693
|
+
| `guide.work-orders-evidence-omni-analysis-browser-20261017` | Omni Analysis Browser 20261017 | Bundled Omnius documentation artifact: docs/work-orders/evidence/omni-analysis-browser-20261017.json. |
|
|
694
|
+
| `guide.work-orders-evidence-omni-analysis-desktop-controls-20261017` | Omni Analysis Desktop Controls 20261017 | Bundled Omnius documentation artifact: docs/work-orders/evidence/omni-analysis-desktop-controls-20261017.png. |
|
|
695
|
+
| `guide.work-orders-evidence-omni-analysis-desktop-evidence-20261017` | Omni Analysis Desktop Evidence 20261017 | Bundled Omnius documentation artifact: docs/work-orders/evidence/omni-analysis-desktop-evidence-20261017.png. |
|
|
696
|
+
| `guide.work-orders-evidence-omni-analysis-execution-verification-20261017` | Omni Analysis Execution Verification 20261017 | Bundled Omnius documentation artifact: docs/work-orders/evidence/omni-analysis-execution-verification-20261017.json. |
|
|
697
|
+
| `guide.work-orders-evidence-omni-analysis-integration-verification-20261017` | Omni Analysis Integration Verification 20261017 | Bundled Omnius documentation artifact: docs/work-orders/evidence/omni-analysis-integration-verification-20261017.json. |
|
|
698
|
+
| `guide.work-orders-evidence-omni-analysis-phone-controls-20261017` | Omni Analysis Phone Controls 20261017 | Bundled Omnius documentation artifact: docs/work-orders/evidence/omni-analysis-phone-controls-20261017.png. |
|
|
699
|
+
| `guide.work-orders-evidence-omni-analysis-phone-model-20261017` | Omni Analysis Phone Model 20261017 | Bundled Omnius documentation artifact: docs/work-orders/evidence/omni-analysis-phone-model-20261017.png. |
|
|
673
700
|
| `guide.work-orders-evidence-opencode-all-free-audit-20261012` | Opencode All Free Audit 20261012 | Bundled Omnius documentation artifact: docs/work-orders/evidence/opencode-all-free-audit-20261012.json. |
|
|
674
701
|
| `guide.work-orders-evidence-opencode-all-free-installed-verification-20261012` | Opencode All Free Installed Verification 20261012 | Bundled Omnius documentation artifact: docs/work-orders/evidence/opencode-all-free-installed-verification-20261012.json. |
|
|
675
702
|
| `guide.work-orders-evidence-opencode-installed-production-stream-live-20261012` | Opencode Installed Production Stream Live 20261012 | Bundled Omnius documentation artifact: docs/work-orders/evidence/opencode-installed-production-stream-live-20261012.json. |
|
|
@@ -720,6 +747,7 @@ Daemon equivalents are `GET /v1/discovery/bootstrap`, `GET /v1/discovery?q=<inte
|
|
|
720
747
|
| `guide.work-orders-evidence-tray-projects-long-menu` | Tray Projects Long Menu | Bundled Omnius documentation artifact: docs/work-orders/evidence/tray-projects-long-menu.png. |
|
|
721
748
|
| `guide.work-orders-evidence-tray-projects-menu` | Tray Projects Menu | Bundled Omnius documentation artifact: docs/work-orders/evidence/tray-projects-menu.png. |
|
|
722
749
|
| `guide.work-orders-evidence-tui-local-admission-live-20261014` | Tui Local Admission Live 20261014 | Bundled Omnius documentation artifact: docs/work-orders/evidence/tui-local-admission-live-20261014.json. |
|
|
750
|
+
| `guide.work-orders-evidence-tui-response-ownership-verification-20261016` | Tui Response Ownership Verification 20261016 | Bundled Omnius documentation artifact: docs/work-orders/evidence/tui-response-ownership-verification-20261016.json. |
|
|
723
751
|
| `guide.work-orders-evidence-update-operation-cancellation-20261016` | Update Operation Cancellation 20261016 | Bundled Omnius documentation artifact: docs/work-orders/evidence/update-operation-cancellation-20261016.json. |
|
|
724
752
|
| `guide.work-orders-evidence-web-instrument-dark-desktop-20261016` | Web Instrument Dark Desktop 20261016 | Bundled Omnius documentation artifact: docs/work-orders/evidence/web-instrument-dark-desktop-20261016.png. |
|
|
725
753
|
| `guide.work-orders-evidence-web-instrument-dark-phone-20261016` | Web Instrument Dark Phone 20261016 | Bundled Omnius documentation artifact: docs/work-orders/evidence/web-instrument-dark-phone-20261016.png. |
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# Real Omni analysis
|
|
2
|
+
|
|
3
|
+
Acceptance: Audio, Video, and Audio/video Analysis must submit actual media to
|
|
4
|
+
the Omni comprehension runtime. Missing weights or an unavailable runtime must
|
|
5
|
+
never yield synthetic scene entities, events, or a successful mock answer.
|
|
6
|
+
|
|
7
|
+
## Findings
|
|
8
|
+
|
|
9
|
+
- The production AV registry seeded deterministic mock adapters. The browser
|
|
10
|
+
modes sent identical requests, and any live role labeled the entire result live.
|
|
11
|
+
- The old Python sidecar implements detection/tracking only; it cannot make its
|
|
12
|
+
advertised semantic roles real merely by downloading weights.
|
|
13
|
+
- The reference `qwen-omni-adapters-main` runtime exposes adapter v1 on `/api/chat`.
|
|
14
|
+
Its model response echoes the request and is not runtime identity attestation.
|
|
15
|
+
- Read-only inspection found an existing Ornith Omni bridge on GPU 1. This differs
|
|
16
|
+
from the earlier GPU 0/Qwen instruction. The user selected that existing Ornith/GPU 1 lane for live verification.
|
|
17
|
+
Actual placement and manifest identity were verified before inference; no service changed.
|
|
18
|
+
|
|
19
|
+
## Work
|
|
20
|
+
|
|
21
|
+
- [x] Delegate discovery across browser, execution, and reference runtime.
|
|
22
|
+
- [x] Implement one bounded, cancellable Omni media client with real input
|
|
23
|
+
conversion, strict adapter/provenance validation, and explicit configuration.
|
|
24
|
+
- [x] Remove implicit production mock fallback; keep explicit offline fixtures.
|
|
25
|
+
- [x] Route the native AV tool and browser modes through the same client.
|
|
26
|
+
- [x] Show model configuration/readiness and distinct transcript/audio/visual
|
|
27
|
+
evidence; do not claim continuous live capture for a local-file form.
|
|
28
|
+
- [x] Preserve project/session/request identity and same-origin authenticated API.
|
|
29
|
+
- [x] Document shared weight ownership, runtime configuration, and limitations.
|
|
30
|
+
- [x] Verify client, route, frontend, and required chat transport regressions.
|
|
31
|
+
- [x] Exercise the source-bundled browser in a private fixture server.
|
|
32
|
+
- [x] Perform live inference only after exact endpoint/model/GPU verification.
|
|
33
|
+
- [x] Commit scoped source changes and push the tracked main branch.
|
|
34
|
+
|
|
35
|
+
## Delivery boundaries
|
|
36
|
+
|
|
37
|
+
Do not overwrite unrelated dirty work, shared build outputs, publishing artifacts,
|
|
38
|
+
or restart the user's running daemon. Test fixtures may simulate the upstream
|
|
39
|
+
contract but must never be reachable as production fallback adapters.
|
|
40
|
+
|
|
41
|
+
## Verification and remaining model limitations
|
|
42
|
+
|
|
43
|
+
- Execution: 71 tests across 9 suites; CLI source routes and required chat
|
|
44
|
+
transport: 76 tests across 5 suites. Final route/browser runtime rerun: 54 tests
|
|
45
|
+
across 2 suites, including the additional project-alias regression.
|
|
46
|
+
- Private source Chromium: 36 checks across desktop/tablet/phone and both themes.
|
|
47
|
+
- CLI + execution source typecheck passed without shared build output.
|
|
48
|
+
- Seven genuine requests used the approved existing Ornith adapter on GPU 1.
|
|
49
|
+
Red → blue → red freshness and speech in audio-only/combined video passed.
|
|
50
|
+
- **Model quality remains imperfect:** the deployed Ornith bridge missed a real
|
|
51
|
+
440 Hz tone, and one combined answer contradicted its correct blue-screen
|
|
52
|
+
observation. These failures were preserved; no mock answer or alternate model
|
|
53
|
+
was substituted. Improving those weights is a separate model-quality repair.
|
|
54
|
+
- Publishing and daemon state were preserved. Configure the deployed process as
|
|
55
|
+
documented in [Omni analysis](../reference/omni-analysis.md).
|
|
56
|
+
|
|
57
|
+
Receipts:
|
|
58
|
+
[execution](../work-orders/evidence/omni-analysis-execution-verification-20261017.json),
|
|
59
|
+
[browser](../work-orders/evidence/omni-analysis-browser-20261017.json),
|
|
60
|
+
[integration and live results](../work-orders/evidence/omni-analysis-integration-verification-20261017.json).
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
# One final response per run
|
|
2
|
+
|
|
3
|
+
Requested after the retained generation panel, completion tool arguments, and
|
|
4
|
+
final receipt displayed the same answer three times, with the final copy losing
|
|
5
|
+
Markdown structure.
|
|
6
|
+
|
|
7
|
+
## Presentation contract
|
|
8
|
+
|
|
9
|
+
- Preserve the accepted assistant reply, including paragraphs, lists, indentation,
|
|
10
|
+
code fences, tables and exact JSON values. Storage summaries remain separate.
|
|
11
|
+
- Show one expanded final answer per run/agent. Do not promote proposed
|
|
12
|
+
`task_complete` arguments before acceptance.
|
|
13
|
+
- Keep generation handling, retries, raw contracts and operational receipts in
|
|
14
|
+
compact disclosures. Successful settlement must not leave a second expanded
|
|
15
|
+
answer; explicit inspection can still reveal retained diagnostics.
|
|
16
|
+
- Preserve real partial output and truthful failed/interrupted states. Do not
|
|
17
|
+
infer acceptance from a tool name, tool-call arguments, or matching prose.
|
|
18
|
+
- Use stable document ownership rather than deleting nearby matching text.
|
|
19
|
+
- Carry the same accepted reply through JSON mode, browser delivery, persistence
|
|
20
|
+
and TUI resume. Avoid duplicate native media previews and playback.
|
|
21
|
+
|
|
22
|
+
## Work
|
|
23
|
+
|
|
24
|
+
- [x] Parallel review of native generation/completion, main/child event paths,
|
|
25
|
+
CLI/browser transport, formatting sanitizers, and restoration.
|
|
26
|
+
- [x] Format-preserving user-reply sanitizer and accepted-reply JSON selection.
|
|
27
|
+
- [x] Main/child response ownership and removal of redundant lifecycle output.
|
|
28
|
+
- [x] Settled inference disclosures and a minimal rich final document.
|
|
29
|
+
- [x] Canonical resume excludes successful completion-control copies while
|
|
30
|
+
retaining rejected completion evidence.
|
|
31
|
+
- [x] Browser waits for accepted output rather than promoting tool arguments.
|
|
32
|
+
- [x] Guard duplicate media rendering/playback owned by native modules.
|
|
33
|
+
- [x] Meaningful regressions, mandatory transport suites, private typecheck and
|
|
34
|
+
actual terminal fixture verification.
|
|
35
|
+
- [x] Scoped commits and push to `origin/main`; record receipts and limits.
|
|
36
|
+
|
|
37
|
+
## Boundaries
|
|
38
|
+
|
|
39
|
+
Keep unrelated dirty work and publishing outputs intact. No model inference,
|
|
40
|
+
model loading, installed-client replacement or production-daemon changes are
|
|
41
|
+
needed to verify this presentation repair. Test fixtures must identify mocked
|
|
42
|
+
inference separately from actual terminal/browser rendering.
|
|
43
|
+
|
|
44
|
+
## Verified changes
|
|
45
|
+
|
|
46
|
+
- `66e68437`: canonical session restore keeps one rich assistant reply and omits
|
|
47
|
+
successful completion-control copies; rejected or unknown outcomes remain
|
|
48
|
+
collapsed evidence. Old imported copies are retired by exact block identity.
|
|
49
|
+
Nine session restore tests passed.
|
|
50
|
+
- `c6de96d6`: format-preserving accepted-reply selection across CLI, API,
|
|
51
|
+
persisted browser sessions and live browser delivery. Removed speculative
|
|
52
|
+
completion-call promotion and automatic post-answer follow-up inference.
|
|
53
|
+
Verification: 85 tests/6 suites including mandatory transport checks, 54
|
|
54
|
+
actual Chromium fixture checks, and CLI no-emit passed.
|
|
55
|
+
[Browser receipt](../work-orders/evidence/final-reply-browser-20261016.json).
|
|
56
|
+
- `ab17b145`: active technical chatter collapses; settled generation becomes a
|
|
57
|
+
compact disclosure; final output is a rich document with a small receipt.
|
|
58
|
+
Fixed blank finalized native Markdown, preserved failed/interrupted partials,
|
|
59
|
+
and migrated old saved generation panels. Verification: 60 tests/9 suites,
|
|
60
|
+
12 actual VTE checks across five sizes, and CLI no-emit passed.
|
|
61
|
+
[Native receipt](../work-orders/evidence/native-completion-presentation-verification-20261016.json).
|
|
62
|
+
- `4a3f1153`: README reduced from 893 to 101 lines of structural overview.
|
|
63
|
+
Removed free-provider/fallback copy, release detail and repeated inventory;
|
|
64
|
+
the sync script now retains a short link to the canonical API reference.
|
|
65
|
+
REST inventory, README sync, manifest, example checks and two endpoint tests
|
|
66
|
+
passed. Full `docs:check` reports stale generated slash-command documentation;
|
|
67
|
+
a separate discovery check also reports stale `docs/DISCOVERY.*`. Those
|
|
68
|
+
unrelated generated files were not overwritten or staged.
|
|
69
|
+
|
|
70
|
+
Test counts overlap and are not a unique repository-wide total. Live UI checks
|
|
71
|
+
use controlled events and owned fixture files; they do not perform inference.
|
|
72
|
+
|
|
73
|
+
- `717e54c9`: main and child response ownership preserves the accepted document,
|
|
74
|
+
fences late events after interruption/reused child views, removes text-match
|
|
75
|
+
deletion, and avoids duplicate native media preview/playback. Session-context
|
|
76
|
+
persistence retains the complete formatted reply instead of flattening/capping
|
|
77
|
+
it at 1,200 characters. Focused lifecycle verification: 97 tests/10 files and
|
|
78
|
+
CLI no-emit passed. Root integration rerun: 20 tests/4 files and a separate
|
|
79
|
+
CLI no-emit passed.
|
|
80
|
+
[Lifecycle receipt](../work-orders/evidence/tui-response-ownership-verification-20261016.json).
|
|
81
|
+
|
|
82
|
+
## Delivery limits
|
|
83
|
+
|
|
84
|
+
Completed source commits are pushed to `origin/main`; installed clients,
|
|
85
|
+
services, shared build outputs and publishing artifacts are unchanged.
|
|
86
|
+
Externally launched subprocesses still expose their raw process logs. They are
|
|
87
|
+
not treated as structured accepted-response receipts; the current Omnius child
|
|
88
|
+
prints one corrected final, but arbitrary older/external programs are not
|
|
89
|
+
rewritten or deduplicated heuristically.
|
|
90
|
+
|
|
91
|
+
The root integration rerun reported a scratch-directory teardown warning from
|
|
92
|
+
an asynchronous identity save; all assertions passed. It did not perform model
|
|
93
|
+
inference. Full documentation drift checks remain limited as recorded above.
|
|
@@ -0,0 +1,251 @@
|
|
|
1
|
+
# Omni media analysis
|
|
2
|
+
|
|
3
|
+
Omnius uses an explicitly configured Omni adapter for genuine audio and video
|
|
4
|
+
comprehension. A model name containing `-omni` does not add audio/video
|
|
5
|
+
execution to stock Ollama: the adapter must decode the media and run the
|
|
6
|
+
corresponding model/projector weights. An unavailable runtime is an unavailable
|
|
7
|
+
capability, never a reason to substitute synthetic observations.
|
|
8
|
+
|
|
9
|
+
## Runtime and model identity
|
|
10
|
+
|
|
11
|
+
The supported wire contract is `robit.ollama.omni-adapter.v1`, implemented by
|
|
12
|
+
[qwen-omni-adapters](https://github.com/robit-man/qwen-omni-adapters). Its public
|
|
13
|
+
model is a logical bundle. The legacy full bundle runs separate comprehension,
|
|
14
|
+
language and speech graphs across a semantic-text boundary. The trained audio
|
|
15
|
+
bridge runs the selected language trunk and its combined vision/audio projector
|
|
16
|
+
in one llama.cpp server, with Qwen3-TTS available separately. Neither profile is
|
|
17
|
+
a claim that incompatible hidden states form one monolithic graph.
|
|
18
|
+
|
|
19
|
+
| Profile | Exact logical tag | Documented artifact size |
|
|
20
|
+
| --- | --- | --- |
|
|
21
|
+
| Qwen audio bridge | `robit/qwen3.8-27b-e03-obliterated-omni-audio-bridge:q4km` | 18.33 GiB |
|
|
22
|
+
| Ornith audio bridge | `robit/ornith-1.5-omni-audio-bridge:q4km` | 8.15 GiB |
|
|
23
|
+
| Legacy full Qwen Omni | `robit/qwen3.8-27b-e03-obliterated-omni:q4km` | Separate full comprehension graph and sidecar |
|
|
24
|
+
|
|
25
|
+
These are distinct profiles. Selecting Qwen must not silently run Ornith.
|
|
26
|
+
Artifact size is neither peak VRAM nor an admission guarantee.
|
|
27
|
+
|
|
28
|
+
## Omnius configuration
|
|
29
|
+
|
|
30
|
+
Configure the daemon/client environment for the already verified deployment:
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
export OMNIUS_OMNI_ADAPTER_URL=http://127.0.0.1:8910
|
|
34
|
+
export OMNIUS_OMNI_MODEL=robit/ornith-1.5-omni-audio-bridge:q4km
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
This example names Ornith explicitly; use the exact Qwen tag only with an
|
|
38
|
+
endpoint actually deployed with that profile. Setting these variables does not
|
|
39
|
+
download, load or switch the server's model.
|
|
40
|
+
|
|
41
|
+
| Variable | Behavior |
|
|
42
|
+
| --- | --- |
|
|
43
|
+
| `OMNIUS_OMNI_ADAPTER_URL` | HTTP(S) **base URL**, default `http://127.0.0.1:8910`; omit `/api/chat`, query and fragment |
|
|
44
|
+
| `OMNIUS_OMNI_MODEL` | Required exact logical `-omni` tag; no model is discovered or selected implicitly |
|
|
45
|
+
| `OMNI_OLLAMA_MODEL` | Inherited compatibility model value when `OMNIUS_OMNI_MODEL` is absent |
|
|
46
|
+
| `OMNIUS_OMNI_ADAPTER_TOKEN` | Optional bearer token for an authenticated adapter boundary; keep secret |
|
|
47
|
+
|
|
48
|
+
An explicitly requested model must match this server-controlled tag. An
|
|
49
|
+
unconfigured, mismatched or unreachable runtime returns a visible error; it
|
|
50
|
+
does not invoke a different model or synthesize evidence. The shared client
|
|
51
|
+
`packages/execution/src/av/omni-comprehension.ts` exposes
|
|
52
|
+
`getOmniAnalysisStatus()` and `analyzeOmniMedia({path, mode, question?, model?,
|
|
53
|
+
signal?})`; its modes are `audio`, `video` and `av`.
|
|
54
|
+
|
|
55
|
+
The client accepts a regular local media file up to 128 MiB, a finite audio
|
|
56
|
+
duration up to 60 seconds or video duration up to 32 seconds, and a question up
|
|
57
|
+
to 8192 characters. It rejects longer input rather than implicitly clipping it.
|
|
58
|
+
CPU FFmpeg normalization produces 16 kHz PCM16 WAV or 1 fps MP4 fitted within
|
|
59
|
+
640 × 640 pixels while preserving aspect ratio; `video` omits audio and `av`
|
|
60
|
+
includes the source audio when present.
|
|
61
|
+
Normalized media is capped at 32 MiB, adapter JSON at 1 MiB, and each returned
|
|
62
|
+
text field at 128 Ki characters. FFmpeg and ffprobe must already be installed;
|
|
63
|
+
analysis does not install dependencies.
|
|
64
|
+
|
|
65
|
+
Status includes `modelIdentityVerified: false`: availability is a compatible
|
|
66
|
+
configuration handshake, not weight identity, GPU residency or a completed
|
|
67
|
+
inference proof.
|
|
68
|
+
|
|
69
|
+
The reference adapter's `/healthz` and `/api/omni/adapter/contract` expose the
|
|
70
|
+
schema and configured stages, **not the identity of loaded weights**. Its
|
|
71
|
+
response `model` echoes the caller's logical tag. The optional
|
|
72
|
+
`adapter.language_backend_model` can be only an internal alias such as
|
|
73
|
+
`local-audio-bridge`. Those fields do not prove that the requested weights ran.
|
|
74
|
+
Bind an endpoint to its operator-verified deployment, model/projector manifest
|
|
75
|
+
digests and actual accelerator placement before interpreting model results.
|
|
76
|
+
|
|
77
|
+
## Endpoint contract
|
|
78
|
+
|
|
79
|
+
Use the adapter's own endpoint, not the ordinary Ollama endpoint or the Omnius
|
|
80
|
+
daemon. The reference launcher uses these loopback addresses:
|
|
81
|
+
|
|
82
|
+
| Address | Purpose |
|
|
83
|
+
| --- | --- |
|
|
84
|
+
| `http://127.0.0.1:8910/api/chat` | Portable nonstreaming Omni requests |
|
|
85
|
+
| `http://127.0.0.1:8910/api/chat/stream` | Adapter NDJSON extension |
|
|
86
|
+
| `http://127.0.0.1:8910/healthz` | Adapter metadata only |
|
|
87
|
+
| `http://127.0.0.1:8910/api/omni/adapter/contract` | Versioned capability/format metadata |
|
|
88
|
+
| `http://127.0.0.1:8901/v1/chat/completions` | Internal llama.cpp worker; not the portable adapter API |
|
|
89
|
+
| `http://127.0.0.1:8920` | Separate authenticated reference portal |
|
|
90
|
+
|
|
91
|
+
Port `11435` is already used by the Omnius daemon in common installations.
|
|
92
|
+
Some standalone adapter/client defaults also name that port; do not copy those
|
|
93
|
+
defaults over a running Omnius service. Component services bind to loopback.
|
|
94
|
+
Use an authenticated boundary when exposing them beyond the trusted host, and
|
|
95
|
+
never put a portal capability URL or access token in logs or documentation.
|
|
96
|
+
|
|
97
|
+
Read-only contract checks do not load a model:
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
curl --fail --silent http://127.0.0.1:8910/healthz
|
|
101
|
+
curl --fail --silent http://127.0.0.1:8910/api/omni/adapter/contract
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
Do not treat successful health checks as successful media inference.
|
|
105
|
+
|
|
106
|
+
## Requests and results
|
|
107
|
+
|
|
108
|
+
The portable route requires `stream: false`, an exact `model`, and nonempty
|
|
109
|
+
Ollama-shaped `messages`. Set `think: false` independently of media handling.
|
|
110
|
+
Ordinary analysis requests use text output and `speech_mode: "never"`.
|
|
111
|
+
|
|
112
|
+
```json
|
|
113
|
+
{
|
|
114
|
+
"model": "robit/ornith-1.5-omni-audio-bridge:q4km",
|
|
115
|
+
"messages": [{
|
|
116
|
+
"role": "user",
|
|
117
|
+
"content": "Describe the audible events and their uncertainty.",
|
|
118
|
+
"audios": [{
|
|
119
|
+
"mime_type": "audio/wav",
|
|
120
|
+
"encoding": "base64",
|
|
121
|
+
"data": "<complete 16 kHz mono PCM16 WAV, base64 encoded>"
|
|
122
|
+
}]
|
|
123
|
+
}],
|
|
124
|
+
"omni": {
|
|
125
|
+
"schema": "robit.ollama.omni-adapter.v1",
|
|
126
|
+
"task": "describe",
|
|
127
|
+
"include_audio_from_video": true
|
|
128
|
+
},
|
|
129
|
+
"response_modalities": ["text"],
|
|
130
|
+
"speech_mode": "never",
|
|
131
|
+
"think": false,
|
|
132
|
+
"cache_prompt": false,
|
|
133
|
+
"stream": false
|
|
134
|
+
}
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
This is a contract example, not a request to run without hardware preflight.
|
|
138
|
+
|
|
139
|
+
| Task | Executed stages |
|
|
140
|
+
| --- | --- |
|
|
141
|
+
| `describe` | Media comprehension; direct observations |
|
|
142
|
+
| `transcribe` | Audio comprehension; direct transcription |
|
|
143
|
+
| `chat` | Media comprehension, then language analysis; optional speech only if requested |
|
|
144
|
+
| `synthesize` | Speech synthesis from text |
|
|
145
|
+
|
|
146
|
+
The reference comprehension worker always receives `cache_prompt: false`.
|
|
147
|
+
Multimodal slot caching can otherwise reuse stale embeddings across new media.
|
|
148
|
+
Keep new attachments request-local. Historical transcript/observations can be
|
|
149
|
+
explicitly labeled history; restored previews and old raw bytes must not become
|
|
150
|
+
new current evidence automatically.
|
|
151
|
+
|
|
152
|
+
Audio input must be a complete uncompressed RIFF/WAVE file containing 16 kHz,
|
|
153
|
+
mono, signed 16-bit PCM. Resample/downmix supported uploads with a bounded local
|
|
154
|
+
decoder before transport. The adapter validates the WAV header and format; a
|
|
155
|
+
renamed MP3 or browser WebM capture is not a WAV envelope.
|
|
156
|
+
|
|
157
|
+
Images accept JPEG, PNG and WebP. Video accepts MP4, WebM and animated GIF with
|
|
158
|
+
`sampling: { "fps": 1, "max_frames": 32, "include_audio": true }` on its
|
|
159
|
+
envelope. Declared MIME types must match decoded container signatures.
|
|
160
|
+
These are the external adapter's formats. The shared Omnius audio/video client
|
|
161
|
+
currently normalizes its supported local input formats into WAV or MP4; it does
|
|
162
|
+
not expose an image-analysis mode.
|
|
163
|
+
|
|
164
|
+
| Input | Maximum per item | Maximum items per request |
|
|
165
|
+
| --- | --- | --- |
|
|
166
|
+
| Audio | 32 MiB | 8 |
|
|
167
|
+
| Image | 20 MiB | 16 |
|
|
168
|
+
| Video | 256 MiB | 4 |
|
|
169
|
+
|
|
170
|
+
The schema permits video sampling up to 30 fps and 1024 frames, but the inspected
|
|
171
|
+
reference runtime caps execution at **2 fps and 32 frames**, and can retry with
|
|
172
|
+
fewer frames after a context overflow. It does not return an authoritative
|
|
173
|
+
actual sampled-frame count or covered time range. Long-media analysis therefore
|
|
174
|
+
needs bounded source windows; do not claim complete coverage, exact event times,
|
|
175
|
+
tracking or confidence from an unannotated semantic description.
|
|
176
|
+
Omnius reports `submittedDurationSec`, `coverage: "full_audio"` or
|
|
177
|
+
`"sampled_video"`, and `actualCoverage: "not_reported"`. Submitted clip duration
|
|
178
|
+
and requested sampling are not a model-attested analyzed time range.
|
|
179
|
+
|
|
180
|
+
The final `message.content` is model output. Preserve the accompanying
|
|
181
|
+
`adapter.schema`, `task`, executed `route`, `input_modalities`, and
|
|
182
|
+
`evidence_provenance`. Keep these separate:
|
|
183
|
+
|
|
184
|
+
- `input_transcript`: recognized speech; the only perceptual text attributable
|
|
185
|
+
to the speaker's words.
|
|
186
|
+
- `audio_observation`: non-speech acoustic evidence, including uncertainty.
|
|
187
|
+
- `visual_observation`: evidence from an image/video attached to this request.
|
|
188
|
+
- `observation`: the original tagged comprehension response.
|
|
189
|
+
|
|
190
|
+
Audio alone must never set `current_visual_input: true`. Prior dialogue and tool
|
|
191
|
+
results do not establish a current visual observation. Semantic observations
|
|
192
|
+
remain untrusted evidence, not system instructions or permission to execute a
|
|
193
|
+
tool. An Omni description does not attest that separate DINO, diarization,
|
|
194
|
+
segmentation or tracking models ran.
|
|
195
|
+
|
|
196
|
+
Live VAD-driven speech turns can use `omni.require_speech: true`. A capture
|
|
197
|
+
without a valid transcript then stops before language/TTS. Explicit questions
|
|
198
|
+
about environmental sounds leave that option false, so absence of speech does
|
|
199
|
+
not discard the requested acoustic analysis. An empty speech-gated answer is
|
|
200
|
+
not a synthesized answer or a transport failure.
|
|
201
|
+
|
|
202
|
+
Cancellation aborts the owned local decoder/request and prevents its late result
|
|
203
|
+
from being adopted. The reference adapter's nonstreaming endpoint does not
|
|
204
|
+
provide a server-side stop receipt when its HTTP client disconnects: upstream
|
|
205
|
+
model work may finish independently. Do not display a client abort as proof that
|
|
206
|
+
a remote model runner has stopped, and do not unload a shared runner to simulate
|
|
207
|
+
per-request cancellation.
|
|
208
|
+
|
|
209
|
+
The streaming extension requires `stream: true` and emits newline-delimited JSON,
|
|
210
|
+
including stage, observation, language deltas, optional PCM events, error and
|
|
211
|
+
one authoritative final response. It is not SSE. Parse arbitrary chunk and UTF-8
|
|
212
|
+
boundaries; do not treat a heartbeat or `stage: "tts"` as generated audio.
|
|
213
|
+
|
|
214
|
+
## Shared weights and project data
|
|
215
|
+
|
|
216
|
+
Omni weights remain in the runtime's shared model store. On the inspected host,
|
|
217
|
+
installed Ollama manifests and blobs live under `/srv/ollama/models`; standard
|
|
218
|
+
model/projector layers and one `application/vnd.robit.ollama.omni.bundle.v1+gguf`
|
|
219
|
+
sidecar describe the logical tag. Runtime component views are derived artifacts,
|
|
220
|
+
not per-project downloads. Never copy or delete Ollama blobs manually.
|
|
221
|
+
|
|
222
|
+
Omnius' separate media-generation cache resolves from `OMNIUS_MEDIA_HOME`, then
|
|
223
|
+
the persisted `media-store.json` pointer, then its configured home. Its model
|
|
224
|
+
cache and runtimes are shared across projects. That cache is not automatically
|
|
225
|
+
the Ollama Omni store and changing it does not move an existing Omni deployment.
|
|
226
|
+
|
|
227
|
+
Generated project artifacts remain under `<project>/.omnius/images`, `videos`
|
|
228
|
+
and `audio`. Analysis of an existing file should reference its captured project
|
|
229
|
+
and revision; switching sessions/projects must not relabel a late result.
|
|
230
|
+
No per-session Omni weight copy is required.
|
|
231
|
+
|
|
232
|
+
## Deployment and validation boundary
|
|
233
|
+
|
|
234
|
+
Changing or starting CUDA workloads requires the host's GPU broker protocol and
|
|
235
|
+
actual runner-placement verification. Confirm the designated endpoint, exact
|
|
236
|
+
tag and GPU before any inference; do not redirect to another accelerator or a
|
|
237
|
+
remote service after a failure. The external runtime's bootstrap/deploy scripts
|
|
238
|
+
can pull weights, build CUDA binaries and replace services, so they are not
|
|
239
|
+
capability probes and are not part of ordinary Omnius analysis.
|
|
240
|
+
|
|
241
|
+
An endpoint may serve a different logical bundle than a neighboring checkout's
|
|
242
|
+
`.env` file. Verify the running deployment and actual model/projector manifest
|
|
243
|
+
identity; neither a checked-out configuration nor the response's echoed tag is
|
|
244
|
+
sufficient evidence.
|
|
245
|
+
|
|
246
|
+
External contract regression sources are `tests/test_omni_adapter.py` for format,
|
|
247
|
+
routing, provenance, no-speech, stale-cache and stream behavior, plus
|
|
248
|
+
`docs/schema/request-v1.schema.json` and `response-v1.schema.json`. Fixture tests
|
|
249
|
+
can validate transport and parsing without running a model. A live acceptance
|
|
250
|
+
receipt must separately name the actual weights/GPU and use distinguishable real
|
|
251
|
+
media, such as red → blue → red frames, rather than asserting a canned answer.
|
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
This is the maintained human inventory for the supported Omnius daemon API. The
|
|
4
4
|
machine contract is generated by `packages/cli/src/api/openapi.ts` and served at
|
|
5
|
-
`/openapi.json`; every operation in that contract must appear here
|
|
6
|
-
|
|
7
|
-
|
|
5
|
+
`/openapi.json`; every operation in that contract must appear here. The root
|
|
6
|
+
README links to this inventory and stays focused on the system overview.
|
|
7
|
+
Browser HTML routes and implementation-only compatibility
|
|
8
8
|
bridges are documented separately and are not part of the stable REST contract.
|
|
9
9
|
|
|
10
10
|
Run the drift check before publishing docs:
|
|
@@ -385,22 +385,35 @@ the unified Omnius ASR cache and are not shipped in the npm package.
|
|
|
385
385
|
|
|
386
386
|
## Generative Media
|
|
387
387
|
|
|
388
|
-
|
|
388
|
+
Model weights and runtimes use the shared `~/.omnius` store. Generated media
|
|
389
|
+
belongs to the invocation project and is available to its terminal and browser
|
|
390
|
+
sessions. Older global-gallery files remain accessible under the explicit
|
|
391
|
+
legacy scope.
|
|
389
392
|
|
|
390
393
|
| Method | Path | Purpose |
|
|
391
394
|
| --- | --- | --- |
|
|
392
395
|
| `GET` | `/v1/media/models` | List available image/video/audio/music models |
|
|
393
396
|
| `GET` | `/v1/media/store` | Unified store disk usage + reclaimable legacy caches |
|
|
397
|
+
| `GET` | `/v1/media/weights` | Inspect an exact model's cached weights, dependencies, paths, and supported actions |
|
|
398
|
+
| `DELETE` | `/v1/media/weights` | Delete an exact cached dependency with sharing and active-use checks |
|
|
399
|
+
| `POST` | `/v1/media/weights/download` | Start or reattach a weight download without loading a model |
|
|
400
|
+
| `GET` | `/v1/media/weights/jobs/{id}` | Read durable weight-download state and byte progress |
|
|
401
|
+
| `POST` | `/v1/media/weights/jobs/{id}/cancel` | Request cancellation of a daemon-owned weight download |
|
|
394
402
|
| `POST` | `/v1/media/migrate` | Dedup + migrate legacy per-group caches into the unified store |
|
|
395
403
|
| `POST` | `/v1/media/relocate` | Relocate the whole media store (weights/venvs/gallery) to a chosen folder |
|
|
396
404
|
| `GET` | `/v1/media/relocate/status` | Status + progress of the media-store relocation job |
|
|
397
|
-
| `
|
|
405
|
+
| `GET` | `/v1/media/av/status` | Inspect configured Omni adapter availability without loading weights; configuration is not GPU/model identity attestation |
|
|
406
|
+
| `POST` | `/v1/media/av/analyze` | Analyze actual audio/video through the configured Omni runtime, scoped to `projectRoot`; separate transcript/audio/visual evidence, no mock fallback |
|
|
398
407
|
| `POST` | `/v1/media/image` | Generate an image |
|
|
399
408
|
| `POST` | `/v1/media/video` | Generate a video |
|
|
400
409
|
| `POST` | `/v1/media/audio` | Generate a sound effect |
|
|
401
410
|
| `POST` | `/v1/media/music` | Generate music |
|
|
402
|
-
| `GET` | `/v1/media/
|
|
411
|
+
| `GET` | `/v1/media/generations/status` | Inspect generation state for the selected project |
|
|
412
|
+
| `POST` | `/v1/media/generations/cancel` | Cancel an owned generation operation |
|
|
413
|
+
| `GET` | `/v1/media/gallery` | List media for the selected project or explicit legacy scope |
|
|
414
|
+
| `HEAD` | `/v1/media/file` | Validate an exact media artifact and its current revision |
|
|
403
415
|
| `GET` | `/v1/media/file` | Stream one generated media file |
|
|
416
|
+
| `DELETE` | `/v1/media/file` | Delete an exact scoped artifact with optional revision matching |
|
|
404
417
|
|
|
405
418
|
Managed TTS `pull` installs the model's runtime prerequisites and weights. It
|
|
406
419
|
does not start inference. Managed TTS `deploy` also verifies the selected CUDA
|