omnius 1.0.643 → 1.0.647

package/docs/ONTOLOGY-LONG-HORIZON-WORK-GRAPH.md

@@ -0,0 +1,329 @@
+# Ontology WorkGraph Review
+
+Date: 2026-08-28
+
+## Objective
+
+Use an ontological graph to control large and long tasks.
+Make each work item, relation, claim, and evidence item addressable.
+Keep one canonical graph across planning, execution, verification, and completion.
+
+## Reference boundary
+
+The [Ontology Workbench](https://robit-man.github.io/ontology/) is the documentation and reference implementation.
+Its [raw source](https://github.com/robit-man/ontology/blob/main/index.html) defines the contracts and behavior.
+
+Use the implementation as a contract reference.
+Do not copy its browser and IndexedDB monolith into Omnius.
+
+## Main conclusion
+
+Omnius already has most required primitives.
+They exist in separate models with different identifiers and completion rules.
+
+Add a canonical `WorkGraph` projection.
+Do not add another independent planner or evidence store.
+
+The graph must join this chain:
+
+```text
+objective
+  -> requirement
+    -> feature
+      -> component or module
+        -> task
+          -> artifact or mutation
+            -> claim
+              -> verification
+                -> evidence
+```
+
+Relations must also represent dependencies, blockers, contradictions, and supersession.
+
+## Useful Ontology Workbench contracts
+
+The reference separates four concerns:
+
+- Type and schema definitions
+- Mutable runtime instances
+- Claims, evidence, and conflicts
+- Objectives, plans, actions, approvals, and executions
+
+The model proposes a revision-bound patch.
+Deterministic code validates and applies the patch.
+The code records an immutable revision receipt.
+
+The reference also uses target-focused evidence collection.
+It scans all target occurrences before it selects evidence.
+It keeps varied semantic roles and document positions.
+It also keeps supporting and contradicting evidence.
+
+These contracts are more valuable than its first chunking algorithm.
+Omnius already has stronger basic chunking, content hashes, staleness checks, and PDF coverage receipts.
+
+## Strong Omnius foundations
+
+### Work decomposition
+
+- `todo-store.ts` has stable identifiers, parents, blockers, owners, dependencies, verifiers, and artifacts.
+- `todoTruth.ts` derives hierarchy, open leaves, dependency state, and parent completion.
+- `featurePlanner.ts` records paths, symbols, tests, evidence, questions, units, and return contracts.
+- `featureNode.ts` provides recursive survey, plan, split, execute, verify, and eviction stages.
+
+### Completion and closure
+
+- `completionLedger.ts` records claims, evidence roles, risks, critiques, unresolved items, and terminal receipts.
+- `deliveryCoverage.ts` has versioned manifests and deterministic coverage evaluation.
+- `integrationClosure.ts` already projects requirements, claims, paths, contracts, evidence, and blockers into a graph.
+- `completion-evidence-gate.ts` requires independent evidence for high-risk claims.
+
+### Evidence and memory
+
+- `evidenceLedger.ts` records content hashes, revisions, ranges, fidelity, mutations, and staleness.
+- `contextMemoryLedger.ts` records authority, trust, temporal validity, relations, and dependency-closed transactions.
+- `evidenceBranch.ts` validates selected ranges against presented windows and original source text.
+- PDF receipts distinguish absence, incomplete coverage, truncation, and omitted pages.
+
+### Retrieval
+
+- Hybrid retrieval combines lexical, vector, and graph candidates.
+- The context assembler supports rank fusion, graph expansion, snippet packing, and request budgets.
+- The memory ledger supports temporal and source-aware retrieval with explicit abstention.
+
+## Critical gaps
+
+### No canonical work identity
+
+Tasks, todos, feature nodes, missions, claims, and memory records use separate identifiers.
+They cannot form one durable requirement-to-evidence path.
+
+### Weak requirement identity
+
+Many requirements and acceptance criteria remain unkeyed prose.
+The integration graph creates one synthetic current-goal requirement.
+Todos do not carry requirement and acceptance identifiers.
+
+### Isolated feature systems
+
+The feature planner, feature tree, and mission artifacts have limited production wiring.
+Their rich contracts do not control the main completion path.
+
+### Advisory completion evidence
+
+The active completion gate records graph gaps but does not stop completion.
+Final reconciliation does not revoke an accepted completion result.
+
+Keep inferred work advisory.
+Make explicit requirements and acceptance paths authoritative.
+
+### No world transaction boundary
+
+`PatchProposal` does not include a base revision or expected source versions.
+It also lacks typed semantic operations and a resulting revision receipt.
+
+Current stale-write controls operate below the proposal contract.
+The system cannot validate one proposal atomically against its observed world.
+
+### No first-class conflict object
+
+Omnius can mark a claim as contradicted.
+It cannot preserve two active incompatible claims in one conflict group.
+
+Add subject, predicate, valid-time, alternatives, authority, confidence, and resolution rationale.
+
+### Incomplete evidence diversity
+
+Several retrieval paths keep one occurrence per file or source.
+Some structured reads select only leading rows or leading text.
+
+This behavior can omit late updates, repeated events, and counterevidence.
+It can also confuse an unobserved fact with an absent fact.
+
+### Fragmented source bindings
+
+Each package uses a different locator and provenance shape.
+Most general records support only line or character ranges.
+
+Add page, sheet, row, heading, table, field, and JSON-path locators.
+
+## Canonical WorkGraph contract
+
+Use these node kinds:
+
+- `objective`
+- `requirement`
+- `feature`
+- `component`
+- `module`
+- `task`
+- `acceptance`
+- `symbol`
+- `artifact`
+- `mutation`
+- `claim`
+- `verification`
+- `evidence`
+- `conflict`
+- `blocker`
+
+Use these core relations:
+
+- `decomposes_to`
+- `part_of`
+- `depends_on`
+- `satisfies`
+- `implements`
+- `modifies`
+- `produces`
+- `verified_by`
+- `evidenced_by`
+- `contradicts`
+- `blocked_by`
+- `supersedes`
+
+Each node and relation must have a stable identifier.
+Each inferred item must record its authority.
+An inferred item remains advisory until code or a user adopts it.
+
+Each action node must define these fields when they apply:
+
+- Target references
+- Inputs
+- Preconditions
+- Expected mutations
+- Side effects
+- Required authority
+- Required approval
+- Risk
+- Verification contract
+
+## Shared evidence contract
+
+Add one `SourceBinding` and `EvidenceRef` schema under `packages/schemas`.
+
+The schema must include these fields:
+
+- Evidence identifier
+- Source identifier and URI
+- Content hash and source revision
+- Observation time and freshness
+- Authority and confidence
+- Typed locator and locator value
+- Supporting or refuting polarity
+- Coverage and truncation status
+- State epoch
+
+Use the same evidence identifier in retrieval, memory, claims, completion, and groundedness.
+Validate each reference against the current source revision.
+
+## Revisioned world reducer
+
+Add a pure reducer under `packages/orchestrator/src/operational-world/`.
+Keep `AgenticRunner`, context systems, and the TUI as consumers.
+
+The model can propose only a `WorldPatch`.
+The patch must include `patchId`, `baseRevision`, and typed operations.
+
+The reducer must do these checks:
+
+1. Verify the base revision.
+2. Validate every runtime schema.
+3. Verify every node, relation, claim, and evidence reference.
+4. Verify expected file hashes and read versions.
+5. Reject duplicate identifiers and dangling relations.
+6. Reject invalid cycles and temporal intervals.
+7. Preserve incompatible claims as a conflict.
+8. Apply operations in a deterministic order.
+9. Emit an immutable patch and revision receipt.
+
+Use `integrationClosure.ts` as the first graph adapter.
+Do not make its bounded display neighborhood the authoritative validation universe.
+
+## Large-document evidence pipeline
+
+Use a source-aware decomposition tree.
+Each child shard must retain its parent and exact locator.
+
+Select shard boundaries by content type:
+
+- Keep prose headings with their paragraphs.
+- Keep JSON paths and array ranges.
+- Repeat table headers for each table shard.
+- Keep PDF page markers.
+- Keep spreadsheet sheet and row markers.
+
+If one extraction fails, subdivide only that shard.
+Preserve the failed shard as unresolved evidence.
+Do not discard it.
+
+For a named target, use this retrieval sequence:
+
+1. Resolve exact target entities.
+2. Scan all target occurrences.
+3. Build section-bounded evidence windows.
+4. Classify each window by semantic role and polarity.
+5. Keep earliest and latest relevant evidence.
+6. Keep early, middle, and late document coverage.
+7. Expand one hop to co-referenced entities.
+8. Include supporting and contradicting evidence.
+9. Trim background evidence before target evidence.
+10. Emit a typed coverage receipt.
+
+Pass one validated `ReasoningSlice` to the reasoner and critic.
+Reject unknown citations and omitted requested targets before model criticism.
+
+## Completion invariant
+
+Use this rule for authoritative work:
+
+> Complete an objective only when every explicit requirement and acceptance path reaches fresh successful evidence after the last relevant mutation.
+
+Do not let a model-created node silently expand the user's required scope.
+Keep each inferred edge advisory until adoption.
+
+## Immediate correctness repairs
+
+Repair these issues before authoritative graph gating:
+
+- Call `verifyUnit` during the feature-node verify stage.
+- Do not mark an exception fallback complete without evidence.
+- Reject missing dependency identifiers and dependency cycles in RALPH plans.
+- Do not serialize independent modules through artificial dependencies.
+- Preserve feature decomposition fields when units become todos.
+- Wire the existing retrieval provider into the production runner.
+- Add free-form lexical retrieval to the context assembler.
+- Remove one-occurrence limits for evidence-mode retrieval.
+
+## Rollout
+
+### Phase 1: Shared identity
+
+Add shared work, relation, source-binding, evidence, conflict, and revision schemas.
+Project current stores into stable graph identifiers.
+
+### Phase 2: Shadow world
+
+Record proposed patches, validation results, revisions, and receipts.
+Compare graph closure with current completion decisions.
+Do not change completion behavior in this phase.
+
+### Phase 3: Retrieval and document evidence
+
+Wire the existing retrieval systems into production context assembly.
+Add evidence diversity, counterevidence, structural locators, and coverage receipts.
+
+### Phase 4: Authoritative explicit closure
+
+Gate only explicit requirement and acceptance paths.
+Keep inferred paths advisory.
+Require fresh evidence after each relevant mutation.
+
+### Phase 5: Parallel long-horizon execution
+
+Schedule ready leaves whose dependency paths are satisfied.
+Assign ownership and authority to each leaf.
+Commit graph patches through the revisioned reducer.
+Reconcile conflicts before dependent work begins.
+
+This design lets Omnius decompose extreme complexity into durable ontological work.
+It also keeps progress, evidence, and completion lossless across long sessions.

package/docs/reference/configuration.md

@@ -27,7 +27,9 @@ Configuration is layered from environment, global user settings, project setting
 | `OMNIUS_SEMANTIC_AUDIO_AUTO_SETUP` | Set to `0` to disable CLAP setup specifically; JetPack default is enabled |
 | `OMNIUS_VISION_AUTO_SETUP` | Set to `0` to disable managed OpenCLIP provisioning on JetPack |
 | `OMNIUS_VISION_PYTHON` | Explicit vendor CUDA Python for the isolated OpenCLIP runtime |
+| `OMNIUS_VISION_MIN_AVAILABLE_MB` | OpenCLIP-specific Jetson unified-memory admission threshold (default `8192`) |
 | `OMNIUS_NEMO_SPEECH_BIN` | Optional existing CUDA-enabled NeMo-Speech.cpp binary; otherwise live diarization builds the pinned source revision |
+| `OMNIUS_DIAR_NVCC` | Optional absolute JetPack CUDA compiler; discovery otherwise checks `/usr/local/cuda-12.2/bin/nvcc` then `/usr/local/cuda/bin/nvcc` |
 | `OMNIUS_DIARIZATION_AUTO_SETUP` | Set to `0` to disable managed diarizer provisioning; live Sortformer defaults on for JetPack |
 | `OMNIUS_HF_TOKEN` | Setup-only Hugging Face token for gated Community-1 download; never persisted or passed to inference |
 | `OMNIUS_PYANNOTE_TERMS_ACCEPTED` | Set to `1` only after accepting Community-1 terms; permits gated daemon bootstrap when a token is present |
@@ -52,7 +54,8 @@ upstream provider credentials.
 
 Linux installations load optional daemon-only overrides from
 `~/.config/omnius/daemon.env`. The postinstall creates this file with mode
-`0600` and the systemd unit references it with an optional `EnvironmentFile`.
+`0600` and creates or rewrites the systemd unit with
+`EnvironmentFile=%h/.config/omnius/daemon.env`.
 This is the appropriate place for `OMNIUS_AUDIO_PYTHON` or a setup-only gated
 model token; credentials do not need to be embedded in the unit or sent in a
 REST request.

package/docs/rest/endpoints/voice-vision.md

@@ -81,9 +81,10 @@ descriptive claims or invoke an LLM.
 First poll `GET /v1/vision/embed/readiness`. It is non-mutating and returns
 HTTP 503 until the isolated runtime and checksum-manifested weights are ready.
 Use the admin-scoped `POST /v1/vision/embed/setup` to create the
-`--system-site-packages` runtime under
-`~/.omnius/runtimes/vision/open-clip`, install pinned non-Torch dependencies,
-and fetch/verify the checkpoint from immutable revision
+private runtime under `~/.omnius/runtimes/vision/open-clip` with
+`include-system-site-packages = false`, link only the already-validated vendor
+Torch provider, install the pinned non-Torch closure (including `wcwidth`), and
+fetch/verify the checkpoint from immutable revision
 `1a25a446712ba5ee05982a381eed697ef9b435cf`. Setup is single-flight and
 daemon-bootstrapped by default on JetPack; set `OMNIUS_VISION_AUTO_SETUP=0`
 to disable it. Inference never installs packages or downloads artifacts, and
@@ -95,8 +96,10 @@ present in the compatible vendor stack or be supplied explicitly with the local
 `OMNIUS_VISION_TORCHVISION_WHEEL_SHA256`; generic PyPI Torch/torchvision is
 blocked. Bootstrap discovery may inspect the selected vendor interpreter, but
 the managed runtime installs its own pinned support closure and proves Torch,
-torchvision, and OpenCLIP with user-site packages disabled. On memory-constrained
-Jetson systems model loading and embedding fail with a typed 503 rather than
+torchvision, and OpenCLIP with user-site packages disabled. OpenCLIP reports
+its own `component: openclip` memory-admission result, governed by
+`OMNIUS_VISION_MIN_AVAILABLE_MB`, rather than reusing CLAP telemetry. On
+memory-constrained Jetson systems model loading and embedding fail with a typed 503 rather than
 evicting a resident ASR/Ollama model. Dependency and
 weight provisioning itself is allowed to finish under transient pressure;
 readiness separately reports `installed`, `weightsReady`,
@@ -280,7 +283,10 @@ startup provisions acoustic, speaker, and semantic roles by default; set
 `OMNIUS_SEMANTIC_AUDIO_AUTO_SETUP=0` to disable CLAP specifically.
 
 CLAP provisioning installs a checksum-locked CPython 3.10/aarch64 wheel
-closure and the immutable model revision without loading the model. It is
+closure—including Pillow and the complete probed Transformers CLAP import
+surface—inside a private venv with `include-system-site-packages = false`.
+Only the validated vendor CUDA Torch site is linked explicitly. The immutable
+model revision is provisioned without loading the model. It is
 allowed to finish while unified memory is busy. Worker activation and
 inference retain the 8 GiB admission gate and idle eviction, so setup cannot
 silently evict ASR, Ollama, or another CUDA workload.
@@ -344,8 +350,11 @@ exact remediation. Inference is likewise non-provisioning and returns typed
 `speaker_diarization_runtime_unavailable` until admin setup succeeds.
 
 Admin setup is asynchronous and single-flight. It returns HTTP 202 while it
-provisions, and readiness exposes the exact phase or terminal error. Existing
-ready workers return HTTP 200. Inference never performs these setup actions.
+provisions, and readiness exposes `setup.stage`, `setup.failed_stage`, the
+complete captured setup `stderr`, and the terminal error. Those diagnostics
+are persisted under the managed runtime so they survive daemon restart.
+Existing ready workers return HTTP 200. Inference never performs these setup
+actions.
 
 On JetPack, live setup with an empty object downloads the immutable,
 checksum-pinned Q8 Sortformer artifact and builds NVIDIA NeMo-Speech.cpp at a
@@ -360,7 +369,9 @@ POST /v1/audio/diarization/live/setup
 The native build requires the JetPack CUDA compiler plus `git` and a C++17
 compiler. Omnius installs pinned CMake/Ninja only in its private build-tools
 venv and never invokes `sudo`; missing native prerequisites are reported by
-readiness for operator installation outside inference. Set
+readiness for operator installation outside inference. CUDA compiler discovery
+checks `OMNIUS_DIAR_NVCC`, `/usr/local/cuda-12.2/bin/nvcc`, then
+`/usr/local/cuda/bin/nvcc`, and reports those exact locations when none exists. Set
 `OMNIUS_NEMO_SPEECH_BIN` to reuse a prebuilt CUDA-enabled binary.
 
 Managed Community-1 setup creates a separate CPU-only CPython environment, so