omnius 1.0.646 → 1.0.648

package/docs/ASD-STE100-COMMUNICATION-AUDIT.md

@@ -0,0 +1,233 @@
+# ASD-STE100 Communication Audit
+
+Date: 2026-08-28
+
+## Objective
+
+Use ASD-STE100 Issue 9 for all Omnius natural-language communication.
+Apply the rule to external replies and internal agent messages.
+Preserve exact machine content without a change.
+
+## Standard boundary
+
+[ASD-STE100 Issue 9](https://www.asd-ste100.org/assets/files/ASD-STE100_ISSUE9.pdf) has 53 writing rules.
+It also has an approved-word dictionary and permits necessary technical terms.
+The standard requires consistent technical terms, short sentences, direct verbs, and clear procedures.
+
+Issue 9 limits a procedure sentence to 20 words.
+It limits a descriptive sentence to 25 words.
+It requires one instruction in most procedure sentences.
+It also requires quoted text to remain unchanged.
+
+The [official STE description](https://www.asd-ste100.org/about.html) identifies an approved dictionary and permitted technical terms.
+The [official checker guidance](https://www.asd-ste100.org/ste-software.html) states that software aids are not complete compliance proof.
+
+Therefore, Omnius can enforce structural STE rules automatically.
+A qualified human must confirm full dictionary compliance.
+Omnius must not claim certified compliance from a model or heuristic check.
+
+## 2026 reliability evidence
+
+Current research supports three separate communication layers.
+
+1. Use controlled natural language for human and agent prose.
+2. Use schemas and exact fields for machine messages.
+3. Use code validators for syntax and meaning.
+
+A 2026 protocol review reports weak support for clarification, context alignment, and semantic verification.
+Omnius now names these duties in the common contract.
+Source: [Beyond Message Passing](https://arxiv.org/abs/2604.02369).
+
+A 2026 harness study reports that prompt text alone did not preserve critical guarantees.
+It assigns deterministic guarantees to code, schemas, manifests, and validators.
+Omnius now uses this rule in the common contract.
+Source: [From Prompts to Contracts](https://arxiv.org/abs/2607.08028).
+
+A 2026 structured-output study separates valid syntax from valid meaning.
+It also reports model-specific failure patterns.
+Omnius must validate JSON structure and field meaning.
+Source: [StructHallu-Drift](https://aclanthology.org/2026.surgellm-1.22/).
+
+A 2026 reliability framework treats retries and verification as different recovery operators.
+Omnius must classify a failure before it selects a recovery method.
+Source: [Cost-Aware Adaptive Reliability](https://arxiv.org/abs/2605.09121).
+
+2025 structured-output studies remain directly applicable.
+They show that schema constraints need adapted prompts, examples, and validators.
+Sources: [The Hidden Cost of Structure](https://aclanthology.org/2025.ranlp-1.124/) and [Schema Reinforcement Learning](https://aclanthology.org/2025.acl-long.243/).
+
+## Lossless exception boundary
+
+Do not change these items during STE conversion:
+
+- Code and code blocks
+- JSON, schemas, keys, and enum values
+- Commands, paths, options, and environment keys
+- API routes, headers, status codes, and media types
+- Tool names and tool argument names
+- Model names and provider names
+- Sentinels, tags, placeholders, and parser labels
+- User input and source quotations
+- Logs, errors, traces, and tool output
+- Required literal replies
+
+Use STE for text that introduces or explains these items.
+If STE conflicts with an exact contract, the exact contract has priority.
+
+## Runtime architecture review
+
+| Surface | Previous condition | Current control | Status |
+|---|---|---|---|
+| Small tier | Large duplicated prose with long sentences | Shared common contract and small-tier addition | Updated |
+| Medium tier | Large duplicated prose with long sentences | Shared common contract and medium-tier addition | Updated |
+| Large tier | Large duplicated prose with long sentences | Shared common contract and large-tier addition | Updated |
+| Ollama and compatible providers | No common language envelope | Provider adds one STE policy message | Updated |
+| Anthropic adapter | Independent system translation | Provider policy enters before translation | Updated |
+| Native Ollama stream | Independent message path | Provider policy enters before native transport | Updated |
+| Nexus remote peer | Independent message path | Nexus adds the same policy | Updated |
+| Cascade backend | Delegates to Ollama or Nexus | Inner backend adds the policy | Updated |
+| Web chat | Separate non-tier system prompt | Web prompt starts with the shared policy | Updated |
+| Realtime and phone | Explicit contraction preference | Shared policy and direct spoken rules | Updated |
+| Task templates | Long compound instructions | Eight source templates use short direct rules | Updated |
+| Telegram router | Many inline contracts and exact schemas | Provider policy governs output | Controlled rewrite required |
+| Sub-agent delegation | Separate preambles and exact envelopes | Provider policy governs output | Controlled rewrite required |
+| RALPH runners | Separate JSON-sensitive prompt files | Provider policy governs output | Controlled rewrite required |
+| Memory and compaction | Separate JSON and summary prompts | Provider policy governs output | Controlled rewrite required |
+| CLI dream and emotion prompts | Creative and machine output contracts | Provider policy governs output | Profile-specific review required |
+| Reference contracts | 39 long Markdown contracts | Included under the common policy | Controlled rewrite required |
+| OpenAPI and help text | Human prose in TypeScript literals | Not model-generated | User-interface review required |
+
+The final provider boundary adds the policy before each model request.
+This rule covers inline prompts that still need a controlled source rewrite.
+It also covers new prompts that a feature adds later.
+The policy does not change exact machine content.
+
+## Source inventory
+
+The prompt audit found 83 source Markdown files.
+Generated files under `publish/` are not source files.
+
+- CLI TUI prompts: 7 files
+- Prompt package contracts and templates: 15 files
+- Orchestrator prompts: 22 files
+- Root reference contracts: 39 files
+
+TypeScript also contains direct model prompts and user-interface messages.
+These strings need a separate literal extractor because many strings contain machine data.
+
+## Main defects found
+
+### Tier prompts
+
+The three tier files duplicated nearly all content.
+Many sentences had more than 25 words.
+Several sentences contained more than one instruction.
+The smaller tiers did not have a materially smaller common contract.
+
+The new design keeps one common contract.
+Each tier file now contains only its tier rules.
+This design prevents language drift between tiers.
+
+### Realtime prompt
+
+The previous prompt explicitly permitted contractions.
+This rule directly conflicted with STE.
+The fallback reply also used a contraction.
+
+The new prompt prohibits contractions.
+The fallback now uses two short sentences.
+An exact user-requested reply remains exempt.
+
+### Task templates
+
+The source templates used long labels, questions, vague terms, and compound instructions.
+The revised templates use one direct action in each procedure sentence.
+Exact placeholders and tool names remain unchanged.
+
+### Telegram and tool contracts
+
+Telegram router prompts contain safety, identity, route, and reply policies.
+They also contain exact JSON fields and enum values.
+A broad text conversion can change router behavior.
+
+Tool contracts contain exact JSON, tool names, argument names, and parser sentinels.
+These elements need token-preservation tests before a controlled prose conversion.
+
+### Corrected prompt defect
+
+`packages/cli/prompts/tui/emotion-center.md` had an unclosed parenthesis.
+Its examples also conflicted with its required emoji-and-word output.
+The revised prompt has short instructions and valid examples.
+A test now protects the placeholders and required output format.
+
+## Telegram router failure review
+
+The live failure was not an unavailable model.
+The broker completed the requests with hidden reasoning tokens.
+Omnius then removed the hidden text and received no visible decision contract.
+
+One predicate controlled two different facts.
+It controlled Ollama transport features and Omnius pool ownership.
+The broker discovery correctly disabled the local Omnius pool.
+That result also disabled Ollama transport behavior by mistake.
+
+The affected requests omitted `think:false` and Ollama context options.
+They also used `/v1/chat/completions` instead of the preferred native `/api/chat` route.
+The model used its reply budget for hidden reasoning.
+
+The backend now uses two predicates.
+
+- `isOllamaTransport` controls Ollama request fields and native routes.
+- `useOllamaPool` controls only local runner acquisition and spawn behavior.
+
+Broker-managed endpoints now use native Ollama transport without a local pool slot.
+Regression tests cover unary and streaming requests through `http://localhost:11434`.
+The tests also prove that Omnius does not acquire a local pool slot.
+
+The Telegram bridge had a separate token-count defect.
+It read usage fields from the wrong level of each stream chunk.
+It now reads the `usage` object, which corrects the false `~0 tok` display.
+
+Each model-generation block now has a bottom `read mode` control.
+The control expands all preserved thinking, content, tool-contract, and handling rows.
+The control remains available after the block completes.
+The Telegram chat fast path now streams into its block and closes the block on success or failure.
+
+## Validation policy
+
+The structural check must mask exact machine content.
+It must then inspect only authored prose.
+
+Use these hard checks for core prompts:
+
+- No contraction
+- No semicolon
+- No unresolved placeholder
+- Balanced delimiters outside protected content
+- Maximum 20 words for a procedure sentence
+- Maximum 25 words for a descriptive sentence
+- One instruction in each procedure sentence
+- Exact preservation of machine tokens
+
+Do not use automatic passive-voice detection as a hard failure.
+Do not use an incomplete dictionary as compliance proof.
+
+## Remaining controlled work
+
+Use a token manifest before any broad prompt conversion.
+Record placeholders, schema keys, enum values, tags, sentinels, and tool names.
+Require the same token set after each prose change.
+
+Convert the remaining groups in this order:
+
+1. Telegram router and reflection prose
+2. Sub-agent and delegation prose
+3. Recovery and completion prose
+4. RALPH runner prose
+5. Memory and compaction prose
+6. CLI setup and recovery text
+7. API descriptions and help text
+8. Reference contracts
+
+Run parser and snapshot tests after each group.
+Do not use a repository-wide blind text replacement.

package/docs/DISCOVERY.json

@@ -28394,6 +28394,39 @@
         }
       ]
     },
+    {
+      "id": "guide.asd-ste100-communication-audit-uppercase",
+      "kind": "guide",
+      "title": "ASD-STE100 Communication Audit",
+      "summary": "Use ASD-STE100 Issue 9 for all Omnius natural-language communication. Apply the rule to external replies and internal agent messages. Preserve exact machine content without a change.",
+      "keywords": [
+        "ASD",
+        "STE100",
+        "COMMUNICATION",
+        "AUDIT",
+        "md"
+      ],
+      "maturity": "stable",
+      "audiences": [
+        "user",
+        "integrator",
+        "coding-agent"
+      ],
+      "layer": "documentation",
+      "interfaces": [
+        {
+          "type": "file",
+          "target": "docs/ASD-STE100-COMMUNICATION-AUDIT.md"
+        }
+      ],
+      "references": [
+        {
+          "type": "documentation",
+          "target": "docs/ASD-STE100-COMMUNICATION-AUDIT.md",
+          "relation": "canonical-artifact"
+        }
+      ]
+    },
     {
       "id": "guide.concept-relational-language",
       "kind": "guide",
@@ -29533,6 +29566,40 @@
         }
       ]
     },
+    {
+      "id": "guide.omnius-reliability-long-horizon-todo-uppercase",
+      "kind": "guide",
+      "title": "Omnius Reliability and Long-Horizon Workboard",
+      "summary": "This file is the durable workboard for the current repair program. It records scope, dependencies, evidence, and closure rules.",
+      "keywords": [
+        "OMNIUS",
+        "RELIABILITY",
+        "LONG",
+        "HORIZON",
+        "TODO",
+        "md"
+      ],
+      "maturity": "stable",
+      "audiences": [
+        "user",
+        "integrator",
+        "coding-agent"
+      ],
+      "layer": "documentation",
+      "interfaces": [
+        {
+          "type": "file",
+          "target": "docs/OMNIUS-RELIABILITY-LONG-HORIZON-TODO.md"
+        }
+      ],
+      "references": [
+        {
+          "type": "documentation",
+          "target": "docs/OMNIUS-RELIABILITY-LONG-HORIZON-TODO.md",
+          "relation": "canonical-artifact"
+        }
+      ]
+    },
     {
       "id": "guide.omnius-self-edit-eval-2026-06-10",
       "kind": "guide",
@@ -29569,6 +29636,40 @@
         }
       ]
     },
+    {
+      "id": "guide.ontology-long-horizon-work-graph-uppercase",
+      "kind": "guide",
+      "title": "Ontology Long-Horizon Work Graph",
+      "summary": "Use an ontological graph to control large and long tasks. Make each work item, relation, claim, and evidence item addressable. Keep one canonical graph across planning, execution, verification, and completion.",
+      "keywords": [
+        "ONTOLOGY",
+        "LONG",
+        "HORIZON",
+        "WORK",
+        "GRAPH",
+        "md"
+      ],
+      "maturity": "stable",
+      "audiences": [
+        "user",
+        "integrator",
+        "coding-agent"
+      ],
+      "layer": "documentation",
+      "interfaces": [
+        {
+          "type": "file",
+          "target": "docs/ONTOLOGY-LONG-HORIZON-WORK-GRAPH.md"
+        }
+      ],
+      "references": [
+        {
+          "type": "documentation",
+          "target": "docs/ONTOLOGY-LONG-HORIZON-WORK-GRAPH.md",
+          "relation": "canonical-artifact"
+        }
+      ]
+    },
     {
       "id": "guide.opencode-agentic-loop-comparison",
       "kind": "guide",

package/docs/DISCOVERY.md

@@ -407,6 +407,7 @@ Daemon equivalents are `GET /v1/discovery/bootstrap`, `GET /v1/discovery?q=<inte
 | `guide.agent-memory-index-uppercase` | Agent-Explorable Documentation | Omnius documentation is exposed to agents through project-local AIWG-style bundles under .aiwg/addons/. |
 | `guide.architecture-agent-system-map` | Omnius Agent System Map | Use this page when you need to understand how a user-visible behavior travels through Omnius, where its state lives, and which package owns a change. For a specific task recipe, search the generated catalog first: |
 | `guide.architecture-overview` | Architecture Overview | Omnius combines a terminal-first agent loop, REST daemon, model routing layer, tool runtime, persistent context, and peer mesh. |
+| `guide.asd-ste100-communication-audit-uppercase` | ASD-STE100 Communication Audit | Use ASD-STE100 Issue 9 for all Omnius natural-language communication. Apply the rule to external replies and internal agent messages. Preserve exact machine content without a change. |
 | `guide.concept-relational-language` | Concept Relational Language (CRL) — Token-Efficient Concept Communication | &gt; Experimental hyper-compressed symbolic notation for mind-mapped logical flow tracking |
 | `guide.context-management-medium-models-proposal` | Context Management for Medium Models (30-40B) — Implementation Proposal | Date: 2026-04-25 Problem: Medium-tier models (~35B params) exhibit excessive repetition at ~35% context fill despite 256K context windows Root Cause: Monolithic context structure + attention degradation + tool schema bloat |
 | `guide.dedup-false-positive-meta-analysis` | Meta-Analysis: False Positive Duplicate Tool Call Detection | Date: 2026-05-14 Scope: packages/orchestrator/src/agenticRunner.ts — proactivePrune() + buildResourceKey() |
@@ -442,7 +443,9 @@ Daemon equivalents are `GET /v1/discovery/bootstrap`, `GET /v1/discovery?q=<inte
 | `guide.memory-integration-analysis` | Memory Integration Analysis — Parallel Systems Gap Assessment | Date: 2026-04-23 Status: ✅ INTEGRATED — All P0 gaps closed Context: Review of literature on memory compression + integration testing of Omnius parallel memory systems |
 | `guide.model-capability-awareness-and-multimodal-memory-root-fix` | Model Capability Awareness And Multimodal Identity Memory Root Fix | Status: planning and integration tracker |
 | `guide.multimodal-identity-memory-implementation` | Multimodal Identity Memory Implementation Tracker | Goal: make voice, text, image, Telegram reply context, CLIP-like embeddings, zettelkasten links, and social profiles converge on one evidence-based identity substrate. |
+| `guide.omnius-reliability-long-horizon-todo-uppercase` | Omnius Reliability and Long-Horizon Workboard | This file is the durable workboard for the current repair program. It records scope, dependencies, evidence, and closure rules. |
 | `guide.omnius-self-edit-eval-2026-06-10` | Omnius Self-Edit Evaluation — Uncommitted Changes &amp; Duplicate-Tool-Call Failure Mode | &gt; Eval of the working-tree changes against docs/opencode-agentic-loop-comparison.md. &gt; Subject: an Omnius instance editing its own orchestrator to implement the P0–P10 &gt; learnings. Generated 2026-06-10. |
+| `guide.ontology-long-horizon-work-graph-uppercase` | Ontology Long-Horizon Work Graph | Use an ontological graph to control large and long tasks. Make each work item, relation, claim, and evidence item addressable. Keep one canonical graph across planning, execution, verification, and completion. |
 | `guide.opencode-agentic-loop-comparison` | OpenCode → Omnius: Agentic Loop &amp; Sub-Agent Delegation Comparison | &gt; Exhaustive comparison between https://github.com/anomalyco/opencode/tree/dev and &gt; packages/orchestrator/src/agenticRunner.ts (omnius). &gt; Generated 2026-06-10. |
 | `guide.operations-delay-analysis` | Unnecessary Causes of Delays Between Agent Actions | Analysis of packages/orchestrator/src/ (95 files) — identified delay sources ranked by impact. |
 | `guide.operations-delay-fix-review` | Delay Fix Review — Commits Since Delay Analysis | Date: 2026-06-10 Reference: /docs/operations/delay-analysis.md (15 delay sources documented) |