ouroboros-ai 0.40.2.dev13__tar.gz → 0.40.2.dev15__tar.gz

{ouroboros_ai-0.40.2.dev13 → ouroboros_ai-0.40.2.dev15}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ouroboros-ai
-Version: 0.40.2.dev13
+Version: 0.40.2.dev15
 Summary: Specification-first workflow engine for AI coding agents. Works with Claude Code and Codex CLI.
 Author-email: Q00 <jqyu.lee@gmail.com>
 License-File: LICENSE

{ouroboros_ai-0.40.2.dev13 → ouroboros_ai-0.40.2.dev15}/docs/agentos/traceguard-vs-legacy-benchmark.md

@@ -74,9 +74,19 @@ Fat-harness baseline report — profile=traceguard_plus_claim_term_guard_fixture
 - Claim-term guard semantic-miss incidents per 100 ACs: -12.5000
 - Claim-term guard median chars ratio: 1.4054
 
+## H1 retry admission
+
+| Fixture | Accepted | Failure class | Retry admission | Evidence refs |
+| --- | --- | --- | --- | ---: |
+| fixture:h1/traceguard/accepted | true |  | ACCEPT | 1 |
+| fixture:h1/traceguard/missing-evidence | false | EVIDENCE_MISSING | RETRY | 0 |
+| fixture:h1/claim-term/semantic-miss | false | SCOPE_CREEP | REDISPATCH | 1 |
+| fixture:h1/traceguard/repeated-fabrication | false | FABRICATION_SUSPECTED | ESCALATE_MODEL | 0 |
+
 ## Gate interpretation
 
 - TraceGuard reduces fixture fabrication incidents to 0 per 100 ACs.
 - The deterministic claim-term guard rejects the fixture semantic miss without reintroducing fabrication.
 - One-shot pass rate drops because unsupported legacy self-reports are rejected instead of counted as accepted.
 - Median chars stay within the <= 1.5x C.4 budget guardrail.
+- H1 admission is typed for accepted, retryable, redispatch, and model-escalation verdicts.

ouroboros_ai-0.40.2.dev15/docs/rfc/verdict-envelope-v1.md

@@ -0,0 +1,96 @@
+# Verdict Envelope v1
+
+Issue: #814
+
+Status: schema-first design RFC. This document fixes the multi-persona verdict
+output shape before subsystem adoption.
+
+## Problem
+
+Ouroboros already has a dispatch-side `_subagent` / `_subagents` envelope, but
+the result side is fragmented:
+
+- `ConsensusResult` in `src/ouroboros/evaluation/models.py` stores votes,
+  approval, majority ratio, and disagreements.
+- Stage 3 events in `src/ouroboros/events/evaluation.py` emit `approved`,
+  `votes`, `majority_ratio`, `disagreements`, and count fields.
+- MCP-facing tools in `src/ouroboros/mcp/tools/definitions.py` return
+  tool-specific text/meta shapes.
+
+That fragmentation makes callers infer verdict semantics from prose or
+subsystem-specific fields. The output contract should instead expose one typed
+envelope.
+
+## Schema
+
+The canonical envelope is JSON-serializable and may be represented by Pydantic
+or dataclass models in implementation PRs.
+
+```json
+{
+  "schema_version": "verdict_envelope.v1",
+  "verdict": "string or null",
+  "status": "PASS | FAIL | BLOCKED | DEFERRED",
+  "members": ["hacker", "architect"],
+  "dissent": [
+    {
+      "persona_a": "hacker",
+      "persona_b": "architect",
+      "topic": "scope risk",
+      "summary": "short disagreement summary"
+    }
+  ],
+  "evidence_used": ["evt_123", "artifact_456"],
+  "transcript_ref": "evt_transcript_789",
+  "follow_up_issue_refs": ["#1306"],
+  "metadata": {
+    "source": "evaluation.stage3"
+  }
+}
+```
+
+Field rules:
+
+- `schema_version` is required and must be `verdict_envelope.v1`.
+- `verdict` is a one-line synthesized conclusion, or `null` when synthesis is
+  deliberately deferred to the user.
+- `status` is required. `DEFERRED` is the default for user-owned decisions such
+  as lateral multi-persona debate.
+- `members` is the ordered set of participating personas/models/roles.
+- `dissent` is optional and contains pairwise or topic-level disagreement.
+- `evidence_used` contains stable event IDs, artifact IDs, fact IDs, or handles
+  that substantiate the verdict.
+- `transcript_ref` is optional until #819 provides durable transcript storage.
+- `follow_up_issue_refs` lists implementation issues opened from this design.
+- `metadata` is for subsystem-specific values that must not define verdict
+  semantics.
+
+## Migration Map
+
+| Existing shape | Envelope mapping | Gap |
+| --- | --- | --- |
+| `ConsensusResult.approved` | `status=PASS` when true, `status=FAIL` when false | No explicit deferred state. |
+| `ConsensusResult.disagreements` | `dissent[].summary` | Lacks structured persona pairs/topics. |
+| `ConsensusResult.votes[].model` | `members[]` | Models are not always personas. |
+| Stage 3 `approved` event field | `status` | Event has no `schema_version`. |
+| Stage 3 `votes` event field | `members` plus `metadata.votes` | Vote details remain subsystem metadata. |
+| Stage 3 `disagreements` event field | `dissent` | Needs structured dissent records. |
+| MCP tool text outputs | `verdict` plus `metadata.rendered_text` | Text must stop being the semantic source of truth. |
+| MCP tool meta outputs | `metadata` | Existing meta can be preserved under namespaced keys. |
+
+## Follow-up Implementation Issues
+
+- #814 follow-up A: add a shared `VerdictEnvelopeV1` model and JSON schema.
+- #814 follow-up B: adapt `ConsensusResult` / deliberative evaluation outputs
+  to expose `VerdictEnvelopeV1`.
+- #814 follow-up C: emit `verdict_envelope` from Stage 3 evaluation events.
+- #814 follow-up D: add MCP tool result metadata for `verdict_envelope` while
+  preserving current rendered text for compatibility.
+- #1306: use typed verifier output and retry admission for TraceGuard-backed H1
+  deliver verdicts.
+
+## Non-goals
+
+- Do not require every user-facing response to auto-synthesize a verdict.
+- Do not replace subsystem-specific detailed result models.
+- Do not make transcript storage mandatory before #819.

{ouroboros_ai-0.40.2.dev13 → ouroboros_ai-0.40.2.dev15}/src/ouroboros/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '0.40.2.dev13'
-__version_tuple__ = version_tuple = (0, 40, 2, 'dev13')
+__version__ = version = '0.40.2.dev15'
+__version_tuple__ = version_tuple = (0, 40, 2, 'dev15')
 
 __commit_id__ = commit_id = None

{ouroboros_ai-0.40.2.dev13 → ouroboros_ai-0.40.2.dev15}/src/ouroboros/backends/capabilities.py

@@ -242,8 +242,8 @@ _CAPABILITIES: tuple[BackendCapability, ...] = (
         name="pi",
         aliases=("pi_cli",),
         supports_runtime=True,
-        supports_llm=False,
-        supports_interview_driver=False,
+        supports_llm=True,
+        supports_interview_driver=True,
         cli_name="pi",
         cli_config_key="pi_cli_path",
         supports_tool_envelope=False,

{ouroboros_ai-0.40.2.dev13 → ouroboros_ai-0.40.2.dev15}/src/ouroboros/config/loader.py

@@ -68,11 +68,13 @@ _CODEX_LLM_BACKENDS = frozenset({"codex", "codex_cli", "opencode", "opencode_cli
 _KIRO_LLM_BACKENDS = frozenset({"kiro", "kiro_cli"})
 _COPILOT_LLM_BACKENDS = frozenset({"copilot", "copilot_cli"})
 _HERMES_LLM_BACKENDS = frozenset({"hermes", "hermes_cli"})
+_PI_LLM_BACKENDS = frozenset({"pi", "pi_cli"})
 _OPENCODE_BACKENDS = frozenset({"opencode", "opencode_cli"})
 _CODEX_DEFAULT_MODEL = "default"
 _KIRO_DEFAULT_MODEL = "default"
 _COPILOT_DEFAULT_MODEL = "default"
 _HERMES_DEFAULT_MODEL = "default"
+_PI_DEFAULT_MODEL = "default"
 _PLACEHOLDER_API_KEY_PREFIX = "YOUR_"
 _PLACEHOLDER_API_KEY_SUFFIX = "_API_KEY"
 _DEFAULT_MAX_PARALLEL_WORKERS = 3
@@ -1253,6 +1255,8 @@ def _default_model_for_backend(
         return _COPILOT_DEFAULT_MODEL
     if resolved in _HERMES_LLM_BACKENDS:
         return _HERMES_DEFAULT_MODEL
+    if resolved in _PI_LLM_BACKENDS:
+        return _PI_DEFAULT_MODEL
     return default_model
 
 
@@ -1291,6 +1295,8 @@ def _normalize_configured_model_for_backend(
         return _COPILOT_DEFAULT_MODEL
     if resolved in _HERMES_LLM_BACKENDS and is_shipped_default:
         return _HERMES_DEFAULT_MODEL
+    if resolved in _PI_LLM_BACKENDS and candidate == default_model:
+        return _PI_DEFAULT_MODEL
 
     return candidate
 

{ouroboros_ai-0.40.2.dev13 → ouroboros_ai-0.40.2.dev15}/src/ouroboros/config/models.py

@@ -133,6 +133,7 @@ class LLMConfig(BaseModel, frozen=True):
         "kiro",
         "hermes",
         "goose",
+        "pi",
     ] = "claude_code"
     permission_mode: Literal["default", "acceptEdits", "bypassPermissions"] = "default"
     opencode_permission_mode: Literal["default", "acceptEdits", "bypassPermissions"] = "acceptEdits"

{ouroboros_ai-0.40.2.dev13 → ouroboros_ai-0.40.2.dev15}/src/ouroboros/harness/deliver_routing.py

@@ -12,6 +12,7 @@ from dataclasses import dataclass
 
 from ouroboros.harness.deliver_gate import DeliverGateVerdict
 from ouroboros.orchestrator.failure_taxonomy import RecoveryAction
+from ouroboros.orchestrator.verifier import RetryAdmission, VerifierStatus, VerifierVerdict
 
 
 @dataclass(frozen=True, slots=True)
@@ -133,11 +134,89 @@ def route_deliver_gate_verdict(
     )
 
 
+def deliver_gate_verifier_verdict(
+    verdict: DeliverGateVerdict,
+    *,
+    rejection_count: int = 1,
+    model_escalation_threshold: int = 2,
+) -> VerifierVerdict:
+    """Promote a TraceGuard deliver verdict into the H1 verifier contract.
+
+    This is the #1306 behavior-change bridge: callers can pass the returned
+    ``VerifierVerdict`` through the existing H1 acceptance boundary instead of
+    treating ``DeliverGateVerdict`` as read-only observation.
+    """
+    route = route_deliver_gate_verdict(
+        verdict,
+        rejection_count=rejection_count,
+        model_escalation_threshold=model_escalation_threshold,
+    )
+    if route.accepted:
+        return VerifierVerdict(
+            passed=True,
+            status=VerifierStatus.PASS,
+            evidence_used=verdict.evidence_event_ids,
+            retry_admission=RetryAdmission.ACCEPT,
+        )
+
+    failure_class = _failure_class_for_route(route)
+    status = VerifierStatus.BLOCKED if failure_class == "BLOCKED" else VerifierStatus.FAIL
+    return VerifierVerdict(
+        passed=False,
+        reasons=(
+            f"{route.reason}: " + "; ".join(route.source_reasons)
+            if route.source_reasons
+            else route.reason,
+        ),
+        failure_class=failure_class,
+        status=status,
+        evidence_used=verdict.evidence_event_ids,
+        retry_admission=_retry_admission_for_route(route),
+    )
+
+
+def _retry_admission_for_route(route: DeliverGateRoute) -> RetryAdmission:
+    if route.action is RecoveryAction.RETRY:
+        return RetryAdmission.RETRY
+    if route.action is RecoveryAction.REDISPATCH:
+        return RetryAdmission.REDISPATCH
+    if route.action is RecoveryAction.ESCALATE_MODEL:
+        return RetryAdmission.ESCALATE_MODEL
+    if route.action is RecoveryAction.ESCALATE_HUMAN:
+        return RetryAdmission.ESCALATE_HUMAN
+    return RetryAdmission.BLOCK
+
+
+def _failure_class_for_route(route: DeliverGateRoute) -> str:
+    reason_codes = tuple(_reason_code(reason) for reason in route.source_reasons)
+    if route.action is RecoveryAction.ESCALATE_HUMAN:
+        return "BLOCKED"
+    if any(reason in _RETRY_REASONS for reason in reason_codes):
+        return "EVIDENCE_MISSING"
+    if any(reason == "semantic_miss" for reason in reason_codes):
+        return "SCOPE_CREEP"
+    if any(
+        reason in {"evidence_handle_mismatch", "chunk_handle_without_fact"}
+        for reason in reason_codes
+    ):
+        return "EVIDENCE_FORM_MISMATCH"
+    if any(
+        reason in {"unsupported_fact_id", "malformed_evidence_claim"} for reason in reason_codes
+    ):
+        return "FABRICATION_SUSPECTED"
+    if route.action is RecoveryAction.ESCALATE_MODEL:
+        return "FABRICATION_SUSPECTED"
+    if route.action is RecoveryAction.REDISPATCH:
+        return "STALL"
+    return "BLOCKED"
+
+
 def _reason_code(reason: str) -> str:
     return reason.split(":", maxsplit=1)[0].strip()
 
 
 __all__ = [
     "DeliverGateRoute",
+    "deliver_gate_verifier_verdict",
     "route_deliver_gate_verdict",
 ]

{ouroboros_ai-0.40.2.dev13 → ouroboros_ai-0.40.2.dev15}/src/ouroboros/orchestrator/parallel_executor.py

@@ -5634,6 +5634,17 @@ Files present:
                     verifier_failure_class=(
                         verifier_verdict.failure_class if verifier_verdict is not None else None
                     ),
+                    verifier_status=(
+                        verifier_verdict.status.value if verifier_verdict is not None else None
+                    ),
+                    retry_admission=(
+                        verifier_verdict.retry_admission.value
+                        if verifier_verdict is not None
+                        else None
+                    ),
+                    verifier_evidence_used=(
+                        list(verifier_verdict.evidence_used) if verifier_verdict is not None else []
+                    ),
                 )
                 result_final_message = (
                     f"{fat_harness_error}\n\nRuntime final message:\n{final_message}"
@@ -6036,6 +6047,9 @@ Files present:
         if verifier_verdict is not None:
             data["verifier_reasons"] = list(verifier_verdict.reasons)
             data["verifier_failure_class"] = verifier_verdict.failure_class
+            data["verifier_status"] = verifier_verdict.status.value
+            data["retry_admission"] = verifier_verdict.retry_admission.value
+            data["verifier_evidence_used"] = list(verifier_verdict.evidence_used)
         if typed_evidence is not None:
             data["typed_evidence_fields"] = sorted(typed_evidence.data)
             data["ignored_out_of_scope_evidence_fields"] = list(

{ouroboros_ai-0.40.2.dev13 → ouroboros_ai-0.40.2.dev15}/src/ouroboros/orchestrator/traceguard_benchmark_capture.py

@@ -10,6 +10,8 @@ from __future__ import annotations
 from dataclasses import dataclass
 from typing import Any
 
+from ouroboros.harness.deliver_gate import DeliverGateVerdict
+from ouroboros.harness.deliver_routing import deliver_gate_verifier_verdict
 from ouroboros.orchestrator.baseline_metrics import (
     DEFAULT_MAX_RETRIES,
     FatHarnessMetricsReport,
@@ -28,6 +30,26 @@ BENCHMARK_PROFILE_TRACEGUARD = "traceguard_deliver_gate_fixture"
 BENCHMARK_PROFILE_TRACEGUARD_CLAIM_TERM_GUARD = "traceguard_plus_claim_term_guard_fixture"
 
 
+@dataclass(frozen=True)
+class H1RetryAdmissionFixtureRow:
+    """Recorded H1 admission outcome for a deliver-gate verdict fixture."""
+
+    fixture: str
+    accepted: bool
+    failure_class: str | None
+    retry_admission: str
+    evidence_used_count: int
+
+    def to_dict(self) -> dict[str, Any]:
+        return {
+            "fixture": self.fixture,
+            "accepted": self.accepted,
+            "failure_class": self.failure_class,
+            "retry_admission": self.retry_admission,
+            "evidence_used_count": self.evidence_used_count,
+        }
+
+
 LEGACY_SELF_REPORT_ROWS: tuple[BaselineMetricFixtureRow, ...] = (
     BaselineMetricFixtureRow(
         "LEG-AC-001",
@@ -118,6 +140,7 @@ class TraceGuardBenchmarkCapture:
     legacy_rows: tuple[BaselineMetricFixtureRow, ...]
     traceguard_rows: tuple[BaselineMetricFixtureRow, ...]
     claim_term_guard_rows: tuple[BaselineMetricFixtureRow, ...]
+    h1_retry_admission_rows: tuple[H1RetryAdmissionFixtureRow, ...]
 
     def to_dict(self) -> dict[str, Any]:
         return {
@@ -133,6 +156,14 @@ class TraceGuardBenchmarkCapture:
                 "report": self.claim_term_guard_report.to_dict(),
                 "rows": [row.to_dict() for row in self.claim_term_guard_rows],
             },
+            "h1_retry_admission": {
+                "rows": [row.to_dict() for row in self.h1_retry_admission_rows],
+                "all_typed": all(row.retry_admission for row in self.h1_retry_admission_rows),
+                "accepted_count": sum(1 for row in self.h1_retry_admission_rows if row.accepted),
+                "routed_count": sum(
+                    1 for row in self.h1_retry_admission_rows if row.retry_admission != "ACCEPT"
+                ),
+            },
             "delta": {
                 "fabrication_incidents_per_100_acs": (
                     self.traceguard_report.fabrication_incidents_per_100_acs
@@ -182,6 +213,7 @@ def build_traceguard_benchmark_capture() -> TraceGuardBenchmarkCapture:
         legacy_rows=LEGACY_SELF_REPORT_ROWS,
         traceguard_rows=traceguard_rows,
         claim_term_guard_rows=claim_term_guard_rows,
+        h1_retry_admission_rows=_h1_retry_admission_rows(),
     )
 
 
@@ -212,12 +244,66 @@ def _claim_term_guard_rows(
     return tuple(guarded)
 
 
+def _h1_retry_admission_rows() -> tuple[H1RetryAdmissionFixtureRow, ...]:
+    fixtures = {
+        "fixture:h1/traceguard/accepted": DeliverGateVerdict(
+            ac_id="AC-1",
+            accepted=True,
+            unsupported_claim_rate=0.0,
+            accepted_fact_ids=("fact_1",),
+            evidence_event_ids=("evt_accept",),
+        ),
+        "fixture:h1/traceguard/missing-evidence": DeliverGateVerdict(
+            ac_id="AC-2",
+            accepted=False,
+            unsupported_claim_rate=1.0,
+            rejected_fact_ids=("fact_missing",),
+            rejected_reasons=("missing_evidence_handle: ev_missing is not present in manifest",),
+        ),
+        "fixture:h1/claim-term/semantic-miss": DeliverGateVerdict(
+            ac_id="AC-3",
+            accepted=False,
+            unsupported_claim_rate=1.0,
+            rejected_fact_ids=("fact_semantic",),
+            rejected_reasons=("semantic_miss: evidence text lacks behavior=admin_delete_denied",),
+            evidence_event_ids=("evt_semantic",),
+        ),
+        "fixture:h1/traceguard/repeated-fabrication": DeliverGateVerdict(
+            ac_id="AC-4",
+            accepted=False,
+            unsupported_claim_rate=1.0,
+            rejected_fact_ids=("fact_fake",),
+            rejected_reasons=("unsupported_fact_id: fact_fake is not present",),
+        ),
+    }
+    rows: list[H1RetryAdmissionFixtureRow] = []
+    for fixture, verdict in fixtures.items():
+        rejection_count = 2 if fixture.endswith("repeated-fabrication") else 1
+        verifier_verdict = deliver_gate_verifier_verdict(
+            verdict,
+            rejection_count=rejection_count,
+            model_escalation_threshold=2,
+        )
+        rows.append(
+            H1RetryAdmissionFixtureRow(
+                fixture=fixture,
+                accepted=verifier_verdict.passed,
+                failure_class=verifier_verdict.failure_class,
+                retry_admission=verifier_verdict.retry_admission.value,
+                evidence_used_count=len(verifier_verdict.evidence_used),
+            )
+        )
+    return tuple(rows)
+
+
 def render_traceguard_benchmark_markdown(
     capture: TraceGuardBenchmarkCapture | None = None,
 ) -> str:
     """Render the benchmark artifact for maintainer review."""
     capture = build_traceguard_benchmark_capture() if capture is None else capture
-    data = capture.to_dict()["delta"]
+    capture_data = capture.to_dict()
+    data = capture_data["delta"]
+    h1_rows = capture_data["h1_retry_admission"]["rows"]
     lines = [
         "# #978 P4 TraceGuard vs legacy baseline benchmark",
         "",
@@ -251,12 +337,29 @@ def render_traceguard_benchmark_markdown(
         f"{data['claim_term_guard_semantic_miss_incidents_per_100_acs']:.4f}",
         f"- Claim-term guard median chars ratio: {data['claim_term_guard_median_chars_ratio']:.4f}",
         "",
+        "## H1 retry admission",
+        "",
+        "| Fixture | Accepted | Failure class | Retry admission | Evidence refs |",
+        "| --- | --- | --- | --- | ---: |",
+        *(
+            "| {fixture} | {accepted} | {failure_class} | {retry_admission} | "
+            "{evidence_used_count} |".format(
+                fixture=row["fixture"],
+                accepted=str(row["accepted"]).lower(),
+                failure_class=row["failure_class"] or "",
+                retry_admission=row["retry_admission"],
+                evidence_used_count=row["evidence_used_count"],
+            )
+            for row in h1_rows
+        ),
+        "",
         "## Gate interpretation",
         "",
         "- TraceGuard reduces fixture fabrication incidents to 0 per 100 ACs.",
         "- The deterministic claim-term guard rejects the fixture semantic miss without reintroducing fabrication.",
         "- One-shot pass rate drops because unsupported legacy self-reports are rejected instead of counted as accepted.",
         "- Median chars stay within the <= 1.5x C.4 budget guardrail.",
+        "- H1 admission is typed for accepted, retryable, redispatch, and model-escalation verdicts.",
     ]
     return "\n".join(lines) + "\n"
 
@@ -274,6 +377,7 @@ __all__ = [
     "BENCHMARK_PROFILE_TRACEGUARD",
     "BENCHMARK_PROFILE_TRACEGUARD_CLAIM_TERM_GUARD",
     "LEGACY_SELF_REPORT_ROWS",
+    "H1RetryAdmissionFixtureRow",
     "TraceGuardBenchmarkCapture",
     "build_traceguard_benchmark_capture",
     "render_traceguard_benchmark_markdown",