debrief-mcp 0.1.0__tar.gz

debrief_mcp-0.1.0/PKG-INFO

@@ -0,0 +1,84 @@
+Metadata-Version: 2.3
+Name: debrief-mcp
+Version: 0.1.0
+Summary: Evidence MCP for agents: what ggufbench measured, where a buildwright run broke, role proposals as text — never writes
+Author: Jordi Marsal
+Author-email: Jordi Marsal <jordi.marsal@gmail.com>
+Requires-Dist: mcp>=2.3,<3
+Requires-Python: >=3.14
+Description-Content-Type: text/markdown
+
+# debrief
+
+The evidence MCP: answers agent questions from receipts — what
+[ggufbench](https://github.com/jordimarsal/ggufbench) measured, and where the
+last [buildwright](https://github.com/jordimarsal/buildwright) run broke — and
+proposes role changes **as text**. It never writes.
+
+> Status: scaffolding. The harness is installed and the parent feature is
+> `pending` — the spec (`harness/specs/debrief/`) is not written yet, so the
+> tool list below is the plan, not the product.
+
+## The idea
+
+- `ggufbench` owns the **catalog**: what a model measured, under which task
+  signature, and why rows were excluded.
+- `buildwright` owns the **episode**: one run, its gates, its failures.
+- `debrief` is the reader. One query surface, one shared finding type, and a
+  proposal that cites both. Two ports that answer different questions:
+
+| Port | Question | Reads | Tools (planned) |
+| --- | --- | --- | --- |
+| `bench` | What did we measure? | ggufbench artifacts | `bench.which_model` · `bench.compare` · `bench.why_excluded` |
+| `run` | Where did this run break? | a buildwright run log | `run.inspect_run` · `run.failures` · `run.tool_calls` |
+| `routing.propose` | What would you change? | findings from both | prints a `BW_ROLE_*` diff as text |
+
+## Product rules (the gates that matter in code)
+
+1. **debrief never writes.** Every tool is read-only or returns a proposal as
+   text. No tool touches `.env`, `BW_ROLE_*`, `models.yaml`, or any other file.
+2. **Proposals target roles, not the registry.** A proposed change is a
+   `BW_ROLE_<ROLE>` diff. `models.yaml` is the registry of what exists — not
+   debrief's to edit.
+3. **bench and run answer from their own source.** `run` may cite `bench`
+   findings; `bench` never reads runs.
+4. **Rows never cross a task signature.** Incomparable rows are refused with a
+   reason — never averaged.
+
+## The gates (process)
+
+The repo runs the [harness](https://github.com/jordimarsal/harness) SDD
+workflow in **hybrid** mode: one in-session agent plays the roles, the human
+gates stay mandatory. Architecture is **hexagonal** (`docs/architecture.md`);
+features are cut into **vertical slices** (`docs/slicing-protocol.md`) —
+S1 `bench_query` → S2 `run_inspect` → S3 `role_propose`, each demoable
+end-to-end.
+
+| Gate | What it checks | Command / artefact |
+| --- | --- | --- |
+| **Spec Gate** (human) | EARS trio in `harness/specs/<name>/`, slice map approved before any slice starts | review + explicit approval |
+| **Batch gates** (every 2–4 tasks) | tests + lint + types, evidence saved | `uv run pytest tests`, `ruff check`, `mypy --strict` — full output to `harness/logs/<feature>/batch-<n>.log` |
+| **Security audit** (`audit_level: standard`) | path traversal, input validation, secrets — run at review time | `bash harness/tools/audit-security.sh` |
+| **Gauntlet G1–G4** (before Completion) | correctness, adversarial input, maintainability, evidence | `bash harness/tools/gauntlet.sh --feature <name>` + report in `harness/progress/gauntlet_<name>.md` |
+| **Completion Gate** (human) | traceability `R<n>` ↔ test, plus a **recorded end-to-end demo** per slice | `python3 harness/tools/check-traceability.py --feature <name>` → `harness/specs/<name>/review.md` |
+| **Session verify** | harness files, docs, tests, state coherence | `harness/init.sh` |
+
+Two gates are non-negotiable and human: the **Spec Gate** and the **Completion
+Gate**. The gauntlet filters what reaches them; it never replaces them.
+
+## Development
+
+```bash
+uv sync
+uv run pytest tests
+```
+
+Requires Python ≥ 3.14 ([uv](https://docs.astral.sh/uv/)).
+
+## Repository map
+
+See [`AGENTS.md`](AGENTS.md) — it is the navigation map for agents (and humans).
+Architecture: [`docs/architecture.md`](docs/architecture.md). Decisions worth
+remembering: [`harness/decisions/`](harness/decisions/).
+
+[![built with harness](https://raw.githubusercontent.com/jordimarsal/harness/main/assets/badge.svg)](https://github.com/jordimarsal/harness)

debrief_mcp-0.1.0/README.md

@@ -0,0 +1,74 @@
+# debrief
+
+The evidence MCP: answers agent questions from receipts — what
+[ggufbench](https://github.com/jordimarsal/ggufbench) measured, and where the
+last [buildwright](https://github.com/jordimarsal/buildwright) run broke — and
+proposes role changes **as text**. It never writes.
+
+> Status: scaffolding. The harness is installed and the parent feature is
+> `pending` — the spec (`harness/specs/debrief/`) is not written yet, so the
+> tool list below is the plan, not the product.
+
+## The idea
+
+- `ggufbench` owns the **catalog**: what a model measured, under which task
+  signature, and why rows were excluded.
+- `buildwright` owns the **episode**: one run, its gates, its failures.
+- `debrief` is the reader. One query surface, one shared finding type, and a
+  proposal that cites both. Two ports that answer different questions:
+
+| Port | Question | Reads | Tools (planned) |
+| --- | --- | --- | --- |
+| `bench` | What did we measure? | ggufbench artifacts | `bench.which_model` · `bench.compare` · `bench.why_excluded` |
+| `run` | Where did this run break? | a buildwright run log | `run.inspect_run` · `run.failures` · `run.tool_calls` |
+| `routing.propose` | What would you change? | findings from both | prints a `BW_ROLE_*` diff as text |
+
+## Product rules (the gates that matter in code)
+
+1. **debrief never writes.** Every tool is read-only or returns a proposal as
+   text. No tool touches `.env`, `BW_ROLE_*`, `models.yaml`, or any other file.
+2. **Proposals target roles, not the registry.** A proposed change is a
+   `BW_ROLE_<ROLE>` diff. `models.yaml` is the registry of what exists — not
+   debrief's to edit.
+3. **bench and run answer from their own source.** `run` may cite `bench`
+   findings; `bench` never reads runs.
+4. **Rows never cross a task signature.** Incomparable rows are refused with a
+   reason — never averaged.
+
+## The gates (process)
+
+The repo runs the [harness](https://github.com/jordimarsal/harness) SDD
+workflow in **hybrid** mode: one in-session agent plays the roles, the human
+gates stay mandatory. Architecture is **hexagonal** (`docs/architecture.md`);
+features are cut into **vertical slices** (`docs/slicing-protocol.md`) —
+S1 `bench_query` → S2 `run_inspect` → S3 `role_propose`, each demoable
+end-to-end.
+
+| Gate | What it checks | Command / artefact |
+| --- | --- | --- |
+| **Spec Gate** (human) | EARS trio in `harness/specs/<name>/`, slice map approved before any slice starts | review + explicit approval |
+| **Batch gates** (every 2–4 tasks) | tests + lint + types, evidence saved | `uv run pytest tests`, `ruff check`, `mypy --strict` — full output to `harness/logs/<feature>/batch-<n>.log` |
+| **Security audit** (`audit_level: standard`) | path traversal, input validation, secrets — run at review time | `bash harness/tools/audit-security.sh` |
+| **Gauntlet G1–G4** (before Completion) | correctness, adversarial input, maintainability, evidence | `bash harness/tools/gauntlet.sh --feature <name>` + report in `harness/progress/gauntlet_<name>.md` |
+| **Completion Gate** (human) | traceability `R<n>` ↔ test, plus a **recorded end-to-end demo** per slice | `python3 harness/tools/check-traceability.py --feature <name>` → `harness/specs/<name>/review.md` |
+| **Session verify** | harness files, docs, tests, state coherence | `harness/init.sh` |
+
+Two gates are non-negotiable and human: the **Spec Gate** and the **Completion
+Gate**. The gauntlet filters what reaches them; it never replaces them.
+
+## Development
+
+```bash
+uv sync
+uv run pytest tests
+```
+
+Requires Python ≥ 3.14 ([uv](https://docs.astral.sh/uv/)).
+
+## Repository map
+
+See [`AGENTS.md`](AGENTS.md) — it is the navigation map for agents (and humans).
+Architecture: [`docs/architecture.md`](docs/architecture.md). Decisions worth
+remembering: [`harness/decisions/`](harness/decisions/).
+
+[![built with harness](https://raw.githubusercontent.com/jordimarsal/harness/main/assets/badge.svg)](https://github.com/jordimarsal/harness)

debrief_mcp-0.1.0/pyproject.toml

@@ -0,0 +1,52 @@
+[project]
+name = "debrief-mcp"
+version = "0.1.0"
+description = "Evidence MCP for agents: what ggufbench measured, where a buildwright run broke, role proposals as text — never writes"
+readme = "README.md"
+requires-python = ">=3.14"
+dependencies = ["mcp>=2.3,<3"]
+
+[[project.authors]]
+name = "Jordi Marsal"
+email = "jordi.marsal@gmail.com"
+
+[project.scripts]
+debrief = "debrief:main"
+debrief-mcp = "debrief:main"
+
+[build-system]
+requires = ["uv_build>=0.12.19,<0.13.0"]
+build-backend = "uv_build"
+
+[dependency-groups]
+dev = [
+    "mypy>=1.15",
+    "pytest>=9.2.0",
+    "ruff>=0.14",
+]
+
+[tool.ruff]
+line-length = 100
+target-version = "py314"
+extend-exclude = ["harness/tools"]
+
+[tool.ruff.lint]
+select = [
+    "E",
+    "F",
+    "W",
+    "I",
+    "UP",
+    "B",
+]
+
+[tool.mypy]
+strict = true
+files = [
+    "src",
+    "tests",
+]
+
+[tool.uv.build-backend]
+module-name = "debrief"
+module-root = "src"

debrief_mcp-0.1.0/pyproject.toml.orig

@@ -0,0 +1,47 @@
+[project]
+name = "debrief-mcp"
+version = "0.1.0"
+description = "Evidence MCP for agents: what ggufbench measured, where a buildwright run broke, role proposals as text — never writes"
+readme = "README.md"
+authors = [
+    { name = "Jordi Marsal", email = "jordi.marsal@gmail.com" }
+]
+requires-python = ">=3.14"
+dependencies = [
+    "mcp>=2.3,<3",
+]
+
+[project.scripts]
+debrief = "debrief:main"
+debrief-mcp = "debrief:main"
+
+[build-system]
+requires = ["uv_build>=0.12.19,<0.13.0"]
+build-backend = "uv_build"
+
+[dependency-groups]
+dev = [
+    "mypy>=1.15",
+    "pytest>=9.2.0",
+    "ruff>=0.14",
+]
+
+[tool.ruff]
+line-length = 100
+target-version = "py314"
+# harness/tools are harness-owned scripts executed by the system python3 (<3.14),
+# so py314-only syntax ruff formats towards must never be applied to them.
+extend-exclude = ["harness/tools"]
+
+[tool.ruff.lint]
+select = ["E", "F", "W", "I", "UP", "B"]
+
+[tool.mypy]
+strict = true
+files = ["src", "tests"]
+
+[tool.uv.build-backend]
+# Distribution is `debrief-mcp` (PyPI name), but the importable package stays
+# `debrief` — module name does not have to match the distribution name.
+module-name = "debrief"
+module-root = "src"

debrief_mcp-0.1.0/src/debrief/__init__.py

@@ -0,0 +1,16 @@
+"""debrief — answers agent questions from receipts; never writes (ADR-0001)."""
+
+from __future__ import annotations
+
+from typing import Any
+
+__all__ = ["main"]
+
+
+def __getattr__(name: str) -> Any:
+    # Lazy on purpose: `import debrief` (and so debrief.domain) must not pull in the mcp SDK.
+    if name == "main":
+        from debrief.mcp_server import main
+
+        return main
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

debrief_mcp-0.1.0/src/debrief/adapters/buildwright_log.py

@@ -0,0 +1,213 @@
+"""buildwright run log adapter: read-only mapping of one run log to Findings (R4).
+
+Format: ``key: value`` header lines (``run:``, ``engine_build:``, ``task:``,
+``roles:``, ``result:``) followed by one ``step:`` line per logged step. The
+adapter is the only place that knows this format (ADR-0002); a run log never
+yields a task signature — it is one episode, not a measurement.
+"""
+
+from __future__ import annotations
+
+import logging
+import re
+import stat
+from dataclasses import dataclass
+from pathlib import Path
+
+from debrief.domain import FAIL, OUTCOMES, PASS, Finding
+from debrief.errors import RUN_SOURCE_MISSING, SOURCE_UNPARSEABLE, DebriefError
+
+log = logging.getLogger(__name__)
+
+# A named tuple, not an inline `except (A, B):`: ruff format (target py314) rewrites
+# the inline spelling towards the PEP 758 bare-comma form, which this repo's review
+# rejected; the named tuple keeps one spelling that is valid on every Python 3.
+_MISSING_PATH_ERRORS = (FileNotFoundError, NotADirectoryError)
+
+_STEP_RE = re.compile(
+    r"^step:\s*(?P<ordinal>\d+)\s+"
+    r"role=(?P<role>\S+)\s+"
+    r"model=(?P<model>\S+)\s+"
+    r"outcome=(?P<outcome>\S+)\s+"
+    r'reason="(?P<reason>[^"]*)"\s+'
+    r"evidence=(?P<evidence>\S+)$"
+)
+
+_RESULT_RE = re.compile(r"^(?P<outcome>\S+)(?:\s+failed_steps=(?P<failed_steps>\S*))?$")
+
+
+class _Unparseable(Exception):
+    """A run log did not match the buildwright log format."""
+
+
+@dataclass(frozen=True)
+class _Step:
+    """One logged step before it becomes a Finding; the ordinal exists only
+    here because the ``result:`` header names failed steps by it, and a
+    Finding never carries run-internal bookkeeping (ADR-0002)."""
+
+    ordinal: str
+    role: str
+    model: str
+    outcome: str
+    reason: str
+    evidence: str
+
+
+class BuildwrightLog:
+    """RunLog over one buildwright run log; opens the file for reading only (R2, R4)."""
+
+    def __init__(self, log_file: Path) -> None:
+        self._log_file = log_file
+
+    def findings(self) -> list[Finding] | DebriefError:
+        # stat()-based probe, mirroring the bench adapter: pathlib's is_file()
+        # answers False for a log whose parent directory is unreadable, which
+        # would mislabel an existing-but-unreadable log as missing. An evidence
+        # tool never silently drops a source it was asked to read.
+        try:
+            st = self._log_file.stat()
+        except _MISSING_PATH_ERRORS:
+            return DebriefError(
+                code=RUN_SOURCE_MISSING,
+                message="no buildwright run log at the requested path",
+                detail=str(self._log_file),
+            )
+        except PermissionError as err:
+            return DebriefError(
+                code=SOURCE_UNPARSEABLE,
+                message=f"{self._log_file} exists but cannot be probed",
+                detail=str(err),
+            )
+        if not stat.S_ISREG(st.st_mode):
+            return DebriefError(
+                code=RUN_SOURCE_MISSING,
+                message="no buildwright run log at the requested path",
+                detail=str(self._log_file),
+            )
+        try:
+            lines = self._log_file.read_text(encoding="utf-8").splitlines()
+        except (OSError, ValueError) as err:
+            # ValueError covers undecodable bytes (UnicodeDecodeError), not just I/O errors.
+            return DebriefError(
+                code=SOURCE_UNPARSEABLE,
+                message=f"{self._log_file} could not be read",
+                detail=str(err),
+            )
+        try:
+            findings = _map_log(lines)
+        except _Unparseable as err:
+            return DebriefError(
+                code=SOURCE_UNPARSEABLE,
+                message=f"{self._log_file} is not a buildwright run log",
+                detail=str(err),
+            )
+        log.info("read %d step(s) from %s", len(findings), self._log_file.name)
+        return findings
+
+
+def _map_log(lines: list[str]) -> list[Finding]:
+    task = _header(lines, "task")
+    steps = [_parse_step(line) for line in lines if line.startswith("step:")]
+    if not steps:
+        # A headers-only log (truncated or step-stripped) would otherwise answer
+        # [] as a structured success and flip routing.propose to the all-passed
+        # claim even when the log's own result: header declares a failure — the
+        # same fabrication class the outcome-enum check guards. An evidence tool
+        # never silently accepts an episode with no logged steps.
+        raise _Unparseable("run log has a task header but no step: lines")
+    _check_result_header(_optional_header(lines, "result"), steps)
+    return [_finding(step, task) for step in steps]
+
+
+def _header(lines: list[str], key: str) -> str:
+    prefix = f"{key}:"
+    for line in lines:
+        if line.startswith(prefix):
+            value = line[len(prefix) :].strip()
+            if value:
+                return value
+    raise _Unparseable(f"no {key}: header line")
+
+
+def _optional_header(lines: list[str], key: str) -> str | None:
+    prefix = f"{key}:"
+    for line in lines:
+        if line.startswith(prefix):
+            value = line[len(prefix) :].strip()
+            return value or None
+    return None
+
+
+def _parse_step(line: str) -> _Step:
+    match = _STEP_RE.match(line)
+    if match is None:
+        raise _Unparseable(f"malformed step line: {line!r}")
+    outcome = match.group("outcome")
+    if outcome not in OUTCOMES:
+        # An unvalidated token would flip _propose to "every logged step passed"
+        # for a failed run, so the enum is checked at this boundary (conventions:
+        # validate at the boundary, trust types inside).
+        raise _Unparseable(f"outcome is not one of {sorted(OUTCOMES)}: {outcome!r}")
+    return _Step(
+        ordinal=match.group("ordinal"),
+        role=match.group("role"),
+        model=match.group("model"),
+        outcome=outcome,
+        reason=match.group("reason"),
+        evidence=match.group("evidence"),
+    )
+
+
+def _finding(step: _Step, task: str) -> Finding:
+    return Finding(
+        model=step.model,
+        task=task,
+        outcome=step.outcome,
+        reason=step.reason,
+        evidence=step.evidence,
+        role=step.role,
+    )
+
+
+def _check_result_header(result_line: str | None, steps: list[_Step]) -> None:
+    """A log's ``result:`` header is evidence too: when present it must not
+    contradict the steps it summarizes. Left unchecked, ``result: pass`` over a
+    logged failure (or a failed_steps list naming steps that passed) would let
+    run.inspect answer a success the log itself calls a lie, and
+    routing.propose the all-passed claim for a failed episode. The header stays
+    optional — producers older than the failed_steps convention omit it — and
+    the check runs only once every step line has parsed."""
+    if result_line is None:
+        return
+    match = _RESULT_RE.fullmatch(result_line)
+    if match is None:
+        raise _Unparseable(f"malformed result: header: {result_line!r}")
+    outcome = match.group("outcome")
+    if outcome not in OUTCOMES:
+        raise _Unparseable(f"result header outcome is not one of {sorted(OUTCOMES)}: {outcome!r}")
+    failed = [step for step in steps if step.outcome == FAIL]
+    if outcome == PASS:
+        if failed:
+            raise _Unparseable(
+                f"result header declares pass but logged step {failed[0].ordinal} failed"
+            )
+        return
+    declared = _declared_failed_steps(match.group("failed_steps"))
+    if not failed:
+        raise _Unparseable("result header declares a failure no logged step records")
+    logged = {step.ordinal.lstrip("0") or "0" for step in failed}
+    if declared != logged:
+        raise _Unparseable(
+            f"result header failed_steps={match.group('failed_steps')!r} does not match"
+            f" the logged failing step(s) {sorted(logged)}"
+        )
+
+
+def _declared_failed_steps(token: str | None) -> set[str]:
+    if token is None:
+        raise _Unparseable("result header declares a failure without a failed_steps list")
+    declared = {part.strip().lstrip("0") or "0" for part in token.split(",") if part.strip()}
+    if not declared:
+        raise _Unparseable(f"result header failed_steps is empty: {token!r}")
+    return declared

debrief_mcp-0.1.0/src/debrief/adapters/ggufbench_store.py

@@ -0,0 +1,250 @@
+"""ggufbench artifacts adapter: read-only mapping of artifact rows to Findings (R3).
+
+Layout: one ``<run-id>/results.json`` per ggufbench run under the artifact root.
+The adapter is the only place that knows this format (ADR-0002).
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import math
+import re
+import stat
+from pathlib import Path
+from typing import Any
+
+from debrief.domain import OUTCOMES, Finding, TaskSignature
+from debrief.errors import BENCH_SOURCE_MISSING, SOURCE_UNPARSEABLE, DebriefError
+
+log = logging.getLogger(__name__)
+
+RESULTS_FILE = "results.json"
+
+# A named tuple, not an inline `except (A, B):`: ruff format (target py314) rewrites
+# the inline spelling towards the PEP 758 bare-comma form, which this repo's review
+# rejected; the named tuple keeps one spelling that is valid on every Python 3.
+_MISSING_PATH_ERRORS = (FileNotFoundError, NotADirectoryError)
+
+# Artifact string fields reach the routing.propose text verbatim, so a newline
+# in a poisoned row could forge arbitrary proposal diff lines (ADR-0004
+# fabrication class). Identifier-like fields must be single-line whitespace-free
+# tokens; free-text fields single-line control-char-free — both validated here,
+# at the boundary, per conventions.
+_TOKEN_RE = re.compile(r"\S+")
+
+
+class _Unparseable(Exception):
+    """An artifact did not match the ggufbench results schema."""
+
+
+class GgufbenchStore:
+    """BenchStore over ggufbench artifacts; opens files for reading only (R2, R3)."""
+
+    def __init__(self, artifact_root: Path) -> None:
+        self._artifact_root = artifact_root
+
+    def findings(self) -> list[Finding] | DebriefError:
+        try:
+            results_files = self._results_files()
+        except OSError as err:
+            return DebriefError(
+                code=SOURCE_UNPARSEABLE,
+                message=f"{self._artifact_root} could not be scanned for {RESULTS_FILE}",
+                detail=str(err),
+            )
+        except _Unparseable as err:
+            # A symlink below the requested path must not turn the store into a
+            # read primitive for files outside the evidence root (conventions:
+            # resolve against an explicit allowed root — security-audit A03).
+            return DebriefError(
+                code=SOURCE_UNPARSEABLE,
+                message=f"a run's {RESULTS_FILE} escapes the ggufbench artifact root",
+                detail=str(err),
+            )
+        if results_files is None:
+            return DebriefError(
+                code=BENCH_SOURCE_MISSING,
+                message="no ggufbench artifacts at the requested path",
+                detail=str(self._artifact_root),
+            )
+        if not results_files:
+            return DebriefError(
+                code=BENCH_SOURCE_MISSING,
+                message=f"no {RESULTS_FILE} under any run directory",
+                detail=str(self._artifact_root),
+            )
+        findings: list[Finding] = []
+        for results_file in results_files:
+            mapped = _read_run(results_file)
+            if isinstance(mapped, DebriefError):
+                return mapped
+            findings.extend(mapped)
+        log.info("read %d ggufbench row(s) from %d run(s)", len(findings), len(results_files))
+        return findings
+
+    def _results_files(self) -> list[Path] | None:
+        """One results.json per run under the root; None when the root itself is absent.
+
+        Scans with stat(), not is_file(): pathlib's is_*() answer False for
+        unreadable directories, which would silently drop a run the tool must
+        report; stat() raises, and findings() maps that to a structured error.
+        Every candidate is resolved and required to stay under the resolved
+        root before anything is read, so symlink indirection below the
+        requested path cannot escape the evidence root (A03).
+        """
+        root = self._artifact_root.resolve()
+        if not root.is_dir():
+            return None
+        own = _contained_results(root, root / RESULTS_FILE)
+        if own is not None:
+            return [own]
+        found: list[Path] = []
+        for path in root.iterdir():
+            contained = _contained_results(root, path / RESULTS_FILE)
+            if contained is not None:
+                found.append(contained)
+        return sorted(found)
+
+
+def _contained_results(root: Path, candidate: Path) -> Path | None:
+    """Resolve `candidate` and return it only when it stays under `root` and is a file.
+
+    A `<run>/results.json` may be a symlink: without this check an artifact
+    could name any file on the host and its bytes would enter findings,
+    bypassing the root boundary every agent-controlled path respects.
+    An escaping candidate refuses the whole scan — an evidence tool never
+    silently drops (or silently reads past its boundary) a run directory.
+    """
+    try:
+        resolved = candidate.resolve()
+    except OSError as err:
+        raise _Unparseable(f"{candidate} could not be resolved") from err
+    if not resolved.is_relative_to(root):
+        raise _Unparseable(f"{candidate} resolves outside the artifact root")
+    return resolved if _is_regular_file(resolved) else None
+
+
+def _is_regular_file(path: Path) -> bool:
+    try:
+        return stat.S_ISREG(path.stat().st_mode)
+    except _MISSING_PATH_ERRORS:
+        return False
+
+
+def _read_run(results_file: Path) -> list[Finding] | DebriefError:
+    try:
+        document: Any = json.loads(results_file.read_text(encoding="utf-8"))
+    except (OSError, ValueError, RecursionError) as err:
+        # ValueError covers JSONDecodeError and undecodable bytes; RecursionError
+        # covers nesting-bomb JSON, which no legitimate artifact carries.
+        return DebriefError(
+            code=SOURCE_UNPARSEABLE,
+            message=f"{results_file} could not be read as JSON",
+            detail=str(err),
+        )
+    try:
+        return _map_document(document)
+    except _Unparseable as err:
+        return DebriefError(
+            code=SOURCE_UNPARSEABLE,
+            message=f"{results_file} is not a ggufbench results document",
+            detail=str(err),
+        )
+
+
+def _map_document(document: Any) -> list[Finding]:
+    if not isinstance(document, dict):
+        raise _Unparseable("document is not an object")
+    engine_build = _required_token(document, "engine_build")
+    baseline = _required_token(document, "baseline")
+    rows = document.get("rows")
+    if not isinstance(rows, list):
+        raise _Unparseable("rows is not an array")
+    return [_map_row(row, baseline, engine_build) for row in rows]
+
+
+def _map_row(row: Any, baseline: str, engine_build: str) -> Finding:
+    if not isinstance(row, dict):
+        raise _Unparseable("row is not an object")
+    graded = _required_counts(row)
+    score = row.get("score")
+    if isinstance(score, bool) or not isinstance(score, (int, float)):
+        raise _Unparseable("score is not a number")
+    try:
+        measure = float(score)
+    except OverflowError as err:
+        # JSON integers are arbitrary-precision; a score too wide for a float is
+        # corrupt evidence, not a crash (R1: every answer is findings or a
+        # structured error).
+        raise _Unparseable(f"score overflows a float: {score}") from err
+    if not math.isfinite(measure):
+        # json.loads accepts the NaN/Infinity/-Infinity tokens; a non-finite
+        # measure would win the proposal ranking and fabricate a role change
+        # (the ADR-0004 fabrication class), so it refuses at this boundary.
+        raise _Unparseable(f"score is not a finite number: {measure!r}")
+    outcome = _required_str(row, "outcome")
+    if outcome not in OUTCOMES:
+        raise _Unparseable(f"outcome is not one of {sorted(OUTCOMES)}: {outcome!r}")
+    return Finding(
+        model=_required_token(row, "model"),
+        task=_required_token(row, "task"),
+        outcome=outcome,
+        reason=_required_line(row, "reason"),
+        evidence=_required_line(row, "artifact"),
+        signature=TaskSignature.of(graded, baseline, engine_build),
+        measure=measure,
+    )
+
+
+def _required_counts(row: dict[str, Any]) -> dict[str, int]:
+    graded = row.get("graded")
+    if not isinstance(graded, dict):
+        raise _Unparseable("graded is not an object of counts")
+    counts: dict[str, int] = {}
+    for dimension, count in graded.items():
+        if not isinstance(dimension, str) or isinstance(count, bool) or not isinstance(count, int):
+            raise _Unparseable("graded counts must map dimension names to integers")
+        if _TOKEN_RE.fullmatch(dimension) is None:
+            # Dimension names join the task signature and the proposal text; a
+            # multi-line name would forge proposal lines exactly like a model id.
+            raise _Unparseable(f"graded dimension is not a single-line token: {dimension!r}")
+        counts[dimension] = count
+    return counts
+
+
+def _required_str(document: dict[str, Any], key: str) -> str:
+    value = document.get(key)
+    if not isinstance(value, str) or not value:
+        raise _Unparseable(f"{key} is not a non-empty string")
+    return value
+
+
+def _required_token(document: dict[str, Any], key: str) -> str:
+    """Identifier-like field: one non-empty, whitespace-free line (no newline forgery)."""
+    value = _required_str(document, key)
+    if _TOKEN_RE.fullmatch(value) is None:
+        raise _Unparseable(f"{key} is not a single-line whitespace-free token: {value!r}")
+    _refuse_control_chars(key, value)
+    return value
+
+
+def _required_line(document: dict[str, Any], key: str) -> str:
+    """Free-text field: one non-empty line with no control characters (incl. newlines).
+
+    The single-line rule is str.splitlines(), not just ``ord(ch) < 0x20``:
+    Unicode line separators (U+0085 NEL, U+2028, U+2029) sit above that range
+    but still split the value everywhere Python splits text, so a poisoned
+    reason or artifact path could smuggle a forged proposal line past a pure
+    control-character check.
+    """
+    value = _required_str(document, key)
+    _refuse_control_chars(key, value)
+    if value.splitlines() != [value]:
+        raise _Unparseable(f"{key} is not a single line: {value!r}")
+    return value
+
+
+def _refuse_control_chars(key: str, value: str) -> None:
+    if any(ord(ch) < 0x20 or ord(ch) == 0x7F for ch in value):
+        raise _Unparseable(f"{key} contains control characters: {value!r}")

debrief_mcp-0.1.0/src/debrief/domain.py

@@ -0,0 +1,79 @@
+"""Domain: the shared currency of both ports and the comparability rule (ADR-0002, ADR-0004).
+
+Imports nothing outward — no ggufbench, buildwright, mcp or filesystem here.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+
+from debrief.errors import INCOMPARABLE_ROWS, DebriefError
+
+PASS = "pass"
+FAIL = "fail"
+OUTCOMES = frozenset({PASS, FAIL})
+
+
+@dataclass(frozen=True)
+class TaskSignature:
+    """Graded counts per dimension plus the baseline and engine build (ADR-0004).
+
+    ggufbench derives it from the data because version labels have lied before;
+    two rows compare only inside one signature.
+    """
+
+    graded_counts: tuple[tuple[str, int], ...]
+    baseline: str
+    engine_build: str
+
+    @classmethod
+    def of(cls, graded: Mapping[str, int], baseline: str, engine_build: str) -> TaskSignature:
+        return cls(tuple(sorted(graded.items())), baseline, engine_build)
+
+    @property
+    def key(self) -> str:
+        graded = ",".join(f"{dimension}:{count}" for dimension, count in self.graded_counts)
+        return f"{graded}@{self.baseline}@{self.engine_build}"
+
+
+@dataclass(frozen=True)
+class Finding:
+    """The shared currency: model, task, outcome, reason, evidence pointer (ADR-0002).
+
+    `signature`, `measure` and `role` carry what one port knows and the other
+    does not: bench rows carry a signature and a score, run steps carry the
+    buildwright role. Neither port fills the other's fields.
+    """
+
+    model: str
+    task: str
+    outcome: str
+    reason: str
+    evidence: str
+    signature: TaskSignature | None = None
+    measure: float | None = None
+    role: str | None = None
+
+
+def shared_signature(rows: Sequence[Finding]) -> TaskSignature | DebriefError:
+    """The one task signature covering every row, or a refusal (ADR-0004).
+
+    Rows that do not share a signature answer different questions, so the
+    caller gets `incomparable_rows` with the groups named — never an average.
+    """
+    signatures = {row.signature for row in rows if row.signature is not None}
+    if len(signatures) > 1:
+        listed = ", ".join(sorted(signature.key for signature in signatures))
+        return DebriefError(
+            code=INCOMPARABLE_ROWS,
+            message="rows span more than one task signature; refusing to rank or average",
+            detail=f"{len(signatures)} signatures: {listed}",
+        )
+    if not signatures:
+        return DebriefError(
+            code=INCOMPARABLE_ROWS,
+            message="no row records a task signature, so comparability cannot be established",
+            detail=f"{len(rows)} row(s): " + ", ".join(f"{row.model}/{row.task}" for row in rows),
+        )
+    return signatures.pop()

debrief_mcp-0.1.0/src/debrief/errors.py

@@ -0,0 +1,24 @@
+"""Structured errors: every debrief answer is findings or one of these (R1)."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+BENCH_SOURCE_MISSING = "bench_source_missing"
+RUN_SOURCE_MISSING = "run_source_missing"
+SOURCE_UNPARSEABLE = "source_unparseable"
+INCOMPARABLE_ROWS = "incomparable_rows"
+PATH_OUTSIDE_ROOT = "path_outside_root"
+
+
+@dataclass(frozen=True)
+class DebriefError:
+    """A refusal a tool returns instead of findings.
+
+    Returned, never raised, so the transport adapter can always render a typed
+    response; `code` is the stable contract an agent branches on.
+    """
+
+    code: str
+    message: str
+    detail: str

debrief_mcp-0.1.0/src/debrief/mcp_server.py

@@ -0,0 +1,218 @@
+"""MCP stdio adapter: three tools that answer from receipts and never write (R1, R2).
+
+Tool names carry the port (bench.*, run.*, routing.propose) so an agent cannot
+confuse what was measured with what happened in one run (ADR-0002).
+"""
+
+from __future__ import annotations
+
+import importlib.metadata
+import logging
+import os
+import signal
+import sys
+from dataclasses import dataclass, field
+from pathlib import Path
+
+from mcp.server.mcpserver import MCPServer
+
+from debrief.adapters.buildwright_log import BuildwrightLog
+from debrief.adapters.ggufbench_store import GgufbenchStore
+from debrief.domain import Finding
+from debrief.errors import PATH_OUTSIDE_ROOT, DebriefError
+from debrief.use_cases import InspectRun, ProposeRole, QueryBench
+
+log = logging.getLogger(__name__)
+
+BENCH_ROOT_ENV = "DEBRIEF_BENCH_ROOT"
+RUN_ROOT_ENV = "DEBRIEF_RUN_ROOT"
+DEFAULT_BENCH_ROOT = "artifacts/ggufbench"
+DEFAULT_RUN_ROOT = "artifacts/buildwright"
+
+_HELP = f"""\
+debrief-mcp {importlib.metadata.version("debrief-mcp")} — answer agent questions from receipts
+MCP stdio server, read-only: every answer is findings or a structured error; never writes.
+
+usage: debrief [-h | --help] [--version]
+
+tools (MCP stdio, JSON-RPC over stdin/stdout; logs go to stderr):
+
+  bench.query      What did ggufbench measure? Findings from ggufbench artifacts.
+                   arg: path — run directory (or the artifact root itself) relative
+                   to the bench root.
+  run.inspect      Where did a buildwright run break? Findings from one run log.
+                   arg: path — the log file relative to the run root.
+  routing.propose  A BW_ROLE_* proposal as text citing both ports' evidence.
+                   args: bench_path (ggufbench artifacts dir), run_path (run log).
+                   Text only; never applied (ADR-0001), targets roles, never
+                   models.yaml (ADR-0003); incomparable rows are refused, never
+                   averaged (ADR-0004).
+
+environment:
+
+  DEBRIEF_BENCH_ROOT  allowed root for ggufbench artifacts
+                      (default: {DEFAULT_BENCH_ROOT})
+  DEBRIEF_RUN_ROOT    allowed root for buildwright run logs
+                      (default: {DEFAULT_RUN_ROOT})
+  DEBRIEF_LOG_LEVEL   stderr logging level (default: INFO)
+
+options:
+
+  -h, --help     show this help and exit
+  --version      show the installed debrief version and exit
+
+exit codes:
+
+  0    normal (help/version, or the client closed the session)
+  2    unknown argument
+  130  interrupted (SIGINT)
+"""
+
+server = MCPServer(
+    "debrief",
+    instructions=(
+        "debrief answers agent questions from receipts: bench.query (ggufbench artifacts), "
+        "run.inspect (one buildwright run log), routing.propose (a BW_ROLE_* diff as text). "
+        "It never writes; every answer is findings or a structured error."
+    ),
+)
+
+
+@dataclass(frozen=True)
+class FindingsResponse:
+    """A bench.* / run.* answer: findings, or a structured error and no findings (R1)."""
+
+    findings: list[Finding] = field(default_factory=list)
+    error: DebriefError | None = None
+
+    @classmethod
+    def of(cls, answer: list[Finding] | DebriefError) -> FindingsResponse:
+        if isinstance(answer, DebriefError):
+            return cls(error=answer)
+        return cls(findings=list(answer))
+
+
+@dataclass(frozen=True)
+class ProposalResponse:
+    """The routing.propose answer: a BW_ROLE_* diff as text, or a structured error (R1, R5)."""
+
+    proposal: str = ""
+    error: DebriefError | None = None
+
+    @classmethod
+    def of(cls, answer: str | DebriefError) -> ProposalResponse:
+        if isinstance(answer, DebriefError):
+            return cls(error=answer)
+        return cls(proposal=answer)
+
+
+@server.tool(
+    name="bench.query",
+    title="What did ggufbench measure?",
+    description=(
+        "Findings from ggufbench artifacts. `path` is a run directory (or the artifact root "
+        "itself) relative to DEBRIEF_BENCH_ROOT. Read-only."
+    ),
+)
+def bench_query(path: str) -> FindingsResponse:
+    resolved = _resolve_within(BENCH_ROOT_ENV, DEFAULT_BENCH_ROOT, path)
+    if isinstance(resolved, DebriefError):
+        return FindingsResponse(error=resolved)
+    return FindingsResponse.of(QueryBench(GgufbenchStore(resolved)).run())
+
+
+@server.tool(
+    name="run.inspect",
+    title="Where did this run break?",
+    description=(
+        "Findings from one buildwright run log. `path` is the log file relative to "
+        "DEBRIEF_RUN_ROOT. Read-only."
+    ),
+)
+def run_inspect(path: str) -> FindingsResponse:
+    resolved = _resolve_within(RUN_ROOT_ENV, DEFAULT_RUN_ROOT, path)
+    if isinstance(resolved, DebriefError):
+        return FindingsResponse(error=resolved)
+    return FindingsResponse.of(InspectRun(BuildwrightLog(resolved)).run())
+
+
+@server.tool(
+    name="routing.propose",
+    title="What would you change?",
+    description=(
+        "A BW_ROLE_* proposal as text, citing bench.query and run.inspect evidence. "
+        "`bench_path` is a ggufbench artifact directory under DEBRIEF_BENCH_ROOT, `run_path` "
+        "a buildwright log under DEBRIEF_RUN_ROOT. Returns text; never writes."
+    ),
+)
+def routing_propose(bench_path: str, run_path: str) -> ProposalResponse:
+    bench_dir = _resolve_within(BENCH_ROOT_ENV, DEFAULT_BENCH_ROOT, bench_path)
+    if isinstance(bench_dir, DebriefError):
+        return ProposalResponse.of(bench_dir)
+    run_file = _resolve_within(RUN_ROOT_ENV, DEFAULT_RUN_ROOT, run_path)
+    if isinstance(run_file, DebriefError):
+        return ProposalResponse.of(run_file)
+    propose = ProposeRole(
+        QueryBench(GgufbenchStore(bench_dir)), InspectRun(BuildwrightLog(run_file))
+    )
+    return ProposalResponse.of(propose.run())
+
+
+def _resolve_within(root_env: str, default_root: str, requested: str) -> Path | DebriefError:
+    """Resolve `requested` against the allowed root; refuse anything that escapes it (A03)."""
+    root = Path(os.environ.get(root_env, default_root)).resolve()
+    try:
+        candidate = (root / requested).resolve() if requested else root
+    except (ValueError, OSError) as err:
+        # ValueError: embedded NUL bytes survive Path construction and only
+        # explode when os calls lstat under resolve(); refuse, never crash.
+        return DebriefError(
+            code=PATH_OUTSIDE_ROOT,
+            message="requested path could not be resolved within the allowed evidence root",
+            detail=f"{requested!r}: {err}",
+        )
+    if not candidate.is_relative_to(root):
+        return DebriefError(
+            code=PATH_OUTSIDE_ROOT,
+            message="requested path is outside the allowed evidence root",
+            detail=f"{requested!r} resolves outside {root}",
+        )
+    return candidate
+
+
+def _on_sigint(signum: int, frame: object) -> None:
+    # Ctrl+C on a stdio server is a normal shutdown, not a crash. Exit
+    # immediately with the conventional SIGINT code: os._exit skips the
+    # interpreter shutdown, where a still-running SDK thread would otherwise
+    # turn a second Ctrl+C into an "Exception ignored" traceback.
+    log.info("debrief MCP server stopping (SIGINT)")
+    sys.stderr.flush()
+    os._exit(130)
+
+
+def main() -> None:
+    args = sys.argv[1:]
+    if "-h" in args or "--help" in args:
+        print(_HELP, end="")
+        return
+    if "--version" in args:
+        print(f"debrief-mcp {importlib.metadata.version('debrief-mcp')}")
+        return
+    if args:
+        # The server takes no positional input: anything else is a typo that
+        # should fail fast instead of silently starting a stdio session that
+        # would then hang waiting for JSON-RPC on the terminal.
+        print(
+            f"debrief: unrecognized argument(s): {' '.join(args)}\ntry 'debrief --help'",
+            file=sys.stderr,
+        )
+        raise SystemExit(2)
+
+    logging.basicConfig(
+        level=os.environ.get("DEBRIEF_LOG_LEVEL", "INFO"),
+        stream=sys.stderr,
+        format="%(levelname)s %(name)s %(message)s",
+    )
+    log.info("debrief MCP server starting on stdio (read-only)")
+    signal.signal(signal.SIGINT, _on_sigint)
+    server.run()

debrief_mcp-0.1.0/src/debrief/ports.py

@@ -0,0 +1,20 @@
+"""Ports the domain drives: two independent evidence sources (ADR-0002)."""
+
+from __future__ import annotations
+
+from typing import Protocol
+
+from debrief.domain import Finding
+from debrief.errors import DebriefError
+
+
+class BenchStore(Protocol):
+    """What ggufbench measured (R3): the only bench source debrief reads."""
+
+    def findings(self) -> list[Finding] | DebriefError: ...
+
+
+class RunLog(Protocol):
+    """One buildwright run (R4): the only run source debrief reads."""
+
+    def findings(self) -> list[Finding] | DebriefError: ...

debrief_mcp-0.1.0/src/debrief/use_cases.py

@@ -0,0 +1,131 @@
+"""Driving use cases: one per agent question (QueryBench S1, InspectRun S2, ProposeRole S3)."""
+
+from __future__ import annotations
+
+import logging
+
+from debrief.domain import FAIL, PASS, Finding, shared_signature
+from debrief.errors import DebriefError
+from debrief.ports import BenchStore, RunLog
+
+log = logging.getLogger(__name__)
+
+
+class QueryBench:
+    """ "What did ggufbench measure?" — answered only from a BenchStore (R3)."""
+
+    def __init__(self, store: BenchStore) -> None:
+        self._store = store
+
+    def run(self) -> list[Finding] | DebriefError:
+        answer = self._store.findings()
+        if isinstance(answer, DebriefError):
+            log.warning("bench query refused: %s", answer.code)
+            return answer
+        log.info("bench query answered from receipts: %d findings", len(answer))
+        return answer
+
+
+class InspectRun:
+    """ "Where did this run break?" — answered only from a RunLog (R4)."""
+
+    def __init__(self, run_log: RunLog) -> None:
+        self._run_log = run_log
+
+    def run(self) -> list[Finding] | DebriefError:
+        answer = self._run_log.findings()
+        if isinstance(answer, DebriefError):
+            log.warning("run inspection refused: %s", answer.code)
+            return answer
+        log.info("run inspection answered from receipts: %d findings", len(answer))
+        return answer
+
+
+class ProposeRole:
+    """ "What would you change?" — a BW_ROLE_* diff as text (R5, ADR-0003).
+
+    The only use case allowed to read findings from both ports (ADR-0002);
+    it renders text and writes nothing (ADR-0001).
+    """
+
+    def __init__(self, bench: QueryBench, run: InspectRun) -> None:
+        self._bench = bench
+        self._run = run
+
+    def run(self) -> str | DebriefError:
+        bench_answer = self._bench.run()
+        if isinstance(bench_answer, DebriefError):
+            return bench_answer
+        run_answer = self._run.run()
+        if isinstance(run_answer, DebriefError):
+            return run_answer
+        return _propose(bench_answer, run_answer)
+
+
+def _propose(bench_rows: list[Finding], run_rows: list[Finding]) -> str | DebriefError:
+    failures = [row for row in run_rows if row.outcome == FAIL]
+    if not failures:
+        return "no proposal: every logged step passed"
+    lines: list[str] = []
+    proposed_roles: set[str] = set()
+    for failure in failures:
+        role = failure.role
+        if role is None:
+            lines.append(
+                f"note: step on {failure.task} recorded no role, nothing to propose"
+                f" ({failure.evidence})"
+            )
+            continue
+        canonical = _canonical_role(role)
+        if canonical in proposed_roles:
+            continue
+        proposal = _role_proposal(canonical, failure, bench_rows)
+        if isinstance(proposal, DebriefError):
+            return proposal
+        lines.extend(proposal)
+        proposed_roles.add(canonical)
+    return "\n".join(lines)
+
+
+def _canonical_role(role: str) -> str:
+    """One env name per role: step lines record the env-var name as often as the
+    bare role, and a token that already carries the canonical prefix would
+    otherwise render a `BW_ROLE_BW_ROLE_*` diff nobody can paste into the
+    buildwright env."""
+    return role.upper().removeprefix("BW_ROLE_")
+
+
+def _role_proposal(
+    role: str, failure: Finding, bench_rows: list[Finding]
+) -> list[str] | DebriefError:
+    role_env = f"BW_ROLE_{role.upper()}"
+    cohort = [row for row in bench_rows if row.task == failure.task]
+    if not cohort:
+        return [
+            f"note: ggufbench never measured {failure.task}, keeping {role_env}={failure.model}"
+        ]
+    signature = shared_signature(cohort)
+    if isinstance(signature, DebriefError):
+        return signature
+    candidates = [
+        row
+        for row in cohort
+        if row.model != failure.model and row.outcome == PASS and row.measure is not None
+    ]
+    if not candidates:
+        return [
+            f"note: no passing model for {failure.task} in the bench cohort,"
+            f" keeping {role_env}={failure.model}"
+        ]
+    winner = sorted(candidates, key=lambda row: (-(row.measure or 0.0), row.model))[0]
+    return [
+        f"proposal: {role_env} holds {failure.model}, which failed task {failure.task}",
+        f"- {role_env}={failure.model}",
+        f"+ {role_env}={winner.model}",
+        f"why run: {failure.model} failed {failure.task} ({failure.reason}) — {failure.evidence}",
+        f"why bench: {winner.model} passed {failure.task} at score {winner.measure:.2f}"
+        f" ({winner.reason}) — {winner.evidence}",
+        f"signature: {signature.key}",
+        "not applied: debrief never writes; paste the + line into the buildwright role env"
+        " (ADR-0001). Proposals target roles, never models.yaml (ADR-0003).",
+    ]