fde-framework 0.1.3__tar.gz → 0.1.4__tar.gz

{fde_framework-0.1.3 → fde_framework-0.1.4}/CHANGELOG.md

@@ -5,6 +5,33 @@ the project is pre-release, so everything sits under 0.1.0 until the first tag.
 
 ## [Unreleased]
 
+## [0.1.4] — 2026-09-09
+
+The measurement release: claims the framework already made, turned into
+numbers and walks.
+
+- Projects with a retrieval component ship `evals/retrieval.py`: recall@10
+  and recall@50 against golden queries in `retrieval_cases.jsonl`, gated in
+  CI, refusing an empty case set. The embedding and index choices set a
+  ceiling nothing downstream recovers; this measures the ceiling by itself,
+  no model in the loop. The diagnosis walk's evidence step now ends at this
+  number, and both embedding approaches record the doctrine: chosen by
+  measured recall on the engagement's own queries, never by leaderboard.
+- Emitted projects ship `ops/diagnosis.md`: the walk that finds which layer
+  a failure lives in -- definitions, evidence, tools, loop, model -- ordered
+  cheapest-to-check first, sections adapted to the components actually in
+  the system. An unclear definition can look like a model error; the
+  expensive habit is re-prompting before finding out.
+- Retrieval corpus: `corpus_churn` dimension (static/periodic/continuous —
+  how fast the document stock turns over, distinct from corpus size and
+  query arrival) and a graph-expanded-retrieval approach: vector entry,
+  entity expansion, rerank exit, for multi-hop questions on a corpus that
+  keeps changing. Full graph-retrieval now steps aside on continuous churn,
+  by name, with the reason on the record. Realizations for plain-python,
+  pgvector and qdrant.
+- README quickstart shows the gate refusal before the passing build, so the
+  first run's refusal is announced rather than a surprise.
+
 ## [0.1.3] — 2026-09-08
 
 A post-launch funnel audit replayed every public transcript against the

{fde_framework-0.1.3 → fde_framework-0.1.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: fde-framework
-Version: 0.1.3
+Version: 0.1.4
 Summary: A framework for Forward Deployed Engineers: from problem statement to a deployable AI project, every decision traced to a fact.
 Project-URL: Homepage, https://github.com/atulkapoor/fde-framework
 Project-URL: Repository, https://github.com/atulkapoor/fde-framework
@@ -76,6 +76,8 @@ pip install fde-framework
 fde start acme --statement "Extract fields from supplier invoices."
 fde ask acme --role admin        # role-scoped discovery interview
 fde architect acme               # the design, with cited rationale
+fde build acme --out project     # refuses: seven gates guard the build
+# ...verify data access, capture the baseline (each gate prints its remedy), then:
 fde build acme --out project     # code + evals + deploy assets + runbook
 ```
 

{fde_framework-0.1.3 → fde_framework-0.1.4}/README.md

@@ -38,6 +38,8 @@ pip install fde-framework
 fde start acme --statement "Extract fields from supplier invoices."
 fde ask acme --role admin        # role-scoped discovery interview
 fde architect acme               # the design, with cited rationale
+fde build acme --out project     # refuses: seven gates guard the build
+# ...verify data access, capture the baseline (each gate prints its remedy), then:
 fde build acme --out project     # code + evals + deploy assets + runbook
 ```
 

fde_framework-0.1.4/framework/approaches/graph-expanded-retrieval.md

@@ -0,0 +1,24 @@
+---
+id: graph-expanded-retrieval
+name: Graph-expanded retrieval
+complexity: 3
+components: [retrieval]
+applies_when: [query_pattern == multi_hop and corpus_churn == continuous]
+avoid_when: [query_pattern == lookup]
+evidence: {case_ids: [structured-extraction], confidence: low, last_verified: 2026-09-09}
+---
+Vector search finds the entry points, the entities they mention open the
+graph, a bounded expansion collects what connects, and a reranker orders the
+merged pool. Most of what a full graph buys on multi-hop questions -- without
+asking the graph to carry recall.
+
+That one reassignment is what survives churn. With vector search carrying
+recall, the graph keeps a lighter contract -- entities and first-class edges,
+rebuilt incrementally -- instead of the exhaustive index whose super-linear
+growth and drifting entity resolution turn a moving corpus's graph
+confidently wrong.
+
+Where the corpus holds still, the full graph still wins: richer edges, deeper
+traversal, no rerank pass to pay for. This exists for the corpus that will
+not hold still. And like the reranker, it is adopted on measurement rather
+than fashion: run the golden multi-hop queries and let the recall gap argue.

{fde_framework-0.1.3 → fde_framework-0.1.4}/framework/approaches/graph-retrieval.md

@@ -4,8 +4,8 @@ name: Graph retrieval
 complexity: 3
 components: [retrieval]
 applies_when: [query_pattern == multi_hop]
-avoid_when: [query_pattern == lookup, query_pattern == comparative]
-evidence: {case_ids: [structured-extraction], confidence: medium, last_verified: 2026-08-21}
+avoid_when: [query_pattern == lookup, query_pattern == comparative, corpus_churn == continuous]
+evidence: {case_ids: [structured-extraction], confidence: medium, last_verified: 2026-09-09}
 ---
 Explicit entities and edges, traversed at query time.
 
@@ -16,7 +16,10 @@ rather than an upgrade.
 
 The costs are real: multi-pass extraction to build it, two to three times the
 end-to-end latency to use it, and an index that grows super-linearly, which is
-what makes incremental updates painful on a corpus that changes.
+what makes incremental updates painful on a corpus that changes. Where the
+corpus changes continuously, that maintenance is the deciding cost --
+graph-expanded retrieval keeps the multi-hop benefit by letting vector search
+carry recall, so the graph can stay small enough to rebuild.
 
 Published gains also warrant scepticism -- judge position bias has been shown to
 swing reported win rates by tens of points. Measure on your own traffic.

{fde_framework-0.1.3 → fde_framework-0.1.4}/framework/approaches/local-embedding.md

@@ -19,6 +19,11 @@ Ruled out where nobody is named to operate it. A model with no owner is a
 liability handed over with a bow on it, and an embedding model whose host dies
 takes the index with it.
 
+Whichever model, it sets the ceiling on retrieval: nothing downstream
+recovers a document that was never encoded well enough to surface. The
+choice is made by measured recall on the engagement's own golden queries --
+the emitted `evals/retrieval.py` is that measurement -- not by leaderboard.
+
 The handover concern is stated precisely rather than broadly: with nobody
 named to operate and data free to leave, the managed alternative is
 strictly simpler and wins. But where data cannot leave, the plain

{fde_framework-0.1.3 → fde_framework-0.1.4}/framework/approaches/managed-embedding.md

@@ -19,4 +19,7 @@ to their source, so this is not a way of anonymising anything on the way out.
 
 One-way, like any vendor call. And expensive to reverse for a second reason --
 changing embedding model means reindexing everything, so leaving is a
-reindexing project rather than a configuration change.
+reindexing project rather than a configuration change. Which is exactly why
+the model is chosen by measured recall on the engagement's own golden
+queries (the emitted `evals/retrieval.py`) rather than by leaderboard: the
+reindexing bill for a wrong guess arrives after the index is full.

fde_framework-0.1.4/framework/dimensions/corpus_churn.md

@@ -0,0 +1,24 @@
+---
+id: corpus_churn
+type: enum
+scope: data
+kind: requirement
+weight: 1.0
+asks: "How often does the document corpus itself change?"
+ask_role: [admin, user]
+values: [static, periodic, continuous]
+recognises:
+  static: [historical archive, frozen corpus, fixed set of documents, one-time snapshot, corpus is static, closed cases]
+  periodic: [refreshed monthly, monthly refresh, refreshed weekly, quarterly refresh, batch refresh, re-indexed every]
+  continuous: [updated continuously, constantly changing, changes daily, keeps changing, new documents arrive daily, updated throughout the day, live document feed]
+---
+`corpus_size` is stock and `arrival_rate` is the flow of questions; this is
+how fast the stock itself turns over. Nobody volunteers it, because no demo
+runs long enough to feel it.
+
+Any index that is expensive to rebuild pays this rate forever, and the
+expensive ones fail politely: retrieval keeps answering, fluently, from a
+corpus that is no longer the client's. An entity graph pays the rate worst --
+extraction is multi-pass, the index grows super-linearly, and entity
+resolution drifts as names and structures change, which is how a graph goes
+from wrong to confidently wrong without an error in any log.

fde_framework-0.1.4/framework/patterns/graph-expanded-retrieval.md

@@ -0,0 +1,15 @@
+---
+id: graph-expanded-retrieval
+component: retrieval
+approach: graph-expanded-retrieval
+realizations:
+  - {stack: plain-python, template: retrieval/graph-expanded-retrieval.plain.py.j2, provides: Retriever}
+  - {stack: pgvector, template: retrieval/graph-expanded-retrieval.pgvector.py.j2, provides: Retriever}
+  - {stack: qdrant, template: retrieval/graph-expanded-retrieval.qdrant.py.j2, provides: Retriever}
+evidence: {case_ids: [structured-extraction], confidence: low, last_verified: 2026-09-09}
+---
+Implements graph-expanded-retrieval for retrieval, satisfying Retriever.
+
+Vector entry, entity expansion, rerank exit. The graph is deliberately the
+junior partner: it contributes connections, never recall, which is what lets
+it stay small enough to rebuild as the corpus moves.

fde_framework-0.1.4/framework/templates/retrieval/graph-expanded-retrieval.pgvector.py.j2

@@ -0,0 +1,117 @@
+"""{{ component }}: {{ approach }}, via {{ stack }}.
+
+{{ rationale }}
+
+Vector entry, entity expansion, rerank exit -- with both halves in the one
+database already being operated. Chunks and their embeddings in one table,
+entity mentions in another, and the expansion is a self-join on mentions
+rather than a graph engine: for co-mention edges at bounded depth, Postgres
+does this well enough that a separate system has to be argued for.
+
+The mentions table is the whole graph, which is the point. Rebuilding it for
+a changed document is a delete and an insert, not a multi-pass re-extraction
+-- the property that lets this survive a corpus that keeps changing.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from typing import Any
+
+interface = "{{ interface }}"
+approach = "{{ approach }}"
+stack = "{{ stack }}"
+
+SCHEMA_SQL = """
+CREATE EXTENSION IF NOT EXISTS vector;
+CREATE TABLE IF NOT EXISTS {table} (
+    id        text PRIMARY KEY,
+    body      text NOT NULL,
+    embedding vector({dimensions})
+);
+CREATE TABLE IF NOT EXISTS {table}_mentions (
+    entity   text NOT NULL,
+    chunk_id text REFERENCES {table}(id) ON DELETE CASCADE,
+    PRIMARY KEY (entity, chunk_id)
+);
+CREATE INDEX IF NOT EXISTS {table}_mentions_entity_idx
+    ON {table}_mentions (entity);
+"""
+
+SEED_SQL = """
+SELECT id, body FROM {table}
+ORDER BY embedding <=> %(query)s::vector
+LIMIT %(k)s;
+"""
+
+# One hop per query, looped in the application: co-mentioned chunks of the
+# frontier, minus everything already found. Bounded by construction.
+EXPAND_SQL = """
+SELECT DISTINCT m2.chunk_id, c.body
+FROM {table}_mentions m1
+JOIN {table}_mentions m2 ON m2.entity = m1.entity
+JOIN {table} c ON c.id = m2.chunk_id
+WHERE m1.chunk_id = ANY(%(frontier)s)
+  AND NOT m2.chunk_id = ANY(%(found)s);
+"""
+
+
+class {{ class_name }}:
+    """{{ interface }}, as {{ approach }}."""
+
+    interface = "{{ interface }}"
+    approach = "{{ approach }}"
+    stack = "{{ stack }}"
+
+    def __init__(
+        self,
+        connection: Any = None,
+        embed: Callable[[str], list[float]] | None = None,
+        rerank: Callable[[str, str], float] | None = None,
+        table: str = "documents",
+        dimensions: int = 768,
+        max_hops: int = 2,
+        seeds: int = 20,
+    ) -> None:
+        self.connection = connection
+        self.embed = embed
+        self.rerank = rerank
+        self.table = table
+        self.dimensions = dimensions
+        self.max_hops = max_hops
+        self.seeds = seeds
+
+    def schema(self) -> str:
+        return SCHEMA_SQL.format(table=self.table, dimensions=self.dimensions)
+
+    def retrieve(self, query: str, top_k: int = 8) -> list[dict[str, Any]]:
+        found: dict[str, dict[str, Any]] = {}
+        with self.connection.cursor() as cursor:
+            cursor.execute(
+                SEED_SQL.format(table=self.table),
+                {"query": self.embed(query), "k": self.seeds},
+            )
+            for row in cursor.fetchall():
+                found[row[0]] = {"id": row[0], "text": row[1], "hop": 0}
+            frontier = list(found)
+            for hop in range(1, self.max_hops + 1):
+                if not frontier:
+                    break
+                cursor.execute(
+                    EXPAND_SQL.format(table=self.table),
+                    {"frontier": frontier, "found": list(found)},
+                )
+                fresh = cursor.fetchall()
+                if not fresh:
+                    break
+                for row in fresh:
+                    found[row[0]] = {"id": row[0], "text": row[1], "hop": hop}
+                frontier = [row[0] for row in fresh]
+        scored = [
+            {**c, "rerank_score": self.rerank(query, c["text"])}
+            for c in found.values()
+        ]
+        return sorted(scored, key=lambda c: -c["rerank_score"])[:top_k]
+
+    def run(self, payload: dict[str, Any]) -> Any:
+        return self.retrieve(payload["query"], top_k=payload.get("k", 8))

fde_framework-0.1.4/framework/templates/retrieval/graph-expanded-retrieval.plain.py.j2

@@ -0,0 +1,65 @@
+"""{{ component }}: {{ approach }}, via {{ stack }}.
+
+{{ rationale }}
+
+Vector entry, entity expansion, rerank exit. Vector search carries recall,
+the adjacency map contributes connections, and the reranker earns the final
+order. The graph stays a junior partner on purpose -- entities and co-mention
+edges only, cheap to rebuild incrementally -- because on a corpus that keeps
+changing, the exhaustive graph is the part that rots first.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+
+from app.contract import RefusedInput
+
+interface = "{{ interface }}"
+approach = "{{ approach }}"
+stack = "{{ stack }}"
+
+TOP_K = 8
+SEEDS = 20     # vector recall opens the graph; it does not need to finish the job
+MAX_HOPS = 2   # each hop multiplies the pool, and two answers most questions
+
+# Injected at wiring time: retrieve(query, k) -> seed chunks (each a dict with
+# at least id, text, entities); rerank(query, text) -> relevance, higher is
+# better.
+retrieve: Callable | None = None
+rerank: Callable | None = None
+
+# entity -> chunk records that mention it, rebuilt incrementally at index
+# time. Co-mention edges only: the point of this variant is a graph small
+# enough to rebuild as the corpus moves.
+adjacency: dict[str, list[dict]] = {}
+
+
+class {{ class_name }}:
+    def run(self, query: str, top_k: int = TOP_K) -> list[dict]:
+        if retrieve is None or rerank is None:
+            raise RefusedInput(
+                "wire retrieve(query, k) and rerank(query, text) -- an "
+                "expansion over unranked seeds would wear a precision it "
+                "does not have"
+            )
+        pool = {c["id"]: {**c, "hop": 0} for c in retrieve(query, SEEDS)}
+        frontier = list(pool)
+        for hop in range(1, MAX_HOPS + 1):
+            connected = [
+                chunk
+                for chunk_id in frontier
+                for entity in pool[chunk_id].get("entities", [])
+                for chunk in adjacency.get(entity, [])
+            ]
+            fresh = {c["id"]: {**c, "hop": hop}
+                     for c in connected if c["id"] not in pool}
+            if not fresh:
+                break
+            pool.update(fresh)
+            frontier = list(fresh)
+        scored = [
+            {**c, "rerank_score": rerank(query, c.get("text", ""))}
+            for c in pool.values()
+        ]
+        return sorted(scored, key=lambda c: -c["rerank_score"])[:top_k]

fde_framework-0.1.4/framework/templates/retrieval/graph-expanded-retrieval.qdrant.py.j2

@@ -0,0 +1,77 @@
+"""{{ component }}: {{ approach }}, via {{ stack }}.
+
+{{ rationale }}
+
+Vector entry, entity expansion, rerank exit. Qdrant answers "what resembles
+this text", the adjacency map answers "what connects to what", and a reranker
+earns the final order over the merged pool -- expanded chunks arrive without
+a similarity score, and ordering them by hop count would pretend distance is
+relevance.
+
+The graph stays entities and co-mention edges only, rebuilt incrementally,
+because this variant exists for the corpus that keeps changing.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from typing import Any
+
+interface = "{{ interface }}"
+approach = "{{ approach }}"
+stack = "{{ stack }}"
+
+COLLECTION = "{{ component }}_chunks"
+TOP_K = 8
+SEEDS = 20
+MAX_HOPS = 2  # bounded: each hop multiplies latency, and two answers most questions
+
+
+class QdrantUnwired(RuntimeError):
+    """No client is wired; retrieval refuses rather than returning nothing."""
+
+
+client: Any = None
+embed: Callable[[str], list[float]] | None = None
+rerank: Callable[[str, str], float] | None = None
+
+# entity -> connected chunk ids, rebuilt incrementally from co-mentions.
+adjacency: dict[str, list[str]] = {}
+
+
+def run(query: str, top_k: int = TOP_K) -> list[dict]:
+    """Seed by similarity, expand along entities, let the reranker order."""
+    if client is None or embed is None or rerank is None:
+        raise QdrantUnwired(
+            "wire a QdrantClient, an embed(text) callable and a "
+            "rerank(query, text) callable before running -- an empty result "
+            "from an unwired store would read as 'no evidence'"
+        )
+    seeds = client.search(
+        collection_name=COLLECTION, query_vector=embed(query),
+        limit=SEEDS, with_payload=True,
+    )
+    found = {str(hit.id): {"id": str(hit.id), "hop": 0, **(hit.payload or {})}
+             for hit in seeds}
+    frontier = list(found)
+    for hop in range(1, MAX_HOPS + 1):
+        connected = [
+            chunk_id
+            for seed_id in frontier
+            for entity in found[seed_id].get("entities", [])
+            for chunk_id in adjacency.get(entity, [])
+        ]
+        fresh = [c for c in dict.fromkeys(connected) if c not in found]
+        if not fresh:
+            break
+        records = client.retrieve(collection_name=COLLECTION, ids=fresh,
+                                  with_payload=True)
+        for record in records:
+            found[str(record.id)] = {"id": str(record.id), "hop": hop,
+                                     **(record.payload or {})}
+        frontier = fresh
+    scored = [
+        {**c, "rerank_score": rerank(query, c.get("text", ""))}
+        for c in found.values()
+    ]
+    return sorted(scored, key=lambda c: -c["rerank_score"])[:top_k]

{fde_framework-0.1.3 → fde_framework-0.1.4}/pyproject.toml

@@ -2,7 +2,7 @@
 # Distribution name matches the repository and stays unique on any index;
 # the import package and the CLI command remain the short `fde`.
 name = "fde-framework"
-version = "0.1.3"
+version = "0.1.4"
 description = "A framework for Forward Deployed Engineers: from problem statement to a deployable AI project, every decision traced to a fact."
 readme = "README.md"
 license = {text = "Apache-2.0"}

{fde_framework-0.1.3 → fde_framework-0.1.4}/src/fde/emit.py

@@ -734,6 +734,15 @@ def _write_evals(architecture: Architecture, out: Path, pairs_path: Path | None)
         _HARNESS.format(metrics=json.dumps(metrics), judged=judged)
     )
 
+    if "retrieval" in architecture.decisions.decided():
+        # The retrieval layer measured alone: the embedding and index set a
+        # ceiling nothing downstream recovers, and an empty case file is a
+        # gap somebody can see rather than a measurement nobody took.
+        (evals / "retrieval.py").write_text(_RETRIEVAL_EVAL)
+        cases_path = evals / "retrieval_cases.jsonl"
+        if not cases_path.exists():
+            cases_path.write_text("")
+
     golden_count = len(suite.golden) if suite else 0
     (evals / "acceptance.md").write_text(_acceptance(architecture, golden_count))
 
@@ -864,6 +873,144 @@ def classify(expected, actual, context=None):
 '''
 
 
+_RETRIEVAL_EVAL = '''#!/usr/bin/env python3
+"""Measure the retrieval layer alone. Exits non-zero on an empty case set,
+an erroring retriever, or recall below the threshold -- so CI can gate on it.
+
+The embedding and index choices set a ceiling on everything downstream: no
+reranking, prompting or model upgrade recovers a document that was never
+retrieved. End-to-end scores blur that ceiling into "quality"; this number is
+the ceiling by itself. When a failing case's answer is missing from the
+evidence, this -- not the prompt -- is the layer to fix (ops/diagnosis.md
+walks the order).
+
+Cases live in retrieval_cases.jsonl beside this file, one JSON object per
+line:
+
+    {"id": "case-1", "query": "...", "relevant": ["doc-7", "doc-12"]}
+
+`relevant` lists the document ids a correct top-K must surface; recall@K is
+the share of them that did. Seed cases from the engagement's own golden
+queries -- the ones the eval owner would grade -- never from a benchmark.
+"""
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+
+HERE = Path(__file__).parent
+KS = (10, 50)
+
+
+def load_cases():
+    path = HERE / "retrieval_cases.jsonl"
+    if not path.exists():
+        return []
+    return [json.loads(line) for line in path.read_text().splitlines() if line.strip()]
+
+
+def resolve_retriever():
+    """The deployed retriever, whatever shape its realization took.
+
+    Returns a callable (query, k) -> ranked results. Wiring problems surface
+    as the retriever's own refusal, which is the correct failure: an eval
+    that silently skipped an unwired store would grade a system that cannot
+    retrieve as though it could.
+    """
+    from app.components import retrieval as mod
+
+    run = getattr(mod, "run", None)
+    if callable(run) and not isinstance(run, type):
+        return lambda query, k: run(query, top_k=k)
+    for name in sorted(dir(mod)):
+        obj = getattr(mod, name)
+        if isinstance(obj, type):
+            if callable(getattr(obj, "retrieve", None)):
+                return lambda query, k, _cls=obj: _cls().retrieve(query, k)
+            if callable(getattr(obj, "run", None)):
+                return lambda query, k, _cls=obj: _cls().run({"query": query, "k": k})
+    raise SystemExit(
+        "no retriever found in app/components/retrieval.py -- expected a "
+        "module-level run(query, top_k=...) or a class with retrieve() or run()"
+    )
+
+
+def surfaced_ids(result):
+    if not isinstance(result, list):
+        return []
+    return [str(r.get("id")) for r in result if isinstance(r, dict) and "id" in r]
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--min-recall", type=float, default=0.0,
+                        help="fail below this mean recall@%d" % KS[0])
+    args = parser.parse_args()
+
+    cases = load_cases()
+    if not cases:
+        print("retrieval_cases.jsonl is empty -- nothing was measured, so "
+              "nothing passed. Seed it with golden queries and the document "
+              "ids a correct top-K must surface.", file=sys.stderr)
+        return 1
+
+    retrieve = resolve_retriever()
+    errors = 0
+    recalls = {k: [] for k in KS}
+    misses = []
+    for case in cases:
+        relevant = [str(r) for r in case.get("relevant", [])]
+        if not relevant:
+            errors += 1
+            misses.append({"id": case.get("id"), "note": "case lists no relevant ids"})
+            continue
+        try:
+            got = surfaced_ids(retrieve(case["query"], max(KS)))
+        except Exception as exc:  # noqa: BLE001
+            errors += 1
+            misses.append({"id": case.get("id"), "note": f"retriever errored: {exc}"})
+            continue
+        for k in KS:
+            top = set(got[:k])
+            recalls[k].append(sum(1 for r in relevant if r in top) / len(relevant))
+        absent = [r for r in relevant if r not in set(got[: max(KS)])]
+        if absent:
+            misses.append({"id": case.get("id"), "missing": absent})
+
+    for k in KS:
+        scored = recalls[k]
+        mean = sum(scored) / len(scored) if scored else 0.0
+        print(f"  recall@{k:<3} {len(scored):>4} cases  {mean:.1%}")
+    for miss in misses[:10]:
+        print(f"    {miss}", file=sys.stderr)
+
+    if errors:
+        print(f"{errors} case(s) errored -- the retriever is not wired end "
+              f"to end yet", file=sys.stderr)
+        return 1
+    gate = recalls[KS[0]]
+    mean = sum(gate) / len(gate) if gate else 0.0
+    if mean <= 0:
+        print("nothing relevant surfaced for any case. Look at the index and "
+              "the embedding model before anything downstream.",
+              file=sys.stderr)
+        return 1
+    if mean < args.min_recall:
+        print(f"recall@{KS[0]} below {args.min_recall:.1%} -- fix retrieval "
+              f"before touching prompts or models: nothing downstream "
+              f"recovers a document that never surfaced", file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
+'''
+
+
 _HARNESS = '''#!/usr/bin/env python3
 """Run the evaluation. Exits non-zero below the threshold, so CI can gate on it.