boost-skill-cli 1.0.317__tar.gz → 1.0.319__tar.gz

{boost_skill_cli-1.0.317 → boost_skill_cli-1.0.319}/CLAUDE.md

@@ -47,20 +47,33 @@ It is five gates and **all must pass**:
 | Gate       | Command                                        | Threshold |
 |------------|------------------------------------------------|-----------|
 | `lint`     | `ruff check boost_cli tests` + `mypy`          | zero errors |
-| `eval`     | `ensure_eval_corpus.sh` + `eval_retrieval.py --build -k 10 --fail-under 0.85` | BM25 recall@k **≥ 0.85** over the pinned corpus |
+| `eval`     | `ensure_eval_corpus.sh` + `eval_retrieval.py --build -k 10` with four floors | BM25 recall@k **≥ 0.85**, hit@1 ≥ 0.65, MRR ≥ 0.74, nDCG@k ≥ 0.78 |
 | `test`     | `pytest tests/unit tests/functional --cov`     | **80%** coverage (`fail_under = 80`) |
 | `smoke`    | `bash tests/smoke.sh`                           | 0 failed |
 | `mutation` | `python3 scripts/mutation_gate.py --run --min 80` | **80%** of `boost_cli/core` mutants killed |
 
 The `eval` gate is the Tier 1 retrieval-quality check: it runs boost's RAG
-engines over the golden set (`tests/eval/golden.jsonl`) and floors BM25
-recall@k. The golden set grades real catalog items **by name**, so it needs a
+engines over the golden set (`tests/eval/golden.jsonl`) and floors **four**
+metrics on BM25 — recall@k, hit@1, MRR and nDCG@k. Flooring recall alone was a
+hole rather than a simplification: a ranker that finds the right answer every
+time and never ranks it first scores recall@10 1.000 with hit@1 0.000, and
+passed. The golden set grades real catalog items **by name**, so it needs a
 corpus: `scripts/ensure_eval_corpus.sh` first taps the pinned repo list in
 `tests/eval/taps.txt` (the minimal set covering all golden targets — `boost tap
---defaults` is NOT enough, it omits every rule/workflow repo). Over that corpus
-BM25 recall is 1.000, so the 0.85 floor has wide margin; regression-vs-baseline
-is relaxed in the gate (`--regression-eps 1`) so upstream repo drift can't flake
-it. Edit `taps.txt` → regenerate `tests/eval/baseline.json`. It runs in CI's
+--defaults` is NOT enough, it omits every rule/workflow repo). Measured over
+that corpus BM25 scores 1.000 / 0.780 / 0.860 / 0.895, and each floor sits about
+0.12 under its measured value — loose enough that upstream drift can't flake the
+build, tight enough to catch a collapse. Regression-vs-baseline stays relaxed
+(`--regression-eps 1`) because the corpus tracks upstream HEAD rather than
+pinned commits, so the absolute floors are the real gate.
+
+**Baselines are keyed by query set** (`name@content-digest`), so one file holds
+both `golden.jsonl` and `golden-natural.jsonl` without either overwriting the
+other. Before that, running the natural-language set printed eight confident
+"REGRESSION vs baseline" lines that were only the gap between two different
+question sets. Add `--floor NAME=VALUE` (repeatable) to gate any metric; a
+misspelled metric name is a hard error rather than a silently skipped floor.
+Edit `taps.txt` → regenerate `tests/eval/baseline.json`. It runs in CI's
 `lint` job (pure-stdlib BM25, no `ANTHROPIC_API_KEY`; needs network to tap). The
 opt-in evals stay out of `check` and all degrade cleanly:
 

{boost_skill_cli-1.0.317 → boost_skill_cli-1.0.319}/Makefile

@@ -98,15 +98,24 @@ dist-check:
 	$(VENV)/bin/check-wheel-contents dist/*.whl
 	$(VENV)/bin/pyroma --min 8 dist/*.tar.gz
 
-# Tier 1 retrieval quality gate: golden-set recall@k over a pinned corpus
-# (tests/eval/taps.txt). Part of `check` and required in CI (the lint job).
-# Taps the pinned repos into a sandboxed $(EVAL_HOME) first (network on the
-# first run; a sentinel skips re-tapping after), then floors BM25 recall@k at
-# 0.85. Regression-vs-baseline is relaxed here (drift tolerance); the absolute
-# floor is the gate.
+# Tier 1 retrieval quality gate over a pinned corpus (tests/eval/taps.txt).
+# Part of `check` and required in CI (the lint job). Taps the pinned repos into
+# a sandboxed $(EVAL_HOME) first (network on the first run; a sentinel skips
+# re-tapping after), then floors FOUR metrics on BM25, not just recall.
+#
+# recall@k alone could not fail this build for a ranker that found the right
+# answer every time and never ranked it first — recall@10 1.000 with hit@1
+# 0.000 passed. Measured on this corpus BM25 scores 1.000 / 0.780 / 0.860 /
+# 0.895, so each floor sits ~0.12 below its measured value: loose enough that
+# upstream repo drift cannot flake the build, tight enough that a collapse
+# fails it. Regression-vs-baseline stays relaxed (--regression-eps 1) because
+# the corpus tracks upstream HEAD rather than pinned commits; the absolute
+# floors are the real gate, and they are drift-tolerant by construction.
 eval:
 	PYTHON=$(PY) BOOST_HOME=$(EVAL_HOME) bash scripts/ensure_eval_corpus.sh
-	BOOST_HOME=$(EVAL_HOME) $(PY) scripts/eval_retrieval.py --build -k 10 --fail-under 0.85 --regression-eps 1
+	BOOST_HOME=$(EVAL_HOME) $(PY) scripts/eval_retrieval.py --build -k 10 \
+	  --fail-under 0.85 --floor hit@1=0.65 --floor MRR=0.74 --floor nDCG@k=0.78 \
+	  --regression-eps 1
 
 # Tier 2a: LLM rerank lift over BM25 on the same golden set. Opt-in and
 # key-gated — needs the `claude` CLI on PATH or ANTHROPIC_API_KEY; skips

{boost_skill_cli-1.0.317/boost_skill_cli.egg-info → boost_skill_cli-1.0.319}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: boost-skill-cli
-Version: 1.0.317
+Version: 1.0.319
 Summary: boost — Homebrew for AI coding skills
 Author: Jonathan Reyes
 License: GNU GENERAL PUBLIC LICENSE

{boost_skill_cli-1.0.317 → boost_skill_cli-1.0.319}/boost_cli/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '1.0.317'
-__version_tuple__ = version_tuple = (1, 0, 317)
+__version__ = version = '1.0.319'
+__version_tuple__ = version_tuple = (1, 0, 319)
 
 __commit_id__ = commit_id = None

{boost_skill_cli-1.0.317 → boost_skill_cli-1.0.319}/boost_cli/core/dense.py

@@ -18,9 +18,9 @@ from pathlib import Path
 from typing import Dict, List, Optional, Tuple
 
 from . import catalog, embed, paths
-from .rag import Hit, chunk, read_body
+from .rag import Hit, chunk, entry_key, read_body
 
-INDEX_VERSION = 1
+INDEX_VERSION = 2
 _BATCH = 128            # texts per embedding request
 _POOL = 8              # chunk over-fetch factor for KNN before per-entry reduce
 
@@ -64,7 +64,7 @@ def _connect() -> Optional[sqlite3.Connection]:
 def _ensure_schema(con: sqlite3.Connection, dim: int) -> None:
     con.execute(
         "CREATE TABLE IF NOT EXISTS chunks (id INTEGER PRIMARY KEY AUTOINCREMENT,"
-        " name TEXT, tap TEXT, kind TEXT, cix INTEGER, snip TEXT)")
+        " name TEXT, tap TEXT, path TEXT, kind TEXT, cix INTEGER, snip TEXT)")
     con.execute("CREATE INDEX IF NOT EXISTS chunks_tap ON chunks(tap)")
     con.execute("CREATE TABLE IF NOT EXISTS meta (k TEXT PRIMARY KEY, v TEXT)")
     con.execute(
@@ -283,7 +283,14 @@ def build(entries: Optional[List[dict]] = None,
         return None
     try:
         meta = _read_meta(con)
-        same_backend = (meta.get("provider") == prov
+        # The stored INDEX_VERSION has to be part of this. `_ensure_schema` uses
+        # CREATE TABLE IF NOT EXISTS, so it cannot add a column to a store built
+        # by an older boost — leaving the build to fail on the first INSERT with
+        # "table chunks has no column named path". `status()` already treated a
+        # version change as a reason to rebuild; only the build path did not.
+        same_version = meta.get("version") == INDEX_VERSION
+        same_backend = (same_version
+                        and meta.get("provider") == prov
                         and meta.get("model") == mdl
                         and meta.get("dim") == dim)
         if force or not same_backend:
@@ -323,7 +330,7 @@ def build(entries: Optional[List[dict]] = None,
         con.commit()
         total = con.execute("SELECT COUNT(*) FROM chunks").fetchone()[0]
         return {
-            "entries": len({(e["name"], e["tap"]) for e in entries}),
+            "entries": len({entry_key(e) for e in entries}),
             "chunks": total,
             "added": added,
             "taps": len(commits),
@@ -384,10 +391,10 @@ def _embed_and_store(con: sqlite3.Connection, entries: List[dict],
             continue
         for (e, ci, text), vec in zip(batch, vecs):
             cur = con.execute(
-                "INSERT INTO chunks (name, tap, kind, cix, snip) "
-                "VALUES (?, ?, ?, ?, ?)",
-                (e["name"], e["tap"], e.get("kind", "skill"), ci,
-                 text[:200].strip()))
+                "INSERT INTO chunks (name, tap, path, kind, cix, snip) "
+                "VALUES (?, ?, ?, ?, ?, ?)",
+                (e["name"], e["tap"], e["skill_md"],
+                 e.get("kind", "skill"), ci, text[:200].strip()))
             con.execute("INSERT INTO vec_chunks (rowid, embedding) VALUES (?, ?)",
                         (cur.lastrowid, mod.serialize_float32(vec)))
             added += 1
@@ -423,31 +430,33 @@ def retrieve(query: str, k: int = 60, kind: Optional[str] = None,
         if not knn:
             return []
         by_id = {r[0]: (r[1], r[2], r[3], r[4]) for r in con.execute(
-            "SELECT id, name, tap, kind, snip FROM chunks WHERE id IN (%s)"  # noqa: S608  interpolates only `?` placeholders; ids are bound params
+            "SELECT id, tap, path, kind, snip FROM chunks WHERE id IN (%s)"  # noqa: S608  interpolates only `?` placeholders; ids are bound params
             % ",".join("?" * len(knn)), [rid for rid, _d in knn])}
     finally:
         con.close()
 
     entries = catalog.all_entries() if entries is None else entries
-    live = {(e["name"], e["tap"]): e for e in entries}
+    live = {entry_key(e): e for e in entries}
     best: Dict[Tuple[str, str], Tuple[float, str]] = {}
     for rid, dist in knn:
         meta = by_id.get(rid)
         if meta is None:
             continue
-        name, tap, ckind, snip = meta
+        tap, path, ckind, snip = meta
         if kind is not None and ckind != kind:
             continue
-        key = (name, tap)
+        key = (tap, path)
         if key not in live:
             continue
         score = 1.0 - dist                       # cosine distance -> similarity
         prev = best.get(key)
         if prev is None or score > prev[0]:
             best[key] = (score, snip)
-    ranked = sorted(best.items(), key=lambda kv: (-kv[1][0], kv[0][0]))
+    # Tie-break on the displayed name (see rag.retrieve for why).
+    ranked = sorted(best.items(),
+                    key=lambda kv: (-kv[1][0], live[kv[0]]["name"], kv[0]))
     hits: List[Hit] = [
-        {"entry": live[(name, tap)], "score": score,
+        {"entry": live[key], "score": score,
          "snippet": snip}  # type: ignore[typeddict-item]
-        for (name, tap), (score, snip) in ranked[:k]]
+        for key, (score, snip) in ranked[:k]]
     return hits

{boost_skill_cli-1.0.317 → boost_skill_cli-1.0.319}/boost_cli/core/rag.py

@@ -33,7 +33,7 @@ from . import ai, catalog, frontmatter, gitutil, paths, registry, util
 # v2: `snip` stores a larger head of the matched chunk (not just SNIP_WIDTH
 # chars) so `retrieve` can window it onto the query terms; bump forces a
 # one-time reindex.
-INDEX_VERSION = 2
+INDEX_VERSION = 3
 ENGINE = "bm25"
 
 # Chunking defaults (documented in docs/rag-architecture.md §4).
@@ -175,7 +175,8 @@ def _make_docs(entries: List[dict], tap_paths: Dict[str, Path]) -> List[dict]:
             if not tf:
                 continue
             docs.append({
-                "n": e["name"], "t": e["tap"], "k": e.get("kind", "skill"),
+                "n": e["name"], "t": e["tap"], "f": e["skill_md"],
+                "k": e.get("kind", "skill"),
                 "c": ci, "l": sum(tf.values()),
                 "snip": piece[:SNIP_STORE].strip(),  # windowed at retrieve time
                 "tf": dict(tf),  # noqa: FURB123  tf is a defaultdict; .copy() would keep the factory
@@ -226,7 +227,7 @@ def build(entries: Optional[List[dict]] = None, force: bool = False) -> dict:
     _save(docs, commits)
     reindexed = sorted({e["tap"] for e in fresh})
     return {
-        "entries": len({(e["name"], e["tap"]) for e in entries}),
+        "entries": len({entry_key(e) for e in entries}),
         "docs": len(docs),
         "taps": len(commits),
         "reindexed": reindexed,
@@ -242,7 +243,7 @@ def _save(docs: List[dict], commits: Dict[str, str]) -> None:
         for term, tf in d["tf"].items():
             postings[term].append([doc_id, tf])
         total_len += d["l"]
-        meta_docs.append({"n": d["n"], "t": d["t"], "k": d["k"],
+        meta_docs.append({"n": d["n"], "t": d["t"], "f": d["f"], "k": d["k"],
                           "c": d["c"], "l": d["l"], "snip": d["snip"]})
     payload = {
         "version": INDEX_VERSION,
@@ -365,6 +366,34 @@ def _passage(text: str, terms: Sequence[str], width: int = SNIP_WIDTH) -> str:
     return lead + snip + tail
 
 
+def entry_key(entry: dict) -> Tuple[str, str]:
+    """Identity of a catalog entry: the tap it came from and its file.
+
+    NOT ``(name, tap)``, which both engines used and which is not unique. On the
+    pinned 6-tap eval corpus 743 entries collapse to 694 such pairs; on a real
+    83-tap install the roadmap item measures 1,557 of 11,147 (14.0%) colliding,
+    worst case ``('survivorforge/cursor-rules', 'rule')`` x47. One tap can
+    legitimately ship two different files under one name —
+
+        docs/rules/backend/nodejs/express-mongodb/admin-interface-rule.mdc
+        docs/rules/backend/nodejs/fullstack-mern-guide/admin-interface-rule.mdc
+
+    — and keying on the name made the second shadow the first in a dict
+    comprehension, so a query matching the first's body was reported under the
+    second's name and description. ``skill_md`` is the tap-relative path, so
+    ``(tap, skill_md)`` is 743/743 and 11,147/11,147 distinct respectively.
+
+    Deliberately strict. An earlier draft fell back to the name when
+    ``skill_md`` was missing, on the theory that a malformed row should not
+    crash a search. It is the worse failure: the index side and the entry side
+    then degrade differently, the keys stop matching, and retrieval silently
+    returns *nothing* — which takes far longer to diagnose than a KeyError
+    naming the field. Every catalog entry has ``skill_md`` (743/743 on the eval
+    corpus), and INDEX_VERSION rejects any index built without it.
+    """
+    return (entry["tap"], entry["skill_md"])
+
+
 def retrieve(query: str, k: int = 60, kind: Optional[str] = None,
              entries: Optional[List[dict]] = None) -> List[Hit]:
     """Top-k full-content hits for ``query``. ``[]`` if the index is empty."""
@@ -375,24 +404,28 @@ def retrieve(query: str, k: int = 60, kind: Optional[str] = None,
     if not terms:
         return []
     entries = catalog.all_entries() if entries is None else entries
-    live = {(e["name"], e["tap"]): e for e in entries}
+    live = {entry_key(e): e for e in entries}
     docs = raw["docs"]
     best: Dict[Tuple[str, str], Tuple[float, str]] = {}
     for doc_id, score in _bm25(terms, raw).items():
         d = docs[doc_id]
         if kind is not None and d["k"] != kind:
             continue
-        key = (d["n"], d["t"])
+        key = (d["t"], d["f"])
         if key not in live:
             continue
         prev = best.get(key)
         if prev is None or score > prev[0]:
             best[key] = (score, d["snip"])
-    ranked = sorted(best.items(), key=lambda kv: (-kv[1][0], kv[0][0]))
+    # Tie-break on the displayed name, not the identity key: the key is now
+    # (tap, path), and ordering by it would silently reshuffle equal-scoring
+    # results relative to every previous release.
+    ranked = sorted(best.items(),
+                    key=lambda kv: (-kv[1][0], live[kv[0]]["name"], kv[0]))
     hits: List[Hit] = [
-        {"entry": live[(name, tap)], "score": score,
+        {"entry": live[key], "score": score,
          "snippet": _passage(snip, terms)}  # type: ignore[typeddict-item]
-        for (name, tap), (score, snip) in ranked[:k]]
+        for key, (score, snip) in ranked[:k]]
     return hits
 
 
@@ -435,9 +468,9 @@ def rrf_fuse(rankings: List[List[Hit]], limit: int = 60) -> List[Hit]:
     calibration step that has to be fitted and re-fitted per corpus; rank
     fusion sidesteps it entirely and is the respected zero-tuning default.
 
-    Entries are identified by ``(name, tap)`` — the same key both engines
-    already dedupe on — and the first ranking's hit object wins, so the caller
-    passes BM25 first and keeps its query-term-highlighted snippet for display.
+    Entries are identified by ``entry_key`` — the same key both engines dedupe
+    on — and the first ranking's hit object wins, so the caller passes BM25
+    first and keeps its query-term-highlighted snippet for display.
 
     The returned ``score`` is the fused rank score, not a BM25 score: it exists
     to order these hits against each other, and comparing it to a raw BM25
@@ -448,7 +481,7 @@ def rrf_fuse(rankings: List[List[Hit]], limit: int = 60) -> List[Hit]:
     for hits in rankings:
         for rank, hit in enumerate(hits, start=1):
             entry = hit["entry"]
-            key = (entry["name"], entry["tap"])
+            key = entry_key(entry)
             scores[key] = scores.get(key, 0.0) + 1.0 / (RRF_K + rank)
             chosen.setdefault(key, hit)
     ordered = sorted(scores, key=lambda key: (-scores[key], key[0], key[1]))

{boost_skill_cli-1.0.317 → boost_skill_cli-1.0.319/boost_skill_cli.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: boost-skill-cli
-Version: 1.0.317
+Version: 1.0.319
 Summary: boost — Homebrew for AI coding skills
 Author: Jonathan Reyes
 License: GNU GENERAL PUBLIC LICENSE

{boost_skill_cli-1.0.317 → boost_skill_cli-1.0.319}/boost_skill_cli.egg-info/SOURCES.txt

@@ -493,7 +493,9 @@ tests/unit/test_docsite_chrome.py
 tests/unit/test_ed25519.py
 tests/unit/test_embed.py
 tests/unit/test_embed_local.py
+tests/unit/test_entry_key.py
 tests/unit/test_errors_and_cli_table.py
+tests/unit/test_eval_baseline.py
 tests/unit/test_eval_faithfulness.py
 tests/unit/test_eval_gate.py
 tests/unit/test_eval_metrics.py

{boost_skill_cli-1.0.317 → boost_skill_cli-1.0.319}/docs/roadmap/items/eval-corpus-cannot-see-real-retrieval.md

@@ -2,14 +2,14 @@
 id: eval-corpus-cannot-see-real-retrieval
 board: code
 section: internals
-status: planned
+status: inflight
 category: Quality · Eval
 complexity: M
 impact: High
 wow: 4
 note: gate reports 1.000 while users get 0.720
 order: 70
-owner:
+owner: loop/eval-corpus-falsifiable
 pr:
 title: The eval gate reports a perfect score on a corpus 4× smaller than a real user's
 ---
@@ -57,6 +57,35 @@ different axes, same conclusion: the number the gate reports is not a number abo
 What it does <b>not</b> fix is the corpus-size half, which is this card's own finding and remains
 open &mdash; 50 queries over 6 taps cannot speak to an 83-tap install. The two are complementary
 rather than overlapping, and combining them (natural-language queries over a realistically sized
-corpus) is the instrument this group actually wants. Also unaddressed here: the kind oracle, the
-recall-only floor and <code>--regression-eps 1</code> are all still exactly as described above.
+corpus) is the instrument this group actually wants.
+
+<b>Two of the three blindnesses are now closed, and the third was never true of the committed
+harness.</b> <b>(1) recall-only flooring</b> is fixed: <code>--floor NAME=VALUE</code> is repeatable
+and gates any metric, and <code>make eval</code> now floors all four &mdash; measured BM25 on the
+pinned corpus is 1.000 / 0.780 / 0.860 / 0.895, with each floor about 0.12 under its measured value
+so upstream drift cannot flake the build. A misspelled metric name is a hard error rather than a
+silently skipped floor, since a floor that never fires is worse than no floor.
+
+<b>(3) the kind oracle does not exist in <code>scripts/eval_retrieval.py</code>.</b> Checked
+directly: every ranker &mdash; <code>catalog_ranker</code>, <code>bm25_ranker</code>,
+<code>dense_ranker</code>, <code>hybrid_ranker</code> &mdash; is called with the query alone, and
+there is no <code>kind=</code> argument anywhere in <code>scripts/</code> or <code>evals/</code>,
+before or after <code>#360</code>. The golden set's <code>kind</code> field feeds the per-kind
+<em>reporting</em> slices in <code>_aggregate</code> and nothing else, so the +0.073 figure quoted
+above cannot have come from this harness. Recording it here rather than silently deleting the claim,
+because the number was real in whatever script measured it.
+
+<b>A defect the card did not predict:</b> the baseline was not keyed to the query set that produced
+it. Running the natural-language set printed eight confident <code>REGRESSION vs baseline</code>
+lines &mdash; BM25 recall 1.000&rarr;0.690, hit@1 0.780&rarr;0.240 &mdash; which were not
+regressions but the gap between two different question sets, and <code>--save-baseline</code> on the
+natural set would have silently overwritten the keyword set's numbers with them. Baselines are now
+keyed by <code>name@content-digest</code>, so editing a query in place invalidates its baseline
+rather than quietly re-grading against numbers that no longer describe it.
+
+<b>Still open:</b> the corpus-size half (23 taps versus a real 83) and
+<code>--regression-eps 1</code>, which is a symptom of the corpus tracking upstream HEAD rather than
+pinned commits. Pinning <code>taps.txt</code> by commit SHA is the change that would make
+regression-vs-baseline meaningful again; the absolute floors added here are drift-tolerant by
+construction and do not depend on it.
 

{boost_skill_cli-1.0.317 → boost_skill_cli-1.0.319}/docs/roadmap/items/near-duplicate-items-eat-the-result-slots.md

@@ -2,14 +2,14 @@
 id: near-duplicate-items-eat-the-result-slots
 board: code
 section: pipeline
-status: planned
+status: inflight
 category: Search · Ranking
 complexity: M
 impact: High
 wow: 4
 note: 56.3% of entries share a name
 order: 71
-owner:
+owner: loop/entry-key-collision
 pr:
 title: Near-duplicate items consume the top-10, and it gets worse with every tap
 ---
@@ -61,3 +61,33 @@ the fix at deduplication before ranking rather than at the fusion rule.
 Worth re-measuring on the 83-tap install with the same script (<code>rrf_fuse</code> is public) to
 turn this bound into a curve.
 
+
+<b>The key half is shipped; the ranking half is not.</b> This card names two things to get right,
+and the second &mdash; <code>(tap, name)</code> is not a unique key &mdash; turned out to be a
+correctness bug rather than a ranking one, so it was fixed first and separately.
+
+Both engines built <code>live = {(name, tap): entry}</code> as a dict comprehension, so the last
+entry silently won. Reproduced on the pinned 6-tap corpus: 743 entries collapse to 694 pairs, leaving
+<b>49 entries (6.6%) unreachable</b> &mdash; the card's 14.0% at 83 taps, at small scale. It is not
+only that a copy is hidden: a query matching a shadowed entry's <em>body</em> was reported under the
+surviving entry's name, description and path. The clearest pair, same tap and same name, is two
+genuinely different rules:
+<code>docs/rules/backend/nodejs/express-mongodb/admin-interface-rule.mdc</code> and
+<code>docs/rules/backend/nodejs/fullstack-mern-guide/admin-interface-rule.mdc</code>.
+
+<code>rag.entry_key</code> is now the single identity function shared by BM25, dense and RRF, keyed
+on <code>(tap, skill_md)</code> &mdash; 743/743 distinct here and 11,147/11,147 on the 83-tap
+install, exactly as this card predicted. Both index versions are bumped, and <code>dense.build</code>
+now wipes on a version change: it compared only provider/model/dim, and since
+<code>_ensure_schema</code> uses <code>CREATE TABLE IF NOT EXISTS</code> an existing store would have
+crashed on the first insert with &ldquo;no column named path&rdquo;.
+
+Measured cost: <b>none</b>. Over the pinned corpus BM25 scores 1.000 / 0.780 / 0.860 / 0.895 both
+before and after, so recovering 49 entries did not disturb ranking. Several fixtures were modelling
+data a real catalog cannot produce &mdash; every entry sharing <code>skill_md="s"</code>, two skills
+at one path &mdash; and were corrected.
+
+<b>Still open:</b> the ranking half. Clustering on content hash, picking a winner with a quality
+prior, and the 83-tap re-measurement of the fusion bound are all untouched. Note the two are
+independent: dedup <em>merges</em> copies that are genuinely the same, whereas this fix stops copies
+that were never the same from being merged by accident.

{boost_skill_cli-1.0.317 → boost_skill_cli-1.0.319}/docs/roadmap.html

@@ -1707,7 +1707,7 @@ a one-line <code>logger.debug(...)</code> in each except branch.</p>
       </article>
 
       <article class="cap rcard" id="eval-corpus-cannot-see-real-retrieval">
-        <div class="head"><span class="pill planned">Planned</span><span class="cat">Quality · Eval</span></div>
+        <div class="head"><span class="pill inflight">In flight</span><span class="cat">Quality · Eval</span></div>
         <h3>The eval gate reports a perfect score on a corpus 4× smaller than a real user's</h3>
         <p><code>make eval</code> reports BM25 <code>recall@10 = 1.000</code> against the
 pinned 23-tap corpus in <code>tests/eval/taps.txt</code>. Run the same script
@@ -1753,8 +1753,37 @@ different axes, same conclusion: the number the gate reports is not a number abo
 What it does <b>not</b> fix is the corpus-size half, which is this card's own finding and remains
 open &mdash; 50 queries over 6 taps cannot speak to an 83-tap install. The two are complementary
 rather than overlapping, and combining them (natural-language queries over a realistically sized
-corpus) is the instrument this group actually wants. Also unaddressed here: the kind oracle, the
-recall-only floor and <code>--regression-eps 1</code> are all still exactly as described above.</p>
+corpus) is the instrument this group actually wants.
+
+<b>Two of the three blindnesses are now closed, and the third was never true of the committed
+harness.</b> <b>(1) recall-only flooring</b> is fixed: <code>--floor NAME=VALUE</code> is repeatable
+and gates any metric, and <code>make eval</code> now floors all four &mdash; measured BM25 on the
+pinned corpus is 1.000 / 0.780 / 0.860 / 0.895, with each floor about 0.12 under its measured value
+so upstream drift cannot flake the build. A misspelled metric name is a hard error rather than a
+silently skipped floor, since a floor that never fires is worse than no floor.
+
+<b>(3) the kind oracle does not exist in <code>scripts/eval_retrieval.py</code>.</b> Checked
+directly: every ranker &mdash; <code>catalog_ranker</code>, <code>bm25_ranker</code>,
+<code>dense_ranker</code>, <code>hybrid_ranker</code> &mdash; is called with the query alone, and
+there is no <code>kind=</code> argument anywhere in <code>scripts/</code> or <code>evals/</code>,
+before or after <code>#360</code>. The golden set's <code>kind</code> field feeds the per-kind
+<em>reporting</em> slices in <code>_aggregate</code> and nothing else, so the +0.073 figure quoted
+above cannot have come from this harness. Recording it here rather than silently deleting the claim,
+because the number was real in whatever script measured it.
+
+<b>A defect the card did not predict:</b> the baseline was not keyed to the query set that produced
+it. Running the natural-language set printed eight confident <code>REGRESSION vs baseline</code>
+lines &mdash; BM25 recall 1.000&rarr;0.690, hit@1 0.780&rarr;0.240 &mdash; which were not
+regressions but the gap between two different question sets, and <code>--save-baseline</code> on the
+natural set would have silently overwritten the keyword set's numbers with them. Baselines are now
+keyed by <code>name@content-digest</code>, so editing a query in place invalidates its baseline
+rather than quietly re-grading against numbers that no longer describe it.
+
+<b>Still open:</b> the corpus-size half (23 taps versus a real 83) and
+<code>--regression-eps 1</code>, which is a symptom of the corpus tracking upstream HEAD rather than
+pinned commits. Pinning <code>taps.txt</code> by commit SHA is the change that would make
+regression-vs-baseline meaningful again; the absolute floors added here are drift-tolerant by
+construction and do not depend on it.</p>
         <div class="meta">
           <span class="m">Complexity <b>M</b></span>
           <span class="m hi">Impact <b>High</b></span>
@@ -3318,7 +3347,7 @@ deliberate, and idempotence was the piece missing from it.</p>
       </article>
 
       <article class="cap rcard" id="near-duplicate-items-eat-the-result-slots">
-        <div class="head"><span class="pill planned">Planned</span><span class="cat">Search · Ranking</span></div>
+        <div class="head"><span class="pill inflight">In flight</span><span class="cat">Search · Ranking</span></div>
         <h3>Near-duplicate items consume the top-10, and it gets worse with every tap</h3>
         <p>Measured on a real 83-tap install: 11,147 catalog entries carry only 6,997
 unique names — <b>1.593 copies per name, and 56.3% of entries share a name with
@@ -3366,7 +3395,38 @@ multiplies them. If the amplification appears at 83 taps it will be because both
 the fix at deduplication before ranking rather than at the fusion rule.
 
 Worth re-measuring on the 83-tap install with the same script (<code>rrf_fuse</code> is public) to
-turn this bound into a curve.</p>
+turn this bound into a curve.
+
+
+<b>The key half is shipped; the ranking half is not.</b> This card names two things to get right,
+and the second &mdash; <code>(tap, name)</code> is not a unique key &mdash; turned out to be a
+correctness bug rather than a ranking one, so it was fixed first and separately.
+
+Both engines built <code>live = {(name, tap): entry}</code> as a dict comprehension, so the last
+entry silently won. Reproduced on the pinned 6-tap corpus: 743 entries collapse to 694 pairs, leaving
+<b>49 entries (6.6%) unreachable</b> &mdash; the card's 14.0% at 83 taps, at small scale. It is not
+only that a copy is hidden: a query matching a shadowed entry's <em>body</em> was reported under the
+surviving entry's name, description and path. The clearest pair, same tap and same name, is two
+genuinely different rules:
+<code>docs/rules/backend/nodejs/express-mongodb/admin-interface-rule.mdc</code> and
+<code>docs/rules/backend/nodejs/fullstack-mern-guide/admin-interface-rule.mdc</code>.
+
+<code>rag.entry_key</code> is now the single identity function shared by BM25, dense and RRF, keyed
+on <code>(tap, skill_md)</code> &mdash; 743/743 distinct here and 11,147/11,147 on the 83-tap
+install, exactly as this card predicted. Both index versions are bumped, and <code>dense.build</code>
+now wipes on a version change: it compared only provider/model/dim, and since
+<code>_ensure_schema</code> uses <code>CREATE TABLE IF NOT EXISTS</code> an existing store would have
+crashed on the first insert with &ldquo;no column named path&rdquo;.
+
+Measured cost: <b>none</b>. Over the pinned corpus BM25 scores 1.000 / 0.780 / 0.860 / 0.895 both
+before and after, so recovering 49 entries did not disturb ranking. Several fixtures were modelling
+data a real catalog cannot produce &mdash; every entry sharing <code>skill_md="s"</code>, two skills
+at one path &mdash; and were corrected.
+
+<b>Still open:</b> the ranking half. Clustering on content hash, picking a winner with a quality
+prior, and the 83-tap re-measurement of the fusion bound are all untouched. Note the two are
+independent: dedup <em>merges</em> copies that are genuinely the same, whereas this fix stops copies
+that were never the same from being merged by accident.</p>
         <div class="meta">
           <span class="m">Complexity <b>M</b></span>
           <span class="m hi">Impact <b>High</b></span>