cribsheet 0.2.2__tar.gz → 0.3.0__tar.gz

cribsheet-0.3.0/.claude-plugin/marketplace.json

@@ -0,0 +1,15 @@
+{
+  "name": "cribsheet",
+  "description": "crib — persistent cross-session markdown memory + a code symbol index for your AI. Ships the `/crib` recall command and registers the crib MCP server via sharedserver; set MCP_COMBINER=1 and it stands down when a combiner already serves crib.",
+  "owner": {
+    "name": "George Harker",
+    "email": "george@georgeharker.com"
+  },
+  "plugins": [
+    {
+      "name": "cribsheet",
+      "source": "./plugins/cribsheet",
+      "description": "Persistent markdown memory + a code symbol index (concept/name lookup, call graph, references, learnings) for your AI. Registers the crib MCP server via sharedserver and keeps one warm crib shared across sessions, carries the reach-for-crib instructions, and ships the `/crib` command. Set MCP_COMBINER=1 (or MCP_COMBINER_SERVES_CRIBSHEET) when a combiner already serves crib and it will not register a second copy."
+    }
+  ]
+}

{cribsheet-0.2.2 → cribsheet-0.3.0}/DESIGN.md

@@ -414,53 +414,88 @@ note markdown through llmkit's rich pipeline (`llmkit.md.render`, the extracted
 successor to the old vendored `crib/render`, carried as the `vendor/llmkit` git
 submodule) when stdout is a tty, and falls back to raw bytes when piped.
 
-### 10.3 Retrieval — hybrid dense ⊕ BM25, fused by RRF
+### 10.3 Retrieval — hybrid dense ⊕ BM25, DENSE-DOMINANT SCORE fusion
 
-![Note & doc retrieval — three recall signals fused by RRF, optional rerank](docs/images/note-retrieval-pipeline.png)
+![Note & doc retrieval — recall signals blended dense-dominant, then reranked](docs/images/note-retrieval-pipeline.png)
 
 <sub>Source: [`note-retrieval-pipeline.svg`](docs/images/note-retrieval-pipeline.svg).</sub>
 
 Dense (vector) retrieval nails paraphrase but underweights *exact terms*, so a
 terse keyword query ("restart server") can rank vaguely-on-topic prose above the
-section that literally documents the command. Measured on real imported docs:
-dense-only put an unrelated "Step 6: Verify" section at rank 1 and the actual
-`:MCPRestartServer` command table at rank 3; hybrid promoted the command table to
-rank 1 and dropped the false positive to rank 5.
-
-**Fusion = Reciprocal Rank Fusion** (`crib/retrieve.py`). Dense cosine (~0–1) and
-BM25 (unbounded, corpus-dependent) are not on comparable scales, so we fuse by
-*rank*, not score: `score(d) = Σ 1/(K + rank_d)` across the two ranked lists,
-`K = rrf_k` (canonical 60). Scale-free, parameter-light, and order-only — the
-alternative (normalize each list then weight-sum) needs per-query normalization
-and a tuned α, which RRF sidesteps.
-
-- **Dense list**: the existing vector query, widened to `max(k*3, 30)` candidates
-  so BM25 can promote items dense ranked low.
-- **Sparse list**: an in-process Okapi BM25 over the project's chunk documents
-  (`store.get_docs`), held in a per-project `LexicalCache` owned by the
-  `IndexEngine` — built lazily, kept warm for the daemon's life, and dropped
-  (dirty-flag) whenever the single write path mutates that project, so the next
-  query rebuilds. BM25 is a ranking aid over the authoritative vector store, so
-  brief staleness only delays a just-written chunk's lexical findability by one
-  query. One-shot callers (`--no-daemon`, tests) build a fresh Crib each run, so
-  the cache is simply cold — no benefit, no harm. Incremental add/remove (vs full
-  rebuild) is the next step once corpora are large enough to feel it; native
-  sparse vectors in Chroma's hybrid `search` (experimental, distributed/hosted
-  only in 1.5.x) are the eventual "BM25 in the store" endgame.
-- The fused order drives ranking, but each hit still shows its **cosine** (a
-  BM25-only finalist's cosine is filled by re-embedding just that handful), so the
-  displayed score is intentionally non-monotonic down the list — the visible
-  fingerprint of fusion.
-
-Config `[retrieve]`: `hybrid` (default on) and `rrf_k`. `hybrid = false` restores
-pure dense. No reindex needed — this is read-path only.
-
-### 10.4 Reranking — options, evidence, and open design
-
-An optional precision stage after fusion: re-score the top `rerank_top_n`
-candidates and blend that order back in. Off by default (`[retrieve].rerank`).
-
-**Fuse, don't replace.** The reranker's order is RRF-fused with the existing
+section that literally documents the command. Hybrid dense⊕BM25 fixes that; the
+open question was always *how to combine* two signals on different scales.
+
+**Fusion = dense-dominant score blend** (was RRF). Dense cosine is already a
+calibrated [0,1] similarity; BM25 is unbounded and corpus-dependent. So we
+normalize **only the sparse side** (min-max over the union candidate pool) and add
+it to the **raw** cosine — *dense-dominant*, so a confident dense match is never
+diluted:  `score(d) = cos(d) + w · minmax(sparse(d))`. This **replaced RRF**: RRF is
+rank-only, discarding the dense ranking's *magnitude*, so a diffuse sparse list
+drowns a confident dense hit. Measured — swapping RRF→score-fusion **+ a matched
+reranker (§10.4)** lifted `code_lookup` MRR **+30%** (4-domain A/B, n=500); RRF
+underperformed in every code slice. On the **notes** path the redesign is
+**precision-neutral, recall-positive** (re-measured 2026-07-13 on the live daemon:
+MRR 0.710 ≈ the old pipeline's 0.714; rerank's isolated lift +0.012 MRR /
++0.033 recall@3; the recall@3 bar clears, and the MRR bar was re-baselined
+0.72 → 0.70 to track corpus drift — see `scripts/eval_retrieval.py`). `rrf_k`
+survives only for the code path's cross-project fan-out, which is still rank-fused.
+
+**Two paths, same ranking, different field construction** — a divergence that's
+*measured*, not accidental:
+
+- **Code** (`crib/codequery.py::_lookup_one`): 2 signals — dense over the LLM
+  *description* ⊕ a **coverage-GATED** BM25 over an *expanded field* (identifier
+  subtokens ⊕ synth keyword phrases, generated with the description in one pass).
+  The gate — `cov = |Q ∩ field|/|Q|`, `gated = cov·bm25` — is a precision filter for
+  **terse** fields: a short identifier field lets one rare-token BM25 hit dominate
+  (IDF), and coverage demotes a match that only shares a stray token.
+- **Notes** (`crib/app.py::_retrieve`): up to 3 signals — dense over the section
+  *body* ⊕ BM25 over `body ⊕ heading ⊕ keyword_index` ⊕ (optional) summary_index
+  aliases as **multi-vector dense**: each alias embedding points at its section, and
+  the section's dense score is `max(body cos, alias cos × summary_weight)` — an
+  alias match IS a dense match, never a rank bonus (the rank-bonus port of the RRF
+  era measured strictly negative; the max-merge measured **+0.013 MRR / +0.010
+  recall@3 over the shipped stack on the n=1876 harvested gold set**, positive in
+  both query segments — `scripts/gen_notes_gold.py`, 2026-07-13).
+  **No coverage gate, body stays in the BM25 field.** Why diverge: a note
+  body is long and informative (unlike a terse name), so distilling it away to a
+  code-style keyword-only field *loses* the exact-term lexical signal the embedding
+  blurs (smoke-tested: code-style field construction regresses notes MRR
+  0.690→0.653). The gate itself was then measured properly (n=1876, 2026-07-13,
+  `[retrieve].keyword_coverage_gate`): **exactly neutral overall and a pure segment
+  trade** — exact-phrase queries +0.021 MRR, paraphrase queries −0.019 — so notes
+  keep it off (paraphrase is their binding constraint) while code keeps it on
+  (terse identifier fields are where it wins). The shared *ranking* transfers; the
+  *field construction and its gate* must not.
+- **Sparse source**: an in-process Okapi BM25 in a per-project `LexicalCache`
+  (`crib/retrieve.py`), built lazily, warm for the daemon's life, dropped on write
+  so the next query rebuilds. Code's equivalent is the resident symbol index's BM25.
+- **Missing-cosine fill**: code has a cosine for every symbol (resident dense over
+  all); notes' dense is a top-k vector query, so a *sparse-only* candidate has its
+  cosine back-filled (re-embed that handful) to give it a real dense base for the
+  blend — cosmetic under RRF, load-bearing under score fusion.
+
+Config `[retrieve]`: `hybrid` (default on), `rrf_k` (cross-project fan-out only).
+`hybrid = false` restores pure dense. Read-path only — no reindex needed.
+
+### 10.4 Reranking — range-matched, ON by default
+
+A cross-encoder precision stage over the top `rerank_top_n` fused candidates,
+folded into the score as a **range-matched** term — `minmax(fusion_score) +
+minmax(rerank_score)`, re-ordered. **On by default** (`[retrieve].rerank`), degrading
+gracefully to the fused order if the model is unavailable. It is the single biggest
+lever (code: +0.09 MRR of the +0.16 total; notes: moves a below-bar state above it).
+
+**Match the rerank to the fusion.** The reranker blends by *score* (range-matched)
+because the fusion is score-based — not by RRF-of-orders (the old approach), which
+discards the fusion magnitude and lets a noisier reranker reorder freely. Measured:
+RRF-rerank **regressed** on svg-mcp while range-matched held; and on notes, RRF-rerank
+suited the old RRF fusion but range-matched suits the new score fusion. **Principle:
+rank-based fusion → RRF-rerank; score-based fusion → range-matched.**
+
+<details><summary>Historical note (pre-score-fusion reranking evidence)</summary>
+
+**Fuse, don't replace.** The reranker's order was RRF-fused with the existing
 dense⊕BM25 order rather than overriding it — so it acts as a *third voter* that
 can promote a better match but can't break a strong hybrid result on one bad
 judgment. Measured: full-replace was a wash (2 wins / 2 regressions on the
@@ -483,6 +518,8 @@ for a human reading the top hit, or any path that feeds only rank-1, the MRR lif
 from the target, e.g. credentials≠tokens) stay at rank-2 under any *small* model —
 those are what a stronger LLM judge is meant to crack (unproven as of writing).
 
+</details>
+
 **Backend options (a tier ladder, picked by host).** `build_reranker` dispatches
 by model name:
 
@@ -504,10 +541,14 @@ judge call (zsh-ai's own calls are streaming chat, no logprobs). (3) OpenAI-
 compatible covers Mac-local llama.cpp *and* Pi-remote with the fast logprob path,
 so it's the lead; Anthropic-as-judge is a secondary text-score adapter.
 
-**Open: machine-aware selection.** Default rerank `off` on plain CPU, `fast` on an
-accelerated host, `llm-judge@localhost` when a local OpenAI-compatible endpoint is
-reachable — probing via `_auto_device` (embed.py) + endpoint reachability. Not yet
-built.
+**Open: machine-aware selection.** Rerank now defaults ON (the ONNX MiniLM stage —
+~0.5s/query cold CPU, cheap warm; the measured net win). The open idea is picking the
+*stage* by host: MiniLM on plain CPU (a Pi may still want `rerank = false`), a heavier
+cross-encoder on an accelerated host, `llm-judge@localhost` when a local
+OpenAI-compatible endpoint is reachable — probing via `_auto_device` (embed.py) +
+endpoint reachability. Not yet built. (NB: the two heavier fastembed cross-encoders
+measured WORSE than MiniLM on the n=1876 notes gold set — any candidate stage must
+re-clear that bar.)
 
 ---
 

{cribsheet-0.2.2 → cribsheet-0.3.0}/PKG-INFO

@@ -1,12 +1,12 @@
 Metadata-Version: 2.4
 Name: cribsheet
-Version: 0.2.2
+Version: 0.3.0
 Summary: Local MCP server: markdown memory with a Chroma embedding index
 Requires-Python: >=3.13
 Requires-Dist: chromadb>=0.5
 Requires-Dist: fastembed>=0.3
 Requires-Dist: fastmcp<4,>=3.4.2
-Requires-Dist: polyllmkit[bridge,md]>=0.1
+Requires-Dist: polyllmkit[all,md]>=0.1.1
 Requires-Dist: pyyaml>=6.0
 Requires-Dist: watchdog>=4.0
 Provides-Extra: st
@@ -85,16 +85,40 @@ git submodule update --init && pip install -e ./vendor/llmkit
 ```
 </details>
 
-**2 — Wire it into Claude Code.** The simplest path installs the MCP server *and* the
-"reach for memory" directive as a plugin:
+**2 — Wire it into Claude Code.** One plugin installs the MCP server, the `/crib`
+command, and the "reach for memory" directive:
 
 ```bash
 claude plugin marketplace add georgeharker/cribsheet
-claude plugin install cribsheet                 # MCP tools + instructions
-# already have crib served another way? use the instructions only:
-claude plugin install cribsheet-instructions
+claude plugin install cribsheet
 ```
 
+It needs [`sharedserver`](https://github.com/georgeharker/claude-sharedserver) on
+`PATH` (`cargo install sharedserver`) — it keeps **one warm crib** serving every
+session, and it's the same process the CLI attaches to. If it's missing the plugin
+still loads and says so; nothing else breaks.
+
+**The plugin writes to your user-scope MCP config** (`claude mcp add|remove`) rather
+than declaring a server in its manifest — that is what lets one plugin serve both
+deployments, and it means crib appears in `claude mcp list` as a user server you can
+inspect or remove.
+
+**Already have crib served another way?** If an aggregator such as
+[mcp-companion](https://github.com/georgeharker/mcp-companion) already proxies
+crib's tools, a second registration would mount every tool twice. Set the switch
+**once**, machine-wide, and the plugin registers nothing — you still get `/crib` and
+the directive:
+
+```sh
+# ~/.zshenv — must be set before Claude Code starts
+export MCP_COMBINER=1                    # a combiner serves my MCPs…
+export MCP_COMBINER_SERVES_CRIBSHEET=0   # …except crib — per-backend override, wins
+```
+
+It's a **global toggle, not per-session**, and it converges both ways: setting it
+removes crib's registration, unsetting it restores it. A change lands in the *next*
+session. Details: [docs/plugin-mcp-registration.md](docs/plugin-mcp-registration.md).
+
 Prefer to wire it by hand? Copy [`CLAUDE.md.example`](CLAUDE.md.example) into your
 global `$CLAUDE_CONFIG_DIR/CLAUDE.md` — that one-line directive is what keeps the
 habit in context (tool descriptions alone load too late to form it).
@@ -149,12 +173,14 @@ runs), so it's fast; `--no-daemon` runs in-process, `--json` gives machine outpu
 
 ## Going further
 
-- **Run the MCP server** — the plugin (Quickstart) is the easy path. To run it
-  yourself: `crib --mcp` (stdio) or `crib serve --http --port 7732`, registered as a
+- **Run the MCP server** — the plugin (Quickstart) is the easy path: it registers
+  crib and keeps it warm for you. To run it yourself instead: `crib --mcp` (stdio) or
+  `crib serve --http --port 7732`, registered as a
   [`sharedserver`](https://github.com/georgeharker/claude-sharedserver)-backed HTTP
   MCP so one warm crib serves every client (and the same process the CLI attaches
-  to). If a combiner already proxies crib's tools to you, install
-  `cribsheet-instructions` (directive only, no second MCP).
+  to). Set `MCP_COMBINER=1` (or `MCP_COMBINER_SERVES_CRIBSHEET=1`) when something
+  else already serves crib — a combiner, or your own external process — and the
+  plugin will stand down rather than register a second copy.
 - **Share across machines** — the data dir is a git repo; notes sync via plain git
   with a frontmatter-aware merge driver so provenance never conflicts:
   ```bash

{cribsheet-0.2.2 → cribsheet-0.3.0}/README.md

@@ -70,16 +70,40 @@ git submodule update --init && pip install -e ./vendor/llmkit
 ```
 </details>
 
-**2 — Wire it into Claude Code.** The simplest path installs the MCP server *and* the
-"reach for memory" directive as a plugin:
+**2 — Wire it into Claude Code.** One plugin installs the MCP server, the `/crib`
+command, and the "reach for memory" directive:
 
 ```bash
 claude plugin marketplace add georgeharker/cribsheet
-claude plugin install cribsheet                 # MCP tools + instructions
-# already have crib served another way? use the instructions only:
-claude plugin install cribsheet-instructions
+claude plugin install cribsheet
 ```
 
+It needs [`sharedserver`](https://github.com/georgeharker/claude-sharedserver) on
+`PATH` (`cargo install sharedserver`) — it keeps **one warm crib** serving every
+session, and it's the same process the CLI attaches to. If it's missing the plugin
+still loads and says so; nothing else breaks.
+
+**The plugin writes to your user-scope MCP config** (`claude mcp add|remove`) rather
+than declaring a server in its manifest — that is what lets one plugin serve both
+deployments, and it means crib appears in `claude mcp list` as a user server you can
+inspect or remove.
+
+**Already have crib served another way?** If an aggregator such as
+[mcp-companion](https://github.com/georgeharker/mcp-companion) already proxies
+crib's tools, a second registration would mount every tool twice. Set the switch
+**once**, machine-wide, and the plugin registers nothing — you still get `/crib` and
+the directive:
+
+```sh
+# ~/.zshenv — must be set before Claude Code starts
+export MCP_COMBINER=1                    # a combiner serves my MCPs…
+export MCP_COMBINER_SERVES_CRIBSHEET=0   # …except crib — per-backend override, wins
+```
+
+It's a **global toggle, not per-session**, and it converges both ways: setting it
+removes crib's registration, unsetting it restores it. A change lands in the *next*
+session. Details: [docs/plugin-mcp-registration.md](docs/plugin-mcp-registration.md).
+
 Prefer to wire it by hand? Copy [`CLAUDE.md.example`](CLAUDE.md.example) into your
 global `$CLAUDE_CONFIG_DIR/CLAUDE.md` — that one-line directive is what keeps the
 habit in context (tool descriptions alone load too late to form it).
@@ -134,12 +158,14 @@ runs), so it's fast; `--no-daemon` runs in-process, `--json` gives machine outpu
 
 ## Going further
 
-- **Run the MCP server** — the plugin (Quickstart) is the easy path. To run it
-  yourself: `crib --mcp` (stdio) or `crib serve --http --port 7732`, registered as a
+- **Run the MCP server** — the plugin (Quickstart) is the easy path: it registers
+  crib and keeps it warm for you. To run it yourself instead: `crib --mcp` (stdio) or
+  `crib serve --http --port 7732`, registered as a
   [`sharedserver`](https://github.com/georgeharker/claude-sharedserver)-backed HTTP
   MCP so one warm crib serves every client (and the same process the CLI attaches
-  to). If a combiner already proxies crib's tools to you, install
-  `cribsheet-instructions` (directive only, no second MCP).
+  to). Set `MCP_COMBINER=1` (or `MCP_COMBINER_SERVES_CRIBSHEET=1`) when something
+  else already serves crib — a combiner, or your own external process — and the
+  plugin will stand down rather than register a second copy.
 - **Share across machines** — the data dir is a git repo; notes sync via plain git
   with a frontmatter-aware merge driver so provenance never conflicts:
   ```bash