opencode-self-improvement 0.6.5 → 0.8.0

package/README.md

@@ -54,6 +54,11 @@ prompt cache on every message. This one is built the other way around:
   a compaction — and stays byte-identical in between. A stable system prefix
   means the provider's prompt cache survives ordinary turns instead of being
   invalidated on every message.
+- **Two-concept evidence gate.** A free-text prompt must share at least two
+  distinct concepts with a memory for it to be injected; keyword-length
+  queries (one or two words — a deliberate search) need only one. Injection
+  stays quiet on single-word coincidences, which `memory_recall` still
+  reaches on the pull path — rare and right beats frequent and maybe.
 - **Hard character budget.** The block is capped at 1200 characters by default
   (see Options) and trimmed at whole-memory boundaries — never cut mid-sentence.
 - **Evidence is never injected**, and the standing system digest is two lines.
@@ -67,10 +72,10 @@ store writes) using the plugin's real injection code:
 
 | Policy | Recomputes | Injected chars (20 turns) | Avg/turn | Worst block |
 | --- | ---: | ---: | ---: | ---: |
-| A — recompute per prompt, uncapped (naive) | 20 | 29460 | 1473.0 | 1765 |
-| B — epoch freeze + 1200-char cap (shipped) | 4 | 19516 | 975.8 | 1139 |
+| A — recompute per prompt, uncapped (naive) | 20 | 25742 | 1287.1 | 1838 |
+| B — epoch freeze + 1200-char cap (shipped) | 4 | 19990 | 999.5 | 1139 |
 
-Policy B injects 33.8% fewer characters — and the bigger saving isn't even in
+Policy B injects 22.3% fewer characters — and the bigger saving isn't even in
 the character count: B's block is byte-identical between store changes, so the
 prompt cache stays warm across ordinary turns. Reproduce (byte-identical
 output):
@@ -123,23 +128,29 @@ right memory (store of 200; k=3 = the production injection limit):
 
 | Query, vs. the stored memory | share | R@1 | R@3 |
 | --- | ---: | ---: | ---: |
-| lexical — reuses your project's words | 46% | 100% | 100% |
-| near — reworded, one overlap | 17% | 44% | 83% |
+| lexical — reuses your project's words | 46% | 98% | 98% |
+| near — reworded, one overlap | 17% | 28% | 28% |
 | strict — zero shared words | 37% | 0% | 0% |
 
 What the benchmark bought us:
 
 - **In-project prompts are overwhelmingly lexical** — you keep saying
   `bun test`, "the rate limiter", "staging deploys". The shipped matcher
-  nails those: 100% R@3, flat from store size 50 to 500.
+  nails those: 98–100% R@3, flat from store size 50 to 500, and the
+  two-concept evidence gate never abstains on one (0 of 48).
 - **It caught a real bug.** Function words ("the", "what", "how") counted as
   matches, so nearly half the time the top memory was the wrong one. The
   matcher now ignores them on the query side: wrong-first rate fell 49% → 34%,
-  and classic BM25 — a tuned scoring algorithm — no longer beats the shipped
-  matcher on any class except the hopeless one.
+  and the two-concept evidence gate + IDF ranking finished the job: **6%**.
+- **One shared word is a coin flip, so injection stopped playing it.** At the
+  near class (a single shared concept) the old matcher force-pushed a guess;
+  free-text prompts now abstain and the pull path covers them. The trade,
+  exactly: near-class R@3 83% → 28% (the remainder is keyword-length queries,
+  which still inject on one concept) for the 34% → 6% wrong-memory number
+  above. A quiet turn beats a confident wrong one.
 - **Strict paraphrase is the real lexical wall**, and we won't pretend
   otherwise. Nothing keyword-side crosses it. So instead of confident noise the
-  matcher now goes *silent* when the vocabulary shares nothing (a third of
+  matcher now goes *silent* when the vocabulary shares nothing (35 of 39
   strict queries inject nothing at all), and the agent's `memory_recall` loop
   remains as the pull path. Embeddings are the only known crossing — the bench
   exposes a scorer seam if you want to measure them — but requiring an
@@ -211,6 +222,32 @@ After changes, `git pull` and restart opencode.
 
 </details>
 
+## Merge gate
+
+Nothing merges without the full gate: CI runs it as required checks (`lint`,
+`test`, `smoke`), and the same sequence is one command locally:
+
+```sh
+bun run gate          # typecheck + lint + unit tests + terminal smoke & BDD
+bun run gate:hooks    # once per clone: run the gate automatically on `git push`
+```
+
+**Feature order is BDD-first.** Every new feature starts by adding (or fixing)
+a BDD scenario in `e2e/` describing the behavior as a user would observe it —
+before any implementation code and before unit tests. Then write the unit
+tests for the pieces, then the code until everything is green. If no scenario
+turns red first, the feature has no observable contract.
+
+The `e2e/` suite is what keeps a load-error release from shipping again: it
+boots the **real opencode CLI** against each candidate in a fully isolated
+instance (fresh config, fresh db, fresh memory store, empty `HOME`) driven by
+a scripted OpenAI-compatible fake model — no API keys anywhere. Scenarios
+cover loading from the source tree and from the **packed npm artifact
+installed runtime-deps-only** (the same semantics as opencode's plugin
+cache), plus BDD flows for remember, recall, exact duplicates, near duplicates,
+ungrounded claims, and forget. Needs `opencode` on `PATH` locally (or set
+`OPENCODE_BIN`).
+
 ## Options
 
 Zero configuration is the default. To tune the plugin, use the object form of
@@ -328,9 +365,12 @@ same data (`memory_status` tool under the hood; no model call):
   - **CONTEXT** — exact chars + estimated tokens riding now, % of the budget,
     requests the current epoch has ridden, session-lifetime totals, and how
     many requests abstained (silence is a feature)
-  - **CACHE** — epoch invalidations this session with the last reason, server
-    recompute count, and the *measured* prompt-cache read rate of the last turn
-    — the receipt, not just the promise
+  - **CACHE** — the receipts, in plain language: cache busts this session
+    (0 = memory has never rewritten a byte of your history), the measured
+    share of prompt tokens the provider served from cache (≥90% reads =
+    "memory is cache-invisible"), or — when the provider reports no usage at
+    all — the structural proof instead: the same block bytes rode every
+    request with zero busts
   - **INJECTED BLOCK** — the verbatim block riding your prompts, or an explicit
     "abstained: no vocabulary overlap"
   - **RECENT BLOCKED WRITES** — that `memory_remember` that didn't stick,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "opencode-self-improvement",
-  "version": "0.6.5",
+  "version": "0.8.0",
   "description": "Durable memory for opencode: remember facts, decisions, and commands across sessions, plus a one-command per-repo setup (/repo-init).",
   "main": "index.ts",
   "exports": {
@@ -57,7 +57,10 @@
     "typescript": "^7.0.2"
   },
   "scripts": {
-    "test": "bun test",
+    "test": "bun test ./test/",
+    "test:e2e": "bun test ./e2e/ --timeout 240000",
+    "gate": "bun scripts/merge-gate.ts",
+    "gate:hooks": "git config core.hooksPath .githooks",
     "typecheck": "bunx tsc --noEmit -p tsconfig.json",
     "bench": "bun bench/inject-cost.ts",
     "bench:recall": "bun bench/recall-quality.ts",

package/src/index.ts

@@ -570,7 +570,7 @@ const impl = {
       editor.add(validated(EmptyArgs, {
         name: "memory_status",
         description:
-          "Plain-text memory report for chat clients (desktop/web/IDE), same sections as the terminal /memory panel: CONTEXT (this session's injected block, budget use, cumulative ride and abstentions), CACHE (epoch invalidations plus the measured session prompt-cache read rate), the block verbatim, recent blocked writes with gate reasons, and store stats. Reads only — no model call.",
+          "Plain-text memory report for chat clients (desktop/web/IDE), same sections as the terminal /memory panel: CONTEXT (this session's injected block, budget use, cumulative ride and abstentions), CACHE (plain-language cache receipts: bust count, and the provider-reported share of prompt tokens served from cache — the proof that memory never mutates your prefix), the block verbatim, recent blocked writes with gate reasons, and store stats. Reads only — no model call.",
         options: direct,
         async execute(): Promise<{ content: string }> {
           // In-process truth: the live epoch + ledger, fresher than any file.

package/src/inject.ts

@@ -82,23 +82,44 @@ export function words(text: string): string[] {
 // Memory-side tokens are never filtered: a memory stating "the" must not
 // gain or lose rank from it, only queries drive matching.
 const QUERY_STOPWORDS = new Set(
-  "a an and are at be but by can did do does for from had has have how i in into is it its me my no not of on or our so than that the their them then there these they this to us was we were what when where which who why will with you your run runs getting want needs right back yet also just still ever".split(" "),
+  "a an and are at be but by can did do does for from had has have how i in into is it its me my no not of on or our so than that the their them then there these they this to us was we were what when where which who why will with you your run runs getting want needs right back yet also just still ever should would could might may must shall ought nothing something anything everything thing things".split(" "),
 );
 
-// Production relevance ranking: score = number of distinct query tokens the
-// memory's (scope+type+content) token set contains. Exported for
-// bench/recall-quality.ts — the benchmark must measure the shipped matcher.
+// A free-text prompt long enough to carry >= this many content tokens must
+// share >= 2 distinct concepts with a memory to inject: with a long query
+// vocabulary, one shared token is near-certain coincidence (a live store hit
+// injected a git memory on the word "should", df=1). A keyword-length query
+// is a deliberate search — one shared concept suffices. memory_recall and the
+// contextual pull keep single-overlap matches reachable; injection just stays
+// quiet unless the vocabulary genuinely agrees twice.
+const LONG_QUERY_TOKENS = 3;
+
+// Production relevance ranking: candidates must clear the concept-overlap
+// gate, then rank by Robertson-IF of the shared tokens (rare concepts beat
+// piles of common ones — the same idf form the bench's BM25 row uses).
+// Exported for bench/recall-quality.ts — the benchmark measures the shipped
+// matcher.
 export function scoreMemories(term: string, mems: Memory[], limit: number): Memory[] {
   const query = new Set(words(term).filter((t) => !QUERY_STOPWORDS.has(t)));
   if (!query.size) return [];
+  const needShared = query.size >= LONG_QUERY_TOKENS ? 2 : 1;
+  const hays = mems.map((m) => new Set(words(`${m.scope} ${m.type} ${m.content}`)));
+  const df = new Map<string, number>();
+  for (const h of hays) for (const t of h) df.set(t, (df.get(t) ?? 0) + 1);
+  const n = mems.length || 1;
   return mems
-    .map((m) => {
-      const hay = new Set(words(`${m.scope} ${m.type} ${m.content}`));
+    .map((m, i) => {
+      let shared = 0;
       let score = 0;
-      for (const w of query) if (hay.has(w)) score++;
-      return { m, score };
+      for (const w of query)
+        if (hays[i]!.has(w)) {
+          shared++;
+          const d = df.get(w) ?? 0;
+          score += Math.log(1 + (n - d + 0.5) / (d + 0.5));
+        }
+      return { m, shared, score };
     })
-    .filter((s) => s.score > 0)
+    .filter((s) => s.shared >= needShared)
     .sort((a, b) => b.score - a.score || openRank(b.m) - openRank(a.m))
     .slice(0, limit)
     .map((s) => s.m);

package/src/status.ts

@@ -262,15 +262,34 @@ export function cacheLines(p: StatusPayload): string[] {
   const st = p.stats ?? emptyStats();
   const inv =
     st.invalidations === 0
-      ? "  invalidations  0 this session"
-      : `  invalidations  ${st.invalidations} this session · last ${
-          st.lastInvalidation ? `${st.lastInvalidation.at.slice(11, 19)} (${st.lastInvalidation.reason})` : "-"
-        }`;
+      ? [
+          "  cache busts    0 — memory has never rewritten a byte of your past",
+          "                 conversation, so the provider can serve history from cache",
+        ]
+      : [
+          `  cache busts    ${st.invalidations} this session · last ${
+            st.lastInvalidation ? `${st.lastInvalidation.at.slice(11, 19)} (${st.lastInvalidation.reason})` : "-"
+          }`,
+          "                 each memory write between turns costs exactly one; the",
+          "                 prefix re-warms on the following turn",
+        ];
   const rate = cacheRate(p.measured);
-  const measured = rate
-    ? `  measured       last turn ${fmtCount(rate.read)}/${fmtCount(rate.prompt)} prompt tok cache-read (${rate.pct}%)`
-    : "  measured       not measured yet — no completed assistant turn";
-  return [inv, `  recomputes     ${p.recomputes} since server start (all sessions)`, measured];
+  const receipts = rate
+    ? [
+        `  receipts       ${fmtCount(rate.read)}/${fmtCount(rate.prompt)} prompt tok served from cache (${rate.pct}%)`,
+        rate.pct >= 90
+          ? "                 — memory is cache-invisible: the same bytes ride every turn"
+          : "                 — below ~90% something re-bills the whole history each turn",
+      ]
+    : [
+        "  receipts       provider reports no cache usage yet — structural proof:",
+        `                 the same block bytes rode ${st.epochRequests} request(s) with ${st.invalidations} busts`,
+      ];
+  return [
+    ...inv,
+    ...receipts,
+    `  recomputes     ${p.recomputes} block builds since server start (all sessions)`,
+  ];
 }
 
 /**
@@ -292,7 +311,7 @@ export function renderMemoryReport(input: {
     );
   } else {
     out.push("CONTEXT", ...contextLines(status));
-    out.push("", "CACHE", ...cacheLines(status));
+    out.push("", "CACHE — the receipts on your prompt-cache cost", ...cacheLines(status));
     out.push("", "INJECTED BLOCK (verbatim)");
     if (status.block) out.push(...status.block.split("\n").map((l) => `  ${l}`));
     else

package/tsconfig.json

@@ -13,5 +13,5 @@
     "jsx": "preserve",
     "jsxImportSource": "@opentui/solid"
   },
-  "include": ["index.ts", "src", "test", "bench"]
+  "include": ["index.ts", "src", "test", "bench", "e2e"]
 }