opencode-self-improvement 0.6.5 → 0.7.0

package/README.md

@@ -54,6 +54,11 @@ prompt cache on every message. This one is built the other way around:
   a compaction — and stays byte-identical in between. A stable system prefix
   means the provider's prompt cache survives ordinary turns instead of being
   invalidated on every message.
+- **Two-concept evidence gate.** A free-text prompt must share at least two
+  distinct concepts with a memory for it to be injected; keyword-length
+  queries (one or two words — a deliberate search) need only one. Injection
+  stays quiet on single-word coincidences, which `memory_recall` still
+  reaches on the pull path — rare and right beats frequent and maybe.
 - **Hard character budget.** The block is capped at 1200 characters by default
   (see Options) and trimmed at whole-memory boundaries — never cut mid-sentence.
 - **Evidence is never injected**, and the standing system digest is two lines.
@@ -67,10 +72,10 @@ store writes) using the plugin's real injection code:
 
 | Policy | Recomputes | Injected chars (20 turns) | Avg/turn | Worst block |
 | --- | ---: | ---: | ---: | ---: |
-| A — recompute per prompt, uncapped (naive) | 20 | 29460 | 1473.0 | 1765 |
-| B — epoch freeze + 1200-char cap (shipped) | 4 | 19516 | 975.8 | 1139 |
+| A — recompute per prompt, uncapped (naive) | 20 | 25742 | 1287.1 | 1838 |
+| B — epoch freeze + 1200-char cap (shipped) | 4 | 19990 | 999.5 | 1139 |
 
-Policy B injects 33.8% fewer characters — and the bigger saving isn't even in
+Policy B injects 22.3% fewer characters — and the bigger saving isn't even in
 the character count: B's block is byte-identical between store changes, so the
 prompt cache stays warm across ordinary turns. Reproduce (byte-identical
 output):
@@ -123,23 +128,29 @@ right memory (store of 200; k=3 = the production injection limit):
 
 | Query, vs. the stored memory | share | R@1 | R@3 |
 | --- | ---: | ---: | ---: |
-| lexical — reuses your project's words | 46% | 100% | 100% |
-| near — reworded, one overlap | 17% | 44% | 83% |
+| lexical — reuses your project's words | 46% | 98% | 98% |
+| near — reworded, one overlap | 17% | 28% | 28% |
 | strict — zero shared words | 37% | 0% | 0% |
 
 What the benchmark bought us:
 
 - **In-project prompts are overwhelmingly lexical** — you keep saying
   `bun test`, "the rate limiter", "staging deploys". The shipped matcher
-  nails those: 100% R@3, flat from store size 50 to 500.
+  nails those: 98–100% R@3, flat from store size 50 to 500, and the
+  two-concept evidence gate never abstains on one (0 of 48).
 - **It caught a real bug.** Function words ("the", "what", "how") counted as
   matches, so nearly half the time the top memory was the wrong one. The
   matcher now ignores them on the query side: wrong-first rate fell 49% → 34%,
-  and classic BM25 — a tuned scoring algorithm — no longer beats the shipped
-  matcher on any class except the hopeless one.
+  and the two-concept evidence gate + IDF ranking finished the job: **6%**.
+- **One shared word is a coin flip, so injection stopped playing it.** At the
+  near class (a single shared concept) the old matcher force-pushed a guess;
+  free-text prompts now abstain and the pull path covers them. The trade,
+  exactly: near-class R@3 83% → 28% (the remainder is keyword-length queries,
+  which still inject on one concept) for the 34% → 6% wrong-memory number
+  above. A quiet turn beats a confident wrong one.
 - **Strict paraphrase is the real lexical wall**, and we won't pretend
   otherwise. Nothing keyword-side crosses it. So instead of confident noise the
-  matcher now goes *silent* when the vocabulary shares nothing (a third of
+  matcher now goes *silent* when the vocabulary shares nothing (35 of 39
   strict queries inject nothing at all), and the agent's `memory_recall` loop
   remains as the pull path. Embeddings are the only known crossing — the bench
   exposes a scorer seam if you want to measure them — but requiring an
@@ -211,6 +222,32 @@ After changes, `git pull` and restart opencode.
 
 </details>
 
+## Merge gate
+
+Nothing merges without the full gate: CI runs it as required checks (`lint`,
+`test`, `smoke`), and the same sequence is one command locally:
+
+```sh
+bun run gate          # typecheck + lint + unit tests + terminal smoke & BDD
+bun run gate:hooks    # once per clone: run the gate automatically on `git push`
+```
+
+**Feature order is BDD-first.** Every new feature starts by adding (or fixing)
+a BDD scenario in `e2e/` describing the behavior as a user would observe it —
+before any implementation code and before unit tests. Then write the unit
+tests for the pieces, then the code until everything is green. If no scenario
+turns red first, the feature has no observable contract.
+
+The `e2e/` suite is what keeps a load-error release from shipping again: it
+boots the **real opencode CLI** against each candidate in a fully isolated
+instance (fresh config, fresh db, fresh memory store, empty `HOME`) driven by
+a scripted OpenAI-compatible fake model — no API keys anywhere. Scenarios
+cover loading from the source tree and from the **packed npm artifact
+installed runtime-deps-only** (the same semantics as opencode's plugin
+cache), plus BDD flows for remember, recall, exact duplicates, near duplicates,
+ungrounded claims, and forget. Needs `opencode` on `PATH` locally (or set
+`OPENCODE_BIN`).
+
 ## Options
 
 Zero configuration is the default. To tune the plugin, use the object form of

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "opencode-self-improvement",
-  "version": "0.6.5",
+  "version": "0.7.0",
   "description": "Durable memory for opencode: remember facts, decisions, and commands across sessions, plus a one-command per-repo setup (/repo-init).",
   "main": "index.ts",
   "exports": {
@@ -57,7 +57,10 @@
     "typescript": "^7.0.2"
   },
   "scripts": {
-    "test": "bun test",
+    "test": "bun test ./test/",
+    "test:e2e": "bun test ./e2e/ --timeout 240000",
+    "gate": "bun scripts/merge-gate.ts",
+    "gate:hooks": "git config core.hooksPath .githooks",
     "typecheck": "bunx tsc --noEmit -p tsconfig.json",
     "bench": "bun bench/inject-cost.ts",
     "bench:recall": "bun bench/recall-quality.ts",

package/src/inject.ts

@@ -82,23 +82,44 @@ export function words(text: string): string[] {
 // Memory-side tokens are never filtered: a memory stating "the" must not
 // gain or lose rank from it, only queries drive matching.
 const QUERY_STOPWORDS = new Set(
-  "a an and are at be but by can did do does for from had has have how i in into is it its me my no not of on or our so than that the their them then there these they this to us was we were what when where which who why will with you your run runs getting want needs right back yet also just still ever".split(" "),
+  "a an and are at be but by can did do does for from had has have how i in into is it its me my no not of on or our so than that the their them then there these they this to us was we were what when where which who why will with you your run runs getting want needs right back yet also just still ever should would could might may must shall ought nothing something anything everything thing things".split(" "),
 );
 
-// Production relevance ranking: score = number of distinct query tokens the
-// memory's (scope+type+content) token set contains. Exported for
-// bench/recall-quality.ts — the benchmark must measure the shipped matcher.
+// A free-text prompt long enough to carry >= this many content tokens must
+// share >= 2 distinct concepts with a memory to inject: with a long query
+// vocabulary, one shared token is near-certain coincidence (a live store hit
+// injected a git memory on the word "should", df=1). A keyword-length query
+// is a deliberate search — one shared concept suffices. memory_recall and the
+// contextual pull keep single-overlap matches reachable; injection just stays
+// quiet unless the vocabulary genuinely agrees twice.
+const LONG_QUERY_TOKENS = 3;
+
+// Production relevance ranking: candidates must clear the concept-overlap
+// gate, then rank by Robertson-IF of the shared tokens (rare concepts beat
+// piles of common ones — the same idf form the bench's BM25 row uses).
+// Exported for bench/recall-quality.ts — the benchmark measures the shipped
+// matcher.
 export function scoreMemories(term: string, mems: Memory[], limit: number): Memory[] {
   const query = new Set(words(term).filter((t) => !QUERY_STOPWORDS.has(t)));
   if (!query.size) return [];
+  const needShared = query.size >= LONG_QUERY_TOKENS ? 2 : 1;
+  const hays = mems.map((m) => new Set(words(`${m.scope} ${m.type} ${m.content}`)));
+  const df = new Map<string, number>();
+  for (const h of hays) for (const t of h) df.set(t, (df.get(t) ?? 0) + 1);
+  const n = mems.length || 1;
   return mems
-    .map((m) => {
-      const hay = new Set(words(`${m.scope} ${m.type} ${m.content}`));
+    .map((m, i) => {
+      let shared = 0;
       let score = 0;
-      for (const w of query) if (hay.has(w)) score++;
-      return { m, score };
+      for (const w of query)
+        if (hays[i]!.has(w)) {
+          shared++;
+          const d = df.get(w) ?? 0;
+          score += Math.log(1 + (n - d + 0.5) / (d + 0.5));
+        }
+      return { m, shared, score };
     })
-    .filter((s) => s.score > 0)
+    .filter((s) => s.shared >= needShared)
     .sort((a, b) => b.score - a.score || openRank(b.m) - openRank(a.m))
     .slice(0, limit)
     .map((s) => s.m);

package/tsconfig.json

@@ -13,5 +13,5 @@
     "jsx": "preserve",
     "jsxImportSource": "@opentui/solid"
   },
-  "include": ["index.ts", "src", "test", "bench"]
+  "include": ["index.ts", "src", "test", "bench", "e2e"]
 }