klypix-mcp 1.85.0 → 1.86.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -789,19 +789,37 @@ Read this section before you build on any of it.
789
789
  Every number here is measured on our own project brain. Nothing below is published, benchmarked or
790
790
  independently validated.
791
791
 
792
- - **Dogfood scale.** KLYPIX itself is built with its own brain: **2,479 cards and 2,018
792
+ - **Dogfood scale.** KLYPIX itself is built with its own brain: **2,695 cards and 2,333
793
793
  connections**, written by multiple concurrent agent sessions, receipts in the file. Current as of
794
- 2026-08-13.
794
+ 2026-09-16.
795
795
  - **Recall.** 73% of past decisions recovered with one search round, 55% brief-only, 0% cold.
796
796
  Caveat that travels with it: n=20, our own brain, self-authored questions, LLM-judged.
797
- - **Ranker.** With the production embedder (the eval harness was fixed 2026-08-10 — it had been
798
- measuring a vector space the product does not use): recall@5 **30%**, recall@10 35%, recall@20
799
- 45%, MRR 0.22 of the true source card on n=20 frozen human-paraphrase questions. Lexical-only
800
- scores 0% on the same set. The previously published "15% → 40% with the reranker" is **retired**:
801
- re-measured validly, the reranker *reduced* recall@5 to 25% and now ships off by default. At n=20
802
- every one of these percentages carries a ±20-point 95% confidence interval — treat them as
803
- directional until the larger frozen set lands. The regressions are recorded next to the wins:
804
- contextual prefixes on short cards, and the reranker itself.
797
+ - **Ranker (`brain_ask`).** Measured 2026-09-16 on the real 2,695-card brain with the production
798
+ embedder, n=107 frozen questions (agent-authored, adversarially verified, four strata), model-free
799
+ rank of the true source card: recall@5 **63%** (95% CI 53–71), recall@10 65%, recall@20 69%, MRR
800
+ 0.43, top-1 33%. The stratum that matters most is the honest one: **paraphrase questions that share
801
+ no words with their card reach recall@5 44%**; status, temporal and multi-hop questions sit at
802
+ 81–93% because they still share vocabulary. Lexical-only scores 0% on the same set.
803
+ - **Second brain.** The same ranker on a different project's brain (1,601 cards, 19 hand-written
804
+ paraphrases, first cross-brain run): recall@5 **42%** (CI 23–64), top-1 11%, MRR 0.25. Paraphrase
805
+ recall transfers between brains; the ranker is not tuned to the brain it was built on. It is
806
+ simply weak on paraphrase everywhere — the embedder's ceiling, documented in `rankForQuestion`.
807
+ - **Retired numbers.** "recall@5 30% (n=20)", "15% → 40% with the reranker" and the "5% → 15% →
808
+ 40%" curve are all **retired**: the first was the n=20 set the larger set replaced, the others were
809
+ measured with a reranker that ships off by default (validly re-measured it *reduced* recall@5) or
810
+ by a harness that had drifted from the production vector space (fixed 2026-08-10). The
811
+ regressions are recorded next to the wins: contextual prefixes on short cards, and the reranker.
812
+ - **The per-prompt hook lane** (the retrieval every Claude Code session receives on every prompt)
813
+ had never been measured before 1.86. `scripts/eval-hook-lane.mjs` now measures it through the
814
+ production rankers on a private capture-pair set built from the machine's own enrichment sidecar
815
+ (`scripts/build-hook-eval-set.mjs`). First measurement, 110 real prompts on our own brain: the
816
+ lexical lane fired on 97% of content-bearing prompts and held the right card in its top five 29%
817
+ of the time, and the lane as a whole injected cards on 93% of prompts that should inject nothing
818
+ (acknowledgements, console echoes, relayed machine turns). 1.86 raises the lexical bar so a
819
+ single title word no longer injects, and skips the uncorroborated semantic guess for prompts with
820
+ fewer than four content words: junk injection 93% → 35%, mean cards injected on junk 3.6 → 1.5,
821
+ at a one-question cost on the 35 real prompts (inside noise). The prompts stay private; the
822
+ sweep tables are in the source next to the bars they chose.
805
823
  - **What we do not publish.** No download count: this package's own 24-hour auto-updater generates
806
824
  most of it, so it is not a user count. No adoption, team or customer figures. No brief-token
807
825
  figure — the last one was measured at ~600 cards and is stale at 2,479.
@@ -704,6 +704,7 @@ server.registerTool('brain_note', {
704
704
  marker: z.enum(['', '?', '!', '+', '✓', '~']).optional().describe('(none)=decision · ?=open question · !=milestone · +=🛠️ skill (reusable how-to/gotcha; always resurfaces, never ages out) · ✓=resolve+archive the best-matching card · ~=update the matching card in place. Default: decision.'),
705
705
  area: z.string().optional().describe('Area/topic — routes the card into that titled container and becomes a #tag (e.g. "Auth", "Release").'),
706
706
  closes: z.string().optional().describe('Title or [[wikilink]] of a strategy/question card this note fulfils — resolves+archives it and draws a "closed by" arrow.'),
707
+ question: z.string().max(240).optional().describe('The question this note ANSWERS, phrased as someone would ask it ("how do we keep a draft separate from what runs?"). Recorded as retrieval enrichment beside the vector cache — never on the canvas — so the card is found by paraphrase, not only by its own words. The session\'s declared intent is recorded as well.'),
707
708
  evidence: z.array(z.object({
708
709
  kind: z.enum(['file', 'pr', 'url', 'commit', 'run']),
709
710
  ref: z.string().min(1).max(1000).describe('A project-relative file path (optional :line or #Lline), or an external reference. External references are stored without fetching them.'),
@@ -724,10 +725,12 @@ server.registerTool('brain_note', {
724
725
  }).optional().describe("GUARD CARDS: make this '+' skill fire BEFORE a matching tool call runs (Claude Code PreToolUse denies on severity block; other hosts warn), not just resurface in briefs. The card stays a normal 🛠️ rule — ✓-resolving it retires the guard, ~ with {remove:true} disarms it."),
725
726
  canvas: z.string().optional().describe('Brain canvas filename/path. Defaults to the project brain ("brain").'),
726
727
  },
727
- }, async ({ text, marker, area, closes, evidence, verify, guard, canvas }, extra) => {
728
+ }, async ({ text, marker, area, closes, question, evidence, verify, guard, canvas }, extra) => {
728
729
  // Both 1.77 and 1.78 ride this call: the enrichment question (the asker's
729
- // vocabulary for retrieval) AND the per-session capture receipt below.
730
- const result = await opBrainNote({ vault: mcpPresence.vault, canvas: boundBrainCanvas(canvas), text, area, marker: marker || '', closes, evidence, verify, guard, via: extra.klypixClientName, enrichmentQuestion: mcpPresence.declaredIntent });
730
+ // vocabulary for retrieval) AND the per-session capture receipt below. Since
731
+ // 1.86 the authored `question` (the pattern) rides beside the declared intent
732
+ // (the instance); the sidecar's quality gate decides what is worth keeping.
733
+ const result = await opBrainNote({ vault: mcpPresence.vault, canvas: boundBrainCanvas(canvas), text, area, marker: marker || '', closes, evidence, verify, guard, via: extra.klypixClientName, enrichmentQuestion: [question, mcpPresence.declaredIntent] });
731
734
  // Per-session capture receipt — this is what stops the uncaptured-work nudge
732
735
  // from firing at a session that DID record its reasoning, just through MCP
733
736
  // rather than a 🧠 marker. The Stop hook and this server share one session-id
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "klypix-mcp",
3
- "version": "1.85.0",
3
+ "version": "1.86.0",
4
4
  "mcpName": "io.github.dahshanlabs/klypix-mcp",
5
5
  "description": "Active state management for multi-agent coding: a shared, versioned project brain over MCP.",
6
6
  "type": "module",
@@ -84,11 +84,13 @@
84
84
  "bench": "node bin/klypix-mcp.mjs bench",
85
85
  "test:bench": "node test/bench.mjs",
86
86
  "pretest": "node test/publish-workflow.mjs",
87
- "test": "node test/publish-verdict.mjs && node test/project-graph.mjs && node test/project-map-cli.mjs && node test/mcp-auto-update.mjs && node test/mcp-supervisor.mjs && node test/runtime-inspector.mjs && node test/codex-hooks.mjs && node test/request-identity.mjs && node test/session-identity-core.mjs && node test/agent-presence.mjs && node test/message-delivery-v3.mjs && node test/claude-message-delivery-v3.mjs && node test/result-reconcile.mjs && node test/evidence-publication-gate.mjs && node test/release-evidence-cli.mjs && node test/intent-guard.mjs && node test/git-capture-install.mjs && node test/brain-history.mjs && node test/brain-graveyard.mjs && node test/archived-visibility.mjs && node test/finding-routing.mjs && node test/finding-routing-hook.mjs && node test/presence-relay.mjs && node test/install-version.mjs && node test/install-rename-backoff.mjs && node test/project-binding-rebind.mjs && node test/context-gateway.mjs && node test/repo-state.mjs && node test/released-tag-guard.mjs && node test/conformance.mjs && node test/brain-doctor.mjs && node test/version-currency.mjs && node test/ship-capture.mjs && node test/capture-gap.mjs && node test/lane-message.mjs && node test/brain-quality.mjs && node test/brain-connect-orphans.mjs && node test/orphan-gardener.mjs && node test/brief-and-recall.mjs && node test/guard-cards.mjs && node test/layout-cluster.mjs && node test/brain-ask.mjs && node test/retrieval-fusion.mjs && node test/eval-retrieval.mjs && node test/field-report-2026-07-04.mjs && node test/autoprop.mjs && node test/overlay-recency-2026-07-12.mjs && node test/brain-challenge.mjs && node test/brain-lens.mjs && node test/brain-kind.mjs && node test/rule-drafts.mjs && node test/claim-engine.mjs && node test/partial-notes.mjs && node test/lifecycle-prefix.mjs && node test/close-link-safety.mjs && node test/resolve-ledger.mjs && node test/plan-fulfillment.mjs && node test/skill-staleness.mjs && node test/canvas-view.mjs && node test/status-completeness.mjs && node test/semantic-security.mjs && node test/semantic-gate.mjs && node test/memory-runtime.mjs && node test/semantic-cache.mjs && node test/enrichment.mjs && node test/decay-status.mjs && node test/decay-hook.mjs && node test/evidence-anchors.mjs && node test/brain-evidence.mjs && node test/presence-visibility.mjs && node test/undeclared-active.mjs && node test/presence-liveness.mjs && node test/observed-scope.mjs && node test/release-lease.mjs && node test/release-reconcile.mjs && node test/release-ancestry.mjs && node test/release-claim-join.mjs && node test/release-claims.mjs && node test/release-handshake.mjs && node test/completion-guard.mjs && node test/merge-brains.mjs && node test/concurrent-writes.mjs && node test/lock-interop.mjs && node test/capture-write-failure.mjs && node test/a2a-smoke.mjs && node test/one-command-setup.mjs && node test/cli-args.mjs && node test/format-guard.mjs && node test/canvas-groups.mjs && node test/git-tools.mjs && node test/uninstall.mjs && node test/current-guidance.mjs && node test/status-shape.mjs && node test/status-hook.mjs",
87
+ "test": "node test/publish-verdict.mjs && node test/project-graph.mjs && node test/project-map-cli.mjs && node test/mcp-auto-update.mjs && node test/mcp-supervisor.mjs && node test/runtime-inspector.mjs && node test/codex-hooks.mjs && node test/request-identity.mjs && node test/session-identity-core.mjs && node test/agent-presence.mjs && node test/message-delivery-v3.mjs && node test/claude-message-delivery-v3.mjs && node test/result-reconcile.mjs && node test/evidence-publication-gate.mjs && node test/release-evidence-cli.mjs && node test/intent-guard.mjs && node test/git-capture-install.mjs && node test/brain-history.mjs && node test/brain-graveyard.mjs && node test/archived-visibility.mjs && node test/finding-routing.mjs && node test/finding-routing-hook.mjs && node test/presence-relay.mjs && node test/install-version.mjs && node test/install-rename-backoff.mjs && node test/project-binding-rebind.mjs && node test/context-gateway.mjs && node test/repo-state.mjs && node test/released-tag-guard.mjs && node test/conformance.mjs && node test/brain-doctor.mjs && node test/version-currency.mjs && node test/ship-capture.mjs && node test/capture-gap.mjs && node test/lane-message.mjs && node test/brain-quality.mjs && node test/brain-connect-orphans.mjs && node test/orphan-gardener.mjs && node test/brief-and-recall.mjs && node test/guard-cards.mjs && node test/layout-cluster.mjs && node test/brain-ask.mjs && node test/retrieval-fusion.mjs && node test/eval-retrieval.mjs && node test/field-report-2026-07-04.mjs && node test/autoprop.mjs && node test/overlay-recency-2026-07-12.mjs && node test/brain-challenge.mjs && node test/brain-lens.mjs && node test/brain-kind.mjs && node test/rule-drafts.mjs && node test/claim-engine.mjs && node test/partial-notes.mjs && node test/lifecycle-prefix.mjs && node test/close-link-safety.mjs && node test/resolve-ledger.mjs && node test/plan-fulfillment.mjs && node test/skill-staleness.mjs && node test/canvas-view.mjs && node test/status-completeness.mjs && node test/semantic-security.mjs && node test/semantic-gate.mjs && node test/memory-runtime.mjs && node test/semantic-cache.mjs && node test/enrichment.mjs && node test/hook-fallback.mjs && node test/eval-hook-lane.mjs && node test/decay-status.mjs && node test/decay-hook.mjs && node test/evidence-anchors.mjs && node test/brain-evidence.mjs && node test/presence-visibility.mjs && node test/undeclared-active.mjs && node test/presence-liveness.mjs && node test/observed-scope.mjs && node test/release-lease.mjs && node test/release-reconcile.mjs && node test/release-ancestry.mjs && node test/release-claim-join.mjs && node test/release-claims.mjs && node test/release-handshake.mjs && node test/completion-guard.mjs && node test/merge-brains.mjs && node test/concurrent-writes.mjs && node test/lock-interop.mjs && node test/capture-write-failure.mjs && node test/a2a-smoke.mjs && node test/one-command-setup.mjs && node test/cli-args.mjs && node test/format-guard.mjs && node test/canvas-groups.mjs && node test/git-tools.mjs && node test/uninstall.mjs && node test/current-guidance.mjs && node test/status-shape.mjs && node test/status-hook.mjs",
88
88
  "test:memory": "node test/memory-runtime.mjs",
89
89
  "test:memory:soak": "node --expose-gc test/memory-soak.mjs",
90
90
  "runtime": "node bin/klypix-runtime.mjs",
91
- "eval:retrieval": "node scripts/eval-retrieval.mjs"
91
+ "eval:retrieval": "node scripts/eval-retrieval.mjs",
92
+ "eval:hook-lane": "node scripts/eval-hook-lane.mjs",
93
+ "eval:hook-lane:build": "node scripts/build-hook-eval-set.mjs"
92
94
  },
93
95
  "dependencies": {
94
96
  "@modelcontextprotocol/ext-apps": "^1.7.5",
@@ -66,18 +66,73 @@ function readFile(file) {
66
66
 
67
67
  const cleanQuestion = (q) => String(q || '').replace(/\s+/g, ' ').trim().slice(0, ENRICHMENT_MAX_QUESTION_CHARS);
68
68
 
69
+ // ── Question quality gate (1.86) ─────────────────────────────────────────────
70
+ // "The nearest human prompt" is the right SOURCE of asker language but is not
71
+ // always a question. Measured on this project's own sidecar (2026-09-16, 187
72
+ // recorded texts): 36% were question-shaped, 18% were acknowledgements ("done,
73
+ // what now", "300mb is ok", "ok do them"), and the rest included npm script
74
+ // echoes ("> klypix@1.3.127 release:register …"), hook feedback re-injected
75
+ // as a user turn, and pasted documents. Every one of those was appended to a
76
+ // card's EMBED input as if it were the words someone would ask with — pulling
77
+ // the vector toward generic chatter, the opposite of the vocabulary bridge
78
+ // enrichment exists to build. The gate rejects text that carries no askable
79
+ // vocabulary and runs on BOTH sides (record and read), so an already-recorded
80
+ // sidecar is cleaned lazily on its next read without a rewrite. A rejection
81
+ // costs nothing but recall the text never carried.
82
+ export const ENRICHMENT_MIN_CONTENT_TOKENS = 4;
83
+ const ENRICHMENT_STOPWORDS = new Set((
84
+ 'a an and are as at be been but by can could did do does for from had has have he her his how i if in is it its '
85
+ + 'just let lets me my no nor not of ok okay on or our out she so than that the their them then there these they this '
86
+ + 'those to up us was we were what when where which who whom why will with would you your yes yep nope sure please '
87
+ + 'thanks thank great good fine done cool right now also very really any all both each more most some such too again '
88
+ + 'about into over under after before while because '
89
+ // Arabic function words and acknowledgements (folded forms; enrichment text is
90
+ // recorded as typed, so the common unfolded spellings are listed too).
91
+ + 'هل ما ماذا لماذا كيف اين أين وين متى من في على الى إلى عن هذا هذه ذلك تلك هو هي هم انا أنا نحن انت أنت و او أو لا نعم تمام طيب اوكي ايوه ايوا'
92
+ ).split(/\s+/).filter(Boolean));
93
+ const ENRICHMENT_MACHINE_RE = /stop hook feedback|<agent-message|\[subagent hand-back\]|<task-notification|system-reminder|\[system notification|<command-(?:name|message|args)|local-command-(?:stdout|stderr)|<hook-[a-z0-9-]+|^\[image:|base directory for this skill:/i;
94
+ const ENRICHMENT_CONSOLE_RE = /^[>$]\s|(?:^|\s)npm (?:err!|warn)\b|\bexited with code \d|[✓✔]|^\s*\{"|^\s*\[\{/i;
95
+ const ENRICHMENT_PASTED_DOC_RE = /^#{1,6}\s|^```|^---\s|^(?:import|export|const|function|class)\s/;
96
+ const contentTokens = (text) => String(text || '').toLowerCase().split(/[^\p{L}\p{N}]+/u)
97
+ .filter((token) => token.length >= 2 && !ENRICHMENT_STOPWORDS.has(token));
98
+
99
+ /**
100
+ * Decide whether a text carries askable vocabulary worth embedding beside a
101
+ * card. Returns { ok, reason, text } — `text` is the cleaned form that would be
102
+ * recorded. Reasons: too-short · machine (harness/hook output) · console (npm /
103
+ * shell echo) · pasted-doc (markdown/code block) · low-content (fewer than
104
+ * ENRICHMENT_MIN_CONTENT_TOKENS non-stopword tokens — the acknowledgement class).
105
+ */
106
+ export function enrichmentQuestionQuality(question) {
107
+ const text = cleanQuestion(question);
108
+ if (text.length < 8) return { ok: false, reason: 'too-short', text };
109
+ if (ENRICHMENT_MACHINE_RE.test(text)) return { ok: false, reason: 'machine', text };
110
+ if (ENRICHMENT_CONSOLE_RE.test(text)) return { ok: false, reason: 'console', text };
111
+ if (ENRICHMENT_PASTED_DOC_RE.test(text)) return { ok: false, reason: 'pasted-doc', text };
112
+ if (contentTokens(text).length < ENRICHMENT_MIN_CONTENT_TOKENS) return { ok: false, reason: 'low-content', text };
113
+ return { ok: true, reason: null, text };
114
+ }
115
+
69
116
  /**
70
117
  * Record question/intent text for captured card bodies. `items` is
71
118
  * [{ body, question }]; entries merge per body key (deduped, newest kept,
72
119
  * capped). Bounded overall: past ENRICHMENT_MAX_ENTRIES the OLDEST entries are
73
120
  * pruned — enrichment is a rolling quality window, not an archive, and unlike
74
121
  * the claims lane nothing downstream depends on any single entry existing.
122
+ * Returns { recorded, rejected } — `rejected` counts texts the quality gate
123
+ * refused (see enrichmentQuestionQuality); they are never written.
75
124
  */
76
125
  export function recordEnrichment(brainPath, items, { home = os.homedir(), now = Date.now() } = {}) {
77
- const list = (Array.isArray(items) ? items : [])
78
- .map((item) => ({ key: enrichmentKeyFor(item?.body), q: cleanQuestion(item?.question) }))
79
- .filter((item) => item.key.length >= 24 && item.q.length >= 8);
80
- if (!list.length) return { recorded: 0 };
126
+ const list = [];
127
+ let rejected = 0;
128
+ for (const item of (Array.isArray(items) ? items : [])) {
129
+ const key = enrichmentKeyFor(item?.body);
130
+ if (key.length < 24) continue;
131
+ const quality = enrichmentQuestionQuality(item?.question);
132
+ if (!quality.ok) { rejected++; continue; }
133
+ list.push({ key, q: quality.text });
134
+ }
135
+ if (!list.length) return { recorded: 0, rejected };
81
136
  const file = enrichmentFileFor(brainPath, home);
82
137
  fs.mkdirSync(path.dirname(file), { recursive: true });
83
138
  const data = readFile(file);
@@ -104,7 +159,7 @@ export function recordEnrichment(brainPath, items, { home = os.homedir(), now =
104
159
  const tmp = `${file}.tmp-${process.pid}`;
105
160
  fs.writeFileSync(tmp, JSON.stringify(data), 'utf8');
106
161
  fs.renameSync(tmp, file);
107
- return { recorded };
162
+ return { recorded, rejected };
108
163
  }
109
164
 
110
165
  /**
@@ -126,9 +181,14 @@ export function readEnrichment(brainPath, { home = os.homedir(), now = Date.now(
126
181
  const memo = readMemo.get(file);
127
182
  if (memo && memo.stamp === stamp) return memo.entries;
128
183
  const data = stamp ? readFile(file) : { entries: {} };
184
+ // The quality gate runs here too: a sidecar written before 1.86 (or by an
185
+ // older hook) is cleaned on read, so its acknowledgements and console echoes
186
+ // stop reaching the embedder without anyone rewriting the file. Entries left
187
+ // with no acceptable text are dropped from the served array entirely.
129
188
  const entries = Object.entries(data.entries)
130
189
  .filter(([, entry]) => now - Number(entry.ts || 0) <= ENRICHMENT_TTL_MS)
131
- .map(([key, entry]) => ({ key, q: (entry.q || []).map(cleanQuestion).filter(Boolean) }));
190
+ .map(([key, entry]) => ({ key, q: (entry.q || []).map((q) => enrichmentQuestionQuality(q)).filter((r) => r.ok).map((r) => r.text) }))
191
+ .filter((entry) => entry.q.length > 0);
132
192
  readMemo.set(file, { stamp, entries });
133
193
  if (readMemo.size > 8) readMemo.delete(readMemo.keys().next().value);
134
194
  return entries;
@@ -105,25 +105,37 @@ function parseEvidence(s) {
105
105
  const prepared = prepareBrainEvidence({ projectRoot: CWD, evidence: refs });
106
106
  return prepared.ok ? prepared.evidence : refs; // legacy malformed refs remain unverified
107
107
  }
108
- // Pull optional `closes:` / `ev:` / `verify:` suffixes off the END of a marker
109
- // body (any order), returning the cleaned body + parsed extras. The suffix
110
- // region starts at the first known key, so a decision can carry none, any, or
111
- // all without the keywords leaking into the card text. The three value regexes
112
- // must stay in LOCKSTEP: each value ends at the NEXT known key (or end of
113
- // line), so omitting a key from one lookahead silently folds "verify: …" into
114
- // that key's value — parseEvidence would mint junk file refs from it.
108
+ // Pull optional `closes:` / `ev:` / `verify:` / `q:` suffixes off the END of a
109
+ // marker body (any order), returning the cleaned body + parsed extras. The
110
+ // suffix region starts at the first known key, so a decision can carry none,
111
+ // any, or all without the keywords leaking into the card text. Every value
112
+ // ends at the NEXT known key (or end of line) through ONE shared lookahead
113
+ // built from the key list, so adding a key cannot silently fold "verify: …"
114
+ // into a neighbour's value — the lockstep bug three hand-written lookaheads
115
+ // used to invite (parseEvidence would mint junk file refs from it).
116
+ //
117
+ // `q:` (1.86) is the question this card ANSWERS, in the words someone would
118
+ // ask it. It is pattern-level retrieval enrichment authored by the model that
119
+ // knows the content — recorded in the sidecar beside the vector cache, never
120
+ // on the canvas — so a card is findable by a paraphrase nobody has typed yet,
121
+ // not only by the prompt that happened to precede its capture.
122
+ const MARKER_SUFFIX_KEYS = ['closes', 'ev', 'verify', 'q'];
123
+ const MARKER_SUFFIX_NEXT = `(?=${MARKER_SUFFIX_KEYS.map(k => `\\s+\\b${k}:`).join('|')}|$)`;
115
124
  function splitMarkerSuffixes(body) {
116
- const m = body.match(/\s+(?:closes|ev|verify):/i);
117
- if (!m) return { body, closes: '', evidence: null, verify: '' };
125
+ const m = body.match(new RegExp(`\\s+(?:${MARKER_SUFFIX_KEYS.join('|')}):`, 'i'));
126
+ if (!m) return { body, closes: '', evidence: null, verify: '', question: '' };
118
127
  const suffix = body.slice(m.index);
119
- const closesM = suffix.match(/\bcloses:\s*(.+?)\s*(?=\s+\bev:|\s+\bverify:|$)/i);
120
- const evM = suffix.match(/\bev:\s*(.+?)\s*(?=\s+\bcloses:|\s+\bverify:|$)/i);
121
- const verifyM = suffix.match(/\bverify:\s*(.+?)\s*(?=\s+\bcloses:|\s+\bev:|$)/i);
128
+ const grab = (key) => {
129
+ const mm = suffix.match(new RegExp(`\\b${key}:\\s*(.+?)\\s*${MARKER_SUFFIX_NEXT}`, 'i'));
130
+ return mm ? mm[1].trim() : '';
131
+ };
132
+ const ev = grab('ev');
122
133
  return {
123
134
  body: body.slice(0, m.index).trim(),
124
- closes: closesM ? closesM[1].trim() : '',
125
- evidence: evM ? parseEvidence(evM[1].trim()) : null,
126
- verify: verifyM ? verifyM[1].trim().slice(0, 200) : '',
135
+ closes: grab('closes'),
136
+ evidence: ev ? parseEvidence(ev) : null,
137
+ verify: grab('verify').slice(0, 200),
138
+ question: grab('q').slice(0, 240),
127
139
  };
128
140
  }
129
141
  // ── Self-healing brain (decision lifecycle, part 3) ──────────────────────────
@@ -2684,7 +2696,7 @@ async function capture(lib) {
2684
2696
  // Strip optional `closes:` / `ev:` / `verify:` suffixes off the body so
2685
2697
  // they don't leak into the card text; they drive the close-link,
2686
2698
  // evidence, and live-probe below.
2687
- const { body: cleanBody, closes, evidence, verify } = splitMarkerSuffixes(body);
2699
+ const { body: cleanBody, closes, evidence, verify, question: markerQuestion } = splitMarkerSuffixes(body);
2688
2700
  body = cleanBody; if (!body) continue;
2689
2701
  const preview = body.slice(0, 90);
2690
2702
  // EXAMPLE/doc guard — rejects marker-SYNTAX documentation (which
@@ -2741,7 +2753,12 @@ async function capture(lib) {
2741
2753
  // per-resolution outcome after the capture.
2742
2754
  if (type === '✓') { resolutions.push({ area, text: body }); ledger.push({ action: 'resolve', area, preview, rIdx: resolutions.length - 1 }); continue; }
2743
2755
  // ~ updates the matching card in place (small corrections).
2744
- if (type === '~') { updates.push({ area, text: body, createdVia: 'claude-code', ...(evidence ? { evidence } : {}), ...(verify ? { verify } : {}) }); ledger.push({ action: 'update', area, preview, ...(evidence ? { ev: evidence.map(e => e.ref) } : {}) }); continue; }
2756
+ if (type === '~') {
2757
+ updates.push({ area, text: body, createdVia: 'claude-code', ...(evidence ? { evidence } : {}), ...(verify ? { verify } : {}) });
2758
+ if (markerQuestion) enrichmentPairs.push({ body, question: markerQuestion });
2759
+ ledger.push({ action: 'update', area, preview, ...(evidence ? { ev: evidence.map(e => e.ref) } : {}) });
2760
+ continue;
2761
+ }
2745
2762
  // Type → scannable prefix + border color: ? open question (amber),
2746
2763
  // ! milestone (blue), + skill (violet), else decision (green). A plain
2747
2764
  // decision whose text reads as a reusable RULE is AUTO-promoted to a
@@ -2759,6 +2776,9 @@ async function capture(lib) {
2759
2776
  const card = (area ? `${area}: ${prefix}${body}` : `${prefix}${body}`) + (tagLine ? `\n${tagLine}` : '');
2760
2777
  cards.push({ text: card, area, borderColor, ...(closes ? { closes } : {}), ...(evidence ? { evidence } : {}), ...(verify ? { verify } : {}) });
2761
2778
  if (lastUserPrompt) enrichmentPairs.push({ body, question: lastUserPrompt });
2779
+ // The authored `q:` rides alongside the incidental prompt: the pattern
2780
+ // (what someone would ask) and the instance (what someone did type).
2781
+ if (markerQuestion) enrichmentPairs.push({ body, question: markerQuestion });
2762
2782
  ledger.push({ action: type === '?' ? 'add-question' : type === '!' ? 'add-milestone' : isSkill ? 'add-skill' : 'add-decision', area, preview, files: fileTags, ...(closes ? { closes } : {}), ...(evidence ? { ev: evidence.map(e => e.ref) } : {}) });
2763
2783
  }
2764
2784
  }
@@ -3561,7 +3581,10 @@ async function promptRetrieve(lib) {
3561
3581
  let hits = [], repeats = [], struct = null;
3562
3582
  if (tokens.length) {
3563
3583
  struct = await cachedStruct(lib);
3564
- hits = lib.scoreCardsAgainstQuery(struct, tokens, { topK: 5, minScore: 3 });
3584
+ // The lexical bar is the engine's measured export (1.86: 4 — one title
3585
+ // word alone no longer injects; see HOOK_LEXICAL_MIN_SCORE). An older
3586
+ // engine keeps the pre-1.86 bar of 3.
3587
+ hits = lib.scoreCardsAgainstQuery(struct, tokens, { topK: 5, minScore: Number.isFinite(lib.HOOK_LEXICAL_MIN_SCORE) ? lib.HOOK_LEXICAL_MIN_SCORE : 3 });
3565
3588
  // PRECISION-first repeat nudge ("you already did this in another session"):
3566
3589
  // only on a do/build request, only completed-work cards, only high confidence.
3567
3590
  // Matched on the PROMPT's stated intent (ptoks), not the git-diff fallback. A
@@ -3579,8 +3602,15 @@ async function promptRetrieve(lib) {
3579
3602
  // prompt never enters this lane → zero added latency. Bulletproof: not installed
3580
3603
  // / timeout / any failure → stays exactly today's pure-lexical behavior. The
3581
3604
  // helper is optional+deploy-gated, so a missing copy degrades cleanly to lexical.
3582
- let semMode = 'lexical';
3583
- if (!repeats.length && !freshHits.length && tokens.length && struct) {
3605
+ let semMode = 'lexical', semTop = null;
3606
+ // Prompt-side admission (1.86): the fallback is an UNCORROBORATED five-card
3607
+ // guess, and the measured cosines cannot tell an acknowledgement from a
3608
+ // real question — but the prompt can. Fewer than FALLBACK_MIN_CONTENT_TOKENS
3609
+ // content tokens ("build best in class", "continue where needed") skips the
3610
+ // lane; the git-diff file tokens do not count, they are not the asker's words.
3611
+ const fallbackEligible = typeof lib.lexicalMissFallbackEligible === 'function' ? lib.lexicalMissFallbackEligible(ptoks) : true;
3612
+ if (!repeats.length && !freshHits.length && tokens.length && struct && !fallbackEligible) semMode = 'sem-skipped';
3613
+ if (!repeats.length && !freshHits.length && tokens.length && struct && fallbackEligible) {
3584
3614
  try {
3585
3615
  const semlib = await import(new URL('./brain-semantic.mjs', import.meta.url).href);
3586
3616
  if (typeof semlib.semanticVecs === 'function') {
@@ -3590,20 +3620,38 @@ async function promptRetrieve(lib) {
3590
3620
  const sem = await semlib.semanticVecs(BRAIN, struct, humanText || '', { timeoutMs: 1500 });
3591
3621
  if (!sem) semMode = 'sem-unavailable';
3592
3622
  else {
3593
- const fresh = Date.now() - 30 * 86_400_000;
3594
- const ranked = struct.cards
3595
- .filter(c => c.type !== 'container' && (c.text || '').trim() && !/^archive$/i.test(c.area || ''))
3596
- .map(c => { const v = sem.vecsMap.get(c.id); return { card: c, s: v ? sem.dot(sem.qv, v) : null }; })
3597
- .filter(x => x.s != null && x.s >= 0.30) // miss-path floor: no lexical corroboration, so demand a real match
3598
- .map(x => { let score = x.s * 10; if ((x.card.createdAt || 0) >= fresh) score += 0.5; return { card: x.card, score }; })
3599
- .sort((a, b) => b.score - a.score).slice(0, 5);
3623
+ // ONE production primitive for this lane (1.86). The ranking
3624
+ // used to live inline here, which meant the surface every
3625
+ // session actually receives was the one no harness could
3626
+ // measure — an eval must IMPORT the ranker, never re-implement
3627
+ // it. rankLexicalMissFallback in klypix-format.mjs is that
3628
+ // primitive (scripts/eval-hook-lane.mjs measures it); its
3629
+ // defaults are the measured ones. An older engine without the
3630
+ // export degrades to the pre-1.86 inline ranking.
3631
+ let ranked;
3632
+ if (typeof lib.rankLexicalMissFallback === 'function') {
3633
+ const r = lib.rankLexicalMissFallback(struct, sem, { topK: 5 });
3634
+ ranked = r.hits; semTop = r.topCos;
3635
+ } else {
3636
+ const fresh = Date.now() - 30 * 86_400_000;
3637
+ ranked = struct.cards
3638
+ .filter(c => c.type !== 'container' && (c.text || '').trim() && !/^archive$/i.test(c.area || ''))
3639
+ .map(c => { const v = sem.vecsMap.get(c.id); return { card: c, s: v ? sem.dot(sem.qv, v) : null }; })
3640
+ .filter(x => x.s != null && x.s >= 0.30)
3641
+ .map(x => { let score = x.s * 10; if ((x.card.createdAt || 0) >= fresh) score += 0.5; return { card: x.card, score }; })
3642
+ .sort((a, b) => b.score - a.score).slice(0, 5);
3643
+ }
3600
3644
  freshHits = ranked;
3601
3645
  semMode = ranked.length ? 'sem-hit' : 'sem-empty';
3602
3646
  }
3603
3647
  }
3604
3648
  } catch { semMode = 'sem-error'; }
3605
- if (semMode !== 'lexical') { try { appendJsonl(HEALTH, { ts: nowIso(), project: path.basename(CWD), mode: 'prompt', sem: semMode, hits: freshHits.length }, 500); } catch { /* */ } }
3606
3649
  }
3650
+ // The health row carries the best cosine the lane saw (`top1`) so the
3651
+ // admission rule can be re-fitted from the field later without a
3652
+ // transcript — the 2026-09-16 audit had only "hits: 5" ×165 to go on. A
3653
+ // skipped lane is logged too (`sem-skipped`, with the content-token count).
3654
+ if (semMode !== 'lexical') { try { appendJsonl(HEALTH, { ts: nowIso(), project: path.basename(CWD), mode: 'prompt', sem: semMode, hits: freshHits.length, ...(Number.isFinite(semTop) ? { top1: Math.round(semTop * 1000) / 1000 } : {}), ...(semMode === 'sem-skipped' ? { content: ptoks.length } : {}) }, 500); } catch { /* */ } }
3607
3655
  // T8 STATUS-DIGEST INJECTION (2026-07-23): a status-shaped prompt gets the
3608
3656
  // COMPUTED current-state digest INSTEAD of card hits — so an agent that
3609
3657
  // never queried the brain still answers "what is remaining?" from state,
@@ -4104,7 +4152,7 @@ function legendFooter() {
4104
4152
  return '\n\n---\n'
4105
4153
  + '🧠 **Capture markers** — write these in your reply; the Stop hook harvests them into the brain (no separate log step). Use sparingly, for real decisions / milestones / discoveries:\n'
4106
4154
  + '`🧠 BRAIN [Area]: decision` · `[Area] ?: open question` · `[Area] !: milestone` · `[Area] +: 🛠️ skill (reusable how-to / gotcha — resurfaces every session, never ages out)` · `[Area] ✓: resolves+archives the matching card` · `[Area] ~: updates it in place` · 🎯 in text = a goal (reads as open).\n'
4107
- + 'Optional suffixes: `closes: <card title / [[wikilink]]>` (resolve the strategy/question this fulfils) · `ev: <file[:line]>, PR#<n>` (anchor to code → auto drift-badge).\n'
4155
+ + 'Optional suffixes: `closes: <card title / [[wikilink]]>` (resolve the strategy/question this fulfils) · `ev: <file[:line]>, PR#<n>` (anchor to code → auto drift-badge) · `q: <the question this answers, as someone would ASK it>` (retrieval enrichment — makes the card findable by paraphrase; sidecar only, never on the canvas).\n'
4108
4156
  + '**Correcting a stale card:** include the word `CORRECTION` (or "was WRONG" / "OBSOLETE" — UPPERCASE; casing is the deliberate-signal, casual prose never fires it) in the decision — the capture then hunts the stale card across ALL areas at a lower match bar and supersedes it (archived + arrowed, with a receipt; restore from Archive if it grabbed the wrong one). A rephrased duplicate `?` merges into the existing open question instead of stacking a twin.\n'
4109
4157
  + '**Verified-fix rule drafts:** when a session FIXES + VERIFIES something trap-shaped that landed as a one-off note, the Stop hook auto-DRAFTS a candidate 🛠️ rule (a per-project sidecar — never a brain card). Approve a real recurring trap with the `+` marker the nudge shows you and it becomes a standing rule that fires EVERY session (like the release-naming rule); ignore the rest and they age out. Draft-only, no blind auto-capture.\n'
4110
4158
  + '**Session brief:** the SessionStart hook prints a ≤2KB ultra brief and writes the FULL brief to `.claude/brain-brief.md` — read that file when planning non-trivial work.\n'
@@ -1622,10 +1622,15 @@ export async function opBrainNote({ vault, canvas, text: noteText, area, marker
1622
1622
  // natural-language question that produced this card — recorded to the
1623
1623
  // retrieval sidecar so brain_ask finds the card in the asker's vocabulary.
1624
1624
  // Additive: any failure costs recall, never the write above.
1625
- if (enrichmentQuestion && (res.stats?.added || 0) > 0) {
1625
+ // Accepts one string or a list (1.86: the caller's authored `question` plus
1626
+ // the declared intent); the sidecar's quality gate drops what carries no
1627
+ // askable vocabulary, so passing both never records junk.
1628
+ const enrichmentQuestions = (Array.isArray(enrichmentQuestion) ? enrichmentQuestion : [enrichmentQuestion])
1629
+ .map((q) => String(q || '').trim()).filter(Boolean);
1630
+ if (enrichmentQuestions.length && (res.stats?.added || 0) > 0) {
1626
1631
  try {
1627
1632
  const enrich = await import('./enrichment.mjs');
1628
- enrich.recordEnrichment(file, [{ body: noteText, question: enrichmentQuestion }]);
1633
+ enrich.recordEnrichment(file, enrichmentQuestions.map((question) => ({ body: noteText, question })));
1629
1634
  } catch { /* sidecar unavailable — additive signal only */ }
1630
1635
  }
1631
1636
  const s = res.stats || {};
@@ -2123,7 +2123,7 @@ export function structToUltraBrief(struct, { freshness = null, briefPath = '.cla
2123
2123
  const tail = [
2124
2124
  '',
2125
2125
  `📖 **Full brief: \`${briefPath}\`** — skills (${skills.length}), milestones, recent decisions, connections, self-heal detail. READ IT before planning non-trivial work.`,
2126
- '🧠 Capture: `🧠 BRAIN [Area]: <decision>` · `?` question · `!` milestone · `+` skill · `✓` resolve · `~` update · a "CORRECTION: …" decision supersedes its stale card across areas · suffixes `closes:` / `ev:` (full legend in the brief file).',
2126
+ '🧠 Capture: `🧠 BRAIN [Area]: <decision>` · `?` question · `!` milestone · `+` skill · `✓` resolve · `~` update · a "CORRECTION: …" decision supersedes its stale card across areas · suffixes `closes:` / `ev:` / `q: <question this answers>` (full legend in the brief file).',
2127
2127
  ];
2128
2128
  const budget = Math.max(400, budgetChars - tail.reduce((s, l) => s + l.length + 1, 0));
2129
2129
  push(`# ${struct.title} — brain (ultra brief)`);
@@ -2469,6 +2469,91 @@ export function scoreCardsAgainstQuery(struct, query, { topK = 6, minScore = 2,
2469
2469
  return scored.filter(s => s.score >= minScore).slice(0, topK);
2470
2470
  }
2471
2471
 
2472
+ // ── Lexical-miss semantic fallback — the per-prompt hook's second lane ───────
2473
+ // scoreCardsAgainstQuery (above) is what every prompt gets first. When it finds
2474
+ // nothing (paraphrase, no keyword overlap) the hook ranks the warm vector cache
2475
+ // against the prompt embedding and injects the top few. That ranking lived
2476
+ // INLINE in the hook until 1.86, so the one surface every session receives was
2477
+ // the one surface no harness could measure. It is a production primitive now:
2478
+ // the hook calls it, scripts/eval-hook-lane.mjs imports it, and the defaults
2479
+ // below are the measured ones, not guesses.
2480
+ //
2481
+ // FLOOR HISTORY. The lane shipped with an absolute cosine floor of 0.30
2482
+ // ("demand a real match"). Under the production contract (bge-small, CLS
2483
+ // pooling, query instruction prefix) cosines never go that low: field health
2484
+ // logs across every project on this machine (2026-09-16, 165 lexical-miss
2485
+ // prompts with a live model) show EXACTLY five cards injected every single
2486
+ // time — the floor never trimmed once. An absolute floor is also the wrong
2487
+ // shape: brain_ask's abstention probe measured true answers as low as 0.498
2488
+ // while out-of-domain top-1 reaches 0.683 (see rankForQuestion).
2489
+ //
2490
+ // MEASURED 2026-09-16 (scripts/eval-hook-lane.mjs, 110 real founder prompts
2491
+ // from this machine's enrichment sidecar, real 2,695-card brain): on the
2492
+ // prompts that reached this lane, the best cosine was 0.58–0.64 for
2493
+ // acknowledgements ("do full publish 100%") and 0.64 for the one real prompt
2494
+ // that missed lexically — NO cosine bar separates them (`minTop` 0.65 zeroes
2495
+ // junk but the sole real row sat at 0.642 with its gold at cosine rank 238).
2496
+ // So `minTop` and `margin` ship as null: the harness keeps sweeping them and
2497
+ // the hook's health row now records `top1` so the field can supply the
2498
+ // evidence this decision lacked. What DOES separate the two classes is the
2499
+ // PROMPT: acknowledgements carry 1–3 content tokens, real prompts 4+ — see
2500
+ // lexicalMissFallbackEligible below, which the hook applies before this lane.
2501
+ // `sem` is the hook's { qv, vecsMap, dot } from brain-semantic.semanticVecs.
2502
+ export function rankLexicalMissFallback(struct, sem, { topK = 5, floor = 0.30, minTop = null, margin = null, now = Date.now(), recentDays = 30 } = {}) {
2503
+ const empty = { hits: [], topCos: null, pool: 0 };
2504
+ if (!struct || !Array.isArray(struct.cards) || !sem || !sem.vecsMap || !sem.qv || typeof sem.dot !== 'function') return empty;
2505
+ const fresh = now - recentDays * 86_400_000;
2506
+ const pool = [];
2507
+ for (const c of struct.cards) {
2508
+ if (c.type === 'container' || !(c.text || '').trim() || /^archive$/i.test(c.area || '')) continue;
2509
+ const v = sem.vecsMap.get(c.id);
2510
+ if (!v) continue;
2511
+ const cos = sem.dot(sem.qv, v);
2512
+ if (!Number.isFinite(cos)) continue;
2513
+ pool.push({ card: c, cos, score: cos * 10 + ((c.createdAt || 0) >= fresh ? 0.5 : 0) });
2514
+ }
2515
+ if (!pool.length) return empty;
2516
+ let topCos = -Infinity;
2517
+ for (const x of pool) if (x.cos > topCos) topCos = x.cos;
2518
+ if (minTop != null && topCos < minTop) return { hits: [], topCos, pool: pool.length };
2519
+ const bar = Math.max(floor ?? -Infinity, margin != null ? topCos - margin : -Infinity);
2520
+ const hits = pool
2521
+ .filter(x => x.cos >= bar)
2522
+ .sort((a, b) => b.score - a.score || (b.card.createdAt || 0) - (a.card.createdAt || 0))
2523
+ .slice(0, Math.max(0, topK));
2524
+ return { hits, topCos, pool: pool.length };
2525
+ }
2526
+
2527
+ // ── The hook lane's two measured bars (1.86) ─────────────────────────────────
2528
+ // Both were swept on 2026-09-16 with scripts/eval-hook-lane.mjs over 110 real
2529
+ // founder prompts (35 capture-pairs with a gold card, 75 that should inject
2530
+ // nothing) against the real 2,695-card brain. The harness reads these exports,
2531
+ // so the number it measures is the number the hook ships.
2532
+ //
2533
+ // HOOK_LEXICAL_MIN_SCORE — scoreCardsAgainstQuery's bar for the per-prompt
2534
+ // lexical lane. A title/tag hit scores 3, a body hit ≤ 1, a standing skill +1:
2535
+ // minScore 3 (pre-1.86) real prompts: fired 97% · right card in top-5 29% · 4.6 cards
2536
+ // junk prompts: injected nothing 27% of the time
2537
+ // minScore 4 (ships) real prompts: fired 91% · right card in top-5 26% · 4.0 cards
2538
+ // junk prompts: injected nothing 69% of the time
2539
+ // minScore 5 real prompts: fired 74% · 20% · 2.4 cards · junk zero 81%
2540
+ // One title word is no longer enough ("do all best in class" used to inject a
2541
+ // card titled "…kill stale-closure CLASS…"); a title word plus corroboration,
2542
+ // or a 🛠️ skill, still is. The one-gold cost (29→26 on n=35) is inside noise.
2543
+ export const HOOK_LEXICAL_MIN_SCORE = 4;
2544
+ // FALLBACK_MIN_CONTENT_TOKENS — the semantic fallback only runs for a prompt
2545
+ // that carries at least this many content tokens (after stopwords and status
2546
+ // vocabulary). Measured: every junk prompt that reached the fallback had 1–3
2547
+ // ("build best in class", "continue where needed"); the real ones had 4+
2548
+ // ("i pressed restart and nothing appears , why"). Terse real prompts keep the
2549
+ // lexical lane and its git-diff fallback — they lose only an uncorroborated
2550
+ // five-card semantic guess that the cosine data says cannot tell junk apart.
2551
+ export const FALLBACK_MIN_CONTENT_TOKENS = 4;
2552
+ export function lexicalMissFallbackEligible(tokens, { minContentTokens = FALLBACK_MIN_CONTENT_TOKENS } = {}) {
2553
+ const n = Array.isArray(tokens) ? tokens.filter(t => typeof t === 'string' && t.trim()).length : 0;
2554
+ return n >= minContentTokens;
2555
+ }
2556
+
2472
2557
  // ── Ask-the-brain — whole-brain, correction-aware retrieval for a question ───
2473
2558
  // The surface a human actually uses ("what did we decide about X?", "where did
2474
2559
  // the auth work land?"). Distinct from the per-prompt recall hook (which injects