@jeffober/thatch 0.1.17 → 0.1.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +56 -2
- package/bin/thatch +73 -7
- package/docs/in-progress/prediction-engine.md +404 -0
- package/docs/qa/README.md +33 -1
- package/docs/qa/use-cases/README.md +4 -0
- package/docs/qa/use-cases/UC-021-prediction-autofire.md +21 -0
- package/docs/qa/use-cases/UC-022-prediction-dedup.md +19 -0
- package/docs/qa/use-cases/UC-023-prediction-confidence-model.md +23 -0
- package/docs/qa/use-cases/UC-024-prediction-delete.md +20 -0
- package/package.json +1 -1
- package/src/db.ts +480 -0
- package/src/extract-queue.ts +0 -7
- package/src/extraction.ts +18 -5
- package/src/index.ts +86 -28
- package/src/mcp.ts +2 -1
- package/src/prompts.ts +89 -5
- package/src/sideband.ts +51 -13
- package/src/tool-defs.ts +203 -0
package/README.md
CHANGED
|
@@ -7,8 +7,8 @@
|
|
|
7
7
|
Persistent memory and useful dev skills for AI coding agents that requires zero
|
|
8
8
|
manual config.
|
|
9
9
|
|
|
10
|
-
Works with **OpenCode** (as a plugin), **Claude Code** (as
|
|
11
|
-
**Cursor** (as a local `stdio` MCP server).
|
|
10
|
+
Works with **OpenCode** (as a plugin), **Claude Code** (as a local `stdio` MCP
|
|
11
|
+
server), and **Cursor** (as a local `stdio` MCP server).
|
|
12
12
|
|
|
13
13
|
Each session inherits the accumulated knowledge of every session before it —
|
|
14
14
|
project architecture, conventions, gotchas, user preferences — so your agent
|
|
@@ -98,6 +98,28 @@ When `thatch` is not on PATH, setup uses the absolute path to `bin/thatch` in
|
|
|
98
98
|
the hooks and MCP config. When it is on PATH (npm global install), setup uses
|
|
99
99
|
the bare `thatch` command so the config survives updates.
|
|
100
100
|
|
|
101
|
+
### Other MCP-compatible harnesses
|
|
102
|
+
|
|
103
|
+
Thatch works as a `stdio` MCP server with any MCP client. Register it in your
|
|
104
|
+
harness's MCP config by pointing to the `thatch` binary:
|
|
105
|
+
|
|
106
|
+
```json
|
|
107
|
+
{
|
|
108
|
+
"mcpServers": {
|
|
109
|
+
"thatch": {
|
|
110
|
+
"command": "thatch",
|
|
111
|
+
"args": ["mcp"]
|
|
112
|
+
}
|
|
113
|
+
}
|
|
114
|
+
}
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
No `setup` step is needed for bare MCP usage; `setup` installs hooks and
|
|
118
|
+
instructions specific to Claude Code and Cursor. For other harnesses, add the
|
|
119
|
+
MCP server config above and include thatch's tools/instructions in your agent's
|
|
120
|
+
system prompt manually (see [docs/dev/mcp-parity.md](docs/dev/mcp-parity.md)
|
|
121
|
+
for the prompt text). [Bun] must be on PATH.
|
|
122
|
+
|
|
101
123
|
## Tools
|
|
102
124
|
|
|
103
125
|
| Tool (opencode) | Tool (MCP) | What it does |
|
|
@@ -111,6 +133,36 @@ the bare `thatch` command so the config survives updates.
|
|
|
111
133
|
| `thatch_find_duplicates` | `mcp__thatch__find_duplicates` | Surface pairs of suspiciously similar memories |
|
|
112
134
|
| `thatch_dedup_mark_checked` | `mcp__thatch__dedup_mark_checked` | Record a reviewed pair so it stops being re-reported |
|
|
113
135
|
| `thatch_extraction_done` | `mcp__thatch__extraction_done` | Acknowledge extraction nudge was dispatched to a sub-agent; drains the buffer |
|
|
136
|
+
| `thatch_prediction_query` | `mcp__thatch__prediction_query` | Query the user decision model for scored predictions matching a context |
|
|
137
|
+
| `thatch_prediction_update` | `mcp__thatch__prediction_update` | Create, reinforce, or weaken a prediction about user preferences |
|
|
138
|
+
| `thatch_prediction_list` | `mcp__thatch__prediction_list` | List all predictions with matchers, confidence, evidence, and provenance |
|
|
139
|
+
| `thatch_prediction_delete` | `mcp__thatch__prediction_delete` | Delete a prediction created in error (cascades to edges and provenance) |
|
|
140
|
+
|
|
141
|
+
## Prediction engine
|
|
142
|
+
|
|
143
|
+
Thatch maintains a statistical model of the user's decision-making preferences.
|
|
144
|
+
When the user sends a prompt, the engine embeds it (sharing the embedding
|
|
145
|
+
already computed for memory recall) and searches for **matchers** — context
|
|
146
|
+
patterns that describe situations where the user has expressed preferences.
|
|
147
|
+
Matching matchers fire linked **predictions** (scored by cosine similarity,
|
|
148
|
+
edge weight, and Bayesian confidence), and a nudge is injected alongside the
|
|
149
|
+
recall nudge:
|
|
150
|
+
|
|
151
|
+
```text
|
|
152
|
+
[thatch] User decision model
|
|
153
|
+
- [0.72 conf, 4 tests] When X: you tend to Y
|
|
154
|
+
- [0.50 conf, 0 tests] When Z: you may prefer W
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
The nudge is descriptive, not directive. The system prompt instructions tell
|
|
158
|
+
the agent to follow strong predictions, surface uncertainty when confidence is
|
|
159
|
+
thin, and update the model when the user responds. Predictions use a Bayesian
|
|
160
|
+
posterior with a Beta prior (K=5, P0=0.5); signals are `confirm`,
|
|
161
|
+
`disconfirm`, `soft` (weak disconfirm, weight 0.25), and `create`.
|
|
162
|
+
|
|
163
|
+
The MCP path (Claude Code, Cursor) fires predictions via the same sideband
|
|
164
|
+
socket used for recall nudges. The `flush-predictions` CLI subcommand provides
|
|
165
|
+
standalone prediction-only output for testing.
|
|
114
166
|
|
|
115
167
|
## Skills
|
|
116
168
|
|
|
@@ -181,6 +233,7 @@ thatch hygiene Print the hygiene report (standalone)
|
|
|
181
233
|
thatch buffer-batch Append Claude Code PostToolBatch output to extraction queue
|
|
182
234
|
thatch buffer-tool Append Cursor postToolUse output to extraction queue
|
|
183
235
|
thatch flush-tools [--json] Peek extraction queue; print extraction + recall nudges
|
|
236
|
+
thatch flush-predictions [--json] Query predictions for the prompt (hook or standalone)
|
|
184
237
|
thatch setup --claude [--cursor] [--global] Install config + instructions + hooks + skills
|
|
185
238
|
```
|
|
186
239
|
|
|
@@ -239,6 +292,7 @@ payload for the fact-extractor skill.
|
|
|
239
292
|
| Session-start reminder | `session.created` event | `SessionStart` hook → `thatch reminder` | `sessionStart` hook → `thatch reminder --json` |
|
|
240
293
|
| Compaction context | `experimental.session.compacting` hook | **Gap**: no equivalent (CLAUDE.md persists) | **Gap**: no equivalent |
|
|
241
294
|
| Extraction nudge | `tool.execute.after` + `chat.message` | `PostToolBatch` + `UserPromptSubmit` (file-backed queue) | `postToolUse` + `beforeSubmitPrompt` (file-backed queue) |
|
|
295
|
+
| Prediction auto-fire | `chat.message` hook, separate try/catch from recall | `flush-tools` tier 2 (sideband socket) | `flush-tools` tier 2 (sideband socket) |
|
|
242
296
|
| Setup detection at startup | n/a (plugin auto-installs) | `checkSetup` in MCP server: detects missing/broken instructions | Same as Claude Code |
|
|
243
297
|
| Skills | 19 (18 shared + coordinator) | 18 (shared only) | 18 (shared only) |
|
|
244
298
|
| Store detection | `worktree` param from plugin | `CLAUDE_PROJECT_DIR` env var | `CURSOR_PROJECT_DIR` then `CLAUDE_PROJECT_DIR` |
|
package/bin/thatch
CHANGED
|
@@ -29,6 +29,7 @@ function usage(): never {
|
|
|
29
29
|
thatch buffer-batch Append a PostToolBatch payload to the queue (Claude Code hook only)
|
|
30
30
|
thatch buffer-tool Append a single postToolUse interaction to the queue (Cursor hook only)
|
|
31
31
|
thatch flush-tools [--json] Peek the queue + print an extraction nudge (hook only)
|
|
32
|
+
thatch flush-predictions [--json] Query predictions for the prompt (hook only)
|
|
32
33
|
thatch setup --claude [--cursor] [--global]
|
|
33
34
|
Install config + instructions + hooks + skills
|
|
34
35
|
|
|
@@ -351,7 +352,7 @@ switch (command) {
|
|
|
351
352
|
|
|
352
353
|
const { peekQueue, getMissedCount, incrementMissedCount } = await import("../src/extract-queue");
|
|
353
354
|
const { buildExtractionPayload } = await import("../src/extraction");
|
|
354
|
-
const { extractionNudge, claudeWriteNudge, claudeRecallNudge } = await import("../src/prompts");
|
|
355
|
+
const { extractionNudge, claudeWriteNudge, claudeRecallNudge, predictionNudge } = await import("../src/prompts");
|
|
355
356
|
|
|
356
357
|
let repo = "unknown";
|
|
357
358
|
try { repo = await detectRepo(); } catch {}
|
|
@@ -369,16 +370,31 @@ switch (command) {
|
|
|
369
370
|
break;
|
|
370
371
|
}
|
|
371
372
|
|
|
372
|
-
// Tier 2: recall nudge via sideband. The prompt text
|
|
373
|
-
// by host (Claude Code: "prompt", Cursor: undocumented).
|
|
373
|
+
// Tier 2: recall nudge + prediction fire via sideband. The prompt text
|
|
374
|
+
// field name varies by host (Claude Code: "prompt", Cursor: undocumented).
|
|
375
|
+
// Both recall and predictions share the same sideband round-trip.
|
|
374
376
|
const promptText = data.prompt ?? data.message ?? data.text ?? data.user_input ?? data.input;
|
|
375
377
|
if (promptText && promptText.length >= 10) {
|
|
376
|
-
const { sidebandMatch, sidebandSocketPath } = await import("../src/sideband");
|
|
378
|
+
const { sidebandMatch, sidebandPredictions, sidebandSocketPath } = await import("../src/sideband");
|
|
377
379
|
const sockPath = sidebandSocketPath(dbPath);
|
|
378
|
-
const
|
|
379
|
-
const
|
|
380
|
+
const recallThreshold = parseFloat(process.env.THATCH_RECALL_THRESHOLD ?? "0.55");
|
|
381
|
+
const predThreshold = parseFloat(process.env.THATCH_PREDICTION_THRESHOLD ?? "0.45");
|
|
382
|
+
|
|
383
|
+
const [matches, predictions] = await Promise.all([
|
|
384
|
+
sidebandMatch(sockPath, promptText, [repo, "global"], recallThreshold, 5),
|
|
385
|
+
sidebandPredictions(sockPath, promptText, [repo, "global"], predThreshold, 5),
|
|
386
|
+
]);
|
|
387
|
+
|
|
388
|
+
const parts: string[] = [];
|
|
380
389
|
if (matches && matches.length > 0) {
|
|
381
|
-
|
|
390
|
+
parts.push(claudeRecallNudge(matches));
|
|
391
|
+
}
|
|
392
|
+
if (predictions && predictions.length > 0) {
|
|
393
|
+
parts.push(predictionNudge(predictions));
|
|
394
|
+
}
|
|
395
|
+
|
|
396
|
+
if (parts.length > 0) {
|
|
397
|
+
const output = parts.join("\n\n");
|
|
382
398
|
if (isJson) console.log(JSON.stringify({ additional_context: output }));
|
|
383
399
|
else console.log(output);
|
|
384
400
|
break;
|
|
@@ -407,6 +423,56 @@ switch (command) {
|
|
|
407
423
|
break;
|
|
408
424
|
}
|
|
409
425
|
|
|
426
|
+
case "flush-predictions": {
|
|
427
|
+
// UserPromptSubmit (Claude Code) / beforeSubmitPrompt (Cursor) hook.
|
|
428
|
+
// Standalone prediction fire: queries the sideband for scored
|
|
429
|
+
// predictions matching the prompt text, prints the prediction nudge
|
|
430
|
+
// if any matchers fire. --json wraps as { additional_context: "..." }
|
|
431
|
+
// for Cursor hooks. Without --json, stdout is plain text.
|
|
432
|
+
//
|
|
433
|
+
// NOTE: flush-tools already fires predictions alongside the recall
|
|
434
|
+
// nudge in its tier 2 block. This subcommand exists for testing and
|
|
435
|
+
// for hosts that want prediction-only output without the recall or
|
|
436
|
+
// extraction nudges.
|
|
437
|
+
const isJson = args.includes("--json");
|
|
438
|
+
try {
|
|
439
|
+
const raw = await Bun.stdin.text();
|
|
440
|
+
const data = JSON.parse(raw) as {
|
|
441
|
+
prompt?: string;
|
|
442
|
+
message?: string;
|
|
443
|
+
text?: string;
|
|
444
|
+
user_input?: string;
|
|
445
|
+
input?: string;
|
|
446
|
+
};
|
|
447
|
+
|
|
448
|
+
const promptText = data.prompt ?? data.message ?? data.text ?? data.user_input ?? data.input;
|
|
449
|
+
if (!promptText || promptText.length < 10) {
|
|
450
|
+
if (isJson) console.log(JSON.stringify({ additional_context: "" }));
|
|
451
|
+
break;
|
|
452
|
+
}
|
|
453
|
+
|
|
454
|
+
let repo = "unknown";
|
|
455
|
+
try { repo = await detectRepo(); } catch {}
|
|
456
|
+
|
|
457
|
+
const { sidebandPredictions, sidebandSocketPath } = await import("../src/sideband");
|
|
458
|
+
const { predictionNudge } = await import("../src/prompts");
|
|
459
|
+
const sockPath = sidebandSocketPath(dbPath);
|
|
460
|
+
const threshold = parseFloat(process.env.THATCH_PREDICTION_THRESHOLD ?? "0.45");
|
|
461
|
+
const predictions = await sidebandPredictions(sockPath, promptText, [repo, "global"], threshold, 5);
|
|
462
|
+
|
|
463
|
+
if (predictions && predictions.length > 0) {
|
|
464
|
+
const output = predictionNudge(predictions);
|
|
465
|
+
if (isJson) console.log(JSON.stringify({ additional_context: output }));
|
|
466
|
+
else console.log(output);
|
|
467
|
+
} else {
|
|
468
|
+
if (isJson) console.log(JSON.stringify({ additional_context: "" }));
|
|
469
|
+
}
|
|
470
|
+
} catch (err) {
|
|
471
|
+
console.error(`[thatch flush-predictions] failed: ${err}`);
|
|
472
|
+
}
|
|
473
|
+
break;
|
|
474
|
+
}
|
|
475
|
+
|
|
410
476
|
default:
|
|
411
477
|
usage();
|
|
412
478
|
}
|
|
@@ -0,0 +1,404 @@
|
|
|
1
|
+
# Prediction engine (in progress)
|
|
2
|
+
|
|
3
|
+
> **Status: implementation complete across both host paths.** The
|
|
4
|
+
> schema, tools, auto-fire (opencode), sideband prediction fire (MCP),
|
|
5
|
+
> prompt instructions, and tests are shipped and passing the quality
|
|
6
|
+
> gate. The feature is ready for field testing. This doc is the live
|
|
7
|
+
> ledger; when the work closes, graduate durable parts into a permanent
|
|
8
|
+
> `docs/dev/prediction-engine.md` and retire this file.
|
|
9
|
+
|
|
10
|
+
## Purpose
|
|
11
|
+
|
|
12
|
+
A **user decision model** that learns the user's preferences for
|
|
13
|
+
judgment calls the coding agent faces every session: scope (how much to
|
|
14
|
+
do), appropriateness (is this the right thing), methodology (the right
|
|
15
|
+
way). The agent gets a "guess what the user would want in this
|
|
16
|
+
situation" system -- inspired by nak's samskara predictive model, but
|
|
17
|
+
different mechanism and domain.
|
|
18
|
+
|
|
19
|
+
The delta over flat thatch memory is three things flat memory cannot do:
|
|
20
|
+
|
|
21
|
+
1. **Graded confidence** -- memory is binary (stored or not).
|
|
22
|
+
Predictions have a Bayesian posterior that is reinforced or weakened
|
|
23
|
+
by user feedback. This enables ambiguity-surfacing: when confidence
|
|
24
|
+
is thin or predictions conflict, the agent asks the user instead of
|
|
25
|
+
guessing.
|
|
26
|
+
2. **Context-disambiguation** -- matchers separate "when" from "what."
|
|
27
|
+
The same topic (tech debt in a PR) can produce opposite predictions
|
|
28
|
+
in different situations (flaky test blocking CI = fix it; unrelated
|
|
29
|
+
to change = skip it). Flat memory encodes this in prose the LLM must
|
|
30
|
+
re-reason about every time; the prediction engine encodes it as
|
|
31
|
+
structure.
|
|
32
|
+
3. **Agent-driven feedback loop** -- the LLM itself decides to
|
|
33
|
+
strengthen or weaken a prediction based on user feedback, creating a
|
|
34
|
+
loop that flat memory lacks.
|
|
35
|
+
|
|
36
|
+
## Design decisions (settled)
|
|
37
|
+
|
|
38
|
+
These came out of design discussion with Jeff, July 2026. They are
|
|
39
|
+
constraints, not open questions.
|
|
40
|
+
|
|
41
|
+
1. **Auto-fire at chat.message.** The hook already embeds the user's
|
|
42
|
+
text for the recall nudge. The prediction fire shares that embedding
|
|
43
|
+
and does a second cosine search against the matchers table. When
|
|
44
|
+
matchers clear a relevance floor, scored predictions are injected as
|
|
45
|
+
a synthetic nudge part. No matchers fire = nothing injected (zero
|
|
46
|
+
tokens, zero latency). No "if appropriate" hedging -- the block's
|
|
47
|
+
presence is the appropriateness signal.
|
|
48
|
+
|
|
49
|
+
2. **Tool-based update, agent-driven.** No local LLM classifier. Prompt
|
|
50
|
+
instructions tell the agent: when the user corrects you, answers a
|
|
51
|
+
question, or provides a clear signal, use prediction_update to
|
|
52
|
+
create/reinforce/weaken a prediction. The LLM decides with
|
|
53
|
+
appropriately nuanced prompt instructions; the tool is the trusted
|
|
54
|
+
boundary that handles embedding, dedup, and confidence math.
|
|
55
|
+
|
|
56
|
+
3. **Statistical model query, not AI.** The query (embed + cosine +
|
|
57
|
+
weighted edges) is mechanical -- same as thatch_memory_recall but
|
|
58
|
+
against different tables. No model call. The agent can also call
|
|
59
|
+
prediction_query explicitly when it wants to check the model before a
|
|
60
|
+
judgment call.
|
|
61
|
+
|
|
62
|
+
4. **No local LLM.** Considered and declined. The host LLM is already
|
|
63
|
+
reading the message and classifying "was this a correction?" as part
|
|
64
|
+
of understanding it. Adding a second classifier pays twice for the
|
|
65
|
+
same answer. The cost of not having a classifier is wasted nudge text
|
|
66
|
+
on turns with no signal -- not wasted model calls.
|
|
67
|
+
|
|
68
|
+
5. **Descriptive context, not directive.** The injected block reads
|
|
69
|
+
"[0.72 conf, 4 tests] When X: you tend to Y" -- same shape as
|
|
70
|
+
samskara's compound summary. The system prompt instructions govern
|
|
71
|
+
how to act on it (follow strong silently, surface weak/competing,
|
|
72
|
+
update on user response). No "if appropriate" or "consider" hedging
|
|
73
|
+
in the injection itself.
|
|
74
|
+
|
|
75
|
+
6. **Confidence mirrors samskara health.** A Bayesian posterior with
|
|
76
|
+
population-prior shrinkage: `confidence = (confirm + k*p0) /
|
|
77
|
+
(confirm + disconfirm + k)`. No wall-clock decay -- relevance-gated,
|
|
78
|
+
like samskara. Being tested (user gives feedback) is what moves
|
|
79
|
+
confidence. Untested predictions stay at p0.
|
|
80
|
+
|
|
81
|
+
7. **LLM is the conflict detector.** The auto-fire returns all
|
|
82
|
+
predictions from matching matchers, not just the top one. The LLM
|
|
83
|
+
reads competing predictions and decides whether they conflict. No
|
|
84
|
+
mechanical conflict heuristic -- the LLM is better at semantic
|
|
85
|
+
conflict detection than any rule.
|
|
86
|
+
|
|
87
|
+
## The normative asymmetry (carried from samskara)
|
|
88
|
+
|
|
89
|
+
Samskara's intents doc identifies a normative asymmetry: the descriptive
|
|
90
|
+
layer (samskara predictions) is ethically free to surface, but the
|
|
91
|
+
normative layer (intents -- "what the model is working toward") carries
|
|
92
|
+
weight because it shapes behavior. The prediction engine is descriptive
|
|
93
|
+
("you tend to prefer X"), not normative ("I'm working toward X with
|
|
94
|
+
you"). It reads user behavior, it does not pursue an agenda. The
|
|
95
|
+
firewall from samskara's efficacy model carries over: the model that
|
|
96
|
+
uses predictions cannot grade its own predictions -- the user's
|
|
97
|
+
feedback is the only signal that moves confidence.
|
|
98
|
+
|
|
99
|
+
## Data model
|
|
100
|
+
|
|
101
|
+
Four new tables in the same SQLite DB, same conventions as the existing
|
|
102
|
+
`entries`/`dedup_pairs` tables (idempotent CREATE TABLE IF NOT EXISTS,
|
|
103
|
+
BLOB embeddings, store column, crypto.randomUUID PKs).
|
|
104
|
+
|
|
105
|
+
### `prediction_matchers`
|
|
106
|
+
|
|
107
|
+
Context patterns describing situations. The "when."
|
|
108
|
+
|
|
109
|
+
- `id TEXT PRIMARY KEY` (crypto.randomUUID).
|
|
110
|
+
- `store TEXT NOT NULL`.
|
|
111
|
+
- `description TEXT NOT NULL` -- "reviewing a PR with pre-existing tech
|
|
112
|
+
debt."
|
|
113
|
+
- `embedding BLOB` -- 384-dim Float32Array, passageEmbed (no query
|
|
114
|
+
prefix; matchers are passages, not queries).
|
|
115
|
+
- `model TEXT`.
|
|
116
|
+
- `created_at`, `updated_at` -- ISO timestamps.
|
|
117
|
+
|
|
118
|
+
### `predictions`
|
|
119
|
+
|
|
120
|
+
User preference statements. The "what."
|
|
121
|
+
|
|
122
|
+
- `id TEXT PRIMARY KEY`.
|
|
123
|
+
- `store TEXT NOT NULL`.
|
|
124
|
+
- `statement TEXT NOT NULL` -- "skip it unless it's blocking this
|
|
125
|
+
change."
|
|
126
|
+
- `rationale TEXT` -- why this prediction exists (inspector context).
|
|
127
|
+
- `embedding BLOB` -- for dedup at creation time.
|
|
128
|
+
- `model TEXT`.
|
|
129
|
+
- `confidence REAL NOT NULL DEFAULT 0.5` -- Bayesian posterior [0,1].
|
|
130
|
+
- `confirm_count REAL NOT NULL DEFAULT 0` -- fractional (soft signals
|
|
131
|
+
are sub-unit).
|
|
132
|
+
- `disconfirm_count REAL NOT NULL DEFAULT 0` -- same.
|
|
133
|
+
- `created_at`, `updated_at`.
|
|
134
|
+
|
|
135
|
+
### `prediction_edges`
|
|
136
|
+
|
|
137
|
+
Weighted matcher-to-prediction links. The "context-dependence."
|
|
138
|
+
|
|
139
|
+
- `matcher_id TEXT NOT NULL` (FK to prediction_matchers, CASCADE).
|
|
140
|
+
- `prediction_id TEXT NOT NULL` (FK to predictions, CASCADE).
|
|
141
|
+
- `weight REAL NOT NULL DEFAULT 1.0`.
|
|
142
|
+
- `created_at`.
|
|
143
|
+
- `PRIMARY KEY (matcher_id, prediction_id)`.
|
|
144
|
+
|
|
145
|
+
### `prediction_provenance`
|
|
146
|
+
|
|
147
|
+
Audit trail. Gives the inspector context and the agent "plenty of
|
|
148
|
+
context" for nuance.
|
|
149
|
+
|
|
150
|
+
- `id TEXT PRIMARY KEY`.
|
|
151
|
+
- `prediction_id TEXT NOT NULL` (FK to predictions, CASCADE).
|
|
152
|
+
- `signal TEXT NOT NULL` -- 'confirm' | 'disconfirm' | 'soft' |
|
|
153
|
+
'create'.
|
|
154
|
+
- `detail TEXT` -- what the user said or did.
|
|
155
|
+
- `created_at`.
|
|
156
|
+
|
|
157
|
+
## Confidence model
|
|
158
|
+
|
|
159
|
+
Mirrors samskara's health posterior with population-prior shrinkage.
|
|
160
|
+
|
|
161
|
+
```
|
|
162
|
+
confidence = (confirm_count + k * p0) / (confirm_count + disconfirm_count + k)
|
|
163
|
+
|
|
164
|
+
k = 5 (prior strength, pseudo-count)
|
|
165
|
+
p0 = 0.5 (population baseline; make population-derived later)
|
|
166
|
+
W_SOFT = 0.25 (soft miss weight, same as samskara)
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
- `confirm` signal: confirm_count += 1.
|
|
170
|
+
- `disconfirm` signal: disconfirm_count += 1.
|
|
171
|
+
- `soft` signal: disconfirm_count += W_SOFT (0.25).
|
|
172
|
+
- `create` signal: no confidence adjustment, just seeds at p0.
|
|
173
|
+
- No wall-clock decay. Relevance-gated: being tested (user gives
|
|
174
|
+
feedback) is what moves confidence. Untested = stays at p0.
|
|
175
|
+
- confirm_count / disconfirm_count are REAL (fractional). An integer
|
|
176
|
+
column would truncate soft signals to 0 (samskara's bug).
|
|
177
|
+
- p0 starts at 0.5 for v1. Later: `populationP0(store)` = sum(confirm) /
|
|
178
|
+
sum(confirm + disconfirm), with a 0.5 fallback under 20 evidence.
|
|
179
|
+
|
|
180
|
+
## Scoring at query time
|
|
181
|
+
|
|
182
|
+
```
|
|
183
|
+
score = cosine(user_text, matcher.embedding) * edge.weight * prediction.confidence
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
The auto-fire returns all predictions from matching matchers (not just
|
|
187
|
+
the top scoring one), so the LLM can read conflicts. The score is for
|
|
188
|
+
ranking and display, not for a mechanical "follow if above X" gate --
|
|
189
|
+
the LLM decides whether to follow, surface, or ignore, informed by the
|
|
190
|
+
confidence and evidence count shown alongside.
|
|
191
|
+
|
|
192
|
+
## Ambiguity-surfacing (the killer feature)
|
|
193
|
+
|
|
194
|
+
- **Strong prediction** (high confidence, sufficient evidence) -> agent
|
|
195
|
+
follows silently, does not interrupt.
|
|
196
|
+
- **No prediction** -> agent asks the user (same as today).
|
|
197
|
+
- **Ambiguous or competing predictions** -> agent surfaces naturally:
|
|
198
|
+
"I think you usually prefer X here, but I'm not sure -- what do you
|
|
199
|
+
want?"
|
|
200
|
+
- **User responds to a surfaced prediction** -> agent calls
|
|
201
|
+
prediction_update to reinforce or weaken.
|
|
202
|
+
|
|
203
|
+
Value = fewer interruptions, not more. High-confidence predictions
|
|
204
|
+
suppress questions the agent would otherwise ask.
|
|
205
|
+
|
|
206
|
+
## Tool surface
|
|
207
|
+
|
|
208
|
+
Four tools, following the existing ToolDef pattern in tool-defs.ts.
|
|
209
|
+
|
|
210
|
+
### `prediction_query`
|
|
211
|
+
|
|
212
|
+
Mechanical search. Returns scored predictions with matcher
|
|
213
|
+
description, confidence, and evidence count.
|
|
214
|
+
|
|
215
|
+
Args: `context` (string), `store` (optional).
|
|
216
|
+
|
|
217
|
+
### `prediction_update`
|
|
218
|
+
|
|
219
|
+
The trusted boundary. The agent says "here's the situation, here's the
|
|
220
|
+
preference, here's what happened." The tool handles: embed matcher,
|
|
221
|
+
find/create by cosine dedup, embed prediction, find/create by cosine
|
|
222
|
+
dedup, adjust confidence, record provenance.
|
|
223
|
+
|
|
224
|
+
Args: `matcher` (string), `prediction` (string), `signal` (confirm |
|
|
225
|
+
disconfirm | soft | create), `rationale` (string), `store` (optional).
|
|
226
|
+
|
|
227
|
+
### `prediction_list`
|
|
228
|
+
|
|
229
|
+
Inspector. Lists all predictions with confidence, evidence, matchers,
|
|
230
|
+
and rationale.
|
|
231
|
+
|
|
232
|
+
Args: `store` (optional).
|
|
233
|
+
|
|
234
|
+
## Auto-fire in chat.message hook
|
|
235
|
+
|
|
236
|
+
Slots into the existing recall-nudge path in index.ts, sharing the
|
|
237
|
+
embedding call. After the recall nudge block, before the catch:
|
|
238
|
+
|
|
239
|
+
```
|
|
240
|
+
embedding = await model.queryEmbed(promptText) // already computed
|
|
241
|
+
|
|
242
|
+
// recall nudge (existing)
|
|
243
|
+
results = db.search([repo, "global"], embedding, { limit: 5 })
|
|
244
|
+
if matches >= RECALL_THRESHOLD -> inject recallNudge
|
|
245
|
+
|
|
246
|
+
// prediction fire (new)
|
|
247
|
+
predItems = db.scorePredictionNudge([repo, "global"], embedding, PREDICTION_THRESHOLD)
|
|
248
|
+
if predItems.length > 0:
|
|
249
|
+
output.parts.push(predictionNudge(predItems)) // synthetic: true
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
One more cosine scan against a separate table. Same brute-force
|
|
253
|
+
pattern, same cost profile. No extra embedding call.
|
|
254
|
+
|
|
255
|
+
## System prompt instructions
|
|
256
|
+
|
|
257
|
+
New section in all three prompt variants (systemPrompt,
|
|
258
|
+
claudeInstructions, cursorInstructions), after "Before Responding."
|
|
259
|
+
|
|
260
|
+
New tools registered in the tool list line:
|
|
261
|
+
`thatch_prediction_query`, `thatch_prediction_update`,
|
|
262
|
+
`thatch_prediction_list` (opencode); bare names for MCP.
|
|
263
|
+
|
|
264
|
+
The injected block format:
|
|
265
|
+
|
|
266
|
+
```
|
|
267
|
+
## User decision model
|
|
268
|
+
- [0.72 conf, 4 tests] When reviewing a PR with pre-existing tech debt: you tend to skip it unless it's blocking the change
|
|
269
|
+
- [0.61 conf, 2 tests] When the tech debt is a flaky test: you've preferred fixing it
|
|
270
|
+
```
|
|
271
|
+
|
|
272
|
+
Prompt instructions:
|
|
273
|
+
- Follow strong predictions silently.
|
|
274
|
+
- Surface weak or competing predictions to the user naturally.
|
|
275
|
+
- When the user responds to a surfaced prediction, use
|
|
276
|
+
prediction_update to reinforce or weaken.
|
|
277
|
+
- When the user corrects you, answers a question, or provides a clear
|
|
278
|
+
signal, use prediction_query then prediction_update.
|
|
279
|
+
|
|
280
|
+
## Constants
|
|
281
|
+
|
|
282
|
+
```
|
|
283
|
+
PREDICTION_THRESHOLD = 0.45 // lower than RECALL_THRESHOLD (0.55)
|
|
284
|
+
MATCHER_DEDUP_COSINE = 0.85 // near-duplicate check at creation
|
|
285
|
+
PREDICTION_DEDUP_COSINE = 0.85
|
|
286
|
+
P0 = 0.5 // prior (population-derived later)
|
|
287
|
+
K = 5 // prior strength
|
|
288
|
+
W_SOFT = 0.25 // soft miss weight
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
## Hook integration
|
|
292
|
+
|
|
293
|
+
The `tool.execute.after` hook (index.ts) already skips
|
|
294
|
+
`thatch_*`-prefixed tools from the extraction buffer.
|
|
295
|
+
`thatch_prediction_*` is caught by that check -- no special handling
|
|
296
|
+
needed. MCP path same: `mcp__thatch__prediction_*` caught by the
|
|
297
|
+
`mcp__thatch__*` filter in extract-queue.ts.
|
|
298
|
+
|
|
299
|
+
Prediction calls do NOT drain the extraction buffer -- they are a
|
|
300
|
+
separate system. The agent may do both in the same turn (save a memory
|
|
301
|
+
AND update a prediction).
|
|
302
|
+
|
|
303
|
+
## Build status
|
|
304
|
+
|
|
305
|
+
Landed so far:
|
|
306
|
+
|
|
307
|
+
1. DB schema + methods (db.ts) -- four new tables (prediction_matchers,
|
|
308
|
+
predictions, prediction_edges, prediction_provenance) in #initSchema,
|
|
309
|
+
plus findMatchers, scorePredictions, findNearestMatcher,
|
|
310
|
+
findNearestPrediction, createMatcher, createPrediction, createEdge,
|
|
311
|
+
adjustConfidence, addProvenance, listPredictions, populationP0,
|
|
312
|
+
getPrediction, getProvenance, deletePrediction, scorePredictionNudge.
|
|
313
|
+
Confidence uses Bayesian posterior with k=5, p0=0.5, W_SOFT=0.25.
|
|
314
|
+
2. Tool definitions (tool-defs.ts) -- prediction_query,
|
|
315
|
+
prediction_update, prediction_list, prediction_delete. Added to
|
|
316
|
+
TOOL_DEFS array (9 -> 13).
|
|
317
|
+
3. Auto-fire (index.ts) -- prediction fire in chat.message hook,
|
|
318
|
+
sharing embedding with recall nudge. Prediction blocks injected
|
|
319
|
+
when matchers clear PREDICTION_THRESHOLD (0.45). predictionNudge
|
|
320
|
+
formatter in prompts.ts.
|
|
321
|
+
4. System prompt (prompts.ts) -- "User Decision Model" section in all
|
|
322
|
+
three variants (systemPrompt, claudeInstructions,
|
|
323
|
+
cursorInstructions). New tools in all three tool lists.
|
|
324
|
+
5. Tools wiring (tools.ts + mcp.ts) -- automatic via TOOL_DEFS
|
|
325
|
+
iteration. No manual changes needed.
|
|
326
|
+
6. Tests -- tests/prediction.test.ts (schema, confidence math, dedup,
|
|
327
|
+
scoring, provenance, listPredictions, populationP0). Updated
|
|
328
|
+
tool-defs.test.ts and plugin.test.ts tool count assertions (9 -> 13).
|
|
329
|
+
7. MCP path (sideband.ts + bin/thatch) -- the sideband server now
|
|
330
|
+
handles `method: "predictions"` alongside `method: "match"`, so
|
|
331
|
+
the warm MCP server can query matchers and score predictions for
|
|
332
|
+
one-shot hook processes. The `flush-tools` CLI subcommand (already
|
|
333
|
+
wired into UserPromptSubmit hooks for Claude Code and Cursor) now
|
|
334
|
+
fires predictions alongside the recall nudge in its tier 2 block,
|
|
335
|
+
sharing one sideband round-trip via `Promise.all`. A standalone
|
|
336
|
+
`flush-predictions` subcommand is also available for testing.
|
|
337
|
+
No new hook installation needed -- the existing `flush-tools` hook
|
|
338
|
+
covers it.
|
|
339
|
+
|
|
340
|
+
Future enhancements:
|
|
341
|
+
|
|
342
|
+
1. **p0 population derivation** -- v1 uses flat 0.5. Once enough
|
|
343
|
+
predictions accumulate, switch to population-derived p0 per store.
|
|
344
|
+
2. **Auto-fire block separation** -- separate synthetic part from the
|
|
345
|
+
recall nudge, or combined? Starting with separate (simpler).
|
|
346
|
+
3. **Inspector UI** -- `thatch prediction list` CLI subcommand for
|
|
347
|
+
terminal inspection. No Svelte UI (thatch has no UI).
|
|
348
|
+
4. **Score formula tuning** -- v1 is `cosine * weight * confidence`.
|
|
349
|
+
If confidence dominates (high confidence on weak cosine swamps low
|
|
350
|
+
confidence on strong cosine), add samskara's `cosine^1.3` power.
|
|
351
|
+
Start simple; let data drive.
|
|
352
|
+
5. **Edge weight tuning** -- v1 is flat 1.0 for all edges. The weight
|
|
353
|
+
encodes context-dependence but we have no signal to set it yet.
|
|
354
|
+
Could later set it from the matcher's cosine at creation time
|
|
355
|
+
(stronger match = higher weight) or from co-occurrence statistics.
|
|
356
|
+
|
|
357
|
+
## Interactions
|
|
358
|
+
|
|
359
|
+
- **Memory** (existing) -- distinct system. Memory is facts the
|
|
360
|
+
agent/user committed; predictions are learned preferences with graded
|
|
361
|
+
confidence. No data flows between them. The agent may write a memory
|
|
362
|
+
about a preference AND update a prediction about the same preference;
|
|
363
|
+
the memory is the "what" (durable fact), the prediction is the "how
|
|
364
|
+
confident" (graded, updated by feedback).
|
|
365
|
+
- **Extraction pipeline** (existing) -- prediction tools are excluded
|
|
366
|
+
from the extraction buffer by the thatch_* prefix filter. No
|
|
367
|
+
feedback loop.
|
|
368
|
+
- **Recall nudge** (existing) -- the prediction fire shares the
|
|
369
|
+
embedding call and runs in the same chat.message hook path (opencode)
|
|
370
|
+
or the same sideband round-trip (MCP). The two nudges are separate
|
|
371
|
+
synthetic parts (opencode) or combined in one output block (MCP);
|
|
372
|
+
either can be absent.
|
|
373
|
+
- **Sideband** (existing, MCP path) -- the sideband server now handles
|
|
374
|
+
a `predictions` method alongside `match`. The `flush-tools` CLI
|
|
375
|
+
subcommand queries both in parallel via `Promise.all`.
|
|
376
|
+
- **Compaction guard** (existing) -- the compacting set check at the
|
|
377
|
+
top of chat.message suppresses ALL nudges, including prediction fire.
|
|
378
|
+
No change needed.
|
|
379
|
+
|
|
380
|
+
## Gotchas
|
|
381
|
+
|
|
382
|
+
- **Matcher embeddings use passageEmbed, not queryEmbed.** Matchers
|
|
383
|
+
are passages (stored content matched against), not queries. The query
|
|
384
|
+
prefix applies to the user's text at fire time. Same pattern as
|
|
385
|
+
remember() which uses passageEmbed for content.
|
|
386
|
+
- **confirm_count / disconfirm_count MUST be REAL.** An integer column
|
|
387
|
+
truncates soft signals (W_SOFT = 0.25) to 0 and freezes confidence at
|
|
388
|
+
p0. This is the same bug samskara had.
|
|
389
|
+
- **Cold start is free.** Empty matchers table, findMatchers returns
|
|
390
|
+
[], nothing injected. The feature is inert until the first
|
|
391
|
+
prediction_update call. No migration, no config.
|
|
392
|
+
- **Prediction calls do not drain the extraction buffer.** They are a
|
|
393
|
+
separate system. The agent may do both in the same turn.
|
|
394
|
+
|
|
395
|
+
## Reference
|
|
396
|
+
|
|
397
|
+
- nak samskara: `docs/dev/samskara.md` in the nak repo (the predictive
|
|
398
|
+
model of the user this feature is inspired by).
|
|
399
|
+
- nak intents: `docs/dev/in-progress/intents.md` in the nak repo (the
|
|
400
|
+
normative layer atop samskara; the C efficacy model and firewall
|
|
401
|
+
between employment and efficacy informed the confidence model here).
|
|
402
|
+
- thatch memory: this doc's "Interactions" section.
|
|
403
|
+
- thatch extraction pipeline: `docs/dev/README.md` + `docs/dev/mcp-parity.md`.
|
|
404
|
+
- thatch hook surface: `docs/dev/setup-and-hooks.md`, `docs/dev/gotchas.md`.
|
package/docs/qa/README.md
CHANGED
|
@@ -91,7 +91,7 @@ correctness on `README.md` and `docs/` excluding `docs/plans/`; see
|
|
|
91
91
|
|
|
92
92
|
### `tests/plugin.test.ts`
|
|
93
93
|
|
|
94
|
-
- `server` export shape: all
|
|
94
|
+
- `server` export shape: all thirteen tools, every hook present, tools carry
|
|
95
95
|
description/args/execute
|
|
96
96
|
- System transform and compaction hooks append their text
|
|
97
97
|
- `tool.execute.after` → `chat.message` extraction round-trip: nudge carries
|
|
@@ -101,6 +101,38 @@ correctness on `README.md` and `docs/` excluding `docs/plans/`; see
|
|
|
101
101
|
- `event` handler: calls `client.session.prompt` on `session.created` with
|
|
102
102
|
correct session ID and nudge text; ignores non-`session.created` events
|
|
103
103
|
- `sessionStartReminder`: includes store name and recall instructions
|
|
104
|
+
- Recall nudge via `chat.message`: surfaces when prompt matches a stored
|
|
105
|
+
memory, no nudge for unrelated or short prompts, extraction priority
|
|
106
|
+
- Prediction auto-fire via `chat.message`: surfaces when prompt matches a
|
|
107
|
+
stored matcher, no nudge for unrelated prompts, prediction and recall
|
|
108
|
+
nudges fire independently as separate synthetic parts
|
|
109
|
+
|
|
110
|
+
### `tests/tool-defs.test.ts`
|
|
111
|
+
|
|
112
|
+
- Tool count (13) and name list
|
|
113
|
+
- Zod schema validation for each tool's args
|
|
114
|
+
- Execute functions: `memory_remember` save/duplicate/overwrite/archived,
|
|
115
|
+
`memory_recall` default/empty/archived, `memory_list`/`show`/`forget`,
|
|
116
|
+
`store_list`, `find_duplicates` clean store
|
|
117
|
+
- Prediction execute functions: `prediction_update` create/confirm/disconfirm/
|
|
118
|
+
create-on-existing (dedup links without disconfirming), `prediction_query`
|
|
119
|
+
matching/unrelated, `prediction_list` with provenance, `prediction_delete`
|
|
120
|
+
found/not-found
|
|
121
|
+
|
|
122
|
+
### `tests/prediction.test.ts`
|
|
123
|
+
|
|
124
|
+
- Schema: tables exist, `populationP0` fallback on empty store
|
|
125
|
+
- Matcher creation and lookup: `findMatchers` ranking, `findNearestMatcher`
|
|
126
|
+
threshold/dedup
|
|
127
|
+
- Prediction creation: `createPrediction` seeds at p0, `adjustConfidence`
|
|
128
|
+
confirm/disconfirm/soft/multiple, non-existent ID no-op
|
|
129
|
+
- Edges and scoring: `createEdge` links, `scorePredictions` empty/dedup/
|
|
130
|
+
weight-preservation, `scorePredictionNudge` threshold filtering + dedup
|
|
131
|
+
- `findNearestPrediction`: store-wide search, threshold, cross-matcher
|
|
132
|
+
- Provenance: `addProvenance` + `getProvenance` read-back, `deletePrediction`
|
|
133
|
+
cascade to edges and provenance
|
|
134
|
+
- `listPredictions`: sorted by confidence, matchers included
|
|
135
|
+
- `populationP0`: under-20 fallback, population hit rate with sufficient evidence
|
|
104
136
|
|
|
105
137
|
### `tests/hygiene.test.ts`
|
|
106
138
|
|
|
@@ -29,6 +29,10 @@ file/IPC/CLI contracts ready to graduate into `bun:test` integration tests.
|
|
|
29
29
|
| [UC-018](UC-018-mcp-startup-setup-detection.md) | MCP startup detects missing/broken setup instructions and warns the agent |
|
|
30
30
|
| [UC-019](UC-019-archived-memory-lifecycle.md) | Archive, search exclusion, update guard, unarchive |
|
|
31
31
|
| [UC-020](UC-020-extraction-escalation.md) | Nudge escalation (polite → insistent → ALL-CAPS), buffer persistence, `extraction_done` drain |
|
|
32
|
+
| [UC-021](UC-021-prediction-autofire.md) | Prediction auto-fire: matcher scores prompt, prediction nudge injected alongside recall |
|
|
33
|
+
| [UC-022](UC-022-prediction-dedup.md) | Prediction create-on-existing: dedup finds match, links edge without disconfirming |
|
|
34
|
+
| [UC-023](UC-023-prediction-confidence-model.md) | Bayesian confidence math: create, confirm, soft (weak disconfirm), disconfirm signals |
|
|
35
|
+
| [UC-024](UC-024-prediction-delete.md) | Prediction delete with cascade, provenance inspection, not-found handling |
|
|
32
36
|
|
|
33
37
|
## Template
|
|
34
38
|
|