@jmtrin/opencode-kevin 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +87 -11
- package/dist/migrations/006_v05_glassbox.sql +118 -0
- package/dist/plugin/Archiver.d.ts +42 -0
- package/dist/plugin/Archiver.js +98 -0
- package/dist/plugin/Archiver.js.map +1 -0
- package/dist/plugin/ContextInjector.d.ts +72 -0
- package/dist/plugin/ContextInjector.js +177 -44
- package/dist/plugin/ContextInjector.js.map +1 -1
- package/dist/plugin/Feedback.d.ts +67 -0
- package/dist/plugin/Feedback.js +137 -0
- package/dist/plugin/Feedback.js.map +1 -0
- package/dist/plugin/InjectionLedger.d.ts +9 -2
- package/dist/plugin/InjectionLedger.js +63 -9
- package/dist/plugin/InjectionLedger.js.map +1 -1
- package/dist/plugin/MemoryService.d.ts +19 -0
- package/dist/plugin/MemoryService.js +119 -15
- package/dist/plugin/MemoryService.js.map +1 -1
- package/dist/plugin/Migrate.js +21 -0
- package/dist/plugin/Migrate.js.map +1 -1
- package/dist/plugin/QualityGate.d.ts +39 -0
- package/dist/plugin/QualityGate.js +38 -8
- package/dist/plugin/QualityGate.js.map +1 -1
- package/dist/plugin/Retrospective.js +12 -0
- package/dist/plugin/Retrospective.js.map +1 -1
- package/dist/plugin/confidence.d.ts +3 -1
- package/dist/plugin/confidence.js +14 -2
- package/dist/plugin/confidence.js.map +1 -1
- package/dist/plugin/index.d.ts +1 -1
- package/dist/plugin/index.js +152 -0
- package/dist/plugin/index.js.map +1 -1
- package/dist/plugin/kevin_audit.d.ts +49 -0
- package/dist/plugin/kevin_audit.js +141 -0
- package/dist/plugin/kevin_audit.js.map +1 -0
- package/dist/plugin/kevin_why.js +23 -2
- package/dist/plugin/kevin_why.js.map +1 -1
- package/dist/plugin/metrics.d.ts +21 -1
- package/dist/plugin/metrics.js +47 -2
- package/dist/plugin/metrics.js.map +1 -1
- package/dist/plugin/replay-types.d.ts +327 -0
- package/dist/plugin/replay-types.js +65 -0
- package/dist/plugin/replay-types.js.map +1 -0
- package/dist/plugin/replay.d.ts +36 -0
- package/dist/plugin/replay.js +193 -0
- package/dist/plugin/replay.js.map +1 -0
- package/migrations/006_v05_glassbox.sql +118 -0
- package/package.json +3 -2
package/README.md
CHANGED
|
@@ -8,6 +8,7 @@ Kevin is an [OpenCode](https://opencode.ai) plugin that **observes** every agent
|
|
|
8
8
|
- **Global memory**: a single `~/.opencode-kevin/kevin.db` shared across all your projects (WAL mode → safe for concurrent sessions). No per-project folders.
|
|
9
9
|
- **Knowledge + Causality (v0.3.0)**: causal failure→fix chains, `kevin_why` explanations, OKF export/import, a supersede model, and human-in-the-loop AGENTS.md suggestions.
|
|
10
10
|
- **Signal over Noise (v0.4.0)**: a quality gate that stores weak lessons without injecting them, an injection ledger with honest `precision_rate`, two-sided confidence, and a fixed compacting hook.
|
|
11
|
+
- **Glass Box (v0.5.0)**: honest measurement replaces estimates — three-way injection settlement (`effective` / `ineffective` / `inconclusive`), human feedback that actually moves confidence, a strict dry-run `kevin_trace`, a read-only `kevin_audit`, memory lifecycle completion (`superseded_by`, `ignored`, `archived`), and a hermetic replay harness. See [How Kevin measures itself](#how-kevin-measures-itself).
|
|
11
12
|
- **Audited**: the v0.4.0 bug catalog (`docs/Kevin_v0.4.0_Bugs.md`) is fully closed — 16/16 bugs fixed and regression-tested (evidence in `kevin_query`/`kevin_get`, OKF round-trip fidelity, causal refresh guard, redaction precision, cross-session isolation).
|
|
12
13
|
- **Standalone**: works without any other plugin. With the ecosystem, it learns more richly.
|
|
13
14
|
|
|
@@ -33,7 +34,7 @@ For a **single project**, put the same `plugin` array in `./opencode.json` or `.
|
|
|
33
34
|
|
|
34
35
|
### 2. Restart OpenCode
|
|
35
36
|
|
|
36
|
-
Config is loaded once at startup and is **not hot-reloaded** — quit and reopen OpenCode after editing. On start, OpenCode resolves the npm spec, caches the plugin in `~/.cache/opencode/packages/@jmtrin/opencode-kevin/`, and exposes
|
|
37
|
+
Config is loaded once at startup and is **not hot-reloaded** — quit and reopen OpenCode after editing. On start, OpenCode resolves the npm spec, caches the plugin in `~/.cache/opencode/packages/@jmtrin/opencode-kevin/`, and exposes thirteen tools: `kevin_save`, `kevin_query`, `kevin_get`, `kevin_recall`, `kevin_status`, `kevin_retrospective`, `kevin_why`, `kevin_export`, `kevin_import`, `kevin_config`, `kevin_feedback`, `kevin_trace`, `kevin_audit` (v0.5.0 adds the last three).
|
|
37
38
|
|
|
38
39
|
### 3. Where data lives
|
|
39
40
|
|
|
@@ -116,7 +117,7 @@ Use `:memory:` for `dbPath` in tests.
|
|
|
116
117
|
┌─────────────────┐ │
|
|
117
118
|
│ SHARE │◄──────────────┘
|
|
118
119
|
│ ContextInjector │ injects relevant lessons pre-prompt
|
|
119
|
-
│ │ (
|
|
120
|
+
│ │ (900 tokens, configurable via pre_prompt_budget_tokens) + on compacting (2000)
|
|
120
121
|
│ │ + <protect> + id: line (v0.2.0)
|
|
121
122
|
│ │ + origin-aware rank (v0.2.0)
|
|
122
123
|
│ │ + <kevin-suggestion> after negative
|
|
@@ -137,7 +138,7 @@ Use `:memory:` for `dbPath` in tests.
|
|
|
137
138
|
|
|
138
139
|
## Tools
|
|
139
140
|
|
|
140
|
-
Kevin exposes
|
|
141
|
+
Kevin exposes 13 tools callable by the agent:
|
|
141
142
|
|
|
142
143
|
### `kevin_save`
|
|
143
144
|
|
|
@@ -188,7 +189,7 @@ kevin_recall({ query: "auth", limit: 3 })
|
|
|
188
189
|
|
|
189
190
|
### `kevin_status`
|
|
190
191
|
|
|
191
|
-
Global counts and metrics. v0.2.0 adds `memories_reflector`, `memories_agent`, `memories_pattern` and a `metrics` object; v0.3.0 adds `memories_causal` and 3 more seeded counters (`patterns_causal`, `causal_links`, `memories_superseded`); v0.4.0 adds the precision block: `injections_total`, `injections_effective`, `injections_ineffective`, `precision_rate`, `patterns_promoted_new`, and per-origin `recurrence_by_origin`.
|
|
192
|
+
Global counts and metrics. v0.2.0 adds `memories_reflector`, `memories_agent`, `memories_pattern` and a `metrics` object; v0.3.0 adds `memories_causal` and 3 more seeded counters (`patterns_causal`, `causal_links`, `memories_superseded`); v0.4.0 adds the precision block: `injections_total`, `injections_effective`, `injections_ineffective`, `precision_rate`, `patterns_promoted_new`, and per-origin `recurrence_by_origin`. v0.5.0 adds `injections_inconclusive`, `coverage_rate`, `blocked`, `memories_ignored`, `memories_archived` and `feedback { positive, negative }`.
|
|
192
193
|
|
|
193
194
|
```
|
|
194
195
|
kevin_status({})
|
|
@@ -197,10 +198,14 @@ kevin_status({})
|
|
|
197
198
|
// "metrics": { "tokens_injected_pre_prompt": 51, "tokens_injected_compacting": 0,
|
|
198
199
|
// "reflections_throttled": 3, "duplicate_suppressions": 2,
|
|
199
200
|
// "tool_calls_deduped": 0, "patterns_mined": 0,
|
|
200
|
-
// "patterns_causal": 1, "causal_links": 2, "memories_superseded": 0
|
|
201
|
-
//
|
|
202
|
-
// "
|
|
203
|
-
// "
|
|
201
|
+
// "patterns_causal": 1, "causal_links": 2, "memories_superseded": 0,
|
|
202
|
+
// "injections_inconclusive": 9, "injections_blocked_seen": 1, ... },
|
|
203
|
+
// "injections_total": 14, "injections_effective": 2, "injections_ineffective": 3,
|
|
204
|
+
// "injections_inconclusive": 9, "precision_rate": 0.40, "coverage_rate": 0.36,
|
|
205
|
+
// "blocked": { "seen": 1, "weak": 0, "recurrence": 2, "stale": 0, "ignored": 1 },
|
|
206
|
+
// "memories_ignored": 1, "memories_archived": 4,
|
|
207
|
+
// "feedback": { "positive": 2, "negative": 1 },
|
|
208
|
+
// "patterns_promoted_new": 2, "recurrence_by_origin": { "reflector": 3, "causal": 1 } }
|
|
204
209
|
```
|
|
205
210
|
|
|
206
211
|
### `kevin_retrospective`
|
|
@@ -245,7 +250,44 @@ kevin_config({ action: "set", key: "quality_gate_enabled", value: "0" })
|
|
|
245
250
|
// → { "ok": true }
|
|
246
251
|
```
|
|
247
252
|
|
|
248
|
-
Known keys: `quality_gate_enabled`, `lesson_snippet_injection`, `llm_reflection_enabled`, `cross_project_enabled`, `patternminer_enabled`, `tool_calls_dedup_enabled` (v0.4.0).
|
|
253
|
+
Known keys: `quality_gate_enabled`, `lesson_snippet_injection`, `llm_reflection_enabled`, `cross_project_enabled`, `patternminer_enabled`, `tool_calls_dedup_enabled` (v0.4.0), `deterministic_retrieval`, `pre_prompt_budget_tokens`, `archive_after_days` (v0.5.0).
|
|
254
|
+
|
|
255
|
+
### `kevin_feedback` (v0.5.0)
|
|
256
|
+
|
|
257
|
+
Rates an injected memory and makes the rating count. `verdict` is `useful` | `wrong` | `outdated` | `ignore`. The first three are stored in `memory_feedback` and move `kevin_why`'s confidence (`+0.05` / `-0.1` per count); **`ignore` is a hard action** — the memory is stamped `ignored = 1` and excluded from retrieval, queries and injection.
|
|
258
|
+
|
|
259
|
+
```
|
|
260
|
+
kevin_feedback({ memory_id: "0195a3b2-...", verdict: "wrong", note: "the fix was wrong" })
|
|
261
|
+
// → { "ok": true, "verdict": "wrong" }
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
### `kevin_trace` (v0.5.0)
|
|
265
|
+
|
|
266
|
+
Strict dry-run: predicts exactly which memories `onSystemTransform` WOULD inject for a query (optionally `session_id`, `tag` and `cap`), with **zero side effects** — no counters, no ledger rows, no seen-set writes, no relevance bumps. Rejected items carry their `GateReason` (`seen_this_session` | `weak` | `recurrence` | `stale` | `ignored`).
|
|
267
|
+
|
|
268
|
+
```
|
|
269
|
+
kevin_trace({ query: "tsc error" })
|
|
270
|
+
// → { "query": "tsc error", "tag": "context", "cap": 900, "would_inject": true,
|
|
271
|
+
// "total_tokens": 82,
|
|
272
|
+
// "admitted": [ { "id": "...", "type": "error", "decision": "admitted", "tokens": 62 } ],
|
|
273
|
+
// "blocked": [ { "id": "...", "type": "error", "decision": "blocked", "reason": "seen_this_session", "tokens": 20 } ] }
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
### `kevin_audit` (v0.5.0)
|
|
277
|
+
|
|
278
|
+
Read-only report of the whole system state: memories by `status`/`origin`/`type`, injection outcomes with `precision_rate`/`coverage_rate`, the five `blocked` counters, feedback by verdict, tokens injected. `verbose: true` adds the settings block. No writes, no LLM; on a pre-006 database it returns what it can with `"partial": true`.
|
|
279
|
+
|
|
280
|
+
```
|
|
281
|
+
kevin_audit({})
|
|
282
|
+
// → { "memories": { "total": 42, "by_status": { "active": 37, "stale": 1, "archived": 4 },
|
|
283
|
+
// "by_origin": { "reflector": 12, "agent": 30 }, "by_type": { "error": 20, ... },
|
|
284
|
+
// "ignored": 1, "archived": 4, "with_feedback": 3, "superseded_with_target": 2 },
|
|
285
|
+
// "injections": { "total": 14, "effective": 2, "ineffective": 3, "inconclusive": 9,
|
|
286
|
+
// "unmeasured": 0, "precision_rate": 0.40, "coverage_rate": 0.36 },
|
|
287
|
+
// "blocked": { "seen": 1, "weak": 0, "recurrence": 2, "stale": 0, "ignored": 1 },
|
|
288
|
+
// "feedback": { "positive": 2, "negative": 1, "by_verdict": { "useful": 2, "wrong": 1 } },
|
|
289
|
+
// "tokens": { "pre_prompt": 51, "compacting": 0 }, "partial": false }
|
|
290
|
+
```
|
|
249
291
|
|
|
250
292
|
---
|
|
251
293
|
|
|
@@ -253,6 +295,27 @@ Known keys: `quality_gate_enabled`, `lesson_snippet_injection`, `llm_reflection_
|
|
|
253
295
|
|
|
254
296
|
Weak lessons — errors the reflector cannot dispatch to a deterministic rule — are **stored but never injected** while `quality_gate_enabled = '1'` (default). Injection now goes through a ledger: every pre-prompt/compacting injection is recorded and settled as effective or ineffective at session idle, so `kevin_status` reports the honest picture (`injections_total`, `injections_effective/ineffective`, `precision_rate`, `patterns_promoted_new`) instead of raw "lessons shared" counts. Recurrences demote lessons (`recurrence_count` → `stale`) and lower confidence. Debug mode: `kevin_config({ action: "set", key: "quality_gate_enabled", value: "0" })` re-injects weak lessons with a `(low confidence)` marker.
|
|
255
297
|
|
|
298
|
+
## How Kevin measures itself (v0.5.0)
|
|
299
|
+
|
|
300
|
+
Every injection is settled at `session.idle` into one of **four outcomes**:
|
|
301
|
+
|
|
302
|
+
| Outcome | Meaning | Counts toward precision? |
|
|
303
|
+
|---|---|---|
|
|
304
|
+
| `effective` | A linked fix was observed after the injection | yes (numerator) |
|
|
305
|
+
| `ineffective` | The same error recurred after the injection | yes (denominator) |
|
|
306
|
+
| `inconclusive` | Neither — the error did not recur, but no fix was seen either | no |
|
|
307
|
+
| `unmeasured` | Session went idle before settlement could run | no |
|
|
308
|
+
|
|
309
|
+
- **`precision_rate`** = `effective / (effective + ineffective)`. The v0.4.0 definition counted "did not recur" as success — measuring absence of recurrence, not effect. **Your precision rate will look lower on v0.5.0. That is the honest number.** Existing `effective` rows were remapped to `inconclusive` by migration 006.
|
|
310
|
+
- **`coverage_rate`** = `(effective + ineffective) / total` — the share of injections that were actually measured. Reported alongside precision so a low measurable fraction stays visible instead of hiding behind a large total.
|
|
311
|
+
- **`blocked`** counts every gate rejection by reason (`seen_this_session`, `weak`, `recurrence`, `stale`, `ignored`) — a rejection you did not count did not happen.
|
|
312
|
+
|
|
313
|
+
`kevin_trace` shows you the plan *before* it happens (dry run, zero side effects); `kevin_audit` reads the whole picture after; `kevin_feedback` lets a human correct it — and the correction moves the confidence number `kevin_why` reports.
|
|
314
|
+
|
|
315
|
+
## Replay harness (v0.5.0)
|
|
316
|
+
|
|
317
|
+
`npm run replay` runs every transcript in `tests/replay/fixtures/` through the plugin against an in-memory database with a frozen clock and prints one table row per transcript (memories created, injection outcomes, `precision_rate`, `coverage_rate`, tokens). Record your own session as a JSON array of typed events (`session.created`, `chat.message`, `tool.before`, `tool.after`, `system.transform`, `compacting`, `session.idle`) with ISO-8601 `at` timestamps, drop it into `tests/replay/fixtures/`, and re-run. The `at` timestamps are the only source of time during replay.
|
|
318
|
+
|
|
256
319
|
---
|
|
257
320
|
|
|
258
321
|
## Hooks
|
|
@@ -263,7 +326,7 @@ Kevin subscribes to 6 OpenCode hooks:
|
|
|
263
326
|
|---|---|
|
|
264
327
|
| `tool.execute.before` | Records tool call start (callID + redacted args) |
|
|
265
328
|
| `tool.execute.after` | Records result (id = callID); on failure → Reflector.invoke async (throttled); on success → CausalChain.onSuccess links the fix (v0.3.0) |
|
|
266
|
-
| `experimental.chat.system.transform` | Injects relevant lessons in `<kevin-context>` (
|
|
329
|
+
| `experimental.chat.system.transform` | Injects relevant lessons in `<kevin-context>` (900 tokens by default, configurable) + optional `<kevin-suggestion>` (v0.3.0) |
|
|
267
330
|
| `experimental.session.compacting` | Re-injects lessons in `<kevin-memory>` after compacting (2000 tokens) + optional `<kevin-suggestion>` |
|
|
268
331
|
| `event` (`session.created`) | Captures current `sessionID` |
|
|
269
332
|
| `event` (`session.idle`) | Generates retrospective.md; boosts positive lessons (v0.2.0); penalizes recurring failures (v0.3.0); promotes causal patterns + mines patterns (opt-in); flushes metrics |
|
|
@@ -292,6 +355,18 @@ KevinPlugin(input, {
|
|
|
292
355
|
});
|
|
293
356
|
```
|
|
294
357
|
|
|
358
|
+
### Settings (v0.5.0)
|
|
359
|
+
|
|
360
|
+
Read/write via `kevin_config({ action: "list" | "set", ... })`. v0.5.0 adds:
|
|
361
|
+
|
|
362
|
+
| Setting | Default | Effect |
|
|
363
|
+
|---|---|---|
|
|
364
|
+
| `deterministic_retrieval` | `"0"` | Freezes Kevin's internal clock (recency factor 1.0, no relevance bumps) — for hermetic tests and the replay harness |
|
|
365
|
+
| `pre_prompt_budget_tokens` | `"900"` | Pre-prompt injection cap, clamped to `[100, 4000]` (was a hard-coded 1500) |
|
|
366
|
+
| `archive_after_days` | `"30"` | Age at which stale non-pattern memories are retired to `archived` on `session.idle` |
|
|
367
|
+
|
|
368
|
+
All settings are stored as TEXT in `kevin_settings`; values are compared against `"1"` for booleans.
|
|
369
|
+
|
|
295
370
|
---
|
|
296
371
|
|
|
297
372
|
## Development
|
|
@@ -302,8 +377,9 @@ cd opencode-kevin
|
|
|
302
377
|
npm install
|
|
303
378
|
npm run typecheck # tsc --noEmit (strict)
|
|
304
379
|
npm run lint # biome check .
|
|
305
|
-
npm test # vitest run (unit + integration + e2e)
|
|
380
|
+
npm test # vitest run (unit + integration + e2e + replay)
|
|
306
381
|
npm run verify # post-install verification
|
|
382
|
+
npm run replay # replay report over tests/replay/fixtures (v0.5.0)
|
|
307
383
|
```
|
|
308
384
|
|
|
309
385
|
### Publishing (maintainer)
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
-- ============================================================================
|
|
2
|
+
-- 006_v05_glassbox.sql — v0.5.0 "Glass Box"
|
|
3
|
+
--
|
|
4
|
+
-- Honest measurement, human feedback, lifecycle completion.
|
|
5
|
+
--
|
|
6
|
+
-- Section 1: rebuild kevin_injections to admit a fourth outcome.
|
|
7
|
+
-- Section 2: human feedback storage.
|
|
8
|
+
-- Section 3: memory lifecycle columns.
|
|
9
|
+
-- Section 4: metric seeds.
|
|
10
|
+
-- Section 5: setting seeds.
|
|
11
|
+
-- Section 6: schema_version.
|
|
12
|
+
-- ============================================================================
|
|
13
|
+
|
|
14
|
+
-- ---------------------------------------------------------------------------
|
|
15
|
+
-- 1. kevin_injections: add 'inconclusive'.
|
|
16
|
+
--
|
|
17
|
+
-- SQLite cannot ALTER a CHECK constraint, so the table must be rebuilt.
|
|
18
|
+
-- Migration 004 set this precedent. kevin_injections has no FTS5 triggers,
|
|
19
|
+
-- so unlike 004 this is a straight four-step rebuild.
|
|
20
|
+
--
|
|
21
|
+
-- Existing rows with outcome='effective' are remapped to 'inconclusive'.
|
|
22
|
+
-- This is not data loss: v0.4's 'effective' meant "the error did not recur",
|
|
23
|
+
-- which is the exact definition of the new 'inconclusive' bucket. Rows that
|
|
24
|
+
-- genuinely earned the new 'effective' will be re-settled naturally, and the
|
|
25
|
+
-- post-apply hook re-derives the counters from the table.
|
|
26
|
+
-- ---------------------------------------------------------------------------
|
|
27
|
+
CREATE TABLE IF NOT EXISTS kevin_injections_new (
|
|
28
|
+
id TEXT PRIMARY KEY,
|
|
29
|
+
memory_id TEXT NOT NULL,
|
|
30
|
+
fingerprint TEXT NOT NULL,
|
|
31
|
+
session_id TEXT NOT NULL,
|
|
32
|
+
hook TEXT NOT NULL CHECK (hook IN ('pre_prompt','compacting')),
|
|
33
|
+
tokens INTEGER NOT NULL,
|
|
34
|
+
injected_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
35
|
+
outcome TEXT NOT NULL DEFAULT 'unmeasured'
|
|
36
|
+
CHECK (outcome IN ('unmeasured','effective','ineffective','inconclusive'))
|
|
37
|
+
);
|
|
38
|
+
|
|
39
|
+
INSERT INTO kevin_injections_new
|
|
40
|
+
(id, memory_id, fingerprint, session_id, hook, tokens, injected_at, outcome)
|
|
41
|
+
SELECT
|
|
42
|
+
id, memory_id, fingerprint, session_id, hook, tokens, injected_at,
|
|
43
|
+
CASE WHEN outcome = 'effective' THEN 'inconclusive' ELSE outcome END
|
|
44
|
+
FROM kevin_injections;
|
|
45
|
+
|
|
46
|
+
DROP TABLE kevin_injections;
|
|
47
|
+
ALTER TABLE kevin_injections_new RENAME TO kevin_injections;
|
|
48
|
+
|
|
49
|
+
CREATE INDEX IF NOT EXISTS idx_injections_fp ON kevin_injections(fingerprint);
|
|
50
|
+
CREATE INDEX IF NOT EXISTS idx_injections_session ON kevin_injections(session_id);
|
|
51
|
+
CREATE INDEX IF NOT EXISTS idx_injections_outcome ON kevin_injections(outcome);
|
|
52
|
+
|
|
53
|
+
-- ---------------------------------------------------------------------------
|
|
54
|
+
-- 2. Human feedback. Append-only audit trail; the hot path reads the
|
|
55
|
+
-- denormalized counters on `memories` (section 3), never this table.
|
|
56
|
+
-- ---------------------------------------------------------------------------
|
|
57
|
+
CREATE TABLE IF NOT EXISTS memory_feedback (
|
|
58
|
+
id TEXT PRIMARY KEY,
|
|
59
|
+
memory_id TEXT NOT NULL,
|
|
60
|
+
verdict TEXT NOT NULL CHECK (verdict IN ('useful','wrong','outdated','ignore')),
|
|
61
|
+
session_id TEXT,
|
|
62
|
+
note TEXT,
|
|
63
|
+
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
|
64
|
+
);
|
|
65
|
+
|
|
66
|
+
CREATE INDEX IF NOT EXISTS idx_feedback_memory ON memory_feedback(memory_id);
|
|
67
|
+
CREATE INDEX IF NOT EXISTS idx_feedback_created ON memory_feedback(created_at);
|
|
68
|
+
|
|
69
|
+
-- ---------------------------------------------------------------------------
|
|
70
|
+
-- 3. Memory lifecycle and feedback columns.
|
|
71
|
+
--
|
|
72
|
+
-- feedback_positive / feedback_negative are SEPARATE from evidence_count
|
|
73
|
+
-- and recurrence_count by design: human judgement is evidence about the
|
|
74
|
+
-- memory, causal counters are evidence about the world. Mixing them was
|
|
75
|
+
-- the confidence-poisoning defect closed in v0.4.0.
|
|
76
|
+
--
|
|
77
|
+
-- superseded_by has no REFERENCES clause on purpose. Store enables
|
|
78
|
+
-- PRAGMA foreign_keys=ON, and a hard FK would block deletion of a memory
|
|
79
|
+
-- that superseded another.
|
|
80
|
+
-- ---------------------------------------------------------------------------
|
|
81
|
+
ALTER TABLE memories ADD COLUMN feedback_positive INTEGER NOT NULL DEFAULT 0;
|
|
82
|
+
ALTER TABLE memories ADD COLUMN feedback_negative INTEGER NOT NULL DEFAULT 0;
|
|
83
|
+
ALTER TABLE memories ADD COLUMN ignored INTEGER NOT NULL DEFAULT 0;
|
|
84
|
+
ALTER TABLE memories ADD COLUMN superseded_by TEXT;
|
|
85
|
+
ALTER TABLE memories ADD COLUMN archived_at TEXT;
|
|
86
|
+
|
|
87
|
+
CREATE INDEX IF NOT EXISTS idx_memories_ignored ON memories(ignored);
|
|
88
|
+
CREATE INDEX IF NOT EXISTS idx_memories_archived ON memories(archived_at);
|
|
89
|
+
|
|
90
|
+
-- ---------------------------------------------------------------------------
|
|
91
|
+
-- 4. Metric seeds. Order matches the additions to METRIC_KEYS in metrics.ts.
|
|
92
|
+
-- ---------------------------------------------------------------------------
|
|
93
|
+
INSERT OR IGNORE INTO kevin_metrics (key, value) VALUES
|
|
94
|
+
('injections_inconclusive', 0),
|
|
95
|
+
('injections_blocked_seen', 0),
|
|
96
|
+
('injections_blocked_weak', 0),
|
|
97
|
+
('injections_blocked_recurrence',0),
|
|
98
|
+
('injections_blocked_stale', 0),
|
|
99
|
+
('injections_blocked_ignored', 0),
|
|
100
|
+
('feedback_positive_total', 0),
|
|
101
|
+
('feedback_negative_total', 0),
|
|
102
|
+
('memories_archived', 0);
|
|
103
|
+
|
|
104
|
+
-- ---------------------------------------------------------------------------
|
|
105
|
+
-- 5. Setting seeds. Values are TEXT, always. Read them with an explicit
|
|
106
|
+
-- string comparison or an explicit Number() parse — never `=== 1`.
|
|
107
|
+
-- (That exact mistake kept cross_project_enabled unreachable for the
|
|
108
|
+
-- whole of v0.3.0.)
|
|
109
|
+
-- ---------------------------------------------------------------------------
|
|
110
|
+
INSERT OR IGNORE INTO kevin_settings (key, value) VALUES
|
|
111
|
+
('deterministic_retrieval', '0'),
|
|
112
|
+
('pre_prompt_budget_tokens', '900'),
|
|
113
|
+
('archive_after_days', '30');
|
|
114
|
+
|
|
115
|
+
-- ---------------------------------------------------------------------------
|
|
116
|
+
-- 6. Version marker.
|
|
117
|
+
-- ---------------------------------------------------------------------------
|
|
118
|
+
INSERT OR IGNORE INTO schema_version (version) VALUES ('006');
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
import type { MemoryService } from "./MemoryService.js";
|
|
2
|
+
import type { Store } from "./Store.js";
|
|
3
|
+
import type { Metrics } from "./metrics.js";
|
|
4
|
+
/**
|
|
5
|
+
* v0.5.0 Archiver (K5-012 / plan §5.4, D5-05).
|
|
6
|
+
*
|
|
7
|
+
* Lifecycle tail of the memory loop: reflectors mark a lesson `stale` when
|
|
8
|
+
* its fingerprint recurred enough times (penalizeRecurringReflectors,
|
|
9
|
+
* migration 003), but nothing ever retires those rows — they keep
|
|
10
|
+
* circulating through status filters and inflating counts. The Archiver
|
|
11
|
+
* runs on `session.idle` and retires stale memories whose last activity
|
|
12
|
+
* (`updated_at`, falling back to `last_verified_at`) is older than
|
|
13
|
+
* `archive_after_days` (kevin_settings, seeded '30' in migration 006).
|
|
14
|
+
*
|
|
15
|
+
* Rules:
|
|
16
|
+
* - Only `status = 'stale'` rows with `archived_at IS NULL`.
|
|
17
|
+
* - `type != 'pattern'`: a pattern is the FIXED form of a fingerprint
|
|
18
|
+
* (K4-025); archiving it would break kevin_why and the recurrence
|
|
19
|
+
* re-admission path (QualityGate rule 4).
|
|
20
|
+
* - Clock: the injection clock (K5-008) — tests freeze it; production
|
|
21
|
+
* defaults to `new Date()`. Never `Date.now()` inside.
|
|
22
|
+
* - `memories_archived` is bumped by the batch size, once per run.
|
|
23
|
+
* - Pre-006 DBs (no `archived_at`) degrade to a no-op.
|
|
24
|
+
*
|
|
25
|
+
* Archived rows remain queryable with `includeSuperseded: true`-style
|
|
26
|
+
* reads (status filter is exclusive, not destructive).
|
|
27
|
+
*/
|
|
28
|
+
export declare class Archiver {
|
|
29
|
+
private readonly store;
|
|
30
|
+
private readonly memoryService;
|
|
31
|
+
private readonly metrics;
|
|
32
|
+
private readonly now;
|
|
33
|
+
constructor(store: Store, memoryService: MemoryService, metrics?: Metrics | null, now?: () => Date);
|
|
34
|
+
/**
|
|
35
|
+
* Archives every eligible stale memory. Returns the number of rows
|
|
36
|
+
* retired. Idempotent: archived rows are excluded by `archived_at IS
|
|
37
|
+
* NULL`, so a second run archives nothing.
|
|
38
|
+
*/
|
|
39
|
+
run(): number;
|
|
40
|
+
private archiveAfterDays;
|
|
41
|
+
private hasArchivedColumn;
|
|
42
|
+
}
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* v0.5.0 Archiver (K5-012 / plan §5.4, D5-05).
|
|
3
|
+
*
|
|
4
|
+
* Lifecycle tail of the memory loop: reflectors mark a lesson `stale` when
|
|
5
|
+
* its fingerprint recurred enough times (penalizeRecurringReflectors,
|
|
6
|
+
* migration 003), but nothing ever retires those rows — they keep
|
|
7
|
+
* circulating through status filters and inflating counts. The Archiver
|
|
8
|
+
* runs on `session.idle` and retires stale memories whose last activity
|
|
9
|
+
* (`updated_at`, falling back to `last_verified_at`) is older than
|
|
10
|
+
* `archive_after_days` (kevin_settings, seeded '30' in migration 006).
|
|
11
|
+
*
|
|
12
|
+
* Rules:
|
|
13
|
+
* - Only `status = 'stale'` rows with `archived_at IS NULL`.
|
|
14
|
+
* - `type != 'pattern'`: a pattern is the FIXED form of a fingerprint
|
|
15
|
+
* (K4-025); archiving it would break kevin_why and the recurrence
|
|
16
|
+
* re-admission path (QualityGate rule 4).
|
|
17
|
+
* - Clock: the injection clock (K5-008) — tests freeze it; production
|
|
18
|
+
* defaults to `new Date()`. Never `Date.now()` inside.
|
|
19
|
+
* - `memories_archived` is bumped by the batch size, once per run.
|
|
20
|
+
* - Pre-006 DBs (no `archived_at`) degrade to a no-op.
|
|
21
|
+
*
|
|
22
|
+
* Archived rows remain queryable with `includeSuperseded: true`-style
|
|
23
|
+
* reads (status filter is exclusive, not destructive).
|
|
24
|
+
*/
|
|
25
|
+
export class Archiver {
|
|
26
|
+
store;
|
|
27
|
+
memoryService;
|
|
28
|
+
metrics;
|
|
29
|
+
now;
|
|
30
|
+
constructor(store, memoryService, metrics, now = () => new Date()) {
|
|
31
|
+
this.store = store;
|
|
32
|
+
this.memoryService = memoryService;
|
|
33
|
+
this.metrics = metrics ?? null;
|
|
34
|
+
this.now = now;
|
|
35
|
+
}
|
|
36
|
+
/**
|
|
37
|
+
* Archives every eligible stale memory. Returns the number of rows
|
|
38
|
+
* retired. Idempotent: archived rows are excluded by `archived_at IS
|
|
39
|
+
* NULL`, so a second run archives nothing.
|
|
40
|
+
*/
|
|
41
|
+
run() {
|
|
42
|
+
if (!this.hasArchivedColumn())
|
|
43
|
+
return 0;
|
|
44
|
+
const days = this.archiveAfterDays();
|
|
45
|
+
if (days <= 0)
|
|
46
|
+
return 0;
|
|
47
|
+
const now = this.now();
|
|
48
|
+
const cutoff = formatSqliteUtc(new Date(now.getTime() - days * 86_400_000));
|
|
49
|
+
this.store
|
|
50
|
+
.prepare(`UPDATE memories
|
|
51
|
+
SET status = 'archived', archived_at = ?, updated_at = ?
|
|
52
|
+
WHERE status = 'stale'
|
|
53
|
+
AND type != 'pattern'
|
|
54
|
+
AND archived_at IS NULL
|
|
55
|
+
AND updated_at < ?`)
|
|
56
|
+
.run(formatSqliteUtc(now), formatSqliteUtc(now), cutoff);
|
|
57
|
+
const changed = this.store.prepare("SELECT changes() AS n").get();
|
|
58
|
+
const n = changed.n;
|
|
59
|
+
if (n > 0)
|
|
60
|
+
this.metrics?.incr("memories_archived", n);
|
|
61
|
+
return n;
|
|
62
|
+
}
|
|
63
|
+
archiveAfterDays() {
|
|
64
|
+
try {
|
|
65
|
+
const raw = this.memoryService.getSetting("archive_after_days", "30");
|
|
66
|
+
const n = Number(raw);
|
|
67
|
+
return Number.isFinite(n) ? n : 30;
|
|
68
|
+
}
|
|
69
|
+
catch {
|
|
70
|
+
return 30;
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
hasArchivedColumn() {
|
|
74
|
+
return hasArchivedColumnCached(this.store);
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
const archivedColumnCache = new WeakMap();
|
|
78
|
+
function hasArchivedColumnCached(store) {
|
|
79
|
+
const cached = archivedColumnCache.get(store);
|
|
80
|
+
if (cached !== undefined)
|
|
81
|
+
return cached;
|
|
82
|
+
try {
|
|
83
|
+
store.prepare("SELECT archived_at FROM memories LIMIT 1").get();
|
|
84
|
+
archivedColumnCache.set(store, true);
|
|
85
|
+
return true;
|
|
86
|
+
}
|
|
87
|
+
catch {
|
|
88
|
+
archivedColumnCache.set(store, false);
|
|
89
|
+
return false;
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
/** 'YYYY-MM-DD HH:MM:SS' UTC — the format SQLite `datetime('now')` uses,
|
|
93
|
+
* so lexicographic comparisons against `updated_at` are valid. */
|
|
94
|
+
function formatSqliteUtc(d) {
|
|
95
|
+
const pad = (n) => String(n).padStart(2, "0");
|
|
96
|
+
return `${d.getUTCFullYear()}-${pad(d.getUTCMonth() + 1)}-${pad(d.getUTCDate())} ${pad(d.getUTCHours())}:${pad(d.getUTCMinutes())}:${pad(d.getUTCSeconds())}`;
|
|
97
|
+
}
|
|
98
|
+
//# sourceMappingURL=Archiver.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"Archiver.js","sourceRoot":"","sources":["../../plugin/Archiver.ts"],"names":[],"mappings":"AAIA;;;;;;;;;;;;;;;;;;;;;;;GAuBG;AACH,MAAM,OAAO,QAAQ;IAKF;IACA;IALD,OAAO,CAAiB;IACxB,GAAG,CAAa;IAEjC,YACkB,KAAY,EACZ,aAA4B,EAC7C,OAAwB,EACxB,MAAkB,GAAG,EAAE,CAAC,IAAI,IAAI,EAAE;QAHjB,UAAK,GAAL,KAAK,CAAO;QACZ,kBAAa,GAAb,aAAa,CAAe;QAI7C,IAAI,CAAC,OAAO,GAAG,OAAO,IAAI,IAAI,CAAC;QAC/B,IAAI,CAAC,GAAG,GAAG,GAAG,CAAC;IAChB,CAAC;IAED;;;;OAIG;IACH,GAAG;QACF,IAAI,CAAC,IAAI,CAAC,iBAAiB,EAAE;YAAE,OAAO,CAAC,CAAC;QACxC,MAAM,IAAI,GAAG,IAAI,CAAC,gBAAgB,EAAE,CAAC;QACrC,IAAI,IAAI,IAAI,CAAC;YAAE,OAAO,CAAC,CAAC;QAExB,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;QACvB,MAAM,MAAM,GAAG,eAAe,CAAC,IAAI,IAAI,CAAC,GAAG,CAAC,OAAO,EAAE,GAAG,IAAI,GAAG,UAAU,CAAC,CAAC,CAAC;QAC5E,IAAI,CAAC,KAAK;aACR,OAAO,CACP;;;;;2BAKuB,CACvB;aACA,GAAG,CAAC,eAAe,CAAC,GAAG,CAAC,EAAE,eAAe,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC,CAAC;QAC1D,MAAM,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,uBAAuB,CAAC,CAAC,GAAG,EAE9D,CAAC;QACF,MAAM,CAAC,GAAG,OAAO,CAAC,CAAC,CAAC;QACpB,IAAI,CAAC,GAAG,CAAC;YAAE,IAAI,CAAC,OAAO,EAAE,IAAI,CAAC,mBAAmB,EAAE,CAAC,CAAC,CAAC;QACtD,OAAO,CAAC,CAAC;IACV,CAAC;IAEO,gBAAgB;QACvB,IAAI,CAAC;YACJ,MAAM,GAAG,GAAG,IAAI,CAAC,aAAa,CAAC,UAAU,CAAC,oBAAoB,EAAE,IAAI,CAAC,CAAC;YACtE,MAAM,CAAC,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;YACtB,OAAO,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;QACpC,CAAC;QAAC,MAAM,CAAC;YACR,OAAO,EAAE,CAAC;QACX,CAAC;IACF,CAAC;IAEO,iBAAiB;QACxB,OAAO,uBAAuB,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IAC5C,CAAC;CACD;AAED,MAAM,mBAAmB,GAAG,IAAI,OAAO,EAAkB,CAAC;AAC1D,SAAS,uBAAuB,CAAC,KAAY;IAC5C,MAAM,MAAM,GAAG,mBAAmB,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;IAC9C,IAAI,MAAM,KAAK,SAAS;QAAE,OAAO,MAAM,CAAC;IACxC,IAAI,CAAC;QACJ,KAAK,CAAC,OAAO,CAAC,0CAA0C,CAAC,CAAC,GAAG,EAAE,CAAC;QAChE,mBAAmB,CAAC,GAAG,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC;QACrC,OAAO,IAAI,CAAC;IACb,CAAC;IAAC,MAAM,CAAC;QACR,mBAAmB,CAAC,GAAG,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;QACtC,OAAO,KAAK,CAAC;IACd,CAAC;AACF,CAAC;AAED;kEACkE;AAClE,SAAS,eAAe,CAAC,CAAO;IAC/B,MAAM,GAAG,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;IACtD,OAAO,GAAG,CAAC,CAAC,cAAc,EAAE,IAAI,GAAG,CAAC,CAAC,CAAC,WAAW,EAAE,GAAG,CAAC,CAAC,IAAI,GAAG,CAC9D,CAAC,CAAC,UAAU,EAAE,CACd,IAAI,GAAG,CAAC,CAAC,CAAC,WAAW,EAAE,CAAC,IAAI,GAAG,CAAC,CAAC,CAAC,aAAa,EAAE,CAAC,IAAI,GAAG,CAAC,CAAC,CAAC,aAAa,EAAE,CAAC,EAAE,CAAC;AACjF,CAAC"}
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import type { InjectionLedger } from "./InjectionLedger.js";
|
|
2
2
|
import type { MemoryService } from "./MemoryService.js";
|
|
3
|
+
import { type GateReason } from "./QualityGate.js";
|
|
3
4
|
import { type Metrics } from "./metrics.js";
|
|
4
5
|
export declare const QUALITY_GATE_SETTING = "quality_gate_enabled";
|
|
5
6
|
export declare const SNIPPET_INJECTION_SETTING = "lesson_snippet_injection";
|
|
@@ -21,6 +22,26 @@ export interface CompactingInput {
|
|
|
21
22
|
export interface CompactingOutput {
|
|
22
23
|
context: string[];
|
|
23
24
|
}
|
|
25
|
+
export interface InjectionPlanItem {
|
|
26
|
+
id: string;
|
|
27
|
+
type: string;
|
|
28
|
+
decision: "admitted" | "blocked";
|
|
29
|
+
/** Gate reason when blocked; null for admitted items. */
|
|
30
|
+
reason?: GateReason;
|
|
31
|
+
/** Estimated tokens this memory would contribute to the block. */
|
|
32
|
+
tokens: number;
|
|
33
|
+
}
|
|
34
|
+
export interface InjectionPlan {
|
|
35
|
+
query: string;
|
|
36
|
+
tag: "context" | "memory";
|
|
37
|
+
cap: number;
|
|
38
|
+
would_inject: boolean;
|
|
39
|
+
/** Estimated tokens of the block that WOULD be produced (0 when
|
|
40
|
+
* nothing would be injected). */
|
|
41
|
+
total_tokens: number;
|
|
42
|
+
admitted: InjectionPlanItem[];
|
|
43
|
+
blocked: InjectionPlanItem[];
|
|
44
|
+
}
|
|
24
45
|
export declare class ContextInjector {
|
|
25
46
|
private memoryService;
|
|
26
47
|
private metrics;
|
|
@@ -76,13 +97,64 @@ export declare class ContextInjector {
|
|
|
76
97
|
* `QualityGate.canInject` (session seen-set + recurrence + strength),
|
|
77
98
|
* and each admitted memory is recorded in the `InjectionLedger`
|
|
78
99
|
* (plan §5.2 — one row per injected memory).
|
|
100
|
+
*
|
|
101
|
+
* v0.5.0 (K5-014 / plan §8.10) — the pipeline is decomposed into
|
|
102
|
+
* `fetchSlice` (ranked retrieval + budget overflow) + `evaluate`
|
|
103
|
+
* (pure gate verdicts) + this orchestrator, so the read-only `plan()`
|
|
104
|
+
* can mirror it without any side effect (D5-08).
|
|
79
105
|
*/
|
|
80
106
|
private inject;
|
|
107
|
+
/**
|
|
108
|
+
* v0.5.0 (K5-014 / plan §8.10) — the ranked-retrieval stage shared by
|
|
109
|
+
* `inject` and `plan`. With `dry = true` it is a strict read:
|
|
110
|
+
*
|
|
111
|
+
* - the probe fetch never bumps (BUG-016), like the live path;
|
|
112
|
+
* - the overflow retry ALSO fetches with `bump: false` (the live path
|
|
113
|
+
* lets the retry fetch bump once — that is the only difference);
|
|
114
|
+
* - the no-retry bump is skipped.
|
|
115
|
+
*
|
|
116
|
+
* This is what lets `plan()` predict the EXACT slice the live path
|
|
117
|
+
* would inject without mutating a single relevance score.
|
|
118
|
+
*/
|
|
119
|
+
private fetchSlice;
|
|
120
|
+
/**
|
|
121
|
+
* v0.5.0 (K5-014 / plan §8.10, D5-08) — PUBLIC read-only prediction of
|
|
122
|
+
* what `inject` WOULD do for a query: same retrieval, same gate, zero
|
|
123
|
+
* side effects. Never moves a counter, never writes the seen-set, never
|
|
124
|
+
* bumps relevance, never records ledger rows. `kevin_trace` (K5-015)
|
|
125
|
+
* surfaces this to the agent; tests freeze the clock + settings around
|
|
126
|
+
* it.
|
|
127
|
+
*/
|
|
128
|
+
plan(query: string, options?: {
|
|
129
|
+
tag?: "context" | "memory";
|
|
130
|
+
cap?: number;
|
|
131
|
+
sessionId?: string;
|
|
132
|
+
}): InjectionPlan;
|
|
133
|
+
/**
|
|
134
|
+
* v0.5.0 (K5-014 / plan §8.10) — PURE gate evaluation shared by `admit`
|
|
135
|
+
* (live path) and `plan` (read-only path): returns the verdict for every
|
|
136
|
+
* candidate plus the seen-set as it WOULD look afterwards. Never writes
|
|
137
|
+
* state — the caller decides whether to persist.
|
|
138
|
+
*/
|
|
139
|
+
private evaluate;
|
|
140
|
+
/**
|
|
141
|
+
* v0.5.0 (K5-017 / plan §8.11, D5-11) — the effective pre-prompt cap,
|
|
142
|
+
* read at call time from `pre_prompt_budget_tokens` (seeded "900" by
|
|
143
|
+
* migration 006). Clamped to [100, 4000]; a non-numeric value falls
|
|
144
|
+
* back to 900. `kevin_trace` reports the value used via `plan().cap`.
|
|
145
|
+
*/
|
|
146
|
+
private prePromptCap;
|
|
81
147
|
/**
|
|
82
148
|
* v0.4.0 (K4-017) — QualityGate admission: filters the ranked slice to
|
|
83
149
|
* memories that may be injected this session, updating the session
|
|
84
150
|
* seen-set.
|
|
85
151
|
*
|
|
152
|
+
* v0.5.0 (K5-007 / plan §5.2, D5-04) — uses `canInjectVerdict` and
|
|
153
|
+
* increments the matching `injections_blocked_*` counter for every
|
|
154
|
+
* rejection, so gate policy becomes measurable. When `dryRun === true`
|
|
155
|
+
* the counters stay untouched (D5-08) — and so does the seen-set
|
|
156
|
+
* (a dry run is a strict read, K5-014).
|
|
157
|
+
*
|
|
86
158
|
* BUG-005 — strength/actionability now go through the REAL
|
|
87
159
|
* `QualityGate.evaluate` semantics (plan §5.1 rules 1-2), which this
|
|
88
160
|
* class had only re-derived from `metadata.dispatch`:
|