opencode-dejavu 2.1.0 → 2.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,43 @@
1
1
  # Changelog
2
2
 
3
+ ## 2.2.1 — 2026-08-24
4
+
5
+ ### Fixed (adversarial-review round)
6
+ - Escalation order: the gate is written to the global store BEFORE being removed from the project store — a crash between the two writes leaves a duplicate (healed by migrate), never a hole.
7
+ - `dejavu:proceed` inside quoted strings no longer bypasses gates (`echo "dejavu:proceed" && gated-cmd` stays enforced); the marker is honored only outside quotes.
8
+ - Concurrent first-encounter race: calls dispatched in the same burst as a REMINDER (within 500ms) are reminded too instead of slipping through as a "retry".
9
+ - CRLF/CR commands normalize identically to LF; `splitChain` splits on CR — no more line-ending fragmentation.
10
+ - `normalizeCommand` is fully idempotent: quoted spans are parameterized BEFORE path rules (a `<str>` substitution inserts spaces that would expose an adjacent `/` to the path rule only on a second pass), fingerprint payloads are trimmed, and already-parameterized payloads are never re-fingerprinted.
11
+ - Interpreter flags glued to their payload (`node -e"code"`) fingerprint identically to the spaced form.
12
+ - Session state maps: inner key sets are capped — long sessions no longer grow unbounded.
13
+
14
+ ### Added
15
+ - Lock degradation (contention > 3s) emits a `degraded` log event — the only window where concurrent writes can lose updates is now visible.
16
+ - `test/property.ts` — property-based tests for the normalization pipeline (idempotency, no nested tokens, output bound, one-liner distinctness, marker neutrality, splitChain atomicity).
17
+ - `test/fuzz.ts` — seeded mutation fuzzer with a metamorphic oracle and case shrinking; both harnesses run in CI. The harnesses caught the idempotency, marker-neutrality, glued-flag and nested-token-detector bugs above before production did.
18
+
19
+ ## 2.2.0 — 2026-08-23
20
+
21
+ ### Added
22
+ - Interpreter one-liner fingerprinting: `python -c` / `node -e` / `bun -e` and friends get their code payload hashed (`<code:sha1-8>`) instead of flattened to `<str>` — distinct scripts no longer share one gate, the same script failing repeatedly still converges.
23
+ - Global cross-project pattern index (`index.json`): counts distinct project dirs per failure key and now drives global escalation — a gate's own `projects` array only ever sees its own store, so escalation was dead code before.
24
+ - `mergeGate`: evidence merge for escalation and dedupe; never demotes a `blocking` gate.
25
+ - `isNoiseError()`: aborted/cancelled tool executions ("Tool execution aborted") are infrastructure noise and are no longer counted as failures.
26
+ - Self-healing stores (`src/validate.ts` invariant layer): every gate read from disk crosses strict parse + mechanical repair; `GateStore.reconcile()` quarantines unparseable gates.json (bytes preserved as `.corrupt-<ts>`), merges duplicate keys, excises unparseable log lines to `log.jsonl.corrupt`; `Stores.reconcileAll()` reconciles the index (prunes orphans, rebuilds missing entries, escalates gates proven in 2+ projects) — runs at every init.
27
+ - `doctor.ts [--repair]` now checks the full invariant set (shape, duplicates, temporal order, nested-token corruption, blocking without evidence, index consistency, stale project copies, missed escalation, log integrity) and heals on demand.
28
+
29
+ ### Changed
30
+ - `canBlock()` rejects bare one-liner shapes (`-c <str>`); existing gates with them are auto-demoted by `migrate()`.
31
+ - Log appends and rotation run under their own lock with atomic writes — concurrent OpenCode windows no longer interleave broken JSONL lines.
32
+ - `migrate()` merges project-local copies of already-global keys into the global gate.
33
+ - `doctor.ts`: NOT-TEACHING/ANNOYING only flag gates that can actually block; non-blockable legacy gates no longer scream.
34
+
35
+ ### Fixed
36
+ - All-digit `<code:...>` fingerprints are no longer re-parameterized by the number rule (~2.3% of payloads collapsed into one key).
37
+ - Signatures with different `<code:...>` fingerprints can no longer fuzzy-merge (random hashes differing in exactly 3 chars passed the distance rule and merged unrelated one-liners).
38
+ - `doctor.ts` no longer crashes on corrupt log lines; it now reports them as a CORRUPT LOG LINES pathology instead.
39
+ - Init failures (corrupt store, failed migrate) are logged instead of swallowed — a plugin starting on broken state is now visible.
40
+
3
41
  ## 2.1.0 — 2026-08-22
4
42
 
5
43
  First public release.
package/README.md CHANGED
@@ -28,7 +28,7 @@ Design decisions (post-mortem of existing approaches):
28
28
  - **Remind first, block on repeat.** Pure blocking starts an arms race — the agent routes around gates (`npm` blocked → uses `pnpm`). A reminder with the correction teaches; the block is reserved for ignored reminders.
29
29
  - **Gate messages are teachers.** Every message carries `CORRECTION:` (what to do instead) and `EVIDENCE:` (N failures across M sessions), not just a prohibition.
30
30
  - **Mechanical pattern-keys only.** No LLM-based error classification in the hot path — the unreliable component doesn't do reliability work.
31
- - **Two scopes.** Repo-specific gotchas live in `<repo>/.opencode/dejavu/` (committable); patterns seen in 2+ project dirs are agent-level habits and move to `~/.config/opencode/dejavu/`.
31
+ - **Two scopes.** Repo-specific gotchas live in `<repo>/.opencode/dejavu/` (committable); patterns seen in 2+ project dirs are agent-level habits and move to `~/.config/opencode/dejavu/`. No single store can see all projects, so a global pattern index (`index.json`) counts distinct project dirs per key and drives the escalation.
32
32
  - **Gates rot — so they expire.** 60 days without recurrence and a gate is dropped. A gate firing 10+ times while the error stopped gets `review: true` for manual inspection.
33
33
  - **The metric is recurrence-after-gate.** Tracked per gate as `recurredAfterGate` — if gates don't reduce recurrence, the whole approach is wrong and you'll see it in the data.
34
34
 
@@ -61,18 +61,21 @@ Restart OpenCode. Gates appear automatically as failures recur — nothing to co
61
61
  ## Robustness & safety
62
62
 
63
63
  - **Blocking policy** — only `bash` commands that are NOT diagnostics may ever become blocking gates. File probes (read/edit/write/glob/grep) and diagnostics (tsc/eslint/pytest/gradle-test/flutter/curl/grep...) stay `watching` forever: measured, visible in reports, but never interrupting the agent. `canBlock()` in `src/patterns.ts` is the single source of truth.
64
+ - **One-liner identity** — for `python -c` / `node -e` / `bun -e` and friends the code payload IS the call, so it is fingerprinted (`<code:hash>`) instead of flattened to `<str>`: different scripts never share a gate, the same script failing repeatedly still converges. Legacy bare `-c <str>` shapes can never block.
64
65
  - **Secret scrubbing** — every signature and snippet passes `scrubSecrets()` (OpenAI/Anthropic/AWS/GitHub/Slack/Stripe/JWT/bearer/DB-conn-string/PEM patterns + `root@host`) before touching disk. Historical data is cleaned by `migrate()` at init or via `bun scripts/migrate.ts <dirs...>` (also scrubs logs).
65
66
  - **Intended non-zero exits** — exit 1 from diagnostics is NOT a failure (that is their normal "found nothing / found issues" outcome). Exit ≥ 2 always counts.
67
+ - **Aborted ≠ failed** — cancelled/aborted tool executions ("Tool execution aborted") are infrastructure noise and are never counted as failures.
66
68
  - **File content is not command output** — text failure signatures are scanned for `bash` only; `read`/`edit`/`write` failures come exclusively from the event channel (a file containing "TypeError" is not a failure).
67
- - **Concurrency** — gates.json mutations run under an exclusive lockfile; writes are tmp+rename with EPERM/EACCES/EBUSY retry (Windows AV/indexer). NT long paths get the `\\?\` prefix.
69
+ - **Concurrency** — gates.json mutations run under an exclusive lockfile; log appends and rotation take their own lock (every OpenCode window shares the global log); writes are tmp+rename with EPERM/EACCES/EBUSY retry (Windows AV/indexer). NT long paths get the `\\?\` prefix. If a lock cannot be acquired within 3s the critical section degrades to unlocked (the tool pipeline must never hang) and emits a `degraded` log event — the only window where updates can be lost is visible.
68
70
  - **Near-duplicate consolidation** — new failures merge into existing patterns via normalized Levenshtein ≤ 0.3 with an absolute floor of 3 edits (replaces token Jaccard, which collapsed all `<str>` placeholders; the floor stops `git push` vs `git pull`-style merges).
69
71
  - **Bounded memory** — per-session maps are capped (200 sessions) and freed on `session.deleted`; handled part IDs evict FIFO; TTL expiry re-runs every 6 h in long-lived processes.
70
- - **Migration** — gates outside the blocking policy are demoted to `watching` automatically; nothing is deleted.
72
+ - **Migration** — gates outside the blocking policy are demoted to `watching` automatically; project copies of already-global gates are merged into the global gate (evidence is consolidated, never deleted).
73
+ - **Self-healing** — every init reconciles the stores: an unparseable `gates.json` is quarantined (bytes preserved as `gates.json.corrupt-<ts>`), gate records are strictly parsed and mechanically repaired (inverted dates swapped, duplicate keys merged, secrets re-scrubbed, stale blocking demoted), unparseable log lines are excised to `log.jsonl.corrupt`, and the cross-project index is reconciled. Every repair is logged as a `repaired`/`quarantined` event.
71
74
 
72
75
  ## Observability (debugging aids)
73
76
 
74
77
  - Every `log.jsonl` gets an `init` event with `PLUGIN_VERSION`; `detected` events carry `channel` (`exit`/`text`/`event`) and the raw exit code; `reminded`/`blocked` carry `via` (`exact`/`fuzzy`/`segment`). Stale plugin sessions are therefore visible in the data.
75
- - `bun scripts/doctor.ts [projectDirs...]` — one-command pathology report: blocking gates outside policy, not-teaching gates (recurredAfterGate 3), annoying gates (reminded 10), secrets on disk, version drift.
78
+ - `bun scripts/doctor.ts [--repair] [projectDirs...]` — one-command report over every invariant the data model implies: gate shape, duplicate keys, temporal order, nested-token corruption, blocking without evidence, policy violations, index↔gates consistency, stale project copies, missed escalation, log integrity, secrets, version drift. `--repair` heals first (idempotent), then reports.
76
79
  - `bun scripts/analyze.ts [projectDirs...]` — store summary: statuses, tools, top patterns.
77
80
  - `/dejavu` command (installed globally) runs doctor first, then reports.
78
81
 
@@ -93,8 +96,10 @@ Not covered (by design, v1): semantically-equivalent-but-syntactically-different
93
96
  | File | Contents |
94
97
  |---|---|
95
98
  | `~/.config/opencode/dejavu/gates.json` | global gates (agent habits) |
99
+ | `~/.config/opencode/dejavu/index.json` | cross-project pattern index: which project dirs each failure key was seen in (escalation evidence) |
96
100
  | `<repo>/.opencode/dejavu/gates.json` | project gates (repo gotchas) |
97
- | `*/dejavu/log.jsonl` | every event: detected, promoted, reminded, blocked, override, expired, recurred-after-gate |
101
+ | `*/dejavu/log.jsonl` | every event: detected, promoted, reminded, blocked, override, expired, recurred-after-gate, repaired, quarantined |
102
+ | `*/dejavu/*.corrupt*` | quarantined corruption (unparseable gates.json, excised log lines) — bytes preserved for forensics; safe to delete after inspection |
98
103
 
99
104
  Both are human-editable. Removing a gate object disables it. Editing `correction` improves what the agent is told.
100
105
 
package/index.ts CHANGED
@@ -6,6 +6,7 @@ import {
6
6
  callSignature,
7
7
  detectFailure,
8
8
  isIntendedNonzero,
9
+ isNoiseError,
9
10
  parameterizeError,
10
11
  patternKey,
11
12
  scrubSecrets,
@@ -24,6 +25,12 @@ const TTL_INTERVAL_MS = 6 * 60 * 60 * 1000
24
25
  const REVIEW_FIRES = 10
25
26
  /** per-session state maps are capped to bound memory in long-lived processes */
26
27
  const SESSION_MAP_CAP = 200
28
+ /** per-session key sets are capped too — one long session must not grow unbounded */
29
+ const SESSION_KEY_CAP = 500
30
+ /** a "retry" arriving this soon after a reminder was dispatched concurrently with it
31
+ * (same tool-call burst) and never saw the reminder — it gets reminded as well.
32
+ * A true agent retry needs a full model turn (≥1s in practice), so 500ms separates both. */
33
+ const REMINDER_RACE_WINDOW_MS = 500
27
34
  /** handled part IDs are capped FIFO-style */
28
35
  const HANDLED_CAP = 5000
29
36
  const HANDLED_KEEP = 2500
@@ -40,10 +47,15 @@ function addToSetMap(map: Map<string, Set<string>>, outer: string, inner: string
40
47
  map.set(outer, set)
41
48
  }
42
49
  set.add(inner)
50
+ while (set.size > SESSION_KEY_CAP) {
51
+ const oldest = set.values().next()
52
+ if (oldest.done) break
53
+ set.delete(oldest.value)
54
+ }
43
55
  }
44
56
 
45
57
  /** Drop oldest entries (Map preserves insertion order) to bound memory. */
46
- function capMap(map: Map<string, Set<string>>, cap: number): void {
58
+ function capMap<K, V>(map: Map<K, V>, cap: number): void {
47
59
  while (map.size > cap) {
48
60
  const oldest = map.keys().next()
49
61
  if (oldest.done) break
@@ -88,8 +100,8 @@ export const Dejavu: Plugin = async ({ directory, client }) => {
88
100
  : null
89
101
  const stores = new Stores(globalStore, projectStore)
90
102
 
91
- /** sessions in which a gate key was already reminded about */
92
- const reminded = new Map<string, Set<string>>()
103
+ /** sessions in which a gate key was already reminded about; value = remind time (race guard) */
104
+ const reminded = new Map<string, Map<string, number>>()
93
105
  /** sessions in which a reminded pattern failed again — next attempt is blocked */
94
106
  const failedAfterReminder = new Map<string, Set<string>>()
95
107
  /** callID -> signature fallback when the after-hook does not receive args */
@@ -105,15 +117,19 @@ export const Dejavu: Plugin = async ({ directory, client }) => {
105
117
  }
106
118
  }
107
119
 
108
- // Init: migrate old data, expire stale gates, rotate logs, warm the caches.
120
+ // Init: heal structural damage, migrate old data, expire stale gates,
121
+ // rotate logs, warm the caches.
109
122
  try {
123
+ await stores.reconcileAll(GLOBAL_PROJECTS)
110
124
  await stores.migrate()
111
125
  await stores.expireAll(TTL_DAYS)
112
126
  await stores.rotateLogs()
113
127
  await stores.logAll({ type: "init", key: "dejavu", version: PLUGIN_VERSION })
114
128
  await logClient("info", `dejavu initialized v${PLUGIN_VERSION}`)
115
- } catch {
116
- // init failures must not prevent hook registration
129
+ } catch (error) {
130
+ // init failures must not prevent hook registration — but must be visible,
131
+ // otherwise a corrupted store silently starts the plugin with no gates
132
+ await logClient("error", `dejavu init failed: ${error instanceof Error ? error.message : String(error)}`)
117
133
  }
118
134
 
119
135
  // Long-lived processes re-run expiry periodically.
@@ -164,7 +180,10 @@ export const Dejavu: Plugin = async ({ directory, client }) => {
164
180
  const session = typeof input.sessionID === "string" ? input.sessionID : "unknown"
165
181
 
166
182
  // Explicit escape hatch — checked only in the actionable text field,
167
- // with word boundaries, so unrelated args cannot bypass gates.
183
+ // with word boundaries, so unrelated args cannot bypass gates. Quoted
184
+ // spans are stripped first: `echo "dejavu:proceed" && gated-cmd` must
185
+ // NOT bypass the gate on the chained command — the marker is a
186
+ // comment-style annotation, not data.
168
187
  const commandText =
169
188
  typeof rawArgs.command === "string"
170
189
  ? rawArgs.command
@@ -173,7 +192,7 @@ export const Dejavu: Plugin = async ({ directory, client }) => {
173
192
  : typeof rawArgs.filePath === "string"
174
193
  ? rawArgs.filePath
175
194
  : ""
176
- if (/\bdejavu:proceed\b/.test(commandText)) {
195
+ if (/\bdejavu:proceed\b/.test(commandText.replace(/"[^"]*"|'[^']*'/g, " "))) {
177
196
  await stores.logAll({ type: "override", key: gate.key, tool: gate.tool, session, project: directory })
178
197
  return
179
198
  }
@@ -188,9 +207,16 @@ export const Dejavu: Plugin = async ({ directory, client }) => {
188
207
  }
189
208
 
190
209
  // First encounter this session -> remind (the call is aborted; agent may retry corrected).
191
- if (!reminded.get(session)?.has(gate.key)) {
192
- addToSetMap(reminded, session, gate.key)
193
- addToSetMap(reminded, session, patternKey(signature)) // exact key too: retry may fuzzy-match differently
210
+ // Race guard: calls dispatched in the same burst all arrive before the agent can
211
+ // have seen any reminder, so a "retry" within REMINDER_RACE_WINDOW_MS of the
212
+ // remind is itself a concurrent first encounter and gets reminded too.
213
+ const sessionReminded = reminded.get(session) ?? new Map<string, number>()
214
+ if (!reminded.has(session)) reminded.set(session, sessionReminded)
215
+ const remindedAt = sessionReminded.get(gate.key)
216
+ if (remindedAt === undefined || Date.now() - remindedAt < REMINDER_RACE_WINDOW_MS) {
217
+ sessionReminded.set(gate.key, Date.now())
218
+ sessionReminded.set(patternKey(signature), Date.now()) // exact key too: retry may fuzzy-match differently
219
+ capMap(sessionReminded, SESSION_KEY_CAP)
194
220
  capMap(reminded, SESSION_MAP_CAP)
195
221
  gate.remindedCount += 1
196
222
  await found.store.save()
@@ -347,6 +373,8 @@ export const Dejavu: Plugin = async ({ directory, client }) => {
347
373
  // Never count our own gate signals as failures — a thrown REMINDER/BLOCK
348
374
  // comes back through this channel as a tool error.
349
375
  if (errorText.includes("[dejavu]")) return
376
+ // Aborted/cancelled executions are infrastructure noise, not mistakes.
377
+ if (isNoiseError(errorText)) return
350
378
  const session = typeof p.sessionID === "string" ? p.sessionID : "unknown"
351
379
 
352
380
  // Prefer the real call signature from the tool input — it keeps the gate
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "opencode-dejavu",
3
- "version": "2.1.0",
3
+ "version": "2.2.1",
4
4
  "description": "Cross-session memory prosthesis for OpenCode: detects recurring tool-call failures and promotes them into enforced gates. Remind first, block on same-session repeat offense.",
5
5
  "type": "module",
6
6
  "main": "index.ts",
package/src/AGENTS.md CHANGED
@@ -9,29 +9,41 @@ Two dependency-free modules: `patterns.ts` (pure functions — call identity, no
9
9
  | Task | File | Symbols |
10
10
  |------|------|---------|
11
11
  | Call identity / gate keys | patterns.ts | `callSignature` → `normalizeCommand`/`normalizeFilePath` → `patternKey` |
12
+ | Interpreter one-liner identity | patterns.ts | `hashInterpreterPayload` — `-c`/`-e` code payload → `<code:hash>` |
12
13
  | Chain-bypass protection | patterns.ts | `splitChain` (quote/paren-aware) → `bashSegmentSignatures` |
13
14
  | Free-form error collapsing | patterns.ts | `parameterizeError` (event channel) vs `normalizeCommand` (bash) |
14
15
  | Near-duplicate merge | patterns.ts | `fuzzySimilar` = normalized `levenshtein` ≤ 0.3 |
15
16
  | Failure text scan | patterns.ts | `detectFailure` + `FAILURE_SIGNATURES` |
17
+ | Noise filtering | patterns.ts | `isNoiseError` + `NOISE_ERRORS` (aborted/cancelled ≠ failed) |
16
18
  | Diagnostic/intended-exit logic | patterns.ts | `DIAGNOSTIC_VERBS`, `isIntendedNonzero`, `canBlock` |
17
- | One scope (gates.json + log.jsonl) | store.ts | `GateStore` — `load`/`save`/`log`/`expire`/`rotateLog` |
18
- | Two-scope logic + promotion | store.ts | `Stores` — `findGate`/`recordFailure`/`migrate`/`blockingGates` |
19
+ | One scope (gates.json + index.json + log.jsonl) | store.ts | `GateStore` — `load`/`save`/`loadIndex`/`saveIndex`/`log`/`expire`/`extract`/`rotateLog`/`reconcile` |
20
+ | Two-scope logic + promotion | store.ts | `Stores` — `findGate`/`recordFailure`/`migrate`/`blockingGates`/`reconcileAll`; `mergeGate` merges duplicate keys |
21
+ | Gate parse/repair boundary | validate.ts | `coerceGateShape` (strict parse), `repairGate` (mechanical coercion), `hasNestedTokens` (corruption fingerprint) |
19
22
  | fs safety | store.ts | `ntPath`, `atomicWrite`, `withLock` |
20
23
 
21
24
  ## INVARIANTS (do not break)
22
25
 
23
26
  - Rule order in `PARAM_RULES` matters: quoted strings first, specific tokens (uuid/sha/ip/url/date), generic numbers last — reordering fragments signatures
27
+ - Rule order in `normalizeCommand` matters too: quoted strings are parameterized BEFORE path rules — a `<str>` substitution inserts spaces that would expose an adjacent `/` to the path rule on a second pass (idempotency); interpreter payload hashing runs while the payload is still raw
24
28
  - `scrubSecrets()` runs on every string before it touches disk; `recordFailure` re-scrubs defensively
25
- - `canBlock(tool, sig)` = `tool === "bash" && !diagnostic` — the ONLY path to `blocking`; probe tools use `PROMOTE_COUNT_PROBE` and never block
29
+ - `canBlock(tool, sig)` = bash && non-diagnostic && not a bare one-liner shape — the ONLY path to `blocking`; probe tools use `PROMOTE_COUNT_PROBE` and never block
26
30
  - `DIAGNOSTIC_VERBS` serves two callers (exit-1 allowlist + blocking policy) — one list, two uses; edit knowing both move
27
- - Lock order is always project → global (see `recordFailure` escalation) — reversing deadlocks
31
+ - Lock order is always project → global, gates → index (see `recordFailure` escalation) — reversing deadlocks; the log lock is separate and leaf-level
32
+ - Cross-project evidence lives ONLY in the global `index.json` — a gate's own `projects` array sees one store and never drives escalation alone
33
+ - Escalation writes the global gate FIRST, then removes the project copy — a crash between the two writes must leave a duplicate (healed by migrate), never a hole
28
34
  - Inside `runLocked` always `load(true)`; unlocked `load()` peeks are routing hints only, never a basis for mutation
29
35
  - `GateStore.load` caches by mtime — after external edits the cache refreshes on next stat; `save()` refreshes it manually
36
+ - Log appends and rotation take the log lock — every OpenCode window shares the global log; unlocked appends interleave into broken JSON
37
+ - Every gate read from disk crosses `coerceGateShape` + `repairGate` in `load()` — enforcement never sees raw state; hopeless records are dropped, repairable ones coerced
38
+ - Quarantine preserves bytes: unparseable files are renamed to `*.corrupt-*`, never deleted; every repair emits a `repaired`/`quarantined` log event
30
39
  - Fuzzy matching is Levenshtein-based on purpose: token Jaccard collapsed all `<str>` placeholders into one bucket; ratio ≤ 0.3 PLUS absolute distance ≥ 3 (verb-level-different commands must never merge)
31
40
 
32
41
  ## ANTI-PATTERNS
33
42
 
34
43
  - Do NOT add a tool to `callSignature` without deciding its class: `PROBE_TOOLS` (higher bar, never blocks) or bash-class
35
44
  - Do NOT widen `FAILURE_SIGNATURES` to cover file-tool output — that text is file content; extend the event channel instead
45
+ - Do NOT flatten interpreter one-liner payloads to `<str>` — `hashInterpreterPayload` must run before string parameterization; a bare `-c <str>` gate would block the whole command family
46
+ - Do NOT count aborted/cancelled executions as failures — filter with `isNoiseError()` at the event channel
47
+ - Do NOT widen `coerceGateShape` drop rules casually — records it deems hopeless are silently dropped on every load; a bad rule silently empties stores
36
48
  - Do NOT write gates.json directly — always `runLocked` + `save()` (atomicWrite); logs are append-only via `log()`
37
49
  - Do NOT let `withLock` throw on contention — it degrades to unlocked after `LOCK_WAIT_MS` by design (pipeline must not hang)
package/src/patterns.ts CHANGED
@@ -43,18 +43,56 @@ export function scrubSecrets(text: string): string {
43
43
 
44
44
  // --- Normalization -----------------------------------------------------------
45
45
 
46
+ /**
47
+ * Interpreter one-liners: the quoted argument IS the program. Parameterizing
48
+ * it to <str> collapsed every script into one key — "python -c <str>" ended up
49
+ * blocking ALL python -c calls after three unrelated failures. Fingerprint the
50
+ * payload instead: same code = same key, different code = different key.
51
+ * Secrets are scrubbed before hashing so they neither persist nor fragment.
52
+ */
53
+ const INTERPRETER_ONELINER =
54
+ /(?:^|[|;&(\n]\s*)(?:\S+[\\/])?(python3?|node|bun|deno|perl|ruby|pwsh|powershell)(?:\.exe)?(?:\s+-\w+)*\s+(-c|-e|--eval|-command)\s*/i
55
+
56
+ function hashInterpreterPayload(command: string): string {
57
+ const match = INTERPRETER_ONELINER.exec(command)
58
+ if (!match) return command
59
+ const payload = command.slice(match.index + match[0].length)
60
+ if (payload.trim() === "") return command
61
+ // Already fingerprinted (re-normalization) — keep the existing token so
62
+ // normalizeCommand stays idempotent.
63
+ if (/^<code:[0-9a-f]+>$/.test(payload.trim())) return command
64
+ // Already-parameterized placeholders are data, not code — never hash them
65
+ // (idempotency: a second pass must not fingerprint a <str>).
66
+ if (/^(?:<(?:str|path|n|hash|uuid|sha|md5|ip|url|email|date)>\s*)+$/.test(payload.trim())) return command
67
+ // For whole (unchained) commands the payload runs to end of string; chain
68
+ // segments are normalized separately, so segment keys stay exact.
69
+ // Trim before hashing: trailing whitespace (e.g. a stripped override marker)
70
+ // is not part of the code's identity.
71
+ const fingerprint = createHash("sha1").update(scrubSecrets(payload.trim())).digest("hex").slice(0, 8)
72
+ return `${command.slice(0, match.index + match[0].length)}<code:${fingerprint}>`
73
+ }
74
+
46
75
  /**
47
76
  * Normalize a bash command into a stable signature.
48
77
  * Paths, numbers, quoted strings, hashes and agent comments are abstracted
49
78
  * away so that "same failure, different instance" collapses into one pattern.
50
79
  */
51
80
  export function normalizeCommand(command: string): string {
52
- let s = command.replace(COMMENT_LINE, "$1").toLowerCase()
81
+ // CRLF/CR commands (Windows pastes, agent multi-line) normalize to LF —
82
+ // otherwise the same command fragments across line-ending styles.
83
+ let s = command.replace(/\r\n?/g, "\n")
84
+ s = s.replace(COMMENT_LINE, "$1").toLowerCase()
85
+ s = hashInterpreterPayload(s)
86
+ // Quoted spans come out FIRST: they are data, and removing them before the
87
+ // path rules keeps normalization idempotent — a <str> replacement inserts
88
+ // spaces that would otherwise expose an adjacent "/" to the path rule only
89
+ // on a second pass.
90
+ s = s.replace(/"[^"]*"|'[^']*'/g, " <str> ")
53
91
  s = s.replace(/[a-z]:[\\/][^\s"']+/gi, " <path> ")
54
92
  s = s.replace(/(^|\s)\/[^\s"']+/g, "$1<path> ")
55
- s = s.replace(/"[^"]*"|'[^']*'/g, " <str> ")
56
- s = s.replace(/\b[0-9a-f]{7,64}\b/gi, " <hash> ")
57
- s = s.replace(/\b\d[\d.]*\b/g, " <n> ")
93
+ // lookbehind: never re-parameterize the <code:...> fingerprint hex
94
+ s = s.replace(/(?<!<code:)\b[0-9a-f]{7,64}\b/gi, " <hash> ")
95
+ s = s.replace(/(?<!<code:)\b\d[\d.]*\b/g, " <n> ")
58
96
  s = s.replace(/\s+/g, " ").trim()
59
97
  return s
60
98
  }
@@ -123,6 +161,14 @@ export function isIntendedNonzero(command: string, exitCode: number): boolean {
123
161
  return exitCode === 1 && isDiagnosticText(command)
124
162
  }
125
163
 
164
+ /**
165
+ * A code flag whose payload was entirely parameterized away (`-c <str>`)
166
+ * carries no identity — blocking that shape blocks the whole command family.
167
+ * New one-liners get <code:...> fingerprints in normalizeCommand; this guard
168
+ * keeps legacy pre-fingerprint gates (and lookalikes) from ever blocking.
169
+ */
170
+ const GENERIC_ONELINER_SHAPE = /(^|\s)(-c|-e|--eval|-command)\s+(?:@\s+)?<str>(?:\s+@)?\s*$/i
171
+
126
172
  /**
127
173
  * Blocking policy: only bash commands that are NOT diagnostics may ever
128
174
  * become enforced gates. File probes and diagnostic queries are measured
@@ -130,7 +176,9 @@ export function isIntendedNonzero(command: string, exitCode: number): boolean {
130
176
  * punishes normal work.
131
177
  */
132
178
  export function canBlock(tool: string, signature: string): boolean {
133
- return tool === "bash" && !isDiagnosticSignature(signature)
179
+ if (tool !== "bash") return false
180
+ if (isDiagnosticSignature(signature)) return false
181
+ return !GENERIC_ONELINER_SHAPE.test(signature)
134
182
  }
135
183
 
136
184
  // --- Chain splitting ---------------------------------------------------------
@@ -180,7 +228,7 @@ export function splitChain(command: string): string[] {
180
228
  continue
181
229
  }
182
230
  if (depth === 0) {
183
- if (ch === ";" || ch === "\n") {
231
+ if (ch === ";" || ch === "\n" || ch === "\r") {
184
232
  flush()
185
233
  i += 1
186
234
  continue
@@ -281,14 +329,25 @@ export function levenshtein(a: string, b: string): number {
281
329
  return prev[n] ?? 0
282
330
  }
283
331
 
332
+ /** Code fingerprints are IDENTITY, not data — they must match exactly. */
333
+ const CODE_FINGERPRINTS = /<code:[0-9a-f]+>/g
334
+
284
335
  /**
285
336
  * Near-duplicate match: normalized edit distance <= 30% AND absolute distance
286
337
  * >= 3. Unlike token-set Jaccard, this does not collapse commands that merely
287
338
  * share placeholder tokens; the absolute floor stops verb-level-different
288
339
  * commands ("git push <str>" vs "git pull <str>" = distance 2) from merging.
340
+ * Signatures carrying <code:...> fingerprints only match if the fingerprints
341
+ * are identical — random hashes differing in 3 chars would otherwise pass the
342
+ * distance rule and merge unrelated one-liners into one gate.
289
343
  */
290
344
  export function fuzzySimilar(a: string, b: string): boolean {
291
345
  if (a === b) return true
346
+ const codesA = a.match(CODE_FINGERPRINTS)
347
+ const codesB = b.match(CODE_FINGERPRINTS)
348
+ if (codesA !== null || codesB !== null) {
349
+ if (codesA === null || codesB === null || codesA.join("\u0000") !== codesB.join("\u0000")) return false
350
+ }
292
351
  const maxLen = Math.max(a.length, b.length)
293
352
  if (maxLen === 0) return true
294
353
  const distance = levenshtein(a, b)
@@ -331,3 +390,22 @@ export function detectFailure(outputText: string): FailureDetection {
331
390
  }
332
391
  return { matched: false, snippet: "" }
333
392
  }
393
+
394
+ // --- Noise filtering ----------------------------------------------------------
395
+
396
+ /**
397
+ * Infrastructure noise, not agent mistakes: aborted/cancelled executions
398
+ * (user hit stop, background task reaped) teach nothing and fragmented the
399
+ * store with unactionable patterns. Aborted != failed.
400
+ */
401
+ const NOISE_ERRORS: RegExp[] = [
402
+ /tool execution aborted/i,
403
+ /execution was aborted/i,
404
+ /\baborted by user\b/i,
405
+ /\bcancelled by user\b/i,
406
+ /\bcanceled by user\b/i,
407
+ ]
408
+
409
+ export function isNoiseError(errorText: string): boolean {
410
+ return NOISE_ERRORS.some((rule) => rule.test(errorText))
411
+ }
package/src/store.ts CHANGED
@@ -1,9 +1,10 @@
1
1
  import { appendFile, mkdir, readFile, rename, stat, unlink, writeFile } from "node:fs/promises"
2
2
  import { dirname, join } from "node:path"
3
3
  import { canBlock, fuzzySimilar, scrubSecrets } from "./patterns"
4
+ import { coerceGateShape, repairGate } from "./validate"
4
5
 
5
6
  /** Bumped on behavior changes; stamped into init log events so stale sessions are visible. */
6
- export const PLUGIN_VERSION = "2.1.0"
7
+ export const PLUGIN_VERSION = "2.2.1"
7
8
 
8
9
  export interface Gate {
9
10
  /** sha1 signature prefix — the pattern identity */
@@ -38,6 +39,17 @@ interface GatesFile {
38
39
  gates: Gate[]
39
40
  }
40
41
 
42
+ /** Cross-project pattern index: which project dirs have seen each key. */
43
+ interface IndexEntry {
44
+ projects: string[]
45
+ lastSeen: string
46
+ }
47
+
48
+ interface IndexFile {
49
+ version: 1
50
+ keys: Record<string, IndexEntry>
51
+ }
52
+
41
53
  export type LogEventType =
42
54
  | "detected"
43
55
  | "promoted"
@@ -48,6 +60,9 @@ export type LogEventType =
48
60
  | "expired"
49
61
  | "recurred-after-gate"
50
62
  | "init"
63
+ | "repaired"
64
+ | "quarantined"
65
+ | "degraded"
51
66
 
52
67
  export interface LogEvent {
53
68
  type: LogEventType
@@ -123,7 +138,7 @@ const LOCK_WAIT_MS = 3000
123
138
  * degradation: if the lock cannot be acquired within LOCK_WAIT_MS the
124
139
  * critical section runs unlocked rather than hanging the tool pipeline.
125
140
  */
126
- async function withLock<T>(lockTarget: string, fn: () => Promise<T>): Promise<T> {
141
+ async function withLock<T>(lockTarget: string, fn: () => Promise<T>, onDegrade?: () => void): Promise<T> {
127
142
  const lock = `${lockTarget}.lock`
128
143
  await mkdir(ntPath(dirname(lock)), { recursive: true })
129
144
  const started = Date.now()
@@ -143,7 +158,11 @@ async function withLock<T>(lockTarget: string, fn: () => Promise<T>): Promise<T>
143
158
  } catch {
144
159
  continue // lock vanished between attempts
145
160
  }
146
- if (Date.now() - started > LOCK_WAIT_MS) break
161
+ if (Date.now() - started > LOCK_WAIT_MS) {
162
+ // The only window where concurrent writes can lose updates — make it visible.
163
+ if (onDegrade) onDegrade()
164
+ break
165
+ }
147
166
  await new Promise((resolve) => setTimeout(resolve, 50))
148
167
  }
149
168
  }
@@ -161,6 +180,8 @@ async function withLock<T>(lockTarget: string, fn: () => Promise<T>): Promise<T>
161
180
  export class GateStore {
162
181
  private gates: Gate[] | null = null
163
182
  private mtimeMs = 0
183
+ private index: IndexFile | null = null
184
+ private indexMtimeMs = 0
164
185
 
165
186
  constructor(public readonly dir: string) {}
166
187
 
@@ -172,12 +193,22 @@ export class GateStore {
172
193
  return join(this.dir, "log.jsonl")
173
194
  }
174
195
 
196
+ private get indexPath(): string {
197
+ return join(this.dir, "index.json")
198
+ }
199
+
175
200
  /** Run a load→mutate→save section under the store's exclusive lock. */
176
201
  async runLocked<T>(fn: () => Promise<T>): Promise<T> {
177
- return withLock(this.gatesPath, fn)
202
+ return withLock(this.gatesPath, fn, () => {
203
+ this.log({ type: "degraded", key: "gates.lock", snippet: `lock contention exceeded ${LOCK_WAIT_MS}ms; critical section ran unlocked` }).catch(() => {})
204
+ })
178
205
  }
179
206
 
180
- /** force=true bypasses the mtime cache (always used inside locks). */
207
+ /**
208
+ * force=true bypasses the mtime cache (always used inside locks).
209
+ * Every record crosses the validation boundary: hopeless records are
210
+ * dropped, repairable ones coerced — enforcement never sees raw state.
211
+ */
181
212
  async load(force = false): Promise<Gate[]> {
182
213
  try {
183
214
  const info = await stat(ntPath(this.gatesPath))
@@ -186,7 +217,15 @@ export class GateStore {
186
217
  }
187
218
  const raw = await readFile(ntPath(this.gatesPath), "utf8")
188
219
  const parsed = JSON.parse(raw) as Partial<GatesFile>
189
- this.gates = Array.isArray(parsed.gates) ? parsed.gates : []
220
+ const records = Array.isArray(parsed.gates) ? parsed.gates : []
221
+ const gates: Gate[] = []
222
+ for (const record of records) {
223
+ const gate = coerceGateShape(record)
224
+ if (gate === null) continue
225
+ repairGate(gate)
226
+ gates.push(gate)
227
+ }
228
+ this.gates = gates
190
229
  this.mtimeMs = info.mtimeMs
191
230
  return this.gates
192
231
  } catch {
@@ -208,10 +247,54 @@ export class GateStore {
208
247
  }
209
248
  }
210
249
 
211
- async log(event: LogEvent): Promise<void> {
250
+ /** Cross-project pattern index; meaningful only on the global store. */
251
+ async loadIndex(force = false): Promise<IndexFile> {
252
+ try {
253
+ const info = await stat(ntPath(this.indexPath))
254
+ if (!force && this.index !== null && info.mtimeMs === this.indexMtimeMs) {
255
+ return this.index
256
+ }
257
+ const raw = await readFile(ntPath(this.indexPath), "utf8")
258
+ const parsed = JSON.parse(raw) as Partial<IndexFile>
259
+ const keys = parsed.keys
260
+ this.index = { version: 1, keys: keys !== null && typeof keys === "object" ? keys : {} }
261
+ this.indexMtimeMs = info.mtimeMs
262
+ return this.index
263
+ } catch {
264
+ // missing or unreadable index — treat as empty
265
+ if (this.index === null) this.index = { version: 1, keys: {} }
266
+ return this.index
267
+ }
268
+ }
269
+
270
+ async saveIndex(): Promise<void> {
271
+ if (this.index === null) return
212
272
  await mkdir(ntPath(this.dir), { recursive: true })
213
- const line = `${JSON.stringify({ ts: new Date().toISOString(), ...event })}\n`
214
- await appendFile(ntPath(this.logPath), line, "utf8")
273
+ await atomicWrite(this.indexPath, `${JSON.stringify(this.index, null, 2)}\n`)
274
+ try {
275
+ this.indexMtimeMs = (await stat(ntPath(this.indexPath))).mtimeMs
276
+ } catch {
277
+ // mtime refresh is best-effort
278
+ }
279
+ }
280
+
281
+ /** Run an index load→mutate→save section under the index's own lock. */
282
+ async runLockedIndex<T>(fn: () => Promise<T>): Promise<T> {
283
+ return withLock(this.indexPath, fn, () => {
284
+ this.log({ type: "degraded", key: "index.lock", snippet: `lock contention exceeded ${LOCK_WAIT_MS}ms; critical section ran unlocked` }).catch(() => {})
285
+ })
286
+ }
287
+
288
+ /**
289
+ * Append under the log lock: every OpenCode window shares the global log,
290
+ * and unlocked concurrent appends interleave into broken JSON lines.
291
+ */
292
+ async log(event: LogEvent): Promise<void> {
293
+ await withLock(this.logPath, async () => {
294
+ await mkdir(ntPath(this.dir), { recursive: true })
295
+ const line = `${JSON.stringify({ ts: new Date().toISOString(), ...event })}\n`
296
+ await appendFile(ntPath(this.logPath), line, "utf8")
297
+ })
215
298
  }
216
299
 
217
300
  /** Caller must hold the lock. */
@@ -225,20 +308,151 @@ export class GateStore {
225
308
  return expired
226
309
  }
227
310
 
311
+ /** Remove gates by key; caller must hold the lock. Returns the removed gates. */
312
+ extract(keys: Set<string>): Gate[] {
313
+ if (this.gates === null) return []
314
+ const removed = this.gates.filter((g) => keys.has(g.key))
315
+ if (removed.length > 0) this.gates = this.gates.filter((g) => !keys.has(g.key))
316
+ return removed
317
+ }
318
+
228
319
  async rotateLog(): Promise<void> {
320
+ await withLock(this.logPath, async () => {
321
+ try {
322
+ const info = await stat(ntPath(this.logPath))
323
+ if (info.size < LOG_ROTATE_BYTES) return
324
+ const raw = await readFile(ntPath(this.logPath), "utf8")
325
+ const lines = raw.split("\n").filter((l) => l.trim() !== "")
326
+ const kept = lines.slice(-LOG_ROTATE_KEEP_LINES)
327
+ await atomicWrite(this.logPath, `${kept.join("\n")}\n`)
328
+ } catch {
329
+ // missing or unreadable log is fine
330
+ }
331
+ })
332
+ }
333
+
334
+ /**
335
+ * Structural self-healing (idempotent): unparseable gates.json is
336
+ * quarantined with its bytes preserved; parseable records are coerced,
337
+ * repaired and deduped; unparseable log lines are excised to
338
+ * log.jsonl.corrupt. Every repair is logged — healing must be visible.
339
+ */
340
+ async reconcile(): Promise<void> {
341
+ await withLock(this.gatesPath, async () => {
342
+ let raw: string | null = null
343
+ try {
344
+ raw = await readFile(ntPath(this.gatesPath), "utf8")
345
+ } catch {
346
+ // no gates file yet — nothing structural to heal
347
+ }
348
+ if (raw !== null && raw.trim() !== "") {
349
+ let parsed: Partial<GatesFile> | null = null
350
+ try {
351
+ parsed = JSON.parse(raw) as Partial<GatesFile>
352
+ } catch {
353
+ parsed = null
354
+ }
355
+ if (parsed === null || typeof parsed !== "object" || !Array.isArray(parsed.gates)) {
356
+ // SQLite-style quarantine: move aside, keep the bytes, start clean.
357
+ const quarantine = `${this.gatesPath}.corrupt-${Date.now()}`
358
+ try {
359
+ await rename(ntPath(this.gatesPath), ntPath(quarantine))
360
+ this.gates = []
361
+ this.mtimeMs = 0
362
+ await this.save()
363
+ await this.log({ type: "quarantined", key: "gates.json", snippet: `unparseable gates file moved to ${quarantine}` })
364
+ } catch {
365
+ // rename failed — next reconcile retries; never destroy the file
366
+ }
367
+ } else {
368
+ let dropped = 0
369
+ let repaired = 0
370
+ const byKey = new Map<string, Gate>()
371
+ for (const record of parsed.gates) {
372
+ const gate = coerceGateShape(record)
373
+ if (gate === null) {
374
+ dropped += 1
375
+ continue
376
+ }
377
+ if (repairGate(gate)) repaired += 1
378
+ const existing = byKey.get(gate.key)
379
+ if (existing) {
380
+ mergeGate(existing, gate)
381
+ } else {
382
+ byKey.set(gate.key, gate)
383
+ }
384
+ }
385
+ const merged = parsed.gates.length - dropped - byKey.size
386
+ this.gates = [...byKey.values()]
387
+ this.mtimeMs = 0
388
+ await this.save()
389
+ if (dropped > 0 || repaired > 0 || merged > 0) {
390
+ await this.log({
391
+ type: "repaired",
392
+ key: "gates.json",
393
+ snippet: `dropped ${dropped} hopeless record(s), repaired ${repaired}, merged ${merged} duplicate key(s)`,
394
+ })
395
+ }
396
+ }
397
+ }
398
+ await this.exciseCorruptLogLines()
399
+ })
400
+ }
401
+
402
+ /** Move unparseable JSONL lines to log.jsonl.corrupt; good lines stay. */
403
+ private async exciseCorruptLogLines(): Promise<void> {
404
+ let raw: string
229
405
  try {
230
- const info = await stat(ntPath(this.logPath))
231
- if (info.size < LOG_ROTATE_BYTES) return
232
- const raw = await readFile(ntPath(this.logPath), "utf8")
233
- const lines = raw.split("\n").filter((l) => l.trim() !== "")
234
- const kept = lines.slice(-LOG_ROTATE_KEEP_LINES)
235
- await writeFile(ntPath(this.logPath), `${kept.join("\n")}\n`, "utf8")
406
+ raw = await readFile(ntPath(this.logPath), "utf8")
236
407
  } catch {
237
- // missing or unreadable log is fine
408
+ return // no log yet
238
409
  }
410
+ const good: string[] = []
411
+ const bad: string[] = []
412
+ for (const line of raw.split("\n")) {
413
+ if (line.trim() === "") continue
414
+ try {
415
+ JSON.parse(line)
416
+ good.push(line)
417
+ } catch {
418
+ bad.push(line)
419
+ }
420
+ }
421
+ if (bad.length === 0) return
422
+ await withLock(this.logPath, async () => {
423
+ await appendFile(ntPath(`${this.logPath}.corrupt`), `${bad.join("\n")}\n`, "utf8")
424
+ await atomicWrite(this.logPath, good.length > 0 ? `${good.join("\n")}\n` : "")
425
+ })
426
+ await this.log({ type: "repaired", key: "log.jsonl", snippet: `excised ${bad.length} corrupt line(s) to log.jsonl.corrupt` })
239
427
  }
240
428
  }
241
429
 
430
+ /** Merge a gate's accumulated evidence into an existing gate with the same key. */
431
+ function mergeGate(target: Gate, source: Gate): void {
432
+ // blocking is the stronger state — a merge must never demote an enforced gate
433
+ if (source.status === "blocking") target.status = "blocking"
434
+ target.count += source.count
435
+ for (const session of source.sessions) {
436
+ if (!target.sessions.includes(session)) target.sessions.push(session)
437
+ }
438
+ if (target.sessions.length > MAX_SESSIONS) target.sessions = target.sessions.slice(-MAX_SESSIONS)
439
+ for (const project of source.projects) {
440
+ if (!target.projects.includes(project)) target.projects.push(project)
441
+ }
442
+ if (target.projects.length > MAX_PROJECTS) target.projects = target.projects.slice(-MAX_PROJECTS)
443
+ if (source.firstSeen < target.firstSeen) target.firstSeen = source.firstSeen
444
+ if (source.lastSeen > target.lastSeen) {
445
+ target.lastSeen = source.lastSeen
446
+ target.snippet = source.snippet
447
+ }
448
+ target.remindedCount += source.remindedCount
449
+ target.blockedCount += source.blockedCount
450
+ target.recurredAfterReminder += source.recurredAfterReminder
451
+ target.recurredAfterGate += source.recurredAfterGate
452
+ if (target.correction === undefined && source.correction !== undefined) target.correction = source.correction
453
+ if (source.review === true) target.review = true
454
+ }
455
+
242
456
  /**
243
457
  * Two-scope gate management: project-local gates live in the repo
244
458
  * (`.opencode/dejavu/`), cross-project agent habits are promoted to the
@@ -309,6 +523,20 @@ export class Stores {
309
523
  }
310
524
  })
311
525
  }
526
+ // The cross-project index rots on the same schedule as the gates.
527
+ await this.globalStore.runLockedIndex(async () => {
528
+ const index = await this.globalStore.loadIndex(true)
529
+ const cutoff = Date.now() - ttlDays * DAY_MS
530
+ let changed = false
531
+ for (const key of Object.keys(index.keys)) {
532
+ const entry = index.keys[key]
533
+ if (entry && Date.parse(entry.lastSeen) < cutoff) {
534
+ delete index.keys[key]
535
+ changed = true
536
+ }
537
+ }
538
+ if (changed) await this.globalStore.saveIndex()
539
+ })
312
540
  }
313
541
 
314
542
  async rotateLogs(): Promise<void> {
@@ -321,6 +549,7 @@ export class Stores {
321
549
  * One-time (idempotent) schema/behavior migration:
322
550
  * - probe-tool gates never block (they were learned under the old policy)
323
551
  * - signatures and snippets are secret-scrubbed (cleans historical leaks)
552
+ * - project copies of already-global keys merge into the global gate
324
553
  */
325
554
  async migrate(): Promise<void> {
326
555
  for (const store of this.scopes()) {
@@ -353,6 +582,112 @@ export class Stores {
353
582
  if (changed) await store.save()
354
583
  })
355
584
  }
585
+
586
+ // A key that reached the global store is global everywhere: merge any
587
+ // leftover project-local copy into the global gate so evidence does not
588
+ // fragment across scopes (stale local copies kept enforcing from the old
589
+ // scope while the global gate starved).
590
+ const projectStore = this.projectStore
591
+ if (projectStore) {
592
+ await projectStore.runLocked(async () => {
593
+ const projGates = await projectStore.load(true)
594
+ const globalKeys = new Set((await this.globalStore.load()).map((g) => g.key))
595
+ const dupes = projGates.filter((g) => globalKeys.has(g.key))
596
+ if (dupes.length === 0) return
597
+ await this.globalStore.runLocked(async () => {
598
+ const globalGates = await this.globalStore.load(true)
599
+ for (const dupe of dupes) {
600
+ const target = globalGates.find((g) => g.key === dupe.key)
601
+ if (target) {
602
+ mergeGate(target, dupe)
603
+ } else {
604
+ globalGates.push(dupe)
605
+ }
606
+ }
607
+ await this.globalStore.save()
608
+ })
609
+ projectStore.extract(new Set(dupes.map((g) => g.key)))
610
+ await projectStore.save()
611
+ })
612
+ }
613
+ }
614
+
615
+ /**
616
+ * Structural self-healing across both scopes plus index reconciliation.
617
+ * Idempotent; runs at plugin init and via `doctor --repair`.
618
+ */
619
+ async reconcileAll(globalProjects = 2): Promise<void> {
620
+ for (const store of this.scopes()) {
621
+ await store.reconcile()
622
+ }
623
+
624
+ // Index-driven escalation healing: a key proven in enough project dirs
625
+ // belongs in the global store even if recordFailure never moved it
626
+ // (racing instances, or stores that predate the index).
627
+ const projectStore = this.projectStore
628
+ if (projectStore) {
629
+ const index = await this.globalStore.loadIndex()
630
+ const toEscalate = (await projectStore.load(true)).filter((g) => {
631
+ const entry = index.keys[g.key]
632
+ return entry !== undefined && entry.projects.length >= globalProjects
633
+ })
634
+ if (toEscalate.length > 0) {
635
+ await projectStore.runLocked(async () => {
636
+ await this.globalStore.runLocked(async () => {
637
+ const globalGates = await this.globalStore.load(true)
638
+ for (const gate of toEscalate) {
639
+ const target = globalGates.find((g) => g.key === gate.key)
640
+ if (target) {
641
+ mergeGate(target, gate)
642
+ } else {
643
+ globalGates.push(gate)
644
+ }
645
+ }
646
+ await this.globalStore.save()
647
+ })
648
+ projectStore.extract(new Set(toEscalate.map((g) => g.key)))
649
+ await projectStore.save()
650
+ })
651
+ await this.globalStore.log({
652
+ type: "repaired",
653
+ key: "index.json",
654
+ snippet: `escalated ${toEscalate.length} gate(s) proven in ${globalProjects}+ project dirs`,
655
+ })
656
+ }
657
+ }
658
+
659
+ // The index must mirror reality: a key absent from every scope is an
660
+ // orphan (its gate expired or was deleted); a global gate missing from
661
+ // the index loses cross-project history. Heal both directions.
662
+ const knownKeys = new Set<string>()
663
+ for (const store of this.scopes()) {
664
+ for (const gate of await store.load(true)) knownKeys.add(gate.key)
665
+ }
666
+ await this.globalStore.runLockedIndex(async () => {
667
+ const index = await this.globalStore.loadIndex(true)
668
+ let pruned = 0
669
+ for (const key of Object.keys(index.keys)) {
670
+ if (!knownKeys.has(key)) {
671
+ delete index.keys[key]
672
+ pruned += 1
673
+ }
674
+ }
675
+ let rebuilt = 0
676
+ for (const gate of await this.globalStore.load(true)) {
677
+ if (!index.keys[gate.key]) {
678
+ index.keys[gate.key] = { projects: [...gate.projects], lastSeen: gate.lastSeen }
679
+ rebuilt += 1
680
+ }
681
+ }
682
+ if (pruned > 0 || rebuilt > 0) {
683
+ await this.globalStore.saveIndex()
684
+ await this.globalStore.log({
685
+ type: "repaired",
686
+ key: "index.json",
687
+ snippet: `pruned ${pruned} orphan key(s), rebuilt ${rebuilt} missing entr(y/ies)`,
688
+ })
689
+ }
690
+ })
356
691
  }
357
692
 
358
693
  async recordFailure(input: {
@@ -426,20 +761,45 @@ export class Stores {
426
761
 
427
762
  await store.save()
428
763
 
429
- // Scope escalation: a pattern seen in enough distinct project directories
430
- // is an agent-level habit, not a repo quirk — move it to the global store.
431
- // Lock order is always project -> global, so no deadlock.
432
- let wentGlobal = false
764
+ // Cross-project evidence lives in the global index: gate.projects only
765
+ // ever sees its own store's directory, so alone it can never reach two
766
+ // projects. A pattern seen in enough distinct project dirs is an
767
+ // agent-level habit, not a repo quirk — move it to the global store.
768
+ // Lock order is always gates -> index and project -> global: no cycles.
433
769
  const moved = gate
434
- if (store !== this.globalStore && this.projectStore && gate.projects.length >= input.globalProjects) {
435
- const idx = gates.findIndex((g) => g.key === moved.key)
436
- if (idx >= 0) gates.splice(idx, 1)
437
- await store.save()
770
+ const indexProjects = await this.globalStore.runLockedIndex(async () => {
771
+ const index = await this.globalStore.loadIndex(true)
772
+ let entry = index.keys[input.key]
773
+ if (!entry) {
774
+ entry = { projects: [], lastSeen: now }
775
+ index.keys[input.key] = entry
776
+ }
777
+ if (input.projectDir !== "" && !entry.projects.includes(input.projectDir)) {
778
+ entry.projects.push(input.projectDir)
779
+ if (entry.projects.length > MAX_PROJECTS) entry.projects = entry.projects.slice(-MAX_PROJECTS)
780
+ }
781
+ entry.lastSeen = now
782
+ await this.globalStore.saveIndex()
783
+ return entry.projects.length
784
+ })
785
+
786
+ let wentGlobal = false
787
+ if (store !== this.globalStore && this.projectStore && indexProjects >= input.globalProjects) {
788
+ // Global FIRST, then remove the local copy: a crash between the two
789
+ // writes must leave a duplicate (healed by migrate), never a hole.
438
790
  await this.globalStore.runLocked(async () => {
439
791
  const globalGates = await this.globalStore.load(true)
440
- if (!globalGates.some((g) => g.key === moved.key)) globalGates.push(moved)
792
+ const existing = globalGates.find((g) => g.key === moved.key)
793
+ if (existing) {
794
+ mergeGate(existing, moved)
795
+ } else {
796
+ globalGates.push(moved)
797
+ }
441
798
  await this.globalStore.save()
442
799
  })
800
+ const idx = gates.findIndex((g) => g.key === moved.key)
801
+ if (idx >= 0) gates.splice(idx, 1)
802
+ await store.save()
443
803
  wentGlobal = true
444
804
  }
445
805
 
@@ -0,0 +1,107 @@
1
+ /**
2
+ * Invariant layer: strict parsing and repair of persisted gate state.
3
+ * Pure functions — no I/O. Used at the persistence boundary (store reconcile)
4
+ * and by diagnostics (doctor). Parse, don't validate: what survives
5
+ * coerceGateShape + repairGate satisfies the data-model invariants.
6
+ */
7
+ import type { Gate } from "./store"
8
+ import { canBlock, scrubSecrets } from "./patterns"
9
+
10
+ /** sha1 prefix-12, the only key shape patternKey ever emits */
11
+ const KEY_SHAPE = /^[0-9a-f]{12}$/
12
+ /** detection truncates snippets at 200 chars on ingest */
13
+ const SNIPPET_MAX = 200
14
+
15
+ /**
16
+ * Structural parse of one persisted gate object. Returns a well-shaped Gate
17
+ * or null when the record is hopeless (missing identity fields, unknown
18
+ * status) — hopeless records are dropped, not guessed at.
19
+ */
20
+ export function coerceGateShape(raw: unknown): Gate | null {
21
+ if (typeof raw !== "object" || raw === null) return null
22
+ const r = raw as Record<string, unknown>
23
+ if (typeof r.key !== "string" || !KEY_SHAPE.test(r.key)) return null
24
+ if (typeof r.signature !== "string" || r.signature.trim() === "") return null
25
+ if (typeof r.tool !== "string" || r.tool.trim() === "") return null
26
+ if (r.status !== "watching" && r.status !== "blocking") return null
27
+
28
+ const num = (v: unknown, fallback: number): number =>
29
+ typeof v === "number" && Number.isFinite(v) && v >= 0 ? Math.floor(v) : fallback
30
+ const strings = (v: unknown): string[] =>
31
+ Array.isArray(v) ? v.filter((x): x is string => typeof x === "string") : []
32
+ const str = (v: unknown, fallback: string): string => (typeof v === "string" ? v : fallback)
33
+ const now = new Date().toISOString()
34
+
35
+ const gate: Gate = {
36
+ key: r.key,
37
+ signature: r.signature,
38
+ tool: r.tool,
39
+ status: r.status,
40
+ count: num(r.count, 0),
41
+ sessions: strings(r.sessions),
42
+ projects: strings(r.projects),
43
+ firstSeen: str(r.firstSeen, now),
44
+ lastSeen: str(r.lastSeen, now),
45
+ snippet: str(r.snippet, ""),
46
+ remindedCount: num(r.remindedCount, 0),
47
+ blockedCount: num(r.blockedCount, 0),
48
+ recurredAfterReminder: num(r.recurredAfterReminder, 0),
49
+ recurredAfterGate: num(r.recurredAfterGate, 0),
50
+ }
51
+ if (typeof r.correction === "string") gate.correction = r.correction
52
+ if (r.review === true) gate.review = true
53
+ return gate
54
+ }
55
+
56
+ /**
57
+ * In-place coercion of everything mechanically repairable. Returns true when
58
+ * anything changed. What it cannot repair (identity fields, hopeless shapes)
59
+ * is rejected earlier by coerceGateShape.
60
+ */
61
+ export function repairGate(gate: Gate): boolean {
62
+ let changed = false
63
+ if (gate.firstSeen > gate.lastSeen) {
64
+ const swap = gate.firstSeen
65
+ gate.firstSeen = gate.lastSeen
66
+ gate.lastSeen = swap
67
+ changed = true
68
+ }
69
+ if (gate.snippet.length > SNIPPET_MAX) {
70
+ gate.snippet = gate.snippet.slice(0, SNIPPET_MAX)
71
+ changed = true
72
+ }
73
+ const signature = scrubSecrets(gate.signature)
74
+ if (signature !== gate.signature) {
75
+ gate.signature = signature
76
+ changed = true
77
+ }
78
+ const snippet = scrubSecrets(gate.snippet)
79
+ if (snippet !== gate.snippet) {
80
+ gate.snippet = snippet
81
+ changed = true
82
+ }
83
+ if (gate.correction !== undefined) {
84
+ const correction = scrubSecrets(gate.correction)
85
+ if (correction !== gate.correction) {
86
+ gate.correction = correction
87
+ changed = true
88
+ }
89
+ }
90
+ // Policy is the single source of truth: a blocking gate that cannot block
91
+ // is a leftover from an older policy and must be demoted.
92
+ if (gate.status === "blocking" && !canBlock(gate.tool, gate.signature)) {
93
+ gate.status = "watching"
94
+ changed = true
95
+ }
96
+ return changed
97
+ }
98
+
99
+ /**
100
+ * Corruption fingerprint of a placeholder re-parameterized inside another
101
+ * token (`<code: <n> >` — a fingerprint eaten by the number rule). Only the
102
+ * `<code:` token carries nested content, so the check is scoped to it —
103
+ * shell text like heredoc `<<eof:` must NOT trip the detector.
104
+ */
105
+ export function hasNestedTokens(signature: string): boolean {
106
+ return /<code:\s*</.test(signature)
107
+ }