nlm-memory 0.4.2 → 0.5.1

package/logs/CHANGELOG/CHANGELOG.md

@@ -1,337 +0,0 @@
-# nlm-memory-ts CHANGELOG
-
-Session-level log per session protocol. Cap: 10 entries — archive older to `CHANGELOG-YYYY.md` when exceeded.
-
-## 2026-05-29 — Security hardening: bind address, timing-safe auth, backup/restore gate
-
-**Changes:**
-- `src/cli/nlm.ts`: `serve()` now passes `hostname: "127.0.0.1"` — daemon no longer listens on all interfaces. Verified: `lsof` shows `127.0.0.1:3940` only.
-- `src/http/app.ts`: `/mcp` bearer token comparison replaced with `timingSafeEqual` (was `!==`).
-- `src/http/app.ts`: `/api/data/backup` and `/api/data/restore` now require `Authorization: Bearer <NLM_MCP_TOKEN>` when that env var is set. When unset (local-only use), the `127.0.0.1` bind is the guard.
-- Import: `timingSafeEqual` from `node:crypto` added.
-
-**Findings that prompted this (peer-reviewed audit 2026-05-29):**
-- MEDIUM: `0.0.0.0` binding exposed all HTTP endpoints to LAN — root finding that amplified everything else.
-- MEDIUM: backup/restore unauthenticated — full DB exfiltration from LAN with one HTTP call (peer reviewer caught this; primary auditor missed).
-- LOW: non-timing-safe bearer comparison on `/mcp`.
-
-**State:** 543 tests passing (no regressions). Daemon reloaded; bind address confirmed.
-
-**Next:** Cursor adapter (NocoDB task #182), Windsurf adapter (#183). Deferred: training pipeline (#185, needs >50 useful hits).
-
-## 2026-05-28 — Scheduler failure backoff (a5c29b0)
-
-**Bug fixed: infinite classify retry storm**
-
-Session `517ca931.jsonl` (230KB, Whtnxt Agent project) was failing DeepSeek classification every 30-min tick because no `adapter_state` row was ever written on failure — so `scanOnce` saw a "new" file every tick. The daemon log showed hundreds of `[scheduler] classifier error for cc_517ca931 — skipping` with no error detail.
-
-Migration 013 adds `failure_count INTEGER DEFAULT 0` to `adapter_state`. `recordFailed()` increments `failure_count` and writes the current `file_size`. `scanOnce` skips files whose size hasn't changed, whether clean or failed — so any stuck file stops retrying until new content arrives. When the file grows, `failure_count` resets to 0.
-
-The scheduler now logs the actual error message and a `failure N/M` counter (e.g. `error: LLM unreachable: deepseek for cc_517ca931 (failure 1/3)`). The real error was `LLM unreachable: deepseek` — now diagnosable. 2 new integration tests; 543 total passing.
-
-**State:** daemon reloaded, migration applied, first retry logged correctly. Session `517ca931` will hit ceiling at failure 3/3 and stop.
-
-**Outstanding:** npm publish v0.4.0 (needs `npm login` in terminal). Scheduler (Phase D) is already fully implemented in `nlm start` — confirm via NocoDB task #184 whether anything remains scoped there.
-
-## 2026-05-28 — Aider adapter + useful-scan cron + v0.4.0 release (ece591a, a222f44, 708f49d)
-
-Three items shipped in this session.
-
-**Aider TranscriptAdapter (ece591a)**
-
-`AiderAdapter` reads `.aider.chat.history.md` (or `$AIDER_CHAT_HISTORY_FILE`). A single file may contain multiple sessions, each opened by a `# aider chat started at YYYY-MM-DD HH:MM:SS` header. User turns are H4 headings (`#### ...`); assistant responses are plain text; blockquote lines (`> ...`) become `[tool_action: ...]` summaries. Session IDs derive from the header timestamp as `ai_YYYYMMDD_HHMMSS`. `endedAt` uses the next session's `startedAt` when available (no per-turn timestamps in markdown). Migration `012_sources_aider.sql` adds `'aider'` to the `sources.kind` CHECK constraint. `seedDefaults()` now seeds 6 presets. 21 new unit tests; 541 total pass.
-
-**useful-scan cron wiring (a222f44)**
-
-`nlm-daily-digest.sh` now calls `nlm useful-scan --days 1` before the Python stats fetch so `useful_hit_rate` is populated in the Telegram digest instead of showing "pending". Scan output goes to `logs/daily-digest/useful-scan.log`; `|| true` keeps the digest alive on scan failure. Dry-run confirmed: 161 recalls scanned, 0% useful (accurate — no `cite_session` calls yet).
-
-**v0.4.0 (708f49d)**
-
-Version bump and `git tag v0.4.0`. npm publish requires `npm login` — tag is set, publish when authenticated.
-
-**State:** build clean, 541 tests green. Outstanding: npm publish (needs `npm login`); C2 Aider and useful-scan cron are now complete.
-
-**Next:** README is rewritten. No outstanding P1s remaining. Consider: npm publish, prose-citation soft labels (deferred), or new feature work.
-
-## 2026-05-28 — HermesAgentAdapter: TranscriptAdapter for NousResearch Hermes Agent (7b9074b)
-
-`src/core/adapters/hermes-agent.ts` — reads `~/.hermes/state.db` (WAL mode, schema v11). Extracts user/assistant/tool turns; tool calls in assistant messages summarized as `[tool_use: <name>]`; tool-role messages summarized as `[tool_result: <name>: <preview>]`; system messages skipped. Label from session `title` field, fallback to first user turn. DB path overridable via `NLM_HERMES_AGENT_DB_PATH` or `HERMES_HOME`.
-
-**Migration 011** (`migrations/011_sources_hermes_agent.sql`) adds `'hermes-agent'` to the `sources.kind` CHECK constraint (same rename-recreate-copy pattern as migration 010).
-
-**Source registry:** `SourceKind` extended; `seedDefaults()` now seeds 5 presets (hermes-agent inserted between hermes and opencode, auto-enabled if state.db exists).
-
-**`from-source.ts`:** `'hermes-agent'` case added, delegates to `HermesAgentAdapter`.
-
-**Tests: 520 pass** (was 501, +19 new in `tests/unit/core/adapters/hermes-agent.test.ts`). `source-registry.test.ts` updated for 5 presets. Build clean, typecheck clean.
-
-**State:** NousResearch Hermes Agent is now fully integrated — plugin hooks (pre/post-turn, session lifecycle) from the previous session + transcript indexing from this session. End-to-end: sessions indexed in SQLite → recalled by the daemon → injected into Hermes Agent via the Python plugin.
-
-**Next:** C2 Aider adapter; B3 extract-triples improvements.
-
-## 2026-05-28 — NousResearch Hermes adapter + README rewrite
-
-**README rewrite**
-
-Lead with the three moats (cross-runtime reach, editable timeline, 97.2% R@5). Dropped "self-improving accuracy" framing. Added OpenCode to the shipped runtime list. Added `nlm connect hermes-agent` to the install table.
-
-**NousResearch Hermes Agent plugin (#165)**
-
-Python plugin for NousResearch Hermes Agent's `plugin.yaml` lifecycle hook system. Covers all 6 events the plugin system exposes.
-
-New files:
-- `plugin-hermes-agent/plugin.yaml` — manifest (`kind: memory`, 6 hooks declared)
-- `plugin-hermes-agent/__init__.py` — Python shim; each hook POSTs to the local nlm daemon (stdlib only, no PyPI deps)
-- `plugin-hermes-agent/README.md` — install guide
-
-New HTTP endpoints in the nlm daemon:
-- `POST /api/hook/hermes-agent/pre-turn` — keyword recall for `pre_llm_call`; updates memo; returns `{"context": str|null}`
-- `POST /api/hook/hermes-agent/post-turn` — prose citation detect for `post_llm_call`; logs to citation-log.jsonl
-- `POST /api/hook/hermes-agent/session-lifecycle` — clears surfaced-ID memo on end/finalize/reset
-
-New install module: `src/install/hermes-agent.ts` — `connectHermesAgent` / `disconnectHermesAgent` (copies plugin dir to `~/.hermes/plugins/nlm-memory/`, enables via `hermes plugins enable` if available).
-
-CLI: `nlm connect hermes-agent` / `nlm disconnect hermes-agent` added to `src/cli/nlm.ts`.
-
-**Tests: 501 pass** (was 488, +13 new in `tests/integration/hermes-agent-hooks.test.ts`). Build clean, typecheck clean.
-
-**State:** all three hermes-agent endpoints tested end-to-end without a TTY. Python plugin is a thin HTTP shim — no Python test harness needed.
-
-**Next:** transcript adapter for NousResearch Hermes sessions (session files stored in `~/.hermes-agent/sessions/` or equivalent); C2 Aider adapter; B3 extract-triples improvements.
-
-## 2026-05-28 — C1: OpenCode adapter (SQLite-based, `opencode/1.0`)
-
-OpenCode stores all sessions in a single SQLite DB (`~/Library/Application Support/opencode/opencode.db` on macOS, `$XDG_DATA_HOME/opencode/opencode.db` on Linux) rather than per-session JSONL files. The adapter reads it via `better-sqlite3` in readonly mode, reusing the same `TranscriptAdapter` port as Claude Code, Hermes, and pi.
-
-**What ships**
-
-- `src/core/adapters/opencode.ts` (new) — `OpenCodeAdapter` class. `detect()` checks for the DB file. `discover()` queries `session WHERE time_archived IS NULL` with optional `time_updated >= since` filter. `parseSession(sessionId)` joins the `session`, `message`, and `part` tables: extracts `text` parts (non-ignored) and `tool` parts (summarized as `[tool: <name>]`), skips structural parts (step-start/finish, reasoning, compaction, snapshot, patch, agent, retry). Label comes from `session.title` unless it's `"New session"`, in which case it falls back to the first user turn. `gitBranch` read from `.git/HEAD` in `session.directory`. `sourcePath` is `${dbPath}::${sessionId}`.
-- `migrations/010_sources_opencode.sql` (new) — SQLite table-recreate migration to add `"opencode"` to the `sources.kind` CHECK constraint (SQLite does not support `ALTER COLUMN`). Copies existing rows, drops old table, renames new.
-- `src/core/adapters/from-source.ts` — `"opencode"` case added to `adapterFromSource` switch.
-- `src/core/sources/source-registry.ts` — `SourceKind` union extended; `seedDefaults()` now seeds 4 presets (added OpenCode row, auto-enabled if DB exists).
-- `tests/unit/core/adapters/opencode.test.ts` (new) — 15 tests: detect enabled/disabled, discover (all sessions, archived exclusion, since filter, absent DB), parseSession (null for unknown, null for no usable turns, turn count + roles, ignored-part skipping, tool-part summarization, title label, fallback label, sourcePath format, projectDir, absent DB, ISO timestamps), and metadata assertions.
-- `tests/integration/source-registry.test.ts` — two assertions updated: "seeds three presets" → "seeds four presets"; kind list updated to include `"opencode"`.
-
-**Architecture note**
-
-The `discover()` / `parseSession()` contract treats session IDs (not file paths) as the identifying string — the interface's `path: string` param is opaque, so this is valid. Users with OpenCode already installed get the source auto-enabled on first `nlm migrate` + daemon restart with no manual configuration.
-
-**Tests: 488 pass** (was 470 before this session). All 57 test files green, build clean.
-
-**Next:** README rewrite (D) — drop "self-improving accuracy" promise; lead with the three moats (editable timeline, cross-runtime MCP reach, 97.2% R@5). Then NousResearch Hermes adapter (#165, P1).
-
-## 2026-05-28 — Code review: HOOK_SCRIPT_MARKERS bug caught and patched (44fec62)
-
-`code-review:code-review` skill run against commits `10c16ac..285fe9e`. One confirmed bug found and fixed: `HOOK_SCRIPT_MARKERS` in `claude-settings.ts` did not include the three Phase 2 hook filenames (`session-start-hook.js`, `pre-compact-hook.js`, `subagent-start-hook.js`). Consequence: `nlm hook uninstall` silently left all three hooks behind; each reinstall appended a duplicate instead of replacing. Live settings had two `SessionStart` NLM entries. Fix: added three filenames to `HOOK_SCRIPT_MARKERS`, updated stale file-level comment, rebuilt, reinstalled. Settings deduplicated (1 entry per event × 6 hooks). 436/436 tests pass. No other confirmed bugs from the review — four lower-confidence items scored below 80 and were not acted on.
-
-**State:** `nlm v0.3.0` installed globally. 6 hooks clean in `~/.claude/settings.json`. Shadow mode live.
-
-**Next:** `nlm useful-scan` CLI (B1 full); C1 OpenCode adapter #180 (P1); B3 extract-triples redesign; tests for `session-start-hook.ts`.
-
-## 2026-05-28 — Deploy v0.3.0: 6 hooks live; cite_session double-count fixed; useful_hit_rate stub; session-start source added
-
-Four commits on main (`976e549` → `d013caf`). All 436 tests green throughout.
-
-1. **B2 double-count fix** (`976e549`): `citation-detect.ts` was re-detecting `cite_session` tool_uses in the Stop hook and writing a second citation log entry. MCP handler already calls `appendCitation()` directly. Fix: skip `cite_session` in Stop hook detector; updated 5 tests in `citation-detect-cite-session.test.ts`.
-2. **B1 stub** (`976e549`): added `useful_hit_rate: null` to `StatsResult` + both `recallStats()` return paths. Daily digest shows "pending" cleanly instead of a field-access error. Unblocks schema for future `nlm useful-scan` CLI.
-3. **Phase 2 hook wiring** (`becb591`): `ALL_HOOKS` now includes SessionStart, PreCompact, SubagentStart. Version string corrected 0.2.0-dev → 0.3.0.
-4. **session-start source** (`d013caf`): `src/hook/session-start-hook.ts` written against current interfaces (stale dist imported `loadSurfacedForBudget` that no longer exists). `ClaudeHookEvent` union extended with `SessionStart` + `SubagentStart`.
-
-**State:** `nlm v0.3.0` installed globally, all 6 hooks active in shadow mode. Live measurement window open. Three weekly metrics per D5 start accumulating: cite_session call rate, useful_hit_rate (shows pending until nlm useful-scan lands), null-recall rate.
-
-**Next:** `nlm useful-scan` CLI (B1 full implementation); B3 extract-triples redesign; C1 OpenCode adapter #180.
-
-## 2026-05-28 — D4 thesis pivot: citation moat downgraded permanently; adapter breadth + editable timeline elevated; Phase 0/2/3 engineering landed
-
-Full-day arc on 2026-05-27 producing three clusters of work: a 3-agent audit exposing recall-layer defects, five engineering branches integrated (Phases 0/2/3 of the 90-day plan), and a D4 strategic-pivot decision ending in a permanent thesis revision. The cite_session MCP tool lands on this branch (`phase-1c-cite-tool`) as the last Phase 0 piece.
-
-**Morning 3-agent audit findings (functional + comparative + operability lenses, parallel dispatch)**
-
-All three agents converged on the same structural defect: `src/hook/citation-detect.ts:54-64` matches session IDs only in tool *inputs*, but `recall_sessions` inputs carry `{query, mode, limit}` — never a session ID. Only `get_session({id})` triggers a match. The dominant agent usage pattern (read the surfaced digest → answer from it without a follow-up `get_session`) generates zero citations. Additional defects found: `useful:true` field never written by any code path (metric permanently reads zero); conversation memo cap of 10 silently kills injection past turn 4 because the memo is never pruned mid-conversation; hook locked to `mode=keyword` so Build F's force-include logic never activates at hook time.
-
-**Phases 0/2/3 engineering shipped (5 branches integrated)**
-
-- Phase 0.1 — Stop hook citation detector fixed: scans ALL assistant turns, not just the last (CHANGELOG entry "Stop-hook multi-turn citation detection" above)
-- Phase 0.2 — `useful:true` write path added; `useful_hit_rate` goes from structurally 0% to a real metric
-- Phase 0.3 — Conversation memo pruning: memo capped at 15 and pruned on oldest-first when full; late-turn injection no longer silently dies at turn 4
-- Phase 0.4 — Hook mode aligned to `hybrid` so Build F's force-include logic activates at hook time
-- Phase 2 — SessionStart injection + opinionated 3-hook subset (SessionEnd, PreCompact, SubagentStart)
-- Phase 3 — Training-data collection scaffolding (citation-log schema extended)
-- Phase 1c (this branch) — `cite_session` MCP tool: explicit citation primitive that lets MCP clients signal "I used session X," bypassing the Stop-hook inference chain entirely
-
-**Phase 1 simulation methodology failures (do not repeat)**
-
-Two simulations run during the day produced misleading numbers and were invalidated before they influenced any decision:
-
-1. A 70-prompt simulation of hook injection reported 15.7% `useful_hit_rate` but used a content-full injection format instead of production's pointer-only format. The content-full format inflates compliance because the session body is present in context; in production the body is NOT injected, only the ID + label. Methodology bug caught by a follow-up validator. The simulation's number was correct for the simulation's format; it is not an estimate of production rates.
-2. A 6-session "Arm C" simulation generated by the evaluating agent itself with deliberate compliance, reporting 100% session compliance. This is not a population estimate — it is a demonstration that the measurement pipeline works end-to-end. The agent knows it is being measured and authored the sessions to pass. Not a useful number for forecasting real-system citation rates.
-
-Rule extracted: simulation is valid for pipeline-validity checks (does the detector fire, does the rate field get written, does the log grow?). Simulation cannot estimate rates because rates are population statistics that require real-system samples.
-
-**Pivot to `cite_session` MCP tool primitive**
-
-The per-prompt ID injection approach (injecting surfaced session IDs into every UserPromptSubmit) was prototyped, tested against real Claude Code behavior, and rejected as not shippable. Real Claude 0% prose citation rate on pointer-only injection: the model does not naturally surface NLM session IDs in its prose responses, and the pointer-only format (no body) doesn't give the model enough content to form a meaningful citation. Reverting that branch (`revert(hook): remove cite-by-ID injection`) was correct before `cite_session` landed as the alternative: an explicit MCP tool the agent can call to record a citation when it genuinely used a session.
-
-**D4 thesis pivot (permanent)**
-
-Senior ML engineer review concluded the citation-trained-reranker moat hypothesis fails on fundamentals:
-
-1. Per-operator reranker trained on ~500-1000 citation events/year with prose-channel noise cannot beat the existing hybrid+RRF baseline at 97.2% R@5. The training set is too small and the signal-to-noise ratio too low for a meaningful quality lift.
-2. Aggregated cross-operator training violates local-first. The privacy property is the product; pooling operator data to make reranking better destroys it.
-3. Per-operator moat math: at ~70 daily recalls with ~15% compliance, the per-operator corpus is ~3,800 labeled rows/year. Cross-encoder reranking needs 10K+ to statistically separate from the baseline. The gap does not close in reasonable product timelines.
-
-**Citation feedback loop's new role (quality-monitoring, not reranker training):** detect trash recalls, populate a "frequently useful" dashboard the operator can act on, seed future supervised work if/when aggregated data becomes available under explicit consent. NOT a moat. NOT a reranker pipeline. Permanently downgraded.
-
-**Three elevated moats (D4 thesis):**
-
-1. **Editable timeline / supersedence** — schema-level, retrofit-impossible for append-only-with-decay competitors (agentmemory, mem0, honcho). The supersedence link is the only non-destructive timeline primitive in any of the four systems.
-2. **Cross-runtime reach via MCP** — Claude Code + Codex shipped. OpenCode (#164) and NousResearch Hermes (#165) adapters UNSHELVED — both elevated to P1. Cross-runtime distribution is now load-bearing, not nice-to-have.
-3. **Passive corpus quality** — 97.2% R@5 on LongMemEval-S (better than agentmemory's published 95.2%). Build F's force-include + hybrid+RRF is the foundation. This number is credible, reproducible, and does not require the citation moat to be true.
-
-**90-day direction (revised)**
-
-Adapter breadth is the primary workstream: #164 OpenCode (~2 weeks), #165 NousResearch Hermes (after #164), then pi.dev/Cursor/Aider via `nlm-wrap` MCP wrapper. Editable-timeline UX visibility (making supersedence queryable from the River UI and the MCP `get_session` surface) is the secondary workstream — the moat is structural but invisible until the UI exposes it. Phase 1 passive measurement window continues; 97.2% R@5 baseline is the floor to defend.
-
-**Next priorities**
-
-1. #164 OpenCode adapter (P1, ~2 weeks) — cross-runtime is load-bearing
-2. #165 NousResearch Hermes adapter (P1, after #164)
-3. Editable-timeline UX: supersedence visible in River + `get_session` response
-4. Retain 97.2% R@5 baseline — no retrieval algorithm changes without clearing this bar
-
-## 2026-05-27 — Codex CLI adapter: marketplace plugin + MCP config wiring + interactive-mode hook dispatch
-
-Cross-runtime adapter work, first target landed. NLM is now installable on Codex CLI via `nlm connect codex`, which registers a local plugin marketplace, installs the `nlm-memory` plugin, writes a sentinel-bracketed `[mcp_servers.nlm-memory]` block to `~/.codex/config.toml`, and (optionally with `--with-hooks`) drops a legacy `~/.codex/hooks.json` fallback. Designed to mirror agentmemory's distribution pattern but the integration surface for current Codex (0.134.0) is materially different from both Codex Desktop and the wiki's 2026-05-23 prediction.
-
-**What ships**
-
-- `plugin/.codex-plugin/plugin.json` — Codex plugin manifest declaring `mcpServers: "./.mcp.json"` and `hooks: "./hooks/hooks.json"` pointers
-- `plugin/hooks/hooks.json` — `UserPromptSubmit` + `Stop` event registrations, scripts referenced via `${CLAUDE_PLUGIN_ROOT}`
-- `plugin/.mcp.json` — MCP server registration (spawns `nlm mcp` over stdio); duplicated by the direct config.toml writer for redundancy
-- `plugin/scripts/{prompt-recall-hook,stop-hook}.mjs` — esbuild single-file bundles of the existing TS hook entries, build pinned in `scripts/build-codex-plugin.mjs`
-- `.agents/plugins/marketplace.json` — marketplace manifest declaring the plugin and its source path (`./plugin`)
-- `src/install/codex.ts` — `connectCodex` / `disconnectCodex` / `writeMcpServerToConfig` / `removeMcpServerFromConfig` / `writeLegacyHooks` / `removeLegacyHooks`. Marketplace + plugin add are delegated to the `codex` binary (it owns trust + snapshot state); MCP config and hooks.json are written directly with sentinel markers so disconnect can strip exact regions without touching user-authored content.
-- `src/cli/nlm.ts` — `nlm connect codex` and `nlm disconnect codex` commands. Flags: `--source <owner/repo>` (default `pbmagnet4/nlm-memory-ts`), `--local` shortcut for dev, `--with-hooks` to also write the legacy fallback, `--dry-run`.
-
-**The four wrong-then-right turns worth keeping in memory**
-
-1. *Codex hooks are not Claude-Code-shape settings.json entries.* The 2026-05-23 wiki claim of "identical schema, ~95% script reuse" was wrong on the install mechanism. Codex uses a marketplace + plugin architecture. Hook *contract* (events, stdin payload, stdout convention) is identical to Claude Code; install path is entirely different. Script logic reuses verbatim.
-2. *Marketplace requires a `.agents/plugins/marketplace.json` at the repo root.* First connect attempt failed with `marketplace root does not contain a supported manifest` until that file landed. Reverse-engineered from `~/.codex/.tmp/plugins/.agents/plugins/marketplace.json` shipped by `openai-curated`.
-3. *The marketplace policy field is enum-constrained.* `authentication: "NONE"` rejected as `unknown variant`; only `"ON_INSTALL"` and `"ON_USE"` accepted. NLM has no auth to do, so `"ON_USE"` was picked as a no-op-on-use default. Marketplace went green after the swap.
-4. *`--dangerously-bypass-hook-trust` is misleadingly named.* The flag warns "hooks may run without review for this invocation" but in practice does not bypass trust at all. Hooks dispatched only after persisting trust via an interactive Codex session. Once trust landed in `[hooks.state]`, hooks fired in subsequent `codex exec` (non-interactive) calls too. The bypass flag's real role is unclear.
-
-**Verified end-to-end** (`019e69fa-4ea1-7b10-8c66-70bda64ba086` is the codex session used for final validation)
-
-- ✅ `codex plugin marketplace add ./` (local source) succeeds
-- ✅ `codex plugin add nlm-memory@nlm-memory-ts` produces `installed, enabled` in `codex plugin list`
-- ✅ Cached plugin at `~/.codex/plugins/cache/nlm-memory-ts/nlm-memory/0.3.0/` contains all expected files including dotfile dirs (`.codex-plugin/`, `.mcp.json`)
-- ✅ `[mcp_servers.nlm-memory]` block written to `~/.codex/config.toml` between sentinels; idempotent under repeated connects; cleanly stripped on disconnect
-- ✅ `UserPromptSubmit` hook dispatches from plugin path: codex stdout shows `hook: UserPromptSubmit` / `hook: UserPromptSubmit Completed`, hook-log gains an entry with codex session UUID (`019e...`), recall ran, gate evaluated, would-inject populated, shadow mode logged correctly
-- ✅ Plugin-only default (`nlm connect codex` without `--with-hooks`) fires UserPromptSubmit exactly once per turn. The earlier double-fire with `--with-hooks` enabled (plugin path + legacy `~/.codex/hooks.json` both fired) is exactly why `--with-hooks` stays opt-in
-- ✅ `codex_features list` confirms `hooks: stable, true` (so the runtime supports them) but `plugin_hooks: removed, false` (the older feature flag is dead; current path is the `hooks` engine with plugin-bundled config pointers)
-
-**Not yet verified**
-
-- ⏳ `Stop` hook dispatch — needs a one-time interactive trust approval before it fires (Codex only prompts for trust on hooks that have a chance to run; `codex exec` -p with bypass-trust did not surface a Stop prompt). Will land on Edward's next interactive `codex` turn.
-- ⏳ Remote marketplace install (`codex plugin marketplace add pbmagnet4/nlm-memory-ts`). The local install is the harder code path (the marketplace.json had to be authored from scratch); remote install reuses the same files via git fetch. Verifying in this session's tail after the GitHub push.
-
-**Trust mechanics, for the future**
-
-Codex persists hook trust per `(source, event, ...)` tuple under `[hooks.state]` in `config.toml`. Once a user approves a hook the first time, subsequent invocations (including `codex exec`) fire without prompting. The hash is content-addressed — a release that changes a script binary requires re-trust. This means `nlm connect codex` from a fresh install always requires one interactive `codex` turn to bootstrap trust before hooks fire; we cannot do that step on the user's behalf.
-
-**Build pipeline**
-
-`npm run build` now chains `build:server` (tsc) + `build:ui` (vite) + `build:codex-plugin` (esbuild). The codex-plugin build is single-file per entry (no dependency tree shipped), platform=node, format=esm, target=node20. Each .mjs is under 10KB.
-
-**Tests**
-
-414 unit + integration pass unchanged. No new test files added in this commit — the install path is exercised by the verified end-to-end smoke flow (`nlm connect codex --local` → `codex exec` → hook-log delta inspection). Test surface for install/codex.ts and the build script should land in a follow-up.
-
-**Wiki correction owed**
-
-`Whtnxt Agent Vault/Ventures/nlm-memory/learnings.md` line 218 lists Codex CLI as "`~/.codex/` JSON-config hooks (identical schema to Claude Code) … ~95% script reuse from Claude Code". The script reuse claim is correct (the .ts files port verbatim); the install-mechanism claim is wrong (marketplace + plugin, not settings.json). Wiki update is the next priority after this commit lands.
-
-**Next priorities** (revised from the morning's stack)
-
-1. Wiki update correcting the 2026-05-23 cross-runtime hook landscape table and adding a Codex plugin Tool Lesson. ← **Up next.**
-2. Stop hook validation on Edward's first interactive codex turn (passive — happens whenever).
-3. NousResearch Hermes Agent (#165) — has the cleanest `plugin.yaml` hook surface and was identified in the wiki as the next runtime worth a real adapter. I can validate it end-to-end without a TTY, unlike Codex.
-4. Mode B pre-mortem and alt-embedding A/B remain shelved.
-
-## 2026-05-27 — Stop-hook multi-turn citation detection: useful_hit_rate goes from structurally 0% to a real metric
-
-Bug-fix to the Stop hook's citation detector. The previous implementation scanned only the LAST assistant turn of the transcript, but `tool_use` blocks live in earlier turns — the typical pattern is `tool_use → tool_result → prose summary`, and Stop fires after the summary. The detector saw prose, found no tool_use, logged 0 citations. Production evidence: 348 Stop firings with surfaced IDs, **zero** citations recorded, despite 23 real `mcp__nlm-memory__*` tool_uses in the matching transcripts over the last 7 days.
-
-**Diagnosis path.** Cross-referenced `~/.nlm/hook-log.jsonl` (stop entries, all `citedIds:[]`) against `~/.claude/projects/<workspace>/<conv>.jsonl` (real assistant turns). Drilled into `1fc5a8f1-00fa-4ff5-85e7-a239072082b2`: recall hook surfaced `cc_7ff73609-…`, the assistant called `get_session({id:"cc_7ff73609-…"})` in turn N-1, then wrote a prose summary in turn N; the Stop hook scanned only turn N and logged `citedIds:[]`. Confirmed by code path at `transcript.ts:48` — the loop returns on the first assistant line found walking from the end.
-
-**Changes**
-- `src/core/hook/transcript.ts` — added `readAllAssistantTurns(transcriptPath): ReadonlyArray<AssistantTurn>` that returns every assistant turn in order. Kept `readLastAssistantTurn` as a thin wrapper (single test caller; back-compat for non-Stop callers).
-- `src/core/hook/cite-memo.ts` (new) — per-conversation cited-set memo mirroring `memo.ts`. Same state dir (`~/.nlm/hook-state/`, overridable via `NLM_HOOK_STATE_DIR`), filename suffix `.cited.json` so memo-sweep's existing dir-walk cleans both surfaced and cited memos by mtime. `loadCited` / `recordCited` / `clearCited`.
-- `src/hook/stop-hook.ts` — `runStopHook` now reads all assistant turns, unions text + tool_uses across them, runs `detectCitations` over the union, dedupes against `loadCited(conversationId)`, posts the fresh ones, and persists via `recordCited`. The `responsePreview` stays as the LAST turn's prose (that's the text Edward saw when Stop fired). Daemon remains blind-append; dedup is hook-local.
-- `src/hook/session-end-hook.ts` — `runSessionEnd` now also calls `clearCited` so both memos are cleaned on session close.
-- `scripts/backfill-citations.mjs` (new) — one-shot historical replay. Walks `~/.nlm/hook-log.jsonl` to collect surfaced-ID sets per conversation, finds matching transcripts under `~/.claude/projects/`, runs the same detector, dedupes against existing `~/.nlm/citation-log.jsonl` entries, appends fresh citations with a `backfill:true` marker. Idempotent. Dry-run by default; `--commit` writes.
-
-**Validation**
-- Tests: 414 unit + integration tests pass (was 396, +18 new). New cases cover: tool_use detected when it's in an earlier turn and the last turn is prose-only (the real-world pattern); dedup across repeated Stop firings on a growing transcript; local memo update even when `postCitation` fails (no double-count on next fire); 10 `cite-memo` cases (load/record/clear/corrupt-file/non-array/path-safety); 3 `readAllAssistantTurns` cases; 2 new session-end cases.
-- Typecheck clean on changes (pre-existing `SessionEnd` error in `hook-claude-settings.test.ts` is unrelated and predates this work).
-- Backfill dry-run against the live `~/.nlm/hook-log.jsonl`: 42 conversations had surfaced IDs, 37 had a matching transcript, **4 conversations contain at least one tool_use citation the old detector missed**. Lower than the upper bound suggests by raw tool-use count (23) because many tool_uses were `recall_sessions`/`recall_facts` (no surfaced-ID-in-input — those are pull, not push-follow-up). The 4 captured citations are the ones where the model actually drilled into a surfaced session via `get_session(id=...)`.
-
-**Impact.** `useful_hit_rate` (cited / surfaced) goes from a structural 0% to a real signal. This is the training-data substrate for the future learned reranker (each row in the citation log is a `(query, returned_id, was_cited)` triple once joined against `~/.nlm/query_log.jsonl` by `conversation_id`). The 348 stop firings that previously generated zero training rows would have generated ~10-15 if the detector had been working — small but real, and growing with every conversation going forward.
-
-**Methodology note worth keeping.** The bug was diagnosable in <10 minutes by cross-referencing two existing log streams (hook-log.jsonl × Claude Code transcripts) before touching code. Tomorrow's-self version of this rule: when a telemetry metric reads structurally zero, scan the raw inputs the metric is supposed to consume before assuming the metric is correct. Filing in `Operations/what-works/code-quality.md` candidate set.
-
-**Next priorities (unchanged from earlier today's update):**
-
-1. ~~Stop hook citation rate.~~ Shipped.
-2. Pre-mortem Mode B before any code. Ceiling +1.5% hybrid temporal — current recommendation is to shelve unless a separate driver emerges.
-3. Cross-runtime hook adapters (Hermes / pi / Codex). Unchanged.
-4. Alt-embedding A/B — still deferred.
-
-**Source:** Whtnxt Agent orchestrator session 2026-05-27 (continuation from Build F ship). Diagnosis grounded in `~/.nlm/hook-log.jsonl` (342 stop entries, 0 citations) and `~/.claude/projects/-Users-echalupa-Documents-Coding-Projects-Whtnxt-Agent/*.jsonl` (23 NLM tool_uses across 7 days).
-
-## 2026-05-27 — Build F shipped: force-include keyword rank-1 on temporal+entity shape; hybrid temporal +3.0 / aggregate +0.8 / hybrid beats keyword for the first time
-
-Single session arc, ~6 hours: Build E′ (asymmetric RRF multiplicative boost) shipped → harness-tested → falsified by head-baseline → reverted → diagnosed via per-question `results.json` → Probes 1 & 2 designed and run → Build F (post-merge force-include) shipped → confirmed by clean A/B head-baseline → shipped. Three full harness runs (1 cold ~50 min + 2 hot ~25s) plus two probe scripts. Zero false ships.
-
-**Build E′ (falsified path, recorded for audit trail).** Built `src/core/recall/query-shape.ts` with `detectQueryShape(query)` returning `{hasTemporal, hasNamedEntity}` (temporal regex covers "N days/weeks/months ago", "last <day>", "when did", "before/after I", "yesterday/today/tomorrow"; named-entity accepts ALL-CAPS acronyms and mixed-case tokens, excludes days of week and month names to avoid Mode B false-fires). Modified `mergeHybrid` to accept a `boostKeyword` param and multiply the keyword leg's `1/(RRF_K + rank_kw)` by 1.75 on shape match. Added 27 unit tests for `detectQueryShape`. Harness run `2026-05-26-16-39-52` (n=500, ~48 min, partial cache): hybrid temporal 91.0 → 92.5 / aggregate 95.8 → 96.4. Head-baseline rerun with boost disabled on the same cache (`2026-05-26-16-57-47`, 26.3s): **byte-identical numbers**. The lift was 100% cache enrichment from the 7,500→5,500 chunk-size change populating new embeddings; the boost contributed zero. Post-mortem probe: detector fires on 23/133 temporal queries, but on those 23 the multiplicative boost changed zero top-5 results — the boost magnitude (1.75×) was too small to overcome the "session appears in both lists at lower rank" advantage in RRF. Reverted; recorded in [[track-record]].
-
-**Build F (shipped).** Replaced the failed multiplicative boost with a post-merge **force-include**: when shape is `temporal && namedEntity`, ensure `kwHits[0].session.id` is in the merged top-`limit` set; if not, insert at position `limit - 1`, displacing the lowest-confidence merged hit. Sidesteps RRF arithmetic entirely. ~10 lines in `forceIncludeKeywordTop()` helper at `src/core/recall/recall-service.ts`; detector unchanged from E′.
-
-**Pre-build probes justified the build.** Probe 1 joined each hybrid temporal miss's keyword `returnedIds` against the dataset's `answer_session_ids` to compute keyword's rank for the gold session — on the 7 KW-FOUND misses, 5 had keyword rank=1 and 2 were within rank 5 (force-include trivially recovers all 7 if the detector fires). Probe 2 measured detector fire rate by `question_type`: 17.3% on temporal-reasoning, 0% on the two paraphrase types (single-session-preference, single-session-assistant), 1.4-2.6% on the other non-temporal types — bounded blast radius of ~5 queries across 367 non-temporal questions.
-
-**Clean A/B (same hot cache, identical code except the force-include branch).** Build F (`2026-05-26-22-47-07`, cold rebuild ~85 min) vs head-baseline boost-off (`2026-05-26-22-56-53`, 22.1s on now-hot cache):
-
-| Metric | Off | On | Δ |
-|---|---|---|---|
-| hybrid aggregate | 96.4 | **97.2** | **+0.8** |
-| hybrid temporal | 92.5 | **95.5** | **+3.0** |
-| all other types | byte-identical | byte-identical | 0 |
-| keyword aggregate | 96.6 | 96.6 | 0 |
-| semantic aggregate | 91.6 | 91.6 | 0 |
-
-Zero regression on any question type. Detector unchanged from E′ — the difference is force-include sidestepping the RRF math rather than trying to outmuscle it.
-
-**Hybrid finally beats keyword on aggregate** (97.2 > 96.6) — first time on this benchmark. Resolves the structural tension from 2026-05-25 where keyword led aggregate R@5. The 2026-05-23 MCP default flip to hybrid is now backed by k=5 numbers, not just the k=20 ablation.
-
-**Gate check vs the 2026-05-26 brief:** target was `hybrid temporal R@5 ≥ +4 (target ~95+)`. Landed at +3.0 / 95.5 — one question shy of +4 but inside the 95+ landing target. The miss is "Who did I meet with during the lunch last Tuesday?" — detector skips because day-of-week is excluded from the named-entity set (necessary to avoid Mode B false-fires). Adding day-of-week as NE would catch this one question but cost the Mode B exclusions. Not worth the trade at scale.
-
-**Tests:** 186 unit tests pass (added 27 for `detectQueryShape`); typecheck clean on changes (pre-existing `SessionEnd` error in `hook-claude-settings.test.ts` unrelated). Daemon unchanged (Build F is recall-path code, not ingest/embed).
-
-**Operational gotcha filed.** Mid-session, `~/.cache/longmemeval/{embeddings.sqlite,longmemeval_s_cleaned.json}` vanished between two harness runs — macOS Sonoma+ auto-cleanup of `~/.cache/` during an idle window. Cost ~90 min of cold rebuild + 277 MB redownload. Mitigation: move the cache outside `~/.cache/` via `LONGMEMEVAL_CACHE_DIR=$HOME/.local/share/longmemeval` before the next harness run. Full diagnosis in `Operations/Tool Lessons/longmemeval-harness.md` (vault) — also captures the harness performance envelope and the pre-build probing methodology.
-
-**Methodology lesson worth keeping.** Two-to-five-line probe scripts catch dead hypotheses cheaper than a full harness run. Pattern: (a) probe detector fire rate on the target distribution, (b) probe detector fire rate on the non-target distribution (blast radius), (c) probe the failure mode's mechanism (rank position, candidate-set membership). Run before harness; the result is right whether or not the build ships. Filed in `Ventures/nlm-memory/track-record.md` and `Operations/Tool Lessons/longmemeval-harness.md`. Candidate addition to `Operations/what-works/code-quality.md` if the pattern recurs outside NLM.
-
-**Next priorities (updated):**
-
-1. **Stop hook citation rate.** Now the highest-leverage moat work — hybrid is structurally sound at 97.2 aggregate; further R@5 work hits diminishing returns until a different lever gets pre-mortem'd.
-2. **Pre-mortem Mode B before any code.** Only 2 of 10 hybrid temporal misses are both-leg misses. Ceiling on a successful Mode B fix is +2/133 = +1.5% hybrid temporal. Probe: can a query-time date parser actually resolve those 2 questions' answer windows? If under 50%, the build doesn't justify itself.
-3. **Cross-runtime hook adapters** (Hermes / pi / Codex). Unchanged from prior handoff.
-4. **Alt-embedding A/B** — still deferred. Hybrid 97.2 is a higher floor than the alt-embedding work was originally framed against. Reopen only when migration 010 is justified by a separate driver.
-
-**Source:** Whtnxt Agent orchestrator session 2026-05-26 → 2026-05-27 (continuation); harness reports `reports/longmemeval/2026-05-26-16-39-52/` (E′ on partial cache), `…16-57-47/` (head-baseline boost off, byte-identical to E′), `…22-47-07/` (Build F on cold rebuild), `…22-56-53/` (head-baseline force-include off, same hot cache as 22-47-07). Probe scripts ephemeral at `/tmp/nlm-eprime/`.
-
-_Older entries archived in CHANGELOG-2026.md_
-
-
-
-_Older entries archived in CHANGELOG-2026.md_

package/migrations/000_initial_schema.sql

@@ -1,174 +0,0 @@
--- NLE Memory canonical store — SQLite schema
--- Default zero-config backend. Postgres mirror lives in schema/postgres.sql (TBD).
---
--- Design principles:
---   • Sessions are immutable once written — supersedence is via edges, never via UPDATE
---   • Entity registry is mutable (canonical merges, retitles, retirements)
---   • All timestamps are ISO 8601 strings in TEXT columns (SQLite lacks a native datetime type;
---     ISO strings sort correctly and are dialect-portable to Postgres)
---   • Foreign keys enforced; ON DELETE CASCADE only on edges, never on sessions
-
-PRAGMA foreign_keys = ON;
-PRAGMA journal_mode = WAL;
-
--- ── Sessions ───────────────────────────────────────────────────────────────
-CREATE TABLE IF NOT EXISTS sessions (
-  id                  TEXT PRIMARY KEY,           -- e.g. sess_2026-05-07T14-32-CAMEL
-  runtime             TEXT NOT NULL,              -- e.g. claude-code/1.0, hermes/0.5
-  runtime_session_id  TEXT,                       -- the runtime's own session identifier
-  started_at          TEXT NOT NULL,              -- ISO 8601
-  ended_at            TEXT,                       -- ISO 8601; NULL while session is active
-  duration_min        INTEGER,                    -- computed; null while active
-  label               TEXT NOT NULL,              -- human-readable session title
-  summary             TEXT NOT NULL,              -- ~80-token classifier output
-  body                TEXT,                       -- full markdown body with inline markers
-  status              TEXT NOT NULL CHECK(status IN ('active','closed','superseded')),
-  transcript_kind     TEXT,                       -- e.g. claude-code-jsonl
-  transcript_path     TEXT,                       -- runtime-resolvable opaque pointer
-  transcript_offset   INTEGER,                    -- byte offset start
-  transcript_length   INTEGER,                    -- byte length
-  created_at          TEXT NOT NULL DEFAULT (datetime('now')),
-  updated_at          TEXT NOT NULL DEFAULT (datetime('now'))
-);
-
-CREATE INDEX IF NOT EXISTS idx_sessions_started_at ON sessions(started_at DESC);
-CREATE INDEX IF NOT EXISTS idx_sessions_status ON sessions(status);
-CREATE INDEX IF NOT EXISTS idx_sessions_runtime ON sessions(runtime);
-
--- ── Entities ───────────────────────────────────────────────────────────────
-CREATE TABLE IF NOT EXISTS entities (
-  canonical           TEXT PRIMARY KEY,           -- preferred spelling — also the primary key
-  type                TEXT NOT NULL,              -- 'candidate' until labeled. Built-in labels: project | tool | contact | service | concept. Custom labels are user-defined via the UI / `nle-daemon action label` and have no CHECK constraint.
-  status              TEXT NOT NULL CHECK(status IN ('active','dormant','retired','rejected','candidate')),
-  source              TEXT,                       -- e.g. 'property:.claude/properties/polysignal.yaml', 'auto-detected', 'user-registered'
-  notes               TEXT,                       -- freeform user notes
-  first_seen_session  TEXT REFERENCES sessions(id),
-  last_seen_session   TEXT REFERENCES sessions(id),
-  session_count       INTEGER NOT NULL DEFAULT 0,
-  created_at          TEXT NOT NULL DEFAULT (datetime('now')),
-  updated_at          TEXT NOT NULL DEFAULT (datetime('now'))
-);
-
-CREATE INDEX IF NOT EXISTS idx_entities_type ON entities(type);
-CREATE INDEX IF NOT EXISTS idx_entities_status ON entities(status);
-
--- ── Entity variants (case-insensitive normalization) ───────────────────────
-CREATE TABLE IF NOT EXISTS entity_variants (
-  variant             TEXT PRIMARY KEY,           -- raw form as it appeared
-  canonical           TEXT NOT NULL REFERENCES entities(canonical) ON DELETE CASCADE,
-  source_session_id   TEXT REFERENCES sessions(id)
-);
-
-CREATE INDEX IF NOT EXISTS idx_entity_variants_canonical ON entity_variants(canonical);
-
--- ── Session ↔ Entity (many-to-many) ───────────────────────────────────────
-CREATE TABLE IF NOT EXISTS session_entities (
-  session_id          TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
-  entity_canonical    TEXT NOT NULL REFERENCES entities(canonical),
-  PRIMARY KEY (session_id, entity_canonical)
-);
-
-CREATE INDEX IF NOT EXISTS idx_session_entities_entity ON session_entities(entity_canonical);
-
--- ── Markers (decisions / open questions) ──────────────────────────────────
--- Extracted from inline body markers. Body is canonical — these rows are a queryable cache.
-CREATE TABLE IF NOT EXISTS markers (
-  id                  INTEGER PRIMARY KEY AUTOINCREMENT,
-  session_id          TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
-  kind                TEXT NOT NULL CHECK(kind IN ('decision','open')),
-  text                TEXT NOT NULL,
-  position            INTEGER NOT NULL DEFAULT 0  -- ordering within session
-);
-
-CREATE INDEX IF NOT EXISTS idx_markers_session ON markers(session_id);
-CREATE INDEX IF NOT EXISTS idx_markers_kind ON markers(kind);
-
--- ── Supersedence + continues edges ────────────────────────────────────────
--- Edge table for non-destructive editing. A session can supersede or continue another.
--- Bidirectional lookups handled by indexes on both columns.
-CREATE TABLE IF NOT EXISTS session_edges (
-  from_session        TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
-  to_session          TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
-  kind                TEXT NOT NULL CHECK(kind IN ('supersedes','continues','branched_from','merged_from')),
-  PRIMARY KEY (from_session, to_session, kind)
-);
-
-CREATE INDEX IF NOT EXISTS idx_session_edges_from ON session_edges(from_session);
-CREATE INDEX IF NOT EXISTS idx_session_edges_to ON session_edges(to_session);
-CREATE INDEX IF NOT EXISTS idx_session_edges_kind ON session_edges(kind);
-
--- ── Full-text search over sessions ────────────────────────────────────────
--- FTS5 virtual table for label/summary/body search. Maintained via triggers.
-CREATE VIRTUAL TABLE IF NOT EXISTS sessions_fts USING fts5(
-  label, summary, body,
-  content='sessions',
-  content_rowid='rowid'
-);
-
-CREATE TRIGGER IF NOT EXISTS sessions_ai AFTER INSERT ON sessions BEGIN
-  INSERT INTO sessions_fts(rowid, label, summary, body)
-  VALUES (new.rowid, new.label, new.summary, new.body);
-END;
-
-CREATE TRIGGER IF NOT EXISTS sessions_au AFTER UPDATE ON sessions BEGIN
-  INSERT INTO sessions_fts(sessions_fts, rowid, label, summary, body)
-  VALUES('delete', old.rowid, old.label, old.summary, old.body);
-  INSERT INTO sessions_fts(rowid, label, summary, body)
-  VALUES (new.rowid, new.label, new.summary, new.body);
-END;
-
-CREATE TRIGGER IF NOT EXISTS sessions_ad AFTER DELETE ON sessions BEGIN
-  INSERT INTO sessions_fts(sessions_fts, rowid, label, summary, body)
-  VALUES('delete', old.rowid, old.label, old.summary, old.body);
-END;
-
--- ── Vector embeddings (sqlite-vec) ────────────────────────────────────────
--- Loaded as an extension at runtime: SELECT load_extension('vec0');
--- Schema declared here for reference; real CREATE happens at daemon startup
--- after the extension is loaded.
---
--- CREATE VIRTUAL TABLE session_embeddings USING vec0(
---   session_id TEXT PRIMARY KEY,
---   embedding float[768]      -- nomic-embed-text dim; configurable
--- );
-
--- ── Schema migrations tracker ─────────────────────────────────────────────
--- Applied by SQLiteStore.migrate() on daemon start. Tracks which versioned
--- migration files in daemon/migrations/ have been run on this database.
-CREATE TABLE IF NOT EXISTS schema_migrations (
-  version     INTEGER PRIMARY KEY,
-  name        TEXT NOT NULL,
-  applied_at  TEXT NOT NULL DEFAULT (datetime('now'))
-);
-
--- ── Adapter state (per-runtime offsets for resumability) ──────────────────
-CREATE TABLE IF NOT EXISTS adapter_state (
-  adapter_name        TEXT NOT NULL,              -- e.g. 'claude-code'
-  source_path         TEXT NOT NULL,              -- e.g. ~/.claude/projects/foo/abc123.jsonl
-  last_offset         INTEGER NOT NULL DEFAULT 0,
-  last_processed_at   TEXT NOT NULL DEFAULT (datetime('now')),
-  PRIMARY KEY (adapter_name, source_path)
-);
-
--- ── Actions (event-sourced action log) ────────────────────────────────────
--- Universal primitive for every interactive change: alert dismiss/snooze, entity
--- retire, link supersedes/continues, mark intentional, undo, etc. Append-only.
--- Computed tables (session_edges, entities.status) become projections of this log.
--- Same schema across web UI, MCP tool calls, CLI, future mobile/api.
-CREATE TABLE IF NOT EXISTS actions (
-  id                  TEXT PRIMARY KEY,           -- act_<iso-ts>_<short-uuid>
-  timestamp           TEXT NOT NULL,              -- ISO 8601 — when the action was taken
-  kind                TEXT NOT NULL,              -- dismiss | snooze | retire_entity | label_entity | merge_entity | link_supersedes | link_continues | resolve_open | mark_intentional | undo | sync_localstorage
-  subject_type        TEXT NOT NULL,              -- alert | entity | session | decision | open_question | action
-  subject_id          TEXT NOT NULL,              -- e.g. 'stale_Squarespace', 'sess_002', 'NocoDB'
-  payload             TEXT,                       -- JSON: action-specific data (snoozed_until, target_session_id, new_type, ...)
-  actor               TEXT NOT NULL DEFAULT 'user', -- user | agent:claude-code | agent:hermes | system
-  runtime             TEXT,                       -- web-ui | mcp:claude-code | cli | mobile-ios | api
-  reverted_by         TEXT REFERENCES actions(id), -- the action that undid this one (null = active)
-  created_at          TEXT NOT NULL DEFAULT (datetime('now'))
-);
-
-CREATE INDEX IF NOT EXISTS idx_actions_subject ON actions(subject_type, subject_id);
-CREATE INDEX IF NOT EXISTS idx_actions_kind ON actions(kind);
-CREATE INDEX IF NOT EXISTS idx_actions_timestamp ON actions(timestamp DESC);
-CREATE INDEX IF NOT EXISTS idx_actions_active ON actions(subject_type, subject_id, reverted_by) WHERE reverted_by IS NULL;

package/migrations/001_entity_type_rename.sql

@@ -1,17 +0,0 @@
--- Migration 001: rename entity types and action kind
---
--- Renames entity.type values: property→project, person→contact, external→service
--- (The CHECK constraint on entities.type was already dropped in the initial schema.)
--- Safe to re-run: CASE expression only matches old values; INSERT OR IGNORE is a no-op
--- if this migration has already been applied.
-
-UPDATE entities
-SET type = CASE
-  WHEN type = 'property' THEN 'project'
-  WHEN type = 'person'   THEN 'contact'
-  WHEN type = 'external' THEN 'service'
-  ELSE type
-END
-WHERE type IN ('property', 'person', 'external');
-
-INSERT OR IGNORE INTO schema_migrations (version, name) VALUES (1, 'entity_type_rename');

package/migrations/002_adapter_state_extend.sql

@@ -1,12 +0,0 @@
--- Migration 002: extend adapter_state with columns needed for supersede-on-resume.
---
--- file_size — bytes-on-disk at last classification (detects file growth)
--- session_id — NLE session id produced by the last classification (target for supersede)
---
--- last_offset stays for future chunking use (Phase 3+). For supersede-on-resume in
--- Phase 2, file_size carries the equivalent signal at whole-file granularity.
-
-ALTER TABLE adapter_state ADD COLUMN file_size INTEGER;
-ALTER TABLE adapter_state ADD COLUMN session_id TEXT;
-
-INSERT OR IGNORE INTO schema_migrations (version, name) VALUES (2, '002_adapter_state_extend');

package/migrations/003_session_embeddings.sql

@@ -1,11 +0,0 @@
--- Migration 003: session_embeddings virtual table via sqlite-vec.
---
--- Requires sqlite-vec loaded at connection time (handled by SQLiteStore.connect()).
--- 768 dims matches nomic-embed-text, the default embedding model.
-
-CREATE VIRTUAL TABLE IF NOT EXISTS session_embeddings USING vec0(
-  session_id TEXT PRIMARY KEY,
-  embedding  float[768]
-);
-
-INSERT OR IGNORE INTO schema_migrations (version, name) VALUES (3, '003_session_embeddings');

package/migrations/004_facts.sql

@@ -1,46 +0,0 @@
--- Migration 004: facts + fact_embeddings.
---
--- Facts are the agent-recall projection of session content: normalized
--- (subject, predicate, value) triples derived from session classifier output,
--- supersedence-aware via the tombstone pointer `superseded_by`. See
--- docs/plans/factstore-design.md.
---
--- Phase B.1 creates the tables. Writes start in Phase B.2 (classifier prompt
--- extension). The fact_embeddings vec0 table is created now so Phase B.3 can
--- light up semantic fact recall without a second migration.
-
-CREATE TABLE IF NOT EXISTS facts (
-  id                 TEXT PRIMARY KEY,
-  kind               TEXT NOT NULL CHECK (kind IN ('decision', 'open', 'attribute')),
-  subject            TEXT NOT NULL,
-  predicate          TEXT NOT NULL,
-  value              TEXT NOT NULL,
-  source_session_id  TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
-  source_quote       TEXT,
-  created_at         TEXT NOT NULL DEFAULT (datetime('now')),
-  superseded_by      TEXT REFERENCES facts(id) ON DELETE SET NULL,
-  confidence         REAL NOT NULL CHECK (confidence >= 0.0 AND confidence <= 1.0)
-);
-
--- Hot path: deterministic supersedence collision check on ingest
--- (subject, predicate) lookups against current rows only.
-CREATE INDEX IF NOT EXISTS idx_facts_subject_predicate_current
-  ON facts(subject, predicate)
-  WHERE superseded_by IS NULL;
-
--- "What do we know about X?" — subject-only browsing.
-CREATE INDEX IF NOT EXISTS idx_facts_subject_current
-  ON facts(subject)
-  WHERE superseded_by IS NULL;
-
--- Reverse lookup: which facts came from this session?
-CREATE INDEX IF NOT EXISTS idx_facts_session
-  ON facts(source_session_id);
-
--- Semantic recall index (Phase B.3). 768 dims matches nomic-embed-text.
-CREATE VIRTUAL TABLE IF NOT EXISTS fact_embeddings USING vec0(
-  fact_id    TEXT PRIMARY KEY,
-  embedding  float[768]
-);
-
-INSERT OR IGNORE INTO schema_migrations (version, name) VALUES (4, '004_facts');