kodelyth-ecc 2.11.0 → 2.13.0

package/CHANGELOG.md

@@ -2,6 +2,132 @@
 
 All notable changes to Kodelyth ECC are documented here.
 
+## v2.13.0 — Arena run #3: eight bugs in the memory store (August 2026)
+
+Pointed the arena at `scripts/memory` — the persistent BM25 store every other
+subsystem trusts, and the one place where a bug costs the user real accumulated
+work. Round 1 found **8 findings, all 8 confirmed by executed repro**. Round 2
+attacked the fixes across 7 vectors and found **1 regression, which is fixed**.
+
+### Fixed — a single English word bricked memory search
+
+Capturing a memory containing the word **"constructor"** crashed indexing, and
+`recall()` then threw on every subsequent call:
+
+```js
+if (!index.tokens[token]) index.tokens[token] = { docs: [], df: 0 };
+index.tokens[token].docs.push(...)   // .docs is undefined
+```
+
+`index.tokens['constructor']` returns `Object.prototype.constructor` — **truthy**
+— so the guard never fires. Same for `toString`, `valueOf`, `hasOwnProperty`,
+`__proto__`, `isPrototypeOf`. No attacker required: writing one memory about a
+constructor, or about overriding `toString`, was enough. The blast radius was
+everything that reads memory — MCP server, dashboard, CLI, and the session-start
+injection hook.
+
+Fixed at 7 sites with null-prototype maps, including sanitising the index after
+`JSON.parse` (which re-introduces a normal prototype). Those tokens are now
+searchable, not merely non-crashing.
+
+### Fixed — the log is genuinely append-only now
+
+`forget()` and `resolveMemory()` read the **entire** log, mutated it in memory,
+and wrote it back with `fs.writeFileSync` — which opens with `'w'`, truncating to
+zero before writing. Measured directly: sampling file size during a rewrite of a
+6.3 MB store observed it at **0.00 MB**, with 32 torn reads in 1423 samples.
+
+Both now **append a patch row**, and `readMemories()` folds rows by id with
+last-write-wins. That is what the file's own header always claimed it was. The
+truncate window is gone — 0 torn reads across 1477 samples — and 8 concurrent
+deletions now all apply, where previously only 3 of 8 survived.
+
+*Honest scope:* the truncate window is proven, and a reader doing its own
+read-modify-write inside it would persist the emptiness. I could **not**
+reproduce a full store wipe end-to-end; what I measured was lost deletions.
+
+### Fixed — memories that were invisible to search, forever
+
+The log and the index were written by separate calls with no reconciliation, and
+`loadIndex()` only rebuilt when the index was *missing* or schema-invalid — never
+when merely incomplete. A memory could sit in the log and return zero hits for
+its own exact text, permanently. The index now stamps the log size it was built
+from and rebuilds on any drift, which self-heals every cause at the price of one
+`stat()`.
+
+### Also fixed
+
+- `forget()` rewrote the whole store even when the id was **not found**.
+- A crash mid-append left a newline-less row; the next `capture()` fused into it
+  and was silently lost while returning an id and reporting success.
+- `capture()` double-indexed on every cold start, inflating `docCount` and giving
+  that memory exactly 2x its true BM25 score.
+- `tags`/`files`/`gotchas` capped their *count* but not each entry's length — a
+  single 5 MB tag was stored whole.
+- `instincts.js` had the identical rewrite pattern; it now uses
+  `safeFs.replaceFileAtomic`.
+
+### Round 2 — one regression, caught and fixed
+
+The new fold promoted an orphan patch row to a phantom memory with
+`problem: undefined`, which would have flowed into `recall()`, the dashboard, and
+the injected session block. A row with no prior and no `problem` is a patch, not
+a memory.
+
+**Backward compatible:** existing logs read correctly — old full-row tombstones
+fold the same way. One real duplicate id in the test store is now correctly
+deduped rather than returned twice.
+
+**569 tests passing**, up from 554.
+
+## v2.12.0 — `scripts/lib/safe-fs.js`: the guard the arena asked for (August 2026)
+
+Across two arena runs the **same containment bug was confirmed four times** in
+three unrelated files — `terse/compress.js`, `dashboard/data.js`,
+`dashboard/server.js`. Every instance was this shape:
+
+```js
+const abs = path.join(root, userInput);
+if (!abs.startsWith(root + path.sep)) return null;   // lexical only
+fs.readFileSync(abs);                                // follows symlinks
+```
+
+`path.join` and `path.resolve` normalise `..`, so a *textual* escape is caught.
+Neither resolves symlinks. A link sitting lexically inside the root passes the
+check while its target is anywhere on disk.
+
+Four spot fixes would have been a fifth bug waiting. The arena's guard proposal
+said to build the guard instead, so here it is.
+
+### Added — `scripts/lib/safe-fs.js`
+
+- **`resolveContained(candidate, root)`** — canonicalises both sides, so a link
+  is judged by where it *points*, not where it sits. Rejects intermediate
+  directory symlinks and dangling links; still allows a symlinked root to serve
+  its own files, and a link that stays inside.
+- **`statRegularFile(abs)`** — refuses symlinks and non-regular files outright,
+  for callers about to rewrite a file.
+- **`safeConfigDir(value, fallback)`** — inspects the **raw** env value for `..`
+  before resolving. (The first draft resolved first and then looked for `..` —
+  dead code, since `path.resolve` collapses it. Its own test caught that.)
+- **`writeNewFile(dest, data, mode)`** — `O_EXCL` so a dangling symlink cannot
+  redirect the write, plus `fchmod` so a restrictive umask cannot silently
+  narrow the mode.
+- **`replaceFileAtomic(abs, contents, mode)`** — random temp name, rename, and
+  `finally`-unlink so a crash leaves no stray copy of the document.
+
+**13 raw `realpath`/`lstat`/`openSync`/`rename` calls across four files became
+zero.** All four now route through one 157-line module with **19 tests**.
+
+### Fixed — the ledger env var, which was never actually fixed
+
+`KODELYTH_TERSE_DIR` was confirmed unvalidated in the first arena run, and the
+run recorded it as *addressed* when no fix had been written. That accounting was
+wrong. It now goes through `safeConfigDir`: a raw `..` falls back to the default,
+a clean absolute path is still honoured.
+
+**554 tests passing**, up from 535.
+
 ## v2.11.0 — Arena run #2: three containment bugs in the dashboard (August 2026)
 
 Pointed the arena at `scripts/dashboard` — the localhost HTTP server that serves

package/CLAUDE.md

@@ -26,7 +26,7 @@ scripts/     → Node.js utilities: MCP server, dashboard, swarm, replay, router
 bundles/     → 3 power bundles (indie-hacker, red-team, enterprise)
 actions/     → GitHub Action (CI/CD integration for PR review)
 docs/        → Feature docs (arena.md, mcp.md, dashboard.md, swarm.md, replay.md, evolve.md, supply-chain.md)
-tests/       → 535 passing tests across 29 test files
+tests/       → 569 passing tests across 30 test files
 ```
 
 ## Running Tests

package/VERSION

@@ -1 +1 @@
-2.11.0
+2.13.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "kodelyth-ecc",
-  "version": "2.11.0",
+  "version": "2.13.0",
   "description": "Production-grade AI coding toolkit — 70 agents (incl. devil-mode adversarial crew), 194 skills, 97 commands, parallel multi-agent commands, semantic intent routing, self-learning memory, and a built-in MCP server (16 tools / 6 prompts / 377 resources) that bridges to Claude Desktop, LangGraph, AutoGen, CrewAI, and OpenAI Agents SDK. Works with Claude Code, Windsurf, Cursor, Codex, Antigravity, OpenCode, Cline, RooCode, Aider, Kimi, and Gemini CLI.",
   "author": "Kodelyth <github.com/sifxprime>",
   "license": "MIT",

package/scripts/dashboard/data.js

@@ -19,6 +19,7 @@
 const fs   = require('fs');
 const os   = require('os');
 const path = require('path');
+const safeFs = require('../lib/safe-fs.js');
 
 const ROOT = path.resolve(__dirname, '..', '..');
 
@@ -305,30 +306,12 @@ function sessionsList({ coordRoot = defaultCoordRoot(), limit = 30 } = {}) {
 
 function sessionDetail({ session, coordRoot = defaultCoordRoot() } = {}) {
   if (!session || session === '..' || session === '.') return null;
-  const dir = path.join(coordRoot, session);
-  // Containment check — ensure the resolved path stays within coordRoot.
-  // path.join normalises "..", so a textual escape is caught here.
-  if (!dir.startsWith(coordRoot + path.sep) && dir !== coordRoot) return null;
-  if (!fs.existsSync(dir)) return null;
-
-  // ...but a SYMLINK is not a textual escape: the link itself sits inside
-  // coordRoot and passes the check above, while its target does not. Reading
-  // through it hands the API task/handoff/status excerpts from anywhere on
-  // disk. realpath resolves the link so containment is checked against the
-  // real destination, which is the only location that matters.
-  let realDir;
-  try {
-    realDir = fs.realpathSync(dir);
-  } catch {
-    return null;   // dangling or unreadable link
-  }
-  let realRoot;
-  try {
-    realRoot = fs.realpathSync(coordRoot);
-  } catch {
-    realRoot = coordRoot;   // root itself may legitimately not be a link
-  }
-  if (!realDir.startsWith(realRoot + path.sep) && realDir !== realRoot) return null;
+  // Containment, including symlink resolution — see scripts/lib/safe-fs.js.
+  // A link sitting inside coordRoot passes a lexical check while pointing
+  // anywhere on disk, which handed the API task/handoff/status excerpts from
+  // outside the root.
+  const dir = safeFs.resolveContained(session, coordRoot);
+  if (!dir) return null;
   const workers = safeReadDir(dir).filter(e => e.isDirectory());
   return {
     session,

package/scripts/dashboard/server.js

@@ -24,6 +24,7 @@ const http = require('http');
 const fs   = require('fs');
 const path = require('path');
 const os   = require('os');
+const safeFs = require('../lib/safe-fs.js');
 const { execFileSync } = require('child_process');
 
 const data = require('./data.js');
@@ -153,27 +154,13 @@ function resolveStatic(reqPath, baseDir = STATIC_DIR) {
   const decoded = decodeURIComponent(reqPath.replace(/^\/+/, ''));
   if (decoded.includes('..')) return null;
   if (decoded === '' || decoded === '/') return path.join(baseDir, 'index.html');
-  const abs = path.resolve(baseDir, decoded);
-  if (!abs.startsWith(baseDir + path.sep) && abs !== path.join(baseDir, 'index.html')) return null;
-
-  // The check above is purely lexical, and path.resolve does not resolve
-  // symlinks — so a link sitting lexically inside baseDir passes it while
-  // readFile follows it to a target anywhere on disk. Canonicalize and re-check
-  // against the real destination, which is the only one that matters.
-  let real;
-  try {
-    real = fs.realpathSync(abs);
-  } catch {
-    return null;   // missing file or dangling link — 404 either way
-  }
-  let realBase;
-  try {
-    realBase = fs.realpathSync(baseDir);
-  } catch {
-    realBase = baseDir;
-  }
-  if (!real.startsWith(realBase + path.sep) && real !== path.join(realBase, 'index.html')) return null;
-  return abs;
+  // Containment, including symlink resolution — see scripts/lib/safe-fs.js.
+  // path.resolve normalises ".." but does not resolve symlinks, so a link
+  // sitting lexically inside baseDir used to pass while readFile followed it
+  // anywhere on disk.
+  return safeFs.resolveContained(decoded, baseDir, {
+    allowExact: path.join(baseDir, 'index.html'),
+  });
 }
 
 // ── route handlers ───────────────────────────────────────────────────────────

package/scripts/lib/safe-fs.js

@@ -0,0 +1,162 @@
+// scripts/lib/safe-fs.js
+//
+// Path containment and safe file replacement, in one place.
+//
+// This module exists because the same bug was confirmed four times across three
+// unrelated files — `scripts/terse/compress.js`, `scripts/dashboard/data.js`,
+// and `scripts/dashboard/server.js`. Every instance was the same mistake:
+//
+//     const abs = path.join(root, userInput);
+//     if (!abs.startsWith(root + path.sep)) return null;   // lexical only
+//     fs.readFileSync(abs);                                // follows symlinks
+//
+// `path.join` and `path.resolve` normalise `..`, so a *textual* escape is
+// caught. Neither resolves symlinks. A link sitting lexically inside the root
+// passes the check while its target is anywhere on disk, and the read follows
+// it. Four spot fixes would have been a fifth bug waiting; this is the guard.
+//
+// Everything here fails closed: on any doubt, return null or throw.
+
+'use strict';
+
+const fs = require('fs');
+const path = require('path');
+const crypto = require('crypto');
+
+// ── Containment ─────────────────────────────────────────────────────────────
+
+/**
+ * Resolve `candidate` and confirm it really lives inside `root`.
+ *
+ * Both sides are canonicalised, so a symlink is judged by where it POINTS, not
+ * where it sits. Returns the resolved absolute path, or null if it escapes, is
+ * missing, or cannot be read.
+ *
+ * `allowExact` names a path that may equal the root itself (some callers serve
+ * an index file at the boundary).
+ */
+function resolveContained(candidate, root, { allowExact = null } = {}) {
+  if (!candidate || !root) return null;
+
+  const absRoot = path.resolve(root);
+  const abs = path.isAbsolute(candidate) ? path.resolve(candidate) : path.resolve(absRoot, candidate);
+
+  // Cheap lexical rejection first — catches `..` without touching the disk.
+  const lexicallyInside = abs === absRoot || abs.startsWith(absRoot + path.sep);
+  if (!lexicallyInside && abs !== allowExact) return null;
+
+  // The check that actually matters. realpath resolves every symlink in the
+  // path, including intermediate directories.
+  let real;
+  try {
+    real = fs.realpathSync(abs);
+  } catch {
+    return null;   // missing, dangling link, or unreadable
+  }
+
+  let realRoot;
+  try {
+    realRoot = fs.realpathSync(absRoot);
+  } catch {
+    realRoot = absRoot;   // the root itself may legitimately not exist yet
+  }
+
+  const reallyInside = real === realRoot || real.startsWith(realRoot + path.sep);
+  if (!reallyInside && real !== allowExact) return null;
+
+  // Callers get the pre-realpath path so user-facing output keeps the name the
+  // caller asked for; containment has already been proven against the target.
+  return abs;
+}
+
+/**
+ * Stat a path that must be a regular file, refusing symlinks outright.
+ *
+ * Used where the caller is about to REWRITE the file: following a link there
+ * means writing through it to a target the user never named, and copying its
+ * contents into a backup beside the link.
+ */
+function statRegularFile(absPath) {
+  let st;
+  try {
+    st = fs.lstatSync(absPath);
+  } catch {
+    throw new Error(`file not found: ${absPath}`);
+  }
+  if (st.isSymbolicLink()) throw new Error(`refusing to operate on a symlink: ${absPath}`);
+  if (!st.isFile()) throw new Error(`not a regular file: ${absPath}`);
+  return st;
+}
+
+/**
+ * A directory path taken from configuration (an env var, usually).
+ *
+ * This is a trusted-config surface — anyone who can set your environment
+ * already has leverage — but an unnormalised value silently creates trees
+ * wherever `..` points, so normalise and reject the obvious escapes.
+ */
+function safeConfigDir(value, fallback) {
+  if (!value) return fallback;
+  const raw = String(value);
+
+  // Inspect the RAW value, not the resolved one. path.resolve collapses `..`
+  // before any check could see it — resolving first and then looking for `..`
+  // is dead code that always passes.
+  if (raw.split(/[\\/]/).includes('..')) return fallback;
+
+  return path.resolve(raw);
+}
+
+// ── Writing ─────────────────────────────────────────────────────────────────
+
+/**
+ * Create a NEW file, refusing to follow a link or overwrite anything.
+ *
+ * `wx` is O_CREAT|O_EXCL|O_WRONLY: if `dest` exists — including as a *dangling*
+ * symlink, which `fs.existsSync` reports as absent — the open fails instead of
+ * writing through the link to a path someone else chose.
+ *
+ * The mode is applied twice on purpose. `open` filters its mode argument
+ * through the process umask, so a 0644 original would come back 0600 under
+ * `umask 077`; `fchmod` ignores the umask and restores it exactly. Passing it
+ * to `open` as well means the file is never briefly world-readable.
+ */
+function writeNewFile(dest, data, mode) {
+  const fd = fs.openSync(dest, 'wx', mode);
+  try {
+    fs.writeFileSync(fd, data);
+    fs.fchmodSync(fd, mode);
+  } finally {
+    fs.closeSync(fd);
+  }
+}
+
+/**
+ * Replace a file's contents atomically, preserving its permissions.
+ *
+ * Writes to a randomly-named sibling and renames over the target, so a crash
+ * cannot leave the user with a truncated file. The name is random rather than
+ * pid-based because a predictable one lets another process pre-plant a symlink
+ * there and capture the write. A failed rename unlinks the temp rather than
+ * leaving the document's contents in a stray world-readable file.
+ */
+function replaceFileAtomic(absPath, contents, mode) {
+  const tmp = `${absPath}.tmp-${crypto.randomBytes(8).toString('hex')}`;
+  let renamed = false;
+  try {
+    writeNewFile(tmp, contents, mode);
+    fs.renameSync(tmp, absPath);
+    renamed = true;
+  } finally {
+    if (!renamed) { try { fs.unlinkSync(tmp); } catch { /* nothing to clean */ } }
+  }
+  return absPath;
+}
+
+module.exports = {
+  resolveContained,
+  statRegularFile,
+  safeConfigDir,
+  writeNewFile,
+  replaceFileAtomic,
+};

package/scripts/memory/instincts.js

@@ -31,6 +31,7 @@ const fs     = require('fs');
 const os     = require('os');
 const path   = require('path');
 const crypto = require('crypto');
+const safeFs = require('../lib/safe-fs.js');
 
 const MEMORY_DIR = process.env.KODELYTH_MEMORY_DIR
   || path.join(os.homedir(), '.kodelythecc', 'memory');
@@ -95,11 +96,15 @@ function loadAll() {
 
 function saveAll(instincts) {
   ensureDir();
-  fs.writeFileSync(
-    INSTINCTS_FILE,
-    instincts.map(i => JSON.stringify(i)).join('\n') + '\n',
-    'utf8'
-  );
+  // Atomic replace, not writeFileSync. 'w' truncates to zero before writing, so
+  // a crash or a concurrent reader can observe an empty file — the same defect
+  // that was measured wiping the memory log mid-rewrite. rename(2) is atomic:
+  // a reader sees either the old file or the new one, never a torn one.
+  const body = instincts.map(i => JSON.stringify(i)).join('\n') + '\n';
+  const mode = (() => {
+    try { return fs.statSync(INSTINCTS_FILE).mode & 0o7777; } catch { return 0o644; }
+  })();
+  safeFs.replaceFileAtomic(INSTINCTS_FILE, body, mode);
 }
 
 // ── Public API ────────────────────────────────────────────────────────────────

package/scripts/memory/store.js

@@ -72,8 +72,23 @@ function newMemoryId() {
 }
 
 // ── Index ────────────────────────────────────────────────────────────────────
+
+// Any map keyed by user text MUST have a null prototype.
+//
+// With a normal object, `map['constructor']` returns Object.prototype's
+// constructor — which is TRUTHY — so a `if (!map[key])` guard silently does not
+// fire, and the code then reads `.docs` off a function and throws. The word
+// "constructor" is ordinary programming vocabulary, so capturing a memory that
+// merely mentions it used to break indexing AND poison recall() for good. Same
+// for toString, valueOf, hasOwnProperty, __proto__, isPrototypeOf.
+function nullMap(source) {
+  const m = Object.create(null);
+  if (source) for (const k of Object.keys(source)) m[k] = source[k];
+  return m;
+}
+
 function emptyIndex() {
-  return { tokens: {}, docCount: 0, avgDocLength: 0, totalLength: 0 };
+  return { tokens: nullMap(), docCount: 0, avgDocLength: 0, totalLength: 0 };
 }
 
 // A valid index for the current schema MUST have a `tokens` object and a
@@ -97,6 +112,19 @@ function loadIndex() {
   } catch {
     return rebuildIndex();
   }
+  // JSON.parse always produces objects with Object.prototype, so an index read
+  // back from disk carries the hazard again. Rebuild the token map without one.
+  if (isValidIndexSchema(parsed)) parsed.tokens = nullMap(parsed.tokens);
+
+  // Self-heal every way the two files can drift apart — a lost update from
+  // concurrent captures, a failed index write (EACCES/ENOSPC), a hand-edited
+  // log. Without this a memory could sit in the log and be invisible to
+  // recall() forever, which for a recall-driven store is indistinguishable
+  // from having lost it.
+  if (isValidIndexSchema(parsed) && parsed.logSize !== logSize()) {
+    return rebuildIndex();
+  }
+
   if (!isValidIndexSchema(parsed)) {
     // Stale or foreign schema on disk — rebuild from the source of truth
     // (the canonical rebuildIndex defined below rebuilds AND persists a valid index).
@@ -105,9 +133,28 @@ function loadIndex() {
   return parsed;
 }
 
+// Stamp the log size this index was built from. The index is DERIVED state, so
+// the cheapest correct thing is to notice when it no longer matches its source
+// and rebuild — rather than trying to keep two files transactionally in sync.
+function logSize() {
+  try { return fs.statSync(PATHS.log).size; } catch { return 0; }
+}
+
 function saveIndex(index) {
   ensureDir(PATHS.dir);
-  fs.writeFileSync(PATHS.index, JSON.stringify(index, null, 2));
+  fs.writeFileSync(PATHS.index, JSON.stringify({ ...index, logSize: logSize() }, null, 2));
+}
+
+// A patch row does not change any searchable text, so the index stays valid —
+// it just needs to learn the log's new size or the staleness check would force
+// a needless rebuild on the next read.
+function restampIndex() {
+  try {
+    if (!fs.existsSync(PATHS.index)) return;
+    const idx = JSON.parse(fs.readFileSync(PATHS.index, 'utf8'));
+    idx.logSize = logSize();
+    fs.writeFileSync(PATHS.index, JSON.stringify(idx, null, 2));
+  } catch { /* a broken index is rebuilt on the next load anyway */ }
 }
 
 function indexMemory(index, memory) {
@@ -116,7 +163,7 @@ function indexMemory(index, memory) {
   const length = tokens.length;
   if (length === 0) return index;
 
-  const tokenFreq = {};
+  const tokenFreq = nullMap();
   for (const token of tokens) {
     tokenFreq[token] = (tokenFreq[token] || 0) + 1;
   }
@@ -147,7 +194,7 @@ function search(query, options = {}) {
   const b  = 0.75;
   const N  = index.docCount;
   const avgDl = index.avgDocLength || 1;
-  const scores = {};
+  const scores = nullMap();
 
   for (const token of tokens) {
     const entry = index.tokens[token];
@@ -183,27 +230,81 @@ function search(query, options = {}) {
 }
 
 // ── Memory I/O ───────────────────────────────────────────────────────────────
+// The log is append-only, so a memory may appear more than once: the original
+// row, then PATCH rows appended by forget()/resolveMemory(). Fold by id with
+// last-write-wins, then drop anything a patch marked deleted.
+//
+// This is what lets mutation be an append instead of a full-file rewrite. The
+// rewrite was the bug: fs.writeFileSync opens with 'w', truncating to zero
+// before writing, and a 6.3 MB store takes several syscalls to write back. A
+// concurrent reader was measured observing the store at 0 bytes mid-write, and
+// any reader doing its own read-modify-write would then persist that emptiness.
 function readMemories(ids = null) {
   if (!fs.existsSync(PATHS.log)) return ids ? {} : [];
   const wantSet = ids ? new Set(ids) : null;
   const lines = fs.readFileSync(PATHS.log, 'utf8').split('\n').filter(Boolean);
-  const out = ids ? {} : [];
+
+  const folded = new Map();   // preserves first-seen order
   for (const line of lines) {
-    let memory;
-    try { memory = JSON.parse(line); } catch { continue; }
-    if (memory.deleted) continue;
-    if (wantSet) {
-      if (wantSet.has(memory.id)) out[memory.id] = memory;
-    } else {
-      out.push(memory);
-    }
+    let row;
+    try { row = JSON.parse(line); } catch { continue; }
+    if (!row || !row.id) continue;
+    if (wantSet && !wantSet.has(row.id)) continue;
+    const prior = folded.get(row.id);
+    if (prior) { folded.set(row.id, { ...prior, ...row }); continue; }
+    // No prior row means this is an orphan patch — a tombstone or a {resolved}
+    // marker whose original is missing (a hand-edited or truncated log). It is
+    // not a memory: promoting it would surface a phantom with problem=undefined
+    // into recall(), the dashboard, and the injected memory block.
+    if (row.problem === undefined) continue;
+    folded.set(row.id, row);
   }
+
+  const live = [...folded.values()].filter(m => !m.deleted);
+  if (!ids) return live;
+  const out = {};
+  for (const m of live) out[m.id] = m;
   return out;
 }
 
+// Append a patch row for an existing memory. Returns false when the id is
+// unknown, so callers keep their found/not-found contract.
+function appendPatch(memoryId, patch) {
+  if (!fs.existsSync(PATHS.log)) return false;
+  const existing = readMemories([memoryId])[memoryId];
+  if (!existing) return false;
+  ensureDir(PATHS.dir);
+  fs.appendFileSync(PATHS.log, JSON.stringify({ id: memoryId, ...patch }) + '\n');
+  return true;
+}
+
+// If a previous append was interrupted (crash, SIGKILL, ENOSPC) the log ends
+// mid-row with no terminator. Appending straight onto that fragment fuses two
+// records into one unparseable line, so readMemories drops BOTH — and capture()
+// still returns an id, telling the caller a memory was saved that does not
+// exist. A leading newline quarantines the fragment on its own line, where the
+// existing JSON.parse skip handles it correctly.
+function endsWithNewline() {
+  try {
+    const { size } = fs.statSync(PATHS.log);
+    if (size === 0) return true;
+    const fd = fs.openSync(PATHS.log, 'r');
+    try {
+      const buf = Buffer.alloc(1);
+      fs.readSync(fd, buf, 0, 1, size - 1);
+      return buf[0] === 0x0a;
+    } finally {
+      fs.closeSync(fd);
+    }
+  } catch {
+    return true;   // no file yet
+  }
+}
+
 function appendMemory(memory) {
   ensureDir(PATHS.dir);
-  fs.appendFileSync(PATHS.log, JSON.stringify(memory) + '\n');
+  const prefix = endsWithNewline() ? '' : '\n';
+  fs.appendFileSync(PATHS.log, prefix + JSON.stringify(memory) + '\n');
 }
 
 // ── Public API ───────────────────────────────────────────────────────────────
@@ -225,16 +326,24 @@ function capture({
     captured_at: new Date().toISOString(),
     problem:     String(problem).slice(0, 500),
     approach:    String(approach).slice(0, 2000),
-    tags:        Array.from(new Set(tags.map(String))).slice(0, 20),
+    // The COUNT was capped but not the length of each tag, so a single
+    // multi-megabyte tag was stored whole — bloating the log, the index, and
+    // the memory block injected at session start. Bound both.
+    tags:        Array.from(new Set(tags.map(t => String(t).slice(0, 60)))).slice(0, 20),
     project:     project ? projectHash(project) : null,
     project_path: project,
     language,
-    files:       files.slice(0, 20),
-    gotchas:     gotchas.slice(0, 10),
+    files:       files.slice(0, 20).map(f => String(f).slice(0, 500)),
+    gotchas:     gotchas.slice(0, 10).map(g => String(g).slice(0, 500)),
     source,
   };
-  appendMemory(memory);
+  // Load the index BEFORE writing the row. If index.json is missing or invalid,
+  // loadIndex() falls through to rebuildIndex(), which re-reads the log — and if
+  // the new row were already there, it would be indexed once by the rebuild and
+  // again by indexMemory() below. That inflated docCount permanently and gave the
+  // doubled document exactly 2x its true BM25 score, on every cold start.
   const index = loadIndex();
+  appendMemory(memory);
   saveIndex(indexMemory(index, memory));
   return memory;
 }
@@ -262,22 +371,10 @@ function listAll() {
 }
 
 function forget(memoryId) {
-  if (!fs.existsSync(PATHS.log)) return false;
-  const lines = fs.readFileSync(PATHS.log, 'utf8').split('\n').filter(Boolean);
-  let found = false;
-  const updated = lines.map(line => {
-    try {
-      const m = JSON.parse(line);
-      if (m.id === memoryId) {
-        found = true;
-        return JSON.stringify({ ...m, deleted: true, deleted_at: new Date().toISOString() });
-      }
-      return line;
-    } catch {
-      return line;
-    }
-  });
-  fs.writeFileSync(PATHS.log, updated.join('\n') + '\n');
+  const found = appendPatch(memoryId, { deleted: true, deleted_at: new Date().toISOString() });
+  // Only rebuild when something actually changed. The old code rewrote the whole
+  // store even when the id was not found, paying the full destruction window for
+  // a no-op delete.
   if (found) rebuildIndex();
   return found;
 }
@@ -291,28 +388,10 @@ function forget(memoryId) {
 // Side-effect: also propagates to instincts.js if the instinct module exists,
 // so low-confidence instincts sourced from the same session get downgraded.
 function resolveMemory(memoryId, resolved) {
-  if (!fs.existsSync(PATHS.log)) return false;
-  const lines = fs.readFileSync(PATHS.log, 'utf8').split('\n').filter(Boolean);
-  let found = false;
-  const updated = lines.map(line => {
-    try {
-      const m = JSON.parse(line);
-      if (m.id === memoryId && !m.deleted) {
-        found = true;
-        return JSON.stringify({
-          ...m,
-          resolved,
-          resolved_at: new Date().toISOString(),
-        });
-      }
-      return line;
-    } catch {
-      return line;
-    }
-  });
-  if (found) {
-    fs.writeFileSync(PATHS.log, updated.join('\n') + '\n');
-  }
+  const found = appendPatch(memoryId, { resolved, resolved_at: new Date().toISOString() });
+  // This hook fires on every Edit/Write, so a full rebuild here would be costly
+  // on a large store. The patch adds no searchable text, so restamping is enough.
+  if (found) restampIndex();
   return found;
 }
 
@@ -323,13 +402,13 @@ function findMemoriesForFile(filePath, options = {}) {
   if (!fs.existsSync(PATHS.log)) return [];
 
   const normFile = path.normalize(filePath);
-  const lines    = fs.readFileSync(PATHS.log, 'utf8').split('\n').filter(Boolean);
+  // Go through readMemories so patch rows are folded in. Reading the log
+  // line-by-line here meant an appended `{resolved}` patch was never applied to
+  // its original row, so a resolved memory kept being returned.
   const matches  = [];
 
-  for (const line of lines) {
-    let m;
-    try { m = JSON.parse(line); } catch { continue; }
-    if (m.deleted || m.resolved !== undefined) continue; // skip already resolved
+  for (const m of readMemories()) {
+    if (m.resolved !== undefined) continue; // skip already resolved
     if (!Array.isArray(m.files) || m.files.length === 0) continue;
     if (projectRoot && m.project_path && m.project_path !== projectRoot) continue;
 
@@ -399,9 +478,9 @@ function rebuildIndex() {
 
 function stats() {
   const memories = readMemories();
-  const byProject = {};
-  const byLanguage = {};
-  const byTag = {};
+  const byProject = nullMap();
+  const byLanguage = nullMap();
+  const byTag = nullMap();
   for (const m of memories) {
     if (m.project) byProject[m.project] = (byProject[m.project] || 0) + 1;
     if (m.language) byLanguage[m.language] = (byLanguage[m.language] || 0) + 1;

package/scripts/terse/compress.js

@@ -12,6 +12,7 @@
 const fs   = require('fs');
 const path = require('path');
 const crypto = require('crypto');
+const safeFs = require('../lib/safe-fs.js');
 
 // ── Substitutions: wordy connective → short form ────────────────────────────
 // These run BEFORE deletions. Order matters: "due to the fact that" must become
@@ -258,27 +259,6 @@ function compressText(source) {
   };
 }
 
-// Write `data` to `dest` with the given mode, refusing to follow a symlink and
-// refusing to overwrite anything that already exists.
-//
-// `wx` is O_CREAT|O_EXCL|O_WRONLY: if `dest` exists — including as a *dangling*
-// symlink, which `fs.existsSync` reports as absent — the open fails instead of
-// silently writing through the link to a path the attacker chose. The mode is
-// applied at create time so the file is never briefly world-readable.
-// The mode is passed to `open` so the file is never briefly world-readable, and
-// then applied again with fchmod: `open` filters its mode argument through the
-// process umask, so under `umask 077` a 0644 original would come back 0600.
-// fchmod ignores the umask, so the original permissions survive exactly.
-function writeNewFile(dest, data, mode) {
-  const fd = fs.openSync(dest, 'wx', mode);
-  try {
-    fs.writeFileSync(fd, data);
-    fs.fchmodSync(fd, mode);
-  } finally {
-    fs.closeSync(fd);
-  }
-}
-
 // ── Public: compress a file, optionally write ───────────────────────────────
 //
 // Note on paths: `filePath` is deliberately unconfined — compressing
@@ -289,21 +269,9 @@ function writeNewFile(dest, data, mode) {
 function compressFile(filePath, { write = false, backup = true } = {}) {
   const abs = path.resolve(filePath);
 
-  // lstat, not existsSync: we need to know whether this is a symlink *before*
-  // reading through it. Following one would copy the link target's bytes into a
-  // backup beside the link — which is how a 0600 secret ends up in a 0644 file.
-  let st;
-  try {
-    st = fs.lstatSync(abs);
-  } catch {
-    throw new Error(`file not found: ${abs}`);
-  }
-  if (st.isSymbolicLink()) {
-    throw new Error(`refusing to compress a symlink: ${abs}`);
-  }
-  if (!st.isFile()) {
-    throw new Error(`not a regular file: ${abs}`);
-  }
+  // Refuses symlinks: following one would copy the link target's bytes into a
+  // backup beside the link, which is how a 0600 secret ends up in a 0644 file.
+  const st = safeFs.statRegularFile(abs);
 
   // Check the size from the stat we already have, before reading. Otherwise an
   // oversized file is pulled fully into memory only to be rejected a line later.
@@ -336,7 +304,7 @@ function compressFile(filePath, { write = false, backup = true } = {}) {
     backupPath = `${abs}.pre-terse.bak`;
     for (let n = 1; ; n++) {
       try {
-        writeNewFile(backupPath, source, mode);
+        safeFs.writeNewFile(backupPath, source, mode);
         break;
       } catch (err) {
         if (err.code !== 'EEXIST') throw err;
@@ -346,19 +314,8 @@ function compressFile(filePath, { write = false, backup = true } = {}) {
     }
   }
 
-  // Write via temp + rename so a crash cannot truncate the user's file. The
-  // suffix is random rather than the pid: a predictable name lets another
-  // process pre-plant a symlink there and capture the write.
-  const tmp = `${abs}.terse-tmp-${crypto.randomBytes(8).toString('hex')}`;
-  let renamed = false;
-  try {
-    writeNewFile(tmp, output, mode);
-    fs.renameSync(tmp, abs);
-    renamed = true;
-  } finally {
-    // A failed rename leaves the document's content sitting in a stray file.
-    if (!renamed) { try { fs.unlinkSync(tmp); } catch { /* nothing to clean */ } }
-  }
+  // Atomic replace preserving the original mode — see scripts/lib/safe-fs.js.
+  safeFs.replaceFileAtomic(abs, output, mode);
 
   return { path: abs, output, stats, wrote: true, backupPath };
 }

package/scripts/terse/ledger.js

@@ -12,9 +12,15 @@
 const fs   = require('fs');
 const os   = require('os');
 const path = require('path');
+const safeFs = require('../lib/safe-fs.js');
 
-const DIR = process.env.KODELYTH_TERSE_DIR
-  || path.join(os.homedir(), '.kodelythecc', 'terse');
+// The env var is a trusted-config surface — anyone who can set your environment
+// already has leverage — but an unnormalised value silently creates a ledger
+// tree wherever ".." points. safeConfigDir rejects those and falls back.
+const DIR = safeFs.safeConfigDir(
+  process.env.KODELYTH_TERSE_DIR,
+  path.join(os.homedir(), '.kodelythecc', 'terse'),
+);
 const LEDGER = path.join(DIR, 'ledger.jsonl');
 
 function ensureDir() { fs.mkdirSync(DIR, { recursive: true }); }