memhouse 0.20.2 → 0.20.3

package/CHANGELOG.md

@@ -4,6 +4,37 @@ Versions before 0.8.0 were beta-only. Beta installs of 0.7.x and earlier should
 uninstall and reinstall — the 0.8.0 layout is new, and an in-place `update` +
 migration path is planned work, not a promise the old versions can cash.
 
+## 0.20.3 — 2026-11-03
+
+Release notes: [`docs/releases/v0.20.3.md`](docs/releases/v0.20.3.md). Read its "Read this
+before upgrading".
+
+- **An editor store memhouse cannot read is an error, never an editor with no sessions.** The
+  normalisers (agy, Copilot CLI, Cursor, cursor-agent, Devin, Gemini CLI, Goose, OpenCode, VS Code,
+  Zed) took any error reading a store for "not there", so a permission denial (macOS privacy on
+  `~/Library/Application Support`, for one) showed as no sessions and said nothing. Now only a path
+  that is not there (ENOENT, ENOTDIR) is no store. Any other error reading the store itself (its
+  folder, its database) stops that editor's pass with its errno, or SQLite's code. Each SQLite store
+  file is probed before it is opened, so a refused open is reported with its errno, not as SQLite's
+  "unable to open".
+  - One session's file or folder that cannot be read inside a readable store is skipped and
+    reported as that session's error; the editor's other sessions are still captured, and the
+    session is read once it can be.
+  - `memhouse status` and `memhouse doctor` name each editor whose store the last pass could not
+    read, with the error. On macOS, a denial under `~/Library` comes with the fix: Full Disk Access
+    for the app that runs memhouse.
+  - The normalised session logs of readable stores do not change.
+- **A changed snapshot opens the next generation** (harness-sessions-sql §4.1), and a pass stopped
+  after archiving one no longer archives it a second time when it runs again. No house reads a
+  snapshot source today, so no row changes.
+- **A pass stopped before its cursor write, then run again on other content, replays as a clean
+  run** (harness-sessions-sql SPEC-QUESTIONS q20). A replay from the archive, and an adapter
+  upgrade's re-derive, took the stopped read as a pass of its own, so a record the re-run no longer
+  saw (say, in a log rewritten between the two passes) was written as a removed version none of the
+  committed reads had. The re-run now writes under the stopped pass's time, and a replay folds the
+  two as one pass. Removed versions an earlier replay or re-derive wrote this way, 0.20.2's first
+  pass among them, stay until something re-derives that session.
+
 ## 0.20.2 — 2026-11-03
 
 Release notes: [`docs/releases/v0.20.2.md`](docs/releases/v0.20.2.md). Read its "Read this

package/bin/memhouse.js

@@ -2303,6 +2303,12 @@ async function cmdStatus() {
       : fail ? warn(`shipper: running — ${out.shipper.via} — but ${fail}`)
       : ok(`shipper: running — ${out.shipper.via}`));
     if (stale) console.log('  restart it:  memhouse stop && memhouse start');
+    // An editor whose store the last pass could not read (a permission denial, macOS privacy): said, with
+    // the advice, never left to read as an editor with no sessions.
+    for (const e of (out.shipper.lastPass && out.shipper.lastPass.normaliseErrors) || []) {
+      console.log(warn(`${e.harness}: a read failed in the last pass (${e.code}): ${e.error}`));
+      if (e.hint) console.log(`  ${e.hint}`);
+    }
   }
   if (out.update) console.log(warn(`update available: ${out.update.target} on '${out.update.channel}' — you have ${PKG.version} — run: memhouse update`));
   console.log(out.daemons.dashboard
@@ -2737,6 +2743,9 @@ async function doctorTail(cfg, add) {
   // The last pass's vanish sweeps (passlog.js vanishChecks): a refused one fails, naming the command an
   // emptied store needs; a listed session's source its list left out that still read is a warning.
   for (const c of require(path.join(REPO_ROOT, 'memhouse', 'shipper', 'passlog')).vanishChecks(sh.lastPass)) add(c.ok, c.label, c.hint, { warn: !!c.warn });
+  // The last pass's normaliser errors (passlog.js normaliseChecks): a store, or a session in one, that
+  // could not be read fails, with the advice for a denial (macOS: Full Disk Access), never silently.
+  for (const c of require(path.join(REPO_ROOT, 'memhouse', 'shipper', 'passlog')).normaliseChecks(sh.lastPass)) add(c.ok, c.label, c.hint);
   // The dashboard is optional: the house, the capture and the /mem skills work without it. Not
   // running is said as a warning, never a failure: doctor's exit is the house's health, and a
   // reader chasing "memory looks wrong" must not be sent after a page nobody opened.

package/docs/releases/README.md

@@ -5,6 +5,7 @@ do, how to upgrade, what is deprecated and when it goes, and what was deferred.
 
 | version | date | headline |
 |---|---|---|
+| [0.20.3](v0.20.3.md) | 2026-11-03 | an editor store memhouse cannot read is an error, with the macOS Full Disk Access hint; a stopped pass replays as a clean run |
 | [0.20.2](v0.20.2.md) | 2026-11-03 | harness-sessions-sql v1.1's columns and a turns view; Cursor sub-agent links; the first capture re-derives every session |
 | [0.20.1](v0.20.1.md) | 2026-11-02 | invites expire; `memhouse plugins install codex`; per-command help and clearer CLI answers |
 | [0.20.0](v0.20.0.md) | 2026-10-29 | schema 3: a session is its store, nothing cut or lost, forks billed once, ten editors through normalisers; Apache-2.0 |

package/docs/releases/v0.20.3.md

@@ -0,0 +1,111 @@
+# memhouse 0.20.3 — 2026-11-03
+
+A patch release on 0.20.2, with no schema change and no re-derive. Install it with `memhouse update`,
+or `npm install -g memhouse`.
+
+- **An editor store memhouse cannot read is an error, never an editor with no sessions.** A
+  permission denial (macOS privacy on `~/Library/Application Support`, for one) used to show as
+  an editor with nothing in it. `memhouse status` and `memhouse doctor` now name the editor, the
+  error and, on macOS, the fix: Full Disk Access for the app that runs memhouse.
+- **A pass stopped before it finished, then run again on other content, replays as a clean run.**
+  A replay from the archive, and an adapter upgrade's re-derive, no longer write removed versions
+  for records no committed read had.
+- **A changed snapshot opens the next generation** (harness-sessions-sql §4.1).
+
+## Read this before upgrading
+
+Nothing in the house changes, and there is no migration or re-derive. Two things behave differently:
+
+- **An unreadable store is reported.** Where an editor's store could not be read, the shipper used
+  to read it as empty; now the pass reports the error for that editor and leaves its sessions as
+  they were. If `memhouse status` shows a line like
+  `cursor: a read failed in the last pass (EACCES): …`, read the hint under it. On macOS it is
+  almost always the privacy setting: give the app that runs memhouse (your terminal, or the
+  service's node) Full Disk Access in System Settings > Privacy & Security, then restart it
+  (`memhouse stop && memhouse start`).
+  - One session's file that cannot be read inside a readable store is reported as that session's
+    error; the editor's other sessions are still captured.
+- **Removed versions a replay wrote by mistake stay until that session is re-derived.** 0.20.2's
+  first pass re-derived every session, and a session whose archive held a pass stopped part way,
+  then re-run after its source changed, may hold a removed version of a record no committed read
+  had. It is never counted and shows only in `memhouse q removed`. Nothing removes it now; the next
+  re-derive of that session does.
+
+```bash
+memhouse update        # then: memhouse status — every line a check mark
+```
+
+## Editor stores memhouse cannot read
+
+The editor normalisers (agy, Copilot CLI, Cursor, cursor-agent, Devin, Gemini CLI, Goose, OpenCode,
+VS Code, Zed) took any error reading a store for "not there".
+
+- Now only a path that is not there (ENOENT, ENOTDIR) is no store. Any other error reading the
+  store itself (its folder, its database) stops that editor's pass with its errno, or SQLite's code,
+  and the editor's sessions keep their logs.
+- A refusal inside a readable store (one session's file or folder) skips that session alone and
+  reports it; it is read once it can be.
+- Each SQLite store file is opened for reading before SQLite opens it, so a refused open is reported
+  with its errno (EACCES, EPERM), not as SQLite's "unable to open database file". A SQLite error on
+  a file that opened stays an error, named by its code (`SQLITE_CANTOPEN`, `SQLITE_NOTADB`).
+- `memhouse status` prints each such error with its hint; `memhouse doctor` fails one check per
+  editor and code. On macOS, a denial under `~/Library` comes with the Full Disk Access hint.
+- The normalised session logs of readable stores do not change.
+
+## A stopped pass, then other content
+
+A pass that stops between its archive write and its cursor write (the shipper killed, a ClickHouse
+error part way through a session) is run again by the next pass. When the editor's store changed in
+between, so that something the stopped pass read is gone, the live state was always right, but a
+replay from the archive treated the stopped read as a pass of its own and wrote what it read as a
+removed version.
+
+- The re-run now writes under the stopped pass's time, and a replay folds the two as one pass, as a
+  clean run of what was committed would (harness-sessions-sql SPEC-QUESTIONS q20).
+- A pass stopped after it wrote its rows has already shown what it read; live keeps any of it the
+  re-run no longer sees as a removed version, which a replay does not make. An adapter upgrade's
+  re-derive removes it.
+
+## Conformance
+
+These notes claim no conformance. The harness-sessions-sql conformance kit v1.1.1 (harness-sessions-sql
+4c7c330), with the session adapters 1.1.0 (harness-skeleton e1e5900), ran against memhouse at 1c6ffbc,
+this release's code (the release commit adds only the version and these notes), on ClickHouse
+26.7.4.58 and 25.11.9.34, with the same result on both. The kit passed every item that applies to
+memhouse: 25 pass.
+
+- **Crash-safe** passes, the crash followed by other content included (0.20.2's one open fail).
+- **Units resolve** passes: 1,170 triples, the broad scenario's 1,060 and a kept vanish's 55 live and
+  55 replayed.
+- **Units only**, **Native is declared** and **Absences replay** (reset and gone keys; vanished, kept)
+  pass.
+- Two **Absences replay** parts are not applicable: memhouse declares only the `kept` vanish policy, so
+  "vanished, removed" and "policy changed" do not apply.
+- **Database profile parity** was not run: memhouse has a ClickHouse profile only.
+
+## Smaller changes
+
+- **A changed snapshot opens the next generation** (harness-sessions-sql §4.1), and a pass stopped
+  after archiving one no longer archives it twice when it runs again. No house reads a snapshot
+  source today (Claude Code, Codex and memhouse's own session logs are streams), so no row changes.
+- **Generation numbering:** no change. harness-sessions-sql v1.1.1 (sessions v1.4.4) makes only the
+  order of a source's generations significant, so memhouse's archives, whose first generation is 0,
+  conform as they are.
+- **The test suites of the capture engine run in CI** on both ClickHouse lines (repository).
+
+## Deferred
+
+- **Cursor sub-agents as appended records.** An anchor would place each after the parent's last
+  message at or before the sub-agent's own `createdAt`; that waits for a live store to confirm
+  that a composer's and its bubbles' `createdAt` share one clock.
+- **`text_len` on the development layout** counts UTF-16 units, not code points.
+- **The capture's per-session rollup query** costs a fixed 0.2 to 0.4 seconds a session; 0.21, with
+  a design note first.
+- **`memhouse reset` and `relocate` on schema 3**: 0.21, with a design note first.
+
+## Deprecated
+
+Unchanged from 0.20.0: passwords on the command line (`--password`, `--member-password`,
+`--to-password`), and `MEMHOUSE_USER` / `MEMHOUSE_PASSWORD` as the admin for `migrate --v3-*`,
+still work with a warning and are refused from the next minor release. See
+[0.20.0's Deprecated](v0.20.0.md#deprecated).

package/editors/normalise/agy.js

@@ -67,6 +67,7 @@
 //   table or column throws. The runner retries a failed session.
 
 const fs = require('fs');
+const fsx = require('./fsx');
 const path = require('path');
 const crypto = require('crypto');
 const { placeChildren } = require('../sessions-log');
@@ -377,10 +378,12 @@ function spillRef(uri, shown, stepsDir, sid) {
   if (!m || m[1] !== sid) return null;
   const file = path.join(stepsDir, m[2], 'output.txt');
   let st;
-  try { st = fs.statSync(file); } catch { return null; }
+  st = fsx.stat(file);
+  if (!st) return null;
   if (!st.isFile() || st.size <= Buffer.byteLength(String(shown || ''), 'utf-8')) return null;
   let buf;
-  try { buf = fs.readFileSync(file); } catch { return null; }
+  buf = fsx.readBytes(file);
+  if (!buf) return null;
   return buf.length > Buffer.byteLength(String(shown || ''), 'utf-8') ? { sha: sha256(buf), len: buf.length } : null;
 }
 
@@ -803,8 +806,8 @@ function fromTranscript(sid, text, summary, ann) {
 // The one error that may read as empty: a table or column an older schema lacks. Everything else
 // (busy, locked, SQLITE_PROTOCOL, malformed, IOERR, NOTADB) fails list() or normalise().
 const isMissing = (e) => /no such (table|column)/i.test(String((e && e.message) || e));
-function readdir(dir) { try { return fs.readdirSync(dir); } catch { return []; } }
-function isFile(p) { try { return fs.statSync(p).isFile(); } catch { return false; } }
+function readdir(dir) { return fsx.readdir(dir) || []; }
+function isFile(p) { return fsx.isFile(p); }
 
 function readSummaries(file) {
   const out = new Map();
@@ -831,30 +834,33 @@ function readSummaries(file) {
  * conversation_summaries.db is not named: what is read from it is in the entry.
  */
 function list(storeRoot = storeRootDefault()) {
-  if (!fs.existsSync(storeRoot)) return [];
+  if (!fsx.stat(storeRoot)) return [];
   const harnessPath = require('../harness-paths').resolveNative(storeRoot);   // = dataFolder('agy') for the default store
   const byId = new Map();
   const add = (id) => {
     if (!byId.has(id)) byId.set(id, { harness: HARNESS, session_id: id, harness_path: harnessPath, root: storeRoot, db_path: null, wal_path: null, transcript_path: null, annotation_path: null, summary: null });
     return byId.get(id);
   };
+  const errors = [];   // a session whose file could not be read: skipped, reported (fsx.entry)
   const conv = path.join(storeRoot, 'conversations');
   for (const f of readdir(conv)) {
     const m = /^(.+)\.db$/.exec(f);
-    if (m && isFile(path.join(conv, f))) { const e = add(m[1]); e.db_path = path.join(conv, f); e.wal_path = `${e.db_path}-wal`; }
+    if (m && fsx.entry(errors, m[1], () => isFile(path.join(conv, f)), false)) { const e = add(m[1]); e.db_path = path.join(conv, f); e.wal_path = `${e.db_path}-wal`; }
   }
   const brain = path.join(storeRoot, 'brain');
   for (const id of readdir(brain)) {
     const p = path.join(brain, id, '.system_generated', 'logs', 'transcript_full.jsonl');
-    if (isFile(p)) add(id).transcript_path = p;
+    if (fsx.entry(errors, id, () => isFile(p), false)) add(id).transcript_path = p;
   }
   const sums = readSummaries(path.join(storeRoot, 'conversation_summaries.db'));
-  for (const e of byId.values()) {
+  for (const e of [...byId.values()]) {
     e.summary = sums.get(e.session_id) || null;
     const ann = path.join(storeRoot, 'annotations', `${e.session_id}.pbtxt`);
-    if (isFile(ann)) e.annotation_path = ann;
+    const has = fsx.entry(errors, e.session_id, () => isFile(ann), null);
+    if (has === null) { byId.delete(e.session_id); continue; }   // its annotation refused: the session waits, reported
+    if (has) e.annotation_path = ann;
   }
-  return [...byId.values()].sort((a, b) => (a.session_id < b.session_id ? -1 : a.session_id > b.session_id ? 1 : 0));
+  return fsx.listed([...byId.values()].sort((a, b) => (a.session_id < b.session_id ? -1 : a.session_id > b.session_id ? 1 : 0)), errors);
 }
 
 function dbSource(db, transcript) {

package/editors/normalise/copilot-cli.js

@@ -45,6 +45,7 @@
 //   not conversation: they are left out of the log.
 
 const fs = require('fs');
+const fsx = require('./fsx');
 const path = require('path');
 const crypto = require('crypto');
 const harnessPaths = require('../harness-paths');
@@ -63,16 +64,17 @@ function list(storeRoot = storeRootDefault()) {
   const base = path.join(storeRoot, 'session-state');
   // The data folder's real path: absolute, symlinks resolved, in the filesystem's own case.
   const harnessPath = harnessPaths.resolveNative(storeRoot);
-  let names = [];
-  try { names = fs.readdirSync(base); } catch { return []; }
+  const names = fsx.readdir(base);
+  if (!names) return [];
   const out = [];
+  const errors = [];   // a session whose file could not be read: skipped, reported (fsx.entry)
   for (const sid of names) {
     const dir = path.join(base, sid);
     const events = path.join(dir, 'events.jsonl');
-    try { if (!fs.statSync(events).isFile()) continue; } catch { continue; }
+    if (!fsx.entry(errors, sid, () => fsx.isFile(events), false)) continue;
     out.push({ harness: HARNESS, harness_path: harnessPath, session_id: sid, dir, events, workspace: path.join(dir, 'workspace.yaml') });
   }
-  return out;
+  return fsx.listed(out, errors);
 }
 
 /** Complete lines only: a partial last line is held back. */
@@ -139,8 +141,7 @@ function normalise(session) {
     : parseEvents(fs.readFileSync(session.events, 'utf-8'));
   let ws = session.workspace;
   if (!ws || typeof ws === 'string') {
-    let txt = '';
-    try { txt = fs.readFileSync(ws || '', 'utf-8'); } catch { txt = ''; }
+    const txt = (ws && fsx.readText(ws)) || '';
     ws = parseWorkspace(txt);
   }
   const start = events.find((e) => e.type === 'session.start');

package/editors/normalise/cursor-agent.js

@@ -78,6 +78,7 @@
 //   A referenced blob that is missing means a torn read: normalise returns null and the pass skips.
 
 const fs = require('fs');
+const fsx = require('./fsx');
 const path = require('path');
 const crypto = require('crypto');
 const { fileURLToPath } = require('url');
@@ -99,7 +100,7 @@ function storeRootDefault() { return harnessPaths.dataFolder(HARNESS); }
 
 const md5 = (s) => crypto.createHash('md5').update(s).digest('hex');
 const sha256 = (b) => crypto.createHash('sha256').update(b).digest('hex');
-function sortedDir(dir) { try { return fs.readdirSync(dir).sort(); } catch { return []; } }
+function sortedDir(dir) { return (fsx.readdir(dir) || []).sort(); }
 
 /**
  * Sessions in one store: every chats/<bucket>/<chatId>/ and acp-sessions/<id>/ that has a store.db.
@@ -109,19 +110,20 @@ function sortedDir(dir) { try { return fs.readdirSync(dir).sort(); } catch { ret
  * not named: it is rewritten (updatedAtMs) on every open, and only its cwd is read, as a fallback.
  */
 function list(storeRoot = storeRootDefault()) {
-  try { if (!fs.statSync(storeRoot).isDirectory()) return []; } catch { return []; }
+  if (!fsx.isDir(storeRoot)) return [];
   const harnessPath = harnessPaths.resolveNative(storeRoot);
   const out = [];
+  const errors = [];   // a chat whose folder or store could not be read: skipped, reported (fsx.entry)
   const add = (dir, sid, bucket) => {
     const db = path.join(dir, 'store.db');
-    try { if (!fs.statSync(db).isFile()) return; } catch { return; }   // empty or refused-headless chat
+    if (!fsx.entry(errors, sid, () => fsx.isFile(db), false)) return;   // empty or refused-headless chat
     out.push({ harness: HARNESS, harness_path: harnessPath, session_id: sid, dir, db, wal: `${db}-wal`, bucket });
   };
   const chats = path.join(storeRoot, 'chats');
-  for (const b of sortedDir(chats)) for (const id of sortedDir(path.join(chats, b))) add(path.join(chats, b, id), id, b);
+  for (const b of sortedDir(chats)) for (const id of fsx.entry(errors, b, () => sortedDir(path.join(chats, b)), [])) add(path.join(chats, b, id), id, b);
   const acp = path.join(storeRoot, 'acp-sessions');
   for (const id of sortedDir(acp)) add(path.join(acp, id), id, null);
-  return out;
+  return fsx.listed(out, errors);
 }
 
 // ── protobuf wire format: just enough to walk the store ───────────────────────────────────────
@@ -468,7 +470,9 @@ function parseMetaValue(v) {
   } catch { return null; }
 }
 function readMetaJsonCwd(dir) {
-  try { const j = JSON.parse(fs.readFileSync(path.join(dir, 'meta.json'), 'utf-8')); return typeof j.cwd === 'string' ? j.cwd : ''; } catch { return ''; }
+  const text = fsx.readText(path.join(dir, 'meta.json'));
+  if (text === null) return '';
+  try { const j = JSON.parse(text); return typeof j.cwd === 'string' ? j.cwd : ''; } catch { return ''; }
 }
 function uriPath(u) {
   if (!u) return '';
@@ -490,7 +494,9 @@ function normalise(session) {
     return build(session, pickMeta(session.meta), (id) => m.get(id), scan);
   }
   let db;
-  try { db = openReadOnly(session.db); } catch { return null; }
+  // Gone since the list: no read this pass. Any other open error (a permission denial, a file SQLite
+  // cannot open) is reported, never taken for an empty chat.
+  try { db = openReadOnly(session.db); } catch (e) { if (fsx.absent(e)) return null; throw e; }
   try {
     db.exec('BEGIN');
     const row = db.prepare("SELECT value FROM meta WHERE key = '0'").get();

package/editors/normalise/cursor.js

@@ -48,6 +48,7 @@
 //   `encrypted_model`. ItemTable is never queried at all (it holds the login rows).
 
 const fs = require('fs');
+const fsx = require('./fsx');
 const path = require('path');
 const { openReadOnly, textOf } = require('../sqlite');
 const harnessPaths = require('../harness-paths');
@@ -165,7 +166,7 @@ const bubbleRange = (cid) => [`bubbleId:${cid}:`, `bubbleId:${cid};`];
  */
 function list(storeRoot = storeRootDefault()) {
   const file = dbPathOf(storeRoot);
-  if (!fs.existsSync(file)) return [];
+  if (!fsx.stat(file)) return [];
   const harnessPath = fs.realpathSync.native(storeRoot);
   const db = openReadOnly(file);
   try {

package/editors/normalise/devin.js

@@ -80,6 +80,7 @@
 //   index. Only a missing table or column reads as empty.
 
 const fs = require('fs');
+const fsx = require('./fsx');
 const os = require('os');
 const path = require('path');
 const crypto = require('crypto');
@@ -126,7 +127,7 @@ const isMissing = (e) => /no such (table|column)/i.test(String((e && e.message)
 function list(storeRoot = storeRootDefault()) {
   const harnessPath = harnessPaths.resolveNative(storeRoot);
   const file = path.join(harnessPath, DB_FILE);
-  try { if (!fs.statSync(file).isFile()) return []; } catch { return []; }   // no store
+  if (!fsx.isFile(file)) return [];   // no store
   // A store that is busy or locked mid-save must not read as "no sessions": an open or a query that
   // fails throws, and the runner keeps its index. Only a missing table or column reads as empty.
   const db = openReadOnly(file);

package/editors/normalise/fsx.js

@@ -0,0 +1,99 @@
+'use strict';
+// The normalisers' file reads, with one rule: a path that is not there (ENOENT, ENOTDIR) is an
+// answer (no store, no session, no file), and any other error is an error. A read that took every
+// error for "not there" turned a permission denial (macOS privacy on ~/Library, a mode change, an
+// unreadable mount) into an editor with no sessions and no word of why.
+//
+// Each helper returns its absent answer (null, or false) only for absence, and throws anything else
+// as an FsError: the path, the operation, and the errno as `code`. normalise-run.js reports it by
+// editor, and `memhouse status` and `doctor` show it with hintOf's advice.
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+
+const ABSENT = new Set(['ENOENT', 'ENOTDIR']);
+/** True for the errors that mean the path is not there. */
+const absent = (e) => !!e && ABSENT.has(e.code);
+
+class FsError extends Error {
+  constructor(op, p, e) {
+    const code = (e && e.code) || 'EIO';
+    super(`cannot ${op} ${p} (${code})`);
+    this.name = 'FsError';
+    this.code = code;
+    this.path = p;
+  }
+}
+const fail = (op, p, e) => (e instanceof FsError ? e : new FsError(op, p, e));
+
+/** A folder's entries, or null when the folder is not there. */
+function readdir(dir, opts) {
+  try { return fs.readdirSync(dir, opts); } catch (e) { if (absent(e)) return null; throw fail('read the folder', dir, e); }
+}
+/** fs.Stats, or null when the path is not there. */
+function stat(p) {
+  try { return fs.statSync(p); } catch (e) { if (absent(e)) return null; throw fail('stat', p, e); }
+}
+/** fs.Stats of the link itself (a symlink is not followed), or null when the path is not there. */
+function lstat(p) {
+  try { return fs.lstatSync(p); } catch (e) { if (absent(e)) return null; throw fail('stat', p, e); }
+}
+const isFile = (p) => { const s = stat(p); return !!s && s.isFile(); };
+const isDir = (p) => { const s = stat(p); return !!s && s.isDirectory(); };
+/** The file's text (UTF-8), or null when it is not there. */
+function readText(p) {
+  try { return fs.readFileSync(p, 'utf-8'); } catch (e) { if (absent(e)) return null; throw fail('read', p, e); }
+}
+/** The file's bytes, or null when it is not there. */
+function readBytes(p) {
+  try { return fs.readFileSync(p); } catch (e) { if (absent(e)) return null; throw fail('read', p, e); }
+}
+/** The resolved path, or null when it is not there. */
+function realpath(p) {
+  try { return fs.realpathSync.native(p); } catch (e) { if (absent(e)) return null; throw fail('resolve', p, e); }
+}
+/** A file descriptor open for reading, or null when the file is not there. */
+function openRead(p) {
+  try { return fs.openSync(p, 'r'); } catch (e) { if (absent(e)) return null; throw fail('open', p, e); }
+}
+/** True when the file opens for reading, false when it is not there; anything else throws. */
+function probe(p) {
+  const fd = openRead(p);
+  if (fd === null) return false;
+  fs.closeSync(fd);
+  return true;
+}
+
+/**
+ * One entry's read inside a store that read (a session's file or folder in a readable root): a refusal
+ * skips that entry alone, recorded in `errors` as { session_id, code, path, error } (the session's id,
+ * or the entry's name when the id is inside the file), and the store's other entries are listed.
+ * One stuck file must not stop an editor's new sessions from being captured. Returns `skipped` then.
+ * Not there is fn's own absent answer, and any other error propagates.
+ */
+function entry(errors, id, fn, skipped) {
+  try { return fn(); } catch (e) {
+    if (!(e instanceof FsError)) throw e;
+    errors.push({ session_id: String(id), code: e.code, path: e.path, error: e.message });
+    return skipped;
+  }
+}
+/** A list's entries, carrying the entries it skipped as `errors` (normalise-run reports them). */
+const listed = (out, errors) => (errors.length ? Object.assign(out, { errors }) : out);
+
+/**
+ * The advice for an error a read met, or ''. On macOS, a denial under ~/Library is almost always the
+ * privacy setting: the app that runs memhouse (the terminal, or the service's node) needs Full Disk
+ * Access to read another app's folder there.
+ */
+function hintOf(err, { platform = process.platform, home = os.homedir() } = {}) {
+  const code = err && err.code;
+  if (code !== 'EACCES' && code !== 'EPERM') return '';
+  const p = String((err && err.path) || '');
+  if (platform === 'darwin' && p && path.resolve(p).startsWith(path.join(home, 'Library') + path.sep)) {
+    return 'macOS privacy: give the app that runs memhouse Full Disk Access (System Settings > Privacy & Security > Full Disk Access), then restart it';
+  }
+  return 'memhouse cannot read it: check the permissions of the path and its folders';
+}
+
+module.exports = { absent, FsError, readdir, stat, lstat, isFile, isDir, readText, readBytes, realpath, openRead, probe, entry, listed, hintOf };

package/editors/normalise/gemini-cli.js

@@ -69,6 +69,7 @@
 //   `authType`, sits in files this module never opens.
 
 const fs = require('fs');
+const fsx = require('./fsx');
 const path = require('path');
 const crypto = require('crypto');
 const { dataFolder, defaultFolder, resolveNative } = require('../harness-paths');
@@ -89,8 +90,8 @@ function storeRootDefault() {
   return defaultFolder(HARNESS);
 }
 
-const readdir = (d) => { try { return fs.readdirSync(d, { withFileTypes: true }).sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0)); } catch { return []; } };
-const readText = (f) => { try { return fs.readFileSync(f, 'utf-8'); } catch { return null; } };
+const readdir = (d) => (fsx.readdir(d, { withFileTypes: true }) || []).sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0));
+const readText = (f) => fsx.readText(f);
 const isObj = (v) => v !== null && typeof v === 'object' && !Array.isArray(v);
 const str = (v) => (typeof v === 'string' ? v : '');
 const ms = (iso) => { const t = Date.parse(iso); return Number.isFinite(t) ? t : null; };
@@ -159,8 +160,8 @@ function topLevelStrings(text, keys) {
 
 /** A file's first bytes: HEAD_CHUNK, or (untilNewline) its first line, null while it has none. */
 function readPrefix(file, untilNewline) {
-  let fd;
-  try { fd = fs.openSync(file, 'r'); } catch { return null; }
+  const fd = fsx.openRead(file);
+  if (fd === null) return null;
   try {
     let buf = Buffer.alloc(0);
     let size = HEAD_CHUNK;
@@ -175,7 +176,7 @@ function readPrefix(file, untilNewline) {
       if (n < size || buf.length >= HEAD_MAX) return null;   // no complete first line (still being written), or no header
       size = Math.min(size * 2, HEAD_MAX - buf.length);
     }
-  } catch { return null; } finally { try { fs.closeSync(fd); } catch { /* closed */ } }
+  } catch (e) { throw new fsx.FsError('read', file, e); } finally { try { fs.closeSync(fd); } catch { /* closed */ } }
 }
 
 /**
@@ -203,7 +204,7 @@ function readHeader(file, format) {
  */
 function list(storeRoot) {
   const root = path.resolve(storeRoot == null ? defaultFolder(HARNESS) : storeRoot);
-  if (!fs.existsSync(root)) return [];
+  if (!fsx.stat(root)) return [];
   // The default store's harness_path is harness-paths' data folder (GEMINI_CLI_HOME included), so the
   // bridge→normaliser switch dedupes; an explicit root is its own resolved path.
   const harnessPath = storeRoot == null ? dataFolder(HARNESS) : resolveNative(root);
@@ -211,8 +212,9 @@ function list(storeRoot) {
   const projectOf = (slug) => {
     if (!bySlug) {
       bySlug = new Map();
+      const text = fsx.readText(path.join(root, 'projects.json'));
       try {
-        const pj = JSON.parse(fs.readFileSync(path.join(root, 'projects.json'), 'utf-8'));
+        const pj = text === null ? null : JSON.parse(text);
         for (const [dir, sl] of Object.entries((pj && pj.projects) || {}).sort()) if (typeof sl === 'string' && !bySlug.has(sl)) bySlug.set(sl, dir);
       } catch { /* no index */ }
     }
@@ -224,30 +226,38 @@ function list(storeRoot) {
     const head = readHeader(file, format);
     if (!head || !head.sessionId) return;
     let size = 0;
-    try { size = fs.statSync(file).size; } catch { return; }
+    const st = fsx.stat(file);
+    if (!st) return;
+    size = st.size;
     const e = { harness: HARNESS, harness_path: harnessPath, session_id: head.sessionId, file, format, kind: head.kind, parent, slug, cwd, size };
     const prev = found.get(e.session_id);
     const better = !prev || (e.format === 'jsonl' && prev.format !== 'jsonl')
       || (e.format === prev.format && (e.size > prev.size || (e.size === prev.size && e.file < prev.file)));
     if (better) found.set(e.session_id, e);
   };
+  const errors = [];
   for (const s of readdir(path.join(root, 'tmp'))) {
     if (!s.isDirectory()) continue;
     const slugDir = path.join(root, 'tmp', s.name);
     const chats = path.join(slugDir, 'chats');
-    const names = readdir(chats);
+    // A project folder that could not be read: its chats are skipped, reported (fsx.entry).
+    const names = fsx.entry(errors, s.name, () => readdir(chats), []);
     if (!names.length) continue;
-    const cwd = (readText(path.join(slugDir, '.project_root')) || '').trim() || projectOf(s.name);
+    const root0 = fsx.entry(errors, s.name, () => readText(path.join(slugDir, '.project_root')), false);
+    if (root0 === false) continue;
+    const cwd = (root0 || '').trim() || projectOf(s.name);
     for (const f of names) {
       const full = path.join(chats, f.name);
       const m = /^session-.+\.(jsonl|json)$/.exec(f.name);
-      if (m && f.isFile()) { consider(full, m[1], s.name, cwd, ''); continue; }
+      if (m && f.isFile()) { fsx.entry(errors, f.name, () => consider(full, m[1], s.name, cwd, '')); continue; }
       if (!f.isDirectory()) continue;
       // A sub-agent directory, named for the main session's (sanitised) id.
-      for (const c of readdir(full)) if (c.isFile() && /\.jsonl$/.test(c.name)) consider(path.join(full, c.name), 'jsonl', s.name, cwd, f.name);
+      for (const c of fsx.entry(errors, f.name, () => readdir(full), [])) {
+        if (c.isFile() && /\.jsonl$/.test(c.name)) fsx.entry(errors, c.name, () => consider(path.join(full, c.name), 'jsonl', s.name, cwd, f.name));
+      }
     }
   }
-  return [...found.values()].map(({ size, ...e }) => e);
+  return fsx.listed([...found.values()].map(({ size, ...e }) => e), errors);
 }
 
 /** Complete lines only (a partial last line is held back); malformed lines are skipped. */

package/editors/normalise/opencode.js

@@ -51,6 +51,7 @@
 //   part, session_message, session_input and event are read.
 
 const fs = require('fs');
+const fsx = require('./fsx');
 const path = require('path');
 const crypto = require('crypto');
 const { fileURLToPath } = require('url');
@@ -71,8 +72,8 @@ function storeRootDefault(env = process.env) {
   return harnessPaths.dataFolder(HARNESS, { env });
 }
 
-function isFile(p) { try { return fs.statSync(p).isFile(); } catch { return false; } }
-function readdir(dir) { try { return fs.readdirSync(dir).sort(); } catch { return []; } }
+function isFile(p) { return fsx.isFile(p); }
+function readdir(dir) { return (fsx.readdir(dir) || []).sort(); }
 
 /**
  * The session DBs of one data folder: OPENCODE_DB when set (absolute, or relative to the folder;
@@ -171,7 +172,7 @@ function keyColumns(sc) {
  */
 function list(storeRoot = storeRootDefault(), { env = process.env } = {}) {
   storeRoot = path.resolve(storeRoot);
-  try { if (!fs.statSync(storeRoot).isDirectory()) return []; } catch { return []; }
+  if (!fsx.isDir(storeRoot)) return [];
   const spills = path.join(harnessPaths.resolveNative(storeRoot), 'tool-output');
   const out = [];
   for (const { file, harness_path } of dbFiles(storeRoot, env)) {
@@ -264,12 +265,12 @@ function spilled(spillDir, p) {
   const ref = path.basename(String(p || ''));
   if (!SPILL.test(ref)) return null;
   if (!spillDir) return { ref, sha: null, len: null };
-  try {
-    const file = path.join(spillDir, ref);
-    if (!fs.lstatSync(file).isFile()) return { ref, sha: null, len: null };
-    const buf = fs.readFileSync(file);
-    return { ref, sha: sha256(buf), len: buf.length };
-  } catch { return { ref, sha: null, len: null }; }
+  const file = path.join(spillDir, ref);
+  const st = fsx.lstat(file);   // a link is never followed
+  if (!st || !st.isFile()) return { ref, sha: null, len: null };
+  const buf = fsx.readBytes(file);
+  if (!buf) return { ref, sha: null, len: null };
+  return { ref, sha: sha256(buf), len: buf.length };
 }
 
 /** A non-file URL cut to scheme://host (its path and query may carry a token), else one marker. */

package/editors/normalise/vscode.js

@@ -39,6 +39,7 @@
 // - redaction by field path: `selectedModel.metadata.auth`, wherever a `selectedModel` sits.
 
 const fs = require('fs');
+const fsx = require('./fsx');
 const path = require('path');
 const { fileURLToPath } = require('url');
 
@@ -64,7 +65,9 @@ function storeRootDefault() {
 /** A workspace's folder from workspaceStorage/<ws>/workspace.json: a path for file URIs, else the URI. */
 function workspaceFolder(wsDir) {
   let j;
-  try { j = JSON.parse(fs.readFileSync(path.join(wsDir, 'workspace.json'), 'utf-8')); } catch { return ''; }
+  const text = fsx.readText(path.join(wsDir, 'workspace.json'));
+  if (text === null) return '';
+  try { j = JSON.parse(text); } catch { return ''; }
   const uri = typeof j.folder === 'string' ? j.folder : (typeof j.workspace === 'string' ? j.workspace : '');
   if (!uri.startsWith('file:')) return uri;
   try { return fileURLToPath(uri); } catch { return decodeURIComponent(uri.replace(/^file:\/\//, '')); }
@@ -78,30 +81,32 @@ function workspaceFolder(wsDir) {
  * while the file stays the same (the mtime only picks between two files).
  */
 function list(storeRoot = storeRootDefault()) {
-  let harnessPath;
-  try { harnessPath = fs.realpathSync.native(storeRoot); } catch { return []; }
+  const harnessPath = fsx.realpath(storeRoot);
+  if (!harnessPath) return [];
   const client = CLIENTS[path.basename(harnessPath)] || 'vscode';
   const user = path.join(harnessPath, 'User');
   const wsRoot = path.join(user, 'workspaceStorage');
   const dirs = [];
-  let wss = [];
-  try { wss = fs.readdirSync(wsRoot).sort(); } catch { wss = []; }
+  const wss = (fsx.readdir(wsRoot) || []).sort();
   for (const ws of wss) dirs.push({ dir: path.join(wsRoot, ws, 'chatSessions'), wsDir: path.join(wsRoot, ws), workspace: ws });
   dirs.push({ dir: path.join(user, 'globalStorage', 'emptyWindowChatSessions'), wsDir: null, workspace: '' });
 
   const bySid = new Map();
+  const errors = [];   // a workspace or chat file that could not be read: skipped, reported (fsx.entry)
   for (const d of dirs) {
-    let names;
-    try { names = fs.readdirSync(d.dir).sort(); } catch { continue; }
+    const where = d.workspace || 'emptyWindowChatSessions';
+    const names = fsx.entry(errors, where, () => fsx.readdir(d.dir), null);
+    if (!names) continue;
+    names.sort();
     let cwd = null;
     for (const name of names) {
       const m = /^(.+)\.(jsonl|json)$/.exec(name);
       if (!m) continue;
       const file = path.join(d.dir, name);
-      let st;
-      try { st = fs.statSync(file); } catch { continue; }
-      if (!st.isFile()) continue;
-      if (cwd === null) cwd = d.wsDir ? workspaceFolder(d.wsDir) : '';
+      const st = fsx.entry(errors, m[1], () => fsx.stat(file), null);
+      if (!st || !st.isFile()) continue;
+      if (cwd === null) cwd = d.wsDir ? fsx.entry(errors, where, () => workspaceFolder(d.wsDir), null) : '';
+      if (cwd === null) break;   // its workspace.json refused: the workspace's chats are skipped, reported
       const e = {
         harness: HARNESS, session_id: m[1], file, format: m[2], workspace: d.workspace, cwd, client, harness_path: harnessPath, mtime: st.mtimeMs,
       };
@@ -110,7 +115,7 @@ function list(storeRoot = storeRootDefault()) {
       if (better) bySid.set(e.session_id, e);
     }
   }
-  return [...bySid.values()].map(({ mtime, ...e }) => e);
+  return fsx.listed([...bySid.values()].map(({ mtime, ...e }) => e), errors);
 }
 
 /** Change-log lines: complete lines only (a partial last line is held back), malformed ones skipped. */
@@ -437,7 +442,8 @@ function load(session) {
   if (Array.isArray(session.ops)) return replay(session.ops.map((o) => (typeof o === 'string' ? JSON.parse(o) : JSON.parse(JSON.stringify(o)))));
   let text = typeof session.text === 'string' ? session.text : null;
   if (text == null) {
-    try { text = fs.readFileSync(session.file, 'utf-8'); } catch { return snap(null); }
+    text = fsx.readText(session.file);
+    if (text === null) return snap(null);
   }
   const json = session.format === 'json' || (session.file && /\.json$/.test(session.file));
   if (json) { try { return snap(JSON.parse(text)); } catch { return snap(null); } }

package/editors/normalise/zed.js

@@ -79,6 +79,7 @@
 //   signatures, the git snapshot (its remote URL) and sandbox grants are never emitted.
 
 const fs = require('fs');
+const fsx = require('./fsx');
 const path = require('path');
 const crypto = require('crypto');
 const zlib = require('zlib');
@@ -154,23 +155,28 @@ function openThreads(file) {
 }
 
 /** session_id → true for the threads any channel's sidebar marks archived (a flag, no time). */
-function readArchived(storeRoot) {
+function readArchived(storeRoot, errors) {
   const out = new Map();
   let channels = [];
-  try { channels = fs.readdirSync(path.join(storeRoot, 'db')).sort(); } catch { return out; }
+  const names = fsx.readdir(path.join(storeRoot, 'db'));
+  if (!names) return out;
+  channels = names.sort();
+  // A channel's workspace DB that could not be read: its archive flags are skipped, reported (fsx.entry).
   for (const ch of channels) {
     const file = path.join(storeRoot, 'db', ch, 'db.sqlite');
-    try { if (!fs.statSync(file).isFile()) continue; } catch { continue; }
-    let db = null;
-    try {
-      db = openReadOnly(file);
-      const cols = columnsOf(db, 'sidebar_threads');
-      if (!cols.has('session_id') || !cols.has('archived')) continue;
-      for (const r of db.prepare('SELECT session_id, archived FROM sidebar_threads WHERE session_id IS NOT NULL').all()) {
-        const sid = String(textOf(r.session_id));
-        if (sid && Number(r.archived) === 1) out.set(sid, true);
-      }
-    } finally { if (db) try { db.close(); } catch { /* closed */ } }
+    fsx.entry(errors, path.join('db', ch), () => {
+      if (!fsx.isFile(file)) return;
+      let db = null;
+      try {
+        db = openReadOnly(file);
+        const cols = columnsOf(db, 'sidebar_threads');
+        if (!cols.has('session_id') || !cols.has('archived')) return;
+        for (const r of db.prepare('SELECT session_id, archived FROM sidebar_threads WHERE session_id IS NOT NULL').all()) {
+          const sid = String(textOf(r.session_id));
+          if (sid && Number(r.archived) === 1) out.set(sid, true);
+        }
+      } finally { if (db) try { db.close(); } catch { /* closed */ } }
+    });
   }
   return out;
 }
@@ -186,7 +192,7 @@ function readArchived(storeRoot) {
  */
 function list(storeRoot = storeRootDefault()) {
   const file = dbPathOf(storeRoot);
-  try { if (!fs.statSync(file).isFile()) return []; } catch { return []; }
+  if (!fsx.isFile(file)) return [];
   const harnessPath = harnessPaths.resolveNative(storeRoot);
   const db = openThreads(file);
   let rows = [];
@@ -196,9 +202,10 @@ function list(storeRoot = storeRootDefault()) {
     rows = db.prepare(`SELECT id, ${cols.has('updated_at') ? 'updated_at' : "'' AS updated_at"} FROM threads ORDER BY id`).all()
       .map((r) => ({ id: String(textOf(r.id)), updated_at: r.updated_at == null ? '' : String(textOf(r.updated_at)) }));
   } finally { db.close(); }
-  const archived = readArchived(storeRoot);
+  const errors = [];
+  const archived = readArchived(storeRoot, errors);
   const rel = path.relative(storeRoot, file);
-  return rows.map((r) => ({ harness: HARNESS, harness_path: harnessPath, session_id: r.id, db_rel: rel, updated_at: r.updated_at, archived: archived.get(r.id) === true }));
+  return fsx.listed(rows.map((r) => ({ harness: HARNESS, harness_path: harnessPath, session_id: r.id, db_rel: rel, updated_at: r.updated_at, archived: archived.get(r.id) === true })), errors);
 }
 
 /** One thread's row and blob, read together so they are of the same save. */

package/editors/sqlite.js

@@ -21,6 +21,7 @@
 const fs = require('fs');
 const path = require('path');
 const { pathToFileURL } = require('url');
+const fsx = require('./normalise/fsx');
 
 let DatabaseSync = null;
 
@@ -48,21 +49,30 @@ function openReadOnly(file) {
   // torn — measured: 3,525 "malformed" and 304 corrupt reads in 10,625 against a live writer, none
   // with the plain open, which in rollback mode creates no file. The header says which (byte 18:
   // 1 rollback, 2 WAL).
-  if (!fs.existsSync(file)) throw new Error(`unable to open database file: ${file}`);
-  if (!fs.existsSync(`${file}-wal`) && isWalHeader(file)) return new DatabaseSync(`${pathToFileURL(path.resolve(file)).href}?immutable=1`, { readOnly: true });
+  //
+  // The file is probed first, by the normalisers' rule (normalise/fsx.js): not there throws with code
+  // ENOENT (a caller's "no store"); any other error, a permission denial or macOS privacy, throws
+  // with its own errno and path, never as SQLite's "unable to open" that a caller could take for
+  // absence. A SQLite error on a file the probe could read is SQLite's, and stays an error.
+  const head = headerOf(file);
+  if (head === null) throw Object.assign(new Error(`unable to open database file: ${file}`), { code: 'ENOENT', path: file });
+  if (!fsx.stat(`${file}-wal`) && isWal(head)) return new DatabaseSync(`${pathToFileURL(path.resolve(file)).href}?immutable=1`, { readOnly: true });
   return new DatabaseSync(file, { readOnly: true });
 }
 
-/** True when the file's SQLite header says WAL mode (file format write version, byte 18, is 2). */
-function isWalHeader(file) {
-  let fd;
+/** The file's first 20 bytes, or null when it is not there; any other error throws (fsx.FsError). */
+function headerOf(file) {
+  const fd = fsx.openRead(file);
+  if (fd === null) return null;
   try {
-    fd = fs.openSync(file, 'r');
     const b = Buffer.alloc(20);
-    return fs.readSync(fd, b, 0, 20, 0) === 20 && b.toString('latin1', 0, 15) === 'SQLite format 3' && b[18] === 2;
-  } catch { return false; } finally { if (fd !== undefined) fs.closeSync(fd); }
+    return b.subarray(0, fs.readSync(fd, b, 0, 20, 0));
+  } catch (e) { throw new fsx.FsError('read', file, e); } finally { fs.closeSync(fd); }
 }
 
+/** True when a SQLite header says WAL mode (file format write version, byte 18, is 2). */
+const isWal = (b) => b.length === 20 && b.toString('latin1', 0, 15) === 'SQLite format 3' && b[18] === 2;
+
 /**
  * A column's value as text, whether SQLite handed back TEXT or BLOB.
  *

package/memhouse/shipper/capture.js

@@ -2036,6 +2036,25 @@ const refusesVanish = (x) => !!(x.vanishRefused || x.stillThere || x.candidateEr
 const storesN = (n) => `${n} store${n === 1 ? '' : 's'}`;
 /** One shell word: as is when it is plain, else single-quoted (a store path may hold spaces). */
 const shellWord = (w) => (/^[A-Za-z0-9_./:@%+=,-]+$/.test(w) ? w : `'${String(w).replace(/'/g, "'\\''")}'`);
+/**
+ * The pass's normaliser errors (normalise-run.js), one line per editor and code, and the advice for
+ * a denial (normalise/fsx.js hintOf): `status` and `doctor` read them back from the log (passlog.js).
+ */
+function normaliseLines(s) {
+  const by = new Map();
+  for (const e of (s.normalise && s.normalise.errors) || []) {
+    const k = `${e.harness}\u0000${e.code || ''}`;
+    if (!by.has(k)) by.set(k, { ...e, n: 0 });
+    by.get(k).n++;
+  }
+  const out = [];
+  for (const e of by.values()) {
+    out.push(`[memhouse] normalise error: ${e.harness} (${e.code || 'error'}): ${e.error}${e.n > 1 ? ` (and ${e.n - 1} more)` : ''}`);
+    const hint = require('../../editors/normalise/fsx').hintOf(e);
+    if (hint) out.push(`[memhouse] normalise hint: ${e.harness} (${e.code || 'error'}): ${hint}`);
+  }
+  return out;
+}
 function vanishLines(s) {
   const out = [];
   for (const x of s.adapters || []) {
@@ -2561,6 +2580,7 @@ async function main(argv = process.argv.slice(2), opts = {}) {
         s = await runPass(ctx);
         console.log(`${summary(s)} in ${((Date.now() - t0) / 1000).toFixed(1)}s`);
         for (const line of vanishLines(s)) console.log(line);
+        for (const line of normaliseLines(s)) console.log(line);
         // Without the package a profile pass captures no Claude Code or Codex session at all; that
         // must not read as a quiet pass with nothing new.
         const ac = require('./adapter-capture');
@@ -2591,7 +2611,7 @@ module.exports = {
   applyAdapters, adapterHarnesses, adapterPass, routeAdapterSessions, ADAPTER_MAX_SESSION_BYTES,
   parseSourceRoots, classifyRoot, discover, Cache, cachePath, captureFile, runPass, buildSession, batches,
   archiveRows, recordsFromArchive, rebuildFromArchive, replayArchive, manifestRow, fkey, splitLines, prefixDigest, extendDigest, digestOf, digestBuffer, HttpHouse, DryHouse,
-  TABLE_COLUMNS, dedupeSession, dedupeSweep, codexRepeatsOf, profileRepeatsOf, profileAdapterPass, profileRouting, main, defaultAdapters, applyBridge, combineRoots, storeRoots, adapterOf, summary, vanishLines,
+  TABLE_COLUMNS, dedupeSession, dedupeSweep, codexRepeatsOf, profileRepeatsOf, profileAdapterPass, profileRouting, main, defaultAdapters, applyBridge, combineRoots, storeRoots, adapterOf, summary, vanishLines, normaliseLines,
   passLock, PASS_LOCK_STALE_MS, PASS_LOCK_REFRESH_MS, duplicatesRefusal, reloadCache,
   BLOCK, ANCHOR, CHUNK, BATCH_ROWS, BATCH_BYTES, FRAGMENT_MS,
 };

package/memhouse/shipper/normalise-run.js

@@ -71,6 +71,18 @@ function messagesIn(file) {
   return n;
 }
 
+// An error as the pass reports it: its first line, and a code `status` and `doctor` key their advice on,
+// with the path when a read named one. The code is the errno of a file read (normalise/fsx.js: EACCES,
+// EPERM, EIO) or SQLite's, named (SQLITE_CANTOPEN on a file the probe could open, SQLITE_NOTADB).
+const SQLITE_CODES = { 5: 'BUSY', 6: 'LOCKED', 10: 'IOERR', 11: 'CORRUPT', 14: 'CANTOPEN', 23: 'AUTH', 26: 'NOTADB' };
+function errorOf(e) {
+  const out = { error: String(e && e.message).split('\n')[0] };
+  if (e && typeof e.code === 'string' && /^E[A-Z0-9]+$/.test(e.code)) out.code = e.code;
+  else if (e && Number.isInteger(e.errcode)) out.code = `SQLITE_${SQLITE_CODES[e.errcode & 0xff] || e.errcode & 0xff}`;
+  if (e && e.path) out.path = String(e.path);
+  return out;
+}
+
 function loadIndex(file) {
   try { const d = JSON.parse(fs.readFileSync(file, 'utf-8')); return d && d.v === 1 && d.sessions ? d : { v: 1, sessions: {} }; } catch { return { v: 1, sessions: {} }; }
 }
@@ -83,7 +95,7 @@ function saveIndex(file, idx) {
 
 /**
  * Bring every normalised log up to date. Returns { harnesses, sessions, normalised, written:
- * { new, append, rewrite, same }, errors: [{ harness, session_id?, error }], ms }.
+ * { new, append, rewrite, same }, errors: [{ harness, session_id?, error, code?, path? }], ms } (errorOf).
  * opts.mods: the modules (default: editors/normalise/); opts.storeRoot(harness): a store to list
  * instead of the module's default (tests).
  */
@@ -99,12 +111,19 @@ function run({ home, mods = modules(), storeRoot = null, log = () => {}, now = D
   for (const m of mods.filter((x) => scope.normaliserInScope(x.HARNESS, editors))) {
     let list;
     try { list = storeRoot && storeRoot(m.HARNESS) ? m.list(storeRoot(m.HARNESS)) : m.list(); } catch (e) {
-      st.errors.push({ harness: m.HARNESS, error: String(e && e.message).split('\n')[0] });
+      st.errors.push({ harness: m.HARNESS, ...errorOf(e) });
       // A store busy for a moment (an editor mid-save) is no reason to forget its sessions: keep
       // their index entries, or the next pass would normalise every one of them again.
       for (const [k, v] of Object.entries(idx.sessions)) if (k.startsWith(`${m.HARNESS}\t`)) seen[k] = v;
       continue;
     }
+    // An entry a readable store could not read (editors/normalise/fsx.js entry: one session's file or
+    // folder refused) is that session's error, the store's other sessions listed. Its index entries are
+    // kept, as a failed list keeps them, so it is not normalised again from scratch once it reads.
+    for (const e of (list && list.errors) || []) {
+      st.errors.push({ harness: m.HARNESS, ...e });
+      for (const [k, v] of Object.entries(idx.sessions)) { const [h, , sid] = k.split('\t'); if (h === m.HARNESS && sid === e.session_id) seen[k] = v; }
+    }
     if (!list || !list.length) continue;
     st.harnesses++;
     for (const s of list) {
@@ -138,7 +157,7 @@ function run({ home, mods = modules(), storeRoot = null, log = () => {}, now = D
         st.normalised++;
         seen[k0] = { sig, file, at: now };
       } catch (e) {
-        st.errors.push({ harness: m.HARNESS, session_id: String(s.session_id), error: String(e && e.message).split('\n')[0] });
+        st.errors.push({ harness: m.HARNESS, session_id: String(s.session_id), ...errorOf(e) });
         seen[k0] = { sig, file: '', failed: true, at: now, tries: prev && prev.failed && prev.sig === sig ? (prev.tries || 0) + 1 : 0 };
       }
     }
@@ -151,4 +170,4 @@ function run({ home, mods = modules(), storeRoot = null, log = () => {}, now = D
   return st;
 }
 
-module.exports = { run, modules, sigOf, DIR };
+module.exports = { run, modules, sigOf, errorOf, DIR };

package/memhouse/shipper/passlog.js

@@ -19,8 +19,12 @@ const FAIL = /\[memhouse\] (?:v3 )?pass failed: ([^\n]*)/;
 // A pass's refused vanish sweeps and its vanish warnings, printed after its pass line (capture.js vanishLines).
 const VANISH = /^\[memhouse\] vanish refused: ([^\n]*)$/gm;
 const WARNED = /^\[memhouse\] vanish warning: ([^\n]*)$/gm;
+// The pass's normaliser errors and their advice, printed after its pass line (capture.js normaliseLines).
+const NORM_ERR = /^\[memhouse\] normalise error: (\S+) \(([^)]*)\): ([^\n]*)$/gm;
+// A hint names its editor and code: one editor can fail on two codes that need different advice.
+const NORM_HINT = /^\[memhouse\] normalise hint: (\S+) \(([^)]*)\): ([^\n]*)$/gm;
 
-/** @returns {{outcome:'ok'|'failed'|null, sessions?:number, rows?:number, why?:string, auth?:boolean, vanishRefused?:string[]}} */
+/** @returns {{outcome:'ok'|'failed'|null, sessions?:number, rows?:number, why?:string, auth?:boolean, vanishRefused?:string[], normaliseErrors?:object[]}} */
 function lastPass(logText) {
   const text = String(logText || '');
   let lastOk = -1; let lastFail = -1; let okM = null; let failM = null;
@@ -33,12 +37,18 @@ function lastPass(logText) {
   }
   const refused = [...text.slice(lastOk).matchAll(VANISH)].map((m) => m[1].trim());
   const warned = [...text.slice(lastOk).matchAll(WARNED)].map((m) => m[1].trim());
+  const hints = new Map([...text.slice(lastOk).matchAll(NORM_HINT)].map((m) => [`${m[1]}\u0000${m[2]}`, m[3].trim()]));
+  const normalise = [...text.slice(lastOk).matchAll(NORM_ERR)].map((m) => {
+    const hint = hints.get(`${m[1]}\u0000${m[2]}`);
+    return { harness: m[1], code: m[2], error: m[3].trim(), ...(hint ? { hint } : {}) };
+  });
   return {
     ...(okM[1] !== undefined ? { outcome: 'ok', sessions: Number(okM[1]), rows: Number(okM[2]) }
       : okM[3] !== undefined ? { outcome: 'ok', sessions: Number(okM[3]), rows: Number(okM[4]), v3: true }
         : { outcome: 'ok', sessions: Number(okM[5]), rows: Number(okM[6]), v3: true, profile: true }),
     ...(refused.length ? { vanishRefused: refused } : {}),
     ...(warned.length ? { vanishWarned: warned } : {}),
+    ...(normalise.length ? { normaliseErrors: normalise } : {}),
   };
 }
 
@@ -86,6 +96,16 @@ function vanishChecks(lp) {
   return out;
 }
 
+/**
+ * doctor's checks for the last pass's normaliser errors: one per editor and code, a failure, with the
+ * advice capture.js printed (macOS: Full Disk Access) or the log to read.
+ */
+function normaliseChecks(lp) {
+  return ((lp && lp.normaliseErrors) || []).map((e) => ({
+    ok: false, label: `${e.harness}: a read failed in the last pass (${e.code}): ${e.error}`, hint: e.hint || 'read: ~/.memhouse/logs/shipper.log',
+  }));
+}
+
 /** The last pass of the shipper log in `logDir` (its last 64 KiB): what `status` and doctor read. */
 function readLastPass(logDir) {
   const fs = require('fs'); const path = require('path');
@@ -97,4 +117,4 @@ function readLastPass(logDir) {
   } catch { return { outcome: null }; }
 }
 
-module.exports = { lastPass, readLastPass, describeFailure, describeVanish, describeVanishWarning, vanishChecks };
+module.exports = { lastPass, readLastPass, describeFailure, describeVanish, describeVanishWarning, vanishChecks, normaliseChecks };

package/memhouse/shipper/profile-capture.js

@@ -7,13 +7,16 @@
 // THE PASS (each line is a decision; the rest is adapter-capture.js's)
 //   archive     session_units: every unit verbatim (enc, sha256, redacted, orig_sha256), its body in
 //               1 MiB chunks; and a read manifest (unit_key '', enc 'manifest') for every read that
-//               opens a generation or delivers units or gone keys (harness-sessions-sql §4.1). Only a reset
-//               opens a generation past the first (sessions §3); a snapshot replaces its one unit.
+//               opens a generation or delivers units or gone keys (harness-sessions-sql §4.1). A read
+//               opens a generation when it is the source's first, a reset or a snapshot (§4.1): a
+//               changed snapshot's one unit goes into a generation of its own.
 //               An archive write carries one captured_at, set by the capture and later than the
 //               session's last one (kept in the local state as archivedAt; on a refold, the
 //               archive's latest), so a capped read's steps stay apart and a clock that steps back
 //               between passes never reorders them; never left to the server's default, which a
-//               parallel parse of a big insert may evaluate per chunk: it is the replay's pass id. A write that fits one batch is one statement, manifests last;
+//               parallel parse of a big insert may evaluate per chunk: it is the replay's pass id. The one
+//               exception is the re-run of a pass that stopped before its cursor write: it writes under
+//               that pass's captured_at, so the two replay as one pass (header: crash). A write that fits one batch is one statement, manifests last;
 //               a bigger one writes its units first (~8 MiB a statement), then all its manifests in
 //               one statement however big: a manifest never lands before its units (a stop between
 //               the two leaves units no manifest names, which the fold ignores and the re-read
@@ -35,9 +38,13 @@
 //               statement holds a pass's manifests, header: archive). Each pass reads the unit
 //               versions it needs and does not hold, at most REPLAY_MAX_BYTES (32 MiB) and 1,000
 //               units a response, and keeps only what the fold holds, so a unit edited at every pass
-//               is not read or held in all its versions at once. A pass
-//               that stopped before its cursor write cannot be told from one that did not: its reads
-//               replay as a pass of their own
+//               is not read or held in all its versions at once. A pass that stopped before its
+//               cursor write cannot be told from one that did not by the archive alone; its re-run,
+//               which sees the archive ahead of the cursor, writes under the stopped pass's
+//               captured_at (header: crash), so the stopped read and the re-run replay as one pass.
+//               Each source's input to a pass's derive is that pass's net change to its fold: with
+//               one manifest a source, as every other pass has, exactly that manifest's units and gone
+//               keys; across a stopped read and its re-run, a unit the re-run took back is no change
 //   re-derive   an upgrade's refold (harness-sessions-sql §8: a new adapter_ver replayed from session_units
 //               alone) replays the archive the same way, under the new version, over the house's rows:
 //               every row it makes, current or a removed version, is written above the house's revs at
@@ -84,13 +91,22 @@
 //   crash       the local state is dropped before the first write and saved after the last, so a
 //               pass that stops anywhere leaves none, and the next one refolds from the house. A
 //               refold reads the archive's manifests first: when a stopped pass left them ahead of the
-//               cursor, its generation, progress and delta fold continue from the archive (a reset
-//               opens the generation past the archive's), so no two manifests of a generation share a
-//               progress and the fold is what the archive says. A source re-read from scratch that
-//               delivers exactly what the generation the stopped pass opened holds (its own read: no
-//               cursor yet, or a reset; never a source_gone generation) stays in it and archives
-//               nothing: only its position moves. So does a stream re-read from the cursor that
-//               delivers only what the stopped pass appended to the cursor's generation
+//               cursor, its generation, progress and delta fold continue from the archive (a reset or
+//               a snapshot opens the generation past the archive's), so no two manifests of a
+//               generation share a progress and the fold is what the archive says. A source re-read
+//               from scratch that delivers exactly what the generation the stopped pass opened holds
+//               (its own read: no cursor yet, a reset or a snapshot; never a source_gone generation)
+//               stays in it and archives nothing: only its position moves. So does a stream re-read
+//               from the cursor that delivers only what the stopped pass appended to the cursor's
+//               generation, and a snapshot that delivers what a stopped pass archived in the cursor's
+//               own generation (an archive from before a snapshot opened a generation). The re-run
+//               writes its archive under the stopped pass's captured_at (archive), so a replay folds
+//               the stopped read and the re-run as one pass: a unit the stopped read delivered that the
+//               re-run no longer sees is never derived, as in a clean run of what was committed
+//               (harness-sessions-sql SPEC-QUESTIONS q20). A stop past the rows has already written the
+//               stopped read's records: the re-run keeps any it no longer sees as removed versions,
+//               which a replay does not make (nor does the reference engine's), and an upgrade's
+//               re-derive tombstones them
 //   edges       after the rows, before the cursor (harness-sessions-sql §5 step 4): the engine edges the session
 //               is the fork or the origin of (profile-edges.js), so the rollup bills a fork's copied
 //               usage once. A fork whose edge moved and is not this session gets its sessions row
@@ -284,6 +300,9 @@ async function captureSession(ctx, ad, d, store, listed) {
     ...archOnly.filter((sk) => ours.has(sk) && !A.get(sk).gone).map((sk) => ({ source_key: sk, mode: '', role: 'main', omitted: true })),
   ];
   const confirmed = new Set();   // omitted sources the adapter answered are not there
+  // The captured_at of a pass that stopped before its cursor write (an archive ahead of the cursor): this
+  // re-run writes under it (header: crash). null when no source is ahead.
+  let stoppedAt = null;
   for (const src of [...listed.sources, ...omitted]) {
     const sk = src.source_key;
     const h = H.get(sk);
@@ -306,6 +325,7 @@ async function captureSession(ctx, ad, d, store, listed) {
         // Only a generation the stopped pass opened (newer than the cursor's) and that holds a read
         // (not a source_gone one) can be the one a re-read from scratch repeats.
         resumed = a.gen > hGen && !a.gone ? a.fold : null;
+        if (a.at && (stoppedAt === null || String(a.at) > stoppedAt)) stoppedAt = String(a.at);
         appendedTo = a.gen === hGen && !a.gone ? { fold: a.fold, progress: String(a.progress) } : null;
       } else prior = h && !goneOf(h) ? { gen: Number(h.generation), progress: String(h.progress), position: parse(h.position, null), known: parse(h.known, {}) } : null;
     }
@@ -339,7 +359,15 @@ async function captureSession(ctx, ad, d, store, listed) {
         units = r.units; gone = []; knownNext = {}; full = true; opens = true;
         break;
       case 'snapshot':
-        units = r.units; knownNext = {};
+        // A changed snapshot opens the next generation (harness-sessions-sql §4.1), unless it is the
+        // stopped pass's own read again (header: crash): the unit a generation the stopped pass opened
+        // holds, or one it archived in the cursor's own generation (before a snapshot opened one).
+        if ((resumed && sameUnits(r.units, resumed)) || (appendedTo && sameUnits(r.units, appendedTo.fold))) {
+          reread = r.units; knownNext = Object.fromEntries(resumed || appendedTo.fold); full = true;
+          break;
+        }
+        gen = prior ? gen + 1 : gen;
+        units = r.units; knownNext = {}; opens = true;
         break;
       case 'appended': case 'delta':
         units = r.units;
@@ -368,7 +396,7 @@ async function captureSession(ctx, ad, d, store, listed) {
     }
     sources[sk] = { gen, progress, position: r.position ?? null, known: src.mode === 'stream' ? {} : knownNext, mode: src.mode || '', role: src.role || 'main' };
     if (changed) fresh.push({ source_key: sk, generation: gen, complete: false, units, gone });
-    if (!prior || r.continuity === 'reset') {
+    if (!prior || r.continuity === 'reset' || r.continuity === 'snapshot') {
       const byKey = new Map();
       for (const u of reread || units) { const k = String(u.unit_key); byKey.delete(k); byKey.set(k, { ...u, unit_key: k }); }
       opened.set(sk, { gen, units: [...byKey.values()] });
@@ -501,7 +529,9 @@ async function captureSession(ctx, ad, d, store, listed) {
       last = known.length || local == null ? Math.max(0, ...known)
         : (typeof ctx.house.archivedAt === 'function' ? await ctx.house.archivedAt(ref) : 0);
     }
-    archivedAt = Math.max(Number(ctx.now()), last + 1);
+    // The re-run of a stopped pass writes under that pass's captured_at (header: crash); every other
+    // write is later than the session's last one.
+    archivedAt = stoppedAt !== null ? Date.parse(`${stoppedAt.replace(' ', 'T')}Z`) : Math.max(Number(ctx.now()), last + 1);
     for (const r of archive) r.captured_at = base.chTs(archivedAt);
     const units = archive.filter((r) => r.enc !== 'manifest');
     const manifests = archive.filter((r) => r.enc === 'manifest');
@@ -792,8 +822,9 @@ async function replayArchive(ctx, ad, d, key, ref, { adapter, adapterVer, order,
   const byOrder = (a, b) => (rank.has(a) ? rank.get(a) : order.length) - (rank.has(b) ? rank.get(b) : order.length) || (a < b ? -1 : a > b ? 1 : 0);
   const sources = new Map();
   // The units a pass needs and does not hold yet are read for that pass (paged by pass), and only
-  // what the fold still holds is kept after it: a snapshot or delta source keeps one generation and
-  // adds a version of a unit at every edit, so holding (or reading at once) every version a
+  // what the fold still holds is kept after it: a delta source keeps one generation and adds a
+  // version of a unit at every edit (so did a snapshot source, in an archive from before a snapshot
+  // opened a generation), so holding (or reading at once) every version a
   // generation ever named grows with the session's history. A read is bounded by bytes, at most
   // `maxBytes` of the units' archived text and 1,000 units a response (one unit larger than that
   // alone); the sizes of every unit a generation's manifests name are read once, with that
@@ -863,12 +894,24 @@ async function replayArchive(ctx, ad, d, key, ref, { adapter, adapterVer, order,
   let view = new Map();
   for (const pass of passes) {
     let full = state === null;
-    const fresh = [];
+    // Each source's input is the pass's net change to its fold (header: replay): with one manifest a
+    // source, that manifest's units and gone keys; across a stopped read and its re-run (one captured_at,
+    // header: crash), a unit the re-run took back is no change, and is never derived.
+    const before = new Map();
     for (const m of pass) {
+      if (!before.has(m.source_key)) { const s = sources.get(m.source_key); before.set(m.source_key, s ? { generation: s.generation, gone: s.gone, units: new Map(s.units) } : null); }
       foldManifest(sources, m);
       if (m.source_gone ? sources.get(m.source_key).gone !== 'kept' : m.continuity === 'reset') full = true;
-      else if (!m.source_gone && ((m.units || []).length || (m.gone || []).length)) {
-        fresh.push({ source_key: m.source_key, generation: m.generation, complete: false, units: await unitsAt(m.source_key, m.generation, m.units || []), gone: m.gone || [] });
+    }
+    const fresh = [];
+    if (!full) {
+      for (const [sk, b] of before) {
+        const s = sources.get(sk);
+        if (s.gone) continue;   // a kept vanish: its units stay as they were
+        const same = !!b && !b.gone && b.generation === s.generation;
+        const units = [...s.units].filter(([uk, sha]) => !same || b.units.get(uk) !== sha);
+        const gone = same ? [...b.units.keys()].filter((uk) => !s.units.has(uk)) : [];
+        if (units.length || gone.length) fresh.push({ source_key: sk, generation: s.generation, complete: false, units: await unitsAt(sk, s.generation, units), gone });
       }
     }
     fresh.sort((a, b) => byOrder(a.source_key, b.source_key));

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "memhouse",
-  "version": "0.20.2",
+  "version": "0.20.3",
   "description": "memhouse — agent conversation memory across 17 editors: parse-on-client shippers, a typed ClickHouse store with one set of rooms per member, and the agentlytics dashboard",
   "main": "bin/memhouse.js",
   "bin": {