memhouse 0.18.10 → 0.18.12

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/AGENTS.md +10 -11
  2. package/CHANGELOG.md +153 -13
  3. package/README.md +2 -2
  4. package/SECURITY.md +4 -2
  5. package/bin/memhouse.js +105 -49
  6. package/docs/guides/admin-password.md +1 -1
  7. package/docs/guides/configuration.md +2 -2
  8. package/docs/guides/install.md +1 -1
  9. package/docs/guides/invite.md +1 -1
  10. package/docs/guides/storage.md +10 -7
  11. package/docs/guides/uninstall.md +4 -2
  12. package/docs/releases/README.md +2 -0
  13. package/docs/releases/v0.18.10.md +7 -7
  14. package/docs/releases/v0.18.11.md +187 -0
  15. package/docs/releases/v0.18.12.md +59 -0
  16. package/docs/tutorials/01-first-house.md +2 -2
  17. package/editors/claude.js +70 -12
  18. package/editors/cursor-agent.js +1 -1
  19. package/examples/01-local-one-machine/README.md +4 -4
  20. package/examples/02-existing-clickhouse/README.md +7 -7
  21. package/memhouse/DESIGN.md +9 -2
  22. package/memhouse/RESUME-AND-UPDATE.md +1 -1
  23. package/memhouse/capabilities.js +1 -1
  24. package/memhouse/convert.js +3 -3
  25. package/memhouse/delivery/PROMPT.md +1 -1
  26. package/memhouse/delivery/plugin/.claude-plugin/plugin.json +1 -1
  27. package/memhouse/delivery/plugin/reference/HOUSE.md +21 -9
  28. package/memhouse/delivery/plugin/skills/access/SKILL.md +2 -2
  29. package/memhouse/delivery/plugin/skills/admin/SKILL.md +3 -3
  30. package/memhouse/delivery/plugin/skills/house/SKILL.md +2 -2
  31. package/memhouse/delivery/plugin/skills/recall/SKILL.md +36 -16
  32. package/memhouse/delivery/plugin/skills/sql/SKILL.md +2 -2
  33. package/memhouse/flags.js +28 -1
  34. package/memhouse/host.js +11 -6
  35. package/memhouse/house/HOUSE.md +6 -6
  36. package/memhouse/house/house.js +31 -5
  37. package/memhouse/invite-target.js +37 -0
  38. package/memhouse/self-update.js +1 -1
  39. package/memhouse/server/queries.js +11 -6
  40. package/memhouse/shipper/ship.js +148 -31
  41. package/package.json +3 -3
  42. package/public/THIRD_PARTY_NOTICES.txt +2114 -0
  43. package/ui/README.md +16 -0
  44. package/ui/vite.config.js +4 -1
@@ -0,0 +1,187 @@
1
+ # memhouse 0.18.11 — 2026-10-10
2
+
3
+ This release does three things:
4
+
5
+ - **Sessions with subagents stop re-shipping.** A Claude Code session that used subagents
6
+ now ships only its new rows. Before, it re-sent itself whole on every pass, and that was
7
+ most of the house's write load.
8
+ - **A security fix.** `invite` no longer sends a stored credential to whatever `--url` is
9
+ typed on the command line.
10
+ - **The 0.18.10 pre-tag list is finished**, and the gentar test kit is bumped.
11
+
12
+ **No drill was run for this release; waived (2026-10-08) under the maintainer's 0.18.10 ruling. The drill runs before 0.19.0.**
13
+
14
+ ## Read this before upgrading
15
+
16
+ **The first pass after the upgrade is a one-time bulk write.** It re-sends, whole and once,
17
+ every stored session that has subagents; after that, those sessions ship only their new
18
+ rows. Two things widen that first pass:
19
+
20
+ - **Sessions whose subagents wrote recently.** A session whose subagent files are newer
21
+ than the house's record of it is read again even if it did not grow, because
22
+ `lastUpdatedAt` now counts subagent writes.
23
+ - **Installs that share session files.** Two installs that ship the same session files
24
+ (two instances on one machine, or a synced directory) must be upgraded together. A
25
+ 0.18.10 and a 0.18.11 number the same session differently, and each would fork it back
26
+ on every pass.
27
+
28
+ **Where that write lands, and when to do it (2026-10-08):**
29
+
30
+ - Upgrade together any installs that ship the same session files (above).
31
+ - A house whose data disk sits on degraded storage: back it up first (data and access
32
+ storage), and upgrade its machines only once the disk is off the degraded array or the
33
+ array is repaired.
34
+ - Any house on healthy storage can upgrade normally.
35
+
36
+ ```bash
37
+ memhouse update # then: memhouse status — every line a check mark
38
+ ```
39
+
40
+ ## Security
41
+
42
+ ### `invite` never sends a stored credential to `--url` (G4)
43
+
44
+ `--url` is the invitee's address, typed on a command line. A typo, a tunnel, or somebody
45
+ else's server received credentials it should never have seen:
46
+
47
+ - the member's own credential, through the reachability probe and the read of its grants;
48
+ - an admin credential the install keeps, through provisioning.
49
+
50
+ The cause: `resolveConfig()` lets a flag outrank the env file, so the "configured house" URL
51
+ the code used *was* `--url`.
52
+
53
+ Now:
54
+
55
+ - **A stored credential goes only to the house it was saved for.** That covers the
56
+ member's own, a `--keep-admin` admin, and a password read from the env file.
57
+ - **`--url` gets an unauthenticated `/ping`**, only to confirm the invitee can reach it.
58
+ - **An admin given for this command** (`--admin-user`, or an exported
59
+ `MEMHOUSE_ADMIN_USER`, with its password from a file, stdin, the environment or a prompt)
60
+ still provisions at `--url`. That is what the person typing the command asked for.
61
+ - **`misc/invite-matrix.sh` points `--url` at a spy server** and asserts that it receives
62
+ nothing but a ping. On 0.18.10 the spy received two authenticated requests.
63
+
64
+ **Action:**
65
+
66
+ - If you ran `memhouse invite … --url <something unusual>` on 0.18.10 or earlier against a
67
+ host you do not control, rotate the member's password with `memhouse passwd`.
68
+ - If an admin credential is kept in the env file and the same applies, rotate that too.
69
+
70
+ ## The re-ship fix (#21)
71
+
72
+ Folded subagent rows were numbered by position, after the parent's turns. One new parent
73
+ turn moved every one of them, so the session forked to a new epoch and was re-sent whole.
74
+ Measured on a real house:
75
+
76
+ - 1.07 M message rows written in 24 h, against 10.2 M in total;
77
+ - one 6,120-line session stored 60 times in a day;
78
+ - one member's house at about 19 M message rows for roughly 295 k real ones.
79
+
80
+ What changed:
81
+
82
+ - **Subagent rows get a numbering block of their own:**
83
+ `seq = 1,000,000,000 + slot × 100,000 + turn`, and the same for tool calls' `idx`.
84
+ `slot` is the subagent's rank by start time, so neither the parent growing, a subagent
85
+ growing, nor a new subagent appearing moves a stored row.
86
+ - **A session without subagents is numbered exactly as before.**
87
+ - **`seq` is sparse.** Count rows with `count()`, never `max(seq) + 1`.
88
+ - **`ORDER BY seq` groups subagents.** Readers see the parent first, then each subagent's
89
+ turns together, in the order the subagents started. They used to be in agent-id order,
90
+ which is random.
91
+ - **A parse too big for the blocks falls back to positional numbering:**
92
+ - a parent past 1e9 turns;
93
+ - a subagent past 100,000 turns;
94
+ - more than 32,949 subagents.
95
+
96
+ Positional numbering is correct, and merely re-ships whole as before.
97
+ - **Subagent writes count as session activity.** A subagent that keeps running after its
98
+ parent goes quiet is shipped on the next pass.
99
+ - **A subagent caught before its first line is written holds its session for one pass.**
100
+ It is then ranked by its real start time; it no longer ranks last and moves later. A
101
+ file that never gets a timestamped line stops holding the session after 10 minutes.
102
+ - **Existing copies are not cleaned up.** Superseded epochs already in a house stay;
103
+ memhouse never deletes to make room. Reclaiming that space is a separate operator
104
+ decision, and no such cleanup may run on a house on degraded storage.
105
+
106
+ ## Also fixed
107
+
108
+ - **The dashboard no longer goes empty on a quiet house (G11).** The stats tables carry a
109
+ one-day TTL and were refreshed only by passes that shipped something.
110
+ - The shipper now also refreshes them when the newest generation is over an hour old.
111
+ - The dashboard re-resolves its tables every five minutes, falling back to the live
112
+ rollup when a stats table is empty.
113
+ - **The skills find their reference again (G26).** All five pointed at
114
+ `../reference/HOUSE.md`, a path that does not exist in the installed plugin; the file is
115
+ at `../../reference/HOUSE.md`.
116
+ - **`/mem:recall`'s queries run (A2).**
117
+ - "Recent sessions" used a column the room does not have.
118
+ - The search now returns the `hit_seq` that the transcript read needs.
119
+ - The read takes its window by position, so the subagent blocks cannot break it.
120
+ - **`HOUSE.md`'s subagent queries keep to the current parse (A3).** They used to double a
121
+ re-parsed session's subagent turns.
122
+ - **An option that takes no value no longer swallows the next word (A6).**
123
+ `update --no-install yes` ran the install it was told to skip, and `share --revoke bob`
124
+ revoked nobody.
125
+ - **`update --channel <name>` pins only a tag the registry has (A5).** Before, it never
126
+ wrote the pin at all.
127
+ - **`update --no-install` restarts a tarball or nightly install.** That is the command
128
+ every nightly build prints for itself. On exactly those builds it stopped at "no
129
+ automatic update for this install" and exited 1, leaving the old shipper running old
130
+ code. It now restarts the daemons, runs any migration, and checks the schema; it needs
131
+ no channel.
132
+ - **`update --no-install` (A6)** no longer prints "files updated", and it is no longer
133
+ refused on a project-dependency install.
134
+ - **The epoch log line counts rows (A4).**
135
+ - **Docs (A8): the host id is derived from the machine's own id**, as it has been since
136
+ 0.18.6, so a reinstall keeps it. Stale prose about timestamps and `extra.turn` is fixed
137
+ too.
138
+
139
+ ## License notices
140
+
141
+ **The dashboard now ships the license notices of what it bundles.** The built dashboard
142
+ (`public/`) includes React, chart.js, lucide-react, react-markdown, tailwindcss and every
143
+ other package it bundles. Their licenses (MIT, ISC) require the notice to travel with each
144
+ copy, and minification strips it from the code.
145
+ - `public/THIRD_PARTY_NOTICES.txt` is now generated at build time from the bundle's module
146
+ graph, and ships in the npm package.
147
+ - CI fails a package that lacks it.
148
+
149
+ Earlier published packages carried those copies without the notices.
150
+
151
+ ## Test infrastructure
152
+
153
+ - **gentar kit v0.9.3 (#18).** In phase 1, `memhouse-house` is reported UNVERIFIED (bench
154
+ only) instead of being dry-run on the CI host.
155
+ - **The suite tears down only a house it created.** It refuses to deploy over an existing
156
+ house, and a failed teardown fails it.
157
+ - **`misc/skill-recipe.sh` runs the skills' queries verbatim, extracted from the docs.**
158
+ The fixture includes a superseded parse and a subagent block.
159
+
160
+ ## Pre-tag list (0.18.10 review): final status
161
+
162
+ | item | where |
163
+ |---|---|
164
+ | A1 version bump | this release |
165
+ | A2 `/mem:recall` queries | #22 |
166
+ | A3 `HOUSE.md` subagent queries, epoch filter | #22 |
167
+ | A4 epoch log count | #21 |
168
+ | A5 `update --channel` validation | #21 |
169
+ | A6 `--no-install` checks; switches swallowing values | #21 |
170
+ | A7 upgrade-cost notes | #21, and this document |
171
+ | A8 stale prose (ts, shrink, `extra` turn, host id) | #21 |
172
+ | G4 `invite --url` and stored credentials (**security**) | #22 |
173
+ | G26 skills' reference path | #22 |
174
+ | G11 idle house's dashboard | #22 |
175
+ | SUB-1 unstamped subagent file | #21 |
176
+
177
+ ## Verification
178
+
179
+ - **Unit tests:** `npm test` passes on main, 125 checks.
180
+ - **CI:** green on ClickHouse 25.11 and 26.7.
181
+ - **Matrices in a lab**, on ClickHouse 26.7.4.58 (production's version) and 26.7.1.1315.
182
+ They were run during #22 on main with #21 and #18 merged:
183
+ - skill recipe 18/18;
184
+ - invite 58/58;
185
+ - one-layout 62/62.
186
+ - **Second looks:** a second model reviewed #21 and #22.
187
+ - **Labs:** kept light, because the lab host's RAID was degraded. No live house was touched.
@@ -0,0 +1,59 @@
1
+ # memhouse 0.18.12 — 2026-10-29
2
+
3
+ A patch release on the 0.18 line. It does one thing for users:
4
+
5
+ - **The `/mem` skills pre-approve read-only commands only.** Each skill's `allowed-tools`
6
+ now lists the memhouse read commands it teaches, instead of `Bash` as a whole.
7
+
8
+ No drill was run for this patch release. Its one user-facing change is held by
9
+ `misc/skill-allowlist-test.js`, which runs in `npm test`.
10
+
11
+ ## Upgrade
12
+
13
+ ```bash
14
+ memhouse update # refreshes the /mem skills wherever they are installed
15
+ ```
16
+
17
+ Then reload the skills in Claude Code with `/reload-plugins`, or restart the session.
18
+
19
+ If you upgraded with `npm install -g memhouse` instead, run `memhouse plugins install claude`
20
+ once, so the installed skills match the package.
21
+
22
+ Nothing else changes: same house layout, same schema, same daemons.
23
+
24
+ ## Changed
25
+
26
+ ### Skills: allowed tools restricted to read-only commands
27
+
28
+ Each `/mem` skill's `allowed-tools` names the memhouse read commands it uses:
29
+
30
+ | skill | runs without a prompt |
31
+ |---|---|
32
+ | `/mem:house` | `memhouse instance`, `whoami`, `status`, `stats`, `doctor`, and `share --list` (with or without `--json`) |
33
+ | `/mem:recall` | `memhouse search`, `resume`, `sessions-query`, `status`, `whoami` |
34
+ | `/mem:sql` | `memhouse sessions-query`, `stats`, `status`, `whoami` |
35
+ | `/mem:access` | `memhouse whoami`, `status`, and `share --list` (with or without `--json`) |
36
+ | `/mem:admin` | `memhouse whoami`, `status`, `instance` |
37
+
38
+ Any other command goes through Claude Code's normal permission prompt, as does a read
39
+ command with another command chained to it. Expect a prompt where a skill used to run a
40
+ command without one; approve it as you would any other command.
41
+
42
+ ## Repository and tests
43
+
44
+ These do not change the installed package's behaviour:
45
+
46
+ - `misc/skill-allowlist-test.js` keeps every skill's `allowed-tools` to read-only memhouse
47
+ commands. It joins `npm test`.
48
+ - The test tooling's deploy targets (lab address, test machine, engine checkout) are read
49
+ from an untracked `deploy/targets.env`. `deploy/targets.example.env` shows the shape.
50
+ - Test fixtures, examples, drill records and docs use stand-in names, hosts and paths.
51
+ `misc/env-guard-test.js` checks tracked files for real environment values, and it joins
52
+ `npm test`.
53
+ - Security reports go through GitHub's private vulnerability reporting (`SECURITY.md`).
54
+ - The 0.18 line is now cut from the `release/0.18` branch. `AGENTS.md` and
55
+ `misc/publish-guard.js` say so.
56
+
57
+ ## Deferred
58
+
59
+ Nothing new. Later work ships on the current line's next release.
@@ -41,8 +41,8 @@ This does four things:
41
41
  1. It starts ClickHouse in a container bound to `127.0.0.1:8123`, so only this machine
42
42
  can reach it. The data lives in a named volume.
43
43
  2. It creates two users. `memhouse_root` administers the house. A **member** named after
44
- your OS user (`polat`, say) ships your sessions, and holds one grant: `mem.polat_*`.
45
- Your rooms are `mem.polat_sessions`, `mem.polat_messages`, `mem.polat_tool_calls`, and
44
+ your OS user (`sam`, say) ships your sessions, and holds one grant: `mem.sam_*`.
45
+ Your rooms are `mem.sam_sessions`, `mem.sam_messages`, `mem.sam_tool_calls`, and
46
46
  so on.
47
47
  3. It writes both credentials to `~/.memhouse/env`, mode 600. A local house is the one
48
48
  case where the admin credential is kept, so that `memhouse invite` works later with
package/editors/claude.js CHANGED
@@ -7,7 +7,7 @@ const HOME = os.homedir();
7
7
 
8
8
  // Discover all Claude Code config roots — ports memory-house agent-sync's
9
9
  // discover_roots("claude-code"). Claude sessions don't live only under ~/.claude:
10
- // every CLAUDE_CONFIG_DIR (e.g. Kommander playbook installs under
10
+ // every CLAUDE_CONFIG_DIR (e.g. playbook installs under
11
11
  // ~/.claude-playbooks/<name>[/playbook]) has its own projects/ dir. A dir qualifies
12
12
  // as a root if it holds a projects/ subdir or a history.jsonl. Deduped by realpath.
13
13
  function discoverClaudeRoots() {
@@ -60,7 +60,7 @@ function getChats() {
60
60
  const dir = path.join(PROJECTS_DIR, projDir);
61
61
  try { if (!fs.statSync(dir).isDirectory()) continue; } catch { continue; }
62
62
 
63
- // Decode folder path from dir name (e.g. -Users-fka-Code-foo -> /Users/fka/Code/foo)
63
+ // Decode folder path from dir name (e.g. -Users-user-Code-foo -> /Users/user/Code/foo)
64
64
  const decodedFolder = projDir.replace(/-/g, '/');
65
65
 
66
66
  // Read sessions-index.json for indexed sessions
@@ -74,13 +74,18 @@ function getChats() {
74
74
  } catch { /* no index */ }
75
75
 
76
76
  // Scan all .jsonl files on disk (some may not be in the index)
77
- let files;
78
- try { files = fs.readdirSync(dir).filter(f => f.endsWith('.jsonl')); } catch { continue; }
77
+ let names;
78
+ try { names = fs.readdirSync(dir); } catch { continue; }
79
+ const files = names.filter(f => f.endsWith('.jsonl'));
80
+ // A session with subagents also has a directory named for it; one without has none, so
81
+ // this listing is how the mtime check below stays free for the sessions it cannot help.
82
+ const sessionDirs = new Set(names.filter(f => !f.endsWith('.jsonl')));
79
83
 
80
84
  for (const file of files) {
81
85
  const sessionId = file.replace('.jsonl', '');
82
86
  const fullPath = path.join(dir, file);
83
87
  const entry = indexed.get(sessionId);
88
+ const subTouched = sessionDirs.has(sessionId) ? newestSubagentMtime(path.join(dir, sessionId, 'subagents')) : 0;
84
89
 
85
90
  if (entry) {
86
91
  // Use index metadata
@@ -89,7 +94,7 @@ function getChats() {
89
94
  composerId: sessionId,
90
95
  name: cleanPrompt(entry.firstPrompt),
91
96
  createdAt: entry.created ? new Date(entry.created).getTime() : null,
92
- lastUpdatedAt: entry.modified ? new Date(entry.modified).getTime() : null,
97
+ lastUpdatedAt: latest(entry.modified ? new Date(entry.modified).getTime() : null, subTouched),
93
98
  mode: 'claude',
94
99
  folder: entry.projectPath || decodedFolder,
95
100
  encrypted: false,
@@ -107,7 +112,7 @@ function getChats() {
107
112
  composerId: sessionId,
108
113
  name: meta.firstPrompt ? cleanPrompt(meta.firstPrompt) : null,
109
114
  createdAt: meta.timestamp || stat.birthtime.getTime(),
110
- lastUpdatedAt: stat.mtime.getTime(),
115
+ lastUpdatedAt: latest(stat.mtime.getTime(), subTouched),
111
116
  mode: 'claude',
112
117
  folder: meta.cwd || decodedFolder,
113
118
  encrypted: false,
@@ -195,7 +200,7 @@ function cleanPrompt(prompt) {
195
200
  // Parse one Claude session .jsonl into the adapter's message shape. When
196
201
  // `isSubagent` is set, each message is tagged `[subagent]` so folded subagent turns
197
202
  // are clearly attributed in the transcript.
198
- function parseSessionFile(filePath, isSubagent, agent = null) {
203
+ function parseSessionFile(filePath, isSubagent, agent = null, info = null) {
199
204
  const messages = [];
200
205
  let lines;
201
206
  try { lines = fs.readFileSync(filePath, 'utf-8').split('\n').filter(Boolean); } catch { return messages; }
@@ -205,7 +210,9 @@ function parseSessionFile(filePath, isSubagent, agent = null) {
205
210
  // cited, or told apart from a sibling's. An agent looking for the fork as its own session
206
211
  // concluded it was never shipped. Every turn now carries the agent's id, the description
207
212
  // the parent gave it, and its position within that subagent — in `extra`, which is not
208
- // part of the line hash, so nothing already shipped is re-shipped for this.
213
+ // part of the line hash. The position is not only a label any more: the shipper numbers
214
+ // a subagent's rows from it (seq = block + turn, see subagentSeq in ship.js), so `turn`
215
+ // must count exactly the turns this parse keeps, in order.
209
216
  let turn = 0;
210
217
  const stamp = (m) => { if (agent) { m._agent = { ...agent, turn: turn++ }; } return m; };
211
218
 
@@ -228,6 +235,7 @@ function parseSessionFile(filePath, isSubagent, agent = null) {
228
235
  // time simply omit it.
229
236
  const _ts = obj.timestamp ? Date.parse(obj.timestamp) : undefined;
230
237
  const at = Number.isFinite(_ts) ? _ts : undefined;
238
+ if (info && info.firstTs === undefined && at !== undefined) info.firstTs = at;
231
239
 
232
240
  if (obj.type === 'user' && obj.message) {
233
241
  const content = extractContent(obj.message.content);
@@ -249,6 +257,10 @@ function parseSessionFile(filePath, isSubagent, agent = null) {
249
257
  return messages;
250
258
  }
251
259
 
260
+ // How long a subagent transcript without a single timestamped line may hold back its
261
+ // session (see getMessages). A real one gets its first line within seconds.
262
+ const SUBAGENT_STAMP_GRACE_MS = 10 * 60 * 1000;
263
+
252
264
  function getMessages(chat) {
253
265
  const filePath = chat._fullPath;
254
266
  if (!filePath || !fs.existsSync(filePath)) return [];
@@ -262,30 +274,76 @@ function getMessages(chat) {
262
274
  const subagentsDir = path.join(filePath.replace(/\.jsonl$/, ''), 'subagents');
263
275
  const subFiles = subagentFiles(subagentsDir);
264
276
  const named = subFiles.length ? subagentNames(filePath) : new Map();
277
+ const folded = [];
278
+ let unstamped = null;
265
279
  for (const { file, workflow } of subFiles) {
266
280
  const id = path.basename(file).replace(/^agent-/, '').replace(/\.jsonl$/, '');
267
281
  const meta = named.get(id) || {};
268
282
  const agent = { id, description: meta.description || '', type: meta.type || (workflow ? 'workflow-subagent' : ''), file: path.basename(file) };
269
283
  if (workflow) agent.workflow = workflow;
270
- const turns = parseSessionFile(file, true, agent);
284
+ const info = {};
285
+ const turns = parseSessionFile(file, true, agent, info);
271
286
  // A fork the parent never named (a Workflow-run agent, or a parent from before the
272
287
  // ids were reported) is described by its own first prompt — what it was told to do.
273
288
  if (!agent.description) {
274
289
  const first = turns.find((m) => m.role === 'user');
275
290
  if (first) { const d = String(first.content).replace(/^\[subagent\] /, '').replace(/\s+/g, ' ').trim().slice(0, 160); for (const m of turns) m._agent.description = d; }
276
291
  }
277
- messages.push(...turns);
292
+ folded.push({ key: path.relative(subagentsDir, file), firstTs: info.firstTs, turns });
293
+ // A subagent caught before its first line is written has no start time, so it ranks
294
+ // last — and when its first line lands with an EARLIER timestamp than a sibling's, it
295
+ // moves, renumbers every sibling after it, and the session forks and re-ships whole
296
+ // (SUB-1). So a fresh one defers the whole session to the next pass, when it has a
297
+ // time. Bounded by age: a file that never gets a timestamped line stops deferring after
298
+ // SUBAGENT_STAMP_GRACE_MS and ranks last, as before.
299
+ if (info.firstTs === undefined && !unstamped) {
300
+ let age = Infinity;
301
+ try { age = Date.now() - fs.statSync(file).mtimeMs; } catch { /* gone: nothing to wait for */ }
302
+ if (age < SUBAGENT_STAMP_GRACE_MS) unstamped = path.relative(subagentsDir, file);
303
+ }
304
+ }
305
+ // Each subagent gets a SLOT, and the shipper numbers its rows from that slot rather than
306
+ // from where they happen to land in this array (see subagentSeq in ship.js). The slot is
307
+ // the subagent's rank by when it started — the timestamp on its transcript's first line —
308
+ // because that is the one order a new subagent cannot disturb: it started after every
309
+ // subagent already on disk, so it ranks last. The file-name order this used to fold in
310
+ // does not have that property: agent ids are random, so a new fork sorted into the middle
311
+ // and pushed every later subagent's rows down, and a session with subagents re-shipped
312
+ // whole on every pass (one 6,120-line session was stored 60 times in a day). A file with
313
+ // no timestamp yet ranks after every one that has one; the path breaks ties.
314
+ folded.sort((a, b) => (a.firstTs === undefined) - (b.firstTs === undefined)
315
+ || (a.firstTs || 0) - (b.firstTs || 0) || (a.key < b.key ? -1 : a.key > b.key ? 1 : 0));
316
+ folded.forEach(({ turns }, slot) => { for (const m of turns) m._agent.slot = slot; messages.push(...turns); });
317
+ if (unstamped) {
318
+ Object.defineProperty(messages, '_defer', { value: `subagent ${unstamped} has no timestamped line yet`, enumerable: false });
278
319
  }
279
320
 
280
321
  return messages;
281
322
  }
282
323
 
324
+ /**
325
+ * When a session was last written, counting its subagents. The shipper's incremental skip
326
+ * compares `lastUpdatedAt` with what the house recorded, and a subagent writes only its own
327
+ * file: a background agent that runs on after its parent goes quiet left the parent's mtime
328
+ * (and the index's `modified`) where they were, so its turns shipped at the parent's next
329
+ * write, or never. Stat calls only, and only for a session that has a directory.
330
+ */
331
+ function newestSubagentMtime(subagentsDir) {
332
+ let newest = 0;
333
+ for (const { file } of subagentFiles(subagentsDir)) {
334
+ try { newest = Math.max(newest, fs.statSync(file).mtimeMs); } catch { /* vanished mid-pass */ }
335
+ }
336
+ return Math.floor(newest);
337
+ }
338
+ function latest(a, b) { return b && (!a || b > a) ? b : a; }
339
+
283
340
  /**
284
341
  * Every subagent transcript under a session, wherever Claude Code put it:
285
342
  * <session>/subagents/agent-<id>.jsonl — Agent tool forks
286
343
  * <session>/subagents/workflows/wf_<id>/agent-<id>.jsonl — Workflow-run agents
287
- * journal.jsonl and *.meta.json are bookkeeping, not transcripts. Sorted for a stable
288
- * fold order. On real data the workflow layer held 100 of 165 forks and shipped none.
344
+ * journal.jsonl and *.meta.json are bookkeeping, not transcripts. Sorted so the listing is
345
+ * deterministic; the fold ORDER is set by start time in getMessages, not by this.
346
+ * On real data the workflow layer held 100 of 165 forks and shipped none.
289
347
  */
290
348
  function subagentFiles(subagentsDir) {
291
349
  const out = [];
@@ -13,7 +13,7 @@ const name = 'cursor-agent';
13
13
 
14
14
  /**
15
15
  * Decode project directory name back to folder path.
16
- * e.g. "Users-fka-Code-Wapple" → "/Users/fka/Code/Wapple"
16
+ * e.g. "Users-user-Code-myapp" → "/Users/user/Code/myapp"
17
17
  */
18
18
  function decodeProjectDir(dirName) {
19
19
  // The encoding replaces "/" with "-". We reconstruct by prepending "/"
@@ -46,8 +46,8 @@ Trimmed from a run on a Linux VM with Docker, ClickHouse 26.7 and memhouse 0.18.
46
46
  == 1. a house: ClickHouse in a container, bound to loopback, plus the install
47
47
  ✓ ClickHouse starting via docker on http://localhost:8123 (loopback only)
48
48
  ✓ ClickHouse ready
49
- ✓ member 'polat' holds mem.polat_* and nothing else; 'memhouse_root' administers the house
50
- ✓ rooms for 'polat': polat_sessions, polat_messages, polat_tool_calls
49
+ ✓ member 'sam' holds mem.sam_* and nothing else; 'memhouse_root' administers the house
50
+ ✓ rooms for 'sam': sam_sessions, sam_messages, sam_tool_calls
51
51
  ✓ installed
52
52
  ✓ shipper started in the background (pid 617049) — loading your history now
53
53
 
@@ -55,10 +55,10 @@ Trimmed from a run on a Linux VM with Docker, ClickHouse 26.7 and memhouse 0.18.
55
55
  [memhouse] shipped 2 sessions (0 skipped) → 6 msg rows, 2 tool rows in 0.7s
56
56
 
57
57
  == 3. what the house holds
58
- ✓ connected: http://localhost:8123 / mem as polat
58
+ ✓ connected: http://localhost:8123 / mem as sam
59
59
  ✓ house: 2 sessions, 6 messages (freshest ingest 2026-10-07 21:08:06 UTC)
60
60
  ✓ fleet: 1 active writer(s)
61
- polat@testbed-7edeca38 0.18.9 last ship 0m ago
61
+ sam@linux-vm-1f2e3d4c 0.18.9 last ship 0m ago
62
62
  ✓ shipper: running — daemon (pid 617049)
63
63
 
64
64
  == 4. find a past conversation
@@ -74,10 +74,10 @@ Trimmed from a run against ClickHouse 26.7.1.1315 with memhouse 0.18.9 (the test
74
74
  credential from: MEMHOUSE_ADMIN_* environment
75
75
  may create users: yes
76
76
 
77
- == 2. install: build member 'polat' with the admin credential, then verify as the member
78
- ✓ created ClickHouse user 'polat'
79
- ✓ granted 'polat' mem.polat_* — their rooms and nothing else; async_insert pinned
80
- ✓ verified as 'polat' with the member credential
77
+ == 2. install: build member 'sam' with the admin credential, then verify as the member
78
+ ✓ created ClickHouse user 'sam'
79
+ ✓ granted 'sam' mem.sam_* — their rooms and nothing else; async_insert pinned
80
+ ✓ verified as 'sam' with the member credential
81
81
  ✓ installed
82
82
  the admin credential was used for this install and not saved. To invite later:
83
83
  with-secret MEMHOUSE_ADMIN_PASSWORD=<reference> -- memhouse invite <name> --url … --admin-user admin
@@ -87,7 +87,7 @@ Trimmed from a run against ClickHouse 26.7.1.1315 with memhouse 0.18.9 (the test
87
87
  [memhouse] shipped 2 sessions (0 skipped) → 6 msg rows, 2 tool rows in 0.2s
88
88
 
89
89
  == 4. whoami: the credential this machine keeps, and what it may do
90
- ✓ polat at https://ch.example.com:8443 — member
90
+ ✓ sam at https://ch.example.com:8443 — member
91
91
  may create users: no
92
92
 
93
93
  == 5. status
@@ -108,8 +108,8 @@ Deletes the example's home. The member and its rooms stay in the house: dropping
108
108
  an administrator's decision. As the admin:
109
109
 
110
110
  ```sql
111
- DROP USER polat;
112
- -- then each of mem.polat_sessions, _messages, _tool_calls, _meta, _events,
111
+ DROP USER sam;
112
+ -- then each of mem.sam_sessions, _messages, _tool_calls, _meta, _events,
113
113
  -- _session_stats, _session_model_stats, _session_tool_stats
114
114
  ```
115
115
 
@@ -134,8 +134,15 @@ Config file: `~/.memhouse/config.json` (`hiddenProjects`, future prefs).
134
134
  - Per-message `ts`: interpolated across the session's `[createdAt, lastUpdatedAt]`
135
135
  (adapters don't expose per-message timestamps; monotonic by `seq`; documented
136
136
  approximation, refine per-adapter later).
137
- - `seq` is the message index within the session (0-based). `tool_calls.idx` is the
138
- call index within the session. `is_subagent` = message content begins with
137
+ - `seq` is the message's place in the session. The session's own turns are dense from 0;
138
+ a folded subagent's turns are numbered from their own block,
139
+ `1,000,000,000 + slot × 100,000 + turn`, where `slot` is the subagent's rank by start
140
+ time. `tool_calls.idx` is laid out the same way (the session's own calls dense from 0,
141
+ a subagent's counted within its block). A row's key therefore never depends on how long
142
+ the parent or any other subagent is, which is what lets a session with subagents ship
143
+ tail-only; positional numbering moved every subagent row whenever the parent grew. A
144
+ parse that does not fit the blocks falls back to positional numbering. Readers order by
145
+ `seq` and count with `count()`; nothing may treat `seq` as dense. `is_subagent` = message content begins with
139
146
  `[subagent]` (the fold marker from the Claude adapter).
140
147
  - Incremental shipping: read existing `(session_id, last_updated_at, message_count)`
141
148
  (FINAL) and skip chats that haven't grown/changed; a full re-ship must remain
@@ -136,7 +136,7 @@ the supervisor's own restart line.
136
136
  ### The gap
137
137
 
138
138
  Nothing tells a running daemon that its files were replaced. `status` prints the version of
139
- the CLI you just typed, not the version the daemon is executing. On macminim a shipper ran
139
+ the CLI you just typed, not the version the daemon is executing. On one machine a shipper ran
140
140
  **1d16h** out of a directory that had been moved.
141
141
 
142
142
  ### Why not `postinstall`
@@ -9,7 +9,7 @@
9
9
  //
10
10
  // SCOPE IS THE WHOLE POINT. A grant is `GRANT <privileges> ON <scope> TO <user>`, and a
11
11
  // privilege only means what its scope allows: an ordinary member holds CREATE DATABASE
12
- // inside `ON polat.*`, which lets them do nothing whatsoever about creating a NEW
12
+ // inside `ON sam.*`, which lets them do nothing whatsoever about creating a NEW
13
13
  // database. Matching the privilege name anywhere in the text — which an earlier draft of
14
14
  // this did — reads that member as a server administrator. So parse the line, keep the
15
15
  // scope, and only count a privilege that is granted server-wide.
@@ -1,8 +1,8 @@
1
1
  // Convert a pre-one-layout house to the one layout — the plan, as SQL, pure.
2
2
  //
3
- // Before 0.18 every member owned a DATABASE named for them: `polat.messages`,
4
- // `polat.house_meta`, granted ALL ON polat.*. The one layout puts everyone in one database
5
- // as `mem.polat_messages`, `mem.polat_meta`, with one wildcard grant and nothing on the
3
+ // Before 0.18 every member owned a DATABASE named for them: `sam.messages`,
4
+ // `sam.house_meta`, granted ALL ON sam.*. The one layout puts everyone in one database
5
+ // as `mem.sam_messages`, `mem.sam_meta`, with one wildcard grant and nothing on the
6
6
  // database itself. Moving is a RENAME per table — metadata only, nothing copied — plus the
7
7
  // grant swap, and shares re-expressed on the new names. The 0.17.1 dashboard views are
8
8
  // dropped: in the one layout the shipper fills the stat tables (a refreshable view needs
@@ -79,6 +79,6 @@ the retained epochs are the only place that content still exists.
79
79
  when the user may want to dig deeper.
80
80
  - Other users' HOUSES (other databases on the same server) are readable only when
81
81
  shared with this credential (`/mem:access` on their side). Query a shared house by
82
- qualified names — `polat.messages` — with the same CUR filter. An empty result
82
+ qualified names — `sam.messages` — with the same CUR filter. An empty result
83
83
  means "nothing visible", not "nothing ever happened".
84
84
  - Do not paste credentials from `{{ENV_FILE}}` into responses, commits, or logs.
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "mem",
3
3
  "description": "Conversation memory for AI agents over the memhouse typed ClickHouse store: recall past sessions and answer from them, run analytics, check the house, bring people in, and administer the server. Skills install as /mem:recall, /mem:sql, /mem:house, /mem:access, and /mem:admin.",
4
- "version": "0.18.10",
4
+ "version": "0.18.12",
5
5
  "author": {
6
6
  "name": "Ramazan Polat"
7
7
  },
@@ -73,8 +73,8 @@ server, not by discipline.
73
73
 
74
74
  **The `sed` is what makes every query below work in both layouts** — write room names
75
75
  bare and it supplies the prefix when there is one. It rewrites only after `FROM`/`JOIN`,
76
- so a needle like `LIKE '%messages%'` is untouched, and `FROM yigit_messages` is left
77
- alone — the room name must follow whitespace, and there is none inside `yigit_messages`.
76
+ so a needle like `LIKE '%messages%'` is untouched, and `FROM bob_messages` is left
77
+ alone — the room name must follow whitespace, and there is none inside `bob_messages`.
78
78
  With no prefix set it expands to nothing and the query passes through verbatim.
79
79
 
80
80
  It is spelled with `[[:space:]]` classes rather than `\b` because **BSD `sed` does not
@@ -84,7 +84,7 @@ which is the right way round — but it fails.
84
84
 
85
85
  **Reading a housemate's memory.** A share is `GRANT SELECT ON mem.<them>_*` — their
86
86
  rooms, not the database. `SHOW TABLES` lists exactly what you may read: your own rooms plus
87
- any shared with you. Name a housemate's rooms in full (`FROM yigit_messages`); the `q()`
87
+ any shared with you. Name a housemate's rooms in full (`FROM bob_messages`); the `q()`
88
88
  rewrite leaves an already-prefixed name alone, since applying yours would point at the
89
89
  wrong rooms. Say whose memory an answer came from.
90
90
 
@@ -101,23 +101,35 @@ session finds nothing — look inside the parent:
101
101
  ```sql
102
102
  -- the subagents a session spawned
103
103
  SELECT toString(extra.agent.id) AS agent, any(toString(extra.agent.description)) AS asked_to, count() AS turns
104
- FROM messages WHERE session_id = '<sid>' AND is_subagent GROUP BY agent -- toString: a JSON path cannot be a GROUP BY key
104
+ FROM messages FINAL
105
+ WHERE session_id = '<sid>' AND user_id = '<uid>' AND is_subagent
106
+ AND (origin != 'ship' OR epoch = (SELECT max(epoch) FROM messages
107
+ WHERE session_id = '<sid>' AND user_id = '<uid>' AND origin = 'ship'))
108
+ GROUP BY agent -- toString: a JSON path cannot be a GROUP BY key
105
109
  -- one subagent's transcript, in order
106
- SELECT seq, role, substring(text, 1, 2000) FROM messages
107
- WHERE session_id = '<sid>' AND extra.agent.id = '<agent>' ORDER BY seq
110
+ SELECT seq, role, substring(text, 1, 2000) AS text FROM messages FINAL
111
+ WHERE session_id = '<sid>' AND user_id = '<uid>' AND toString(extra.agent.id) = '<agent>'
112
+ AND (origin != 'ship' OR epoch = (SELECT max(epoch) FROM messages
113
+ WHERE session_id = '<sid>' AND user_id = '<uid>' AND origin = 'ship'))
114
+ ORDER BY seq
108
115
  ```
109
116
 
117
+ Both carry the current-epoch filter (trap 1 below). A session whose subagents were
118
+ re-parsed holds every parse, so an unfiltered count doubles each subagent's turns, and an
119
+ unfiltered transcript repeats them. `<uid>` is the session's `user_id`, from whatever
120
+ query found it (trap 2).
121
+
110
122
  **`messages`** — one row per turn.
111
123
 
112
124
  | column | meaning |
113
125
  |---|---|
114
126
  | `session_id` | `<source>:<adapter-local id>`, e.g. `claude-code:6b1f…` |
115
- | `seq` | position in the session, from 0 |
127
+ | `seq` | order within the session. The session's own turns are `0, 1, 2, …`; a subagent's turns sit in their own block, `1,000,000,000 + slot × 100,000 + turn` (slot = the subagent's rank by start time), so `seq` is **sparse** — count rows with `count()`, never `max(seq) + 1` |
116
128
  | `source` | editor: `claude-code`, `codex`, `cursor`, … |
117
- | `host` | the machine, `<hostname>-<8 hex>` — random per install, not derived |
129
+ | `host` | the machine, `<hostname>-<8 hex>` — the 8 hex are derived from the OS machine id (random only where none can be read), so a reinstall keeps it and same-named machines differ |
118
130
  | `ts` | when the turn happened (`DateTime64(3,'UTC')`) |
119
131
  | `role` | `user`, `assistant`, `system` |
120
- | `is_subagent` | 1 for a turn that a Claude Code **subagent** (a fork) produced; folded into the parent session after the parent's own turns. Subagents are never separate sessions. |
132
+ | `is_subagent` | 1 for a turn that a Claude Code **subagent** (a fork) produced; folded into the parent session after the parent's own turns, each subagent's turns contiguous and the subagents in the order they started. Subagents are never separate sessions. |
121
133
  | `extra` | JSON. For a subagent turn: `extra.agent.id`, `extra.agent.description` (what the parent asked it to do, else the fork's own first prompt), `extra.agent.type` (`general-purpose`, `workflow-subagent`, …), `extra.agent.turn` (position within that subagent), and `extra.agent.workflow` (`wf_…`) when a Workflow run spawned it. Empty `{}` otherwise, and on rows shipped before 0.18.3. |
122
134
  | `model`, `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_write_tokens` | usage; **empty for several adapters** — see below |
123
135
  | `text` | the turn, **truncated at 50,000 chars** |