memhouse 0.18.10 → 0.18.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +10 -11
- package/CHANGELOG.md +153 -13
- package/README.md +2 -2
- package/SECURITY.md +4 -2
- package/bin/memhouse.js +105 -49
- package/docs/guides/admin-password.md +1 -1
- package/docs/guides/configuration.md +2 -2
- package/docs/guides/install.md +1 -1
- package/docs/guides/invite.md +1 -1
- package/docs/guides/storage.md +10 -7
- package/docs/guides/uninstall.md +4 -2
- package/docs/releases/README.md +2 -0
- package/docs/releases/v0.18.10.md +7 -7
- package/docs/releases/v0.18.11.md +187 -0
- package/docs/releases/v0.18.12.md +59 -0
- package/docs/tutorials/01-first-house.md +2 -2
- package/editors/claude.js +70 -12
- package/editors/cursor-agent.js +1 -1
- package/examples/01-local-one-machine/README.md +4 -4
- package/examples/02-existing-clickhouse/README.md +7 -7
- package/memhouse/DESIGN.md +9 -2
- package/memhouse/RESUME-AND-UPDATE.md +1 -1
- package/memhouse/capabilities.js +1 -1
- package/memhouse/convert.js +3 -3
- package/memhouse/delivery/PROMPT.md +1 -1
- package/memhouse/delivery/plugin/.claude-plugin/plugin.json +1 -1
- package/memhouse/delivery/plugin/reference/HOUSE.md +21 -9
- package/memhouse/delivery/plugin/skills/access/SKILL.md +2 -2
- package/memhouse/delivery/plugin/skills/admin/SKILL.md +3 -3
- package/memhouse/delivery/plugin/skills/house/SKILL.md +2 -2
- package/memhouse/delivery/plugin/skills/recall/SKILL.md +36 -16
- package/memhouse/delivery/plugin/skills/sql/SKILL.md +2 -2
- package/memhouse/flags.js +28 -1
- package/memhouse/host.js +11 -6
- package/memhouse/house/HOUSE.md +6 -6
- package/memhouse/house/house.js +31 -5
- package/memhouse/invite-target.js +37 -0
- package/memhouse/self-update.js +1 -1
- package/memhouse/server/queries.js +11 -6
- package/memhouse/shipper/ship.js +148 -31
- package/package.json +3 -3
- package/public/THIRD_PARTY_NOTICES.txt +2114 -0
- package/ui/README.md +16 -0
- package/ui/vite.config.js +4 -1
|
@@ -0,0 +1,187 @@
|
|
|
1
|
+
# memhouse 0.18.11 — 2026-10-10
|
|
2
|
+
|
|
3
|
+
This release does three things:
|
|
4
|
+
|
|
5
|
+
- **Sessions with subagents stop re-shipping.** A Claude Code session that used subagents
|
|
6
|
+
now ships only its new rows. Before, it re-sent itself whole on every pass, and that was
|
|
7
|
+
most of the house's write load.
|
|
8
|
+
- **A security fix.** `invite` no longer sends a stored credential to whatever `--url` is
|
|
9
|
+
typed on the command line.
|
|
10
|
+
- **The 0.18.10 pre-tag list is finished**, and the gentar test kit is bumped.
|
|
11
|
+
|
|
12
|
+
**No drill was run for this release; waived (2026-10-08) under the maintainer's 0.18.10 ruling. The drill runs before 0.19.0.**
|
|
13
|
+
|
|
14
|
+
## Read this before upgrading
|
|
15
|
+
|
|
16
|
+
**The first pass after the upgrade is a one-time bulk write.** It re-sends, whole and once,
|
|
17
|
+
every stored session that has subagents; after that, those sessions ship only their new
|
|
18
|
+
rows. Two things widen that first pass:
|
|
19
|
+
|
|
20
|
+
- **Sessions whose subagents wrote recently.** A session whose subagent files are newer
|
|
21
|
+
than the house's record of it is read again even if it did not grow, because
|
|
22
|
+
`lastUpdatedAt` now counts subagent writes.
|
|
23
|
+
- **Installs that share session files.** Two installs that ship the same session files
|
|
24
|
+
(two instances on one machine, or a synced directory) must be upgraded together. A
|
|
25
|
+
0.18.10 and a 0.18.11 number the same session differently, and each would fork it back
|
|
26
|
+
on every pass.
|
|
27
|
+
|
|
28
|
+
**Where that write lands, and when to do it (2026-10-08):**
|
|
29
|
+
|
|
30
|
+
- Upgrade together any installs that ship the same session files (above).
|
|
31
|
+
- A house whose data disk sits on degraded storage: back it up first (data and access
|
|
32
|
+
storage), and upgrade its machines only once the disk is off the degraded array or the
|
|
33
|
+
array is repaired.
|
|
34
|
+
- Any house on healthy storage can upgrade normally.
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
memhouse update # then: memhouse status — every line a check mark
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Security
|
|
41
|
+
|
|
42
|
+
### `invite` never sends a stored credential to `--url` (G4)
|
|
43
|
+
|
|
44
|
+
`--url` is the invitee's address, typed on a command line. A typo, a tunnel, or somebody
|
|
45
|
+
else's server received credentials it should never have seen:
|
|
46
|
+
|
|
47
|
+
- the member's own credential, through the reachability probe and the read of its grants;
|
|
48
|
+
- an admin credential the install keeps, through provisioning.
|
|
49
|
+
|
|
50
|
+
The cause: `resolveConfig()` lets a flag outrank the env file, so the "configured house" URL
|
|
51
|
+
the code used *was* `--url`.
|
|
52
|
+
|
|
53
|
+
Now:
|
|
54
|
+
|
|
55
|
+
- **A stored credential goes only to the house it was saved for.** That covers the
|
|
56
|
+
member's own, a `--keep-admin` admin, and a password read from the env file.
|
|
57
|
+
- **`--url` gets an unauthenticated `/ping`**, only to confirm the invitee can reach it.
|
|
58
|
+
- **An admin given for this command** (`--admin-user`, or an exported
|
|
59
|
+
`MEMHOUSE_ADMIN_USER`, with its password from a file, stdin, the environment or a prompt)
|
|
60
|
+
still provisions at `--url`. That is what the person typing the command asked for.
|
|
61
|
+
- **`misc/invite-matrix.sh` points `--url` at a spy server** and asserts that it receives
|
|
62
|
+
nothing but a ping. On 0.18.10 the spy received two authenticated requests.
|
|
63
|
+
|
|
64
|
+
**Action:**
|
|
65
|
+
|
|
66
|
+
- If you ran `memhouse invite … --url <something unusual>` on 0.18.10 or earlier against a
|
|
67
|
+
host you do not control, rotate the member's password with `memhouse passwd`.
|
|
68
|
+
- If an admin credential is kept in the env file and the same applies, rotate that too.
|
|
69
|
+
|
|
70
|
+
## The re-ship fix (#21)
|
|
71
|
+
|
|
72
|
+
Folded subagent rows were numbered by position, after the parent's turns. One new parent
|
|
73
|
+
turn moved every one of them, so the session forked to a new epoch and was re-sent whole.
|
|
74
|
+
Measured on a real house:
|
|
75
|
+
|
|
76
|
+
- 1.07 M message rows written in 24 h, against 10.2 M in total;
|
|
77
|
+
- one 6,120-line session stored 60 times in a day;
|
|
78
|
+
- one member's house at about 19 M message rows for roughly 295 k real ones.
|
|
79
|
+
|
|
80
|
+
What changed:
|
|
81
|
+
|
|
82
|
+
- **Subagent rows get a numbering block of their own:**
|
|
83
|
+
`seq = 1,000,000,000 + slot × 100,000 + turn`, and the same for tool calls' `idx`.
|
|
84
|
+
`slot` is the subagent's rank by start time, so neither the parent growing, a subagent
|
|
85
|
+
growing, nor a new subagent appearing moves a stored row.
|
|
86
|
+
- **A session without subagents is numbered exactly as before.**
|
|
87
|
+
- **`seq` is sparse.** Count rows with `count()`, never `max(seq) + 1`.
|
|
88
|
+
- **`ORDER BY seq` groups subagents.** Readers see the parent first, then each subagent's
|
|
89
|
+
turns together, in the order the subagents started. They used to be in agent-id order,
|
|
90
|
+
which is random.
|
|
91
|
+
- **A parse too big for the blocks falls back to positional numbering:**
|
|
92
|
+
- a parent past 1e9 turns;
|
|
93
|
+
- a subagent past 100,000 turns;
|
|
94
|
+
- more than 32,949 subagents.
|
|
95
|
+
|
|
96
|
+
Positional numbering is correct, and merely re-ships whole as before.
|
|
97
|
+
- **Subagent writes count as session activity.** A subagent that keeps running after its
|
|
98
|
+
parent goes quiet is shipped on the next pass.
|
|
99
|
+
- **A subagent caught before its first line is written holds its session for one pass.**
|
|
100
|
+
It is then ranked by its real start time; it no longer ranks last and moves later. A
|
|
101
|
+
file that never gets a timestamped line stops holding the session after 10 minutes.
|
|
102
|
+
- **Existing copies are not cleaned up.** Superseded epochs already in a house stay;
|
|
103
|
+
memhouse never deletes to make room. Reclaiming that space is a separate operator
|
|
104
|
+
decision, and no such cleanup may run on a house on degraded storage.
|
|
105
|
+
|
|
106
|
+
## Also fixed
|
|
107
|
+
|
|
108
|
+
- **The dashboard no longer goes empty on a quiet house (G11).** The stats tables carry a
|
|
109
|
+
one-day TTL and were refreshed only by passes that shipped something.
|
|
110
|
+
- The shipper now also refreshes them when the newest generation is over an hour old.
|
|
111
|
+
- The dashboard re-resolves its tables every five minutes, falling back to the live
|
|
112
|
+
rollup when a stats table is empty.
|
|
113
|
+
- **The skills find their reference again (G26).** All five pointed at
|
|
114
|
+
`../reference/HOUSE.md`, a path that does not exist in the installed plugin; the file is
|
|
115
|
+
at `../../reference/HOUSE.md`.
|
|
116
|
+
- **`/mem:recall`'s queries run (A2).**
|
|
117
|
+
- "Recent sessions" used a column the room does not have.
|
|
118
|
+
- The search now returns the `hit_seq` that the transcript read needs.
|
|
119
|
+
- The read takes its window by position, so the subagent blocks cannot break it.
|
|
120
|
+
- **`HOUSE.md`'s subagent queries keep to the current parse (A3).** They used to double a
|
|
121
|
+
re-parsed session's subagent turns.
|
|
122
|
+
- **An option that takes no value no longer swallows the next word (A6).**
|
|
123
|
+
`update --no-install yes` ran the install it was told to skip, and `share --revoke bob`
|
|
124
|
+
revoked nobody.
|
|
125
|
+
- **`update --channel <name>` pins only a tag the registry has (A5).** Before, it never
|
|
126
|
+
wrote the pin at all.
|
|
127
|
+
- **`update --no-install` restarts a tarball or nightly install.** That is the command
|
|
128
|
+
every nightly build prints for itself. On exactly those builds it stopped at "no
|
|
129
|
+
automatic update for this install" and exited 1, leaving the old shipper running old
|
|
130
|
+
code. It now restarts the daemons, runs any migration, and checks the schema; it needs
|
|
131
|
+
no channel.
|
|
132
|
+
- **`update --no-install` (A6)** no longer prints "files updated", and it is no longer
|
|
133
|
+
refused on a project-dependency install.
|
|
134
|
+
- **The epoch log line counts rows (A4).**
|
|
135
|
+
- **Docs (A8): the host id is derived from the machine's own id**, as it has been since
|
|
136
|
+
0.18.6, so a reinstall keeps it. Stale prose about timestamps and `extra.turn` is fixed
|
|
137
|
+
too.
|
|
138
|
+
|
|
139
|
+
## License notices
|
|
140
|
+
|
|
141
|
+
**The dashboard now ships the license notices of what it bundles.** The built dashboard
|
|
142
|
+
(`public/`) includes React, chart.js, lucide-react, react-markdown, tailwindcss and every
|
|
143
|
+
other package it bundles. Their licenses (MIT, ISC) require the notice to travel with each
|
|
144
|
+
copy, and minification strips it from the code.
|
|
145
|
+
- `public/THIRD_PARTY_NOTICES.txt` is now generated at build time from the bundle's module
|
|
146
|
+
graph, and ships in the npm package.
|
|
147
|
+
- CI fails a package that lacks it.
|
|
148
|
+
|
|
149
|
+
Earlier published packages carried those copies without the notices.
|
|
150
|
+
|
|
151
|
+
## Test infrastructure
|
|
152
|
+
|
|
153
|
+
- **gentar kit v0.9.3 (#18).** In phase 1, `memhouse-house` is reported UNVERIFIED (bench
|
|
154
|
+
only) instead of being dry-run on the CI host.
|
|
155
|
+
- **The suite tears down only a house it created.** It refuses to deploy over an existing
|
|
156
|
+
house, and a failed teardown fails it.
|
|
157
|
+
- **`misc/skill-recipe.sh` runs the skills' queries verbatim, extracted from the docs.**
|
|
158
|
+
The fixture includes a superseded parse and a subagent block.
|
|
159
|
+
|
|
160
|
+
## Pre-tag list (0.18.10 review): final status
|
|
161
|
+
|
|
162
|
+
| item | where |
|
|
163
|
+
|---|---|
|
|
164
|
+
| A1 version bump | this release |
|
|
165
|
+
| A2 `/mem:recall` queries | #22 |
|
|
166
|
+
| A3 `HOUSE.md` subagent queries, epoch filter | #22 |
|
|
167
|
+
| A4 epoch log count | #21 |
|
|
168
|
+
| A5 `update --channel` validation | #21 |
|
|
169
|
+
| A6 `--no-install` checks; switches swallowing values | #21 |
|
|
170
|
+
| A7 upgrade-cost notes | #21, and this document |
|
|
171
|
+
| A8 stale prose (ts, shrink, `extra` turn, host id) | #21 |
|
|
172
|
+
| G4 `invite --url` and stored credentials (**security**) | #22 |
|
|
173
|
+
| G26 skills' reference path | #22 |
|
|
174
|
+
| G11 idle house's dashboard | #22 |
|
|
175
|
+
| SUB-1 unstamped subagent file | #21 |
|
|
176
|
+
|
|
177
|
+
## Verification
|
|
178
|
+
|
|
179
|
+
- **Unit tests:** `npm test` passes on main, 125 checks.
|
|
180
|
+
- **CI:** green on ClickHouse 25.11 and 26.7.
|
|
181
|
+
- **Matrices in a lab**, on ClickHouse 26.7.4.58 (production's version) and 26.7.1.1315.
|
|
182
|
+
They were run during #22 on main with #21 and #18 merged:
|
|
183
|
+
- skill recipe 18/18;
|
|
184
|
+
- invite 58/58;
|
|
185
|
+
- one-layout 62/62.
|
|
186
|
+
- **Second looks:** a second model reviewed #21 and #22.
|
|
187
|
+
- **Labs:** kept light, because the lab host's RAID was degraded. No live house was touched.
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# memhouse 0.18.12 — 2026-10-29
|
|
2
|
+
|
|
3
|
+
A patch release on the 0.18 line. It does one thing for users:
|
|
4
|
+
|
|
5
|
+
- **The `/mem` skills pre-approve read-only commands only.** Each skill's `allowed-tools`
|
|
6
|
+
now lists the memhouse read commands it teaches, instead of `Bash` as a whole.
|
|
7
|
+
|
|
8
|
+
No drill was run for this patch release. Its one user-facing change is held by
|
|
9
|
+
`misc/skill-allowlist-test.js`, which runs in `npm test`.
|
|
10
|
+
|
|
11
|
+
## Upgrade
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
memhouse update # refreshes the /mem skills wherever they are installed
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Then reload the skills in Claude Code with `/reload-plugins`, or restart the session.
|
|
18
|
+
|
|
19
|
+
If you upgraded with `npm install -g memhouse` instead, run `memhouse plugins install claude`
|
|
20
|
+
once, so the installed skills match the package.
|
|
21
|
+
|
|
22
|
+
Nothing else changes: same house layout, same schema, same daemons.
|
|
23
|
+
|
|
24
|
+
## Changed
|
|
25
|
+
|
|
26
|
+
### Skills: allowed tools restricted to read-only commands
|
|
27
|
+
|
|
28
|
+
Each `/mem` skill's `allowed-tools` names the memhouse read commands it uses:
|
|
29
|
+
|
|
30
|
+
| skill | runs without a prompt |
|
|
31
|
+
|---|---|
|
|
32
|
+
| `/mem:house` | `memhouse instance`, `whoami`, `status`, `stats`, `doctor`, and `share --list` (with or without `--json`) |
|
|
33
|
+
| `/mem:recall` | `memhouse search`, `resume`, `sessions-query`, `status`, `whoami` |
|
|
34
|
+
| `/mem:sql` | `memhouse sessions-query`, `stats`, `status`, `whoami` |
|
|
35
|
+
| `/mem:access` | `memhouse whoami`, `status`, and `share --list` (with or without `--json`) |
|
|
36
|
+
| `/mem:admin` | `memhouse whoami`, `status`, `instance` |
|
|
37
|
+
|
|
38
|
+
Any other command goes through Claude Code's normal permission prompt, as does a read
|
|
39
|
+
command with another command chained to it. Expect a prompt where a skill used to run a
|
|
40
|
+
command without one; approve it as you would any other command.
|
|
41
|
+
|
|
42
|
+
## Repository and tests
|
|
43
|
+
|
|
44
|
+
These do not change the installed package's behaviour:
|
|
45
|
+
|
|
46
|
+
- `misc/skill-allowlist-test.js` keeps every skill's `allowed-tools` to read-only memhouse
|
|
47
|
+
commands. It joins `npm test`.
|
|
48
|
+
- The test tooling's deploy targets (lab address, test machine, engine checkout) are read
|
|
49
|
+
from an untracked `deploy/targets.env`. `deploy/targets.example.env` shows the shape.
|
|
50
|
+
- Test fixtures, examples, drill records and docs use stand-in names, hosts and paths.
|
|
51
|
+
`misc/env-guard-test.js` checks tracked files for real environment values, and it joins
|
|
52
|
+
`npm test`.
|
|
53
|
+
- Security reports go through GitHub's private vulnerability reporting (`SECURITY.md`).
|
|
54
|
+
- The 0.18 line is now cut from the `release/0.18` branch. `AGENTS.md` and
|
|
55
|
+
`misc/publish-guard.js` say so.
|
|
56
|
+
|
|
57
|
+
## Deferred
|
|
58
|
+
|
|
59
|
+
Nothing new. Later work ships on the current line's next release.
|
|
@@ -41,8 +41,8 @@ This does four things:
|
|
|
41
41
|
1. It starts ClickHouse in a container bound to `127.0.0.1:8123`, so only this machine
|
|
42
42
|
can reach it. The data lives in a named volume.
|
|
43
43
|
2. It creates two users. `memhouse_root` administers the house. A **member** named after
|
|
44
|
-
your OS user (`
|
|
45
|
-
Your rooms are `mem.
|
|
44
|
+
your OS user (`sam`, say) ships your sessions, and holds one grant: `mem.sam_*`.
|
|
45
|
+
Your rooms are `mem.sam_sessions`, `mem.sam_messages`, `mem.sam_tool_calls`, and
|
|
46
46
|
so on.
|
|
47
47
|
3. It writes both credentials to `~/.memhouse/env`, mode 600. A local house is the one
|
|
48
48
|
case where the admin credential is kept, so that `memhouse invite` works later with
|
package/editors/claude.js
CHANGED
|
@@ -7,7 +7,7 @@ const HOME = os.homedir();
|
|
|
7
7
|
|
|
8
8
|
// Discover all Claude Code config roots — ports memory-house agent-sync's
|
|
9
9
|
// discover_roots("claude-code"). Claude sessions don't live only under ~/.claude:
|
|
10
|
-
// every CLAUDE_CONFIG_DIR (e.g.
|
|
10
|
+
// every CLAUDE_CONFIG_DIR (e.g. playbook installs under
|
|
11
11
|
// ~/.claude-playbooks/<name>[/playbook]) has its own projects/ dir. A dir qualifies
|
|
12
12
|
// as a root if it holds a projects/ subdir or a history.jsonl. Deduped by realpath.
|
|
13
13
|
function discoverClaudeRoots() {
|
|
@@ -60,7 +60,7 @@ function getChats() {
|
|
|
60
60
|
const dir = path.join(PROJECTS_DIR, projDir);
|
|
61
61
|
try { if (!fs.statSync(dir).isDirectory()) continue; } catch { continue; }
|
|
62
62
|
|
|
63
|
-
// Decode folder path from dir name (e.g. -Users-
|
|
63
|
+
// Decode folder path from dir name (e.g. -Users-user-Code-foo -> /Users/user/Code/foo)
|
|
64
64
|
const decodedFolder = projDir.replace(/-/g, '/');
|
|
65
65
|
|
|
66
66
|
// Read sessions-index.json for indexed sessions
|
|
@@ -74,13 +74,18 @@ function getChats() {
|
|
|
74
74
|
} catch { /* no index */ }
|
|
75
75
|
|
|
76
76
|
// Scan all .jsonl files on disk (some may not be in the index)
|
|
77
|
-
let
|
|
78
|
-
try {
|
|
77
|
+
let names;
|
|
78
|
+
try { names = fs.readdirSync(dir); } catch { continue; }
|
|
79
|
+
const files = names.filter(f => f.endsWith('.jsonl'));
|
|
80
|
+
// A session with subagents also has a directory named for it; one without has none, so
|
|
81
|
+
// this listing is how the mtime check below stays free for the sessions it cannot help.
|
|
82
|
+
const sessionDirs = new Set(names.filter(f => !f.endsWith('.jsonl')));
|
|
79
83
|
|
|
80
84
|
for (const file of files) {
|
|
81
85
|
const sessionId = file.replace('.jsonl', '');
|
|
82
86
|
const fullPath = path.join(dir, file);
|
|
83
87
|
const entry = indexed.get(sessionId);
|
|
88
|
+
const subTouched = sessionDirs.has(sessionId) ? newestSubagentMtime(path.join(dir, sessionId, 'subagents')) : 0;
|
|
84
89
|
|
|
85
90
|
if (entry) {
|
|
86
91
|
// Use index metadata
|
|
@@ -89,7 +94,7 @@ function getChats() {
|
|
|
89
94
|
composerId: sessionId,
|
|
90
95
|
name: cleanPrompt(entry.firstPrompt),
|
|
91
96
|
createdAt: entry.created ? new Date(entry.created).getTime() : null,
|
|
92
|
-
lastUpdatedAt: entry.modified ? new Date(entry.modified).getTime() : null,
|
|
97
|
+
lastUpdatedAt: latest(entry.modified ? new Date(entry.modified).getTime() : null, subTouched),
|
|
93
98
|
mode: 'claude',
|
|
94
99
|
folder: entry.projectPath || decodedFolder,
|
|
95
100
|
encrypted: false,
|
|
@@ -107,7 +112,7 @@ function getChats() {
|
|
|
107
112
|
composerId: sessionId,
|
|
108
113
|
name: meta.firstPrompt ? cleanPrompt(meta.firstPrompt) : null,
|
|
109
114
|
createdAt: meta.timestamp || stat.birthtime.getTime(),
|
|
110
|
-
lastUpdatedAt: stat.mtime.getTime(),
|
|
115
|
+
lastUpdatedAt: latest(stat.mtime.getTime(), subTouched),
|
|
111
116
|
mode: 'claude',
|
|
112
117
|
folder: meta.cwd || decodedFolder,
|
|
113
118
|
encrypted: false,
|
|
@@ -195,7 +200,7 @@ function cleanPrompt(prompt) {
|
|
|
195
200
|
// Parse one Claude session .jsonl into the adapter's message shape. When
|
|
196
201
|
// `isSubagent` is set, each message is tagged `[subagent]` so folded subagent turns
|
|
197
202
|
// are clearly attributed in the transcript.
|
|
198
|
-
function parseSessionFile(filePath, isSubagent, agent = null) {
|
|
203
|
+
function parseSessionFile(filePath, isSubagent, agent = null, info = null) {
|
|
199
204
|
const messages = [];
|
|
200
205
|
let lines;
|
|
201
206
|
try { lines = fs.readFileSync(filePath, 'utf-8').split('\n').filter(Boolean); } catch { return messages; }
|
|
@@ -205,7 +210,9 @@ function parseSessionFile(filePath, isSubagent, agent = null) {
|
|
|
205
210
|
// cited, or told apart from a sibling's. An agent looking for the fork as its own session
|
|
206
211
|
// concluded it was never shipped. Every turn now carries the agent's id, the description
|
|
207
212
|
// the parent gave it, and its position within that subagent — in `extra`, which is not
|
|
208
|
-
// part of the line hash
|
|
213
|
+
// part of the line hash. The position is not only a label any more: the shipper numbers
|
|
214
|
+
// a subagent's rows from it (seq = block + turn, see subagentSeq in ship.js), so `turn`
|
|
215
|
+
// must count exactly the turns this parse keeps, in order.
|
|
209
216
|
let turn = 0;
|
|
210
217
|
const stamp = (m) => { if (agent) { m._agent = { ...agent, turn: turn++ }; } return m; };
|
|
211
218
|
|
|
@@ -228,6 +235,7 @@ function parseSessionFile(filePath, isSubagent, agent = null) {
|
|
|
228
235
|
// time simply omit it.
|
|
229
236
|
const _ts = obj.timestamp ? Date.parse(obj.timestamp) : undefined;
|
|
230
237
|
const at = Number.isFinite(_ts) ? _ts : undefined;
|
|
238
|
+
if (info && info.firstTs === undefined && at !== undefined) info.firstTs = at;
|
|
231
239
|
|
|
232
240
|
if (obj.type === 'user' && obj.message) {
|
|
233
241
|
const content = extractContent(obj.message.content);
|
|
@@ -249,6 +257,10 @@ function parseSessionFile(filePath, isSubagent, agent = null) {
|
|
|
249
257
|
return messages;
|
|
250
258
|
}
|
|
251
259
|
|
|
260
|
+
// How long a subagent transcript without a single timestamped line may hold back its
|
|
261
|
+
// session (see getMessages). A real one gets its first line within seconds.
|
|
262
|
+
const SUBAGENT_STAMP_GRACE_MS = 10 * 60 * 1000;
|
|
263
|
+
|
|
252
264
|
function getMessages(chat) {
|
|
253
265
|
const filePath = chat._fullPath;
|
|
254
266
|
if (!filePath || !fs.existsSync(filePath)) return [];
|
|
@@ -262,30 +274,76 @@ function getMessages(chat) {
|
|
|
262
274
|
const subagentsDir = path.join(filePath.replace(/\.jsonl$/, ''), 'subagents');
|
|
263
275
|
const subFiles = subagentFiles(subagentsDir);
|
|
264
276
|
const named = subFiles.length ? subagentNames(filePath) : new Map();
|
|
277
|
+
const folded = [];
|
|
278
|
+
let unstamped = null;
|
|
265
279
|
for (const { file, workflow } of subFiles) {
|
|
266
280
|
const id = path.basename(file).replace(/^agent-/, '').replace(/\.jsonl$/, '');
|
|
267
281
|
const meta = named.get(id) || {};
|
|
268
282
|
const agent = { id, description: meta.description || '', type: meta.type || (workflow ? 'workflow-subagent' : ''), file: path.basename(file) };
|
|
269
283
|
if (workflow) agent.workflow = workflow;
|
|
270
|
-
const
|
|
284
|
+
const info = {};
|
|
285
|
+
const turns = parseSessionFile(file, true, agent, info);
|
|
271
286
|
// A fork the parent never named (a Workflow-run agent, or a parent from before the
|
|
272
287
|
// ids were reported) is described by its own first prompt — what it was told to do.
|
|
273
288
|
if (!agent.description) {
|
|
274
289
|
const first = turns.find((m) => m.role === 'user');
|
|
275
290
|
if (first) { const d = String(first.content).replace(/^\[subagent\] /, '').replace(/\s+/g, ' ').trim().slice(0, 160); for (const m of turns) m._agent.description = d; }
|
|
276
291
|
}
|
|
277
|
-
|
|
292
|
+
folded.push({ key: path.relative(subagentsDir, file), firstTs: info.firstTs, turns });
|
|
293
|
+
// A subagent caught before its first line is written has no start time, so it ranks
|
|
294
|
+
// last — and when its first line lands with an EARLIER timestamp than a sibling's, it
|
|
295
|
+
// moves, renumbers every sibling after it, and the session forks and re-ships whole
|
|
296
|
+
// (SUB-1). So a fresh one defers the whole session to the next pass, when it has a
|
|
297
|
+
// time. Bounded by age: a file that never gets a timestamped line stops deferring after
|
|
298
|
+
// SUBAGENT_STAMP_GRACE_MS and ranks last, as before.
|
|
299
|
+
if (info.firstTs === undefined && !unstamped) {
|
|
300
|
+
let age = Infinity;
|
|
301
|
+
try { age = Date.now() - fs.statSync(file).mtimeMs; } catch { /* gone: nothing to wait for */ }
|
|
302
|
+
if (age < SUBAGENT_STAMP_GRACE_MS) unstamped = path.relative(subagentsDir, file);
|
|
303
|
+
}
|
|
304
|
+
}
|
|
305
|
+
// Each subagent gets a SLOT, and the shipper numbers its rows from that slot rather than
|
|
306
|
+
// from where they happen to land in this array (see subagentSeq in ship.js). The slot is
|
|
307
|
+
// the subagent's rank by when it started — the timestamp on its transcript's first line —
|
|
308
|
+
// because that is the one order a new subagent cannot disturb: it started after every
|
|
309
|
+
// subagent already on disk, so it ranks last. The file-name order this used to fold in
|
|
310
|
+
// does not have that property: agent ids are random, so a new fork sorted into the middle
|
|
311
|
+
// and pushed every later subagent's rows down, and a session with subagents re-shipped
|
|
312
|
+
// whole on every pass (one 6,120-line session was stored 60 times in a day). A file with
|
|
313
|
+
// no timestamp yet ranks after every one that has one; the path breaks ties.
|
|
314
|
+
folded.sort((a, b) => (a.firstTs === undefined) - (b.firstTs === undefined)
|
|
315
|
+
|| (a.firstTs || 0) - (b.firstTs || 0) || (a.key < b.key ? -1 : a.key > b.key ? 1 : 0));
|
|
316
|
+
folded.forEach(({ turns }, slot) => { for (const m of turns) m._agent.slot = slot; messages.push(...turns); });
|
|
317
|
+
if (unstamped) {
|
|
318
|
+
Object.defineProperty(messages, '_defer', { value: `subagent ${unstamped} has no timestamped line yet`, enumerable: false });
|
|
278
319
|
}
|
|
279
320
|
|
|
280
321
|
return messages;
|
|
281
322
|
}
|
|
282
323
|
|
|
324
|
+
/**
|
|
325
|
+
* When a session was last written, counting its subagents. The shipper's incremental skip
|
|
326
|
+
* compares `lastUpdatedAt` with what the house recorded, and a subagent writes only its own
|
|
327
|
+
* file: a background agent that runs on after its parent goes quiet left the parent's mtime
|
|
328
|
+
* (and the index's `modified`) where they were, so its turns shipped at the parent's next
|
|
329
|
+
* write, or never. Stat calls only, and only for a session that has a directory.
|
|
330
|
+
*/
|
|
331
|
+
function newestSubagentMtime(subagentsDir) {
|
|
332
|
+
let newest = 0;
|
|
333
|
+
for (const { file } of subagentFiles(subagentsDir)) {
|
|
334
|
+
try { newest = Math.max(newest, fs.statSync(file).mtimeMs); } catch { /* vanished mid-pass */ }
|
|
335
|
+
}
|
|
336
|
+
return Math.floor(newest);
|
|
337
|
+
}
|
|
338
|
+
function latest(a, b) { return b && (!a || b > a) ? b : a; }
|
|
339
|
+
|
|
283
340
|
/**
|
|
284
341
|
* Every subagent transcript under a session, wherever Claude Code put it:
|
|
285
342
|
* <session>/subagents/agent-<id>.jsonl — Agent tool forks
|
|
286
343
|
* <session>/subagents/workflows/wf_<id>/agent-<id>.jsonl — Workflow-run agents
|
|
287
|
-
* journal.jsonl and *.meta.json are bookkeeping, not transcripts. Sorted
|
|
288
|
-
*
|
|
344
|
+
* journal.jsonl and *.meta.json are bookkeeping, not transcripts. Sorted so the listing is
|
|
345
|
+
* deterministic; the fold ORDER is set by start time in getMessages, not by this.
|
|
346
|
+
* On real data the workflow layer held 100 of 165 forks and shipped none.
|
|
289
347
|
*/
|
|
290
348
|
function subagentFiles(subagentsDir) {
|
|
291
349
|
const out = [];
|
package/editors/cursor-agent.js
CHANGED
|
@@ -13,7 +13,7 @@ const name = 'cursor-agent';
|
|
|
13
13
|
|
|
14
14
|
/**
|
|
15
15
|
* Decode project directory name back to folder path.
|
|
16
|
-
* e.g. "Users-
|
|
16
|
+
* e.g. "Users-user-Code-myapp" → "/Users/user/Code/myapp"
|
|
17
17
|
*/
|
|
18
18
|
function decodeProjectDir(dirName) {
|
|
19
19
|
// The encoding replaces "/" with "-". We reconstruct by prepending "/"
|
|
@@ -46,8 +46,8 @@ Trimmed from a run on a Linux VM with Docker, ClickHouse 26.7 and memhouse 0.18.
|
|
|
46
46
|
== 1. a house: ClickHouse in a container, bound to loopback, plus the install
|
|
47
47
|
✓ ClickHouse starting via docker on http://localhost:8123 (loopback only)
|
|
48
48
|
✓ ClickHouse ready
|
|
49
|
-
✓ member '
|
|
50
|
-
✓ rooms for '
|
|
49
|
+
✓ member 'sam' holds mem.sam_* and nothing else; 'memhouse_root' administers the house
|
|
50
|
+
✓ rooms for 'sam': sam_sessions, sam_messages, sam_tool_calls
|
|
51
51
|
✓ installed
|
|
52
52
|
✓ shipper started in the background (pid 617049) — loading your history now
|
|
53
53
|
|
|
@@ -55,10 +55,10 @@ Trimmed from a run on a Linux VM with Docker, ClickHouse 26.7 and memhouse 0.18.
|
|
|
55
55
|
[memhouse] shipped 2 sessions (0 skipped) → 6 msg rows, 2 tool rows in 0.7s
|
|
56
56
|
|
|
57
57
|
== 3. what the house holds
|
|
58
|
-
✓ connected: http://localhost:8123 / mem as
|
|
58
|
+
✓ connected: http://localhost:8123 / mem as sam
|
|
59
59
|
✓ house: 2 sessions, 6 messages (freshest ingest 2026-10-07 21:08:06 UTC)
|
|
60
60
|
✓ fleet: 1 active writer(s)
|
|
61
|
-
|
|
61
|
+
sam@linux-vm-1f2e3d4c 0.18.9 last ship 0m ago
|
|
62
62
|
✓ shipper: running — daemon (pid 617049)
|
|
63
63
|
|
|
64
64
|
== 4. find a past conversation
|
|
@@ -74,10 +74,10 @@ Trimmed from a run against ClickHouse 26.7.1.1315 with memhouse 0.18.9 (the test
|
|
|
74
74
|
credential from: MEMHOUSE_ADMIN_* environment
|
|
75
75
|
may create users: yes
|
|
76
76
|
|
|
77
|
-
== 2. install: build member '
|
|
78
|
-
✓ created ClickHouse user '
|
|
79
|
-
✓ granted '
|
|
80
|
-
✓ verified as '
|
|
77
|
+
== 2. install: build member 'sam' with the admin credential, then verify as the member
|
|
78
|
+
✓ created ClickHouse user 'sam'
|
|
79
|
+
✓ granted 'sam' mem.sam_* — their rooms and nothing else; async_insert pinned
|
|
80
|
+
✓ verified as 'sam' with the member credential
|
|
81
81
|
✓ installed
|
|
82
82
|
the admin credential was used for this install and not saved. To invite later:
|
|
83
83
|
with-secret MEMHOUSE_ADMIN_PASSWORD=<reference> -- memhouse invite <name> --url … --admin-user admin
|
|
@@ -87,7 +87,7 @@ Trimmed from a run against ClickHouse 26.7.1.1315 with memhouse 0.18.9 (the test
|
|
|
87
87
|
[memhouse] shipped 2 sessions (0 skipped) → 6 msg rows, 2 tool rows in 0.2s
|
|
88
88
|
|
|
89
89
|
== 4. whoami: the credential this machine keeps, and what it may do
|
|
90
|
-
✓
|
|
90
|
+
✓ sam at https://ch.example.com:8443 — member
|
|
91
91
|
may create users: no
|
|
92
92
|
|
|
93
93
|
== 5. status
|
|
@@ -108,8 +108,8 @@ Deletes the example's home. The member and its rooms stay in the house: dropping
|
|
|
108
108
|
an administrator's decision. As the admin:
|
|
109
109
|
|
|
110
110
|
```sql
|
|
111
|
-
DROP USER
|
|
112
|
-
-- then each of mem.
|
|
111
|
+
DROP USER sam;
|
|
112
|
+
-- then each of mem.sam_sessions, _messages, _tool_calls, _meta, _events,
|
|
113
113
|
-- _session_stats, _session_model_stats, _session_tool_stats
|
|
114
114
|
```
|
|
115
115
|
|
package/memhouse/DESIGN.md
CHANGED
|
@@ -134,8 +134,15 @@ Config file: `~/.memhouse/config.json` (`hiddenProjects`, future prefs).
|
|
|
134
134
|
- Per-message `ts`: interpolated across the session's `[createdAt, lastUpdatedAt]`
|
|
135
135
|
(adapters don't expose per-message timestamps; monotonic by `seq`; documented
|
|
136
136
|
approximation, refine per-adapter later).
|
|
137
|
-
- `seq` is the message
|
|
138
|
-
|
|
137
|
+
- `seq` is the message's place in the session. The session's own turns are dense from 0;
|
|
138
|
+
a folded subagent's turns are numbered from their own block,
|
|
139
|
+
`1,000,000,000 + slot × 100,000 + turn`, where `slot` is the subagent's rank by start
|
|
140
|
+
time. `tool_calls.idx` is laid out the same way (the session's own calls dense from 0,
|
|
141
|
+
a subagent's counted within its block). A row's key therefore never depends on how long
|
|
142
|
+
the parent or any other subagent is, which is what lets a session with subagents ship
|
|
143
|
+
tail-only; positional numbering moved every subagent row whenever the parent grew. A
|
|
144
|
+
parse that does not fit the blocks falls back to positional numbering. Readers order by
|
|
145
|
+
`seq` and count with `count()`; nothing may treat `seq` as dense. `is_subagent` = message content begins with
|
|
139
146
|
`[subagent]` (the fold marker from the Claude adapter).
|
|
140
147
|
- Incremental shipping: read existing `(session_id, last_updated_at, message_count)`
|
|
141
148
|
(FINAL) and skip chats that haven't grown/changed; a full re-ship must remain
|
|
@@ -136,7 +136,7 @@ the supervisor's own restart line.
|
|
|
136
136
|
### The gap
|
|
137
137
|
|
|
138
138
|
Nothing tells a running daemon that its files were replaced. `status` prints the version of
|
|
139
|
-
the CLI you just typed, not the version the daemon is executing. On
|
|
139
|
+
the CLI you just typed, not the version the daemon is executing. On one machine a shipper ran
|
|
140
140
|
**1d16h** out of a directory that had been moved.
|
|
141
141
|
|
|
142
142
|
### Why not `postinstall`
|
package/memhouse/capabilities.js
CHANGED
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
//
|
|
10
10
|
// SCOPE IS THE WHOLE POINT. A grant is `GRANT <privileges> ON <scope> TO <user>`, and a
|
|
11
11
|
// privilege only means what its scope allows: an ordinary member holds CREATE DATABASE
|
|
12
|
-
// inside `ON
|
|
12
|
+
// inside `ON sam.*`, which lets them do nothing whatsoever about creating a NEW
|
|
13
13
|
// database. Matching the privilege name anywhere in the text — which an earlier draft of
|
|
14
14
|
// this did — reads that member as a server administrator. So parse the line, keep the
|
|
15
15
|
// scope, and only count a privilege that is granted server-wide.
|
package/memhouse/convert.js
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
// Convert a pre-one-layout house to the one layout — the plan, as SQL, pure.
|
|
2
2
|
//
|
|
3
|
-
// Before 0.18 every member owned a DATABASE named for them: `
|
|
4
|
-
// `
|
|
5
|
-
// as `mem.
|
|
3
|
+
// Before 0.18 every member owned a DATABASE named for them: `sam.messages`,
|
|
4
|
+
// `sam.house_meta`, granted ALL ON sam.*. The one layout puts everyone in one database
|
|
5
|
+
// as `mem.sam_messages`, `mem.sam_meta`, with one wildcard grant and nothing on the
|
|
6
6
|
// database itself. Moving is a RENAME per table — metadata only, nothing copied — plus the
|
|
7
7
|
// grant swap, and shares re-expressed on the new names. The 0.17.1 dashboard views are
|
|
8
8
|
// dropped: in the one layout the shipper fills the stat tables (a refreshable view needs
|
|
@@ -79,6 +79,6 @@ the retained epochs are the only place that content still exists.
|
|
|
79
79
|
when the user may want to dig deeper.
|
|
80
80
|
- Other users' HOUSES (other databases on the same server) are readable only when
|
|
81
81
|
shared with this credential (`/mem:access` on their side). Query a shared house by
|
|
82
|
-
qualified names — `
|
|
82
|
+
qualified names — `sam.messages` — with the same CUR filter. An empty result
|
|
83
83
|
means "nothing visible", not "nothing ever happened".
|
|
84
84
|
- Do not paste credentials from `{{ENV_FILE}}` into responses, commits, or logs.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "mem",
|
|
3
3
|
"description": "Conversation memory for AI agents over the memhouse typed ClickHouse store: recall past sessions and answer from them, run analytics, check the house, bring people in, and administer the server. Skills install as /mem:recall, /mem:sql, /mem:house, /mem:access, and /mem:admin.",
|
|
4
|
-
"version": "0.18.
|
|
4
|
+
"version": "0.18.12",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Ramazan Polat"
|
|
7
7
|
},
|
|
@@ -73,8 +73,8 @@ server, not by discipline.
|
|
|
73
73
|
|
|
74
74
|
**The `sed` is what makes every query below work in both layouts** — write room names
|
|
75
75
|
bare and it supplies the prefix when there is one. It rewrites only after `FROM`/`JOIN`,
|
|
76
|
-
so a needle like `LIKE '%messages%'` is untouched, and `FROM
|
|
77
|
-
alone — the room name must follow whitespace, and there is none inside `
|
|
76
|
+
so a needle like `LIKE '%messages%'` is untouched, and `FROM bob_messages` is left
|
|
77
|
+
alone — the room name must follow whitespace, and there is none inside `bob_messages`.
|
|
78
78
|
With no prefix set it expands to nothing and the query passes through verbatim.
|
|
79
79
|
|
|
80
80
|
It is spelled with `[[:space:]]` classes rather than `\b` because **BSD `sed` does not
|
|
@@ -84,7 +84,7 @@ which is the right way round — but it fails.
|
|
|
84
84
|
|
|
85
85
|
**Reading a housemate's memory.** A share is `GRANT SELECT ON mem.<them>_*` — their
|
|
86
86
|
rooms, not the database. `SHOW TABLES` lists exactly what you may read: your own rooms plus
|
|
87
|
-
any shared with you. Name a housemate's rooms in full (`FROM
|
|
87
|
+
any shared with you. Name a housemate's rooms in full (`FROM bob_messages`); the `q()`
|
|
88
88
|
rewrite leaves an already-prefixed name alone, since applying yours would point at the
|
|
89
89
|
wrong rooms. Say whose memory an answer came from.
|
|
90
90
|
|
|
@@ -101,23 +101,35 @@ session finds nothing — look inside the parent:
|
|
|
101
101
|
```sql
|
|
102
102
|
-- the subagents a session spawned
|
|
103
103
|
SELECT toString(extra.agent.id) AS agent, any(toString(extra.agent.description)) AS asked_to, count() AS turns
|
|
104
|
-
FROM messages
|
|
104
|
+
FROM messages FINAL
|
|
105
|
+
WHERE session_id = '<sid>' AND user_id = '<uid>' AND is_subagent
|
|
106
|
+
AND (origin != 'ship' OR epoch = (SELECT max(epoch) FROM messages
|
|
107
|
+
WHERE session_id = '<sid>' AND user_id = '<uid>' AND origin = 'ship'))
|
|
108
|
+
GROUP BY agent -- toString: a JSON path cannot be a GROUP BY key
|
|
105
109
|
-- one subagent's transcript, in order
|
|
106
|
-
SELECT seq, role, substring(text, 1, 2000) FROM messages
|
|
107
|
-
WHERE session_id = '<sid>' AND extra.agent.id = '<agent>'
|
|
110
|
+
SELECT seq, role, substring(text, 1, 2000) AS text FROM messages FINAL
|
|
111
|
+
WHERE session_id = '<sid>' AND user_id = '<uid>' AND toString(extra.agent.id) = '<agent>'
|
|
112
|
+
AND (origin != 'ship' OR epoch = (SELECT max(epoch) FROM messages
|
|
113
|
+
WHERE session_id = '<sid>' AND user_id = '<uid>' AND origin = 'ship'))
|
|
114
|
+
ORDER BY seq
|
|
108
115
|
```
|
|
109
116
|
|
|
117
|
+
Both carry the current-epoch filter (trap 1 below). A session whose subagents were
|
|
118
|
+
re-parsed holds every parse, so an unfiltered count doubles each subagent's turns, and an
|
|
119
|
+
unfiltered transcript repeats them. `<uid>` is the session's `user_id`, from whatever
|
|
120
|
+
query found it (trap 2).
|
|
121
|
+
|
|
110
122
|
**`messages`** — one row per turn.
|
|
111
123
|
|
|
112
124
|
| column | meaning |
|
|
113
125
|
|---|---|
|
|
114
126
|
| `session_id` | `<source>:<adapter-local id>`, e.g. `claude-code:6b1f…` |
|
|
115
|
-
| `seq` |
|
|
127
|
+
| `seq` | order within the session. The session's own turns are `0, 1, 2, …`; a subagent's turns sit in their own block, `1,000,000,000 + slot × 100,000 + turn` (slot = the subagent's rank by start time), so `seq` is **sparse** — count rows with `count()`, never `max(seq) + 1` |
|
|
116
128
|
| `source` | editor: `claude-code`, `codex`, `cursor`, … |
|
|
117
|
-
| `host` | the machine, `<hostname>-<8 hex>` — random
|
|
129
|
+
| `host` | the machine, `<hostname>-<8 hex>` — the 8 hex are derived from the OS machine id (random only where none can be read), so a reinstall keeps it and same-named machines differ |
|
|
118
130
|
| `ts` | when the turn happened (`DateTime64(3,'UTC')`) |
|
|
119
131
|
| `role` | `user`, `assistant`, `system` |
|
|
120
|
-
| `is_subagent` | 1 for a turn that a Claude Code **subagent** (a fork) produced; folded into the parent session after the parent's own turns. Subagents are never separate sessions. |
|
|
132
|
+
| `is_subagent` | 1 for a turn that a Claude Code **subagent** (a fork) produced; folded into the parent session after the parent's own turns, each subagent's turns contiguous and the subagents in the order they started. Subagents are never separate sessions. |
|
|
121
133
|
| `extra` | JSON. For a subagent turn: `extra.agent.id`, `extra.agent.description` (what the parent asked it to do, else the fork's own first prompt), `extra.agent.type` (`general-purpose`, `workflow-subagent`, …), `extra.agent.turn` (position within that subagent), and `extra.agent.workflow` (`wf_…`) when a Workflow run spawned it. Empty `{}` otherwise, and on rows shipped before 0.18.3. |
|
|
122
134
|
| `model`, `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_write_tokens` | usage; **empty for several adapters** — see below |
|
|
123
135
|
| `text` | the turn, **truncated at 50,000 chars** |
|