memhouse 0.7.1 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/README.md +79 -43
  2. package/bin/memhouse.js +153 -147
  3. package/editors/antigravity.js +7 -15
  4. package/editors/cursor.js +18 -24
  5. package/editors/goose.js +9 -21
  6. package/editors/index.js +19 -70
  7. package/editors/opencode.js +6 -15
  8. package/editors/sqlite.js +74 -0
  9. package/editors/windsurf.js +3 -3
  10. package/editors/zed.js +10 -15
  11. package/memhouse/COMPETITION.md +9 -5
  12. package/memhouse/DESIGN.md +29 -30
  13. package/memhouse/RESUME-AND-UPDATE.md +7 -10
  14. package/memhouse/delivery/AGENT-INSTALL.md +15 -13
  15. package/memhouse/delivery/PROMPT.md +7 -9
  16. package/memhouse/delivery/kernel-install.md +18 -33
  17. package/memhouse/delivery/plugin/skills/search/SKILL.md +25 -38
  18. package/memhouse/delivery/plugin/skills/sessions/SKILL.md +15 -28
  19. package/memhouse/delivery/plugin/skills/sql/SKILL.md +23 -36
  20. package/memhouse/host.js +1 -1
  21. package/memhouse/house/HOUSE.md +74 -0
  22. package/memhouse/house/house.js +170 -0
  23. package/memhouse/{per-member/schema-member.sql.tpl → house/schema.sql.tpl} +8 -11
  24. package/memhouse/resume.js +1 -1
  25. package/memhouse/self-update.js +4 -4
  26. package/memhouse/server/queries.js +8 -2
  27. package/memhouse/shipper/ship.js +30 -37
  28. package/package.json +2 -6
  29. package/memhouse/per-member/INSTALL.md +0 -228
  30. package/memhouse/per-member/PLAN.md +0 -174
  31. package/memhouse/per-member/PROVISIONING.md +0 -177
  32. package/memhouse/per-member/README.md +0 -48
  33. package/memhouse/per-member/SCHEMA.md +0 -120
  34. package/memhouse/per-member/SHARING.md +0 -149
  35. package/memhouse/per-member/provision.js +0 -297
  36. package/memhouse/per-member/rooms.js +0 -252
  37. package/memhouse/per-member/schema-merge.sql.tpl +0 -24
@@ -1,120 +0,0 @@
1
- # SCHEMA — rooms, keys, Merge rooms, the rollup
2
- The room shapes, keys and Merge rooms are
3
- unchanged. What changes is that v4's decision about `sessions_v` — *a saved query, not a
4
- room* — is now the implemented one, after a detour through a stored view that this
5
- version records rather than hides.
6
-
7
- ## `sessions_v` is a saved query, not a room
8
-
9
- v4 said so. Round 1 of the review campaign implemented it as a **stored view** instead —
10
- reactively, to fix a finding that the read layer named the shared rooms — without going
11
- back to this file. Everything that followed was the cost of that override:
12
-
13
- - the view needed a name, and `sessions_v_<m>` sits inside the `^sessions_` pattern the
14
- Merge rooms select on. `all_sessions` silently absorbed every member's aggregate view
15
- alongside their base room: **161 sessions counted as 322**.
16
- - so the name moved to `v_sessions_<m>`, which then needed a reserved prefix so no member
17
- handle could re-create the collision;
18
- - and a fourth grant per member, because a recipient given the three rooms could read rows
19
- and use none of the product's read paths;
20
- - and a fourth object to create, roll forward, check in `doctor`, and eventually drop.
21
-
22
- None of that buys anything the query does not already do. Reverted.
23
-
24
- ### What it is now
25
-
26
- `rooms.js` exposes `sessions_v` as **SQL text** — a parenthesised `SELECT` over the
27
- caller's own rooms. It drops into the same `FROM … AS c` position a view name would have
28
- occupied, so the read layer treats it as just another resolved room name and the
29
- substitution point is one function. (The shared layout, where this key resolved to a
30
- stored view's *name*, is gone; the text form is the only one now.)
31
-
32
- ```sql
33
- FROM {{sessions_v}} AS c -- resolves to: FROM ( SELECT … FROM sessions_alice … ) AS c
34
- ```
35
-
36
- It runs **under the caller's credential**, substituted with the caller's own room names,
37
- so it inherits their grants exactly and raises no view-ownership question at all.
38
-
39
- ### The two properties that had to survive
40
-
41
- - the join is a **LEFT JOIN**, so a session with no messages still appears with zero
42
- aggregates instead of vanishing from every count;
43
- - `join_use_nulls = 1` makes the unmatched message columns NULL, so `coalesce` yields true
44
- zeros rather than counting the placeholder row.
45
-
46
- The view carried the second as a trailing `SETTINGS` clause. **A subquery cannot**, so it
47
- moved to the caller: `READ_SETTINGS = { final: 1, join_use_nulls: 1 }`, applied by the
48
- dashboard client, the shipper and the CLI's raw HTTP alike. A unit check asserts the text
49
- carries no `SETTINGS` of its own, because putting it back would break every read that uses
50
- the rollup.
51
-
52
- ### The cost, stated plainly
53
-
54
- An agent writing ad-hoc SQL has no name to type. `memhouse sessions-query` prints the
55
- rollup resolved for whoever the configured credential is, and the three delivery skills
56
- point at it. That is a worse affordance than a name, and it is the price of the property
57
- that made v4 choose a query: nothing to own, nothing to grant, nothing in a namespace.
58
-
59
- ## Room naming, keys, Merge rooms
60
-
61
- Unchanged from v4. Rooms are type-first — `sessions_<m>`, `messages_<m>`,
62
- `tool_calls_<m>` — so the Merge rooms anchor on a fixed room type and never match
63
- themselves. With the view gone, `^sessions_` now matches **only** base session rooms,
64
- which is what it was always supposed to mean.
65
-
66
- > **OPEN** (carried from v4): is `all_sessions` a Merge over the member `sessions_*`
67
- > rooms, or a saved query over `all_messages`? The first is cheaper; the second guarantees
68
- > the aggregates match the per-member rollup exactly.
69
-
70
- ## Not yet written
71
-
72
- - the kernel capability that provisions a member
73
- - member removal: drop the three rooms, revoke, decide what happens to rooms shared outward
74
-
75
- ## What a member can still learn about the others
76
-
77
- Isolation is a grant that is absent, so no member can read another's rows — verified by
78
- trying every read and write against another member's rooms, plus `system.parts`,
79
- `system.query_log`, `system.processes` and the rest, all of which refuse. Two things do
80
- leak, both structural rather than bugs in the code, and neither is fixed by tightening a
81
- grant:
82
-
83
- **The roster.** A member's handle is part of their table name, and ClickHouse distinguishes
84
- "forbidden" from "absent": `SELECT count() FROM messages_bob` returns `Code: 497
85
- ACCESS_DENIED` (HTTP 500) if bob exists and `Code: 60 UNKNOWN_TABLE` (HTTP 404) if he does
86
- not. So any member can enumerate the membership of the house by guessing handles.
87
- `system.tables` correctly hides the other rooms; this reopens it. `EXISTS TABLE` does not
88
- leak — it answers 497 either way — so it is specifically the natural probe that talks. If
89
- the membership of a house is itself sensitive, the per-member layout is the wrong shape for
90
- it.
91
-
92
- **Volume.** `system.tables.total_rows` and `total_bytes` are readable for the Merge rooms,
93
- and a Merge room spans every member:
94
-
95
- name total_rows total_bytes
96
- all_messages 87 42455 <- the whole house
97
- messages_alice 33 25070 <- what alice can actually read
98
-
99
- So a member can see the house's combined row count and byte volume, and by polling it, when
100
- other members are shipping. Withholding `SELECT` on the Merge rooms is not the fix — it
101
- makes the team room deny rather than narrow, which is the property those rooms exist for.
102
-
103
- ## Why members are not granted ALL on their own rooms
104
-
105
- `GRANT ALL ON <db>.messages_<m>` expands to 45 privileges on 26.7, and one of them is
106
- `CREATE TABLE` **on that name**. A member would then own the name rather than the data:
107
-
108
- DROP TABLE messages_alice;
109
- CREATE TABLE messages_alice AS all_messages ENGINE = Merge('<db>','^messages_');
110
-
111
- `all_messages` is `Merge(db,'^messages_')`, so it now contains a Merge over its own
112
- namespace and every other member's rows are counted twice in the team room — measured, a
113
- house reading alice 6030 / bob 2401 / carol 12 became bob 4802 / carol 24, with no error.
114
- Confidentiality survives (the Merge still narrows by grants) but integrity does not, and a
115
- team-wide number is what the room is for. It is the same failure recorded above under the
116
- `sessions_v` rename, reached on purpose instead of by accident.
117
-
118
- Members therefore get exactly what the shipper uses — `SELECT, INSERT, ALTER UPDATE, ALTER
119
- DELETE, ALTER ADD COLUMN, OPTIMIZE` — and nothing that defines an object. Replacing a room
120
- is the admin's job.
@@ -1,149 +0,0 @@
1
- # SHARING — two tiers
2
-
3
- **A share is three grants**, one per room, and there is no fourth: the rollup is a saved
4
- query over exactly those rooms rather than a stored object, so a recipient who can read
5
- the rooms can run it.
6
-
7
- A share is also **read-only by construction**. `WITH GRANT OPTION` is attached to the
8
- member's `SELECT` grant and not to their `ALL` grant ([PROVISIONING](PROVISIONING.md)),
9
- so alice cannot hand bob mutation or drop rights on her room even by mistake.
10
-
11
- ## Tier 1 — whole room, self-serve
12
-
13
- A member holds `WITH GRANT OPTION` on their own three rooms — there is no fourth object
14
- to hold anything on — so they share without an operator:
15
-
16
- ```sql
17
- GRANT SELECT ON mem.messages_alice TO bob;
18
- GRANT SELECT ON mem.sessions_alice TO bob;
19
- GRANT SELECT ON mem.tool_calls_alice TO bob;
20
- ```
21
-
22
- **Three statements.** The rollup every read path uses is a saved query over exactly these
23
- rooms, run under the recipient's own credential — so a recipient who can read the rooms
24
- can run it, and there is no fourth grant to forget. That is the concrete payoff of the
25
- query over the view, and it is why SCHEMA v4 chose it before the implementation briefly
26
- went the other way.
27
-
28
- `memhouse sessions-query` prints the rollup for whoever is connected, which is what a
29
- recipient pastes into a `FROM (...) AS c` when writing ad-hoc SQL.
30
-
31
- This is the tier per-member rooms exist to enable. Under a row policy the same act needs
32
- `CREATE ROW POLICY`, which members do not have and the kernel withholds from agency
33
- owners by default.
34
-
35
- Bob reading `messages_alice` sees `user_id = 'alice'` on every row. That stamp is
36
- unforgeable **by members**, since `EXECUTE AS` is a grantable privilege they do not hold,
37
- and forgeable by the owner or anyone granted it. Trust boundary is the owner, not the
38
- engine.
39
-
40
- The stamp is also pinned **server-side**: the member settings profile carries
41
- `async_insert = 0 CONST`. A MATERIALIZED column is computed during the insert, and an
42
- async insert flushes outside that context — measured on 25.11.9.34, `async_insert=1`
43
- stores `user_id` as the empty string while a sync insert on the same table stamps
44
- correctly. The shipper always passed `async_insert=0`, so memhouse was never the risk;
45
- anything else holding a member credential was. With the constraint in force a client that
46
- *asks* for `async_insert=1` is refused (`SETTING_CONSTRAINT_VIOLATION`), and one that
47
- simply does not mention the setting is held at 0 — so the insert still succeeds, stamped,
48
- rather than landing unowned. It is the override that is refused, not the write.
49
-
50
- Two limits worth stating plainly. The constraint binds the **ClickHouse user**, not the
51
- database — if you reuse a member's credential for unrelated high-throughput ingestion
52
- elsewhere on that server, async inserts are refused there too. And it only takes effect
53
- once the profile is applied: a house provisioned by an older memhouse keeps the old
54
- profile until any member is provisioned again, which now updates the shared profile for
55
- everyone at once. `doctor`'s blank-`user_id` check remains as the detector for rows
56
- written before that.
57
-
58
- ## What the boundary does NOT hide: the roster
59
-
60
- **Members can discover which other members exist, and this is not fixable here.** A
61
- handle is a table name, and ClickHouse answers differently for a table you may not read
62
- than for one that is not there:
63
-
64
- ```
65
- SELECT count() FROM mem.messages_alice -- exists, not granted → Code 497, Not enough privileges
66
- SELECT count() FROM mem.messages_nobody -- does not exist → Code 60, Unknown table
67
- ```
68
-
69
- So probing names enumerates the membership. Measured on 26.7.3.19 that this is
70
- **ClickHouse's behaviour and not a consequence of our grant set**: a user holding *zero*
71
- grants anywhere in the database gets exactly the same 497/60 split. No grant, revoke or
72
- role arrangement changes it.
73
-
74
- It *is* closable, and the price is the reason it stays open. Suffixing rooms with a random
75
- token (`messages_alice_8f9b2a`) would still match the `^messages_` Merge selectors, and
76
- would still leave the share in this document working — alice grants on the room she owns
77
- and knows, and bob never needs its name. What it costs is that a room name stops being
78
- **derivable**: `roomNames(member)` is pure today, and every client — the shipper, the
79
- skills, `sessions-query`, a person typing SQL — would instead have to look its own name up
80
- first. A real trade, deliberately not taken; not an impossibility.
81
-
82
- What leaks is **membership, not content**: names and the fact of existence, never rows,
83
- counts or sizes. In the setting this is built for — a team who already know they are a
84
- team — that is not a secret. Say so out loud rather than implying the boundary is
85
- tighter than it is.
86
-
87
- Bob's Merge-room reads widen automatically once alice grants him a room — `Merge` reduces
88
- to the caller's actual grants, so nothing else needs changing. Measured end to end on
89
- 25.11: alice's `GRANT` took what bob sees of her rows through `all_messages` from 0 to
90
- 22,413, and her
91
- `REVOKE` returned it to 0, and his direct read was denied again.
92
-
93
- **A member must hold SELECT on the Merge rooms themselves** for any of that to work —
94
- provisioning issues it. Without that grant the team room does not narrow to what the
95
- caller can read, it denies outright, which looks identical to isolation and is not.
96
-
97
- ### Grant-option covers the whole grant
98
-
99
- Alice's grant-option is over `SELECT, INSERT, ALTER UPDATE, ALTER DELETE`, so she *can*
100
- hand bob the mutation rights on her own rooms, not merely `SELECT`. Her data, her call —
101
- but a share is `SELECT`-only by convention, and nothing in the engine enforces the
102
- convention. A `share` verb should encode it.
103
-
104
- > **OPEN:** a `share` / `unshare` CLI verb wrapping the three grants, so a member never
105
- > types SQL. Mirrors memory-house's `mh-share`.
106
-
107
- ## Tier 2 — some rows, owner-mediated
108
-
109
- Row policies remain the only way to share a *subset*. The member asks the owner, who
110
- holds house-scoped policy rights:
111
-
112
- ```sql
113
- GRANT SELECT ON mem.messages_alice TO bob; -- privilege
114
- CREATE ROW POLICY share_alice_to_bob ON mem.messages_alice
115
- FOR SELECT USING <predicate> TO bob; -- restriction
116
- ```
117
-
118
- Both halves are required: a policy filters, it never grants. Privilege without a policy
119
- shares everything; a policy without privilege shares nothing.
120
-
121
- > **OPEN:** which predicates — by project, by date range, by `source`? Free-form SQL from
122
- > a member is an injection surface, so this likely wants a small fixed vocabulary.
123
-
124
- > **OPEN:** what a partial recipient's rollup should show. The rollup is a saved query
125
- > over `sessions_alice` and `messages_alice`, so a policy on `messages_alice` filters the
126
- > rows it aggregates and the recipient sees totals computed over their own slice — which
127
- > is arguably right, and is untested. Nothing extra has to be shared for it to work,
128
- > which is one fewer moving part than the stored view had.
129
-
130
- ## Why the tiers are asymmetric
131
-
132
- Deliberate. Whole-room sharing is the common case, is coarse, and is safe to self-serve —
133
- the worst outcome is oversharing your own data. Partial sharing writes a filter into the
134
- house's access model, so it goes through the owner.
135
-
136
- ## What sharing does not do
137
-
138
- **It is not a boundary between agents sharing one credential.** A crew running under a
139
- single member credential reads and writes the same rooms; any per-agent tag is
140
- client-supplied. Enforced separation needs separate ClickHouse users, which means
141
- separate rooms — this design one level down. Deferred with `agent_id`.
142
-
143
- ## Not yet written
144
-
145
- - `memhouse share <member>` / `unshare`
146
- - how the dashboard and skills list rooms a caller can read but does not own — filter
147
- `system.tables` by grants (it *is* grant-filtered: alice sees exactly her own three
148
- three rooms, measured), or read the Merge rooms and group by `_table`
149
- - revocation when a member is removed while their rooms are shared outward
@@ -1,297 +0,0 @@
1
- #!/usr/bin/env node
2
- // mem — provision one member's rooms and grants. Idempotent; also the schema-rollout hook.
3
- //
4
- // provision.js --member <name> [--merge] [--owner-grants]
5
- //
6
- // Run as the OWNER. Steps, all IF NOT EXISTS / re-runnable:
7
- // 1. create the member's three rooms from schema-member.sql.tpl
8
- // 2. grant the member their own three rooms, TWO statements each: MEMBER_PRIVS (not
9
- // re-grantable), then `SELECT WITH GRANT OPTION`. The split is the point —
10
- // MEMBER_PRIVS is exactly what the shipper uses and nothing that DEFINES an object,
11
- // and attaching grant-option to `SELECT` alone makes a share read-only by
12
- // construction. It is deliberately NOT `ALL`: that includes CREATE TABLE on the
13
- // member's own room name, which lets them replace the room with a Merge over every
14
- // member's and double the others' rows in the team room. See rooms.js.
15
- //
16
- // THREE ROOMS, not four objects: the session rollup is a saved query over these
17
- // same rooms, not a stored view, so it needs no object and no grant of its own.
18
- // 3. --merge: create/refresh the three Merge rooms, borrowing this member's columns
19
- // 4. grant the member SELECT on whichever Merge rooms exist — a Merge reduces to the
20
- // rooms the caller can already read, so this is what makes the team room fail
21
- // CLOSED rather than deny outright
22
- //
23
- // Grants are explicit, per room, and NEVER `ON mem.*` — a member who can read the whole
24
- // house can read every other member's rooms, which would make this a shared house with
25
- // longer table names. No wildcards are used anywhere; the only pattern in the design is
26
- // the Merge regex, anchored on a fixed room type.
27
- //
28
- // Env: MEM_URL / MEM_USER / MEM_PASSWORD / MEM_DB, falling back to MEMHOUSE_*. URL and
29
- // USER are REQUIRED — see the note at cfg. MEM_DB defaults to 'mem'.
30
-
31
- const fs = require('fs');
32
- const crypto = require('crypto');
33
- const path = require('path');
34
- // Named, not thrown as a raw MODULE_NOT_FOUND with a require stack. This runs as a
35
- // SUBPROCESS of `memhouse install`, after the admin path has already created a ClickHouse
36
- // user — so its failure mode is a half-provisioned house, and "Cannot find module" plus a
37
- // stack trace is the least useful thing to print at that moment. Seen for real in a
38
- // checkout where `npm install` had not been run.
39
- let createClient, ClickHouseLogLevel;
40
- try { ({ createClient, ClickHouseLogLevel } = require('@clickhouse/client')); }
41
- catch {
42
- console.error("provision.js: dependency '@clickhouse/client' is not installed.");
43
- console.error(` From a checkout: npm install --prefix ${path.join(__dirname, '..', '..')}`);
44
- console.error(` From an npm install: ${require('./rooms').installCommand()}`);
45
- process.exit(2);
46
- }
47
- const { ROOM_TYPES, mergeRooms, assertUsableMember, MEMBER_PRIVS, MEMBER_PROFILE, MEMBER_PROFILE_SETTINGS } = require('./rooms');
48
-
49
- const args = process.argv.slice(2);
50
- const flag = (n) => args.includes(`--${n}`);
51
- const opt = (n, d = null) => { const i = args.indexOf(`--${n}`); return i >= 0 ? args[i + 1] : d; };
52
-
53
- // Defaults MUST match what the shipper and CLI use, or an operator who overrides only
54
- // the thing that differs — a password, say — silently provisions rooms in a house the
55
- // product never reads, or fails to authenticate as a user that does not exist.
56
- //
57
- // Which is why there is no default URL or user any more, here either. This file CREATES
58
- // USERS AND GRANTS; pointing it at a guessed http://localhost:8123 as memhouse_root aims
59
- // the most privileged operation in the product at whatever house happens to be listening.
60
- // Observed: running it with no env at all made an authentication attempt against the
61
- // pilot's own ClickHouse.
62
- const cfg = {
63
- url: process.env.MEM_URL || process.env.MEMHOUSE_URL,
64
- username: process.env.MEM_USER || process.env.MEMHOUSE_USER,
65
- password: process.env.MEM_PASSWORD || process.env.MEMHOUSE_PASSWORD || '',
66
- database: process.env.MEM_DB || process.env.MEMHOUSE_DB || 'mem',
67
- };
68
- if (!cfg.url || !cfg.username) {
69
- console.error('provision.js: no house given. Set MEM_URL and MEM_USER (or MEMHOUSE_URL/MEMHOUSE_USER).');
70
- console.error(' This creates users and grants — it will not guess http://localhost:8123 as memhouse_root.');
71
- process.exit(2);
72
- }
73
-
74
- // Statements are split on ';' at end of line — the templates contain no ';' inside a
75
- // statement body, and keeping the splitter dumb keeps the templates readable.
76
- function statements(sql) {
77
- return sql
78
- .split(/;\s*$/m)
79
- .map((s) => s.replace(/^\s*--.*$/gm, '').trim())
80
- .filter(Boolean);
81
- }
82
-
83
- /** Which of the three Merge rooms actually exist — a member may be provisioned first. */
84
- async function mergeRoomsPresent(client, database) {
85
- const names = Object.values(mergeRooms());
86
- const rs = await client.query({
87
- query: `SELECT name FROM system.tables WHERE database = {db:String} AND name IN ({names:Array(String)}) ORDER BY name`,
88
- query_params: { db: database, names },
89
- format: 'JSONEachRow',
90
- });
91
- return (await rs.json()).map((r) => r.name);
92
- }
93
-
94
- async function main() {
95
- const member = opt('member');
96
- if (!member) { console.error('usage: provision.js --member <name> [--merge]'); process.exit(2); }
97
- assertUsableMember(member);
98
-
99
- const here = __dirname;
100
-
101
- // Create the house with a client that has NOT selected it — on a fresh server the
102
- // database does not exist yet, and selecting it fails before it can be created.
103
- // See the note in shipper/ship.js: the driver's ERROR-level dump buries every
104
- // refusal this file prints. MEMHOUSE_DEBUG=1 restores it.
105
- const quiet = { level: process.env.MEMHOUSE_DEBUG ? ClickHouseLogLevel.DEBUG : ClickHouseLogLevel.OFF };
106
- const bootstrap = createClient({ ...cfg, database: '', log: quiet });
107
- await bootstrap.command({ query: `CREATE DATABASE IF NOT EXISTS ${cfg.database}` });
108
- await bootstrap.close();
109
-
110
- console.log(`[mem] provisioning in ${cfg.url} database '${cfg.database}' as '${cfg.username}'`);
111
- const client = createClient({ ...cfg, log: quiet, clickhouse_settings: { async_insert: 0 } });
112
-
113
- // 0. the member themselves.
114
- //
115
- // This file never created the ClickHouse user, so the path `memhouse install` prints and
116
- // INSTALL.md repeats — `provision.js --member <name> --merge` — created three rooms and
117
- // then died on `There is no role \`<name>\` in \`user directories\``, leaving orphan rooms
118
- // that nothing removes and that the --merge step then folded into all_sessions. It also
119
- // ignored --member-password. The documented second-member path did not work at all.
120
- const existing = await client.query({
121
- query: `SELECT name FROM system.users WHERE name = {n:String}`,
122
- query_params: { n: member }, format: 'JSONEachRow',
123
- });
124
- if ((await existing.json()).length === 0) {
125
- const pw = opt('member-password') || crypto.randomBytes(24).toString('base64url');
126
- await client.command({ query: `CREATE USER ${member} IDENTIFIED BY '${pw.replace(/'/g, "\\'")}'` });
127
- console.log(`[mem] created ClickHouse user '${member}'`);
128
- if (!opt('member-password')) {
129
- console.log(`[mem] password for '${member}': ${pw}`);
130
- console.log('[mem] shown once — hand it over, or pass --member-password next time');
131
- }
132
- console.log(`[mem] they finish with: memhouse install --url ${cfg.url} --db ${cfg.database} --user ${member} --password '…'`);
133
- } else if (flag('user-just-created')) {
134
- // The caller made it moments ago and has already said so. Saying "already exists"
135
- // here contradicts the line above it, and does so using the exact phrase that
136
- // elsewhere means "someone else owns this handle, refusing" — which reads as a
137
- // warning in the middle of a successful first install.
138
- console.log(`[mem] using the ClickHouse user '${member}' just created — provisioning rooms`);
139
- } else {
140
- // NOT "already exists". That phrase is what the identity-takeover REFUSAL says, so
141
- // reading it here — during a successful provision, and especially while resuming a
142
- // first install that failed after CREATE USER — reads as a warning that someone else
143
- // owns the handle. Both branches of this if now describe a success, in plain words.
144
- console.log(`[mem] using the existing ClickHouse user '${member}' — provisioning rooms`);
145
- }
146
-
147
- // 1. rooms
148
- const tpl = fs.readFileSync(path.join(here, 'schema-member.sql.tpl'), 'utf-8');
149
- const roomSql = statements(tpl.replaceAll('{{MEMBER}}', member));
150
- for (const q of roomSql) {
151
- // allow_experimental_full_text_index: a no-op on 26.x, required on 25.x for the
152
- // messages text indexes.
153
- await client.command({ query: q, clickhouse_settings: { allow_experimental_full_text_index: 1 } });
154
- }
155
- // CREATE TABLE IF NOT EXISTS is a no-op on an existing room, so re-running provision.js
156
- // over a room that has DRIFTED left it lossy while printing "rooms ready". The owner's
157
- // route has to heal as well as create — it is the one route that always holds the rights.
158
- {
159
- const { templateColumns } = require('../shipper/ship');
160
- const want = templateColumns(tpl);
161
- for (const ty of ROOM_TYPES) {
162
- const room = `${ty}_${member}`;
163
- const rs = await client.query({
164
- query: 'SELECT name FROM system.columns WHERE database = {d:String} AND table = {n:String}',
165
- query_params: { d: cfg.database, n: room }, format: 'JSONEachRow',
166
- });
167
- const have = new Set((await rs.json()).map((r) => r.name));
168
- if (!have.size) continue;
169
- for (const c of (want[ty] || [])) {
170
- if (have.has(c.name)) continue;
171
- await client.command({
172
- query: `ALTER TABLE ${cfg.database}.${room} ADD COLUMN IF NOT EXISTS ${c.name} ${c.type}`,
173
- clickhouse_settings: { allow_experimental_full_text_index: 1 },
174
- });
175
- console.log(`[mem] added missing column ${room}.${c.name}`);
176
- }
177
- }
178
- }
179
- console.log(`[mem] rooms ready for '${member}': ${ROOM_TYPES.map((t) => `${t}_${member}`).join(', ')}`);
180
-
181
- // 2. grants — two statements per room, and the split is the point.
182
- //
183
- // ALL, NOT re-grantable: the member owns their room outright, DROP and TRUNCATE
184
- // included, because it is their memory to destroy. It also covers the mutation
185
- // privileges the shipper needs without naming them — the shipper clears a session's
186
- // rows before re-inserting, and which of ALTER UPDATE / ALTER DELETE a server demands
187
- // varies by version (`DELETE FROM` is a lightweight delete, implemented as
188
- // `ALTER TABLE … UPDATE _row_exists = 0`, so 25.11 wants ALTER UPDATE where 26.7 wanted
189
- // ALTER DELETE). Granting one and not the other produces a pass that fails elsewhere.
190
- //
191
- // SELECT, re-grantable: a share is READ-ONLY BY CONSTRUCTION rather than by convention.
192
- // `GRANT ALL … WITH GRANT OPTION` expands to 45 privileges on 26.7 — DROP TABLE,
193
- // TRUNCATE, CREATE ROW POLICY, SYSTEM DROP REPLICA among them — and every one of those
194
- // would become something a member could hand to a colleague while meaning "let them
195
- // read my sessions". Measured: with this split, `GRANT SELECT … TO bob` succeeds and
196
- // `GRANT DROP TABLE … TO bob` is refused 497.
197
- // NOT `GRANT ALL`. On 26.7 that expands to 45 privileges including CREATE TABLE **on
198
- // the member's own room name** — which means a member owns the NAME, not just the data,
199
- // and can put any engine behind it. Measured on both versions:
200
- //
201
- // DROP TABLE messages_alice; -- allowed
202
- // CREATE TABLE messages_alice AS all_messages
203
- // ENGINE = Merge('<db>','^messages_'); -- allowed
204
- //
205
- // `all_messages` is Merge(db,'^messages_'), so it then contains a Merge over its own
206
- // namespace and every OTHER member's rows are counted twice in the team room: a house
207
- // reading alice 6030 / bob 2401 / carol 12 became bob 4802 / carol 24, silently, no
208
- // error. Confidentiality survives — the Merge still narrows by grants, so alice reads
209
- // nothing new — but integrity does not, and a team-wide number is the whole point of
210
- // the room. SCHEMA.md already records the same failure reached by accident; this is the
211
- // same failure reachable on purpose by any member.
212
- //
213
- // So: the privileges the shipper actually uses, and nothing that lets a member define
214
- // an object. ALTER UPDATE and ALTER DELETE both, because `DELETE FROM` is a lightweight
215
- // delete implemented as `ALTER TABLE … UPDATE _row_exists = 0` and which of the two a
216
- // server demands varies by version (25.11 wants UPDATE where 26.7 wanted DELETE).
217
- // ALTER ADD COLUMN for ensureSchema's rollout. No CREATE TABLE, no DROP TABLE, no
218
- // TRUNCATE: the admin who provisioned the room is the one who can replace it.
219
- //
220
- // SELECT is re-grantable so a share is READ-ONLY BY CONSTRUCTION rather than by
221
- // convention — `GRANT SELECT … TO bob` succeeds, `GRANT DROP TABLE … TO bob` is 497.
222
- for (const t of ROOM_TYPES) {
223
- const room = `${cfg.database}.${t}_${member}`;
224
- await client.command({ query: `GRANT ${MEMBER_PRIVS} ON ${room} TO ${member}` });
225
- await client.command({ query: `GRANT SELECT ON ${room} TO ${member} WITH GRANT OPTION` });
226
- }
227
- console.log(`[mem] granted ${MEMBER_PRIVS} on 3 rooms to '${member}'; SELECT is the only re-grantable one`);
228
-
229
- // 2b. A settings profile with CEILINGS, not just defaults.
230
- //
231
- // Every isolation test passed on confidentiality and none existed for availability: a
232
- // member could `SETTINGS max_memory_usage=100000000000`, `max_execution_time=0` and
233
- // `max_threads=64` on a shared house, and take the server down for everyone. The
234
- // constraint form (`MAX`) is what makes it a ceiling — a plain default is advisory and a
235
- // member simply overrides it, which is what they were doing.
236
- //
237
- // Generous on purpose: a full re-ship of a large house is a big INSERT, and the point is
238
- // to stop one member exhausting the box, not to make honest work fail.
239
- // CREATE ... OR REPLACE, which this used, DETACHES every user already assigned to the
240
- // profile. One shared profile plus a replace on every provision meant that adding a
241
- // second member silently stripped the first member's ceilings — measured: provision
242
- // kyle, then provision lena, and `SHOW CREATE USER kyle` no longer carries the profile
243
- // at all. In a three-member house only the most recently provisioned member had any
244
- // limit, and nothing said so.
245
- //
246
- // CREATE IF NOT EXISTS + ALTER updates the same object in place and keeps every existing
247
- // assignment, so one provision now brings ALL members up to the current settings —
248
- // which is also what makes the async_insert pin reach members provisioned earlier.
249
- // SETTINGS NONE first, because ALTER only upserts what it lists: a setting dropped from
250
- // MEMBER_PROFILE_SETTINGS in a later version would otherwise stay enforced forever on
251
- // every upgraded house, and anything an operator had added to a profile of the same name
252
- // would survive too. Clearing then applying gives the same absolute state OR REPLACE had,
253
- // without its detachment.
254
- //
255
- // The gap between the two statements is admin-only and sub-second — but it is not
256
- // self-healing: a crash or a dropped connection between them leaves the profile EMPTY,
257
- // so every member is unbounded until someone provisions again. Nothing here notices, and
258
- // `doctor` does not check the profile. Re-run provision if one dies mid-way.
259
- try {
260
- await client.command({ query: `CREATE SETTINGS PROFILE IF NOT EXISTS ${MEMBER_PROFILE}` });
261
- await client.command({ query: `ALTER SETTINGS PROFILE ${MEMBER_PROFILE} SETTINGS NONE` });
262
- await client.command({ query: `ALTER SETTINGS PROFILE ${MEMBER_PROFILE} SETTINGS ${MEMBER_PROFILE_SETTINGS}` });
263
- await client.command({ query: `ALTER USER ${member} SETTINGS PROFILE '${MEMBER_PROFILE}'` });
264
- console.log(`[mem] settings profile '${MEMBER_PROFILE}' applied to '${member}' (memory, time and thread ceilings)`);
265
- } catch (e) {
266
- // A server where the admin cannot create profiles still gets a working member; say so
267
- // rather than failing the provision.
268
- console.log(`[mem] note: could not apply the '${MEMBER_PROFILE}' settings profile — ${e.message}`);
269
- console.log('[mem] the member works, but nothing bounds their query resources on this server');
270
- }
271
-
272
- // 3. Merge rooms
273
- if (flag('merge')) {
274
- const mtpl = fs.readFileSync(path.join(here, 'schema-merge.sql.tpl'), 'utf-8');
275
- for (const q of statements(mtpl.replaceAll('{{TEMPLATE_MEMBER}}', member))) {
276
- await client.command({ query: q });
277
- }
278
- console.log('[mem] merge rooms ready: all_sessions, all_messages, all_tool_calls');
279
- }
280
-
281
- // 4. Merge-room grants. Broad on purpose, and safe for the reason the whole layout
282
- // rests on: a Merge table reduces to the underlying rooms the CALLER holds grants for.
283
- // Measured on 25.11 — bob with SELECT on all_messages and nothing on messages_alice
284
- // reads his own row and none of alice's, with `_table` showing only messages_bob, and
285
- // no error. Withholding this grant does not make anything safer; it makes the team
286
- // room deny outright instead of failing closed, which is a different and worse thing.
287
- const merged = await mergeRoomsPresent(client, cfg.database);
288
- for (const room of merged) {
289
- await client.command({ query: `GRANT SELECT ON ${cfg.database}.${room} TO ${member}` });
290
- }
291
- if (merged.length) console.log(`[mem] granted SELECT on ${merged.join(', ')} to '${member}'`);
292
- else console.log('[mem] no merge rooms yet — run once with --merge to create them');
293
-
294
- await client.close();
295
- }
296
-
297
- main().catch((e) => { console.error(`[mem] provision failed: ${e.message}`); process.exit(1); });