memhouse 0.7.0 → 0.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/bin/memhouse.js CHANGED
@@ -673,7 +673,12 @@ ${mergeGrants}
673
673
  -- CONST, not MAX: a plain default is advisory, and MAX is not enough either because 0 means
674
674
  -- UNLIMITED in ClickHouse and 0 satisfies any MAX. Measured — with a MAX ceiling in force,
675
675
  -- SETTINGS max_memory_usage = 0 was accepted and the ceiling was gone.
676
- CREATE SETTINGS PROFILE OR REPLACE ${MEMBER_PROFILE} SETTINGS ${MEMBER_PROFILE_SETTINGS};
676
+ -- CREATE ... OR REPLACE, which this used, DETACHES every user already on the profile, so
677
+ -- provisioning a second member silently stripped the first member's ceilings. IF NOT
678
+ -- EXISTS + ALTER edits the same object in place and keeps every assignment.
679
+ CREATE SETTINGS PROFILE IF NOT EXISTS ${MEMBER_PROFILE};
680
+ ALTER SETTINGS PROFILE ${MEMBER_PROFILE} SETTINGS NONE;
681
+ ALTER SETTINGS PROFILE ${MEMBER_PROFILE} SETTINGS ${MEMBER_PROFILE_SETTINGS};
677
682
  ALTER USER ${member} SETTINGS PROFILE '${MEMBER_PROFILE}';
678
683
  `;
679
684
  }
@@ -756,7 +761,16 @@ async function adminBootstrap(cfg, admin) {
756
761
  // 3. rooms, 4. grants, 5. Merge rooms + their grant — provision.js owns all of it, so
757
762
  // there is one implementation of the grant set rather than two that drift.
758
763
  const provision = path.join(REPO_ROOT, 'memhouse', 'per-member', 'provision.js');
759
- const rc = spawnSync(process.execPath, [provision, '--member', admin.member, '--merge'], {
764
+ // Tell it who made the user. Without this the child finds a user that exists — because
765
+ // THIS command created it two lines ago — and says so, producing two consecutive lines
766
+ // of one command's output that contradict each other:
767
+ // ✓ created ClickHouse user 'henry'
768
+ // [mem] ClickHouse user 'henry' already exists — provisioning rooms only
769
+ // and "already exists" is the exact phrase memhouse uses for the identity-takeover
770
+ // REFUSAL, so a first-time installer reads a success as a warning about someone else.
771
+ const provisionArgs = [provision, '--member', admin.member, '--merge'];
772
+ if (createdUser) provisionArgs.push('--user-just-created');
773
+ const rc = spawnSync(process.execPath, provisionArgs, {
760
774
  stdio: 'inherit',
761
775
  env: { ...process.env, MEM_URL: cfg.url, MEM_USER: admin.user, MEM_PASSWORD: admin.password, MEM_DB: cfg.db },
762
776
  });
@@ -37,6 +37,53 @@ unforgeable **by members**, since `EXECUTE AS` is a grantable privilege they do
37
37
  and forgeable by the owner or anyone granted it. Trust boundary is the owner, not the
38
38
  engine.
39
39
 
40
+ The stamp is also pinned **server-side**: the member settings profile carries
41
+ `async_insert = 0 CONST`. A MATERIALIZED column is computed during the insert, and an
42
+ async insert flushes outside that context — measured on 25.11.9.34, `async_insert=1`
43
+ stores `user_id` as the empty string while a sync insert on the same table stamps
44
+ correctly. The shipper always passed `async_insert=0`, so memhouse was never the risk;
45
+ anything else holding a member credential was. With the constraint in force a client that
46
+ *asks* for `async_insert=1` is refused (`SETTING_CONSTRAINT_VIOLATION`), and one that
47
+ simply does not mention the setting is held at 0 — so the insert still succeeds, stamped,
48
+ rather than landing unowned. It is the override that is refused, not the write.
49
+
50
+ Two limits worth stating plainly. The constraint binds the **ClickHouse user**, not the
51
+ database — if you reuse a member's credential for unrelated high-throughput ingestion
52
+ elsewhere on that server, async inserts are refused there too. And it only takes effect
53
+ once the profile is applied: a house provisioned by an older memhouse keeps the old
54
+ profile until any member is provisioned again, which now updates the shared profile for
55
+ everyone at once. `doctor`'s blank-`user_id` check remains as the detector for rows
56
+ written before that.
57
+
58
+ ## What the boundary does NOT hide: the roster
59
+
60
+ **Members can discover which other members exist, and this is not fixable here.** A
61
+ handle is a table name, and ClickHouse answers differently for a table you may not read
62
+ than for one that is not there:
63
+
64
+ ```
65
+ SELECT count() FROM mem.messages_alice -- exists, not granted → Code 497, Not enough privileges
66
+ SELECT count() FROM mem.messages_nobody -- does not exist → Code 60, Unknown table
67
+ ```
68
+
69
+ So probing names enumerates the membership. Measured on 26.7.3.19 that this is
70
+ **ClickHouse's behaviour and not a consequence of our grant set**: a user holding *zero*
71
+ grants anywhere in the database gets exactly the same 497/60 split. No grant, revoke or
72
+ role arrangement changes it.
73
+
74
+ It *is* closable, and the price is the reason it stays open. Suffixing rooms with a random
75
+ token (`messages_alice_8f9b2a`) would still match the `^messages_` Merge selectors, and
76
+ would still leave the share in this document working — alice grants on the room she owns
77
+ and knows, and bob never needs its name. What it costs is that a room name stops being
78
+ **derivable**: `roomNames(member)` is pure today, and every client — the shipper, the
79
+ skills, `sessions-query`, a person typing SQL — would instead have to look its own name up
80
+ first. A real trade, deliberately not taken; not an impossibility.
81
+
82
+ What leaks is **membership, not content**: names and the fact of existence, never rows,
83
+ counts or sizes. In the setting this is built for — a team who already know they are a
84
+ team — that is not a secret. Say so out loud rather than implying the boundary is
85
+ tighter than it is.
86
+
40
87
  Bob's Merge-room reads widen automatically once alice grants him a room — `Merge` reduces
41
88
  to the caller's actual grants, so nothing else needs changing. Measured end to end on
42
89
  25.11: alice's `GRANT` took what bob sees of her rows through `all_messages` from 0 to
@@ -130,8 +130,18 @@ async function main() {
130
130
  console.log('[mem] shown once — hand it over, or pass --member-password next time');
131
131
  }
132
132
  console.log(`[mem] they finish with: memhouse install --url ${cfg.url} --db ${cfg.database} --user ${member} --password '…'`);
133
+ } else if (flag('user-just-created')) {
134
+ // The caller made it moments ago and has already said so. Saying "already exists"
135
+ // here contradicts the line above it, and does so using the exact phrase that
136
+ // elsewhere means "someone else owns this handle, refusing" — which reads as a
137
+ // warning in the middle of a successful first install.
138
+ console.log(`[mem] using the ClickHouse user '${member}' just created — provisioning rooms`);
133
139
  } else {
134
- console.log(`[mem] ClickHouse user '${member}' already exists — provisioning rooms only`);
140
+ // NOT "already exists". That phrase is what the identity-takeover REFUSAL says, so
141
+ // reading it here — during a successful provision, and especially while resuming a
142
+ // first install that failed after CREATE USER — reads as a warning that someone else
143
+ // owns the handle. Both branches of this if now describe a success, in plain words.
144
+ console.log(`[mem] using the existing ClickHouse user '${member}' — provisioning rooms`);
135
145
  }
136
146
 
137
147
  // 1. rooms
@@ -226,8 +236,30 @@ async function main() {
226
236
  //
227
237
  // Generous on purpose: a full re-ship of a large house is a big INSERT, and the point is
228
238
  // to stop one member exhausting the box, not to make honest work fail.
239
+ // CREATE ... OR REPLACE, which this used, DETACHES every user already assigned to the
240
+ // profile. One shared profile plus a replace on every provision meant that adding a
241
+ // second member silently stripped the first member's ceilings — measured: provision
242
+ // kyle, then provision lena, and `SHOW CREATE USER kyle` no longer carries the profile
243
+ // at all. In a three-member house only the most recently provisioned member had any
244
+ // limit, and nothing said so.
245
+ //
246
+ // CREATE IF NOT EXISTS + ALTER updates the same object in place and keeps every existing
247
+ // assignment, so one provision now brings ALL members up to the current settings —
248
+ // which is also what makes the async_insert pin reach members provisioned earlier.
249
+ // SETTINGS NONE first, because ALTER only upserts what it lists: a setting dropped from
250
+ // MEMBER_PROFILE_SETTINGS in a later version would otherwise stay enforced forever on
251
+ // every upgraded house, and anything an operator had added to a profile of the same name
252
+ // would survive too. Clearing then applying gives the same absolute state OR REPLACE had,
253
+ // without its detachment.
254
+ //
255
+ // The gap between the two statements is admin-only and sub-second — but it is not
256
+ // self-healing: a crash or a dropped connection between them leaves the profile EMPTY,
257
+ // so every member is unbounded until someone provisions again. Nothing here notices, and
258
+ // `doctor` does not check the profile. Re-run provision if one dies mid-way.
229
259
  try {
230
- await client.command({ query: `CREATE SETTINGS PROFILE OR REPLACE ${MEMBER_PROFILE} SETTINGS ${MEMBER_PROFILE_SETTINGS}` });
260
+ await client.command({ query: `CREATE SETTINGS PROFILE IF NOT EXISTS ${MEMBER_PROFILE}` });
261
+ await client.command({ query: `ALTER SETTINGS PROFILE ${MEMBER_PROFILE} SETTINGS NONE` });
262
+ await client.command({ query: `ALTER SETTINGS PROFILE ${MEMBER_PROFILE} SETTINGS ${MEMBER_PROFILE_SETTINGS}` });
231
263
  await client.command({ query: `ALTER USER ${member} SETTINGS PROFILE '${MEMBER_PROFILE}'` });
232
264
  console.log(`[mem] settings profile '${MEMBER_PROFILE}' applied to '${member}' (memory, time and thread ceilings)`);
233
265
  } catch (e) {
@@ -103,6 +103,23 @@ const MEMBER_PROFILE_SETTINGS = [
103
103
  'max_execution_time = 600 MIN 1 MAX 600',
104
104
  'max_threads = 16 MIN 1 MAX 16',
105
105
  'max_result_rows = 10000000 MIN 1 MAX 10000000',
106
+ // Provenance, pinned server-side rather than trusted to every client.
107
+ //
108
+ // `user_id MATERIALIZED currentUser()` is what records who wrote a row — but a
109
+ // MATERIALIZED column is computed during the INSERT, and an ASYNC insert flushes
110
+ // outside that context: measured on 25.11.9.34, `async_insert=1` stores user_id as the
111
+ // EMPTY STRING while a sync insert on the same table stamps correctly. An unattributed
112
+ // row is invisible to every identity-bound path at once, including its own owner's
113
+ // `reset`.
114
+ //
115
+ // The shipper already passes async_insert=0 on every insert, so memhouse itself was
116
+ // never the risk. The risk is everything ELSE holding a member credential — an agent
117
+ // following the /memhouse:sql skill, a migration script, a psql-habit one-liner. CONST
118
+ // makes the house refuse (SETTING_CONSTRAINT_VIOLATION) instead of silently accepting
119
+ // rows nobody owns, which is the difference between a guarantee and a convention.
120
+ // `doctor` still checks for blank user_id — that stays, as the detector for rows
121
+ // written before this profile existed.
122
+ 'async_insert = 0 CONST',
106
123
  ].join(', ');
107
124
 
108
125
  // ClickHouse usernames are permissive; room names are not. Refuse anything that would
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "memhouse",
3
- "version": "0.7.0",
3
+ "version": "0.7.1",
4
4
  "description": "memhouse — agent conversation memory across 17 editors: parse-on-client shippers, a typed ClickHouse store with one set of rooms per member, kernel-agency install, and the agentlytics dashboard",
5
5
  "main": "bin/memhouse.js",
6
6
  "bin": {