memhouse 0.7.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -36,6 +36,13 @@ makes it worse — [EACCES on install](#eacces-on-install) tells them apart.
36
36
  memhouse deploy --local # a loopback-bound ClickHouse, then install and ship
37
37
  ```
38
38
 
39
+ The house it stands up is **yours by name**: its superuser defaults to your OS
40
+ username (`polat`, not `memhouse_root` — override with `--user`), and the house
41
+ database defaults to the same name, so your memory lives at `polat.messages` and
42
+ `WHERE user_id = 'polat'` reads like it should. The name
43
+ and password are fixed when the data volume is first initialised, like any ClickHouse
44
+ container, and reused on every redeploy after that.
45
+
39
46
  Otherwise point at one you already run, or hand the SQL to whoever administers it:
40
47
 
41
48
  ```bash
@@ -157,19 +164,30 @@ immediate.
157
164
 
158
165
  ## How your memory is stored
159
166
 
160
- **One house, one set of rooms per member.** You own `sessions_<you>`,
161
- `messages_<you>`, `tool_calls_<you>` in the `mem` database, and hold grants on
162
- those and nothing else. Isolation is a grant that is simply *absent*, so it fails
163
- closed — no row policy that has to be right on every table and every read path.
167
+ **A house is a database; its rooms are three shared tables.** `sessions`, `messages`
168
+ and `tool_calls` live in whatever database you point at — your own name by default
169
+ (`polat.messages`), a team's (`team_a.messages`), even `default`. Everyone in the
170
+ house writes into the same tables with their own credential, and every row says where
171
+ it came from: `user_id`, stamped by the server (`MATERIALIZED currentUser()`, with
172
+ async inserts pinned off so the stamp cannot be skipped), and `host`, the machine's
173
+ install fingerprint.
174
+
175
+ `WHERE user_id = 'alice'` is one person. `WHERE host = '…'` is one machine. No filter
176
+ is the whole house — which is exactly what a team dashboard wants.
164
177
 
165
- All of *your* machines ship into *your* rooms; the `host` column tells them apart.
166
- Another member's machines never do. A team-wide read is a `Merge` room
167
- (`all_sessions`) plus a `GRANT`, which narrows to whatever the caller can already
168
- see. Sharing a whole room needs no operator.
178
+ Sharing is not a feature bolted on top; it IS the house. A team makes a database,
179
+ grants each person `ALL` on it, and their shippers all write into the same rooms:
180
+
181
+ ```sql
182
+ CREATE DATABASE team_a;
183
+ CREATE USER alice IDENTIFIED BY '…';
184
+ GRANT ALL ON team_a.* TO alice; -- repeat per housemate
185
+ ```
169
186
 
170
- The member name comes from the server — `SELECT currentUser()` — not from your
171
- config. A client that could name its own member could write into someone else's
172
- rooms.
187
+ `ALL` on your own house reaches nothing outside it — the database is the boundary,
188
+ which is also why joining a ClickHouse someone else runs (a kernel's, a team's) needs
189
+ no negotiation beyond a database and a credential. Housemates are collaborators;
190
+ groups that should not see each other get separate houses.
173
191
 
174
192
  ### Imported history is protected
175
193
 
package/bin/memhouse.js CHANGED
@@ -23,7 +23,7 @@ const SHIP_JS = path.join(REPO_ROOT, 'memhouse', 'shipper', 'ship.js');
23
23
  const SERVER_JS = path.join(REPO_ROOT, 'memhouse', 'server', 'server.js');
24
24
  const DELIVERY = path.join(REPO_ROOT, 'memhouse', 'delivery');
25
25
  const PKG = require(path.join(REPO_ROOT, 'package.json'));
26
- const { roomNames, ROOM_TYPES, mergeRooms, MEMBER_PRIVS, MEMBER_PROFILE, MEMBER_PROFILE_SETTINGS, installCommand } = require(path.join(REPO_ROOT, 'memhouse', 'per-member', 'rooms'));
26
+ const { roomNames, ROOM_TYPES, MEMBER_PIN, installCommand } = require(path.join(REPO_ROOT, 'memhouse', 'house', 'house'));
27
27
  const envfile = require(path.join(REPO_ROOT, 'memhouse', 'envfile'));
28
28
 
29
29
  const HOME_DIR = process.env.MEMHOUSE_HOME || path.join(os.homedir(), '.memhouse');
@@ -65,7 +65,10 @@ function resolveConfig() {
65
65
  url: pick('url', 'MEMHOUSE_URL', 'http://localhost:8123'),
66
66
  user: pick('user', 'MEMHOUSE_USER', 'memhouse_root'),
67
67
  password: pick('password', 'MEMHOUSE_PASSWORD', ''),
68
- db: pick('db', 'MEMHOUSE_DB', 'mem'),
68
+ // The house defaults to the USER'S OWN NAME (`polat` ships into `polat.messages`),
69
+ // because a house is a database and the natural first house is your own. Any name
70
+ // works — user 'alice' with house 'default' is a supported pairing.
71
+ db: pick('db', 'MEMHOUSE_DB', pick('user', 'MEMHOUSE_USER', 'memhouse_root')),
69
72
  port: pick('port', 'MEMHOUSE_PORT', '4640'),
70
73
  // Did anything actually SAY which house this is, or are the values above just the
71
74
  // defaults? The defaults are not neutral — localhost:8123 as memhouse_root is a real
@@ -600,38 +603,20 @@ function generatePassword() {
600
603
  * same templates provision.js applies, so the two cannot drift into different houses.
601
604
  */
602
605
  function memberSql(db, member, password) {
603
- const here = path.join(REPO_ROOT, 'memhouse', 'per-member');
604
- // The templates are written unqualified because provision.js applies them with the
605
- // house already selected. A human pastes this somewhere unknown — clickhouse-client,
606
- // the play UI, curl — so every name is qualified here and there is no `USE`.
607
- //
608
- // `currentDatabase()` in the Merge engine is the trap, and it is silent:
609
- // it is evaluated at CREATE time, not at read time, so a block run without the house
610
- // selected builds Merge rooms pointing at `default` that return zero rows forever. The
611
- // literal name goes in instead.
606
+ // The template is written unqualified because the shipper applies it with the house
607
+ // already selected. A human pastes this somewhere unknown — clickhouse-client, the
608
+ // play UI, curl — so every name is qualified here and there is no `USE`.
612
609
  const qualify = (sql) => sql
613
- .replace(/CREATE TABLE IF NOT EXISTS (\w+)/g, `CREATE TABLE IF NOT EXISTS ${db}.$1`)
614
- .replace(/ AS (\w+)\nENGINE = Merge/g, ` AS ${db}.$1\nENGINE = Merge`)
615
- .replace(/Merge\(currentDatabase\(\)/g, `Merge('${db}'`);
616
- const rooms = qualify(fs.readFileSync(path.join(here, 'schema-member.sql.tpl'), 'utf-8')
617
- .replaceAll('{{MEMBER}}', member));
618
- const merge = qualify(fs.readFileSync(path.join(here, 'schema-merge.sql.tpl'), 'utf-8')
619
- .replaceAll('{{TEMPLATE_MEMBER}}', member));
620
- // The SAME grant set provision.js issues. These two paths build the same house, and when
621
- // only one of them was narrowed, `install --print-sql` — the path INSTALL.md calls the
622
- // common case — still emitted GRANT ALL, which includes CREATE TABLE on the member's own
623
- // room name. A member provisioned that way could drop their room, recreate it as
624
- // Merge('<db>','^messages_'), and double every other member's rows in the team room:
625
- // exactly the attack the narrowing was for, fully reachable, one release later.
626
- const grants = ROOM_TYPES.flatMap((t) => [
627
- `GRANT ${MEMBER_PRIVS} ON ${db}.${t}_${member} TO ${member};`,
628
- `GRANT SELECT ON ${db}.${t}_${member} TO ${member} WITH GRANT OPTION;`,
629
- ]).join('\n');
630
- const mergeGrants = Object.values(mergeRooms())
631
- .map((n) => `GRANT SELECT ON ${db}.${n} TO ${member};`).join('\n');
610
+ .replace(/CREATE TABLE IF NOT EXISTS (\w+)/g, `CREATE TABLE IF NOT EXISTS ${db}.$1`);
611
+ const rooms = qualify(fs.readFileSync(path.join(REPO_ROOT, 'memhouse', 'house', 'schema.sql.tpl'), 'utf-8'));
632
612
  return `-- memhouse: everything '${member}' needs in house '${db}'. Run as a user with
633
613
  -- ACCESS MANAGEMENT (a stock 'default' with access_management=1 will do).
634
614
  --
615
+ -- A house is a database; its rooms are three shared tables. Everyone granted on the
616
+ -- database writes into the same tables, and every row carries who (user_id, stamped by
617
+ -- the server) and where from (host, the install's fingerprint). Adding a housemate later
618
+ -- is the same two statements: CREATE USER, GRANT ALL ON the house.
619
+ --
635
620
  -- Every name is qualified and there is no USE, so this runs anywhere: clickhouse-client,
636
621
  -- the play UI, curl, a GUI. Order matters only in that the database comes first.
637
622
  --
@@ -655,26 +640,16 @@ CREATE USER IF NOT EXISTS ${member} IDENTIFIED BY '${password.replace(/'/g, "\\'
655
640
 
656
641
  ${rooms.trim()}
657
642
 
658
- -- Exactly what the shipper uses, and nothing that defines an object: no CREATE TABLE, no
659
- -- DROP, no TRUNCATE. A member who owned the room NAME rather than its data could replace
660
- -- it with a Merge over every member's rooms and double their rows in the team room.
661
- -- Only SELECT is re-grantable, so a share can only ever be read-only.
662
- ${grants}
663
-
664
- -- Team-wide reads. A Merge room reduces to the rooms the CALLER holds grants for, so
665
- -- this narrows rather than denies, and picks up members added later with no DDL.
666
- ${merge.trim()}
667
-
668
- ${mergeGrants}
669
-
670
- -- Resource ceilings. Confidentiality isolation is a grant that is absent; this is the
671
- -- availability half, and it did not exist until 0.4.7 — a member could raise
672
- -- max_memory_usage, zero max_execution_time and take a shared server down for everyone.
673
- -- CONST, not MAX: a plain default is advisory, and MAX is not enough either because 0 means
674
- -- UNLIMITED in ClickHouse and 0 satisfies any MAX. Measured — with a MAX ceiling in force,
675
- -- SETTINGS max_memory_usage = 0 was accepted and the ceiling was gone.
676
- CREATE SETTINGS PROFILE OR REPLACE ${MEMBER_PROFILE} SETTINGS ${MEMBER_PROFILE_SETTINGS};
677
- ALTER USER ${member} SETTINGS PROFILE '${MEMBER_PROFILE}';
643
+ -- The whole database. ALL on your own house reaches nothing outside it — the database is
644
+ -- the boundary — and it is what lets the shipper create and evolve its own tables.
645
+ GRANT ALL ON ${db}.* TO ${member};
646
+
647
+ -- Pin async_insert on the user (ADD SETTING merges; a bare SETTINGS clause would replace
648
+ -- the user's whole list). The pin keeps the user_id stamp honest: a MATERIALIZED
649
+ -- currentUser() is computed during the INSERT, and an async flush stores it as the empty
650
+ -- string. CONST refuses the override; a client that never mentions the setting is held
651
+ -- at 0 and its insert lands stamped.
652
+ ALTER USER ${member} ADD SETTING ${MEMBER_PIN};
678
653
  `;
679
654
  }
680
655
 
@@ -702,22 +677,19 @@ async function adminBootstrap(cfg, admin) {
702
677
  catch { console.log(bad(`house '${cfg.db}' does not exist and '${admin.user}' cannot create it: ${e.message}`)); return null; }
703
678
  }
704
679
 
705
- // 2. the user. An EXISTING user is refused: adopting one silently hands the second
706
- // human the first's identity and rooms, and user_id MATERIALIZED currentUser() would
707
- // stamp them identically, so nothing downstream would ever notice.
708
- // A member who passes --admin-user gets ACCESS_DENIED here, because no member holds
709
- // SELECT ON system.users. That is the isolation working — but the raw 497 names neither
710
- // side, while INSTALL.md promises install "refuses, names both sides, and creates
711
- // nothing". Say who you are not, and what the owner has to run instead.
680
+ // 2. the user. A shared house makes an existing user unremarkable — rows are told apart
681
+ // by the server-stamped user_id, so "the handle is taken" hands nobody anything. What
682
+ // still matters is proof: connecting AS the member below is what validates whichever
683
+ // password is in play, existing user or fresh one.
712
684
  let exists;
713
685
  try {
714
686
  exists = (await chRows(adminCfg, `SELECT name FROM system.users WHERE name = '${admin.member}'`, { database: '' })).length > 0;
715
687
  } catch (e) {
716
688
  if (/Not enough privileges|ACCESS_DENIED|system\.users/i.test(e.message || '')) {
717
689
  console.log(bad(`'${admin.user}' is not an admin of this house — it cannot read system.users, so it cannot provision a member`));
718
- console.log(' Adding a member is the house owner\'s job. Ask them for:');
719
- console.log(` node ${path.join(REPO_ROOT, 'memhouse', 'per-member', 'provision.js')} --member ${admin.member} --merge`);
720
- console.log(' If you already HAVE a member credential on this house, you do not need admin:');
690
+ console.log(' Joining a house is two statements for whoever owns it:');
691
+ console.log(` CREATE USER ${admin.member} IDENTIFIED BY '…'; GRANT ALL ON ${cfg.db}.* TO ${admin.member};`);
692
+ console.log(' If you already HAVE a credential on this house, you do not need admin:');
721
693
  console.log(` memhouse install --url ${adminCfg.url} --user ${admin.member} --password …`);
722
694
  return null;
723
695
  }
@@ -725,19 +697,14 @@ async function adminBootstrap(cfg, admin) {
725
697
  }
726
698
  let password = flags['member-password'] || null;
727
699
  if (exists) {
728
- if (flags['adopt-user'] !== true) {
729
- console.log(bad(`ClickHouse user '${admin.member}' already exists in this house.`));
730
- console.log(' Creating rooms for them would hand you their identity — user_id is stamped from');
731
- console.log(' currentUser(), so their rows and yours would be indistinguishable.');
732
- console.log(' If this is you on a new machine, prove it:');
733
- console.log(` memhouse install --adopt-user --member ${admin.member} --member-password '…' …`);
734
- console.log(` If it is someone else, pick another handle: --member <name>`);
700
+ if (!password) {
701
+ console.log(bad(`ClickHouse user '${admin.member}' already exists — pass --member-password to install as them.`));
702
+ console.log(` Or pick another handle: --member <name>`);
735
703
  return null;
736
704
  }
737
- if (!password) { console.log(bad('--adopt-user requires --member-password — the password is the proof')); return null; }
738
705
  try { await ch({ ...cfg, user: admin.member, password }, 'SELECT 1', { database: '' }); }
739
- catch { console.log(bad(`--adopt-user: '${admin.member}' did not authenticate with that password`)); return null; }
740
- console.log(ok(`adopted existing user '${admin.member}' (password verified)`));
706
+ catch { console.log(bad(`'${admin.member}' did not authenticate with that password`)); return null; }
707
+ console.log(ok(`using the existing ClickHouse user '${admin.member}' (password verified)`));
741
708
  } else {
742
709
  if (!password) {
743
710
  password = generatePassword();
@@ -753,24 +720,18 @@ async function adminBootstrap(cfg, admin) {
753
720
  createdUser = admin.member;
754
721
  }
755
722
 
756
- // 3. rooms, 4. grants, 5. Merge rooms + their grant — provision.js owns all of it, so
757
- // there is one implementation of the grant set rather than two that drift.
758
- const provision = path.join(REPO_ROOT, 'memhouse', 'per-member', 'provision.js');
759
- const rc = spawnSync(process.execPath, [provision, '--member', admin.member, '--merge'], {
760
- stdio: 'inherit',
761
- env: { ...process.env, MEM_URL: cfg.url, MEM_USER: admin.user, MEM_PASSWORD: admin.password, MEM_DB: cfg.db },
762
- });
763
- if (rc.status !== 0) {
764
- // "nothing was written" was a lie whenever the CREATE USER above had already run —
765
- // and it runs before this. Observed: provision.js died on a missing dependency, this
766
- // printed the all-clear, and the next install refused with the identity-takeover
767
- // message because the user it said it had not created was there. Say what exists, and
768
- // name the two ways forward.
769
- console.log(bad('provisioning failed'));
723
+ // 3. the grant and the pin. GRANT ALL on the house is the whole tenancy model: it
724
+ // reaches nothing outside the database, and it is what lets the member's own shipper
725
+ // create and evolve the rooms (--ensure-schema below).
726
+ try {
727
+ await q(`GRANT ALL ON ${cfg.db}.* TO ${admin.member}`, { database: '' });
728
+ await q(`ALTER USER ${admin.member} ADD SETTING ${MEMBER_PIN}`, { database: '' });
729
+ console.log(ok(`granted the house: ALL ON ${cfg.db}.* to '${admin.member}', async_insert pinned`));
730
+ } catch (e) {
731
+ console.log(bad(`could not grant the house to '${admin.member}': ${e.message}`));
770
732
  if (createdUser) {
771
- console.log(` ClickHouse user '${createdUser}' WAS created before this failed; its rooms were not.`);
772
- console.log(' Nothing was written to disk. To continue once the cause is fixed:');
773
- console.log(` memhouse install --adopt-user --member ${createdUser} --member-password '<the password above>' …`);
733
+ console.log(` ClickHouse user '${createdUser}' WAS created before this failed.`);
734
+ console.log(` Continue once fixed: memhouse install --member ${createdUser} --member-password '<the password above>' …`);
774
735
  console.log(` Or undo it as the admin: DROP USER ${createdUser}`);
775
736
  } else {
776
737
  console.log(' nothing was written');
@@ -779,11 +740,15 @@ async function adminBootstrap(cfg, admin) {
779
740
  }
780
741
 
781
742
  // The step that makes this trustworthy: stop being admin, and prove the credential we
782
- // are about to persist actually reaches the rooms.
743
+ // are about to persist can build and reach the rooms itself.
783
744
  const memberCfg = { ...cfg, user: admin.member, password };
745
+ if (run(SHIP_JS, ['--ensure-schema'], memberCfg) !== 0) {
746
+ console.log(bad(`'${admin.member}' could not create the rooms in '${cfg.db}' — nothing written to disk`));
747
+ return null;
748
+ }
784
749
  try {
785
- const seen = await chRows(memberCfg, `SELECT count() AS n FROM system.tables WHERE database = '${cfg.db}' AND name IN ('sessions_${admin.member}','messages_${admin.member}','tool_calls_${admin.member}')`, { database: '' });
786
- if (Number(seen[0]?.n) !== 3) { console.log(bad(`'${admin.member}' cannot see all three of their rooms — nothing written`)); return null; }
750
+ const seen = await chRows(memberCfg, `SELECT count() AS n FROM system.tables WHERE database = '${cfg.db}' AND name IN ('sessions','messages','tool_calls')`, { database: '' });
751
+ if (Number(seen[0]?.n) !== 3) { console.log(bad(`'${admin.member}' cannot see the three rooms — nothing written`)); return null; }
787
752
  } catch (e) { console.log(bad(`'${admin.member}' could not connect after provisioning: ${e.message}`)); return null; }
788
753
  console.log(ok(`verified as '${admin.member}' — admin credential discarded, not stored`));
789
754
  return memberCfg;
@@ -841,6 +806,16 @@ async function cmdInstall({ interactive }) {
841
806
  let cfg = resolveConfig();
842
807
  const adminUser = flags['admin-user'];
843
808
 
809
+ // The house name is spliced unquoted into CREATE DATABASE and GRANT ALL ON <db>.* —
810
+ // validate it HERE, on every install mode, not only where deploy happens to choose it.
811
+ // Found by the acceptance run, driven end to end: `install --db system` sailed through,
812
+ // granted a member CHECK/SHOW/SELECT/INSERT/ALTER/DROP/TRUNCATE/… ON system.* and
813
+ // created sessions/messages/tool_calls INSIDE the server's system database, exit 0.
814
+ // On a shared or kernel ClickHouse that is a granted-everything-on-system user minted
815
+ // by a memhouse one-liner.
816
+ try { require(path.join(REPO_ROOT, 'memhouse', 'house', 'house')).assertUsableName(cfg.db, 'house'); }
817
+ catch (e) { console.log(bad(e.message)); return 1; }
818
+
844
819
  // Option 1 — print the SQL and stop. For the common case: you have admin on this
845
820
  // ClickHouse and would rather run four statements yourself than hand a credential to an
846
821
  // installer. Nothing is written and nothing is contacted.
@@ -879,7 +854,7 @@ async function cmdInstall({ interactive }) {
879
854
  if (keyProblem) {
880
855
  console.log(bad(`the rooms are here, but their sorting keys are wrong: ${keyProblem}`));
881
856
  console.log(' a ship pass would corrupt them, so `memhouse ship` will refuse.');
882
- console.log(` rebuild them from ${path.join(REPO_ROOT, 'memhouse', 'per-member', 'schema-member.sql.tpl')}, then: memhouse ship --full`);
857
+ console.log(` rebuild them from ${path.join(REPO_ROOT, 'memhouse', 'house', 'schema.sql.tpl')}, then: memhouse ship --full`);
883
858
  console.log(' no config was written — nothing here reads as installed.');
884
859
  return 1;
885
860
  }
@@ -893,8 +868,8 @@ async function cmdInstall({ interactive }) {
893
868
  }
894
869
  if (flags['no-ship'] !== true && run(SHIP_JS, [], cfg) !== 0) return 1;
895
870
  console.log(ok('installed — next: memhouse start (dashboard + shipper loop)'));
896
- console.log(' admin is needed again only for: a second member, or an ADD COLUMN rollout.');
897
- console.log(` node ${path.join(REPO_ROOT, 'memhouse', 'per-member', 'provision.js')} --member <name> --merge`);
871
+ console.log(' adding a housemate later is two statements for the admin:');
872
+ console.log(` CREATE USER <name> IDENTIFIED BY '…'; GRANT ALL ON ${cfg.db}.* TO <name>;`);
898
873
  return 0;
899
874
  }
900
875
 
@@ -903,7 +878,7 @@ async function cmdInstall({ interactive }) {
903
878
  // authenticating as memhouse_root instead: the three flags did nothing, --yes satisfied
904
879
  // haveAll so no prompt asked who you were, and resolveConfig's default was used AS A
905
880
  // CREDENTIAL. Refuse rather than do something else silently.
906
- const adminOnly = ['adopt-user', 'member', 'member-password'].filter((k) => flags[k] !== undefined);
881
+ const adminOnly = ['member', 'member-password'].filter((k) => flags[k] !== undefined);
907
882
  if (!adminUser && adminOnly.length) {
908
883
  console.log(bad(`--${adminOnly.join(', --')} ${adminOnly.length > 1 ? 'are' : 'is'} only read with --admin-user`));
909
884
  console.log(' Those flags provision a member, which needs house admin. Without them this');
@@ -1016,7 +991,7 @@ async function cmdInstall({ interactive }) {
1016
991
  if (keyProblem) {
1017
992
  console.log(bad(`the rooms are here, but their sorting keys are wrong: ${keyProblem}`));
1018
993
  console.log(' a ship pass would corrupt them, so `memhouse ship` will refuse.');
1019
- console.log(` rebuild them from ${path.join(REPO_ROOT, 'memhouse', 'per-member', 'schema-member.sql.tpl')}, then: memhouse ship --full`);
994
+ console.log(` rebuild them from ${path.join(REPO_ROOT, 'memhouse', 'house', 'schema.sql.tpl')}, then: memhouse ship --full`);
1020
995
  console.log(' no config was written — nothing here reads as installed.');
1021
996
  return 1;
1022
997
  }
@@ -1306,7 +1281,7 @@ async function cmdDoctor() {
1306
1281
  add(t === objects.length, t === objects.length
1307
1282
  ? `schema: 3/3 rooms in '${cfg.db}' (${objects.join(', ')})`
1308
1283
  : `schema: ${t}/${objects.length} rooms in '${cfg.db}' — missing ${absent.join(', ')}`,
1309
- `run: memhouse install, or as the owner: node ${path.join(REPO_ROOT, 'memhouse', 'per-member', 'provision.js')} --member ${rooms.member}`);
1284
+ `run: memhouse install — with ALL on the house, ship --ensure-schema builds them`);
1310
1285
  } catch (e) { add(false, 'schema check', `${e.message} — run: memhouse install`); }
1311
1286
  // Columns, not just rooms. A room with a column missing accepts every insert and
1312
1287
  // discards that field — measured, 85 rows shipped with the value thrown away while
@@ -1314,7 +1289,7 @@ async function cmdDoctor() {
1314
1289
  // counted rooms.
1315
1290
  try {
1316
1291
  const { templateColumns } = require(path.join(REPO_ROOT, 'memhouse', 'shipper', 'ship.js'));
1317
- const tpl = fs.readFileSync(path.join(REPO_ROOT, 'memhouse', 'per-member', 'schema-member.sql.tpl'), 'utf-8');
1292
+ const tpl = fs.readFileSync(path.join(REPO_ROOT, 'memhouse', 'house', 'schema.sql.tpl'), 'utf-8');
1318
1293
  const want = templateColumns(tpl);
1319
1294
  // Names AND types AND the MATERIALIZED kind. Comparing names alone left every other
1320
1295
  // kind of drift invisible, with measured consequences: a `UInt64` column narrowed to
@@ -1379,7 +1354,7 @@ async function cmdDoctor() {
1379
1354
  add(false, `sorting keys: only ${checked}/${ROOM_TYPES.length} rooms are MergeTree — the rest have no sorting key at all`,
1380
1355
  'a room was replaced by a Merge/View/Log engine; rebuild it from the schema template as the house owner');
1381
1356
  } else add(wrongKeys.length === 0, `sorting keys${wrongKeys.length ? `: wrong on ${wrongKeys.join(', ')}` : ` carry origin correctly (${checked}/${ROOM_TYPES.length} rooms)`}`,
1382
- `rebuild those rooms from the schema template, then: memhouse ship --full\n template: ${path.join(REPO_ROOT, 'memhouse', 'per-member', 'schema-member.sql.tpl')}`);
1357
+ `rebuild those rooms from the schema template, then: memhouse ship --full\n template: ${path.join(REPO_ROOT, 'memhouse', 'house', 'schema.sql.tpl')}`);
1383
1358
  } catch (e) { add(false, 'sorting keys', e.message); }
1384
1359
  try {
1385
1360
  // countIf, not any(). `any()` returns an arbitrary row's value, so on a house with
@@ -1917,8 +1892,8 @@ async function cmdReset() {
1917
1892
  } catch { /* pre-origin house: nothing to protect, ensureSchema will add the column */ }
1918
1893
 
1919
1894
  if (flags.yes !== true) {
1920
- // Name the rooms. Under the per-member layout this only ever empties the caller's own,
1921
- // and "ALL rows in 'mem'" would misdescribe that in the alarming direction.
1895
+ // Name the rooms, and the scope honestly: the tables are shared, but reset only ever
1896
+ // deletes rows whose user_id is the CALLER's — a housemate's rows are untouched.
1922
1897
  const scope = allOrigins
1923
1898
  ? `EVERY row, including ${imported} imported one(s) that re-shipping CANNOT restore,`
1924
1899
  : 'the rows the shipper wrote';
@@ -2257,10 +2232,43 @@ async function cmdUninstall() {
2257
2232
  : process.env.MEMHOUSE_PASSWORD ? 'MEMHOUSE_PASSWORD' : null;
2258
2233
  if (initialised && rotateAsk) {
2259
2234
  console.log(bad(`${rotateAsk} cannot change the credential of an existing house — the image only applies it when it initialises the data directory.`));
2260
- console.log(' to rotate: ALTER USER memhouse_root IDENTIFIED BY \'…\' inside the house, then: memhouse setup --password …');
2235
+ console.log(` to rotate: ALTER USER ${priorCfg.user || 'memhouse_root'} IDENTIFIED BY '…' inside the house, then: memhouse setup --password …`);
2261
2236
  console.log(' to start over (DESTROYS the memory): memhouse deploy --down');
2262
2237
  process.exitCode = 2; break;
2263
2238
  }
2239
+ // Who owns this house. A local deploy is the pilot's own machine and their own
2240
+ // memory, so the house user is THEM — suggested from the OS username (`polat`, not
2241
+ // `memhouse_root`) — created by the container itself at first init with full
2242
+ // rights. No admin/member split, no provisioning dance: a URI and a credential is
2243
+ // the whole install. The name is fixed at volume init exactly like the password,
2244
+ // so an existing house keeps the user it was born with.
2245
+ const osName = (os.userInfo().username || '').replace(/[^A-Za-z0-9_]/g, '_').replace(/^[0-9_]+/, '');
2246
+ let houseUser;
2247
+ if (initialised) {
2248
+ // The env FILE, not resolveConfig(): flags outrank the file there, so comparing
2249
+ // `--user other` against priorCfg.user compares the flag against itself and the
2250
+ // refusal below can never fire — while the volume stays initialised with the
2251
+ // original name and every later connection fails auth.
2252
+ const storedUser = readEnvFile().MEMHOUSE_USER;
2253
+ // No stored user means the env file is gone or emptied — the stored PASSWORD is
2254
+ // gone with it, so the `initialised && !reusable` refusal below already owns this
2255
+ // case. Refusing `--user` here on a guessed 'memhouse_root' would tell the pilot
2256
+ // their house "was initialised with memhouse_root" on no evidence at all.
2257
+ if (storedUser && flags.user && flags.user !== true && flags.user !== storedUser) {
2258
+ console.log(bad(`--user cannot rename the user of an existing house — the image creates it only when it initialises the data directory.`));
2259
+ console.log(` this house was initialised with '${storedUser}'. To start over (DESTROYS the memory): memhouse deploy --down`);
2260
+ process.exitCode = 2; break;
2261
+ }
2262
+ houseUser = storedUser || 'memhouse_root';
2263
+ } else if (flags.user && flags.user !== true) {
2264
+ houseUser = String(flags.user);
2265
+ } else if (flags.yes !== true && process.stdin.isTTY) {
2266
+ houseUser = (await ask(`House user (this becomes your member name)`, osName || 'memhouse_root')).trim() || osName || 'memhouse_root';
2267
+ } else {
2268
+ houseUser = osName || 'memhouse_root';
2269
+ }
2270
+ try { require(path.join(REPO_ROOT, 'memhouse', 'house', 'house')).assertUsableName(houseUser, 'house user'); }
2271
+ catch (e) { console.log(bad(`'${houseUser}' cannot be the house user — ${e.message}`)); process.exitCode = 2; break; }
2264
2272
  // An initialised volume with no credential to reuse — env file deleted, emptied, or
2265
2273
  // never written — is the same lockout by another route: a generated password would
2266
2274
  // be ignored by the server and then written over the config as if it worked.
@@ -2287,7 +2295,17 @@ async function cmdUninstall() {
2287
2295
  const tag = flags.tag || process.env.MEMHOUSE_CH_TAG || dep.managedTag() || dep.DEFAULT_TAG;
2288
2296
  // The database is part of the destination, not a detail of it — see the service
2289
2297
  // check below, which must compare it.
2290
- const targetDb = flags.db || process.env.MEMHOUSE_DB || priorCfg.db || 'mem';
2298
+ // STATED sources only, then the house user. priorCfg.db is unusable here: resolveConfig
2299
+ // computes a DEFAULT for it (the user's name, itself defaulted), so going through it
2300
+ // handed a fresh deploy `memhouse_root` as the house name while the superuser was
2301
+ // 'polat' — the memory would live in a database named after a role account nobody
2302
+ // chose.
2303
+ const targetDb = flags.db || process.env.MEMHOUSE_DB || readEnvFile().MEMHOUSE_DB || houseUser;
2304
+ // Same check install runs, but EARLY — deploy initialises the container volume and
2305
+ // writes the env file before its install step, so a refusal that waits for install
2306
+ // arrives after the damage.
2307
+ try { require(path.join(REPO_ROOT, 'memhouse', 'house', 'house')).assertUsableName(targetDb, 'house'); }
2308
+ catch (e) { console.log(bad(e.message)); process.exitCode = 2; break; }
2291
2309
  {
2292
2310
  // A service-managed shipper keeps the environment it was installed with, so ANY
2293
2311
  // switch that repoints the config leaves it shipping somewhere else — not only an
@@ -2341,7 +2359,7 @@ async function cmdUninstall() {
2341
2359
  }
2342
2360
  const pw = reusable || crypto.randomBytes(16).toString('hex');
2343
2361
  if (reusable) console.log(ok('reusing the existing house credential (its data volume is already initialised)'));
2344
- const r = dep.up({ password: pw, port, tag });
2362
+ const r = dep.up({ password: pw, port, tag, user: houseUser });
2345
2363
  if (!r.ok) { console.log(bad(r.msg)); process.exitCode = 1; break; }
2346
2364
  console.log(ok(`ClickHouse starting via ${r.engine} on ${r.url} (loopback only)`));
2347
2365
 
@@ -2353,7 +2371,10 @@ async function cmdUninstall() {
2353
2371
  // house that eventually came up is unreachable forever. The image applies the
2354
2372
  // password only at first init, so there is no way back from that.
2355
2373
  if (!reusable) {
2356
- writeEnvFile({ ...priorCfg, url: r.url, user: 'memhouse_root', password: pw });
2374
+ // db INCLUDED: this file is read back by the install below, and writing
2375
+ // priorCfg's computed default here made that install refuse over a mismatch
2376
+ // with a file this same command had just written.
2377
+ writeEnvFile({ ...priorCfg, url: r.url, user: houseUser, password: pw, db: targetDb });
2357
2378
  console.log(ok(`credential saved to ${ENV_FILE} before waiting — the volume is initialised with it`));
2358
2379
  }
2359
2380
 
@@ -2403,10 +2424,10 @@ async function cmdUninstall() {
2403
2424
  // anticipates it) and it is NOT the narrow set INSTALL.md's grant-set argument
2404
2425
  // describes. A user who later adds a second member should know which house they
2405
2426
  // have.
2406
- console.log(warn(`this house is yours alone: '${'memhouse_root'}' is its superuser, and that is the credential being saved`));
2407
- console.log(' Adding a second person later? Give them their own member instead:');
2408
- console.log(` node ${path.join(REPO_ROOT, 'memhouse', 'per-member', 'provision.js')} --member <name> --merge`);
2409
- flags.url = r.url; flags.user = 'memhouse_root'; flags.password = pw;
2427
+ console.log(warn(`this house is yours alone: '${houseUser}' is its superuser, and that is the credential being saved`));
2428
+ console.log(' Adding a housemate later is two statements:');
2429
+ console.log(` CREATE USER <name> IDENTIFIED BY '…'; GRANT ALL ON ${targetDb}.* TO <name>;`);
2430
+ flags.url = r.url; flags.user = houseUser; flags.password = pw;
2410
2431
  flags.db = targetDb;
2411
2432
  flags.yes = true;
2412
2433
  process.exitCode = await cmdInstall({ interactive: false });
@@ -5,10 +5,12 @@ on your machines — across **all 17 editors** agentlytics supports — parsed l
5
5
  shipped to a typed ClickHouse store, shareable with a team, installable on the
6
6
  ultimagent kernel as an **agency**, and visible through the agentlytics dashboard.
7
7
 
8
- **One house, a set of rooms per member** — `sessions_<m>`, `messages_<m>`,
9
- `tool_calls_<m>`. All of one member's machines ship into that member's rooms, told
10
- apart by the `host` column; no member writes into another's. The shared-table layout
11
- this once had was removed in 0.4.0; see `per-member/`.
8
+ **A house is a database; its rooms are three shared tables** — `sessions`, `messages`,
9
+ `tool_calls`. Everyone in the house writes into the same tables with their own
10
+ credential; the server-stamped `user_id` says who, the install fingerprint `host` says
11
+ which machine. Sharing IS the house; the boundary between groups is the database. (The
12
+ layout has moved twice: shared-tables-with-row-policies fell in 0.4.0, per-member
13
+ suffixed rooms fell in 0.8.0 — see `house/HOUSE.md` for why.)
12
14
 
13
15
  **Agency, precisely.** In constellation terms (`../TERMINOLOGY.md`) memhouse is a
14
16
  **house** — the `mem` database — **plus a resident**: the shipper. That pairing
@@ -24,23 +26,21 @@ on = "loop" # a daemon — not an insert trigger, not a schedule
24
26
  ```
25
27
 
26
28
  The session rollup is a **routine** — house machinery, not a second resident. Since
27
- 0.4.0 it is not even an object: it is SQL text (`per-member/rooms.js`), substituted with
28
- the caller's own room names and computed during your query, for your query, writing
29
- nothing. memhouse has **no materialized views at all** — in fact no stored views at all
29
+ 0.4.0 it is not even an object: it is SQL text (`house/house.js`), computed during your
30
+ query, for your query, writing nothing. memhouse has **no materialized views at all** — in fact no stored views at all
30
31
  — so the shipper is not merely a resident, it is the only candidate in the tree. Strip
31
32
  it and every remaining moving part is a routine over rows nobody is writing any more.
32
33
 
33
34
  Note that `user_id MATERIALIZED currentUser()` is a materialized *column*: it computes
34
- during your own insert and is part of the table's definition. It is not what isolates
35
- members — the room they hold no grant on does that — but it records who wrote each row,
36
- which keeps provenance across a share and keeps the `Merge` rooms meaningful. An identity
37
- property, not what makes this an agency.
35
+ during your own insert and is part of the table's definition. It is what tells housemates'
36
+ rows apart in the shared tables — provenance, stamped by the server, with async_insert
37
+ pinned so it cannot be skipped. An identity property, not what makes this an agency.
38
38
 
39
39
  **Positioning:** an alternative agency **competing with memory-house**. It borrows
40
40
  memory-house's proven ideas (server-stamped identity, idempotent shipping,
41
41
  skills/plugin delivery) and agentlytics' proven assets (adapters, UI, cost engine) —
42
- but **not** its shared-table-plus-row-policy layout, which 0.4.0 removed in favour of
43
- one set of rooms per member. If it wins, it can become memory-house v4; until then the
42
+ but **not** its row-policy machinery: memhouse's tables are shared by people who chose
43
+ one house, and the boundary between houses is the database itself. If it wins, it can become memory-house v4; until then the
44
44
  two run side by side as separate agencies, each in its own house. Naming: the product is
45
45
  **memhouse**; the agency is **`memhouse`** and its house is the **`mem`** database
46
46
  (identifiers can't carry a dash; the owner kept the longer name, `memhouse_root`).
@@ -60,20 +60,19 @@ them free and further along; **bet 3 is the one nobody else attempts.**
60
60
  write axis this bet is precisely *moving work from routine to resident*:
61
61
  memory-house parses when you query, memhouse parses before anyone asks and
62
62
  writes the result down.
63
- 2. **Typed common schema** (`per-member/schema-member.sql.tpl`, applied once per
64
- member; `schema-merge.sql.tpl` for the team rooms). Physical typed columns (what
65
- memory-house derives in views) + `tool_calls_<m>` (memory-house has no tool table)
66
- + one `extra JSON` escape hatch per room so unnormalized adapter fields are never
67
- lost. `ReplacingMergeTree(ingested_at)`; `messages_<m>` keyed
68
- `(session_id, user_id, seq)` so a re-ship of a grown/changed session replaces stale
69
- rows (latest-wins). FTS text indexes built in (CH ≥ 26.2).
70
- 3. **Kernel-installable agency.** Same install path proven for agentlytics-agency:
71
- `install-agency{memhouse}` → house + `memhouse_root` + credential; members via
72
- `register-member` + owner `GRANT` on that member's own rooms. Own-only visibility
73
- needs no policy: a member is granted their rooms and nobody else's. The shipper is
74
- the resident that lands with it. Identity is `user_id MATERIALIZED currentUser()`
75
- (requires `async_insert=0`). Sharing is a further GRANT, issued by the member
76
- themselves (grant-option) or the owner.
63
+ 2. **Typed common schema** (`house/schema.sql.tpl` — three shared tables per house).
64
+ Physical typed columns (what memory-house derives in views) + `tool_calls`
65
+ (memory-house has no tool table) + one `extra JSON` escape hatch per room so
66
+ unnormalized adapter fields are never lost. `ReplacingMergeTree(ingested_at)`;
67
+ `messages` keyed `(session_id, user_id, origin, seq)` so a re-ship of a
68
+ grown/changed session replaces stale rows (latest-wins) and two housemates' rows
69
+ never collapse into one. FTS text indexes built in (CH ≥ 26.2).
70
+ 3. **Kernel-installable agency.** Joining any ClickHouse — a kernel's included — is a
71
+ database plus a credential: `CREATE USER`, `GRANT ALL ON <house>.* `, done. The
72
+ shipper is the resident that lands with it and creates its own rooms. Identity is
73
+ `user_id MATERIALIZED currentUser()` with `async_insert = 0 CONST` pinned on the
74
+ user; machines are told apart by the `host` fingerprint. The boundary between
75
+ agencies on one server is the database — a house's ALL reaches nothing outside it.
77
76
  4. **Borrowed UI, zero fork.** The React SPA consumes REST JSON, not tables. The
78
77
  memhouse server implements the **same `/api/*` contract** as agentlytics'
79
78
  `server.js` (same routes, same response shapes) over the memhouse schema, and
@@ -83,9 +82,9 @@ them free and further along; **bet 3 is the one nobody else attempts.**
83
82
 
84
83
  | Path | What | Notes |
85
84
  |---|---|---|
86
- | `per-member/rooms.js` | room-name resolution + the session rollup | one naming rule for both transports |
87
- | `per-member/schema-member.sql.tpl` | one member's three rooms (typed) | the owner via `provision.js`, or the member on a house they own |
88
- | `per-member/schema-merge.sql.tpl` | the three team Merge rooms | owner-managed; reduce to the caller's grants |
85
+ | `house/house.js` | room names + the session rollup + the user pin | one rule for both transports |
86
+ | `house/schema.sql.tpl` | the house's three shared tables (typed) | applied by each member's own shipper |
87
+ | `house/HOUSE.md` | the model, and what it replaced | |
89
88
  | `shipper/ship.js` | parse-on-client shipper CLI — the resident (`worker`) | reuses `../../editors`; incremental; idempotent |
90
89
  | `server/server.js` | REST API + dashboard | same API contract as agentlytics; serves `../../public` |
91
90
  | `delivery/` | delivery kit | installer, skills, plugin, AGENT-INSTALL.md, prompt |
@@ -26,22 +26,20 @@ FIRST. Only say you don't know after a search comes back empty.
26
26
  (full-text over messages), `/memhouse:sessions` (list/filter sessions),
27
27
  `/memhouse:sql` (free-form read-only SQL). Without skills, query directly.
28
28
 
29
- **Table names first.** Every member owns their rooms, so every table below takes your
30
- username as a suffix — `messages_<you>`, `sessions_<you>`, `tool_calls_<you>`. There
31
- are no unsuffixed rooms; the plain names fail with `UNKNOWN_TABLE` rather than
32
- returning nothing. Get the suffix from `SELECT currentUser()`. **There is no
33
- `sessions_v` object** — the rollup is a saved query over those same rooms, and
29
+ **Table names.** The house's rooms are three plain, shared tables — `messages`,
30
+ `sessions`, `tool_calls` — in the connection's database. Everyone in the house writes
31
+ into the same tables; `user_id` (server-stamped) says whose row it is and `host` says
32
+ which machine. Query the whole house by default; add `WHERE user_id = '<name>'` for one
33
+ person. **There is no `sessions_v` object** — the rollup is a saved query, and
34
34
  `memhouse sessions-query` prints it ready to drop into a `FROM (...) AS c`.
35
- `all_messages` / `all_sessions` / `all_tool_calls` read across every member you hold a
36
- grant for.
37
35
 
38
36
  - Find sessions about a topic (FTS, lowercase your terms):
39
- `SELECT DISTINCT session_id, any(project), min(ts) FROM messages_<you>
37
+ `SELECT DISTINCT session_id, any(project), min(ts) FROM messages
40
38
  WHERE hasToken(text_word, 'clickhouse') GROUP BY session_id
41
39
  ORDER BY 3 DESC LIMIT 10 SETTINGS final=1, join_use_nulls=1 FORMAT PrettyCompact`
42
40
  - Recent sessions: `SELECT session_id, source, project, started, first_prompt
43
41
  FROM $(memhouse sessions-query) AS c ORDER BY started DESC LIMIT 20 SETTINGS final=1, join_use_nulls=1`
44
- - Replay one session: `SELECT role, text FROM messages_<you>
42
+ - Replay one session: `SELECT role, text FROM messages
45
43
  WHERE session_id = '<id>' ORDER BY seq SETTINGS final=1, join_use_nulls=1`
46
44
 
47
45
  **Rules.**