memhouse 0.7.1 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +79 -43
- package/bin/memhouse.js +153 -147
- package/editors/antigravity.js +7 -15
- package/editors/cursor.js +18 -24
- package/editors/goose.js +9 -21
- package/editors/index.js +19 -70
- package/editors/opencode.js +6 -15
- package/editors/sqlite.js +74 -0
- package/editors/windsurf.js +3 -3
- package/editors/zed.js +10 -15
- package/memhouse/COMPETITION.md +9 -5
- package/memhouse/DESIGN.md +29 -30
- package/memhouse/RESUME-AND-UPDATE.md +7 -10
- package/memhouse/delivery/AGENT-INSTALL.md +15 -13
- package/memhouse/delivery/PROMPT.md +7 -9
- package/memhouse/delivery/kernel-install.md +18 -33
- package/memhouse/delivery/plugin/skills/search/SKILL.md +25 -38
- package/memhouse/delivery/plugin/skills/sessions/SKILL.md +15 -28
- package/memhouse/delivery/plugin/skills/sql/SKILL.md +23 -36
- package/memhouse/host.js +1 -1
- package/memhouse/house/HOUSE.md +74 -0
- package/memhouse/house/house.js +170 -0
- package/memhouse/{per-member/schema-member.sql.tpl → house/schema.sql.tpl} +8 -11
- package/memhouse/resume.js +1 -1
- package/memhouse/self-update.js +4 -4
- package/memhouse/server/queries.js +8 -2
- package/memhouse/shipper/ship.js +30 -37
- package/package.json +2 -6
- package/memhouse/per-member/INSTALL.md +0 -228
- package/memhouse/per-member/PLAN.md +0 -174
- package/memhouse/per-member/PROVISIONING.md +0 -177
- package/memhouse/per-member/README.md +0 -48
- package/memhouse/per-member/SCHEMA.md +0 -120
- package/memhouse/per-member/SHARING.md +0 -149
- package/memhouse/per-member/provision.js +0 -297
- package/memhouse/per-member/rooms.js +0 -252
- package/memhouse/per-member/schema-merge.sql.tpl +0 -24
|
@@ -1,228 +0,0 @@
|
|
|
1
|
-
# INSTALL — how a house gets a member
|
|
2
|
-
|
|
3
|
-
Three options. The first two are finished and measured; the third is not.
|
|
4
|
-
|
|
5
|
-
| | you hold admin | who runs the SQL | admin credential |
|
|
6
|
-
|---|---|---|---|
|
|
7
|
-
| **1. print the SQL** | yes | you | never leaves your terminal |
|
|
8
|
-
| **2. hand it over** | yes | the installer | used, then discarded |
|
|
9
|
-
| **3. kernel** | no — the realm does | not finalized | — |
|
|
10
|
-
|
|
11
|
-
## The identity rule, first — everything else follows from it
|
|
12
|
-
|
|
13
|
-
**The ClickHouse user is the identity.** Rooms are `sessions_<member>`,
|
|
14
|
-
`messages_<member>`, `tool_calls_<member>`, and `<member>` is whatever the server answers
|
|
15
|
-
to `SELECT currentUser()` — never what the config file says, never what the OS says.
|
|
16
|
-
|
|
17
|
-
The home directory produces a **suggestion**:
|
|
18
|
-
|
|
19
|
-
```
|
|
20
|
-
/Users/polat -> polat
|
|
21
|
-
/Users/ramazan.polat -> ramazan_polat (sanitized, and the change is printed)
|
|
22
|
-
/home/JohnSmith -> JohnSmith (case preserved; ClickHouse is case-sensitive)
|
|
23
|
-
/root -> refused (a container artefact, not a person)
|
|
24
|
-
```
|
|
25
|
-
|
|
26
|
-
Sanitization is `[A-Za-z][A-Za-z0-9_]*`. It can collide — `ramazan.polat` and
|
|
27
|
-
`ramazan-polat` both land on `ramazan_polat` — so a changed suggestion is printed, and
|
|
28
|
-
`--member` overrides it.
|
|
29
|
-
|
|
30
|
-
## Option 1 — print the SQL, run it yourself
|
|
31
|
-
|
|
32
|
-
The common case: you installed ClickHouse, you have `default` with
|
|
33
|
-
`access_management = 1`, and you would rather run four kinds of statement than hand a
|
|
34
|
-
superuser credential to an installer.
|
|
35
|
-
|
|
36
|
-
```
|
|
37
|
-
memhouse install --print-sql [--member polat] [--db mem]
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
Prints the whole install as SQL and exits. Nothing is contacted, nothing is written. It
|
|
41
|
-
generates a password and prints it inside the `CREATE USER`, or takes `--member-password`.
|
|
42
|
-
The last line is the command to run afterwards, with the credential filled in.
|
|
43
|
-
|
|
44
|
-
Two properties that make the block safe to paste anywhere:
|
|
45
|
-
|
|
46
|
-
- **Every name is qualified and there is no `USE`.** It runs in `clickhouse-client`, the
|
|
47
|
-
play UI, `curl`, or a GUI, in any database context.
|
|
48
|
-
- **The Merge engine gets the literal house name, not `currentDatabase()`.** That call is
|
|
49
|
-
evaluated at CREATE time, not read time, so a block run without the house selected would
|
|
50
|
-
otherwise build Merge rooms pointing at `default` that return zero rows forever, with no
|
|
51
|
-
error at any point.
|
|
52
|
-
|
|
53
|
-
Measured: 18 statements (the 17 objects plus a leading `SET
|
|
54
|
-
allow_experimental_full_text_index = 1`), run over HTTP with no database selected, 18
|
|
55
|
-
succeeded on both 26.7.2.59 and 25.11.9.34, and the
|
|
56
|
-
Merge rooms came out bound to `Merge('mem', '^sessions_')`.
|
|
57
|
-
|
|
58
|
-
## Option 2 — give the installer admin, it runs the SQL
|
|
59
|
-
|
|
60
|
-
```
|
|
61
|
-
memhouse install --admin-user … --admin-password … [--member polat]
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
The same statements, applied by `provision.js` so there is one implementation of the grant
|
|
65
|
-
set rather than two that drift. Then the step that makes the mode trustworthy:
|
|
66
|
-
**disconnect from admin, reconnect as the member, and prove that credential reaches all
|
|
67
|
-
three rooms before writing anything.** A mode that only proves the admin could do it has
|
|
68
|
-
proven nothing about the credential it is about to persist.
|
|
69
|
-
|
|
70
|
-
**The admin credential is never persisted** — not to the env file, not to a unit file, not
|
|
71
|
-
to a log. A shipper running as the house owner would make per-member rooms decoration.
|
|
72
|
-
|
|
73
|
-
## Option 3 — the kernel
|
|
74
|
-
|
|
75
|
-
**Not finalized.** The realm holds the privileged credential and mints members; what the
|
|
76
|
-
installer asks it for, and what it gets back, is not designed yet. Everything above about
|
|
77
|
-
the room layout, the grant set and the identity rule holds regardless — what is open is
|
|
78
|
-
who plays admin and how the handle is chosen.
|
|
79
|
-
|
|
80
|
-
## The grant set — two statements per room, and the split is the point
|
|
81
|
-
|
|
82
|
-
```sql
|
|
83
|
-
GRANT SELECT, INSERT, ALTER UPDATE, ALTER DELETE, ALTER ADD COLUMN, OPTIMIZE
|
|
84
|
-
ON mem.sessions_polat TO polat;
|
|
85
|
-
GRANT SELECT ON mem.sessions_polat TO polat WITH GRANT OPTION;
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
**Exactly what the shipper uses, not `ALL`.** `ALL` expands to 45 privileges on 26.7 and
|
|
89
|
-
one of them is `CREATE TABLE` **on the member's own room name** — so the member owns the
|
|
90
|
-
NAME, not just the data, and can do this:
|
|
91
|
-
|
|
92
|
-
```sql
|
|
93
|
-
DROP TABLE messages_polat;
|
|
94
|
-
CREATE TABLE messages_polat AS all_messages ENGINE = Merge('mem','^messages_');
|
|
95
|
-
```
|
|
96
|
-
|
|
97
|
-
`all_messages` is `Merge(mem,'^messages_')`, so it then contains a Merge over its own
|
|
98
|
-
namespace and **every other member's rows are counted twice in the team room**. Measured:
|
|
99
|
-
a house reading alice 6030 / bob 2401 / carol 12 became bob 4802 / carol 24, with no error
|
|
100
|
-
on either ClickHouse version. Confidentiality survives — the Merge still narrows by grants,
|
|
101
|
-
so nothing new becomes readable — but a team-wide number is what that room is for.
|
|
102
|
-
|
|
103
|
-
`ALTER UPDATE` and `ALTER DELETE` are both named because `DELETE FROM` is a lightweight
|
|
104
|
-
delete implemented as `ALTER TABLE … UPDATE _row_exists = 0`, and which of the two a server
|
|
105
|
-
demands varies by version (25.11 wants one where 26.7 wanted the other). `ALTER ADD COLUMN`
|
|
106
|
-
is for `ensureSchema`'s rollout. Replacing a room is the admin's job, not the member's.
|
|
107
|
-
|
|
108
|
-
**`SELECT`, re-grantable.** A share is read-only **by construction**, not by convention:
|
|
109
|
-
the member can hand a colleague `SELECT` and nothing else, because they hold nothing else
|
|
110
|
-
to hand over.
|
|
111
|
-
|
|
112
|
-
Measured on 26.7.2 and 25.11.9:
|
|
113
|
-
|
|
114
|
-
```
|
|
115
|
-
polat grants SELECT to bob ok
|
|
116
|
-
polat grants DROP TABLE to bob 497 Not enough privileges
|
|
117
|
-
polat drops their own room 497 Not enough privileges
|
|
118
|
-
polat creates any table 497 Not enough privileges
|
|
119
|
-
polat installs, ships, resets ok
|
|
120
|
-
bob ends up with GRANT SELECT ON mem.sessions_polat
|
|
121
|
-
```
|
|
122
|
-
|
|
123
|
-
One definition, in `rooms.js` as `MEMBER_PRIVS`, used by both `provision.js` and
|
|
124
|
-
`install --print-sql`. They drifted once — the first was narrowed and the second went on
|
|
125
|
-
emitting `GRANT ALL` for a release, so the attack above stayed reachable through the path
|
|
126
|
-
this document calls the common case.
|
|
127
|
-
|
|
128
|
-
Never `ON mem.*`. A member who can read `mem.*` can read every other member's rooms, and
|
|
129
|
-
then this is a shared house with longer table names.
|
|
130
|
-
|
|
131
|
-
`SELECT` on the three Merge rooms is granted broadly and is safe: a Merge table reduces to
|
|
132
|
-
the underlying rooms the *caller* holds grants for, so it narrows rather than denies, and
|
|
133
|
-
picks up members added later with no DDL.
|
|
134
|
-
|
|
135
|
-
## Refusals — the complete list
|
|
136
|
-
|
|
137
|
-
Install refuses, names both sides, and creates nothing:
|
|
138
|
-
|
|
139
|
-
| Case | Why |
|
|
140
|
-
|---|---|
|
|
141
|
-
| handle does not match `[A-Za-z][A-Za-z0-9_]*` after sanitization | it would need quoting, and a quoted room name breaks the Merge selectors |
|
|
142
|
-
| handle would be `root`, or the home directory is `/root` | a container artefact, not a person |
|
|
143
|
-
| **option 2: the ClickHouse user already exists** | the second human would take over the first's identity and rooms, and `user_id MATERIALIZED currentUser()` would stamp them identically. Requires `--adopt-user` AND `--member-password`, verified by reconnecting before anything is written |
|
|
144
|
-
| option 2: `mem` absent and admin cannot `CREATE DATABASE` | nothing downstream can work |
|
|
145
|
-
| an env file already points at a different url/db | it would orphan a running shipper. `memhouse setup` is the deliberate way to move; `--force` overrides |
|
|
146
|
-
| rooms missing and you cannot create them | you are not the owner — both other options are printed |
|
|
147
|
-
|
|
148
|
-
`--adopt-user` is legitimate — the same person on a new laptop — but it is
|
|
149
|
-
indistinguishable from a takeover without the password, so the password is the proof.
|
|
150
|
-
|
|
151
|
-
## After install — when admin is needed again
|
|
152
|
-
|
|
153
|
-
- **a second member** — `provision.js --member <name> --merge`, or `--print-sql --member <name>`
|
|
154
|
-
- **schema rollout** — `ADD COLUMN` on the **Merge room first**, then the member rooms;
|
|
155
|
-
Merge rejects mutations and tolerates drift, which is what makes a progressive rollout
|
|
156
|
-
possible
|
|
157
|
-
- **offboarding** — `DROP USER <member>`. The rooms are deliberately left: they hold that
|
|
158
|
-
person's transcripts, and deleting someone's memory is a decision, not a cleanup step
|
|
159
|
-
|
|
160
|
-
## Upgrading from 0.3.x — read this before you do
|
|
161
|
-
|
|
162
|
-
**0.4.0 cannot read a 0.3.x house.** An earlier version of this document said an existing
|
|
163
|
-
house "keeps working untouched". That was wrong, and it was wrong in the release notes and
|
|
164
|
-
the `v0.4.0` tag message too.
|
|
165
|
-
|
|
166
|
-
What is true: the DATABASE NAME survives. `MEMHOUSE_DB` sits in `~/.memhouse/env` and
|
|
167
|
-
explicit config outranks the new `mem` default, so nothing repoints you at a different
|
|
168
|
-
house. What does not survive is the TABLES. 0.3.x kept `sessions`, `messages`,
|
|
169
|
-
`tool_calls` and a stored `sessions_v`; 0.4.0 looks for `sessions_<you>` and finds nothing.
|
|
170
|
-
Measured against a real 0.3.2 house:
|
|
171
|
-
|
|
172
|
-
```
|
|
173
|
-
memhouse status -> not connected: Code: 60 ... Unknown table expression
|
|
174
|
-
identifier 'sessions_memhouse_root'
|
|
175
|
-
memhouse doctor -> schema: 0/3 rooms in 'memhouse'
|
|
176
|
-
memhouse ship -> pass failed: UNKNOWN_TABLE
|
|
177
|
-
```
|
|
178
|
-
|
|
179
|
-
`install` and `doctor` now detect this and name it, rather than telling you to run an
|
|
180
|
-
install you already ran.
|
|
181
|
-
|
|
182
|
-
**Your transcripts are not lost.** memhouse ships FROM your local session stores, so the
|
|
183
|
-
new rooms rebuild from disk and the old tables are left untouched:
|
|
184
|
-
|
|
185
|
-
```
|
|
186
|
-
memhouse install --admin-user <user> --admin-password <pw> --member <you>
|
|
187
|
-
memhouse ship --full
|
|
188
|
-
```
|
|
189
|
-
|
|
190
|
-
The one thing that does not come back that way is a session whose transcript you have
|
|
191
|
-
since deleted locally — it exists only in the old tables. Read those before dropping
|
|
192
|
-
anything: `SELECT * FROM <db>.sessions`.
|
|
193
|
-
|
|
194
|
-
## Where the credential lives
|
|
195
|
-
|
|
196
|
-
`~/.memhouse/env`, mode 0600, `MEMHOUSE_*` keys. **Not** a `.env` in a working directory:
|
|
197
|
-
the skills and docs source this path (`. ~/.memhouse/env`), and a `.env` beside a checkout
|
|
198
|
-
gets committed eventually.
|
|
199
|
-
|
|
200
|
-
The env file is written **last**, after everything above has proved out. Written first, a
|
|
201
|
-
failed install leaves a config behind that the next command reads as truth.
|
|
202
|
-
|
|
203
|
-
`memhouse service install` inlines the same values into the unit file, because a service
|
|
204
|
-
must not depend on a shell-quoted file it does not parse the same way. That is a second
|
|
205
|
-
copy, also 0600, removed by `memhouse service uninstall`.
|
|
206
|
-
|
|
207
|
-
## `--adopt-user` needs house admin — it is not the new-machine path
|
|
208
|
-
|
|
209
|
-
`--adopt-user` is read only on the `--admin-user` branch, so it provisions a member and
|
|
210
|
-
requires admin on the house. Passing it without `--admin-user` used to be ignored silently;
|
|
211
|
-
it is refused now.
|
|
212
|
-
|
|
213
|
-
The actual "same person, new machine" path needs no admin at all — you already have a
|
|
214
|
-
member credential, so just use it:
|
|
215
|
-
|
|
216
|
-
memhouse install --url https://… --user <member> --password …
|
|
217
|
-
|
|
218
|
-
That finds the existing rooms and writes the config. `--adopt-user` is for the case where
|
|
219
|
-
an admin is *creating or re-attaching* a member and wants the existing password verified
|
|
220
|
-
before anything is written.
|
|
221
|
-
|
|
222
|
-
## Offboarding, from the member's side
|
|
223
|
-
|
|
224
|
-
`DROP USER <member>` leaves their rooms and revokes everything (verified: no orphan grants
|
|
225
|
-
survive). From that member's machine every command then fails with `AUTHENTICATION_FAILED`
|
|
226
|
-
and advises reinstalling — which cannot work, and is indistinguishable from a wrong
|
|
227
|
-
password. There is no signal that says "you were offboarded"; ClickHouse does not
|
|
228
|
-
distinguish the two either. Worth knowing before you use it on someone.
|
|
@@ -1,174 +0,0 @@
|
|
|
1
|
-
# per-member rooms
|
|
2
|
-
**v6 reverts one decision v5 made under review pressure.** v5 gave every member a stored
|
|
3
|
-
`sessions_v`. SCHEMA v4 had already decided that the rollup is a *saved query*, and
|
|
4
|
-
overriding that reactively — inside a review round, to fix a read-layer finding — cost a
|
|
5
|
-
name inside the Merge selector's namespace (161 sessions counted as 322), a rename, a
|
|
6
|
-
reserved prefix, a fourth grant and a fourth object to provision and roll forward. The
|
|
7
|
-
query is back. SCHEMA v5 records the detour in full.
|
|
8
|
-
|
|
9
|
-
**v5 corrects the grant set, adds the member's own view, and stops calling this a
|
|
10
|
-
skeleton.** Four changes:
|
|
11
|
-
|
|
12
|
-
1. The grant set was `SELECT, INSERT`, which is insufficient — the shipper clears a
|
|
13
|
-
session's rows before re-inserting, so every incremental pass needs both
|
|
14
|
-
`ALTER UPDATE` and `ALTER DELETE` (which of the two is demanded varies by server
|
|
15
|
-
version). The error surfaced only on the *second* ship of a *changed* session, which is
|
|
16
|
-
why four document versions carried it. See PROVISIONING v5.
|
|
17
|
-
2. The delete predicate **must bind the user** rather than call `currentUser()`. A mutation
|
|
18
|
-
does not evaluate it in the caller's context, so the delete matched nothing and removed
|
|
19
|
-
nothing, silently. Measured on 25.11 and on chdb; it affected the shared layout too.
|
|
20
|
-
3. The read layer resolves the rollup as **SQL text** rather than a name: the stored view
|
|
21
|
-
in the shared layout, a subquery over the caller's rooms in the per-member one. Without
|
|
22
|
-
that resolution a per-member house ships fine and reads back nothing.
|
|
23
|
-
4. v1–v4 were design skeletons. This one is implemented and measured: `rooms.js`,
|
|
24
|
-
`provision.js`, the two DDL templates, and room routing through the read layer.
|
|
25
|
-
|
|
26
|
-
## The model
|
|
27
|
-
|
|
28
|
-
House **`mem`**. Three rooms per member:
|
|
29
|
-
|
|
30
|
-
```
|
|
31
|
-
mem.sessions_<member>
|
|
32
|
-
mem.messages_<member>
|
|
33
|
-
mem.tool_calls_<member>
|
|
34
|
-
```
|
|
35
|
-
|
|
36
|
-
Owner-managed Merge rooms for team-wide reads:
|
|
37
|
-
|
|
38
|
-
```sql
|
|
39
|
-
mem.all_sessions = Merge('mem', '^sessions_')
|
|
40
|
-
mem.all_messages = Merge('mem', '^messages_')
|
|
41
|
-
mem.all_tool_calls = Merge('mem', '^tool_calls_')
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
Canon: house = database, room = table, member = a registered user with grants.
|
|
45
|
-
|
|
46
|
-
## Why
|
|
47
|
-
|
|
48
|
-
**Isolation is structural.** A grant that is absent exposes nothing; a row policy has to
|
|
49
|
-
be correct on every room for every role, and one wrong predicate exposes everything. It
|
|
50
|
-
fails closed.
|
|
51
|
-
|
|
52
|
-
**Sharing is self-serve.** A member holding grant-option on their own rooms shares with a
|
|
53
|
-
colleague directly. Under row policies the same act needs `CREATE ROW POLICY`, which the
|
|
54
|
-
kernel withholds from agency owners.
|
|
55
|
-
|
|
56
|
-
**Personal reads stop paying for everyone else.** The shared rooms order
|
|
57
|
-
`(session_id, user_id, seq)`, so filtering by member cannot prefix-scan. At a hundred
|
|
58
|
-
members every personal query and full-text lookup scans roughly 100× what it needs.
|
|
59
|
-
|
|
60
|
-
**Per-member lifecycle is atomic** — delete, export, retention, and quota are per-room
|
|
61
|
-
operations rather than mutations across a shared table.
|
|
62
|
-
|
|
63
|
-
## Grants
|
|
64
|
-
|
|
65
|
-
Two statements per room, six per member, explicit — plus SELECT on the Merge rooms.
|
|
66
|
-
`PROVISIONING.md` is the authority; this is the shape:
|
|
67
|
-
|
|
68
|
-
```sql
|
|
69
|
-
GRANT ALL ON mem.sessions_<m> TO <m>;
|
|
70
|
-
GRANT SELECT ON mem.sessions_<m> TO <m> WITH GRANT OPTION;
|
|
71
|
-
-- and the same pair for messages_<m> and tool_calls_<m>
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
**The split is the point.** `ALL` covers the mutation privileges the shipper's re-ship
|
|
75
|
-
needs without naming them — clear-then-insert, so a growing transcript cannot leave a
|
|
76
|
-
stale `seq` tail when a re-parse yields fewer rows, and which of `ALTER UPDATE` /
|
|
77
|
-
`ALTER DELETE` a server demands varies by version. Grant-option is attached to the
|
|
78
|
-
`SELECT` statement and **not** to the `ALL` one, so a share is read-only by construction:
|
|
79
|
-
a member cannot hand a colleague `DROP TABLE` while meaning "let them read my sessions".
|
|
80
|
-
Never `ON mem.*` — see PROVISIONING.
|
|
81
|
-
|
|
82
|
-
## What stays
|
|
83
|
-
|
|
84
|
-
- `user_id MATERIALIZED currentUser()` on every room; `async_insert=0` on writers.
|
|
85
|
-
- Typed columns, `extra JSON`, `ReplacingMergeTree(ingested_at)`, `final=1` reads, both
|
|
86
|
-
full-text layers on messages.
|
|
87
|
-
- Parse-on-client; the shipper runs the 17 adapters and writes typed rows.
|
|
88
|
-
- `sessions_v` as a **saved query** run under the caller, not a stored view — the v1–v4
|
|
89
|
-
decision, restored. The read layer resolves it as text in the same place it resolves a
|
|
90
|
-
room name, so the rollup and the rooms travel through one substitution point.
|
|
91
|
-
- The REST contract and the borrowed dashboard.
|
|
92
|
-
|
|
93
|
-
## Measured facts (ClickHouse 26.7.1)
|
|
94
|
-
|
|
95
|
-
1. Members **self-provision** inside their own prefix and are denied outside it.
|
|
96
|
-
2. `Merge` **fails closed on permissions** — a caller sees only the underlying rooms they
|
|
97
|
-
hold grants for, with no leak and no error. This requires SELECT on the Merge room
|
|
98
|
-
ITSELF: without it the team room denies outright instead of narrowing, which is a
|
|
99
|
-
different and worse behaviour. Provisioning grants it (re-measured on 25.11).
|
|
100
|
-
3. `Merge` **auto-discovers** rooms created after it exists.
|
|
101
|
-
4. `Merge` **tolerates schema drift**, returning a column's default for rooms lacking it —
|
|
102
|
-
so schema rollout is progressive, not lock-step. It **rejects mutations**; use
|
|
103
|
-
`ADD COLUMN`, Merge room first.
|
|
104
|
-
5. Row-policy rights are **house-scopeable** (`GRANT ACCESS MANAGEMENT ON mem.*`). Owner
|
|
105
|
-
only — the scope is the house, not a room.
|
|
106
|
-
6. **`EXECUTE AS` exists**, changes `currentUser()`, and is a grantable privilege denied to
|
|
107
|
-
members by default.
|
|
108
|
-
7. **The shipper needs both mutation grants** on its own rooms, or incremental passes
|
|
109
|
-
fail — 26.7 named `ALTER DELETE`, 25.11 named `ALTER UPDATE(_row_exists)`. New in v5.
|
|
110
|
-
8. **`currentUser()` in a mutation predicate matches nothing.** A `DELETE` filtered on
|
|
111
|
-
`user_id = currentUser()` removed 0 rows where the same predicate with the bound value
|
|
112
|
-
removed 2000 — on ClickHouse 25.11 and on chdb alike, with no error either time. New
|
|
113
|
-
in v5.
|
|
114
|
-
|
|
115
|
-
## Measured end-to-end (the implementation, not the design)
|
|
116
|
-
|
|
117
|
-
On ClickHouse 25.11, two members, real local session stores:
|
|
118
|
-
|
|
119
|
-
- alice ships 161 sessions / 22,413 message rows / 16,197 tool rows into her own rooms;
|
|
120
|
-
bob's rooms are untouched.
|
|
121
|
-
- alice reading **or writing** bob's rooms is denied, and so is reading bob's view.
|
|
122
|
-
- the whole read layer resolves: 13 of 14 dashboard query functions run green against
|
|
123
|
-
per-member rooms (the fourteenth needs a SQL argument), and `status`, `search`, `doctor`,
|
|
124
|
-
`stats` all report alice's own data. The same queries against the shared names fail with
|
|
125
|
-
`UNKNOWN_TABLE`, which is what they did before this change.
|
|
126
|
-
- a planted stale `seq` tail is cleared by the next re-ship, which is what the two ALTER
|
|
127
|
-
grants exist for.
|
|
128
|
-
- the whole gauntlet scenario's SQL assertions pass when run by hand against a two-member
|
|
129
|
-
house: 22,413 rows in alice's room, none unstamped, none in bob's, bob reading the team
|
|
130
|
-
Merge room sees zero of alice's, alice sees her own. Sharing round-trips: alice's
|
|
131
|
-
`GRANT` widens bob's Merge view to 22,413 and her `REVOKE` returns it to zero, with the
|
|
132
|
-
direct read denied again.
|
|
133
|
-
- the shared-room layout has been removed; this is the only layout.
|
|
134
|
-
|
|
135
|
-
## `user_id` — what it guarantees
|
|
136
|
-
|
|
137
|
-
Unforgeable **by a member**, since members do not hold `EXECUTE AS`. Forgeable by the
|
|
138
|
-
owner or any grantee. The trust boundary is the owner, not the engine — the normal
|
|
139
|
-
arrangement, but do not overstate it anywhere user-facing. **Members must never hold
|
|
140
|
-
`EXECUTE AS`.**
|
|
141
|
-
|
|
142
|
-
## Phases
|
|
143
|
-
|
|
144
|
-
1. Schema + naming — room DDL template, key order, Merge rooms. **Done.**
|
|
145
|
-
2. Provisioning — kernel capability and standalone path. **Standalone done**
|
|
146
|
-
(`provision.js`); the kernel capability is not written.
|
|
147
|
-
3. Query layer — the member view and room-name resolution. **Done** — `rooms.js` plus
|
|
148
|
-
templated room names through `queries.js`, `ship.js` and the CLI.
|
|
149
|
-
4. Sharing — room-level self-serve; row-level via the owner. **Room-level proven**;
|
|
150
|
-
row-level not implemented.
|
|
151
|
-
|
|
152
|
-
Migration is not a phase and must not shape MVP decisions.
|
|
153
|
-
|
|
154
|
-
## Open questions
|
|
155
|
-
|
|
156
|
-
- **May members create and drop their own rooms?** Works, safely scoped; but a member
|
|
157
|
-
could drop their own memory and schema rollout stops being central. Deferred with the
|
|
158
|
-
uninstall/drop work.
|
|
159
|
-
- **Key order** — is `user_id` still worth an `ORDER BY` slot now the room is the tenant?
|
|
160
|
-
- **Standalone path** — a user with no kernel is both owner and member; one credential or
|
|
161
|
-
two?
|
|
162
|
-
- **Is `EXECUTE AS` scopeable** to `mem.*` rather than `*.*`? Bounds what an owner
|
|
163
|
-
credential can impersonate.
|
|
164
|
-
- ~~**Should a share be `SELECT`-only?**~~ **RESOLVED.** Grant-option is attached to the
|
|
165
|
-
member's `SELECT` grant and not to their `ALL` grant, so a share can only ever be
|
|
166
|
-
read-only. Measured on 26.7.2: `GRANT SELECT … TO bob` succeeds,
|
|
167
|
-
`GRANT DROP TABLE … TO bob` is refused 497.
|
|
168
|
-
|
|
169
|
-
## Out of scope
|
|
170
|
-
|
|
171
|
-
- Migration. Deferred; unblocked in principle by `EXECUTE AS`.
|
|
172
|
-
- `agent_id` / crew identity. Within a crew sharing one credential an agent tag is
|
|
173
|
-
client-supplied and is not a security boundary.
|
|
174
|
-
- Adapters, REST contract, dashboard.
|
|
@@ -1,177 +0,0 @@
|
|
|
1
|
-
# PROVISIONING — minting a member and their rooms
|
|
2
|
-
|
|
3
|
-
Two statements per room, six per member, and no seventh for the rollup: the session
|
|
4
|
-
rollup is a saved query over exactly these rooms, so a member who can read them can run
|
|
5
|
-
it. See [INSTALL.md](INSTALL.md) for the three ways this gets applied.
|
|
6
|
-
|
|
7
|
-
## A note on names before the SQL
|
|
8
|
-
|
|
9
|
-
The house is `mem`. `provision.js`, the shipper, the read layer and the CLI all default to
|
|
10
|
-
it, so the SQL below is what a default install actually provisions. The owner is
|
|
11
|
-
`memhouse_root` — the **database** was shortened to `mem`, the owning role was not.
|
|
12
|
-
|
|
13
|
-
They must agree or rooms get provisioned somewhere the product never reads. Override with
|
|
14
|
-
`MEM_DB` / `MEM_USER` (or the `MEMHOUSE_*` equivalents); `provision.js` prints the
|
|
15
|
-
connection it resolved so a mismatch is visible rather than silent.
|
|
16
|
-
|
|
17
|
-
An install predating this default carries `MEMHOUSE_DB` in `~/.memhouse/env`, and explicit
|
|
18
|
-
config outranks the default, so nothing repoints you at a different house. **That does not
|
|
19
|
-
mean a 0.3.x house keeps working** — the database name survives the upgrade, the tables do
|
|
20
|
-
not. See [INSTALL.md](INSTALL.md#upgrading-from-03x--read-this-before-you-do).
|
|
21
|
-
|
|
22
|
-
## The grant set
|
|
23
|
-
|
|
24
|
-
Two statements per room. The split is the point.
|
|
25
|
-
|
|
26
|
-
```sql
|
|
27
|
-
GRANT ALL ON mem.sessions_<m> TO <m>;
|
|
28
|
-
GRANT SELECT ON mem.sessions_<m> TO <m> WITH GRANT OPTION;
|
|
29
|
-
GRANT ALL ON mem.messages_<m> TO <m>;
|
|
30
|
-
GRANT SELECT ON mem.messages_<m> TO <m> WITH GRANT OPTION;
|
|
31
|
-
GRANT ALL ON mem.tool_calls_<m> TO <m>;
|
|
32
|
-
GRANT SELECT ON mem.tool_calls_<m> TO <m> WITH GRANT OPTION;
|
|
33
|
-
```
|
|
34
|
-
|
|
35
|
-
Plus `SELECT` on whichever Merge rooms exist, which is what makes the team room narrow to
|
|
36
|
-
the caller's own rooms instead of denying outright.
|
|
37
|
-
|
|
38
|
-
**`ALL`, not re-grantable.** The member owns their room outright — `DROP` and `TRUNCATE`
|
|
39
|
-
included, because it is their memory to destroy. It also covers the mutation privileges
|
|
40
|
-
below without naming them, which is what stops the version-dependent trap.
|
|
41
|
-
|
|
42
|
-
**`SELECT`, re-grantable.** A share is read-only **by construction**. `GRANT ALL … WITH
|
|
43
|
-
GRANT OPTION` expands to 45 privileges on 26.7 — `DROP TABLE`, `TRUNCATE`,
|
|
44
|
-
`CREATE ROW POLICY`, `SYSTEM DROP REPLICA` among them — every one of which a member could
|
|
45
|
-
then hand to a colleague while meaning "let them read my sessions". Measured on 26.7.2:
|
|
46
|
-
`GRANT SELECT … TO bob` succeeds, `GRANT DROP TABLE … TO bob` is refused 497, and the
|
|
47
|
-
member can still drop their own room.
|
|
48
|
-
|
|
49
|
-
Never `ON mem.*`. A member who can read `mem.*` can read every other member's rooms, and
|
|
50
|
-
then this is a shared house with longer table names.
|
|
51
|
-
|
|
52
|
-
### Why the mutation grants are not optional
|
|
53
|
-
|
|
54
|
-
v4 said `SELECT, INSERT`. A member provisioned that way ships **once** and then fails on
|
|
55
|
-
every subsequent pass.
|
|
56
|
-
|
|
57
|
-
Both `ALTER` privileges are needed, and **which one bites depends on the server version**.
|
|
58
|
-
`DELETE FROM` is a *lightweight* delete, implemented as
|
|
59
|
-
`ALTER TABLE ... UPDATE _row_exists = 0`: ClickHouse 26.7 refused it for want of
|
|
60
|
-
`ALTER DELETE`, and 25.11 refused the same statement for want of
|
|
61
|
-
`ALTER UPDATE(_row_exists)`. Granting the one you happened to test against produces a
|
|
62
|
-
member who cannot ship on the other server, so grant both.
|
|
63
|
-
|
|
64
|
-
The shipper is not append-only. A session file grows while the agent works, so each pass
|
|
65
|
-
re-parses the whole transcript and re-ships it; to keep that idempotent it **clears the
|
|
66
|
-
session's existing rows and re-inserts them**. `ReplacingMergeTree(ingested_at)` collapses
|
|
67
|
-
duplicates by key eventually, but it cannot remove a row the new parse no longer produces —
|
|
68
|
-
a re-parse that yields fewer rows would otherwise leave a stale `seq` tail behind forever.
|
|
69
|
-
So the clear is a real `ALTER TABLE ... DELETE`, and without the privilege the first
|
|
70
|
-
incremental pass dies with
|
|
71
|
-
|
|
72
|
-
```
|
|
73
|
-
Not enough privileges. To execute this query, it's necessary to have the grant
|
|
74
|
-
ALTER UPDATE(_row_exists) ON mem.messages_<m>
|
|
75
|
-
```
|
|
76
|
-
|
|
77
|
-
The failure is invisible in a first-run test, which is exactly why it survived four
|
|
78
|
-
document versions: provision, ship, count rows, declare victory. It only appears on the
|
|
79
|
-
*second* ship of a *changed* session.
|
|
80
|
-
|
|
81
|
-
### The delete predicate must bind the user, not call `currentUser()`
|
|
82
|
-
|
|
83
|
-
Related, and worse, because it fails **silently**. The clear was written as
|
|
84
|
-
|
|
85
|
-
```sql
|
|
86
|
-
DELETE FROM messages_<m> WHERE session_id = {id:String} AND user_id = currentUser()
|
|
87
|
-
```
|
|
88
|
-
|
|
89
|
-
A mutation does not evaluate `currentUser()` in the caller's context. Measured on
|
|
90
|
-
ClickHouse 25.11 and on chdb: the identical predicate with the literal value deleted 2000
|
|
91
|
-
rows where `currentUser()` deleted **0** — no error, no warning, nothing in the log. The
|
|
92
|
-
stale tail the delete exists to remove simply survives. The value is bound instead, read
|
|
93
|
-
from the server over the same connection, so it is the same identity with none of the
|
|
94
|
-
context dependence. This affected the **shared** layout too, and predates this fork.
|
|
95
|
-
|
|
96
|
-
### What the privilege lets a member do
|
|
97
|
-
|
|
98
|
-
Delete and mutate rows in their own rooms. That is their own memory, and they can already
|
|
99
|
-
overwrite it by re-shipping — so this widens what a member can destroy about themselves,
|
|
100
|
-
and nothing about anyone else. Grants remain per-room and explicit, so `ALTER DELETE` on
|
|
101
|
-
`messages_alice` says nothing about `messages_bob`.
|
|
102
|
-
|
|
103
|
-
`DROP TABLE` is included too (see below): a member can remove their own room, not just
|
|
104
|
-
empty it.
|
|
105
|
-
|
|
106
|
-
### Consequence of `WITH GRANT OPTION`
|
|
107
|
-
|
|
108
|
-
Grant-option covers every privilege **in the statement it is attached to**, which is why
|
|
109
|
-
it is attached to the `SELECT` statement and not the `ALL` one. Alice can hand bob read
|
|
110
|
-
access to `messages_alice` and nothing else; she cannot hand him the mutation or drop
|
|
111
|
-
rights even by mistake. The convention `SHARING` describes is now enforced by the engine
|
|
112
|
-
rather than by everyone remembering it.
|
|
113
|
-
|
|
114
|
-
A share is the three rooms and nothing else. The recipient runs the same rollup query over
|
|
115
|
-
them under their own credential, so there is no fourth grant to forget — which is the
|
|
116
|
-
concrete reason the query beat the view.
|
|
117
|
-
|
|
118
|
-
`WITH GRANT OPTION` is what makes whole-room sharing self-serve. Without it every share
|
|
119
|
-
becomes an operator ticket, which is most of the reason this fork exists.
|
|
120
|
-
|
|
121
|
-
The owner (`memhouse_root` — the house was shortened to `mem`, the owning role was not)
|
|
122
|
-
holds `ALL ON mem.* WITH GRANT OPTION`, so a second member joining
|
|
123
|
-
an existing house needs no new owner setup — house, owner, and Merge rooms are already
|
|
124
|
-
there.
|
|
125
|
-
|
|
126
|
-
**RESOLVED:** the member does get `CREATE TABLE` / `DROP TABLE` on their own rooms — it
|
|
127
|
-
is inside `ALL`. That is what lets `--ensure-schema` run as the member and makes the solo
|
|
128
|
-
install one command with no owner involvement. The cost is accepted: a member can drop
|
|
129
|
-
their own memory. The thing that was actually worth preventing — a member handing those
|
|
130
|
-
rights to someone else — is prevented by the grant-option split above.
|
|
131
|
-
|
|
132
|
-
## Kernel path
|
|
133
|
-
|
|
134
|
-
**Not finalized.** The realm holds the privileged credential and mints members; what the
|
|
135
|
-
installer asks it for and what it gets back is not designed yet.
|
|
136
|
-
|
|
137
|
-
What is settled regardless: the room layout, the grant set above, and the identity rule
|
|
138
|
-
(the ClickHouse user is the identity). What is open is only who plays admin and how the
|
|
139
|
-
handle is chosen. Whatever that turns out to be, provisioning wants to be ONE idempotent
|
|
140
|
-
step — three rooms plus their grants — because it is also the hook for rolling a schema
|
|
141
|
-
change across rooms.
|
|
142
|
-
|
|
143
|
-
Nothing touches the Merge rooms on member join: they are a regex over room names and
|
|
144
|
-
**auto-discover** rooms created after them.
|
|
145
|
-
|
|
146
|
-
## Standalone path
|
|
147
|
-
|
|
148
|
-
No kernel, so no capability layer; the CLI performs the same steps. `provision.js` is that
|
|
149
|
-
path today: it creates the house with a client that has *not* selected it (on a fresh
|
|
150
|
-
server the database does not exist yet, and selecting it fails before it can be created),
|
|
151
|
-
then the rooms, then the grants.
|
|
152
|
-
|
|
153
|
-
> **OPEN:** a standalone user is simultaneously owner and member. One credential holding
|
|
154
|
-
> both roles, or two? Today's `memhouse install` assumes a single credential that owns
|
|
155
|
-
> everything, which no longer maps cleanly.
|
|
156
|
-
|
|
157
|
-
## Row-policy rights
|
|
158
|
-
|
|
159
|
-
`GRANT ACCESS MANAGEMENT ON mem.*` expands to
|
|
160
|
-
`CREATE/ALTER/DROP/SHOW ROW POLICY ON mem.*` — house-scoped, no global access management
|
|
161
|
-
required. This is what lets the owner serve partial-share requests.
|
|
162
|
-
|
|
163
|
-
**Owner only.** The scope is the house, not a room, so any holder could attach a policy to
|
|
164
|
-
another member's rooms. Policies filter rather than grant, so it is not a data leak — but
|
|
165
|
-
it is a denial of service on a colleague's own reads.
|
|
166
|
-
|
|
167
|
-
## `EXECUTE AS`
|
|
168
|
-
|
|
169
|
-
Owner may hold it; **members must not**. It changes `currentUser()`, so a member holding
|
|
170
|
-
it could write rows stamped as someone else and defeat the provenance the schema relies
|
|
171
|
-
on.
|
|
172
|
-
|
|
173
|
-
## Not yet written
|
|
174
|
-
|
|
175
|
-
- the capability script
|
|
176
|
-
- member removal: drop the three rooms, revoke, and decide what happens to rooms they had
|
|
177
|
-
shared outward
|
|
@@ -1,48 +0,0 @@
|
|
|
1
|
-
# per-member rooms
|
|
2
|
-
|
|
3
|
-
**One house, a set of rooms per member.** This replaced a shared layout — three rooms
|
|
4
|
-
named `sessions`/`messages`/`tool_calls` for everyone, separated by row policies — which
|
|
5
|
-
has been removed. There is no switch and no fallback: this is the data model.
|
|
6
|
-
|
|
7
|
-
**The design is implemented** — `rooms.js`, `provision.js`, the two DDL templates, and
|
|
8
|
-
room routing through the read layer — and proven end to end on ClickHouse 26.7.1 and
|
|
9
|
-
25.11.
|
|
10
|
-
|
|
11
|
-
| Document | What |
|
|
12
|
-
|---|---|
|
|
13
|
-
| [`PLAN.md`](PLAN.md) | the model, why, phases, measured facts |
|
|
14
|
-
| [`PROVISIONING.md`](PROVISIONING.md) | the grant set (three per member), kernel capability, standalone path |
|
|
15
|
-
| [`SCHEMA.md`](SCHEMA.md) | room naming, shapes, sort keys, Merge rooms, and why `sessions_v` is a saved query |
|
|
16
|
-
| [`SHARING.md`](SHARING.md) | whole-room self-serve (three grants), partial rows via the owner |
|
|
17
|
-
|
|
18
|
-
These four documents describe the design as it stands. Earlier revisions are in
|
|
19
|
-
`git log`, not in this directory — a superseded document kept beside the current one is a
|
|
20
|
-
contradiction waiting to be quoted, and this directory produced two review findings that
|
|
21
|
-
way before the versions were removed.
|
|
22
|
-
|
|
23
|
-
## The one-paragraph version
|
|
24
|
-
|
|
25
|
-
The house is **`mem`**. Each member gets `mem.sessions_<member>`, `mem.messages_<member>`,
|
|
26
|
-
`mem.tool_calls_<member>`, with three grants over them —
|
|
27
|
-
`SELECT, INSERT, ALTER UPDATE, ALTER DELETE`, all `WITH GRANT OPTION`. The session rollup
|
|
28
|
-
is a saved query over those same rooms, not a fourth object. Isolation stops
|
|
29
|
-
being a row policy that must be right everywhere and becomes a grant that is simply
|
|
30
|
-
absent — it fails closed. Sharing a whole room needs no operator. Team-wide reads come
|
|
31
|
-
from `Merge` rooms anchored on the room type (`^messages_`), which reduce to each caller's
|
|
32
|
-
own grants, auto-discover new members, and tolerate schema drift — provided the caller
|
|
33
|
-
holds `SELECT` on the Merge room itself, which provisioning issues; without it the team
|
|
34
|
-
room denies instead of narrowing.
|
|
35
|
-
`user_id MATERIALIZED currentUser()` stays on every room: unforgeable by members, who are
|
|
36
|
-
not granted `EXECUTE AS`.
|
|
37
|
-
|
|
38
|
-
## Biggest open questions
|
|
39
|
-
|
|
40
|
-
1. **May members create and drop their own rooms?** It works and is safely scoped, but it
|
|
41
|
-
lets a member drop their own memory and makes schema rollout non-central.
|
|
42
|
-
2. **Key order** — is `user_id` still worth an `ORDER BY` slot now the room is the tenant?
|
|
43
|
-
3. **Standalone path** — a user with no kernel is both owner and member; one credential or
|
|
44
|
-
two?
|
|
45
|
-
4. **Is `EXECUTE AS` scopeable** to `mem.*` rather than `*.*`?
|
|
46
|
-
5. **Is a saved rollup enough for agents?** It has no name to type. `memhouse
|
|
47
|
-
sessions-query` prints it, which is a worse affordance than a name and the price of
|
|
48
|
-
having nothing to own, grant or collide.
|