memhouse 0.3.1 → 0.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +83 -6
- package/bin/memhouse.js +882 -40
- package/editors/zed.js +55 -6
- package/index.js +2 -2
- package/mem-house/COMPETITION.md +217 -0
- package/mem-house/DESIGN.md +61 -32
- package/mem-house/delivery/AGENT-INSTALL.md +23 -3
- package/mem-house/delivery/PROMPT.md +23 -10
- package/mem-house/delivery/install.sh +9 -7
- package/mem-house/delivery/kernel-install.md +56 -25
- package/mem-house/delivery/skills/memhouse-search/SKILL.md +46 -16
- package/mem-house/delivery/skills/memhouse-sessions/SKILL.md +38 -7
- package/mem-house/delivery/skills/memhouse-sql/SKILL.md +48 -17
- package/mem-house/deploy.js +333 -0
- package/mem-house/envfile.js +49 -0
- package/mem-house/per-member/INSTALL.md +183 -0
- package/mem-house/per-member/PLAN.md +174 -0
- package/mem-house/per-member/PROVISIONING.md +177 -0
- package/mem-house/per-member/README.md +48 -0
- package/mem-house/per-member/SCHEMA.md +73 -0
- package/mem-house/per-member/SHARING.md +102 -0
- package/mem-house/per-member/provision.js +142 -0
- package/mem-house/per-member/rooms.js +132 -0
- package/mem-house/per-member/schema-member.sql.tpl +85 -0
- package/mem-house/per-member/schema-merge.sql.tpl +24 -0
- package/mem-house/server/queries.js +103 -53
- package/mem-house/service.js +400 -0
- package/mem-house/shipper/ship.js +61 -18
- package/package.json +3 -3
- package/pricing.js +20 -2
- package/pricing.json +216 -24
- package/public/assets/{index-CCPh7a2Z.js → index-CtH-5i95.js} +5 -5
- package/public/index.html +1 -1
- package/ui/src/App.jsx +3 -2
- package/ui/src/pages/SqlViewer.jsx +39 -21
- package/mem-house/rls.sql +0 -17
- package/mem-house/schema.sql +0 -145
|
@@ -2,14 +2,23 @@
|
|
|
2
2
|
|
|
3
3
|
How to run mem-house as an **agency** on an ultimagent kernel — a ClickHouse server
|
|
4
4
|
(a **town**) promoted into a realm — instead of a plain ClickHouse. Same flow proven
|
|
5
|
-
for the agentlytics agency (`agency/AGENCY.md`); the agency
|
|
6
|
-
**`
|
|
7
|
-
`
|
|
8
|
-
`on = "loop"`) — that
|
|
9
|
-
|
|
5
|
+
for the agentlytics agency (`agency/AGENCY.md`); the agency is named **`memhouse`** and
|
|
6
|
+
its house is the **`mem`** database (CH identifiers can't carry a dash, and the owning
|
|
7
|
+
role kept the longer name — `memhouse_root`). What lands is a **house** (the `mem`
|
|
8
|
+
database) with a **resident** (the shipper — `kind = "worker"`, `on = "loop"`) — that
|
|
9
|
+
pairing is the agency. Residents write; routines read, and the session rollup is a
|
|
10
|
+
routine. Terms: `../../TERMINOLOGY.md`.
|
|
11
|
+
|
|
12
|
+
**One set of rooms per member.** Each member owns `sessions_<them>`, `messages_<them>`,
|
|
13
|
+
`tool_calls_<them>` and holds grants on those and nothing else, so isolation is an
|
|
14
|
+
absent grant rather than a row policy — it fails closed. There is no shared
|
|
15
|
+
`sessions`/`messages`/`tool_calls` layout and no row policy in the default path; 0.4.0
|
|
16
|
+
removed both. Team-wide reads are the `Merge` rooms plus a `GRANT`. The session rollup
|
|
17
|
+
is a **saved query**, not a stored view — `memhouse sessions-query` prints it.
|
|
18
|
+
Design and measurements: `../per-member/`.
|
|
10
19
|
|
|
11
20
|
This is an operator runbook, so it speaks machine vocabulary throughout — ClickHouse,
|
|
12
|
-
`GRANT`,
|
|
21
|
+
`GRANT`, `currentUser()`. That is correct for this audience.
|
|
13
22
|
|
|
14
23
|
## 1. Provision (operator, one-time — from the kernel checkout)
|
|
15
24
|
|
|
@@ -17,8 +26,8 @@ This is an operator runbook, so it speaks machine vocabulary throughout — Clic
|
|
|
17
26
|
python3 executor/executor.py provision memhouse
|
|
18
27
|
```
|
|
19
28
|
|
|
20
|
-
Submits `install-agency{name:'memhouse'}`; the
|
|
21
|
-
|
|
29
|
+
Submits `install-agency{name:'memhouse'}`; the human owner approves; the executor
|
|
30
|
+
(running as the realm's privileged user) creates database `mem`
|
|
22
31
|
(the house), owner `memhouse_root`
|
|
23
32
|
(`realm_user` profile), and owner grants. It returns a one-time **credential** —
|
|
24
33
|
rotate on first connect (`issue-credential{user:'memhouse_root'}`).
|
|
@@ -31,7 +40,7 @@ memhouse install --yes \
|
|
|
31
40
|
--url https://<kernel-host>:8443 \
|
|
32
41
|
--user memhouse_root \
|
|
33
42
|
--password <credential-from-provision> \
|
|
34
|
-
--db
|
|
43
|
+
--db mem
|
|
35
44
|
```
|
|
36
45
|
|
|
37
46
|
That writes `~/.memhouse/env`, applies the schema, and runs the first ship.
|
|
@@ -50,34 +59,56 @@ injection surface). It is the only thing here that writes without being asked.
|
|
|
50
59
|
For each person joining, split across the two authorities:
|
|
51
60
|
|
|
52
61
|
```bash
|
|
53
|
-
# The
|
|
62
|
+
# The realm's privileged user mints the identity; the human owner approves:
|
|
54
63
|
python3 executor/executor.py submit register-member '{"handle":"alice"}'
|
|
55
64
|
python3 executor/executor.py approve <call_id> && python3 executor/executor.py drain
|
|
56
65
|
|
|
57
|
-
# OWNER
|
|
58
|
-
#
|
|
66
|
+
# OWNER mints that member's three rooms and their grants (memhouse_root has
|
|
67
|
+
# grant-option on mem.*). One idempotent step — see ../per-member/PROVISIONING.md:
|
|
68
|
+
MEM_URL=https://<kernel-host>:8443 MEM_USER=memhouse_root \
|
|
69
|
+
MEM_PASSWORD=<credential> MEM_DB=mem \
|
|
70
|
+
node mem-house/per-member/provision.js --member alice --merge
|
|
59
71
|
```
|
|
60
72
|
|
|
73
|
+
**Never `GRANT … ON mem.*` to a member.** A member who can read `mem.*` can read every
|
|
74
|
+
other member's rooms, and then this is a shared house with longer table names.
|
|
75
|
+
`provision.js` grants per room: `ALL` on each of alice's three, plus a re-grantable
|
|
76
|
+
`SELECT` on each (that split is what makes a share read-only by construction), plus
|
|
77
|
+
`SELECT` on the `Merge` rooms.
|
|
78
|
+
|
|
61
79
|
Each member then runs the shipper with **their own** credential
|
|
62
|
-
(`MEMHOUSE_USER=alice`) —
|
|
63
|
-
|
|
80
|
+
(`MEMHOUSE_USER=alice`) — it writes to `sessions_alice` / `messages_alice` /
|
|
81
|
+
`tool_calls_alice`, resolved from `SELECT currentUser()` rather than from config, and
|
|
82
|
+
the house stamps `user_id='alice'` on their rows, un-spoofably
|
|
83
|
+
(`MATERIALIZED currentUser()`, `async_insert=0`).
|
|
84
|
+
|
|
85
|
+
## 4. Visibility
|
|
86
|
+
|
|
87
|
+
**Own-only is the default and needs no policy.** Alice is granted her own three rooms
|
|
88
|
+
and nobody else's, so bob's rooms are not hidden from her — they are simply not hers to
|
|
89
|
+
read, and the failure mode of a missing grant is a denial rather than a leak.
|
|
64
90
|
|
|
65
|
-
|
|
91
|
+
**Team-wide reads are the `Merge` rooms** (`all_sessions`, `all_messages`,
|
|
92
|
+
`all_tool_calls`) plus a `GRANT SELECT` on them, which `provision.js --merge` issues. A
|
|
93
|
+
`Merge` room narrows to whatever underlying rooms the caller already holds grants for —
|
|
94
|
+
measured on 26.7.1: no leak, no error — so it can be granted broadly. It also
|
|
95
|
+
auto-discovers rooms created after it, so onboarding a member needs no DDL there.
|
|
66
96
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
ACCESS MANAGEMENT, which the kernel withholds from agency owners by design.
|
|
71
|
-
- **Team pool** (everyone sees everything): apply no policy; the owner GRANTs from
|
|
72
|
-
step 3 are the whole model.
|
|
97
|
+
A member widens what a colleague sees by granting their own rooms directly
|
|
98
|
+
(`GRANT SELECT ON mem.messages_alice TO bob`) — self-serve, no operator, because the
|
|
99
|
+
member holds grant-option on their own `SELECT`. See `../per-member/SHARING.md`.
|
|
73
100
|
|
|
74
|
-
|
|
101
|
+
Row policies are still the only way to share a *subset* of rows, which is owner-mediated
|
|
102
|
+
and documented in `SHARING.md` — they are not how isolation works.
|
|
75
103
|
|
|
76
104
|
## 5. Verify
|
|
77
105
|
|
|
78
106
|
```bash
|
|
79
107
|
memhouse stats # per-source counts as the owner
|
|
80
|
-
# as a member
|
|
81
|
-
curl -s -u "alice:<pw>" "$MEMHOUSE_URL/?database=
|
|
82
|
-
--data-binary "SELECT count() FROM
|
|
108
|
+
# as a member: the rollup resolves to alice's own rooms, so counts are hers alone
|
|
109
|
+
curl -s -u "alice:<pw>" "$MEMHOUSE_URL/?database=mem" \
|
|
110
|
+
--data-binary "SELECT count() FROM $(memhouse sessions-query) AS c SETTINGS final=1, join_use_nulls=1"
|
|
111
|
+
# and the team room narrows rather than denying — alice sees herself plus whoever granted her
|
|
112
|
+
curl -s -u "alice:<pw>" "$MEMHOUSE_URL/?database=mem" \
|
|
113
|
+
--data-binary "SELECT user_id, count() FROM all_messages GROUP BY user_id SETTINGS final=1"
|
|
83
114
|
```
|
|
@@ -8,20 +8,21 @@ allowed-tools: Bash(set -a*), Bash(. *), Bash(curl*)
|
|
|
8
8
|
|
|
9
9
|
# memhouse-search — search conversation memory
|
|
10
10
|
|
|
11
|
-
Search the full message history in the memhouse house
|
|
12
|
-
|
|
11
|
+
Search the full message history in the memhouse house. Every room is named for your
|
|
12
|
+
ClickHouse user — `messages_<you>`, `sessions_<you>` — see **Room names** below.
|
|
13
13
|
|
|
14
14
|
## Connection
|
|
15
15
|
|
|
16
16
|
Credentials come from `~/.memhouse/env` (or already-exported `MEMHOUSE_*` vars).
|
|
17
|
-
Every query runs over ClickHouse HTTP with `final=1`
|
|
18
|
-
stale row versions until merges; `final=1` collapses to
|
|
17
|
+
Every query runs over ClickHouse HTTP with `final=1` and `join_use_nulls=1`
|
|
18
|
+
(ReplacingMergeTree keeps stale row versions until merges; `final=1` collapses to
|
|
19
|
+
latest-wins — always
|
|
19
20
|
include it on reads):
|
|
20
21
|
|
|
21
22
|
```bash
|
|
22
23
|
set -a; [ -f ~/.memhouse/env ] && . ~/.memhouse/env; set +a
|
|
23
24
|
curl -sS --fail-with-body --user "${MEMHOUSE_USER:-memhouse_root}:${MEMHOUSE_PASSWORD:-}" \
|
|
24
|
-
--data-binary @- "${MEMHOUSE_URL:-http://localhost:8123}/?database=${MEMHOUSE_DB:-
|
|
25
|
+
--data-binary @- "${MEMHOUSE_URL:-http://localhost:8123}/?database=${MEMHOUSE_DB:-mem}&final=1&join_use_nulls=1" <<'SQL'
|
|
25
26
|
<the query>
|
|
26
27
|
FORMAT PrettyCompact
|
|
27
28
|
SQL
|
|
@@ -30,9 +31,38 @@ SQL
|
|
|
30
31
|
`FORMAT PrettyCompact` for display; `FORMAT JSONEachRow` when you want to parse
|
|
31
32
|
the rows yourself.
|
|
32
33
|
|
|
34
|
+
## Room names — always suffixed
|
|
35
|
+
|
|
36
|
+
Every member owns their own rooms, named for their ClickHouse user:
|
|
37
|
+
`messages_<you>`, `sessions_<you>`, `tool_calls_<you>`. There are no unsuffixed rooms;
|
|
38
|
+
the bare names fail with `UNKNOWN_TABLE` rather than returning nothing. You hold no grant
|
|
39
|
+
on anyone else's rooms, so isolation is not something a query can work around.
|
|
40
|
+
|
|
41
|
+
Resolve your own name once and substitute it into every table name below:
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
MEM_ME="$(curl -sS --fail-with-body --user "$MEMHOUSE_USER:$MEMHOUSE_PASSWORD" \
|
|
45
|
+
--data-binary "SELECT currentUser() FORMAT TabSeparated" \
|
|
46
|
+
"${MEMHOUSE_URL:-http://localhost:8123}/" | tr -d '\r\n')"
|
|
47
|
+
# rooms: messages_$MEM_ME, sessions_$MEM_ME, tool_calls_$MEM_ME
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
**There is no `sessions_v` object.** The session rollup is a saved query over those same
|
|
51
|
+
rooms — `memhouse sessions-query` prints it for whoever you are connected as, ready to
|
|
52
|
+
paste into a `FROM (...) AS c` position.
|
|
53
|
+
|
|
54
|
+
**Read the rollup with `join_use_nulls=1`.** The connection recipe above sets it. Without
|
|
55
|
+
it, ClickHouse gives an unmatched `m.seq` a default instead of NULL, so a session with no
|
|
56
|
+
messages reports `total_msgs = 1` rather than 0 — measured, not theoretical. A subquery
|
|
57
|
+
has no `SETTINGS` clause of its own, so the setting has to come from the caller.
|
|
58
|
+
|
|
59
|
+
If you hold `SELECT` on them, `all_messages` / `all_sessions` / `all_tool_calls` read
|
|
60
|
+
across every member at once, narrowed to whatever grants you actually have — a Merge room
|
|
61
|
+
reduces to the rooms the caller can read, so it fails closed rather than denying outright.
|
|
62
|
+
|
|
33
63
|
## How to search (the FTS columns)
|
|
34
64
|
|
|
35
|
-
`
|
|
65
|
+
`messages_<you>` carries two MATERIALIZED lowercase copies of `text`, each with a
|
|
36
66
|
text index — search those, display `text`:
|
|
37
67
|
|
|
38
68
|
- `text_ngram` (ngram index) → substring match: `text_ngram LIKE '%term%'`
|
|
@@ -48,8 +78,8 @@ phrase → one `LIKE` clause for the whole phrase.
|
|
|
48
78
|
SELECT m.ts, m.source, m.project, m.role, m.session_id,
|
|
49
79
|
s.name AS session,
|
|
50
80
|
substring(m.text, 1, 300) AS snippet
|
|
51
|
-
FROM
|
|
52
|
-
LEFT JOIN
|
|
81
|
+
FROM messages_<you> AS m
|
|
82
|
+
LEFT JOIN sessions_<you> AS s USING (session_id)
|
|
53
83
|
WHERE m.text_ngram LIKE '%postgres%' AND m.text_ngram LIKE '%migration%'
|
|
54
84
|
ORDER BY m.ts DESC
|
|
55
85
|
LIMIT 30
|
|
@@ -60,7 +90,7 @@ FORMAT PrettyCompact
|
|
|
60
90
|
|
|
61
91
|
```sql
|
|
62
92
|
SELECT ts, project, session_id, substring(text, 1, 200) AS snippet
|
|
63
|
-
FROM
|
|
93
|
+
FROM messages_<you>
|
|
64
94
|
WHERE hasToken(text_word, 'rls')
|
|
65
95
|
AND source = 'claude-code'
|
|
66
96
|
AND ts > now() - INTERVAL 14 DAY
|
|
@@ -74,8 +104,8 @@ FORMAT PrettyCompact
|
|
|
74
104
|
```sql
|
|
75
105
|
SELECT m.session_id, any(s.name) AS session, any(m.project) AS project,
|
|
76
106
|
count() AS hits, max(m.ts) AS last_hit
|
|
77
|
-
FROM
|
|
78
|
-
LEFT JOIN
|
|
107
|
+
FROM messages_<you> AS m
|
|
108
|
+
LEFT JOIN sessions_<you> AS s USING (session_id)
|
|
79
109
|
WHERE m.text_ngram LIKE '%clickhouse cache%'
|
|
80
110
|
GROUP BY m.session_id
|
|
81
111
|
ORDER BY hits DESC
|
|
@@ -91,14 +121,14 @@ FORMAT PrettyCompact
|
|
|
91
121
|
`cursor`, `cursor-agent`, `vscode`, `zed`, `opencode`, `gemini-cli`,
|
|
92
122
|
`windsurf`, `antigravity`, `copilot-cli`, `goose`, `kiro`, …)
|
|
93
123
|
- Skip subagent noise → `AND NOT m.is_subagent`
|
|
94
|
-
- Only user/assistant text lives in `
|
|
95
|
-
`
|
|
96
|
-
if the user explicitly wants tool calls searched.
|
|
124
|
+
- Only user/assistant text lives in `messages_<you>`; tool invocations are in
|
|
125
|
+
`tool_calls_<you>` (`tool_name`, `args`) — search `args ILIKE '%term%'` there
|
|
126
|
+
only if the user explicitly wants tool calls searched.
|
|
97
127
|
|
|
98
128
|
## Output
|
|
99
129
|
|
|
100
130
|
Summarize hits grouped by session (name, source, project, date, 1–2 best
|
|
101
131
|
snippets each) and give the total hit count. Offer to pull the full transcript
|
|
102
|
-
of the best session (`SELECT role, text FROM
|
|
132
|
+
of the best session (`SELECT role, text FROM messages_<you> WHERE session_id = '…'
|
|
103
133
|
ORDER BY seq`). If zero hits, retry with broader/fewer terms and report the
|
|
104
|
-
corpus size (`SELECT count() FROM
|
|
134
|
+
corpus size (`SELECT count() FROM messages_<you>`).
|
|
@@ -8,7 +8,8 @@ allowed-tools: Bash(set -a*), Bash(. *), Bash(curl*)
|
|
|
8
8
|
|
|
9
9
|
# memhouse-sessions — browse session history
|
|
10
10
|
|
|
11
|
-
List/filter sessions from the
|
|
11
|
+
List/filter sessions from the session rollup — a saved query over your own rooms, not an
|
|
12
|
+
object; `memhouse sessions-query` prints it (session metadata +
|
|
12
13
|
message aggregates) in the memhouse house.
|
|
13
14
|
|
|
14
15
|
## When to use
|
|
@@ -20,17 +21,47 @@ message aggregates) in the memhouse house.
|
|
|
20
21
|
## Connection
|
|
21
22
|
|
|
22
23
|
Credentials from `~/.memhouse/env` (or exported `MEMHOUSE_*`). Always read with
|
|
23
|
-
`final=1` (collapses ReplacingMergeTree duplicates to latest-wins)
|
|
24
|
+
`final=1` (collapses ReplacingMergeTree duplicates to latest-wins) and
|
|
25
|
+
`join_use_nulls=1` (see below — the session rollup needs it):
|
|
24
26
|
|
|
25
27
|
```bash
|
|
26
28
|
set -a; [ -f ~/.memhouse/env ] && . ~/.memhouse/env; set +a
|
|
27
29
|
curl -sS --fail-with-body --user "${MEMHOUSE_USER:-memhouse_root}:${MEMHOUSE_PASSWORD:-}" \
|
|
28
|
-
--data-binary @- "${MEMHOUSE_URL:-http://localhost:8123}/?database=${MEMHOUSE_DB:-
|
|
30
|
+
--data-binary @- "${MEMHOUSE_URL:-http://localhost:8123}/?database=${MEMHOUSE_DB:-mem}&final=1&join_use_nulls=1" <<'SQL'
|
|
29
31
|
<the query>
|
|
30
32
|
FORMAT PrettyCompact
|
|
31
33
|
SQL
|
|
32
34
|
```
|
|
33
35
|
|
|
36
|
+
## Room names — always suffixed
|
|
37
|
+
|
|
38
|
+
Every member owns their own rooms, named for their ClickHouse user:
|
|
39
|
+
`messages_<you>`, `sessions_<you>`, `tool_calls_<you>`. There are no unsuffixed rooms;
|
|
40
|
+
the bare names fail with `UNKNOWN_TABLE` rather than returning nothing. You hold no grant
|
|
41
|
+
on anyone else's rooms, so isolation is not something a query can work around.
|
|
42
|
+
|
|
43
|
+
Resolve your own name once and substitute it into every table name below:
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
MEM_ME="$(curl -sS --fail-with-body --user "$MEMHOUSE_USER:$MEMHOUSE_PASSWORD" \
|
|
47
|
+
--data-binary "SELECT currentUser() FORMAT TabSeparated" \
|
|
48
|
+
"${MEMHOUSE_URL:-http://localhost:8123}/" | tr -d '\r\n')"
|
|
49
|
+
# rooms: messages_$MEM_ME, sessions_$MEM_ME, tool_calls_$MEM_ME
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
**There is no `sessions_v` object.** The session rollup is a saved query over those same
|
|
53
|
+
rooms — `memhouse sessions-query` prints it for whoever you are connected as, ready to
|
|
54
|
+
paste into a `FROM (...) AS c` position.
|
|
55
|
+
|
|
56
|
+
**Read the rollup with `join_use_nulls=1`.** The connection recipe above sets it. Without
|
|
57
|
+
it, ClickHouse gives an unmatched `m.seq` a default instead of NULL, so a session with no
|
|
58
|
+
messages reports `total_msgs = 1` rather than 0 — measured, not theoretical. A subquery
|
|
59
|
+
has no `SETTINGS` clause of its own, so the setting has to come from the caller.
|
|
60
|
+
|
|
61
|
+
If you hold `SELECT` on them, `all_messages` / `all_sessions` / `all_tool_calls` read
|
|
62
|
+
across every member at once, narrowed to whatever grants you actually have — a Merge room
|
|
63
|
+
reduces to the rooms the caller can read, so it fails closed rather than denying outright.
|
|
64
|
+
|
|
34
65
|
## Default listing
|
|
35
66
|
|
|
36
67
|
```sql
|
|
@@ -38,7 +69,7 @@ SELECT session_id, name, source, host, project,
|
|
|
38
69
|
started, ended, total_msgs, user_msgs, assistant_msgs,
|
|
39
70
|
output_tokens,
|
|
40
71
|
substring(first_prompt, 1, 80) AS first_prompt
|
|
41
|
-
FROM
|
|
72
|
+
FROM (the rollup from `memhouse sessions-query`)
|
|
42
73
|
ORDER BY ended DESC
|
|
43
74
|
LIMIT 20
|
|
44
75
|
FORMAT PrettyCompact
|
|
@@ -59,7 +90,7 @@ Per-editor rollup ("where has my work been happening?"):
|
|
|
59
90
|
```sql
|
|
60
91
|
SELECT source, count() AS sessions, sum(total_msgs) AS msgs,
|
|
61
92
|
sum(output_tokens) AS out_tokens, max(ended) AS last_activity
|
|
62
|
-
FROM
|
|
93
|
+
FROM (the rollup from `memhouse sessions-query`)
|
|
63
94
|
GROUP BY source
|
|
64
95
|
ORDER BY sessions DESC
|
|
65
96
|
FORMAT PrettyCompact
|
|
@@ -69,7 +100,7 @@ Busiest projects this month:
|
|
|
69
100
|
|
|
70
101
|
```sql
|
|
71
102
|
SELECT project, count() AS sessions, sum(assistant_msgs) AS assistant_msgs
|
|
72
|
-
FROM
|
|
103
|
+
FROM (the rollup from `memhouse sessions-query`)
|
|
73
104
|
WHERE ended > now() - INTERVAL 30 DAY AND project != ''
|
|
74
105
|
GROUP BY project
|
|
75
106
|
ORDER BY sessions DESC
|
|
@@ -90,5 +121,5 @@ FORMAT PrettyCompact
|
|
|
90
121
|
|
|
91
122
|
A compact table: when, name (or first prompt), source, host, project, msgs,
|
|
92
123
|
tokens. Offer to pull a full transcript next
|
|
93
|
-
(`SELECT role, text FROM
|
|
124
|
+
(`SELECT role, text FROM messages_<you> WHERE session_id = '…' ORDER BY seq`) or to
|
|
94
125
|
run memhouse-search for a specific quote.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: memhouse-sql
|
|
3
|
-
description: Run free-form read-only SQL against memhouse conversation memory (typed sessions/messages/tool_calls
|
|
3
|
+
description: Run free-form read-only SQL against memhouse conversation memory (typed sessions/messages/tool_calls rooms on ClickHouse, one set per member). Use for ad-hoc analytics the other memhouse skills don't cover — token spend, model/editor usage, tool rankings, activity heatmaps, busiest days/projects, cache-hit ratios, or any custom question over conversation data.
|
|
4
4
|
user-invocable: true
|
|
5
5
|
argument-hint: "<question or SQL>"
|
|
6
6
|
allowed-tools: Bash(set -a*), Bash(. *), Bash(curl*)
|
|
@@ -11,21 +11,23 @@ allowed-tools: Bash(set -a*), Bash(. *), Bash(curl*)
|
|
|
11
11
|
If the user gives SQL, run it (append `FORMAT PrettyCompact` if no FORMAT
|
|
12
12
|
given). If they give a question, write the SQL yourself from the schema below.
|
|
13
13
|
|
|
14
|
-
**Read-only rule:** the shipper (`ship.js`) is the only writer. Never INSERT/
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
14
|
+
**Read-only rule:** the shipper (`ship.js`) is the only writer. Never INSERT/ALTER/DROP
|
|
15
|
+
from here. A member credential normally holds `ALL` on its **own three rooms** — enough
|
|
16
|
+
to destroy that memory — and nothing at all on anyone else's. Reads need no scoping
|
|
17
|
+
clause: the rooms you can name are already only yours, and there is no policy to work
|
|
18
|
+
around.
|
|
18
19
|
|
|
19
20
|
## Connection
|
|
20
21
|
|
|
21
22
|
Credentials from `~/.memhouse/env` (or exported `MEMHOUSE_*`). Always read with
|
|
22
23
|
`final=1` — every table is `ReplacingMergeTree(ingested_at)`, so without it you
|
|
23
|
-
can double-count stale row versions
|
|
24
|
+
can double-count stale row versions — and `join_use_nulls=1`, which the session
|
|
25
|
+
rollup depends on (see below):
|
|
24
26
|
|
|
25
27
|
```bash
|
|
26
28
|
set -a; [ -f ~/.memhouse/env ] && . ~/.memhouse/env; set +a
|
|
27
29
|
curl -sS --fail-with-body --user "${MEMHOUSE_USER:-memhouse_root}:${MEMHOUSE_PASSWORD:-}" \
|
|
28
|
-
--data-binary @- "${MEMHOUSE_URL:-http://localhost:8123}/?database=${MEMHOUSE_DB:-
|
|
30
|
+
--data-binary @- "${MEMHOUSE_URL:-http://localhost:8123}/?database=${MEMHOUSE_DB:-mem}&final=1&join_use_nulls=1" <<'SQL'
|
|
29
31
|
<the query>
|
|
30
32
|
FORMAT PrettyCompact
|
|
31
33
|
SQL
|
|
@@ -33,20 +35,49 @@ SQL
|
|
|
33
35
|
|
|
34
36
|
`FORMAT PrettyCompact` for display, `FORMAT JSONEachRow` to parse.
|
|
35
37
|
|
|
38
|
+
## Room names — always suffixed
|
|
39
|
+
|
|
40
|
+
Every member owns their own rooms, named for their ClickHouse user:
|
|
41
|
+
`messages_<you>`, `sessions_<you>`, `tool_calls_<you>`. There are no unsuffixed rooms;
|
|
42
|
+
the bare names fail with `UNKNOWN_TABLE` rather than returning nothing. You hold no grant
|
|
43
|
+
on anyone else's rooms, so isolation is not something a query can work around.
|
|
44
|
+
|
|
45
|
+
Resolve your own name once and substitute it into every table name below:
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
MEM_ME="$(curl -sS --fail-with-body --user "$MEMHOUSE_USER:$MEMHOUSE_PASSWORD" \
|
|
49
|
+
--data-binary "SELECT currentUser() FORMAT TabSeparated" \
|
|
50
|
+
"${MEMHOUSE_URL:-http://localhost:8123}/" | tr -d '\r\n')"
|
|
51
|
+
# rooms: messages_$MEM_ME, sessions_$MEM_ME, tool_calls_$MEM_ME
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
**There is no `sessions_v` object.** The session rollup is a saved query over those same
|
|
55
|
+
rooms — `memhouse sessions-query` prints it for whoever you are connected as, ready to
|
|
56
|
+
paste into a `FROM (...) AS c` position.
|
|
57
|
+
|
|
58
|
+
**Read the rollup with `join_use_nulls=1`.** The connection recipe above sets it. Without
|
|
59
|
+
it, ClickHouse gives an unmatched `m.seq` a default instead of NULL, so a session with no
|
|
60
|
+
messages reports `total_msgs = 1` rather than 0 — measured, not theoretical. A subquery
|
|
61
|
+
has no `SETTINGS` clause of its own, so the setting has to come from the caller.
|
|
62
|
+
|
|
63
|
+
If you hold `SELECT` on them, `all_messages` / `all_sessions` / `all_tool_calls` read
|
|
64
|
+
across every member at once, narrowed to whatever grants you actually have — a Merge room
|
|
65
|
+
reduces to the rooms the caller can read, so it fails closed rather than denying outright.
|
|
66
|
+
|
|
36
67
|
## Schema (the house)
|
|
37
68
|
|
|
38
69
|
| Object | Kind | Columns |
|
|
39
70
|
|---|---|---|
|
|
40
|
-
| `
|
|
41
|
-
| `
|
|
42
|
-
| `
|
|
43
|
-
| `
|
|
71
|
+
| `sessions_<you>` | table, 1 row/session | `session_id, source, host, name, mode, folder, project, git_branch, created_at, last_updated_at, message_count, path, extra JSON, user_id, ingested_at` |
|
|
72
|
+
| `messages_<you>` | table, 1 row/message | `session_id, seq, source, host, ts, role, model, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, text, project, folder, is_subagent, extra JSON, line_hash, user_id, ingested_at` + FTS columns `text_ngram`/`text_word` (lowercased; see memhouse-search) |
|
|
73
|
+
| `tool_calls_<you>` | table, 1 row/tool call | `session_id, seq, idx, source, host, tool_name, args, ts, project, folder, user_id, ingested_at` |
|
|
74
|
+
| the rollup | **saved query**, not an object — `memhouse sessions-query` prints it | `session_id, source, host, name, mode, folder, project, git_branch, user_id, created_at, last_updated_at, started, ended, duration_sec, total_msgs, user_msgs, assistant_msgs, subagent_msgs, models, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, user_chars, assistant_chars, first_prompt` |
|
|
44
75
|
|
|
45
76
|
Notes: `source` = editor id (`claude-code`, `codex`, `cursor`, `cursor-agent`,
|
|
46
77
|
`vscode`, `zed`, `opencode`, `gemini-cli`, `windsurf`, `antigravity`, …);
|
|
47
78
|
`seq` = message index within its session (0-based); `tool_calls.idx` = call
|
|
48
79
|
index within the session; `user_id` is server-stamped (`currentUser()`);
|
|
49
|
-
`models` in
|
|
80
|
+
`models` in the rollup is an `Array(String)`. Exclude `''` and
|
|
50
81
|
`'<synthetic>'` from model aggregates.
|
|
51
82
|
|
|
52
83
|
## Examples
|
|
@@ -56,7 +87,7 @@ Token spend by model:
|
|
|
56
87
|
```sql
|
|
57
88
|
SELECT model, sum(input_tokens) AS in_tok, sum(output_tokens) AS out_tok,
|
|
58
89
|
sum(cache_read_tokens) AS cache_read
|
|
59
|
-
FROM
|
|
90
|
+
FROM messages_<you>
|
|
60
91
|
WHERE model NOT IN ('', '<synthetic>')
|
|
61
92
|
GROUP BY model
|
|
62
93
|
ORDER BY out_tok DESC
|
|
@@ -67,7 +98,7 @@ Which tools do I use most (memhouse exclusive — memory-house has no tool table
|
|
|
67
98
|
|
|
68
99
|
```sql
|
|
69
100
|
SELECT tool_name, count() AS calls, uniqExact(session_id) AS sessions
|
|
70
|
-
FROM
|
|
101
|
+
FROM tool_calls_<you>
|
|
71
102
|
GROUP BY tool_name
|
|
72
103
|
ORDER BY calls DESC
|
|
73
104
|
LIMIT 20
|
|
@@ -78,7 +109,7 @@ Busiest days, last two weeks:
|
|
|
78
109
|
|
|
79
110
|
```sql
|
|
80
111
|
SELECT toDate(ts) AS day, uniqExact(session_id) AS sessions, count() AS msgs
|
|
81
|
-
FROM
|
|
112
|
+
FROM messages_<you>
|
|
82
113
|
WHERE ts > now() - INTERVAL 14 DAY
|
|
83
114
|
GROUP BY day
|
|
84
115
|
ORDER BY day DESC
|
|
@@ -89,7 +120,7 @@ Activity by editor this week / subagent share:
|
|
|
89
120
|
|
|
90
121
|
```sql
|
|
91
122
|
SELECT source, count() AS msgs, countIf(is_subagent) AS subagent_msgs
|
|
92
|
-
FROM
|
|
123
|
+
FROM messages_<you>
|
|
93
124
|
WHERE ts > now() - INTERVAL 7 DAY
|
|
94
125
|
GROUP BY source
|
|
95
126
|
ORDER BY msgs DESC
|
|
@@ -100,7 +131,7 @@ Longest sessions by wall-clock:
|
|
|
100
131
|
|
|
101
132
|
```sql
|
|
102
133
|
SELECT name, project, source, duration_sec, total_msgs, output_tokens
|
|
103
|
-
FROM
|
|
134
|
+
FROM (the rollup from `memhouse sessions-query`)
|
|
104
135
|
ORDER BY duration_sec DESC
|
|
105
136
|
LIMIT 10
|
|
106
137
|
FORMAT PrettyCompact
|