@martintrojer/murmur 0.1.3 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ARCHITECTURE.md +742 -219
- package/CHANGELOG.md +172 -0
- package/README.md +136 -27
- package/dist/cli.js +1419 -869
- package/dist/cli.js.map +1 -1
- package/dist/extension/murmur-pi.js +190 -102
- package/dist/extension/murmur-pi.js.map +1 -1
- package/dist/extension/store.js +413 -190
- package/dist/extension/store.js.map +1 -1
- package/dist/index.d.ts +492 -147
- package/dist/index.js +970 -641
- package/dist/index.js.map +1 -1
- package/package.json +4 -3
package/ARCHITECTURE.md
CHANGED
|
@@ -17,204 +17,556 @@ spends most of its effort refusing to solve harder problems nearby.
|
|
|
17
17
|
## The shape
|
|
18
18
|
|
|
19
19
|
```
|
|
20
|
-
pi agent ──in-process── murmur store ──
|
|
20
|
+
pi agent ──in-process── murmur store ── state.db (one per node)
|
|
21
21
|
│
|
|
22
|
-
|
|
22
|
+
ssh murmur export
|
|
23
23
|
│
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
24
|
+
collector ── view ── picker
|
|
25
|
+
│
|
|
26
|
+
jump: local switch, or ssh -t
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
-
Each node keeps
|
|
30
|
-
|
|
31
|
-
|
|
29
|
+
Each node keeps one SQLite database describing the current state of its own
|
|
30
|
+
panes. `murmur export` prints that state as a single complete JSON document — a
|
|
31
|
+
**snapshot**. Any node pulls its peers' snapshots over ssh, caches one per peer,
|
|
32
|
+
and renders the union as an attention-sorted list. No daemon, no listening
|
|
33
|
+
socket, no master node.
|
|
32
34
|
|
|
33
35
|
murmur observes and connects. It does not place work. That is an orchestrator's
|
|
34
36
|
job, and mixing the two is how you end up owning scheduling, credentials and
|
|
35
37
|
artifact movement.
|
|
36
38
|
|
|
39
|
+
## The model in one page
|
|
40
|
+
|
|
41
|
+
Three facts, independent, each with exactly one writer:
|
|
42
|
+
|
|
43
|
+
| Fact | Meaning | Stored as | Written by |
|
|
44
|
+
| --- | --- | --- | --- |
|
|
45
|
+
| **activity** | is a process working in this pane | `agents.activity` = `running` \| `stopped` | the pane's owning process only |
|
|
46
|
+
| **attention** | does someone need to look at this pane | rows in `attention`, kind `done` \| `blocked` \| `crashed` | owner (`done`), external notifier (`blocked`), local reconciliation (`crashed`) |
|
|
47
|
+
| **freshness** | how recently we reached the node that reported | `peers.fetched_at` | the collector |
|
|
48
|
+
|
|
49
|
+
They are never folded into one enum, and no stored value spans two of them.
|
|
50
|
+
There is no `cleared`: absence of an attention row *is* "nothing to see", and
|
|
51
|
+
absence of an agent row *is* "no agent here". The words a surface paints
|
|
52
|
+
(`crashed`, `blocked`, `done`, `running`, `idle`) are derived at read time by
|
|
53
|
+
`renderState` and stored nowhere.
|
|
54
|
+
|
|
55
|
+
Why three and not one. A crashed agent on an unreachable host is crashed *and*
|
|
56
|
+
stale, on different axes; a running agent that a notifier flagged as blocked is
|
|
57
|
+
both, and the picker shows both. Every attempt to collapse these produced a
|
|
58
|
+
question with no answer, and one of them shipped: a focus hook that could
|
|
59
|
+
overwrite an agent's state replaced `working` with `blocked` on live panes and
|
|
60
|
+
nulled the owner metadata while all three processes were running. The fix is
|
|
61
|
+
structural — `attention` has no column an agent field could live in.
|
|
62
|
+
|
|
63
|
+
Identity and address are separated:
|
|
64
|
+
|
|
65
|
+
- **node identity** — `identity.json`, `{host_id, display_name}`. Created only
|
|
66
|
+
by `murmur init`, and only by it. Survives a store wipe.
|
|
67
|
+
- **agent identity** — a random UUID minted per *process instance* when it
|
|
68
|
+
claims a pane. It is not derived from the pane, so a new process in the same
|
|
69
|
+
pane is a different agent, and a late write from a replaced owner matches no
|
|
70
|
+
row.
|
|
71
|
+
- **pane** — the *address*. `UNIQUE` in `agents`, part of the primary key in
|
|
72
|
+
`attention`. One top-level instrumented agent per pane, enforced by SQLite.
|
|
73
|
+
|
|
74
|
+
Truth about a node lives only on that node. Other nodes hold one opaque,
|
|
75
|
+
validated snapshot per peer, replaced whole or not at all.
|
|
76
|
+
|
|
37
77
|
## Why this is not a distributed system
|
|
38
78
|
|
|
39
|
-
It is single-writer-per-partition
|
|
40
|
-
|
|
41
|
-
|
|
79
|
+
It is single-writer-per-partition caching. Each node owns the state of its own
|
|
80
|
+
panes and publishes it; no other node writes about them, ever. Merging is a
|
|
81
|
+
concatenation of one local read and one cached document per peer.
|
|
42
82
|
|
|
43
83
|
Calling it "distributed" invites machinery the problem does not have: consensus,
|
|
44
84
|
conflict resolution, vector clocks, leader election. If a change starts to need
|
|
45
85
|
conflict resolution, that is the signal the single-writer invariant has been
|
|
46
86
|
broken somewhere.
|
|
47
87
|
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
- **A node may not
|
|
51
|
-
corrections. A reader that learns something
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
-
|
|
55
|
-
|
|
56
|
-
|
|
88
|
+
Three consequences worth stating, because each is easy to violate by accident:
|
|
89
|
+
|
|
90
|
+
- **A node may not write state about another node's panes.** Not even
|
|
91
|
+
corrections. A reader that learns something — a jump proving a pane is gone —
|
|
92
|
+
*reports* it and writes nothing. The next collect reconciles, because the
|
|
93
|
+
owning node is the one that reconciles.
|
|
94
|
+
- **Only a pane's own process may report for that pane.** `claimAgent` answers
|
|
95
|
+
`refused` to a second live claimant, and a refused caller registers no
|
|
96
|
+
handlers, writes nothing and paints no badge. This replaced an environment
|
|
97
|
+
marker and three helper functions: a nested pi inherits `$TMUX_PANE` and used
|
|
98
|
+
to report *as* the pane's real agent, so one pane accumulated six reporting
|
|
99
|
+
pids of which one was alive, and the live agent read idle while it worked.
|
|
100
|
+
Silence is correct for a process with nothing true to say.
|
|
101
|
+
- **`murmur notify` is the one deliberate exception, and it is narrow by
|
|
102
|
+
construction.** Its whole request type is `{kind, location, message, source}`
|
|
103
|
+
— there is no `agent_id`, no pid, no activity and no metadata field, so it
|
|
104
|
+
cannot say anything about a process being alive even by accident. It is also
|
|
105
|
+
restricted to `blocked` by the callers that use it; `running`, `done` and
|
|
106
|
+
`crashed` remain the owner's and reconciliation's.
|
|
57
107
|
|
|
58
108
|
## A tmux pane is the agent's address
|
|
59
109
|
|
|
60
|
-
Everything below assumes agents run inside tmux.
|
|
61
|
-
|
|
62
|
-
one.
|
|
110
|
+
Everything below assumes agents run inside tmux. A pane is how murmur names an
|
|
111
|
+
agent and how a jump reaches one.
|
|
63
112
|
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
113
|
+
An agent IS a pane. A session and a window are only where that pane currently
|
|
114
|
+
lives: a pane keeps its id across `move-pane`, `break-pane`, and a window closed
|
|
115
|
+
and reopened, while a recorded window id goes stale as a matter of course. So
|
|
116
|
+
**only a pane may decide whether an agent exists** — a window id is location,
|
|
117
|
+
never evidence of life. tmux says the same thing with its sigils, `$25` / `@75`
|
|
118
|
+
/ `%89`, and the three ids are branded types (`SessionId`, `WindowId`, `PaneId`)
|
|
119
|
+
so that passing one where another is meant does not compile.
|
|
120
|
+
|
|
121
|
+
The extension resolves its pane from `$TMUX_PANE` and returns early without one,
|
|
122
|
+
so a pi started in a plain terminal records nothing and never appears in the
|
|
123
|
+
picker. That is the honest outcome rather than a gap: with no pane there is no
|
|
124
|
+
address, and a row you cannot jump to is worse than no row.
|
|
68
125
|
|
|
69
126
|
Two nearby calls look contradictory and are not:
|
|
70
127
|
|
|
71
128
|
- `currentWindow()` answers "which pane am I in", so only `$TMUX_PANE` can tell
|
|
72
129
|
it. Asking tmux instead reports whichever pane the server considers active,
|
|
73
|
-
which would let a non-tmux pi record itself in an unrelated agent's pane
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
130
|
+
which would let a non-tmux pi record itself in an unrelated agent's pane.
|
|
131
|
+
- `livePanes()` answers "which panes exist on this host", which is server-wide
|
|
132
|
+
and correct from anywhere. `export` runs over ssh with no pane of its own and
|
|
133
|
+
still has to see the panes.
|
|
134
|
+
|
|
135
|
+
`livePanes()` returns `null` for "could not tell", which is deliberately
|
|
136
|
+
distinct from an empty set, and `reconcileLocal` treats `null` as no evidence
|
|
137
|
+
and writes nothing. Conflating them deletes every agent on the host the moment
|
|
138
|
+
tmux is unreachable.
|
|
139
|
+
|
|
140
|
+
The rule binds the JUMP as well as reconciliation, and that took two goes to
|
|
141
|
+
learn. A local jump asks `livePanes()`, and the remote probe is `tmux list-panes
|
|
142
|
+
-a -F '#{pane_id}'` — not `list-windows`, because no answer about windows can
|
|
143
|
+
say whether a pane exists. Asking about windows there condemned healthy agents
|
|
144
|
+
on one keypress. **A failed jump now mutates nothing**: it returns a
|
|
145
|
+
`JumpResult` with a reason and a message, and `pane_gone` is a report rather
|
|
146
|
+
than a deletion. That is the same single-writer rule one level down — the jump
|
|
147
|
+
runs on the reader, and only the owning node may retire a pane.
|
|
148
|
+
|
|
149
|
+
The remote jump inherits the tmux requirement, since it is `ssh -t <host> tmux
|
|
80
150
|
attach`. tmux must be running on the far side, which is why a dead remote tmux
|
|
81
|
-
server gets its own diagnosis rather than being reported as an unreachable
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
151
|
+
server gets its own diagnosis rather than being reported as an unreachable host.
|
|
152
|
+
|
|
153
|
+
The brands (`src/ids.ts`) are phantom types on `string`, so they cost nothing at
|
|
154
|
+
runtime and are applied at the edges: `asPaneId` and friends are called where a
|
|
155
|
+
bare string arrives — argv, a tmux query, the wire — and everything inside deals
|
|
156
|
+
in branded values. What that prevented was the mix-up that deleted ten live
|
|
157
|
+
agents: comparing a `WindowId` against a set of pane ids is a compile error.
|
|
158
|
+
|
|
159
|
+
What it does NOT buy, since that bounds how far the types can be trusted: both
|
|
160
|
+
jump paths once compared a `WindowId` against a `Set<WindowId>`, which is
|
|
161
|
+
internally coherent and compiles cleanly. Branding stops you MIXING the three
|
|
162
|
+
ids; it cannot stop you asking the wrong one a question. A type system polices
|
|
163
|
+
which noun you passed, never whether the question was worth asking. Only a test
|
|
164
|
+
that moves a pane out from under a recorded window catches that.
|
|
165
|
+
|
|
166
|
+
The badge also has to be cleared from outside, which is why `murmur clear --pane
|
|
167
|
+
<id>` exists and why tmux hooks call it. The status bar and the picker read a
|
|
168
|
+
tmux window option, so it outlives the agent unless something clears it — and
|
|
169
|
+
the agent cannot, because "you looked at it" is an event only the multiplexer
|
|
170
|
+
sees. Hooks run in the tmux server with no `$TMUX_PANE`, so the pane id is
|
|
171
|
+
passed explicitly: the badge belongs to the window, the looking belongs to one
|
|
172
|
+
pane.
|
|
173
|
+
|
|
174
|
+
## The units
|
|
93
175
|
|
|
94
176
|
| Unit | Does | Depends on |
|
|
95
177
|
| ---- | ---- | ---------- |
|
|
96
178
|
| `identity` | Read/create this node's `{host_id, display_name}` | state dir |
|
|
97
|
-
| `store` |
|
|
98
|
-
| `
|
|
179
|
+
| `store` | Claim, report, reconcile, cache. **The only module touching SQL** | `identity` (as a parameter) |
|
|
180
|
+
| `snapshot` | **Pure.** `parseSnapshot`: validate one document, totally | nothing |
|
|
181
|
+
| `view` | **Pure.** `SnapshotPane[]` in, `PaneView[]` out | `store` (types) |
|
|
99
182
|
| `channel` | Seam: `exec(target, argv) -> stdout`. One impl: ssh | OS |
|
|
100
|
-
| `collector` |
|
|
101
|
-
| `mux` | Seam: window/pane queries,
|
|
183
|
+
| `collector` | Fetch each peer, validate, replace its cache whole | `channel`, `store` |
|
|
184
|
+
| `mux` | Seam: window/pane queries, badge, attach. One impl: tmux | OS |
|
|
185
|
+
|
|
186
|
+
`view` and `snapshot` being pure and `store` being the only SQL is the boundary
|
|
187
|
+
that carries the design. All three are testable without a machine, a network or
|
|
188
|
+
a multiplexer.
|
|
189
|
+
|
|
190
|
+
**One local read.** `Store.localPanes()` is the only way to read local state,
|
|
191
|
+
and it returns `SnapshotPane[]` — the same shape a peer's cached snapshot holds.
|
|
192
|
+
So `paneViews` maps one type to `PaneView` exactly once, and a local pane and a
|
|
193
|
+
remote pane travel the same code path, differing only in the `host_id`, `local`
|
|
194
|
+
and freshness fields the caller supplies. There is no `agentForPane`, no
|
|
195
|
+
`attentionFor`, no `agents()`: it was the proliferation of narrow reads that let
|
|
196
|
+
`clear` and the extension each derive their own idea of what a pane's state was,
|
|
197
|
+
across fourteen sites and six shipped bugs.
|
|
198
|
+
|
|
199
|
+
`owner_pid` is not in that shape. It is local-only, never in a snapshot and
|
|
200
|
+
never on the wire, which makes remote liveness inference *unrepresentable*
|
|
201
|
+
rather than discouraged. A remote pid names a process in another machine's table.
|
|
202
|
+
|
|
203
|
+
The `Store` interface is closed, and adding a method is a contract change. Some
|
|
204
|
+
shapes are forbidden outright, each because it once let a writer say something
|
|
205
|
+
it had no standing to say: anything that takes a row and writes it (`append`,
|
|
206
|
+
`ingest`, `put`), any log read, any partial-row or column-map update, any
|
|
207
|
+
activity write keyed on pane alone rather than on `agent_id` plus `owner_pid`,
|
|
208
|
+
any attention method that accepts an agent field, and any escape hatch exposing
|
|
209
|
+
the database handle.
|
|
210
|
+
|
|
211
|
+
Two seams exist with exactly one implementation each: `channel` (ssh) and `mux`
|
|
212
|
+
(tmux). Defined so a second backend is possible; not designed for one that does
|
|
213
|
+
not exist. The harness is not a third: pi reports in-process through the
|
|
214
|
+
extension, and every other harness comes in through `murmur notify`, which is a
|
|
215
|
+
command rather than an interface to implement.
|
|
102
216
|
|
|
103
|
-
|
|
104
|
-
the design. Four heuristics live in one of those two modules: attention
|
|
105
|
-
ordering, staleness, crash synthesis, and retention. Both modules are testable
|
|
106
|
-
without a machine, a network or a multiplexer.
|
|
217
|
+
## The data model
|
|
107
218
|
|
|
108
|
-
Three
|
|
109
|
-
`
|
|
110
|
-
|
|
219
|
+
Three tables in `state.db`, all `STRICT`, `user_version = 3`. `agents` and
|
|
220
|
+
`attention` are local truth; `peers` is a cache of other nodes.
|
|
221
|
+
|
|
222
|
+
```sql
|
|
223
|
+
CREATE TABLE agents (
|
|
224
|
+
agent_id TEXT NOT NULL PRIMARY KEY, -- a UUID per process instance
|
|
225
|
+
pane TEXT NOT NULL UNIQUE, -- the address
|
|
226
|
+
owner_pid INTEGER NOT NULL CHECK (owner_pid > 0),
|
|
227
|
+
activity TEXT NOT NULL CHECK (activity IN ('running', 'stopped')),
|
|
228
|
+
session TEXT NOT NULL, -- location, may change
|
|
229
|
+
window TEXT NOT NULL,
|
|
230
|
+
session_name TEXT, window_name TEXT,
|
|
231
|
+
agent_name TEXT, pi_session TEXT, workstream TEXT, role TEXT,
|
|
232
|
+
cli TEXT NOT NULL,
|
|
233
|
+
driver TEXT NOT NULL CHECK (driver IN ('human', 'orchestrated')),
|
|
234
|
+
claimed_at INTEGER NOT NULL,
|
|
235
|
+
updated_at INTEGER NOT NULL
|
|
236
|
+
) STRICT;
|
|
237
|
+
|
|
238
|
+
CREATE TABLE attention (
|
|
239
|
+
pane TEXT NOT NULL,
|
|
240
|
+
kind TEXT NOT NULL CHECK (kind IN ('done', 'blocked', 'crashed')),
|
|
241
|
+
message TEXT NOT NULL,
|
|
242
|
+
source TEXT NOT NULL,
|
|
243
|
+
session TEXT NOT NULL, -- its own location: see below
|
|
244
|
+
window TEXT NOT NULL,
|
|
245
|
+
session_name TEXT, window_name TEXT,
|
|
246
|
+
requested_at INTEGER NOT NULL,
|
|
247
|
+
PRIMARY KEY (pane, kind)
|
|
248
|
+
) STRICT;
|
|
249
|
+
|
|
250
|
+
CREATE TABLE peers (
|
|
251
|
+
name TEXT NOT NULL PRIMARY KEY, target TEXT NOT NULL,
|
|
252
|
+
host_id TEXT, display_name TEXT,
|
|
253
|
+
snapshot TEXT, snapshot_at INTEGER, -- the whole document, their clock
|
|
254
|
+
fetched_at INTEGER, last_attempt_at INTEGER, last_error TEXT,
|
|
255
|
+
murmur_version TEXT, snapshot_version INTEGER
|
|
256
|
+
) STRICT;
|
|
257
|
+
```
|
|
111
258
|
|
|
112
|
-
|
|
259
|
+
Schema facts that are load-bearing, and the reason each is in the schema rather
|
|
260
|
+
than in a comment:
|
|
261
|
+
|
|
262
|
+
1. **`agents.pane` is `UNIQUE`.** "One top-level instrumented agent per pane" is
|
|
263
|
+
enforced by SQLite, not by a caller.
|
|
264
|
+
2. **`agents.agent_id` is a UUID, not `host:pane`.** A replacement owner is a
|
|
265
|
+
different row, so a late write from the previous owner matches nothing and is
|
|
266
|
+
silently ineffective rather than destructive.
|
|
267
|
+
3. **`owner_pid` is local-only**, per the previous section.
|
|
268
|
+
4. **`attention` has no `agent_id` and no `owner_pid` column.** An attention
|
|
269
|
+
writer structurally cannot address an agent's identity, activity or metadata.
|
|
270
|
+
This is the whole fix for the live-corruption incident described above.
|
|
271
|
+
5. **`PRIMARY KEY (pane, kind)`.** Kinds coexist: a `crashed` row is not
|
|
272
|
+
clobbered by a later `blocked`, and "focus clears all attention for the pane"
|
|
273
|
+
is one `DELETE ... WHERE pane = ?`.
|
|
274
|
+
6. **`attention` carries its own location.** An attention-only pane — a codex
|
|
275
|
+
agent murmur never instrumented — is listable and jumpable with no agent row.
|
|
276
|
+
`attention.pane` deliberately does not reference `agents.pane`; a constraint
|
|
277
|
+
saying otherwise would make that case unrepresentable.
|
|
278
|
+
7. **`peers.snapshot` is one TEXT column** holding the validated document.
|
|
279
|
+
Whole-peer atomic replacement is therefore structural: there is no partial
|
|
280
|
+
apply to get wrong.
|
|
281
|
+
8. **`CHECK` constraints on `activity`, `driver`, `kind`.** An unknown value
|
|
282
|
+
cannot reach storage, so no sort, count or render path needs a fallback
|
|
283
|
+
branch. An unknown state that sorted as `NaN` was a real bug.
|
|
284
|
+
|
|
285
|
+
No index beyond the declared keys: both local tables are bounded by the number
|
|
286
|
+
of live panes on one machine.
|
|
287
|
+
|
|
288
|
+
**Who writes each fact**, since "it is in the enum" is not the same as
|
|
289
|
+
"something produces it" — `blocked` sat in an enum unproduced for months while
|
|
290
|
+
three surfaces carried machinery for it:
|
|
291
|
+
|
|
292
|
+
| Fact | Written by | On |
|
|
293
|
+
| --- | --- | --- |
|
|
294
|
+
| `activity = running` | pi extension | `agent_start` |
|
|
295
|
+
| `activity = stopped` | pi extension | `agent_end` |
|
|
296
|
+
| attention `done` | pi extension | `agent_settled`, pane unfocused, `driver = human` |
|
|
297
|
+
| attention `blocked` | `murmur notify` | an outside-in call from another harness |
|
|
298
|
+
| attention `crashed` | `reconcileLocal` | pane alive, owner pid gone, activity was `running` |
|
|
299
|
+
| (row removed) | `releaseAgent` | `session_shutdown` |
|
|
300
|
+
| (attention removed) | `acknowledgePane` | `murmur clear`, i.e. tmux focus |
|
|
301
|
+
|
|
302
|
+
Nothing writes `crashed` from inside an agent, for the obvious reason.
|
|
303
|
+
|
|
304
|
+
**Versioning is one strategy, not two.** On open, if `user_version` is not 3,
|
|
305
|
+
murmur salvages `SELECT name, target FROM peers` — the two fields a human typed
|
|
306
|
+
— deletes the database and its `-wal`/`-shm` sidecars, recreates the schema, and
|
|
307
|
+
re-inserts those peers with every observed column `NULL`. There is no
|
|
308
|
+
`ALTER TABLE` anywhere and no additive path to forget to use. Any change to any
|
|
309
|
+
table bumps the version and costs a rebuild, which is affordable precisely
|
|
310
|
+
because nothing here is history: everything in the file is either current state
|
|
311
|
+
or a cache.
|
|
312
|
+
|
|
313
|
+
**Identity is never minted by the store.** `openStore()` takes no arguments and
|
|
314
|
+
does not read `identity.json`. `createIdentity` is called only by `murmur init`;
|
|
315
|
+
`setDisplayName` backs `murmur init --name` on an existing node and keeps its
|
|
316
|
+
`host_id`. Every command that needs a `host_id` — `export`, `collect`, `status`,
|
|
317
|
+
`pick`, `peer` — calls `loadIdentity()` and fails with `murmur is not initialised
|
|
318
|
+
on this node; run: murmur init`. A node that came into existence as a side
|
|
319
|
+
effect of a status-bar tick has an identity nobody chose.
|
|
320
|
+
|
|
321
|
+
`notify` and `clear` are absent from that list as a consequence of the model
|
|
322
|
+
rather than as an exemption: both address a pane, and `attention` is keyed on
|
|
323
|
+
pane alone, so neither reads `identity.json` and neither can fail for want of an
|
|
324
|
+
identity.
|
|
325
|
+
|
|
326
|
+
**Transactions.** `claimAgent` and `reconcileLocal` are `IMMEDIATE`: both read
|
|
327
|
+
then write, and a deferred transaction would start as a reader and fail
|
|
328
|
+
`SQLITE_BUSY_SNAPSHOT` on upgrade — measured at 5 of 8 concurrent writers
|
|
329
|
+
failing. Everything else is a single statement, atomic by construction.
|
|
330
|
+
`localPanes` is a deferred read transaction, so a pane cannot appear with an
|
|
331
|
+
agent and without the attention that was there when the agent was read.
|
|
332
|
+
`buildLocalSnapshot` is two transactions rather than one, because a write
|
|
333
|
+
transaction held open across the read would serialise every focus hook on the
|
|
334
|
+
machine behind an export.
|
|
335
|
+
|
|
336
|
+
Deliberately absent from the model: a generic `put`/`del` op-log (that needs
|
|
337
|
+
conflict resolution, which single-writer partitioning lets us skip, and it would
|
|
338
|
+
simply *be* `mu`), task/DAG structure (observing a task graph means owning it),
|
|
339
|
+
and any hybrid logical clock (nothing depends on cross-node causality; clock
|
|
340
|
+
skew affects display order only).
|
|
341
|
+
|
|
342
|
+
**TypeScript, on install-story grounds.** pi is an npm package, so every node
|
|
343
|
+
running agents already has Node and npm. Python would mean inventing a
|
|
344
|
+
cross-machine install story for the ecosystem that handles it worst, on machines
|
|
345
|
+
where the system interpreter is least yours to touch.
|
|
346
|
+
|
|
347
|
+
## The snapshot
|
|
348
|
+
|
|
349
|
+
`murmur export` takes no options and prints one complete document. There is no
|
|
350
|
+
`--since`, no watermark, no delta form and no JSONL.
|
|
351
|
+
|
|
352
|
+
```jsonc
|
|
353
|
+
{
|
|
354
|
+
"murmur_snapshot": 1,
|
|
355
|
+
"host_id": "1d2ee96e-3a94-41b2-90fa-5f1ee2f04276", // from identity.json
|
|
356
|
+
"display_name": "mtrojer-mac",
|
|
357
|
+
"murmur_version": "0.2.0", // read from package.json, never restated
|
|
358
|
+
"generated_at": 1788105698997, // this node's clock at build time
|
|
359
|
+
"panes": [
|
|
360
|
+
{
|
|
361
|
+
"pane": "%250",
|
|
362
|
+
"session": "$25",
|
|
363
|
+
"window": "@75",
|
|
364
|
+
"session_name": "hacking/murmur",
|
|
365
|
+
"window_name": "worker-1",
|
|
366
|
+
"agent": {
|
|
367
|
+
"agent_id": "c0ffee00-1111-2222-3333-444444444444",
|
|
368
|
+
"activity": "running",
|
|
369
|
+
"agent_name": "worker-1",
|
|
370
|
+
"pi_session": null,
|
|
371
|
+
"workstream": "murmur",
|
|
372
|
+
"role": null,
|
|
373
|
+
"cli": "pi",
|
|
374
|
+
"driver": "orchestrated",
|
|
375
|
+
"claimed_at": 1788105600000,
|
|
376
|
+
"updated_at": 1788105690000
|
|
377
|
+
},
|
|
378
|
+
"attention": [
|
|
379
|
+
{ "kind": "blocked", "message": "needs input", "source": "codex",
|
|
380
|
+
"requested_at": 1788105680000 }
|
|
381
|
+
]
|
|
382
|
+
}
|
|
383
|
+
]
|
|
384
|
+
}
|
|
385
|
+
```
|
|
386
|
+
|
|
387
|
+
Rules, each of which a reader depends on:
|
|
388
|
+
|
|
389
|
+
1. **The document is complete, so absence is absence.** A peer that answers has
|
|
390
|
+
said everything it knows, and a pane present in the previous fetch and
|
|
391
|
+
missing from this one is gone. This is what makes whole replacement correct.
|
|
392
|
+
2. `owner_pid` is absent, on purpose. A reader has no pid to probe.
|
|
393
|
+
3. A pane with `"agent": null` is an attention-only pane: valid, listable,
|
|
394
|
+
jumpable. A pane with `"agent": null` and `"attention": []` is not emitted.
|
|
395
|
+
4. `panes` order is presentation-only. Emitted sorted by pane id so the output
|
|
396
|
+
diffs cleanly; readers sort for themselves.
|
|
397
|
+
5. `generated_at` is the *producing* node's clock. What it is not is when the
|
|
398
|
+
reader fetched it — see freshness below.
|
|
399
|
+
|
|
400
|
+
`buildLocalSnapshot` reconciles before it reads, which is what makes the
|
|
401
|
+
document authoritative: a snapshot built from unreconciled rows would publish
|
|
402
|
+
agents whose panes are gone, and a reader has no way to tell.
|
|
403
|
+
|
|
404
|
+
**Validation is total and strict**, and happens before storage.
|
|
405
|
+
`parseSnapshot` rejects an unknown key, a missing key, a wrong type, a
|
|
406
|
+
`murmur_snapshot` other than `1`, an unknown `activity`, `driver` or `kind`, a
|
|
407
|
+
duplicate pane, and a pane that is neither an agent nor an attention. Nothing is
|
|
408
|
+
coerced, defaulted or carried through. The error names the first failing path
|
|
409
|
+
(`panes[3].attention[0].kind`), and the collector turns it into a failed fetch:
|
|
410
|
+
a peer that answers with a bad document is **reachable but broken** and visibly
|
|
411
|
+
so, not silently stale.
|
|
412
|
+
|
|
413
|
+
**Forward compatibility is not offered**, and that is the honest report rather
|
|
414
|
+
than a shortcut. A higher `murmur_snapshot` is rejected like any other wrong
|
|
415
|
+
value, because a reader that carried fields it did not understand would be
|
|
416
|
+
guessing about state a human acts on. A version mismatch is an operator-visible
|
|
417
|
+
pairing problem: upgrade the other node.
|
|
418
|
+
|
|
419
|
+
### Collecting
|
|
420
|
+
|
|
421
|
+
Per peer, independently, in a bounded pool (`MAX_CONCURRENT_PEERS = 8`) under a
|
|
422
|
+
whole-collect deadline (`COLLECT_DEADLINE_MS = 4000`, sized under a tmux
|
|
423
|
+
status-bar tick):
|
|
424
|
+
|
|
425
|
+
1. `ssh <target> murmur export` — one round trip, no arguments. There is never a
|
|
426
|
+
second "refetch from zero" trip, because there is no watermark to be wrong.
|
|
427
|
+
2. `parseSnapshot(stdout)`.
|
|
428
|
+
3. `replacePeerSnapshot(name, {ok: true, snapshot, at: now})`, which takes
|
|
429
|
+
`host_id`, `display_name`, `murmur_version` and `snapshot_version` out of the
|
|
430
|
+
document itself. The caller passes no metadata alongside it, so the cache
|
|
431
|
+
cannot disagree with the snapshot it holds.
|
|
432
|
+
|
|
433
|
+
Any failure at any step replaces nothing: `last_attempt_at` and `last_error` are
|
|
434
|
+
set, the previous snapshot stands verbatim, and the peer ages into `stale` on its
|
|
435
|
+
own. There is no third outcome and no path that writes part of a document.
|
|
436
|
+
|
|
437
|
+
Concurrency is about the unreachable peers, not the reachable ones. A sleeping
|
|
438
|
+
laptop holds a forked ssh client for the full connect timeout, and a serial loop
|
|
439
|
+
charged that to every peer behind it: three asleep laptops froze the status bar
|
|
440
|
+
for thirty seconds.
|
|
441
|
+
|
|
442
|
+
`collect` also calls `reconcileLocal` once per invocation, including with zero
|
|
443
|
+
peers. That is the only housekeeping left, and it lives here rather than on
|
|
444
|
+
`export` because `export` only runs when a peer asks, so a single-machine node
|
|
445
|
+
would otherwise reconcile never.
|
|
446
|
+
|
|
447
|
+
### Reconciliation
|
|
448
|
+
|
|
449
|
+
`reconcileLocal` is the only thing that retires local rows. It consults each
|
|
450
|
+
pane and each owner pid once:
|
|
451
|
+
|
|
452
|
+
| Pane live? | Owner pid alive? | `activity` | Action |
|
|
453
|
+
| --- | --- | --- | --- |
|
|
454
|
+
| no | — | — | delete the agent row and all attention for the pane |
|
|
455
|
+
| yes | yes | — | nothing |
|
|
456
|
+
| yes | no | `running` | set `stopped`, upsert `crashed` attention |
|
|
457
|
+
| yes | no | `stopped` | delete the agent row, **keep** attention |
|
|
458
|
+
|
|
459
|
+
Then, unconditionally, attention for any pane that no longer exists is deleted —
|
|
460
|
+
which is what reaps an attention-only pane whose window was closed.
|
|
461
|
+
|
|
462
|
+
The asymmetry in the last two rows is the point. A dead *running* owner is an
|
|
463
|
+
unreported crash and must leave a durable trace; a dead *stopped* owner finished
|
|
464
|
+
normally, so its row is noise, but a `done` it raised is a fact a human has not
|
|
465
|
+
yet seen. For the same reason `releaseAgent` deletes the agent row and not its
|
|
466
|
+
attention: completion must survive the process exiting.
|
|
467
|
+
|
|
468
|
+
`requested_at` is never updated by a repeat, so re-reconciling changes nothing —
|
|
469
|
+
crash attention is idempotent for free, and age keeps meaning "how long this has
|
|
470
|
+
gone unmet".
|
|
113
471
|
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
**
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
**An event log, not a state snapshot.** Current state is a fold over the log,
|
|
147
|
-
which buys three things: there is no state table that can disagree with the log,
|
|
148
|
-
incremental sync is a watermark ("everything after N", idempotent), and history
|
|
149
|
-
is not optional, because the picker's preview shows recent events, so the
|
|
150
|
-
protocol has to ship events rather than derived state.
|
|
151
|
-
|
|
152
|
-
**TypeScript, on install-story grounds.** pi is itself an npm package, so every
|
|
153
|
-
node that runs agents necessarily has Node and npm, so npm is not a new
|
|
154
|
-
dependency on exactly the machines that matter. Python would mean inventing a cross-machine
|
|
155
|
-
install story for the ecosystem that handles it worst, on machines where the
|
|
156
|
-
system interpreter is least yours to touch. The "keep the working script"
|
|
157
|
-
argument is weaker than it looks: those lines worked because dotfiles symlinked
|
|
158
|
-
them, so the moment a second machine needs them they must be packaged from
|
|
159
|
-
scratch either way.
|
|
160
|
-
|
|
161
|
-
## Three ideas to understand before changing anything
|
|
162
|
-
|
|
163
|
-
Everything else is mechanical. These three are the ones to understand before
|
|
164
|
-
changing anything.
|
|
165
|
-
|
|
166
|
-
### 1. State and freshness are different axes, and freshness is two
|
|
167
|
-
|
|
168
|
-
`stale` is not an agent state. The enum stays `working | blocked | done |
|
|
169
|
-
crashed | cleared`.
|
|
170
|
-
|
|
171
|
-
- **state** — what the agent is doing. Authored by the agent, in the log.
|
|
172
|
-
- **freshness** — how current our replica is. Known only by the reader.
|
|
173
|
-
|
|
174
|
-
Putting `stale` in the enum produces unanswerable questions: is a crashed agent
|
|
175
|
-
on an unreachable host `crashed` or `stale`? Both, on different axes.
|
|
176
|
-
|
|
177
|
-
Freshness then splits again, and missing this shipped a bug:
|
|
472
|
+
The liveness probe **fails closed**. `pidAlive` reports death only on `ESRCH`,
|
|
473
|
+
so a probe that cannot answer (`EPERM`) reads as alive. An unknown must never
|
|
474
|
+
let a second writer displace a possibly-live owner, and must never manufacture a
|
|
475
|
+
crash.
|
|
476
|
+
|
|
477
|
+
## The view
|
|
478
|
+
|
|
479
|
+
`paneViews(store, identity, now)` builds `PaneView[]` from `store.localPanes()`
|
|
480
|
+
and each cached peer snapshot, through one mapping function. `renderState`
|
|
481
|
+
picks one word, `viewSort` orders the list, and `RENDER_PRIORITY` is the single
|
|
482
|
+
ordering table both the status bar and the picker import rather than restating.
|
|
483
|
+
|
|
484
|
+
`identity` is required and non-null, because every caller is a command that
|
|
485
|
+
already fails without one. Optional-chaining it is what previously classed every
|
|
486
|
+
row as remote — including local ones — whenever identity was absent.
|
|
487
|
+
|
|
488
|
+
### State and freshness are different axes, and the ages are two
|
|
489
|
+
|
|
490
|
+
`stale` is not a state. It is a property of a NODE.
|
|
491
|
+
|
|
492
|
+
- **Local views are always fresh.** We are the node that authored them.
|
|
493
|
+
- **A remote view takes the freshness of its node**: `fresh` when
|
|
494
|
+
`now - fetched_at <= STALENESS_MS` (60s), else `stale`. A peer never reached
|
|
495
|
+
is stale, not fresh — `null` means the first collect has not succeeded yet, and
|
|
496
|
+
an unreachable host you just added must not read as up to date.
|
|
497
|
+
- **Freshness is never per agent, and no liveness is inferred for a remote
|
|
498
|
+
pane.** A stale node keeps its last-known fields verbatim, beside an explicit
|
|
499
|
+
"stale host" flag. That is the honest presentation: the fields are real, they
|
|
500
|
+
are simply old, and hiding them would lose the only information available.
|
|
501
|
+
|
|
502
|
+
Two clocks, and collapsing them shipped a bug where a dead host's agents
|
|
503
|
+
rendered as live, because the *replica* really was current:
|
|
178
504
|
|
|
179
505
|
| | asks | answers |
|
|
180
506
|
| --- | --- | --- |
|
|
181
|
-
| `fetched_at` | how current is my copy | is the host reachable |
|
|
182
|
-
|
|
|
507
|
+
| `fetched_at` (our clock) | how current is my copy | is the host reachable |
|
|
508
|
+
| `updated_at` (their clock) | how old is this pane's news | is the row worth believing |
|
|
509
|
+
|
|
510
|
+
`snapshot_at` is the third and belongs to the peer too: when that node *built*
|
|
511
|
+
the document. A peer polled one second ago can be serving a three-hour-old
|
|
512
|
+
fact, so `peer list` and `status --json` report both, and the picker shows the
|
|
513
|
+
pane's own age rather than ours.
|
|
514
|
+
|
|
515
|
+
`age()` and `freshness()` are the only two places a duration becomes text or a
|
|
516
|
+
verdict.
|
|
183
517
|
|
|
184
|
-
|
|
185
|
-
these made a dead host's agents render as live, because the replica really was
|
|
186
|
-
current.
|
|
518
|
+
### Facts only the owner can know are recorded by the owner
|
|
187
519
|
|
|
188
|
-
|
|
520
|
+
Pids, pane liveness and window names mean something only on the machine that
|
|
521
|
+
owns them, so reconciliation runs on the owning node and the results travel in
|
|
522
|
+
its snapshot.
|
|
189
523
|
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
the
|
|
524
|
+
Do it on the reader and every remote `running` pane is marked `crashed`, which
|
|
525
|
+
looks exactly like a real crash and so goes uninvestigated. This is the easiest
|
|
526
|
+
thing in the codebase to get backwards, and the model now makes it impossible
|
|
527
|
+
rather than merely inadvisable: the reader holds no pid to probe.
|
|
193
528
|
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
529
|
+
The same rule gives names their home. Window ids are machine-local, so resolving
|
|
530
|
+
a remote id against the local tmux would label an agent with whatever *this*
|
|
531
|
+
machine has at that id. Names travel in the snapshot instead.
|
|
197
532
|
|
|
198
|
-
|
|
199
|
-
a remote id against the local tmux labels an agent with whatever *this* machine
|
|
200
|
-
has at that id. Names travel on the event instead.
|
|
533
|
+
### `clear` may only cancel a request for attention
|
|
201
534
|
|
|
202
|
-
|
|
535
|
+
`murmur clear --pane` runs from tmux focus hooks, so the single fact it knows is
|
|
536
|
+
*the user looked at this pane*. It is one `DELETE FROM attention WHERE pane = ?`
|
|
537
|
+
plus the badge, and it reads no agent row to decide anything.
|
|
538
|
+
|
|
539
|
+
There is no state focus must refuse to clear, because attention is the only
|
|
540
|
+
thing focus can address. That whole class of bug — a whitelist, a resolver call,
|
|
541
|
+
a metadata copy-forward, and the reasoning about which states were clearable —
|
|
542
|
+
is deleted along with the ability to get it wrong. It was the worst bug found
|
|
543
|
+
here: 50 of 84 turns on one agent were cleared within a minute of starting,
|
|
544
|
+
because switching back to a pane wiped the state of the agent running in it.
|
|
545
|
+
|
|
546
|
+
The badge is a window option while "you looked" is true of one pane, so `clear`
|
|
547
|
+
keeps the badge lit when another pane in the same window still wants attention.
|
|
548
|
+
That question is asked of attention only: a busy agent next door is not a reason
|
|
549
|
+
to keep an attention badge on. It fails safe by keeping the badge — wrongly
|
|
550
|
+
keeping one is recoverable by focusing the pane, wrongly clearing one loses the
|
|
551
|
+
signal — and it is silent and total, because it runs inside the tmux server.
|
|
552
|
+
|
|
553
|
+
### The single-machine case is the same code path
|
|
203
554
|
|
|
204
555
|
murmur replaced a 1500-line script that was the daily local tool. Zero peers is
|
|
205
556
|
therefore the common case:
|
|
206
557
|
|
|
207
|
-
- the
|
|
208
|
-
- the status bar and picker read the
|
|
209
|
-
|
|
558
|
+
- the extension claims its pane and reports activity
|
|
559
|
+
- the status bar and picker read `localPanes()` through the same mapping a peer
|
|
560
|
+
snapshot goes through
|
|
561
|
+
- the collector iterates the peer list, finds nothing, and reconciles once
|
|
210
562
|
|
|
211
563
|
No network, no ssh, no daemon, no added latency. Federation is strictly
|
|
212
|
-
additive: a
|
|
564
|
+
additive: a loop over an empty array. Measured first paint with zero peers:
|
|
565
|
+
~50 ms, against 250 ms for the picker it replaces.
|
|
213
566
|
|
|
214
567
|
This is a constraint, not an observation. A tool that only pays for itself at
|
|
215
|
-
three nodes charges rent daily for
|
|
216
|
-
|
|
217
|
-
48 ms against 250 ms for the picker it replaces.
|
|
568
|
+
three nodes charges rent daily for capability used occasionally — one of the
|
|
569
|
+
things herdr was rejected for.
|
|
218
570
|
|
|
219
571
|
## Design choices, and what they cost
|
|
220
572
|
|
|
@@ -223,41 +575,67 @@ latencies. Both are the reader pulling, and OpenSSH `ControlMaster` collapses
|
|
|
223
575
|
them, because the persisted control socket *is* the tunnel. Push needs a
|
|
224
576
|
listener, which is the thing this design does not have.
|
|
225
577
|
|
|
226
|
-
**The collector never
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
578
|
+
**The collector never prompts.** It reuses a warm `ControlMaster` socket when
|
|
579
|
+
there is one and cold-connects otherwise, so with key auth a peer is visible
|
|
580
|
+
whether or not you ssh'd there recently. `BatchMode=yes` makes the cold path
|
|
581
|
+
acceptable: no password, passphrase or host-key prompt, so a peer that cannot
|
|
582
|
+
authenticate fails fast and shows stale instead of blocking on a human.
|
|
583
|
+
|
|
584
|
+
Unhandled: a host demanding a hardware-token touch per connection, which
|
|
585
|
+
`BatchMode` cannot detect before the token blinks. `hasWarmSocket` exists for
|
|
586
|
+
that — gating collect on it per peer would restore the strict posture for those
|
|
587
|
+
hosts only. Not wired up, because no peer in use needs it.
|
|
588
|
+
|
|
589
|
+
**A node being down is the common case, so nothing routine reports it.** A fleet
|
|
590
|
+
normally has a laptop asleep and a box switched off. The collector used to write
|
|
591
|
+
two lines of ssh diagnostics per failed peer, and it runs from `murmur status`
|
|
592
|
+
on every status-bar tick and from `murmur pick` inside a display-popup — so one
|
|
593
|
+
sleeping node wrote to stderr several times a minute, forever, and into a UI.
|
|
594
|
+
|
|
595
|
+
Failures now travel in `collect`'s return value. `murmur collect`, which a human
|
|
596
|
+
runs on purpose, is the only thing that prints, and it distinguishes unreachable
|
|
597
|
+
(expected, exit 0) from reachable-but-broken (an invalid snapshot, a missing
|
|
598
|
+
binary, an auth failure; exit 1). `Permission denied` is deliberately classed as
|
|
599
|
+
reachable-but-broken: an auth misconfiguration is an operator task, not a
|
|
600
|
+
sleeping laptop, and calling it "asleep, probably" is how a fixable setup error
|
|
601
|
+
stays invisible for weeks. Errors are normalised once, where they are caught, so
|
|
602
|
+
`peer list`, `status --json` and any SDK consumer get the diagnosis rather than
|
|
603
|
+
140 characters of murmur's own ssh invocation.
|
|
231
604
|
|
|
232
605
|
**Membership is local and asymmetric.** No shared node list, no registry, no
|
|
233
|
-
join protocol. Reachability is not symmetric: a laptop reaches a server, and
|
|
234
|
-
|
|
606
|
+
join protocol. Reachability is not symmetric: a laptop reaches a server, and the
|
|
607
|
+
server does not reach a laptop behind NAT that sleeps. A global list would
|
|
235
608
|
advertise peers half the fleet cannot use. And nothing needs one: only a node
|
|
236
609
|
rendering a picker needs targets, and only ones it can reach. Identity is
|
|
237
|
-
*discovered
|
|
238
|
-
node's
|
|
610
|
+
*discovered* — config holds an ssh target, and the first export returns the
|
|
611
|
+
node's `host_id` and display name, which `peer add` records immediately rather
|
|
612
|
+
than throwing away.
|
|
239
613
|
|
|
240
|
-
**Zero knobs.** Every setting
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
where the tests go.
|
|
614
|
+
**Zero knobs.** Every exported setting is one the user can get wrong invisibly.
|
|
615
|
+
Two are irreducible: `peers` (only the operator knows their fleet) and `theme`.
|
|
616
|
+
Collection concurrency, deadlines and the staleness threshold are constants. The
|
|
617
|
+
trade: a wrong constant needs a release, not an edit. Heuristics replace knobs,
|
|
618
|
+
so heuristics are where the tests go.
|
|
246
619
|
|
|
247
620
|
**`driver` distinguishes who is waiting.** An agent you are talking to and an
|
|
248
621
|
agent an orchestrator placed want opposite treatment: when the orchestrated one
|
|
249
622
|
finishes, its supervisor consumes the result and nobody needs to acknowledge
|
|
250
|
-
anything. Same
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
623
|
+
anything. Same facts, opposite attention. So an orchestrated agent raises no
|
|
624
|
+
`done` at all, and its rows are hidden from the picker and the status bar unless
|
|
625
|
+
its attention includes `blocked` or `crashed` — the two kinds only a human can
|
|
626
|
+
answer. That list is `NEEDS_HUMAN`, shared by both surfaces, because it was two
|
|
627
|
+
literals in two files and that is how a row needing a human became one a human
|
|
628
|
+
could not see.
|
|
629
|
+
|
|
630
|
+
It is per *agent*, not per node, because the normal case is one machine running
|
|
631
|
+
your session and six spawned workers at once.
|
|
254
632
|
|
|
255
633
|
**Glance, not remote rendering.** "Render any pane from the master" hides two
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
634
|
+
different problems: a stateless `capture-pane` (cheap, and what the preview
|
|
635
|
+
does) and continuous frame streaming with resize negotiation and input routing
|
|
636
|
+
(most of herdr's codebase). Glance plus jump gets everything except never
|
|
637
|
+
leaving the local frame, and you are jumping there to work anyway. This
|
|
638
|
+
deferral is the main reason murmur is small.
|
|
261
639
|
|
|
262
640
|
## Why not something else
|
|
263
641
|
|
|
@@ -269,7 +647,7 @@ run locally rather than judged from their READMEs.
|
|
|
269
647
|
| herdr | no — 1:1, planned, blocked | yes | screen-scraped |
|
|
270
648
|
| T3 Code | no — "unbuilt" by its own docs | no — 14-method adapter | driven |
|
|
271
649
|
| mu | state sync yes, agents no | yes | reported |
|
|
272
|
-
| **murmur** | **yes** | **yes, in-process** | **
|
|
650
|
+
| **murmur** | **yes** | **yes, in-process** | **reported from inside** |
|
|
273
651
|
|
|
274
652
|
### herdr
|
|
275
653
|
|
|
@@ -277,25 +655,23 @@ A Rust terminal multiplexer built for coding agents: workspaces, tabs, panes, a
|
|
|
277
655
|
per-pane `idle/working/blocked/done` sidebar, a socket API. Evaluated as a tmux
|
|
278
656
|
replacement and rejected on its own terms.
|
|
279
657
|
|
|
280
|
-
On multi-machine it is strictly 1:1. `--remote <target>` takes a single
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
progress for some time.
|
|
658
|
+
On multi-machine it is strictly 1:1. `--remote <target>` takes a single target,
|
|
659
|
+
no subcommand has a `--host` flag, and `--remote` *replaces* the view rather
|
|
660
|
+
than adding to it: two machines means two sessions and a full switch between
|
|
661
|
+
them. Multi-client is the maintainer's stated top priority, gated behind a
|
|
662
|
+
long-running server/client refactor.
|
|
286
663
|
|
|
287
|
-
Waiting would not help
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
push-based state from inside the agent for screen-scraping.
|
|
664
|
+
Waiting would not help. The scope is one client attaching to multiple *herdr*
|
|
665
|
+
servers, so every machine must run herdr — including machines where you cannot
|
|
666
|
+
choose the multiplexer. And herdr detects state by matching terminal output, so
|
|
667
|
+
adopting it trades reporting from inside the agent for screen-scraping.
|
|
292
668
|
|
|
293
669
|
Taken from it: integration installs that write hooks into each agent's own
|
|
294
670
|
config directory (`murmur link pi`), reusing one authenticated connection, and
|
|
295
|
-
its own stated non-goals: no merged PTYs across machines, no moving work
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
671
|
+
its own stated non-goals: no merged PTYs across machines, no moving work between
|
|
672
|
+
hosts, host as a lightweight label. Rejected: the always-present sidebar, not
|
|
673
|
+
for its ~4 columns but because it is fixed to one edge and cannot become a
|
|
674
|
+
horizontal strip, while a status row is overhead already paid.
|
|
299
675
|
|
|
300
676
|
### T3 Code
|
|
301
677
|
|
|
@@ -303,14 +679,15 @@ An "agent harness control surface": a server owning agent sessions plus web,
|
|
|
303
679
|
desktop and mobile clients over one RPC WebSocket.
|
|
304
680
|
|
|
305
681
|
Its remote access is well ahead of herdr: direct ws/wss, bearer pairing, relay
|
|
306
|
-
tunnels, mesh-VPN serve and desktop-managed SSH, all shipped. But the aggregated
|
|
307
|
-
|
|
682
|
+
tunnels, mesh-VPN serve and desktop-managed SSH, all shipped. But the aggregated
|
|
683
|
+
view is unbuilt by its own internals docs — multiple live connections exist, a
|
|
684
|
+
fused cross-machine overview does not.
|
|
308
685
|
|
|
309
686
|
It also does not support pi, and adding it is expensive: a provider needs a
|
|
310
|
-
driver plus
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
687
|
+
driver plus a fourteen-method adapter, and the reference implementation is over
|
|
688
|
+
1700 lines. murmur's adapter problem is smaller structurally: T3 Code drives an
|
|
689
|
+
agent it does not live inside, while murmur's extension runs *in process* and
|
|
690
|
+
calls the store directly.
|
|
314
691
|
|
|
315
692
|
Taken from it: the rule that *remoteness is expressed at the connection layer,
|
|
316
693
|
never by splitting the runtime* (murmur's channel seam is exactly this),
|
|
@@ -320,72 +697,218 @@ than a hostname.
|
|
|
320
697
|
### mu
|
|
321
698
|
|
|
322
699
|
An agent orchestrator: workstreams, a task DAG, agents in panes, isolated
|
|
323
|
-
workspaces. It
|
|
324
|
-
|
|
700
|
+
workspaces. It solved machine identity and cross-machine sync for its own
|
|
701
|
+
problem, which is a genuinely harder one: mu replicates *writes* from many
|
|
702
|
+
nodes, so it needs an op-log and per-peer watermarks.
|
|
325
703
|
|
|
326
704
|
Two ideas taken directly. Sync is ambient rather than a daemon: every invocation
|
|
327
|
-
syncs before the verb, and no watcher outlives the command. And sync
|
|
328
|
-
|
|
705
|
+
syncs before the verb, and no watcher outlives the command. And sync never fails
|
|
706
|
+
a command — every ambient entry point is total, and a dead peer warns and
|
|
707
|
+
returns.
|
|
329
708
|
|
|
330
709
|
One idea deliberately not taken: mu's generic replicated KV. A generic op-log
|
|
331
|
-
needs conflict resolution, which single-writer
|
|
332
|
-
a murmur with `put`/`del` over arbitrary entities would just *be* mu.
|
|
710
|
+
needs conflict resolution, which single-writer-per-node caching skips entirely,
|
|
711
|
+
and a murmur with `put`/`del` over arbitrary entities would just *be* mu. The
|
|
712
|
+
same reasoning is why murmur ships no log of its own — nothing here is authored
|
|
713
|
+
by two parties, so there is nothing to merge.
|
|
333
714
|
|
|
334
715
|
**Relationship:** murmur observes, mu orchestrates. Merging is plausible later,
|
|
335
716
|
since murmur would give mu global agent addressing and remote observation. But
|
|
336
717
|
remote *orchestration* is much harder than remote observation, and none of it is
|
|
337
718
|
murmur's problem.
|
|
338
719
|
|
|
720
|
+
## The extension's lifecycle assumptions
|
|
721
|
+
|
|
722
|
+
The extension is the only part of murmur that lives inside another program's
|
|
723
|
+
process, and every bug in it has come from assuming that process is simpler than
|
|
724
|
+
it is.
|
|
725
|
+
|
|
726
|
+
At load, in order, stopping at the first failure: resolve the pane, load the
|
|
727
|
+
identity (never mint one), open the store, and `claimAgent`. Then:
|
|
728
|
+
|
|
729
|
+
| Outcome | Meaning | What the extension does |
|
|
730
|
+
| --- | --- | --- |
|
|
731
|
+
| `claimed` | the pane had no owner | keep the `agent_id`, register handlers |
|
|
732
|
+
| `retained` | same pid re-claiming | keep the same `agent_id` and activity |
|
|
733
|
+
| `replaced` | previous owner is dead | fresh `agent_id`; the old occupant's attention is cleared |
|
|
734
|
+
| `refused` | another live process owns the pane | register nothing, paint nothing, close the store |
|
|
735
|
+
|
|
736
|
+
`retained` is what makes pi's `/reload` a no-op: pi re-runs the extension
|
|
737
|
+
factory in the same process, and a check that could not recognise its own claim
|
|
738
|
+
would silence the real agent. `refused` is permanent for the process — a nested
|
|
739
|
+
agent is deliberately invisible, and it is checked at both the store call and
|
|
740
|
+
the badge, because the badge is painted from the same handler that reports.
|
|
741
|
+
|
|
742
|
+
`replaced` clears the previous occupant's attention because that attention
|
|
743
|
+
described a process that is gone, and a human looking at the pane now sees a
|
|
744
|
+
different agent.
|
|
745
|
+
|
|
746
|
+
Handlers are serialised through a single-slot promise chain. `activity` is
|
|
747
|
+
last-write-wins, so an inverted `start`/`end` pair would leave a finished agent
|
|
748
|
+
`running`.
|
|
749
|
+
|
|
750
|
+
`setActivity` returning `false` is not an error and is not retried: it means
|
|
751
|
+
this process is no longer the owner of record, and the correct response is
|
|
752
|
+
silence. The badge is gated on that boolean, so a process that cannot report
|
|
753
|
+
cannot paint either.
|
|
754
|
+
|
|
755
|
+
Three assumptions that were wrong, all of which failed silently:
|
|
756
|
+
|
|
757
|
+
- **`session_shutdown` is not "the process is exiting".** pi fires it for
|
|
758
|
+
`/reload`, session switch, resume and fork, then rebinds and continues with the
|
|
759
|
+
same instance. Cleanup belongs there; re-arming belongs in `session_start`,
|
|
760
|
+
which fires afterwards. Treating shutdown as terminal stopped reporting
|
|
761
|
+
permanently on the first `/reload`.
|
|
762
|
+
- **`agent_end` is per RUN, and is not the end of the work.** `turn_start` /
|
|
763
|
+
`turn_end` are the per-turn events; one prompt with three tool calls fires one
|
|
764
|
+
`agent_start`, three `turn_end`, one `agent_end`. But `agent_end` can fire
|
|
765
|
+
several times before the agent is finished, because pi re-enters the loop for a
|
|
766
|
+
retry, a compaction or a queued continuation, each with its own `agent_start`.
|
|
767
|
+
So `start, end, start, end, settled` is a normal sequence, and a single
|
|
768
|
+
`agent_end` cannot mean "waiting for a human". `agent_settled` is the event
|
|
769
|
+
that means it: fired once, last. Listening to the wrong event is why the
|
|
770
|
+
highest-attention state in the model had no producer for months.
|
|
771
|
+
- **The pane outlives the window.** A pane can move between windows, keeping its
|
|
772
|
+
id while the window id changes, so the location is re-read on every report
|
|
773
|
+
rather than cached at startup, and a move hands the badge over to the new
|
|
774
|
+
window.
|
|
775
|
+
|
|
776
|
+
The store handle has three states — `untried`, `open`, `absent` — because those
|
|
777
|
+
are three real situations. Collapsing them is what previously latched reporting
|
|
778
|
+
off for the life of a process after one transient failure.
|
|
779
|
+
|
|
780
|
+
The general rule: nothing in the extension may be permanent except "murmur is
|
|
781
|
+
not installed here" and "this pane is not mine". Every other giving-up must be
|
|
782
|
+
recoverable by the next event, because the process it lives in can run for days
|
|
783
|
+
and the user can fix the cause from outside without restarting it.
|
|
784
|
+
|
|
339
785
|
## Testing posture
|
|
340
786
|
|
|
341
787
|
Bug-driven, not coverage-driven. A thing earns a test when it can fail *without
|
|
342
|
-
you noticing
|
|
788
|
+
you noticing*. The suite is 238 tests over 28 files, and the ones that matter
|
|
789
|
+
assert what is **impossible** rather than what is implemented:
|
|
343
790
|
|
|
344
791
|
| Target | Why it can fail silently |
|
|
345
792
|
| ------ | ------------------------ |
|
|
346
|
-
|
|
|
347
|
-
|
|
|
348
|
-
|
|
|
349
|
-
|
|
|
350
|
-
|
|
|
793
|
+
| A notifier cannot touch an agent row | The corruption it caused looked like a state change |
|
|
794
|
+
| `acknowledgePane` cannot change activity | A focus hook wiping a busy agent reads as the agent stopping |
|
|
795
|
+
| A second live claimant is refused | A nested run reporting looks like the real agent misbehaving |
|
|
796
|
+
| A replaced owner's writes return false | A late write from a dead process corrupts a live row |
|
|
797
|
+
| Reconciliation asymmetry | An idle agent vanishing reads as "not there" |
|
|
798
|
+
| Whole-snapshot replacement | A pane that should be gone lingers forever |
|
|
799
|
+
| Validation before storage | An unknown value reaching a sort or a count |
|
|
800
|
+
| No `owner_pid` on any read path | Remote liveness inference creeping back in |
|
|
801
|
+
| Freshness derivation | A dead peer showing last-known state as current |
|
|
802
|
+
| Jump failure mutating nothing | A keypress deleting a healthy agent |
|
|
803
|
+
|
|
804
|
+
Several of those are asserted structurally — over the whole returned object
|
|
805
|
+
graph, not by reading the type — because a type says what the author intended
|
|
806
|
+
and a test says what the object contains.
|
|
351
807
|
|
|
352
808
|
Not tested: ssh transport (OpenSSH's job), tmux wrappers (thin and loud), TUI
|
|
353
809
|
rendering, packaging.
|
|
354
810
|
|
|
355
811
|
New tests are verified by breaking the code they cover and watching them fail. A
|
|
356
|
-
test that has never failed has not been shown to test anything.
|
|
357
|
-
|
|
358
|
-
|
|
812
|
+
test that has never failed has not been shown to test anything. A test asserting
|
|
813
|
+
a *wrong* behaviour has been written twice here, so this step is not ceremony.
|
|
814
|
+
|
|
815
|
+
Three trap shapes this caught, all in tests rather than code. A `setTimeout(0)`
|
|
816
|
+
standing in for a barrier passed about nine runs in ten — worse than a failing
|
|
817
|
+
test, because it teaches people to re-run. A `until()` helper that threw on
|
|
818
|
+
timeout made a mutation "pass", because the regression died inside the helper
|
|
819
|
+
instead of at the assertion describing it. And `expect(output).not.toContain(n)`
|
|
820
|
+
over a live number passes for the wrong reason as soon as `n` changes; asserting
|
|
821
|
+
over a closed key set is strictly better, and this appeared three separate
|
|
822
|
+
times.
|
|
823
|
+
|
|
824
|
+
Tests must not touch the developer's own state. `stateDir()` is repointed for
|
|
825
|
+
every test process, in-process and spawned, and no test addresses the caller's
|
|
826
|
+
`$TMUX_PANE`. This is guarded by a test of its own, because it is not
|
|
827
|
+
hypothetical: writing the rewrite contract required one `npm run check` to
|
|
828
|
+
confirm the tree was green, and that run corrupted the author's live state for
|
|
829
|
+
the pane it was running in — the third independent reproduction of the same bug.
|
|
830
|
+
|
|
831
|
+
Where a fake is the test's weak point, the test talks to the real thing.
|
|
832
|
+
`test/mux-targets.test.ts` drives a private tmux server on its own `-L` socket,
|
|
833
|
+
because every other test fakes the `Mux` and a malformed tmux target passes them
|
|
834
|
+
all — two wrong target spellings did exactly that.
|
|
359
835
|
|
|
360
836
|
**The thing to know before adding a feature:** most bugs worth fixing here were
|
|
361
837
|
invisible to unit tests and to reading the code. A silently no-opping extension,
|
|
362
838
|
a remote jump broken by two independent shell-quoting layers, a 75-second hang
|
|
363
|
-
on an unreachable peer, ages that measured the wrong clock. All
|
|
364
|
-
|
|
365
|
-
|
|
839
|
+
on an unreachable peer, ages that measured the wrong clock. All surfaced by
|
|
840
|
+
running the thing on two real machines. Unit tests protect the heuristics; they
|
|
841
|
+
do not tell you the tool is usable.
|
|
366
842
|
|
|
367
843
|
## Deliberate non-goals
|
|
368
844
|
|
|
369
845
|
Each was considered and refused with reasons above:
|
|
370
846
|
|
|
371
847
|
- a daemon or listening socket
|
|
848
|
+
- history of any kind: an event log, a state timeline, an audit trail
|
|
372
849
|
- interactive remote terminal rendering
|
|
373
850
|
- orchestration or work placement
|
|
374
851
|
- a generic put/del op-log
|
|
375
852
|
- HLC or clock reconciliation
|
|
376
|
-
- gossip replication
|
|
853
|
+
- gossip replication
|
|
377
854
|
- configuration knobs beyond peers and theme
|
|
378
|
-
-
|
|
855
|
+
- multiplexers other than tmux, channels other than ssh
|
|
856
|
+
- an in-process integration for any harness other than pi. `murmur notify` is
|
|
857
|
+
the outside-in path for the rest, and it is not the same thing: a harness that
|
|
858
|
+
can only run a command when something happens can ask for attention, but
|
|
859
|
+
cannot report activity or ownership.
|
|
860
|
+
|
|
861
|
+
## Accepted limitations
|
|
862
|
+
|
|
863
|
+
These are design, not surprise. Each is a consequence of the model above, and
|
|
864
|
+
each was cheaper to accept than to solve:
|
|
865
|
+
|
|
866
|
+
1. **No history.** Only current state is stored. There is no event log, no
|
|
867
|
+
retention horizon, and no way to ask what an agent was doing an hour ago. The
|
|
868
|
+
picker's preview is a live `capture-pane`, not a replay.
|
|
869
|
+
2. **No compatibility with older nodes.** A pre-rewrite `events.db` is not
|
|
870
|
+
migrated, and any `state.db` from a different `user_version` is rebuilt with
|
|
871
|
+
only peer names and targets salvaged. A node serving the old event format is
|
|
872
|
+
reported as reachable-but-broken, which is the honest description: the two
|
|
873
|
+
cannot federate.
|
|
874
|
+
3. **No incremental sync.** Every collect transfers each peer's whole snapshot.
|
|
875
|
+
It is bounded by live pane count, so it is small — but it is O(panes) per
|
|
876
|
+
tick rather than O(changes).
|
|
877
|
+
4. **No nested agents.** A second live process in one pane is refused and
|
|
878
|
+
reports nothing. A pane shows at most one agent.
|
|
879
|
+
5. **No remote liveness inference.** `owner_pid` never crosses the wire, so a
|
|
880
|
+
remote pane's `activity` is whatever its own node last said. A stale node
|
|
881
|
+
keeps its last-known values beside a warning, and murmur will not guess.
|
|
882
|
+
6. **PID reuse can hide a crash.** If a pane's owner dies and the OS reassigns
|
|
883
|
+
its pid before the next `reconcileLocal`, that owner reads as alive and the
|
|
884
|
+
crash is missed until something else changes. The window is short and the
|
|
885
|
+
failure is temporary; the alternatives (pid start time, cgroups, a watcher)
|
|
886
|
+
each buy a platform dependency for a case measured as rare.
|
|
887
|
+
7. **A claim costs a liveness probe, and an unprobeable owner blocks the pane.**
|
|
888
|
+
`claimAgent` fails closed, so a pid that answers `EPERM` refuses the new
|
|
889
|
+
claimant and the pane stays unclaimable until it goes away. Deliberate: the
|
|
890
|
+
alternative is letting a second writer displace a possibly-live owner.
|
|
891
|
+
8. **Attention is per pane, not per agent.** A pane whose agent is replaced
|
|
892
|
+
loses the previous occupant's attention, which is intended, and two
|
|
893
|
+
sequential agents in one pane cannot each hold their own `done`.
|
|
379
894
|
|
|
380
895
|
## Known gaps
|
|
381
896
|
|
|
382
|
-
- **
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
897
|
+
- **The remote wrapper session is verified against tmux, not against use.**
|
|
898
|
+
Options, the switch, and the return to the origin window are verified on real
|
|
899
|
+
tmux servers with a stub ssh. Sitting in a live remote pane and working in it
|
|
900
|
+
is not.
|
|
901
|
+
- **Interactive attach is unverified.** Federation, staleness, snapshot
|
|
902
|
+
replacement and the jump target are verified across two machines over real
|
|
903
|
+
ssh; sitting in a remote pane and working in it is not, and it needs a human
|
|
904
|
+
at a terminal.
|
|
905
|
+
- **A deadline-cut peer reports `unreachable: false`.** The message says the
|
|
906
|
+
collect deadline passed, which is accurate and is what every surface prints,
|
|
907
|
+
but the flag reads as "reachable". No surface misreports it today; the flag is
|
|
908
|
+
still the wrong shape for that one case.
|
|
909
|
+
- **`peer add` cannot always refuse a self-add.** The refusal keys on the
|
|
910
|
+
`host_id` in the probe's snapshot, so a target that fails to answer is added
|
|
911
|
+
on the operator's word. Correct by design, but best-effort rather than a
|
|
912
|
+
guarantee.
|
|
388
913
|
- **The hardware-token path is verified only in the cold-fail direction.** The
|
|
389
914
|
second test node authenticates by key, so it never needed a warm socket.
|
|
390
|
-
- **Non-pi harnesses have no attention path.** Agents that cannot report from
|
|
391
|
-
inside themselves need an outside-in `notify` verb, which does not exist.
|