@dumbmice/pi-mailbridge 0.0.0-stage → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +836 -0
- package/LICENSE +21 -0
- package/README.md +598 -2
- package/bin/pi-mailbridge.mjs +31 -0
- package/config.example.json +54 -0
- package/dist/core/attachments.js +87 -0
- package/dist/core/authz.js +65 -0
- package/dist/core/commands.js +72 -0
- package/dist/core/config-freshness.js +24 -0
- package/dist/core/config.js +286 -0
- package/dist/core/deliveries.js +180 -0
- package/dist/core/discover.js +282 -0
- package/dist/core/header-encoding.js +70 -0
- package/dist/core/html.js +48 -0
- package/dist/core/inline-images.js +355 -0
- package/dist/core/mail-command.js +62 -0
- package/dist/core/mailer.js +126 -0
- package/dist/core/markdown.js +96 -0
- package/dist/core/oauth-device.js +119 -0
- package/dist/core/oauth-openid.js +48 -0
- package/dist/core/oauth.js +137 -0
- package/dist/core/prime.js +51 -0
- package/dist/core/reply-parse.js +134 -0
- package/dist/core/sent-index.js +135 -0
- package/dist/core/settle.js +122 -0
- package/dist/core/setup.js +100 -0
- package/dist/core/smoke.js +164 -0
- package/dist/core/status.js +56 -0
- package/dist/core/token-source.js +27 -0
- package/dist/core/token.js +196 -0
- package/dist/core/version.js +44 -0
- package/dist/daemon/api.js +46 -0
- package/dist/daemon/imapflow.js +75 -0
- package/dist/daemon/main.js +802 -0
- package/dist/daemon/process.js +270 -0
- package/dist/daemon/scan-queue.js +38 -0
- package/dist/daemon/script.js +44 -0
- package/dist/daemon/send-budget.js +30 -0
- package/dist/daemon/service.js +455 -0
- package/dist/daemon/source-proxy.js +64 -0
- package/dist/daemon/spawn.js +227 -0
- package/dist/daemon/watcher.js +308 -0
- package/dist/extension.js +1233 -0
- package/dist/store/journal.js +92 -0
- package/dist/store/mode-store.js +47 -0
- package/dist/store/oauth-store.js +50 -0
- package/dist/store/paths.js +33 -0
- package/dist/store/pending-store.js +212 -0
- package/dist/store/registry.js +61 -0
- package/dist/store/routes.js +223 -0
- package/dist/store/schema.js +93 -0
- package/dist/store/secret.js +70 -0
- package/dist/store/threads.js +145 -0
- package/dist/store/uid-cursor.js +112 -0
- package/docs/ARCHITECTURE.md +356 -0
- package/docs/MAPPING.md +201 -0
- package/docs/OAUTH.md +448 -0
- package/docs/SECURITY.md +273 -0
- package/docs/images/flow.svg +124 -0
- package/docs/images/notification.png +0 -0
- package/package.json +85 -4
- package/scripts/render-teaser.ts +56 -0
- package/src/core/attachments.ts +129 -0
- package/src/core/authz.ts +87 -0
- package/src/core/commands.ts +99 -0
- package/src/core/config-freshness.ts +33 -0
- package/src/core/config.ts +373 -0
- package/src/core/deliveries.ts +233 -0
- package/src/core/discover.ts +333 -0
- package/src/core/header-encoding.ts +75 -0
- package/src/core/html.ts +62 -0
- package/src/core/inline-images.ts +421 -0
- package/src/core/mail-command.ts +76 -0
- package/src/core/mailer.ts +193 -0
- package/src/core/markdown.ts +114 -0
- package/src/core/oauth-device.ts +195 -0
- package/src/core/oauth-openid.ts +61 -0
- package/src/core/oauth.ts +182 -0
- package/src/core/prime.ts +74 -0
- package/src/core/reply-parse.ts +153 -0
- package/src/core/sent-index.ts +164 -0
- package/src/core/settle.ts +179 -0
- package/src/core/setup.ts +139 -0
- package/src/core/smoke.ts +207 -0
- package/src/core/status.ts +75 -0
- package/src/core/token-source.ts +34 -0
- package/src/core/token.ts +240 -0
- package/src/core/version.ts +45 -0
- package/src/daemon/api.ts +52 -0
- package/src/daemon/imapflow.ts +101 -0
- package/src/daemon/main.ts +832 -0
- package/src/daemon/process.ts +348 -0
- package/src/daemon/scan-queue.ts +45 -0
- package/src/daemon/script.ts +45 -0
- package/src/daemon/send-budget.ts +36 -0
- package/src/daemon/service.ts +550 -0
- package/src/daemon/source-proxy.ts +88 -0
- package/src/daemon/spawn.ts +332 -0
- package/src/daemon/watcher.ts +427 -0
- package/src/extension.ts +1381 -0
- package/src/store/journal.ts +104 -0
- package/src/store/mode-store.ts +60 -0
- package/src/store/oauth-store.ts +55 -0
- package/src/store/paths.ts +38 -0
- package/src/store/pending-store.ts +282 -0
- package/src/store/registry.ts +91 -0
- package/src/store/routes.ts +267 -0
- package/src/store/schema.ts +104 -0
- package/src/store/secret.ts +69 -0
- package/src/store/threads.ts +177 -0
- package/src/store/uid-cursor.ts +135 -0
|
@@ -0,0 +1,356 @@
|
|
|
1
|
+
# ARCHITECTURE
|
|
2
|
+
|
|
3
|
+
## 1. Shape
|
|
4
|
+
|
|
5
|
+
```
|
|
6
|
+
one mailbox
|
|
7
|
+
│
|
|
8
|
+
┌──────────────▼───────────────┐
|
|
9
|
+
│ daemon (one per machine) │ IMAP IDLE, 1 connection
|
|
10
|
+
│ port 4177, token-auth'd │ UID-cursor scan
|
|
11
|
+
│ │
|
|
12
|
+
│ pending store (durable) │
|
|
13
|
+
│ registry (1 decision/sess) │
|
|
14
|
+
└───────┬──────────────┬───────┘
|
|
15
|
+
│ poll: /pending│ poll: /mode
|
|
16
|
+
┌─────────▼────┐ ┌─────▼────────┐
|
|
17
|
+
│ pi session A │ │ pi session B │ thin clients (in-process extensions)
|
|
18
|
+
│ claim → ack │ │ claim → ack │
|
|
19
|
+
└──────────────┘ └──────────────┘
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
**One watcher, many sessions.** The daemon owns the single IMAP connection (plus one short-lived read-back per notification when
|
|
23
|
+
`mail.imap.sentFolder` is set); pi
|
|
24
|
+
sessions are thin clients over loopback HTTP: they poll `/pending` and replay it, which
|
|
25
|
+
needs no reconnect logic because replay is idempotent. (`/events` also offers SSE for a
|
|
26
|
+
push client; nothing uses it today.) This is the same
|
|
27
|
+
topology as `opencode-afk` and it is the reason multi-session works: there is no
|
|
28
|
+
N-way race for mailbox state, because only one process reads the mailbox.
|
|
29
|
+
|
|
30
|
+
The extension (inside pi) never touches IMAP or SMTP on the delivery path: it sends mail
|
|
31
|
+
through the daemon and receives replies from it. The one exception is the `/mailbridge
|
|
32
|
+
smoke` diagnostic, which sends a real mail and then opens the folder over its own
|
|
33
|
+
short-lived connection to prove both sides work (and `/mailbridge prime`, which opens one
|
|
34
|
+
to move the cursor without reading anything).
|
|
35
|
+
|
|
36
|
+
## 2. Delivery pipeline (the part that must not lose mail)
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
new UID ──► fetch ──► self-echo marker ──► authorize ──► route (id / token / chain) ──► persist pending (durable)
|
|
40
|
+
│
|
|
41
|
+
▼
|
|
42
|
+
waiting for the next poll tick
|
|
43
|
+
│
|
|
44
|
+
┌─────────────────────────┴──────────┐
|
|
45
|
+
│ every session sees it; only the │
|
|
46
|
+
│ one whose session id matches claims│
|
|
47
|
+
└─────────────────────────┬──────────┘
|
|
48
|
+
▼
|
|
49
|
+
pi.sendUserMessage(body, followUp)
|
|
50
|
+
│
|
|
51
|
+
▼
|
|
52
|
+
ack ──► \Seen + journal append
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Invariants:
|
|
56
|
+
|
|
57
|
+
- **Nothing is marked `\Seen` before the ack.** A crash at any earlier point
|
|
58
|
+
leaves mail unseen and the pending entry durable → retried, not lost.
|
|
59
|
+
- **The cursor only advances after persist**, never after broadcast.
|
|
60
|
+
- **At-least-once, deduped by journal.** A duplicated delivery is caught by the
|
|
61
|
+
journal (UID already acked), not by timing.
|
|
62
|
+
- **The claim is short-lived.** It prevents two sessions with the same id (an
|
|
63
|
+
old and new runtime during `/reload`) from injecting twice.
|
|
64
|
+
|
|
65
|
+
## 3. Daemon HTTP surface
|
|
66
|
+
|
|
67
|
+
Loopback only by default (`daemon.host`). Every request except `/health` carries
|
|
68
|
+
`Authorization: Bearer <token>` from
|
|
69
|
+
`$XDG_STATE_HOME/pi-mailbridge/secret.key`; comparison is constant-time.
|
|
70
|
+
|
|
71
|
+
| Method | Path | Purpose |
|
|
72
|
+
| --- | --- | --- |
|
|
73
|
+
| GET | `/health` | liveness probe; **unauthenticated** by design (mode, pending count, the decision-registry size, emitted as both `sessions` and `registry` — sessions *with an outstanding decision*, not live sessions; live ones are `clients` — ping timing, SSE subscribers, the last scan's summary, `configLoadedAt`, `version`, `apiVersion` and `capabilities`; no content, no addresses) |
|
|
74
|
+
| GET | `/events` | SSE: `delivery`, `mode` (no client uses it; sessions poll) |
|
|
75
|
+
| GET | `/pending` | the delivery channel: polled every 10 s, replayed on reconnect; returns the daemon's own device's records plus untagged ones |
|
|
76
|
+
| POST | `/claim` | `{uid, instanceId, uidValidity?}` — atomic claim; refused as `stale-space` for another UID space and `foreign-device` for a record another device scanned |
|
|
77
|
+
| POST | `/ack` | `{uid, instanceId, uidValidity?}` — claimant-only; refused as `stale-space` when the record (or the caller) belongs to another UID space; journals in the **record's** space, then `\Seen` + prune |
|
|
78
|
+
| POST | `/register` | reserve a session's single outstanding decision |
|
|
79
|
+
| POST | `/bye` | a session is leaving; the daemon exits when the last one goes |
|
|
80
|
+
| POST | `/ping` | heartbeat, the other half of the idle exit |
|
|
81
|
+
| DELETE | `/register` | release the reservation |
|
|
82
|
+
| GET/POST | `/mode` | global `on`/`off`, persisted + broadcast |
|
|
83
|
+
| POST | `/send` | send one mail via the daemon's SMTP config; at most ten `notification` sends a minute, then `429 send_budget` (`kind: "settle"` and `"decision"` are exempt) |
|
|
84
|
+
| POST | `/reload` | re-read config: stop the watcher, reload, redial. On failure it restores the previous config and warns; a refused connect returns `ok` with a warning |
|
|
85
|
+
| POST | `/shutdown` | stop the daemon (used by the restart path) |
|
|
86
|
+
|
|
87
|
+
Bounds: body ≤ 1 MB, registry ≤ 10 000 entries, SSE clients ≤ 100. A second daemon start exits 0
|
|
88
|
+
on `daemon.lock` (and, as a second guard, on `EADDRINUSE`) — quiet by design.
|
|
89
|
+
|
|
90
|
+
## 4. Routing token
|
|
91
|
+
|
|
92
|
+
Subject format:
|
|
93
|
+
|
|
94
|
+
```
|
|
95
|
+
[pi:f913c9] <human subject>
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
A six- to eight-hex-character **local route id**, resolved from `store/routes.ts`. It replaced a
|
|
99
|
+
signed token (39 characters, in an 87-character subject) that made every subject unreadable in a notification
|
|
100
|
+
preview. Signed tokens still verify, so mail already in flight keeps working.
|
|
101
|
+
|
|
102
|
+
The id is a lookup key, not a credential: an unknown id resolves to nothing, and
|
|
103
|
+
authorization remains the sender allowlist plus DMARC (SECURITY.md §2.2) — except in the
|
|
104
|
+
single-mailbox layout, where DMARC is off (README, One mailbox or two?). Because routes are
|
|
105
|
+
local, an id minted on one machine simply does not resolve on another — which is the
|
|
106
|
+
desired behaviour, not an error.
|
|
107
|
+
|
|
108
|
+
- A local lookup key, not a signature: an id that was never minted here resolves to
|
|
109
|
+
nothing. (Legacy tokens *are* HMAC-signed, and are verified when they appear.)
|
|
110
|
+
- Records the **device that minted it**. A daemon only persists a delivery whose
|
|
111
|
+
route (or chain entry, or legacy token) names its own device, so several machines
|
|
112
|
+
sharing one mailbox never hold stale copies of each other's replies. A record with
|
|
113
|
+
no device claim is accepted (single-device installs).
|
|
114
|
+
- Replies keep the id because mail clients prefix — `Re: [pi:…] …`. Parsing
|
|
115
|
+
tolerates `Re:`, `RE:`, `AW:`, `Re[2]:`.
|
|
116
|
+
- Expiry bounds replay: an id older than `tokenTtlHours` resolves to nothing, and the
|
|
117
|
+
reply chain ages out on the same clock.
|
|
118
|
+
|
|
119
|
+
### What identifies a reply
|
|
120
|
+
|
|
121
|
+
An inbound message is matched to a session by the first rule that succeeds:
|
|
122
|
+
|
|
123
|
+
1. **Short route id** — `[pi:f913c9]`, matched anywhere in the subject, so `Re:` /
|
|
124
|
+
`RE:` / `Aw:` / `Re[2]:` prefixes are irrelevant. This is what every new
|
|
125
|
+
notification carries. It is a lookup key into this machine's `store/routes.ts`, not a
|
|
126
|
+
signature: an id minted elsewhere (or expired) resolves to nothing, and then rule 2 is
|
|
127
|
+
tried.
|
|
128
|
+
2. **Legacy signed token** — `[pi:<payload>.<mac>]`, a 39-character HMAC-signed payload
|
|
129
|
+
carrying `sessionId`, expiry and the minting device. Still verified so mail sent before
|
|
130
|
+
the short id keeps working; no longer minted.
|
|
131
|
+
3. **Reply chain** — `In-Reply-To`, or any id in `References`, equals a Message-ID
|
|
132
|
+
recorded in `store/threads.ts`. This is what survives a human editing the
|
|
133
|
+
subject or a client rewriting it. The chain is searched newest-first, so the
|
|
134
|
+
closest ancestor wins, and entries age out with `security.tokenTtlHours` so the chain
|
|
135
|
+
cannot route mail forever.
|
|
136
|
+
Some providers replace the `Message-ID` we stamp (QQ rewrites it to `<tencent_…@qq.com>`),
|
|
137
|
+
so the id the reply actually references is one we never minted. With
|
|
138
|
+
`mail.imap.sentFolder` set — which `setup` writes automatically for `qq.com` and
|
|
139
|
+
`foxmail.com`, so a QQ install holds this extra connection by default — the daemon reads the
|
|
140
|
+
just-sent copy back and records the
|
|
141
|
+
delivered id as well. Best-effort: it retries for about two minutes — six attempts totalling 112 s (the provider appends the
|
|
142
|
+
copy asynchronously) and matches on the `X-Pi-Mailbridge-Id` header we stamp, so it can only
|
|
143
|
+
ever record the copy of that exact notification. A capture in flight when the daemon stops
|
|
144
|
+
is lost, and that notification falls back to the subject id.
|
|
145
|
+
4. Otherwise the message is rejected (`no-token`) and left unread - the cursor has moved
|
|
146
|
+
past it, so the retry is a new reply, not an edited subject.
|
|
147
|
+
|
|
148
|
+
None of these is authentication: they route, they do not authorize. Authorization is the
|
|
149
|
+
allowlist plus DMARC (off in the single-mailbox layout), and it happens before any of this
|
|
150
|
+
(§2). A signed token is not
|
|
151
|
+
stronger than a short id for routing purposes — the signature stops someone *predicting*
|
|
152
|
+
an id, but guessing is pointless when any unresolvable id simply falls through to no
|
|
153
|
+
route. `security.allowHeaderThreading: false` disables rule 3 only.
|
|
154
|
+
|
|
155
|
+
The token is **not** authentication. It routes; it does not authorize. See
|
|
156
|
+
SECURITY.md.
|
|
157
|
+
|
|
158
|
+
### Device identity
|
|
159
|
+
|
|
160
|
+
`device.name` defaults to the first label of `os.hostname()`. It appears in every **settle**
|
|
161
|
+
notification's subject and body, and is stored on the routing record so the daemon that
|
|
162
|
+
minted the reply can claim it. (The two agent tools carry the routing id and their own
|
|
163
|
+
text, not a device line.) It exists for two reasons:
|
|
164
|
+
|
|
165
|
+
1. **Human routing.** With pi on a laptop, a desktop and a build box, "which
|
|
166
|
+
machine is asking?" must be answerable from a notification preview.
|
|
167
|
+
2. **Machine routing.** Two daemons watching one mailbox would otherwise both
|
|
168
|
+
fetch every reply. The device claim lets the non-owner skip it at scan time
|
|
169
|
+
(`other-device`), without persisting and without marking it seen.
|
|
170
|
+
|
|
171
|
+
### UIDVALIDITY, and what a renumbered mailbox does
|
|
172
|
+
|
|
173
|
+
UIDs are only meaningful within one `UIDVALIDITY`. If a provider recreates, migrates or
|
|
174
|
+
"repairs" a folder, UIDs restart at 1 - and a persisted high-water cursor then skips every
|
|
175
|
+
future message, with `examined 0` as the only symptom.
|
|
176
|
+
|
|
177
|
+
The daemon records the mailbox's `UIDVALIDITY` beside the cursor and compares it on every
|
|
178
|
+
connect. On a change it resets the cursor (except the first observation after an upgrade, unless the
|
|
179
|
+
cursor has fallen outside the new range) and ignores `journal.json` entries from the old
|
|
180
|
+
numbering (they describe a space that no longer exists). Queued records tagged with another numbering are dropped
|
|
181
|
+
on **every** connect, not only on a change - a crash between the cursor write and the drop
|
|
182
|
+
would otherwise leave a record that nothing can claim and nothing ever removes. Records with
|
|
183
|
+
*no* space tag predate that field: an ordinary connect keeps them (an upgrade must not discard
|
|
184
|
+
a queued reply), but a detected renumbering drops them too, because they cannot be shown to
|
|
185
|
+
belong to the new numbering - live, one was claimed there and delivered a different message's
|
|
186
|
+
body.
|
|
187
|
+
The cost is explicit: **a renumbering can re-deliver replies that were acked inside the
|
|
188
|
+
route TTL**, because the only durable dedupe is UID-based. Nothing is lost; recent mail may
|
|
189
|
+
arrive once more. The daemon logs the change, the reset and how many records it dropped.
|
|
190
|
+
|
|
191
|
+
## 5. Modes
|
|
192
|
+
|
|
193
|
+
`/mailbridge on` → mode `on`: decisions mail the human and end the turn.
|
|
194
|
+
`/mailbridge off` → mode `off`: decisions use the terminal as usual. A drain re-checks the mode
|
|
195
|
+
between records, so an `off` arriving as a reply stops the batch it came in — the records behind
|
|
196
|
+
it stay pending and unseen until the mode goes back on.
|
|
197
|
+
|
|
198
|
+
Mode is global per machine (one human, one mailbox) and persisted, so a daemon
|
|
199
|
+
restart does not silently re-enable mailing. Each session reads it on its poll tick (the
|
|
200
|
+
daemon also broadcasts it to any `/events` subscriber, of which there are none today).
|
|
201
|
+
|
|
202
|
+
## 6. Deliberately out of scope: creating sessions
|
|
203
|
+
|
|
204
|
+
`ctx.newSession()` **replaces** the active session in-process, and only a
|
|
205
|
+
command handler can call it — which is why `opencode-afk` needs a two-phase
|
|
206
|
+
`create`→`resume` dance and why `pi-wechat-assistant` gates on `ctx.isIdle()`.
|
|
207
|
+
|
|
208
|
+
pi-mailbridge does not create or switch sessions. It notifies about, and
|
|
209
|
+
resumes, sessions the human already started. Anything else needs a different
|
|
210
|
+
process topology (an RPC supervisor owning `pi --mode rpc` children).
|
|
211
|
+
|
|
212
|
+
## 6a. Lifetime: singleton, and it stops when you do
|
|
213
|
+
|
|
214
|
+
**Who starts it.** Whichever comes first: the extension on `session_start`; `/mailbridge
|
|
215
|
+
setup` or `/mailbridge restart` when the session has no runtime yet; or the poll tick, which respawns a daemon that exited while the session was
|
|
216
|
+
still alive. It is spawned *detached* so it outlives the process that started it and can
|
|
217
|
+
serve other sessions. When a session finds one already running, it compares `/health`'s
|
|
218
|
+
`configLoadedAt` with `config.json`'s mtime and issues `/reload` if the file is newer, so a
|
|
219
|
+
hand-edited config is not served stale. Two limits worth knowing: the comparison is mtime
|
|
220
|
+
based, so an edit that preserves the mtime (`cp -p`, a restore) is not noticed until
|
|
221
|
+
`smoke`/`restart`; and only the daemon re-reads — the session's own copy of the config
|
|
222
|
+
refreshes on the next `/mailbridge` command or pi restart.
|
|
223
|
+
|
|
224
|
+
**Singleton.** `daemon.lock` in the state directory is the lock, with the owner's pid inside: a
|
|
225
|
+
live holder makes the newcomer exit 0, and a file left by a crash is stolen by an atomic rename
|
|
226
|
+
(not an unlink-then-retake, which lets two starters both win). The listening port is a second
|
|
227
|
+
guard, for a foreign listener - a bind failure also exits 0. `ensureDaemon()` then probes `/health` and reuses the winner.
|
|
228
|
+
|
|
229
|
+
**Two versions, two clocks.** `/health` carries both: `apiVersion` (the wire contract, `API_VERSION`)
|
|
230
|
+
and `version` (the package build, diagnostic). A session replaces a daemon whose `apiVersion`
|
|
231
|
+
differs - never negotiates - and additive wire changes are gated by `capabilities` rather than a
|
|
232
|
+
protocol bump. State files carry `v` (`STATE_VERSION`): absent means schema 1 and is read and
|
|
233
|
+
stamped on the next save, older is migrated on read, newer is refused for reading *and* writing.
|
|
234
|
+
|
|
235
|
+
**A device rename strands what it scanned.** Records carry the device that scanned them, and
|
|
236
|
+
`/pending` offers only this device's, so renaming `device.name` while replies are queued leaves
|
|
237
|
+
those records neither offered nor claimable. `/mailbridge status` reports the count under "held for
|
|
238
|
+
another device". They are your own mail and the mailbox still holds them unread, but the UID cursor
|
|
239
|
+
has already passed them, so the recovery is to restore the previous `device.name` (the records are
|
|
240
|
+
delivered again) or to have the human reply again. Rename before a quiet mailbox, not during.
|
|
241
|
+
|
|
242
|
+
**One owner for replacement.** A dead daemon, a protocol mismatch and `/mailbridge restart` all go
|
|
243
|
+
through one `replaceRuntime()`: capture the notifier, rebuild, flush the pending settle. Three
|
|
244
|
+
paths each doing it themselves is how the same dropped notification was fixed three times.
|
|
245
|
+
|
|
246
|
+
**A command that stops the daemon parks the poll first.** The tick runs every ten seconds and a
|
|
247
|
+
restart waits up to five for the port to close, so a tick landing inside that window would hear the
|
|
248
|
+
dying daemon's ping fail, respawn it, and hand the command the "something still answers" evidence
|
|
249
|
+
it is measuring. `restart` therefore parks the loop, *joins* a tick that is already in flight
|
|
250
|
+
(bounded by an internal budget longer than a spawn, so a daemon that tick is starting exists
|
|
251
|
+
first), and re-arms in a `finally`. The pause flag is checked after every await in the tick, because a tick suspended
|
|
252
|
+
before the park cannot be recalled - only made to stop at its next step.
|
|
253
|
+
|
|
254
|
+
**It starts when a session needs it.** `pi install` runs nothing. On `session_start`, if the config
|
|
255
|
+
is usable, the extension probes `/health` and spawns the daemon detached when nothing answers; in-pi
|
|
256
|
+
`/mailbridge setup` does the same after writing the config, while headless `--setup` only writes
|
|
257
|
+
the file. The daemon is not tied to the mode: `on`/`off` gate scanning and delivery, not existence.
|
|
258
|
+
|
|
259
|
+
**It stops when the last session does.** Every session pings `/ping` on its poll tick
|
|
260
|
+
(10s) and sends `/bye` on `session_shutdown`. Once no ping has arrived for
|
|
261
|
+
`daemon.idleTimeoutSecs` (default 120), the daemon exits. This is deliberate:
|
|
262
|
+
|
|
263
|
+
- an updated package cannot leave an orphaned process running old code - the exact trap
|
|
264
|
+
that cost an evening, where a daemon started at 15:09 kept serving after several code
|
|
265
|
+
changes because nothing could replace it. Idle exit alone is not enough while a session is
|
|
266
|
+
open, so `/health` reports the daemon's protocol and a session replaces a daemon on another
|
|
267
|
+
protocol (throttled to once a minute);
|
|
268
|
+
- no session means no consumer: settle notifications are sent *by* a session, and no
|
|
269
|
+
session can claim a pending reply;
|
|
270
|
+
- a daemon with no sessions still fetches and parses every incoming message looking for
|
|
271
|
+
a routing tag. Stopping it stops reading a mailbox nothing will use.
|
|
272
|
+
|
|
273
|
+
Mail that arrives while it is down is not lost: the **catch-up scan** runs on the next
|
|
274
|
+
start, from the stored UID cursor.
|
|
275
|
+
|
|
276
|
+
There is deliberately no switch to keep it resident: with no sessions it scans but cannot deliver,
|
|
277
|
+
so a daemon that outlived pi would read a mailbox for nobody. Idleness is measured from the
|
|
278
|
+
daemon's own start until a session pings, so a hand-started daemon gets one full grace period and
|
|
279
|
+
then exits - it is not exempt.
|
|
280
|
+
|
|
281
|
+
## 7. Failure modes
|
|
282
|
+
|
|
283
|
+
| Failure | Behaviour |
|
|
284
|
+
| --- | --- |
|
|
285
|
+
| IMAP drops | exponential backoff 1 s → 60 s; IDLE renews at 25 min |
|
|
286
|
+
| IDLE `EXISTS` never fires | a catch-up scan runs every `daemon.scanIntervalSecs` (default 60, `0` disables) while mode is on and the watcher is healthy — observed live after a reply sat unscanned |
|
|
287
|
+
| Daemon dies | sessions keep working; extension reconnects; missed mail recovered from `/pending` |
|
|
288
|
+
| Session exits before ack | mail stays unseen, pending durable → redelivered |
|
|
289
|
+
| SMTP fails | decision is *not* reserved as outstanding; error surfaces to the tool |
|
|
290
|
+
| Mailbox full of non-token mail | cursor advances, nothing is claimed, no state grows |
|
|
291
|
+
| Clock skew | expiry is compared against the local clock (`expiresAt <= now`), so a clock more than a TTL **ahead** refuses fresh ids, and a clock **behind** keeps expired ones alive |
|
|
292
|
+
|
|
293
|
+
## 8. Module layout
|
|
294
|
+
|
|
295
|
+
```
|
|
296
|
+
src/
|
|
297
|
+
extension.ts pi entry: tools, commands, polling client, session lifecycle
|
|
298
|
+
core/
|
|
299
|
+
attachments.ts image/markdown attachment rendering
|
|
300
|
+
authz.ts loop/bounce → robot → allowlist → DMARC (off for one mailbox)
|
|
301
|
+
commands.ts /mailbridge parsing + usage text
|
|
302
|
+
config.ts layered config, env overrides, type validation
|
|
303
|
+
config-freshness.ts is the daemon's config older than the file?
|
|
304
|
+
deliveries.ts claim → inject → ack, serialized per process
|
|
305
|
+
discover.ts account autodiscovery
|
|
306
|
+
header-encoding.ts RFC 2047 decode/encode
|
|
307
|
+
html.ts text → HTML with quoted-history folding
|
|
308
|
+
inline-images.ts local images → cid: parts; linked files → attachments (inside the project)
|
|
309
|
+
mailer.ts nodemailer transport (password | XOAUTH2), headers
|
|
310
|
+
mail-command.ts a reply that is a command → the safe subset, else text
|
|
311
|
+
prime.ts "watch from now on" - first run only, guarded by the cursor
|
|
312
|
+
markdown.ts notifications render as markdown
|
|
313
|
+
oauth*.ts device-code + OpenID flows, one-retry token provider
|
|
314
|
+
reply-parse.ts quote stripping, address + AR-header pinning
|
|
315
|
+
settle.ts idle debounce, turn-settled notification
|
|
316
|
+
status.ts scan summary → the lines /mailbridge status prints
|
|
317
|
+
version.ts this build's version (diagnostic; replacement keys on apiVersion)
|
|
318
|
+
sent-index.ts record the Message-ID the provider actually delivered
|
|
319
|
+
setup.ts first-run scaffolding
|
|
320
|
+
smoke.ts end-to-end send-and-read-back check
|
|
321
|
+
token.ts HMAC sign/verify, subjects
|
|
322
|
+
token-source.ts XOAUTH2 command tokens
|
|
323
|
+
daemon/
|
|
324
|
+
api.ts the wire contract: API_VERSION, capabilities, endpoint list
|
|
325
|
+
imapflow.ts imapflow adapter
|
|
326
|
+
main.ts CLI + lifecycle
|
|
327
|
+
process.ts UID-cursor scan → authorize → route → parse → persist
|
|
328
|
+
scan-queue.ts one scan at a time; bursts coalesce into a follow-up
|
|
329
|
+
script.ts which daemon entry to spawn: the build when one exists
|
|
330
|
+
send-budget.ts outbound mail per minute, so a loop cannot spend the quota
|
|
331
|
+
source-proxy.ts the MailSource the daemon really uses: live watcher + UIDVALIDITY
|
|
332
|
+
service.ts the HTTP surface: /health /pending /claim /ack /send /events …
|
|
333
|
+
spawn.ts ensureDaemon, single-instance
|
|
334
|
+
watcher.ts imapflow IDLE, renew, backoff
|
|
335
|
+
store/
|
|
336
|
+
journal.ts acked UIDs — durable replay protection
|
|
337
|
+
mode-store.ts on/off
|
|
338
|
+
oauth-store.ts refresh tokens, 0600
|
|
339
|
+
paths.ts XDG dirs
|
|
340
|
+
pending-store.ts durable pending + claim/ack state machine
|
|
341
|
+
registry.ts one outstanding decision per session
|
|
342
|
+
schema.ts on-disk schema version: refuse newer, migrate older
|
|
343
|
+
routes.ts short routing ids, TTL
|
|
344
|
+
secret.ts shared HMAC/bearer key, atomic 0600
|
|
345
|
+
threads.ts outbound Message-IDs: threading + reply chain
|
|
346
|
+
uid-cursor.ts cursor + UID space; resets when the mailbox is renumbered
|
|
347
|
+
test/ hermetic unit tests; no IMAP, no SMTP, no sockets
|
|
348
|
+
```
|
|
349
|
+
|
|
350
|
+
`/events` (SSE) exists on the daemon for push clients; the extension currently polls
|
|
351
|
+
`/pending` every few seconds instead, because replaying it is idempotent and needs no
|
|
352
|
+
reconnect logic.
|
|
353
|
+
|
|
354
|
+
`MailSource` (in `daemon/process.ts`) is the seam: `watcher.ts` implements it over
|
|
355
|
+
imapflow, tests implement it over an in-memory fixture. That is why the whole
|
|
356
|
+
pipeline is testable without a mailbox.
|
package/docs/MAPPING.md
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
# docs/MAPPING.md — `opencode-afk` → `pi-mailbridge`
|
|
2
|
+
|
|
3
|
+
Module-by-module mapping from the reference implementation
|
|
4
|
+
[`7emotions/afk`](https://github.com/7emotions/afk) (npm `opencode-afk`, MIT,
|
|
5
|
+
v0.1.5, 3,988 lines JS) onto Pi's extension API.
|
|
6
|
+
|
|
7
|
+
Read this alongside `ARCHITECTURE.md`. The goal is to port a proven design,
|
|
8
|
+
not to reinvent it — while deleting the parts that only exist because OpenCode
|
|
9
|
+
is driven out-of-process.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## 0. The one structural difference
|
|
14
|
+
|
|
15
|
+
`opencode-afk` injects replies through the **OpenCode SDK over HTTP**, because
|
|
16
|
+
its daemon lives outside the agent process. Pi extensions run **inside** the
|
|
17
|
+
agent process, so injection is a local function call.
|
|
18
|
+
|
|
19
|
+
That single change deletes three pieces of machinery:
|
|
20
|
+
|
|
21
|
+
| afk machinery | Why it exists | pi-mailbridge |
|
|
22
|
+
| --- | --- | --- |
|
|
23
|
+
| `resolveRootSessionID()` parent-chain walk | opencode subagent sessions have `parentID`; the plugin must find the ROOT session to stamp the token with | Not needed. Pi reports the session id directly via `ctx.sessionManager.getSessionId()`. Revisit only if pi subagents surface as child sessions. |
|
|
24
|
+
| Ownership self-check by `session.directory` | daemon broadcasts to all instances; each must decide if a reply is "for me" | Direct: compare the token's session id to `getSessionId()`. |
|
|
25
|
+
| `POST /claim {sessionID, instanceId}` with HTTP 500-on-gone handling | the SDK call may fail because the session vanished | Not needed for routing. The claim remains, but only to make **ack ordering** crash-safe (see §3). |
|
|
26
|
+
|
|
27
|
+
Everything else ports essentially as-is.
|
|
28
|
+
|
|
29
|
+
---
|
|
30
|
+
|
|
31
|
+
## 1. Module map
|
|
32
|
+
|
|
33
|
+
| `opencode-afk` file | Lines | pi-mailbridge target | Change |
|
|
34
|
+
| --- | --- | --- | --- |
|
|
35
|
+
| `index.js` | 391 | `src/extension.ts` | Rewritten: opencode tool/command registration → `pi.registerTool` / `pi.registerCommand` |
|
|
36
|
+
| `daemon.js` | 473 | `src/daemon/main.ts` + `src/daemon/service.ts` | Ported: HTTP surface + SSE in `service.ts`, CLI/lifecycle and single-instance lock in `main.ts` (port **4177**, not 4100) |
|
|
37
|
+
| `core/watcher.js` | 261 | `src/daemon/watcher.ts` | Ported verbatim — `imapflow` IDLE, renew < 29 min, exponential backoff |
|
|
38
|
+
| `core/process.js` | 314 | `src/daemon/process.ts` | Ported — UID-cursor incremental scan → fetch → parse → persist |
|
|
39
|
+
| `core/reply-parse.js` | 265 | `src/core/reply-parse.ts` | Ported — quote stripping, address/AR-header pinning (token verify lives in `core/token.ts`) |
|
|
40
|
+
| `core/inject.js` | 181 | `src/extension.ts` (in-process) | **Replaced** by `pi.sendUserMessage(body, { deliverAs: "followUp" })` |
|
|
41
|
+
| `core/pause-notify.js` | 143 | `src/extension.ts` → `pi.on("agent_settled")` | Replaced by the pi event (§4) |
|
|
42
|
+
| `core/subscribe.js` | 271 | `src/extension.ts` | Replaced — polls `/pending` and replays it (idempotent); `/events` SSE stays available for push clients |
|
|
43
|
+
| `core/new-session.js` | 103 | **out of scope** | Deferred. Pi can do it (`ctx.newSession`) but it replaces the active session; see ARCHITECTURE §6 |
|
|
44
|
+
| `request-decision.js` | 187 | `src/extension.ts` | Ported as a `pi.registerTool` |
|
|
45
|
+
| `notify.js` | 125 | `src/extension.ts` | Ported as a `pi.registerTool` |
|
|
46
|
+
| `mailer.js` | 86 | `src/core/mailer.ts` | Ported — `[omo:…]` → `[pi:…]`, nodemailer transport |
|
|
47
|
+
| `config.js` | 184 | `src/core/config.ts` | Ported — layered config + env overrides |
|
|
48
|
+
| `messages.js` | 102 | *(inlined)* | Folded into the modules that emit the text (`commands.ts`, `settle.ts`, `smoke.ts`) — no separate table |
|
|
49
|
+
| `install.js` | 168 | **dropped** | opencode-specific installer; pi installs via `pi install npm:@dumbmice/pi-mailbridge` |
|
|
50
|
+
| `store/paths.js` | 88 | `src/store/paths.ts` | Ported — XDG state dir |
|
|
51
|
+
| `store/secret.js` | 116 | `src/store/secret.ts` | Ported — HMAC key file, first-writer-wins |
|
|
52
|
+
| `store/pending-store.js` | 215 | `src/store/pending-store.ts` | Ported — durable pending replies |
|
|
53
|
+
| `store/uid-cursor.js` | 123 | `src/store/uid-cursor.ts` | Ported, then extended — monotonic within a UID space; resets when the mailbox renumbers |
|
|
54
|
+
| `store/mode-store.js` | 86 | `src/store/mode-store.ts` | Ported — global on/off via `/mailbridge on` · `off` |
|
|
55
|
+
| `store/registry.js` | 106 | `src/store/registry.ts` | Ported — one-outstanding-decision guard |
|
|
56
|
+
| `command/afk.md`, `command/back.md` | short | `/mailbridge on` · `off` subcommands | Renamed and namespaced: `back` is too generic a top-level name |
|
|
57
|
+
| `AGENTS.md` | — | `AGENTS.md` | Rewritten for contributors + agents |
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## 2. API substitutions
|
|
62
|
+
|
|
63
|
+
### Registering a tool
|
|
64
|
+
|
|
65
|
+
```js
|
|
66
|
+
// opencode-afk
|
|
67
|
+
export const request_decision = tool({
|
|
68
|
+
description: "…",
|
|
69
|
+
args: { subject: z.string(), question: z.string() },
|
|
70
|
+
async execute(args, ctx) { /* … */ },
|
|
71
|
+
})
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
```ts
|
|
75
|
+
// pi-mailbridge
|
|
76
|
+
pi.registerTool({
|
|
77
|
+
name: "request_decision",
|
|
78
|
+
label: "Request decision",
|
|
79
|
+
description: "Email the user a decision request and end the turn.",
|
|
80
|
+
parameters: Type.Object({
|
|
81
|
+
subject: Type.String(),
|
|
82
|
+
question: Type.String(),
|
|
83
|
+
options: Type.Optional(Type.Array(Type.String())),
|
|
84
|
+
recommendation: Type.Optional(Type.String()),
|
|
85
|
+
}),
|
|
86
|
+
async execute(toolCallId, params, signal, onUpdate, ctx) {
|
|
87
|
+
/* … */
|
|
88
|
+
return { content: [{ type: "text", text: "Email sent." }], details: {} };
|
|
89
|
+
},
|
|
90
|
+
})
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
### Inbound injection
|
|
94
|
+
|
|
95
|
+
```js
|
|
96
|
+
// opencode-afk (core/inject.js)
|
|
97
|
+
await client.session.prompt({
|
|
98
|
+
path: { id: sessionID }, query: { directory },
|
|
99
|
+
body: { parts: [{ type: "text", text: body }] },
|
|
100
|
+
})
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
```ts
|
|
104
|
+
// pi-mailbridge — in-process, cannot "not find" the session
|
|
105
|
+
pi.sendUserMessage(body, { deliverAs: "followUp", expandPromptTemplates: false })
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
`deliverAs: "followUp"` queues the reply if a run is in flight — matching afk's
|
|
109
|
+
behaviour when a decision is answered late.
|
|
110
|
+
|
|
111
|
+
`expandPromptTemplates: false` is a correctness requirement, not an optimisation. With expansion
|
|
112
|
+
on, pi dispatches any message starting with `/` to a registered command and returns without a
|
|
113
|
+
turn, and it returns `void` — the caller cannot tell a command that ran from a prompt pi rejected
|
|
114
|
+
(no model, expired credentials, compaction in progress), while this package acks and marks
|
|
115
|
+
`\Seen` after that call returns. Commands by mail are therefore dispatched by this package
|
|
116
|
+
(`core/mail-command.ts`), which limits them to the safe subset; everything else stays text.
|
|
117
|
+
|
|
118
|
+
### Lifecycle signal
|
|
119
|
+
|
|
120
|
+
```js
|
|
121
|
+
// opencode-afk: polling/idle heuristics in core/pause-notify.js
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
```ts
|
|
125
|
+
// pi-mailbridge: a real event, and the correct one
|
|
126
|
+
pi.on("agent_settled", (event) => {
|
|
127
|
+
if (event.aborted && !config.notify.notifyOnAborted) return // Escape ≠ finished
|
|
128
|
+
scheduleSettledMail() // debounced
|
|
129
|
+
})
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
`agent_end` is **wrong** here: pi may still retry, compact, or run queued
|
|
133
|
+
follow-ups after it.
|
|
134
|
+
|
|
135
|
+
### Commands
|
|
136
|
+
|
|
137
|
+
```js
|
|
138
|
+
// opencode-afk: command/afk.md + command/back.md
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
```ts
|
|
142
|
+
pi.registerCommand("mailbridge", {
|
|
143
|
+
description: "Status, on/off, setup, check, auth, smoke, restart",
|
|
144
|
+
handler: (args, ctx) => …, // one command; away/here became on/off, and no top-level commands
|
|
145
|
+
})
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
---
|
|
149
|
+
|
|
150
|
+
## 3. What we keep exactly (these are the hard-won parts)
|
|
151
|
+
|
|
152
|
+
1. **UID-cursor polling, not `\Seen`.** afk searches `UID > cursor`, never
|
|
153
|
+
`UNSEEN` and never by SUBJECT. `\Seen` is shared mutable state — any other
|
|
154
|
+
mail client can set it and silently eat a reply.
|
|
155
|
+
2. **Ack ordering.** Parse → persist to durable pending → notify → inject →
|
|
156
|
+
**then** `\Seen` + journal append. A crash anywhere before the ack leaves
|
|
157
|
+
the mail unseen and the pending durable, so nothing is lost. Never mark
|
|
158
|
+
seen at parse time.
|
|
159
|
+
3. **Single watcher per machine.** One daemon, one IMAP IDLE connection, fixed
|
|
160
|
+
port, `EADDRINUSE` = "already running, exit quietly". This is what makes
|
|
161
|
+
multi-session polling sane: N pi sessions, 1 mailbox connection.
|
|
162
|
+
4. **A routing id in the subject — but no longer a signature.** Shipped as a 6-hex
|
|
163
|
+
local route id (`[pi:f913c9]`, six hex, eight on the rare collision), looked up in
|
|
164
|
+
`store/routes.ts`. This is the one
|
|
165
|
+
deliberate reversal from the baseline: a signed token made every subject preview
|
|
166
|
+
unreadable (a 39-character token, in a subject over 80 characters). HMAC tokens still
|
|
167
|
+
verify for mail already in flight, and
|
|
168
|
+
authorization never depended on the token either way.
|
|
169
|
+
5. **Global mode store.** `/mailbridge on` and `/mailbridge off` are explicit, persisted, and
|
|
170
|
+
independent of focus heuristics.
|
|
171
|
+
6. **Backoff + IDLE renew.** Renew well under the ~29-minute server timeout;
|
|
172
|
+
exponential reconnect 1 s → 60 s.
|
|
173
|
+
7. **Bounded resources.** Registry cap, SSE client cap, request body cap,
|
|
174
|
+
constant-time token comparison.
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## 4. Deliberate differences
|
|
179
|
+
|
|
180
|
+
| Area | afk | pi-mailbridge | Why |
|
|
181
|
+
| --- | --- | --- | --- |
|
|
182
|
+
| New session from mail (`/new`) | supported | deferred | `ctx.newSession()` *replaces* the active session; needs its own design pass |
|
|
183
|
+
| Service | opencode/qq mail tuned | provider-neutral | `config.mail.*`; no hard-coded host |
|
|
184
|
+
| Transport | SMTP password | SMTP password **or** OAuth2/XOAUTH2 | Outlook requires OAuth2 since 2024-09-16 |
|
|
185
|
+
| Reply parsing | hand-rolled | hand-rolled, quote-stripping covered by tests | no `inreply` dependency: the stripping rules are ours and tested |
|
|
186
|
+
| Auth check | allowlist | allowlist + DMARC header (pinned when `trustedAuthservId` is set; **off for a single mailbox** — SECURITY.md §5) | see SECURITY.md §2 |
|
|
187
|
+
| Port | 4100 | 4177 | avoid colliding with a running afk |
|
|
188
|
+
|
|
189
|
+
---
|
|
190
|
+
|
|
191
|
+
## 5. Publish checklist
|
|
192
|
+
|
|
193
|
+
- [x] `package.json` `files` limited to `dist`, `src`, `bin`, `docs`, `config.example.json`,
|
|
194
|
+
`CHANGELOG.md`, `scripts/render-teaser.ts`. `dist` is load-bearing: the installed extension
|
|
195
|
+
spawns `dist/daemon/main.js` (Node cannot strip types inside `node_modules`)
|
|
196
|
+
- [x] no secrets in the package; `config.json` is gitignored and never shipped
|
|
197
|
+
- [x] `pi` manifest points at `./src/extension.ts`
|
|
198
|
+
- [x] peer deps on `@earendil-works/pi-coding-agent`, `@earendil-works/pi-ai` and `@sinclair/typebox`, not hard deps
|
|
199
|
+
- [x] tests run with `node --test` and never touch real IMAP/SMTP
|
|
200
|
+
- [x] docs state the daemon is a **local, authenticated** service that can inject prompts
|
|
201
|
+
- [x] README states the threat model before the install steps ("Security first", and the install section points at SECURITY.md)
|