@bom0792/herdgate 0.9.28 → 0.9.31

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/DEPLOY.md ADDED
@@ -0,0 +1,341 @@
1
+ # Deploying herdgate publicly (cloudflared, this machine)
2
+
3
+ The cloud binds `127.0.0.1` and is published through a Cloudflare tunnel. Two shapes:
4
+
5
+ ## A. Quick tunnel (no DNS setup, throwaway URL)
6
+
7
+ ```sh
8
+ # terminal 1 — the cloud
9
+ cd /home/bom/source/herdgate
10
+ export HERDGATE_ACCOUNT_TOKEN=$(openssl rand -base64 32)
11
+ export HERDGATE_STATE=$HOME/.herdgate/state.json
12
+ npm start
13
+
14
+ # terminal 2 — a trycloudflare URL (random, changes every run)
15
+ cloudflared tunnel --url http://127.0.0.1:8787
16
+ ```
17
+
18
+ Use the printed `https://<random>.trycloudflare.com` as the public URL. Nothing to configure; fine for testing.
19
+
20
+ ## B. Named tunnel + your domain (stable hostname)
21
+
22
+ ```sh
23
+ cloudflared tunnel login # once, picks a zone
24
+ cloudflared tunnel create herdgate
25
+ cloudflared tunnel route dns herdgate herd.example.com
26
+
27
+ cat > ~/.cloudflared/herdgate.yml <<'YML'
28
+ tunnel: herdgate
29
+ credentials-file: /home/bom/.cloudflared/<TUNNEL-ID>.json
30
+ ingress:
31
+ - hostname: herd.example.com
32
+ service: http://127.0.0.1:8787
33
+ - service: http_status:404
34
+ YML
35
+
36
+ cloudflared tunnel run --config ~/.cloudflared/herdgate.yml
37
+ ```
38
+
39
+ Then start the cloud with the hostname pinned:
40
+
41
+ ```sh
42
+ export HERDGATE_ACCOUNT_TOKEN=$(openssl rand -base64 32)
43
+ export HERDGATE_STATE=$HOME/.herdgate/state.json
44
+ export HERDGATE_PUBLIC_HOST=herd.example.com
45
+ export HERDGATE_PUBLIC_URL=https://herd.example.com
46
+ npm start
47
+ ```
48
+
49
+ `HERDGATE_PUBLIC_HOST` makes the gateway answer **only** that Host (421 otherwise); `HERDGATE_PUBLIC_URL` builds
50
+ the pairing links shown to hosts.
51
+
52
+ ### Funnel vs Tunnel
53
+
54
+ - **Tunnel** (above): the hostname resolves for everyone, but traffic only flows for this tunnel — which is
55
+ what we want, because herdgate authenticates every request and socket itself (hello-first tokens). Use this.
56
+ - **Funnel** (`cloudflared tunnel --url … --hello-world` variants / funnel ingress) would additionally let
57
+ arbitrary internet clients terminate at your origin without a CF account — herdgate's gate is the only
58
+ protection then. It works with the same code (the gate never trusts CF headers), but adds no value and more
59
+ exposure. Not recommended.
60
+
61
+ ## Rehearse before going live (edge-sim)
62
+
63
+ `npm run edge` runs a local Cloudflare-edge simulator: TLS termination + `Host` rewrite +
64
+ WebSocket Upgrade splice — byte-for-byte the origin-side job cloudflared does. It lets you
65
+ exercise the public contract (`expectedHost` pinning → 421 on wrong host, the PWA over
66
+ TLS, both wss legs) on this machine, with a throwaway dev cert:
67
+
68
+ ```sh
69
+ openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:P-256 \
70
+ -keyout /tmp/edge-key.pem -out /tmp/edge-cert.pem -days 2 -nodes -subj "/CN=127.0.0.1"
71
+
72
+ HERDGATE_PUBLIC_HOST=gate.local HERDGATE_SSH_PORT=0 npm start # terminal 1
73
+ HERDGATE_TARGET_PORT=8787 HERDGATE_PINNED_HOST=gate.local \
74
+ HERDGATE_EDGE_CERT=/tmp/edge-cert.pem HERDGATE_EDGE_KEY=/tmp/edge-key.pem \
75
+ npm run edge # terminal 2
76
+ # → https://127.0.0.1:<port> (trust the cert or pass --ignore-certificate-errors)
77
+ ```
78
+
79
+ Verified by `test/edge.test.ts` and a full headless-Chrome run: sign in → `online` →
80
+ machine online → pair read → live Herdr view + SSH hint, all over TLS with both wss legs
81
+ through the splice.
82
+
83
+ ### A note on `trycloudflare.com` quick tunnels
84
+
85
+ On this machine, quick tunnels **register** but the edge answers **404** for the
86
+ trycloudflare hostname from both IPv4 and IPv6 — the free-for-all hostname space is
87
+ blocked somewhere between this machine and Cloudflare. **Named tunnels on your own zone
88
+ do not have this problem**: `gate.phuongnb.io.vn` (tunnel `herdgate`, id `dfc578ef`) is
89
+ live — PWA + CSP through the edge, owner + host wss legs through the edge, pairing and
90
+ the live Herdr view all verified in headless Chrome with the real Cloudflare cert (no
91
+ `--ignore-certificate-errors`). The live recipe on this machine:
92
+
93
+ ```sh
94
+ cloudflared tunnel create herdgate
95
+ cloudflared tunnel route dns -f <tunnel-uuid> gate.phuongnb.io.vn # -f overwrites; the
96
+ # bare form once logged the WRONG tunnel id (fell back to config.yml) — always pass
97
+ # the UUID and check the log line says your tunnel id
98
+ # ~/.cloudflared/herdgate.yml → ingress gate.phuongnb.io.vn → http://127.0.0.1:8833
99
+ HERDGATE_PUBLIC_HOST=gate.phuongnb.io.vn HERDGATE_PORT=8833 HERDGATE_SSH_PORT=2299 npm start
100
+ cloudflared --config ~/.cloudflared/herdgate.yml tunnel run herdgate
101
+ ```
102
+
103
+ Gotchas hit along the way: port 8787 was already taken by the hermes webui (the old
104
+ `config.yml` routes `hermes.phuongnb.io.vn` there — an ingress clash would silently serve
105
+ the wrong app through our hostname), and running the gateway script without `&` in a
106
+ shell helper blocks forever. Local `curl http://127.0.0.1:8833/` returns **421** by
107
+ design — the gateway pins `Host: gate.phuongnb.io.vn`; that is the pinning contract
108
+ working, not a bug.
109
+
110
+ ### The SSH leg — passwordless logins (publickey)
111
+
112
+ `hc machine add` shells out to the system `ssh`, which prefers keys and the agent and
113
+ refuses to answer prompts when it has no terminal. A device whose machine should add
114
+ it non-interactively registers that machine's public key first:
115
+
116
+ 1. In the PWA, open the host's room → **SSH for the Herdr app**, select the exact device ID,
117
+ and add the controlling machine's public key. For Herdr Mobile, add the phone's public key to
118
+ its device ID instead. Do not edit the publisher's `authorized_keys` or copy a private key.
119
+ 2. Confirm the fingerprint presented by the client matches the Gate's known SSH host fingerprint.
120
+ Pin that key; the TCP edge does not present the VPS admin SSH key.
121
+ 3. Use the room's device ID as the SSH username. For example, a Core client can run
122
+ `hc machine add d_device@ssh.example.com:2299`. On Mobile, use the reachable TCP edge
123
+ hostname or IPv4 address and port in the machine settings. Password auth with the paired
124
+ device secret also works.
125
+
126
+ The REST surface (same two identities as `/v1/device/push`: the device's own secret,
127
+ or the account token): `GET/POST/DELETE /v1/device/sshkey`. Ed25519, RSA and ECDSA
128
+ (nistp256/384/521) keys are accepted; signatures are verified per RFC 4252 §7.
129
+
130
+ ## The SSH leg is not a Cloudflare-TCP leg — three ways in
131
+
132
+ Phones speak plain SSH to `host:port`; Cloudflare Tunnel only carries TCP for clients
133
+ running `cloudflared access`, which the phone does not. Reachability options, in order
134
+ of setup cost:
135
+
136
+ 1. **LAN / same network** — `HERDGATE_SSH_BIND=0.0.0.0`, phones dial the machine.
137
+ 2. **IPv6 AAAA** (this machine's setup) — a DNS-only `AAAA` record to the gate
138
+ machine's global IPv6; `~/.local/bin/herdgate-ssh-ddns` + the `herdgate-ddns`
139
+ timer keep it current (needs a Cloudflare API token + `ufw allow`).
140
+ 3. **Tokenless TCP edge (recommended for a VPS)** — one TCP connection from Herdr
141
+ Mobile maps to one WSS connection to the public gate. The edge never terminates
142
+ SSH. Use the existing Herdgate SSH host key on the phone, not the VPS admin key.
143
+
144
+ ### Deploy the tokenless TCP edge on a VPS
145
+
146
+ Use a public IPv4 VPS with Node 22+ installed system-wide. Build the standalone
147
+ JavaScript on a machine with TypeScript-enabled Node 22.18+ before copying it to
148
+ the VPS (Ubuntu's packaged Node may not support running `.ts` directly):
149
+
150
+ ```sh
151
+ node --input-type=module -e 'import { readFileSync } from "node:fs"; import { stripTypeScriptTypes } from "node:module"; process.stdout.write(stripTypeScriptTypes(readFileSync("src/cloud/ssh-tcp-edge.ts", "utf8")))' > /tmp/ssh-tcp-edge.mjs
152
+ ```
153
+
154
+ Copy the output to `/opt/herdgate-tcp-edge/ssh-tcp-edge.mjs` and install only the
155
+ locked `ws` version there (`npm install --save-exact --omit=dev --ignore-scripts --prefix /opt/herdgate-tcp-edge ws@8.23.0`).
156
+ Install `templates/herdgate-tcp-edge.service` into `/etc/systemd/system/` (adjust
157
+ `ExecStart` if Node is not at `/usr/bin/node`), and put this in
158
+ `/etc/herdgate-tcp-edge.env`:
159
+
160
+ ```dotenv
161
+ HERDGATE_URL=https://gate.phuongnb.io.vn
162
+ HERDGATE_TCP_EDGE_BIND=0.0.0.0
163
+ HERDGATE_TCP_EDGE_PORT=2299
164
+ ```
165
+
166
+ Do **not** copy any Herdgate credential or SSH host key to the VPS. Enable the unit
167
+ with `sudo systemctl enable --now herdgate-tcp-edge` and allow TCP 2299 in the VPS
168
+ provider and OS firewalls. Restrict the VPS's separate admin SSH access. The edge
169
+ caps concurrent sessions at 3 per IP and 6 total by default; the gate's own tunnel
170
+ caps still apply, and the gate may see several clients as one source behind a proxy.
171
+
172
+ Test `VPS_IP:2299` with the existing mobile username and key over cellular before
173
+ changing DNS. Confirm the host key matches the gate's pinned fingerprint. To switch
174
+ `ssh.phuongnb.io.vn`, add a DNS-only `A` record for the VPS and remove the old `AAAA`
175
+ unless it also points to a verified edge. Disable the old home AAAA DDNS timer if
176
+ using that hostname for the VPS. Keep the HTTPS hostname unchanged.
177
+
178
+ ### Legacy authenticated SSH bridge
179
+
180
+ The older SSH bridge is also available, but requires an account owner token on
181
+ its VPS and shares one upstream connection:
182
+
183
+ ```sh
184
+ # on the VPS (Node ≥ 22, one file, no dependencies):
185
+ node ssh-bridge.ts # env: HERDGATE_URL, HERDGATE_BRIDGE_TOKEN (the account token),
186
+ # HERDGATE_BRIDGE_BIND/PORT (default 0.0.0.0:2299),
187
+ # HERDGATE_BRIDGE_PUBLIC_HOST/PORT — advertise what phones dial
188
+ ```
189
+
190
+ The bridge dials the cloud over an OUTBOUND WebSocket and pipes raw SSH bytes both
191
+ ways; the cloud feeds them into its sshd through ssh2's `injectSocket`. Device-key
192
+ auth, act-scope enforcement, the read-only allowlist and brute-force strikes all stay
193
+ in the cloud. The legacy bridge credential is the account (owner) token. While a bridge with an advertised address is connected, the
194
+ device hello's `ssh {host, port}` points phones at it; disconnect it and the hello
195
+ falls back to the gate's own address. Verified end to end through the real Cloudflare
196
+ tunnel: pair over wss → SSH the bridge → `hello` (32 methods), `server.status`, and
197
+ `terminal.send` refused with the act-scope error.
198
+
199
+ The public HTTP/WSS surface continues to go through the tunnel.
200
+
201
+ ### When the publisher and gateway run on the same machine
202
+
203
+ Set `HERDGATE_AGENT_LOOPBACK_PORT=8833` in the publisher's `herdgate.env`
204
+ (use the gateway's actual port). Keep `HERDGATE_URL` set to the canonical public
205
+ URL. The publisher dials `127.0.0.1` with the public `Host` header, so gateway host
206
+ validation and room-token authentication still apply. This opt-in mode uses the
207
+ existing `ws` dependency from the npm installation; the standalone plugin uses
208
+ Node's native WebSocket by default. Restart only the publisher. Remove the line
209
+ to restore the public transport. No client SSH configuration changes are needed.
210
+
211
+ For the standalone Linux Mobile TUI publisher (`src/host/mobile-tui.ts`),
212
+ set `HERDGATE_AGENT_LOOPBACK_PORT` in its **process environment** (for systemd,
213
+ a drop-in with `[Service]` and `Environment=HERDGATE_AGENT_LOOPBACK_PORT=8833`,
214
+ using the actual local port). Keep the public `HERDGATE_URL`, the existing room
215
+ and the device scope. Only enable this when the Gate is on the same machine.
216
+ Restart that Mobile publisher alone in a maintenance window and verify its
217
+ journal reports `cloud hello: … via loopback 127.0.0.1:…`. Reconnect Mobile
218
+ and test the existing entry. Remove the drop-in setting and restart the same
219
+ publisher to roll back. This does not change the public WSS leg on the VPS,
220
+ the SSH host key, or the Herdr server.
221
+
222
+ ## Measure Mobile latency
223
+
224
+ For a local baseline with an isolated Herdr 0.9.5 server, pane, Gate and TCP
225
+ edge (no live rooms or keys), run:
226
+
227
+ ```sh
228
+ TEST_HERDR_BIN="$(command -v herdr)" node scripts/bench-mobile-tui.mjs 25
229
+ ```
230
+
231
+ Compare native Herdr, the Core adapter over local pipes, and the full Mobile
232
+ transport over loopback. The script reports key-to-ANSI, Send-to-ACK and
233
+ Send-to-ANSI separately; attach is measured separately from steady-state
234
+ input. This does not include cellular RTT or the official app's rendering.
235
+ An observe/read-only TUI attachment instead polls every 1.25 seconds; its
236
+ refresh interval is not comparable to the native interactive stream.
237
+
238
+ ## Keep it running
239
+
240
+ On this machine (Linux + systemd) a user unit keeps the gate up; the tunnel is a second unit:
241
+
242
+ ```sh
243
+ mkdir -p ~/.config/systemd/user
244
+ cat > ~/.config/systemd/user/herdgate.service <<'UNIT'
245
+ [Unit]
246
+ Description=herdgate cloud relay hub
247
+ After=network-online.target
248
+
249
+ [Service]
250
+ ExecStart=/home/bom/.nvm/versions/node/v22.23.1/bin/node /home/bom/source/herdgate/src/cloud/main.ts
251
+ Environment=HERDGATE_ACCOUNT_TOKEN=change-me
252
+ Environment=HERDGATE_PUBLIC_HOST=gate.phuongnb.io.vn
253
+ Environment=HERDGATE_PORT=8833
254
+ Environment=HERDGATE_STATE=/home/bom/.local/state/herdgate/state.json
255
+ WorkingDirectory=/home/bom/source/herdgate
256
+ Restart=always
257
+ RestartSec=3
258
+
259
+ [Install]
260
+ WantedBy=default.target
261
+ UNIT
262
+ systemctl --user daemon-reload && systemctl --user enable --now herdgate
263
+ # tunnel: cloudflared --config ~/.cloudflared/herdgate.yml tunnel run herdgate
264
+ # (a second unit wrapping that line, same shape)
265
+ ```
266
+
267
+ Other platforms:
268
+
269
+ ```sh
270
+ # launchd agent sketch (adjust paths); or use tmux/systemd-in-wsl as you prefer
271
+ cat > ~/Library/LaunchAgents/dev.herdgate.cloud.plist <<'PLIST'
272
+ <?xml version="1.0" encoding="UTF-8"?>
273
+ <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
274
+ <plist version="1.0"><dict>
275
+ <key>Label</key><string>dev.herdgate.cloud</string>
276
+ <key>ProgramArguments</key><array>
277
+ <string>/Users/bom/.nvm/versions/node/v22.23.1/bin/node</string>
278
+ <string>/home/bom/source/herdgate/src/cloud/main.ts</string>
279
+ </array>
280
+ <key>RunAtLoad</key><true/>
281
+ <key>KeepAlive</key><true/>
282
+ <key>EnvironmentVariables</key><dict>
283
+ <key>HERDGATE_ACCOUNT_TOKEN</key><string>…</string>
284
+ <key>HERDGATE_STATE</key><string>/Users/bom/.herdgate/state.json</string>
285
+ <key>HERDGATE_PUBLIC_HOST</key><string>herd.example.com</string>
286
+ <key>HERDGATE_PUBLIC_URL</key><string>https://herd.example.com</string>
287
+ </dict>
288
+ </dict></plist>
289
+ PLIST
290
+ launchctl load ~/Library/LaunchAgents/dev.herdgate.cloud.plist
291
+ ```
292
+
293
+ (The plist above is a sketch — fill the token, keep `node` on your PATH or use the absolute binary.)
294
+
295
+ ## Migrate the Linux TUI publisher
296
+
297
+ On the host, use Node 22+ and run as the user who owns the running Herdr TUI
298
+ and the systemd user unit. Verify publisher/remote access before starting and
299
+ pick a window when a short remote disconnection is acceptable:
300
+
301
+ ```sh
302
+ systemctl --user is-active herdgate-home-tui.service
303
+ herdr status server --json
304
+ npx --yes @bom0792/herdgate@latest migrate
305
+ npx --yes @bom0792/herdgate@latest status
306
+ ```
307
+
308
+ For a new host with no previous publisher unit, use `herdgate setup` after
309
+ pairing instead. Migration only adopts the known legacy Herdgate home unit.
310
+ It stages the publisher under `${XDG_DATA_HOME:-$HOME/.local/share}/herdgate/home-tui/releases`,
311
+ reuses the existing room config, and checks for a Gate hello from the new
312
+ systemd invocation. If the new publisher cannot connect, it attempts to
313
+ restore the previous service. It does not stop or update the Herdr server.
314
+ If Herdr's executable or socket is not at the default path, pass
315
+ `HERDR_BIN_PATH` and/or `HERDR_SOCKET_PATH` when running setup or update.
316
+
317
+ After success, reconnect from the remote Herdr TUI and confirm the existing
318
+ session is reachable. If migration fails, check
319
+ `journalctl --user -u herdgate-home-tui.service` locally; do not re-pair or
320
+ share tokens or keys. Use `herdgate update` for later main-publisher updates.
321
+
322
+ ### Upgrade boundaries
323
+
324
+ - The publisher CLI updates the main TUI service only. It does not deploy
325
+ the Gate, web/PWA, Cloudflare tunnel, VPS TCP edge, or independent Mobile
326
+ read/control services. Maintain and verify those separately.
327
+ - Herdr Mobile Core, Mobile TUI read and Mobile TUI control are different
328
+ targets. Preserve the room, SSH key and scope of each entry. Do not turn a
329
+ read device into an interactive device just to resolve a connection issue.
330
+ - If several phone entries use the same public IP and a new one cannot
331
+ connect, close unused entries and retry. Check the TCP edge logs before
332
+ attributing the failure to the connection limits.
333
+
334
+ ## Operational notes
335
+
336
+ - Preserve the Gate state and SSH host key in protected backups. Deleting
337
+ state loses room/device registrations; replacing the host key invalidates
338
+ clients' pins.
339
+ - **Rotate the account token** by restarting with a new `HERDGATE_ACCOUNT_TOKEN` (devices must update it).
340
+ - **Logs** go to stderr (agent connect/pair events). Payloads are never logged — keep it that way.
341
+ - **Watch the exposure**: `curl -s https://host/healthz` public; everything else needs tokens.
package/DESIGN.md ADDED
@@ -0,0 +1,216 @@
1
+ # herdgate — design
2
+
3
+ Architecture and trust boundaries. For installation and upgrade commands, see [README.md](README.md) and
4
+ [DEPLOY.md](DEPLOY.md).
5
+
6
+ ## 1. What herdgate is
7
+
8
+ A relay hub between **host apps** and **devices** (browsers, PWAs). The first host target is official
9
+ [Herdr](https://herdr.dev). A herdgate-owned local adapter will use a verified upstream interface; the adapter
10
+ runs on the host and the agent connects outbound to the gateway. No fork, third-party plugin, or relay project
11
+ is required. The hub handles identity, routing, and presence; content and policy stay in the host adapter. Payloads crossing the cloud are **opaque by rule**: the cloud never parses, logs or
12
+ persists them.
13
+
14
+ Three deployment properties:
15
+
16
+ 1. Hosts and devices both connect **outbound**; only the cloud is publicly reachable (via cloudflared on the
17
+ machine that runs it).
18
+ 2. The cloud is the **identity provider**. No Cloudflare Access in front, no trusted proxy headers. If
19
+ `HERDGATE_PUBLIC_HOST` is set, the Host header is enforced (anti host-confusion) but nothing else about the
20
+ edge is trusted.
21
+ 3. Single-owner v0: accounts come from config (env tokens). Multi-account signup is roadmap, not code.
22
+
23
+ ## 2. Identity: accounts, rooms, devices
24
+
25
+ | Layer | Who | Credential | Minted by |
26
+ |---|---|---|---|
27
+ | account | a person | config token (v0) | operator (env) |
28
+ | room | one host app | room token (random 32B, stored **hashed**) | device-flow approval |
29
+ | device | one room-scoped browser or SSH client | device ID + device secret, or registered SSH public key for SSH auth | `pair.begin` and key registration |
30
+
31
+ - **Host provisioning (device flow).** The host mints a code (`POST /v1/agent/code`), a human approves it from a
32
+ logged-in device (`POST /v1/agent/approve`), the host polls `GET /v1/agent/code/:code/status` and receives the
33
+ room token exactly once. The host stores it; re-provisioning rotates it.
34
+ - **Device pairing.** A device hellos with its account token, then `pair.begin {roomId, name, scope}`.
35
+ `read` pairs immediately. `act` triggers `pair.request` **down to the host** — the human approves on the host
36
+ (approve-on-host), the cloud grants on `pair.response`. Timeout 60s → denied.
37
+ - **Scopes.** `read` = view; `act` = may cause effects. The cloud knows scopes and reports them
38
+ (`room.update`, `hello.devices`), but **enforcement of what an `act` payload may do is host-side** — the cloud
39
+ cannot see payloads. This split is deliberate (see §5).
40
+
41
+ ## 3. Wire protocol (v1)
42
+
43
+ One JSON object per WS text frame: `{v:1, t:"…", …}`. Unknown `v` → close. Legs:
44
+
45
+ - `GET /v1/agent/ws` — host leg. `hello {roomToken, info}` first (10s deadline, else close 4001). Then
46
+ `frame {device, payload}` ⇄, `presence`, `pair.request`/`pair.response`, `ping`/`pong`, `room.update`.
47
+ One live agent per room: a second connection supersedes the first (4000).
48
+ - `GET /v1/device/ws` — device leg. `hello {accountToken | device+deviceSecret, resume?}` first. Then `frame {payload}`
49
+ (routed to the paired room's host), `pair.begin` → `pair.granted`/`pair.denied`, `presence`, `ping`/`pong`.
50
+
51
+ ### Room streams and resume (v0.6.0)
52
+
53
+ Every host→device message that matters while a phone is away (frames, presence) carries a
54
+ room-scoped `seq` and is buffered briefly (last 256 KiB / 120 s per room). Device hello
55
+ responses carry a `stream {gen, seq}` header; a reconnecting device adds
56
+ `resume {room, gen, seq}` to its hello and receives `resumed:true` plus the buffered tail
57
+ in order. `gen` changes on every gateway process start, so stale bookkeeping from a
58
+ previous life invalidates in one comparison — the device gets a fresh hello (`resumed:false`)
59
+ and re-requests state. A resume from before what was kept answers `stream.gap:true`
60
+ (replay what survived; the device re-requests its view). Frames with an `ssh.*` op are
61
+ **ephemeral**: sent raw, unstamped, never buffered — they belong to an SSH channel that
62
+ died with the drop, and replaying them into a new SSH session would corrupt its wire
63
+ protocol. `pair.granted` (carries the one-time device secret) is likewise never replayed.
64
+
65
+ HTTP: `GET /healthz`, `POST /v1/agent/code`, `POST /v1/agent/approve` (bearer account token),
66
+ `GET /v1/agent/code/:code/status`.
67
+
68
+ Limits: frames 256 KiB (`maxPayload`), 60 msg/s per socket (token bucket), hello within 10s, failed-hello
69
+ strikes per IP (5/min → 60s block), and WebSocket heartbeat pings (8s by default) with termination on silence.
70
+
71
+ ## 4. State
72
+
73
+ `state.json` (atomic tmp+rename, 0600): rooms (token hashed) + devices. In-memory pending pairing codes never
74
+ touch disk. v0 loses open sockets on restart; the room stream resumes devices after a drop (§3), and the `gen`
75
+ header makes a restart a clean re-sync rather than a wrong replay.
76
+
77
+ ## 5. Threat model — what the cloud promises
78
+
79
+ | Threat | Mitigation (v0) |
80
+ |---|---|
81
+ | Guessing a room token / account token | 256-bit random, hashed at rest, constant-time compare, failed-hello strikes |
82
+ | Brute-forcing pairing codes | 8-char no-lookalike alphabet, 10-min TTL, single retrieval, rate-limited mint per IP |
83
+ | A stolen `act` device | approve-on-host at pair time; host re-confirmation of dangerous ops is host policy |
84
+ | Hostile room claiming | codes bound to the approving account; rooms never cross accounts |
85
+ | DoS | per-IP blocks, per-socket rate, frame caps, supersede rule, unbounded fan-out is a bug |
86
+ | Cloud reading content | Gate routes application frames without interpreting them; only a negotiated Noise session protects that payload from the Gate. SSH terminates at the Gate. |
87
+ | Host confusion via cloudflared | optional `expectedHost` pin; nothing else about the edge is trusted |
88
+
89
+ Per-device secrets and Noise_XX payload sessions now exist. Noise applies to its own application payload
90
+ channel, not to SSH, which terminates at the Gate. Do not infer that all payloads use Noise or that the
91
+ single-owner deployment model provides multi-tenant isolation.
92
+
93
+ ## 6. Roadmap
94
+
95
+ 1. ~~**Per-device secrets**~~ — **done (v0.5.0)**: `pair.granted` returns a device secret once; hello uses it;
96
+ the account token stays in the dashboard only (and is never an SSH credential).
97
+ 2. ~~**Resume**~~ — **done (v0.6.0)**: room streams with `seq`/`gen`, a bounded replay buffer, gap declaration;
98
+ see §3. Ciphertext-friendly by construction (the buffer stores opaque envelopes).
99
+ 3. ~~**E2E payloads**~~ — **done (v0.7.0)**: Noise_XX (P-256/SHA-256/AES-256-GCM, WebCrypto-only so the
100
+ plugin and the PWA run one identical module) over the payload channel; the device initiates after
101
+ pairing, the host acks with `e2e.ready` (nothing encrypts before the ack — a frame racing the
102
+ handshake's last step would desync the counters forever), and a decrypt failure drops the session for
103
+ renegotiation. Host static key is TOFU: the device pins its fingerprint and screams if it changes.
104
+ See DESIGN §3 and `plugin/server/e2e.mjs`.
105
+ 4. ~~**Official Herdr adapter**~~ — **done (v0.1.0)**, read-only. `src/host/herdr.ts` runs
106
+ `herdr api snapshot` (verified on Herdr 0.9.5, protocol 22) and maps `workspaces[]`, `tabs[]`, `panes[]`,
107
+ `agents[]` and the focused ids into a versioned view. Device ops are `view` and `ping`; every other op is
108
+ refused on the host with `read-only`. Next in this line: an `act` op set once per-op policy and audit exist.
109
+ 5. **Device PWA** — shipped: pairing, multi-room dashboard, per-device SSH key management and read-only pane views.
110
+ 5b. **SSH through the Gate** — shipped: per-device public keys, TUI bridge, real Core and the scoped Mobile
111
+ TUI adapter (see §8).
112
+ 5c. **SSH without inbound publisher ports** — shipped: SSH over WSS for desktop clients and a tokenless
113
+ TCP edge for Mobile. The old owner-token SSH bridge is a separate legacy deployment.
114
+ 6. ~~**Push wake**~~ — **done (v0.8.0)**: VAPID keys beside the state file; a paired device stores its web-push
115
+ subscription (`/v1/device/push`); the host sends `wake {device}` and the cloud delivers a silent,
116
+ payload-less push — but only when the device is not connected (a connected one is already live). The
117
+ PWA's service worker raises a notification only when no window is open.
118
+ 7. Multi-account signup, quotas, Postgres state, audit log.
119
+
120
+ ## 7. Host payload contract (herdgate.herdr.v1)
121
+
122
+ The cloud routes `payload` blind, so the host and its devices own their own schema. Current ops:
123
+
124
+ | Direction | Message | Reply |
125
+ |---|---|---|
126
+ | device → host | `{op:"view"}` | `{op:"view", view}` (lazy read if none yet) |
127
+ | device → host | `{op:"ping", n?}` | `{op:"pong", n, at}` |
128
+ | host → device | `{op:"view", view}` | pushed when the view changes, and to a newly paired device |
129
+ | host → device | `{op:"error", code, message}` | `read-only` for any other op; `host-unavailable` when Herdr cannot be read |
130
+
131
+ `view` = `{v:1, source:"herdr", herdrVersion, protocol, takenAt, focused{workspaceId,tabId,paneId},
132
+ workspaces[{id,label,number,tabs,panes,agentStatus,focused}], agents[{name,kind,status,workspaceId,tabId,paneId,cwd,title}]}`.
133
+ A new Herdr version that adds keys keeps working (every field is read defensively); a missing key degrades to an
134
+ empty/honest value rather than throwing.
135
+
136
+ The table describes the standalone metadata adapter. The TUI publisher also exposes read-only
137
+ `term.list` and `term.read` for the PWA. Browser payload operations do not grant terminal input;
138
+ SSH access is a separate channel with its own authorization.
139
+
140
+ ## 8. How herdgate attaches to Herdr (official surface only)
141
+
142
+ Herdr is not forked, patched, or re-implemented. herdgate is a **Herdr plugin**: a directory with
143
+ `herdr-plugin.toml` whose commands Herdr validates, launches, and logs — the surface Herdr documents for
144
+ extensions (`docs/plugins.mdx`) and lists in the marketplace (`herdr-plugin` GitHub topic, `/tui/plugins/`).
145
+
146
+ | Manifest entry | Command | Why |
147
+ |---|---|---|
148
+ | `[[startup]]` | `node server/index.mjs serve` | Herdr runs it once after the socket API is ready; it spawns the long-lived publisher and exits, so a failed publisher cannot fail the Herdr server |
149
+ | `[[actions]]` | `pair`, `status`, `stop` | the human-facing operations, invoked as `herdr plugin action invoke dev.herdgate.host.pair` |
150
+ | `[[panes]]` | `node server/index.mjs pane` | a `split` pane inside Herdr showing honest publish state (light 10s poll) |
151
+
152
+ Two official surfaces are used, one per program: the **plugin** (§ above, `herdr-plugin.toml`) for the TUI, and
153
+ the **app mod** (§ below, `mods/<name>/`) for the desktop app.
154
+
155
+ The plugin uses Herdr's CLI for metadata and pane reads, and fixed Herdr bridge commands for remote TUI
156
+ connections. The interactive Mobile adapter additionally uses Herdr's native terminal session control.
157
+ These integrations use the running TUI server's configured executable and socket, not a second Herdr
158
+ server. Nothing patches Herdr.
159
+
160
+ Credentials go in `HERDR_PLUGIN_CONFIG_DIR`, not the managed plugin checkout; runtime state goes in
161
+ `HERDR_PLUGIN_STATE_DIR`. The standalone plugin uses Node's native `WebSocket`; the opt-in Linux Gate
162
+ loopback transport uses the bundled `ws` dependency.
163
+
164
+ Note on Herdr's moving parts: the **TUI** (`herdr`) exposes workspaces/tabs/panes/agents; **herdr-core** (`hc`)
165
+ is the separate runtime under the Herdr app and its iPhone/iPad/Android app, with a JSON-RPC API of its own.
166
+ The plugin attaches to the TUI's documented CLI.
167
+
168
+ ### The Herdr app mod (`app-mod/herdgate/`)
169
+
170
+ The desktop app extends through **mods**, a different official surface (`docs/mods.md`,
171
+ `defaults/AGENTS.md`): `mods/<name>/` with `mod.json` + `main.js`, written in gpui-kit. A mod is trusted code
172
+ with `fetch`, `exec`, and unrestricted network — but **no Node globals** (`process`, `require`, `setTimeout`
173
+ are absent). Verified in a real window: `herdr-app check` passes, `mod.open` reports `overlays:["herdgate"]`
174
+ with `errors: []`, and a probe mod reached a live herdgate cloud with `fetch` (HTTP 200).
175
+
176
+ The mod and the TUI plugin share one publisher: the mod owns the pairing conversation (mint a code, poll,
177
+ write the token 0600) and shells out to `plugin/server/index.mjs serve` for the long-lived connection. One
178
+ implementation, two surfaces.
179
+
180
+ ### SSH clients: TUI, real Core and the Mobile TUI adapter
181
+
182
+ The Gate (`src/cloud/sshd.ts`) terminates SSH and authenticates the device ID with its registered public
183
+ key or device secret, never with the owner token. Each exec is an ephemeral channel bound to the
184
+ authenticated device and room. The publisher dispatches supported commands to fixed argv, not a shell.
185
+ A VPS TCP edge, when used, carries raw SSH over WSS and holds no Gate credentials; the client verifies
186
+ and pins the Gate's SSH host key. SSH channels are not replayed or resumed after a disconnect.
187
+
188
+ | Target | Host scope enforcement |
189
+ |---|---|
190
+ | Existing Herdr TUI via the desktop TUI bridge | `act` only; the native two-way bridge is not a read filter. |
191
+ | Real `herdr-core` | A `read` device gets filtered plain JSON-RPC; framed hello is downgraded. Approved `act` gets byte-transparent Core RPC. |
192
+ | Existing TUI panes via the Mobile adapter | Read/observe attaches show a text snapshot; no input or resize. An explicitly configured device with matching `act` scope can use Send, keys, Interrupt and native attach with ANSI, input and resize. |
193
+
194
+ The Mobile adapter speaks the Core protocol subset needed for those operations and rejects unsupported
195
+ RPCs. It does not implement Core task lifecycle, templates or file operations. See
196
+ `plugin/server/mobile-tui-core.mjs` and `plugin/server/mobile-tui-stream.mjs`.
197
+
198
+ ### Publisher releases and migration
199
+
200
+ The platform-specific publisher runtime is installed outside npm's cache. On Windows `herdgate install`
201
+ installs a per-user, Node-pinned command from npm. First setup also provisions it. Its `command` junction
202
+ is separate from the publisher's `current` junction, so publisher rollback does not remove the command.
203
+ User PATH registration is optional and never changes the machine-wide PATH. Migration stages and validates
204
+ first, then saves the existing device mode only after the new publisher connects; failure triggers a
205
+ rollback attempt. On Linux the CLI installs a user systemd unit, recognizes only the known legacy home
206
+ unit for adoption, and checks the new invocation's Gate hello. Both preserve the room config and the
207
+ running Herdr server. The Gate, TCP edge, and standalone Linux Mobile publishers have independent
208
+ lifecycles.
209
+
210
+ ## 9. Herdr-relay comparison (why a new project)
211
+
212
+ herdr-relay: Mac-local bridge, Tailscale identity, renders Herdr's mods in sandboxed processes, diffs trees
213
+ bridge-side. herdgate: the cloud in the middle, own identity, host renders, cloud routes opaque frames. The
214
+ ideas kept: hello-first auth, hashed tokens, scope-aware routing, single-use artifacts, rate limits everywhere.
215
+ The parts dropped: mod sandboxing (the host is trusted code on its own machine), tree recording (the host owns
216
+ its UI), Tailscale (public via cloudflared).