@bom0792/herdgate 0.9.29 → 0.9.34

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (53) hide show
  1. package/.agents/skills/herdr/SKILL.md +81 -0
  2. package/.agents/skills/herdr/herdr_pi.py +272 -0
  3. package/.agents/skills/herdr/profiles.example.toml +18 -0
  4. package/.agents/skills/herdr-advisor/SKILL.md +12 -0
  5. package/.agents/skills/herdr-committee/SKILL.md +13 -0
  6. package/.agents/skills/herdr-handoff/SKILL.md +19 -0
  7. package/.agents/skills/herdr-help/SKILL.md +14 -0
  8. package/.agents/skills/herdr-plugin/SKILL.md +21 -0
  9. package/DEPLOY.md +106 -120
  10. package/DESIGN.md +9 -4
  11. package/README.md +55 -2
  12. package/bin/herdgate.mjs +12 -2
  13. package/bin/home-tui.mjs +2 -0
  14. package/bin/pi-skills.mjs +99 -0
  15. package/dist-readme.md +9 -2
  16. package/package.json +38 -9
  17. package/plugin/server/e2e.d.mts +2 -2
  18. package/plugin/server/e2e.mjs +51 -32
  19. package/plugin/server/index.mjs +128 -32
  20. package/plugin/server/pi-receipts.d.mts +28 -0
  21. package/plugin/server/pi-receipts.mjs +77 -0
  22. package/plugin/server/pi-worker.d.mts +47 -0
  23. package/plugin/server/pi-worker.mjs +127 -0
  24. package/plugin/server/ssh.mjs +12 -4
  25. package/scripts/bench-mobile-tui.mjs +246 -0
  26. package/scripts/build-host-bundle.sh +14 -8
  27. package/scripts/build-release.sh +47 -0
  28. package/scripts/deploy-vps.sh +5 -78
  29. package/scripts/install-pi-skills.mjs +4 -0
  30. package/src/cloud/gateway.ts +163 -134
  31. package/src/cloud/main.ts +72 -15
  32. package/src/cloud/pairing.ts +7 -2
  33. package/src/cloud/push.ts +54 -0
  34. package/src/cloud/sshd.ts +76 -38
  35. package/src/cloud/util.ts +5 -5
  36. package/src/host/mobile-tui.ts +11 -1
  37. package/src/web/app.js +88 -61
  38. package/src/web/e2e.js +51 -32
  39. package/src/web/sw.js +1 -1
  40. package/templates/cloud/.dockerignore +7 -0
  41. package/templates/cloud/Caddyfile.example +3 -0
  42. package/templates/cloud/Dockerfile +10 -0
  43. package/templates/cloud/README.md +74 -0
  44. package/templates/cloud/compose.env.example +18 -0
  45. package/templates/cloud/compose.yml +41 -0
  46. package/templates/cloud/gitignore.example +4 -0
  47. package/templates/cloud/herdgate-cloud-edge.service +18 -0
  48. package/templates/cloud/herdgate-cloud.service +26 -0
  49. package/tsconfig.cloud.json +11 -0
  50. package/tsconfig.json +18 -0
  51. package/windows/install-cli.mjs +15 -5
  52. package/windows/setup-tui.mjs +1 -0
  53. package/src/web/herdgate-host.zip +0 -0
package/DEPLOY.md CHANGED
@@ -1,15 +1,65 @@
1
- # Deploying herdgate publicly (cloudflared, this machine)
1
+ # Deploying Herdgate
2
2
 
3
- The cloud binds `127.0.0.1` and is published through a Cloudflare tunnel. Two shapes:
3
+ ## Deploying a clean Gate
4
4
 
5
- ## A. Quick tunnel (no DNS setup, throwaway URL)
5
+ From a checkout with Node 22+, npm, Bash, `tar`, `zip` and `sha256sum`:
6
+
7
+ ```sh
8
+ npm ci --ignore-scripts
9
+ npm run build:release
10
+ # Or stage into a new directory without changing the PWA currently being served:
11
+ npm run build:release -- /path/to/new-output
12
+ ```
13
+
14
+ The default `dist/<version>/` contains `herdgate-cloud-<version>.tar.gz`,
15
+ `herdgate-host-<version>.zip`, the npm tarball and `SHA256SUMS`. Verify the set
16
+ with `sha256sum -c SHA256SUMS`. The Cloud archive contains compiled JS, the PWA
17
+ with the matching host ZIP, a dependency lockfile, and Compose/systemd examples.
18
+ The standalone npm command `herdgate cloud serve` is also available from a full
19
+ npm installation on Node with TypeScript stripping, like `npm start`.
20
+
21
+ Extract the Cloud archive and follow its [README](templates/cloud/README.md)
22
+ to deploy via Docker Compose or systemd. Create credentials on the destination:
23
+ no build step copies state, tokens, SSH or VAPID keys, or modifies a running Gate.
24
+ The old `scripts/deploy-vps.sh` is retired and does not connect to the target.
25
+ For an existing Gate, back up and migrate explicitly, keeping the public origin,
26
+ owner token, state, SSH key and any VAPID keys. A clean Gate starts with no rooms.
27
+
28
+ From v0.9.33, Push wake is off by default. If you need it, set `HERDGATE_ENABLE_PUSH=1`
29
+ (and optionally `HERDGATE_PUSH_SUB` to a VAPID contact URI); `HERDGATE_NO_PUSH=1`
30
+ takes precedence. Existing VAPID keys are untouched while wake is off.
31
+
32
+ A PWA room with a Noise host pin no longer accepts plaintext pane data or a different
33
+ host key.
34
+
35
+ ### Client addresses behind a proxy (v0.9.34)
36
+
37
+ Abuse limits (failed-hello strikes, pairing-code minting, SSH-over-WSS caps) key on
38
+ the client address. Connections arriving over loopback — a same-host proxy such as
39
+ cloudflared — automatically get a **per-connection** budget, so one abusive client
40
+ cannot lock everyone behind the proxy out, and the Gate admits SSH tunnels up to its
41
+ global total rather than a collapsed per-address cap. If your edge forwards a real
42
+ client IP header, set `HERDGATE_TRUSTED_PROXY=1` (and optionally
43
+ `HERDGATE_REAL_IP_HEADER`, default `CF-Connecting-IP`) to restore per-client budgets;
44
+ a raw TCP edge cannot convey real addresses, so size the shared budget with
45
+ `HERDGATE_SSH_TUNNELS_PER_IP` and `HERDGATE_SSH_TUNNELS_TOTAL`. Pairing-code minting
46
+ is bounded per client (10 per 10 minutes) and globally (256 pending). Matching legacy pins are upgraded without re-pairing. Keep the publisher’s
47
+ `e2e-host.json` in backups; on a key-change warning verify the identity with the host
48
+ admin rather than clearing browser data. Noise does not cover the separate SSH leg.
49
+
50
+ ## Source checkout behind cloudflared
51
+
52
+ If you are developing or already run from the checkout, bind the cloud to
53
+ `127.0.0.1` and publish it through a Cloudflare tunnel. Two shapes:
54
+
55
+ ### A. Quick tunnel (no DNS setup, throwaway URL)
6
56
 
7
57
  ```sh
8
58
  # terminal 1 — the cloud
9
- cd /home/bom/source/herdgate
10
- export HERDGATE_ACCOUNT_TOKEN=$(openssl rand -base64 32)
11
- export HERDGATE_STATE=$HOME/.herdgate/state.json
12
- npm start
59
+ # From the checkout; use a disposable private directory.
60
+ trial=$(mktemp -d)
61
+ (umask 077; openssl rand -base64 32 > "$trial/owner-token")
62
+ HERDGATE_ACCOUNT_TOKEN_FILE="$trial/owner-token" HERDGATE_STATE="$trial/state.json" npm start
13
63
 
14
64
  # terminal 2 — a trycloudflare URL (random, changes every run)
15
65
  cloudflared tunnel --url http://127.0.0.1:8787
@@ -17,7 +67,7 @@ cloudflared tunnel --url http://127.0.0.1:8787
17
67
 
18
68
  Use the printed `https://<random>.trycloudflare.com` as the public URL. Nothing to configure; fine for testing.
19
69
 
20
- ## B. Named tunnel + your domain (stable hostname)
70
+ ### B. Named tunnel + your domain (stable hostname)
21
71
 
22
72
  ```sh
23
73
  cloudflared tunnel login # once, picks a zone
@@ -26,21 +76,24 @@ cloudflared tunnel route dns herdgate herd.example.com
26
76
 
27
77
  cat > ~/.cloudflared/herdgate.yml <<'YML'
28
78
  tunnel: herdgate
29
- credentials-file: /home/bom/.cloudflared/<TUNNEL-ID>.json
79
+ credentials-file: /absolute/path/to/.cloudflared/<TUNNEL-ID>.json
30
80
  ingress:
31
81
  - hostname: herd.example.com
32
82
  service: http://127.0.0.1:8787
33
83
  - service: http_status:404
34
84
  YML
35
85
 
36
- cloudflared tunnel run --config ~/.cloudflared/herdgate.yml
86
+ cloudflared --config ~/.cloudflared/herdgate.yml tunnel run
37
87
  ```
38
88
 
39
89
  Then start the cloud with the hostname pinned:
40
90
 
41
91
  ```sh
42
- export HERDGATE_ACCOUNT_TOKEN=$(openssl rand -base64 32)
43
- export HERDGATE_STATE=$HOME/.herdgate/state.json
92
+ install -d -m 700 "$HOME/.herdgate"
93
+ # First install only; preserve the token file on upgrades.
94
+ (umask 077; set -C; openssl rand -base64 32 > "$HOME/.herdgate/owner-token")
95
+ export HERDGATE_ACCOUNT_TOKEN_FILE="$HOME/.herdgate/owner-token"
96
+ export HERDGATE_STATE="$HOME/.herdgate/state.json"
44
97
  export HERDGATE_PUBLIC_HOST=herd.example.com
45
98
  export HERDGATE_PUBLIC_URL=https://herd.example.com
46
99
  npm start
@@ -69,7 +122,9 @@ TLS, both wss legs) on this machine, with a throwaway dev cert:
69
122
  openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:P-256 \
70
123
  -keyout /tmp/edge-key.pem -out /tmp/edge-cert.pem -days 2 -nodes -subj "/CN=127.0.0.1"
71
124
 
72
- HERDGATE_PUBLIC_HOST=gate.local HERDGATE_SSH_PORT=0 npm start # terminal 1
125
+ # terminal 1, reuse a local test token file
126
+ HERDGATE_ACCOUNT_TOKEN_FILE=/path/to/test-token \
127
+ HERDGATE_PUBLIC_HOST=gate.local HERDGATE_NO_SSH=1 npm start
73
128
  HERDGATE_TARGET_PORT=8787 HERDGATE_PINNED_HOST=gate.local \
74
129
  HERDGATE_EDGE_CERT=/tmp/edge-cert.pem HERDGATE_EDGE_KEY=/tmp/edge-key.pem \
75
130
  npm run edge # terminal 2
@@ -80,32 +135,8 @@ Verified by `test/edge.test.ts` and a full headless-Chrome run: sign in → `onl
80
135
  machine online → pair read → live Herdr view + SSH hint, all over TLS with both wss legs
81
136
  through the splice.
82
137
 
83
- ### A note on `trycloudflare.com` quick tunnels
84
-
85
- On this machine, quick tunnels **register** but the edge answers **404** for the
86
- trycloudflare hostname from both IPv4 and IPv6 — the free-for-all hostname space is
87
- blocked somewhere between this machine and Cloudflare. **Named tunnels on your own zone
88
- do not have this problem**: `gate.phuongnb.io.vn` (tunnel `herdgate`, id `dfc578ef`) is
89
- live — PWA + CSP through the edge, owner + host wss legs through the edge, pairing and
90
- the live Herdr view all verified in headless Chrome with the real Cloudflare cert (no
91
- `--ignore-certificate-errors`). The live recipe on this machine:
92
-
93
- ```sh
94
- cloudflared tunnel create herdgate
95
- cloudflared tunnel route dns -f <tunnel-uuid> gate.phuongnb.io.vn # -f overwrites; the
96
- # bare form once logged the WRONG tunnel id (fell back to config.yml) — always pass
97
- # the UUID and check the log line says your tunnel id
98
- # ~/.cloudflared/herdgate.yml → ingress gate.phuongnb.io.vn → http://127.0.0.1:8833
99
- HERDGATE_PUBLIC_HOST=gate.phuongnb.io.vn HERDGATE_PORT=8833 HERDGATE_SSH_PORT=2299 npm start
100
- cloudflared --config ~/.cloudflared/herdgate.yml tunnel run herdgate
101
- ```
102
-
103
- Gotchas hit along the way: port 8787 was already taken by the hermes webui (the old
104
- `config.yml` routes `hermes.phuongnb.io.vn` there — an ingress clash would silently serve
105
- the wrong app through our hostname), and running the gateway script without `&` in a
106
- shell helper blocks forever. Local `curl http://127.0.0.1:8833/` returns **421** by
107
- design — the gateway pins `Host: gate.phuongnb.io.vn`; that is the pinning contract
108
- working, not a bug.
138
+ For a permanent deployment, prefer a named tunnel on a domain you control. If a quick tunnel
139
+ registers but returns 404, check the active tunnel and ingress settings before changing DNS.
109
140
 
110
141
  ### The SSH leg — passwordless logins (publickey)
111
142
 
@@ -143,37 +174,15 @@ of setup cost:
143
174
 
144
175
  ### Deploy the tokenless TCP edge on a VPS
145
176
 
146
- Use a public IPv4 VPS with Node 22+ installed system-wide. Build the standalone
147
- JavaScript on a machine with TypeScript-enabled Node 22.18+ before copying it to
148
- the VPS (Ubuntu's packaged Node may not support running `.ts` directly):
149
-
150
- ```sh
151
- node --input-type=module -e 'import { readFileSync } from "node:fs"; import { stripTypeScriptTypes } from "node:module"; process.stdout.write(stripTypeScriptTypes(readFileSync("src/cloud/ssh-tcp-edge.ts", "utf8")))' > /tmp/ssh-tcp-edge.mjs
152
- ```
153
-
154
- Copy the output to `/opt/herdgate-tcp-edge/ssh-tcp-edge.mjs` and install only the
155
- locked `ws` version there (`npm install --save-exact --omit=dev --ignore-scripts --prefix /opt/herdgate-tcp-edge ws@8.23.0`).
156
- Install `templates/herdgate-tcp-edge.service` into `/etc/systemd/system/` (adjust
157
- `ExecStart` if Node is not at `/usr/bin/node`), and put this in
158
- `/etc/herdgate-tcp-edge.env`:
159
-
160
- ```dotenv
161
- HERDGATE_URL=https://gate.phuongnb.io.vn
162
- HERDGATE_TCP_EDGE_BIND=0.0.0.0
163
- HERDGATE_TCP_EDGE_PORT=2299
164
- ```
165
-
166
- Do **not** copy any Herdgate credential or SSH host key to the VPS. Enable the unit
167
- with `sudo systemctl enable --now herdgate-tcp-edge` and allow TCP 2299 in the VPS
168
- provider and OS firewalls. Restrict the VPS's separate admin SSH access. The edge
169
- caps concurrent sessions at 3 per IP and 6 total by default; the gate's own tunnel
170
- caps still apply, and the gate may see several clients as one source behind a proxy.
171
-
172
- Test `VPS_IP:2299` with the existing mobile username and key over cellular before
173
- changing DNS. Confirm the host key matches the gate's pinned fingerprint. To switch
174
- `ssh.phuongnb.io.vn`, add a DNS-only `A` record for the VPS and remove the old `AAAA`
175
- unless it also points to a verified edge. Disable the old home AAAA DDNS timer if
176
- using that hostname for the VPS. Keep the HTTPS hostname unchanged.
177
+ Use a public IPv4 VPS with Node 22+ installed system-wide. The Cloud release
178
+ includes the compiled TCP edge and a separate systemd unit. See the
179
+ [Cloud README](templates/cloud/README.md#optional-native-herdr-mobile-on-cellular).
180
+ For an existing standalone deployment, its old unit and config still work.
181
+ The edge caps concurrent sessions at 3 per IP and 6 total by default; the
182
+ Gate's tunnel caps still apply, and it may see several clients as one source
183
+ behind a proxy. Test the edge directly from Mobile over cellular and verify
184
+ the pinned Gate SSH key before switching any DNS-only records. Keep the HTTPS
185
+ Gate hostname unchanged.
177
186
 
178
187
  ### Legacy authenticated SSH bridge
179
188
 
@@ -208,62 +217,39 @@ existing `ws` dependency from the npm installation; the standalone plugin uses
208
217
  Node's native WebSocket by default. Restart only the publisher. Remove the line
209
218
  to restore the public transport. No client SSH configuration changes are needed.
210
219
 
211
- ## Keep it running
220
+ For the standalone Linux Mobile TUI publisher (`src/host/mobile-tui.ts`),
221
+ set `HERDGATE_AGENT_LOOPBACK_PORT` in its **process environment** (for systemd,
222
+ a drop-in with `[Service]` and `Environment=HERDGATE_AGENT_LOOPBACK_PORT=8833`,
223
+ using the actual local port). Keep the public `HERDGATE_URL`, the existing room
224
+ and the device scope. Only enable this when the Gate is on the same machine.
225
+ Restart that Mobile publisher alone in a maintenance window and verify its
226
+ journal reports `cloud hello: … via loopback 127.0.0.1:…`. Reconnect Mobile
227
+ and test the existing entry. Remove the drop-in setting and restart the same
228
+ publisher to roll back. This does not change the public WSS leg on the VPS,
229
+ the SSH host key, or the Herdr server.
230
+
231
+ ## Measure Mobile latency
212
232
 
213
- On this machine (Linux + systemd) a user unit keeps the gate up; the tunnel is a second unit:
233
+ For a local baseline with an isolated Herdr 0.9.5 server, pane, Gate and TCP
234
+ edge (no live rooms or keys), run:
214
235
 
215
236
  ```sh
216
- mkdir -p ~/.config/systemd/user
217
- cat > ~/.config/systemd/user/herdgate.service <<'UNIT'
218
- [Unit]
219
- Description=herdgate cloud relay hub
220
- After=network-online.target
221
-
222
- [Service]
223
- ExecStart=/home/bom/.nvm/versions/node/v22.23.1/bin/node /home/bom/source/herdgate/src/cloud/main.ts
224
- Environment=HERDGATE_ACCOUNT_TOKEN=change-me
225
- Environment=HERDGATE_PUBLIC_HOST=gate.phuongnb.io.vn
226
- Environment=HERDGATE_PORT=8833
227
- Environment=HERDGATE_STATE=/home/bom/.local/state/herdgate/state.json
228
- WorkingDirectory=/home/bom/source/herdgate
229
- Restart=always
230
- RestartSec=3
231
-
232
- [Install]
233
- WantedBy=default.target
234
- UNIT
235
- systemctl --user daemon-reload && systemctl --user enable --now herdgate
236
- # tunnel: cloudflared --config ~/.cloudflared/herdgate.yml tunnel run herdgate
237
- # (a second unit wrapping that line, same shape)
237
+ TEST_HERDR_BIN="$(command -v herdr)" node scripts/bench-mobile-tui.mjs 25
238
238
  ```
239
239
 
240
- Other platforms:
240
+ Compare native Herdr, the Core adapter over local pipes, and the full Mobile
241
+ transport over loopback. The script reports key-to-ANSI, Send-to-ACK and
242
+ Send-to-ANSI separately; attach is measured separately from steady-state
243
+ input. This does not include cellular RTT or the official app's rendering.
244
+ An observe/read-only TUI attachment instead polls every 1.25 seconds; its
245
+ refresh interval is not comparable to the native interactive stream.
241
246
 
242
- ```sh
243
- # launchd agent sketch (adjust paths); or use tmux/systemd-in-wsl as you prefer
244
- cat > ~/Library/LaunchAgents/dev.herdgate.cloud.plist <<'PLIST'
245
- <?xml version="1.0" encoding="UTF-8"?>
246
- <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
247
- <plist version="1.0"><dict>
248
- <key>Label</key><string>dev.herdgate.cloud</string>
249
- <key>ProgramArguments</key><array>
250
- <string>/Users/bom/.nvm/versions/node/v22.23.1/bin/node</string>
251
- <string>/home/bom/source/herdgate/src/cloud/main.ts</string>
252
- </array>
253
- <key>RunAtLoad</key><true/>
254
- <key>KeepAlive</key><true/>
255
- <key>EnvironmentVariables</key><dict>
256
- <key>HERDGATE_ACCOUNT_TOKEN</key><string>…</string>
257
- <key>HERDGATE_STATE</key><string>/Users/bom/.herdgate/state.json</string>
258
- <key>HERDGATE_PUBLIC_HOST</key><string>herd.example.com</string>
259
- <key>HERDGATE_PUBLIC_URL</key><string>https://herd.example.com</string>
260
- </dict>
261
- </dict></plist>
262
- PLIST
263
- launchctl load ~/Library/LaunchAgents/dev.herdgate.cloud.plist
264
- ```
247
+ ## Keep the Gate running
265
248
 
266
- (The plist above is a sketch — fill the token, keep `node` on your PATH or use the absolute binary.)
249
+ Use the packaged [systemd unit or Compose](templates/cloud/README.md) on Linux.
250
+ For a foreground source/npm install, set a persistent state file and use
251
+ `HERDGATE_ACCOUNT_TOKEN_FILE` instead of storing the token in a unit, plist, or
252
+ shell history. Run the reverse proxy or tunnel separately.
267
253
 
268
254
  ## Migrate the Linux TUI publisher
269
255
 
@@ -309,6 +295,6 @@ share tokens or keys. Use `herdgate update` for later main-publisher updates.
309
295
  - Preserve the Gate state and SSH host key in protected backups. Deleting
310
296
  state loses room/device registrations; replacing the host key invalidates
311
297
  clients' pins.
312
- - **Rotate the account token** by restarting with a new `HERDGATE_ACCOUNT_TOKEN` (devices must update it).
298
+ - **Rotate the account token** by replacing the private token file (or old token environment value) and restarting the Gate. Update browsers that use the owner token.
313
299
  - **Logs** go to stderr (agent connect/pair events). Payloads are never logged — keep it that way.
314
300
  - **Watch the exposure**: `curl -s https://host/healthz` public; everything else needs tokens.
package/DESIGN.md CHANGED
@@ -57,7 +57,9 @@ responses carry a `stream {gen, seq}` header; a reconnecting device adds
57
57
  in order. `gen` changes on every gateway process start, so stale bookkeeping from a
58
58
  previous life invalidates in one comparison — the device gets a fresh hello (`resumed:false`)
59
59
  and re-requests state. A resume from before what was kept answers `stream.gap:true`
60
- (replay what survived; the device re-requests its view). Frames with an `ssh.*` op are
60
+ (replay what survived; the device re-requests its view). Device-targeted frames are
61
+ visible only to their destination device, live and in replay; sequence gaps between
62
+ its frames are normal because the cursor is room-wide. Frames with an `ssh.*` op are
61
63
  **ephemeral**: sent raw, unstamped, never buffered — they belong to an SSH channel that
62
64
  died with the drop, and replaying them into a new SSH session would corrupt its wire
63
65
  protocol. `pair.granted` (carries the one-time device secret) is likewise never replayed.
@@ -100,8 +102,9 @@ single-owner deployment model provides multi-tenant isolation.
100
102
  plugin and the PWA run one identical module) over the payload channel; the device initiates after
101
103
  pairing, the host acks with `e2e.ready` (nothing encrypts before the ack — a frame racing the
102
104
  handshake's last step would desync the counters forever), and a decrypt failure drops the session for
103
- renegotiation. Host static key is TOFU: the device pins its fingerprint and screams if it changes.
104
- See DESIGN §3 and `plugin/server/e2e.mjs`.
105
+ renegotiation. Cipher operations are serialized per direction. Host identity is TOFU: the PWA uses
106
+ a full SHA-256 pin, upgrades a matching legacy pin and refuses changed keys or plaintext pane data
107
+ on a pinned room. See `plugin/server/e2e.mjs`.
105
108
  4. ~~**Official Herdr adapter**~~ — **done (v0.1.0)**, read-only. `src/host/herdr.ts` runs
106
109
  `herdr api snapshot` (verified on Herdr 0.9.5, protocol 22) and maps `workspaces[]`, `tabs[]`, `panes[]`,
107
110
  `agents[]` and the focused ids into a versioned view. Device ops are `view` and `ping`; every other op is
@@ -114,7 +117,9 @@ single-owner deployment model provides multi-tenant isolation.
114
117
  6. ~~**Push wake**~~ — **done (v0.8.0)**: VAPID keys beside the state file; a paired device stores its web-push
115
118
  subscription (`/v1/device/push`); the host sends `wake {device}` and the cloud delivers a silent,
116
119
  payload-less push — but only when the device is not connected (a connected one is already live). The
117
- PWA's service worker raises a notification only when no window is open.
120
+ PWA's service worker raises a notification only when no window is open. Off by default since v0.9.33;
121
+ opt-in wake uses a fixed HTTPS provider list and connects only to an approved public IPv4 DNS result,
122
+ retaining TLS verification for the provider name and never following redirects.
118
123
  7. Multi-account signup, quotas, Postgres state, audit log.
119
124
 
120
125
  ## 7. Host payload contract (herdgate.herdr.v1)
package/README.md CHANGED
@@ -27,7 +27,7 @@ Both remote connections are outbound; the cloud gateway binds loopback.
27
27
  | Herdr Mobile (Core) | A real `herdr-core` runtime over SSH | Core terminals and the Core API, subject to the device scope |
28
28
  | Herdr Mobile (TUI adapter) | Existing Herdr TUI panes over SSH | Read-only snapshots, or opt-in Send, keys, Interrupt and native attach/resize for one approved `act` device; not the full Core API |
29
29
 
30
- The Gateway also supports bounded room-stream resume and VAPID push wake. Pairing, publisher updates and
30
+ The Gateway also supports bounded room-stream resume and opt-in VAPID push wake (off by default). Pairing, publisher updates and
31
31
  Mobile authorization are separate: changing the publisher release does not move a room or grant a new scope.
32
32
 
33
33
  ## Layout
@@ -43,6 +43,15 @@ src/web/ the PWA a device opens — served by the cloud, same origin as t
43
43
  test/ e2e relay, the adapter, and the plugin's own commands
44
44
  ```
45
45
 
46
+ ## Self-hosted release
47
+
48
+ Herdgate ships as one versioned release set. Build it from the checkout with
49
+ `npm ci --ignore-scripts` and `npm run build:release`. The output under `dist/<version>/`
50
+ contains a clean **Cloud** archive (Gate, PWA, optional Mobile TCP edge), a **host** ZIP,
51
+ the **npm** package (CLI, publisher, plugin/mod and Pi skills), and `SHA256SUMS`.
52
+ See [Deploying a clean Gate](DEPLOY.md#deploying-a-clean-gate) for Docker Compose
53
+ or systemd. No credentials, keys, or live state are copied into the Cloud archive.
54
+
46
55
  ## Quickstart
47
56
 
48
57
  ```sh
@@ -132,7 +141,8 @@ algorithm list) that makes real OpenSSH clients silently abandon publickey auth
132
141
  an upstream release carries the fix; the system-ssh test below guards the regression.
133
142
 
134
143
  Gateway flags: `HERDGATE_SSH_PORT` (default 2222, `0` disables), `HERDGATE_SSH_BIND` (loopback; `0.0.0.0` for
135
- LAN phones), `HERDGATE_SSH_KEY` (host key file). Verified against real herdr-core 0.1.21.
144
+ LAN phones), `HERDGATE_SSH_KEY` (host key file). Set `HERDGATE_SSH_PUBLIC_PORT` if an external TCP edge
145
+ advertises a different port. Verified against real herdr-core 0.1.21.
136
146
  Anything that is not `view`/`ping` is refused **on the host** with `read-only` — the cloud cannot see payloads,
137
147
  so the host is the last trusted decision point. `act` scope is denied unless `HERDGATE_ALLOW_ACT=1`.
138
148
 
@@ -350,6 +360,23 @@ Set `HERDR_SOCKET_PATH` if needed to select the existing TUI session. Without
350
360
  mode, use a separate room and set that variable to the exact existing `act`
351
361
  device ID. The publisher never auto-approves new `act` pairings.
352
362
 
363
+ ### Mobile TUI latency (0.9.30+)
364
+
365
+ Interactive Mobile TUI sends the first output after an idle period immediately,
366
+ while continuing to batch bursts. This applies to the Mobile TUI adapter on
367
+ Windows and Linux, not to real Core or read-only snapshot polling.
368
+
369
+ For a standalone Linux Mobile publisher on the **same machine** as the Gate,
370
+ opt into `HERDGATE_AGENT_LOOPBACK_PORT` to avoid routing that leg through the
371
+ public Gate. Keep the canonical HTTPS `HERDGATE_URL`; the room token and
372
+ public Host check still apply. See [deployment and rollback](DEPLOY.md#when-the-publisher-and-gateway-run-on-the-same-machine).
373
+ Do not set loopback on a laptop or other host with a remote Gate.
374
+
375
+ On Windows, `herdgate update` deploys the new adapter and preserves the
376
+ existing Mobile mode and device. On Linux, update and restart standalone
377
+ Mobile services separately; `herdgate update` updates only the main TUI
378
+ publisher. Neither operation updates the Gate or the VPS edge.
379
+
353
380
  ## The PWA (the device side)
354
381
 
355
382
  The cloud serves the PWA itself (`src/web/`, `HERDGATE_WEB_ROOT`, `HERDGATE_NO_WEB=1` to disable). A phone
@@ -373,6 +400,32 @@ Verified live: real Herdr → publisher → cloud → **headless Chrome** signed
373
400
 
374
401
  Going public: [DEPLOY.md](DEPLOY.md) (cloudflared, on this machine).
375
402
 
403
+ ## Herdr skills for Pi
404
+
405
+ The npm package bundles six skills: `herdr`, `herdr-advisor`, `herdr-committee`,
406
+ `herdr-handoff`, `herdr-help` and `herdr-plugin` (sources: [`.agents/skills/`](.agents/skills)).
407
+ Starting with `0.9.31`, a global npm install with scripts enabled, the Windows
408
+ `herdgate install` bootstrap, and successful npm publisher `setup`, `migrate` or
409
+ `update` sync them for the current user. A local npm dependency install, a bare
410
+ `npx` fetch and a standalone host ZIP do not install user skills.
411
+
412
+ Pi discovers the managed pack in `~/.pi/agent/skills/herdgate`, or
413
+ `$PI_CODING_AGENT_DIR/skills/herdgate` for a custom agent directory. Use `/reload`
414
+ in an open Pi session or open Pi again; try `/skill:herdr-help`. Herdgate does not
415
+ install Pi, change its settings/trust, install an extension, or start an agent.
416
+ Python 3.11+ is optional for the profile and JSONL observation helper.
417
+
418
+ `herdgate pi-skills` syncs just the pack without touching the publisher. Set
419
+ `HERDGATE_NO_PI_SKILLS=1` before installation or update to skip syncing. The
420
+ installer leaves other skill folders and locally edited packs alone. Existing
421
+ same-name skills can cause Pi collision diagnostics; keep one copy active.
422
+ To remove only the managed copy, back up or remove the `skills/herdgate` folder
423
+ in your Pi profile; skip sync on later installs if you want it to stay removed.
424
+ The source checkout still provides project skills under the normal Pi trust rules.
425
+
426
+ For users working in this checkout: [skill guide](docs/herdr-skills/HERDR_SKILLS_GUIDE.md)
427
+ and [Herdgate research notes](docs/herdr-skills/HERDGATE_PI_OBSERVATION.md).
428
+
376
429
  ## Herdr-relay lineage
377
430
 
378
431
  herdgate borrows its *ideas* from [herdr-relay](https://github.com/Lapis0875/herdr-relay) — hello-first token
package/bin/herdgate.mjs CHANGED
@@ -11,6 +11,7 @@ import { homedir, hostname } from "node:os";
11
11
  import { spawnSync } from "node:child_process";
12
12
  import { fileURLToPath } from "node:url";
13
13
  import { manageCloud, resolveCloud } from "./cloud-config.mjs";
14
+ import { trySyncPiSkills } from "./pi-skills.mjs";
14
15
  import { readConfig } from "../plugin/server/common.mjs";
15
16
  import { withLatest } from "../windows/npm-update.mjs";
16
17
  import { windowsNode } from "../windows/node-runtime.mjs";
@@ -21,8 +22,10 @@ const ROOT = dirname(HERE); // <pkg>
21
22
  const help = `herdgate — every machine, one herd
22
23
 
23
24
  herdgate install [--no-path] Install the persistent Windows command (no publisher restart).
25
+ herdgate pi-skills Sync the bundled skills into your Pi profile.
24
26
  herdgate cloud add <name> <https-url>
25
27
  herdgate cloud list
28
+ herdgate cloud serve Run a self-hosted Gate from the full npm package (uses HERDGATE_* env).
26
29
  herdgate pair --cloud <name> Pair this machine with owner approval.
27
30
  herdgate setup [--cloud <name>] [--mobile-act <device-id>]
28
31
  Install/configure the publisher for this OS.
@@ -347,8 +350,15 @@ function managePublisher(cmd, args) {
347
350
 
348
351
  const [cmd, ...args] = process.argv.slice(2);
349
352
  try {
350
- if (cmd === "setup-host") setupHost();
351
- else if (cmd === "cloud") manageCloud(args);
353
+ if (cmd === "setup-host") {
354
+ setupHost();
355
+ trySyncPiSkills();
356
+ } else if (cmd === "pi-skills" && !args.length) trySyncPiSkills();
357
+ else if (cmd === "cloud" && args.length === 1 && args[0] === "serve") {
358
+ const entry = join(ROOT, "src", "cloud", "main.ts");
359
+ if (!existsSync(entry)) throw new Error("cloud serve requires the full npm package; use the Cloud archive for Docker or systemd");
360
+ runNode(entry, []);
361
+ } else if (cmd === "cloud") manageCloud(args);
352
362
  else if (cmd === "pair") pairHost(args);
353
363
  else if (process.platform === "win32" && (
354
364
  ["install", "setup", "migrate", "start", "update", "install-tui"].includes(cmd) ||
package/bin/home-tui.mjs CHANGED
@@ -8,6 +8,7 @@ import { dirname, isAbsolute, join } from 'node:path';
8
8
  import { fileURLToPath } from 'node:url';
9
9
  import { setTimeout as delay } from 'node:timers/promises';
10
10
  import { readConfig } from '../plugin/server/common.mjs';
11
+ import { trySyncPiSkills } from './pi-skills.mjs';
11
12
 
12
13
  const unit = 'herdgate-home-tui.service';
13
14
  const marker = '# Managed by Herdgate CLI\n';
@@ -136,6 +137,7 @@ async function setup(args) {
136
137
  }
137
138
  throw error;
138
139
  }
140
+ trySyncPiSkills();
139
141
  console.log('Herdgate publisher connected to Gate; no Herdr process was restarted.');
140
142
  } finally { rmSync(lock, { recursive: true, force: true }); }
141
143
  }
@@ -0,0 +1,99 @@
1
+ // Install only shipped resources into Pi's user skill directory; never edit Pi settings.
2
+ import { createHash } from 'node:crypto';
3
+ import { lstatSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, renameSync, rmdirSync, rmSync, writeFileSync } from 'node:fs';
4
+ import { homedir } from 'node:os';
5
+ import { dirname, join, resolve } from 'node:path';
6
+ import { fileURLToPath } from 'node:url';
7
+
8
+ export const piSkillFiles = [
9
+ ...['herdr', 'herdr-advisor', 'herdr-committee', 'herdr-handoff', 'herdr-help', 'herdr-plugin'].map(name => `${name}/SKILL.md`),
10
+ 'herdr/herdr_pi.py', 'herdr/profiles.example.toml',
11
+ ].sort();
12
+ const source = fileURLToPath(new URL('../.agents/skills/', import.meta.url));
13
+ const marker = '.herdgate-skills-v1';
14
+ const stat = path => {
15
+ try { return lstatSync(path); } catch (error) { if (error.code === 'ENOENT') return null; throw error; }
16
+ };
17
+
18
+ const packDirs = new Set(piSkillFiles.map(name => name.split('/')[0]));
19
+ const packFiles = new Set(piSkillFiles);
20
+ function hasExactLayout(root) {
21
+ const dirs = readdirSync(root, { withFileTypes: true });
22
+ if (dirs.length !== packDirs.size + 1) return false;
23
+ let count = 0;
24
+ for (const dir of dirs) {
25
+ if (dir.name === marker && dir.isFile()) continue;
26
+ if (!packDirs.has(dir.name) || !dir.isDirectory()) return false;
27
+ for (const entry of readdirSync(join(root, dir.name), { withFileTypes: true })) {
28
+ if (!entry.isFile() || !packFiles.has(`${dir.name}/${entry.name}`)) return false;
29
+ count++;
30
+ }
31
+ }
32
+ return count === packFiles.size;
33
+ }
34
+ const digest = files => {
35
+ const hash = createHash('sha256');
36
+ for (const [path, bytes] of files) hash.update(`${path}\0${bytes.length}\0`).update(bytes);
37
+ return hash.digest('hex');
38
+ };
39
+ const readPack = root => piSkillFiles.map(name => [name, readFileSync(join(root, name))]);
40
+
41
+ function syncPiSkills() {
42
+ if (process.env.HERDGATE_NO_PI_SKILLS === '1' || !stat(source)) return;
43
+ const home = homedir();
44
+ const override = process.env.PI_CODING_AGENT_DIR;
45
+ const agent = override ? resolve(override === '~' ? home : /^~[\\/]/.test(override) ? join(home, override.slice(2)) : override) : join(home, '.pi', 'agent');
46
+ const skills = join(agent, 'skills');
47
+ const dest = join(skills, 'herdgate');
48
+ const lock = join(skills, '.herdgate-skills.lock');
49
+ const files = readPack(source);
50
+ const next = digest(files);
51
+ mkdirSync(skills, { recursive: true, mode: 0o700 });
52
+ try { mkdirSync(lock, { mode: 0o700 }); }
53
+ catch (error) {
54
+ if (error.code !== 'EEXIST') throw error;
55
+ throw new Error('another skill sync is in progress (or left a stale .herdgate-skills.lock)');
56
+ }
57
+ try {
58
+ const existing = stat(dest);
59
+ if (existing) {
60
+ if (!existing.isDirectory() || !stat(join(dest, marker))?.isFile()) {
61
+ throw new Error('the destination is not managed by Herdgate; left untouched');
62
+ }
63
+ const prior = readFileSync(join(dest, marker), 'utf8');
64
+ if (!hasExactLayout(dest) || prior !== `${digest(readPack(dest))}\n`) {
65
+ throw new Error('the installed skill pack is locally modified; left untouched');
66
+ }
67
+ if (prior === `${next}\n`) return;
68
+ }
69
+ // Hidden staging/backup directories are not auto-discovered by Pi.
70
+ const stage = mkdtempSync(join(skills, '.herdgate-stage-'));
71
+ const backup = join(stage, 'previous');
72
+ const pending = join(stage, 'next');
73
+ try {
74
+ for (const [name, bytes] of files) {
75
+ const path = join(pending, name);
76
+ mkdirSync(dirname(path), { recursive: true, mode: 0o700 });
77
+ writeFileSync(path, bytes, { flag: 'wx', mode: 0o600 });
78
+ }
79
+ writeFileSync(join(pending, marker), `${next}\n`, { flag: 'wx', mode: 0o600 });
80
+ if (existing) renameSync(dest, backup);
81
+ try { renameSync(pending, dest); }
82
+ catch (error) {
83
+ if (existing) renameSync(backup, dest);
84
+ throw error;
85
+ }
86
+ rmSync(backup, { recursive: true, force: true });
87
+ } finally {
88
+ // If rollback also failed, preserve the backup rather than deleting the user's old pack.
89
+ if (stat(backup)) console.error(`Herdgate: previous skill pack kept at ${backup}`);
90
+ else rmSync(stage, { recursive: true, force: true });
91
+ }
92
+ console.error(`Herdgate: ${existing ? 'updated' : 'installed'} Pi skills at ${dest} (use /reload in Pi).`);
93
+ } finally { rmdirSync(lock); }
94
+ }
95
+
96
+ export function trySyncPiSkills() {
97
+ try { syncPiSkills(); }
98
+ catch (error) { console.error(`Herdgate: could not sync Pi skills: ${error.message}`); }
99
+ }
package/dist-readme.md CHANGED
@@ -2,7 +2,8 @@
2
2
 
3
3
  Bộ này dùng cho máy đã cài **herdr** (app desktop hoặc TUI): cài mod + publisher,
4
4
  máy sẽ tự kết nối **out** tới gate (không mở port, không VPN) và xuất hiện trên
5
- phone/PWA của bạn.
5
+ phone/PWA của bạn. ZIP này không chạy Gate; để tự host Gate mới, dùng
6
+ `herdgate-cloud-<version>.tar.gz` trong cùng bộ phát hành.
6
7
 
7
8
  ## Yêu cầu
8
9
 
@@ -45,6 +46,8 @@ khi cài mới hoặc `herdgate migrate`
45
46
  để tiếp quản publisher cũ. Nếu bản cũ chưa lưu mode, khai đúng chế độ đang dùng:
46
47
  `--mobile-act d_device`, `--mobile-read d_device`, `--hc <path>` hoặc `--plain`.
47
48
  Không dừng publisher hay pair lại trước. `herdgate update` giữ mode đã lưu.
49
+ Bản npm từ 0.9.31 tự đồng bộ bộ skill vào profile Pi của user khi cài/cập nhật
50
+ Herdgate. Dùng `/reload` trong Pi để nhận thay đổi. ZIP này không chứa bộ skill.
48
51
 
49
52
  Nếu dùng ZIP, mở PowerShell ở thư mục vừa giải nén. Trỏ thẳng đến Node 22 nếu
50
53
  máy có nhiều bản Node (sửa đường dẫn nếu cần):
@@ -73,7 +76,10 @@ Mặc định publisher chỉ mở cầu nối TUI, không dùng desktop app hay
73
76
  Mobile là opt-in: `--hc <path>` dùng Core thật; `--mobile-tui d_device`
74
77
  chỉ đọc pane; `--mobile-tui-act d_device` cho phép nhập và resize đối với
75
78
  đúng thiết bị đã được duyệt `act`. Adapter TUI không phải bản thay thế đầy đủ
76
- cho Core. Cài ZIP không tự migrate hay nâng cấp máy nhà, Gate hoặc VPS.
79
+ cho Core. Từ `0.9.30`, Mobile TUI Act gửi ngay phản hồi đầu tiên sau lúc rảnh;
80
+ không cần bật thêm tùy chọn trên Windows. Loopback chỉ dành cho publisher
81
+ Linux chạy cùng máy với Gate, không dành cho laptop dùng Gate từ xa.
82
+ Cài ZIP không tự migrate hay nâng cấp máy nhà, Gate hoặc VPS.
77
83
 
78
84
  ## Bộ gồm
79
85
 
@@ -81,6 +87,7 @@ cho Core. Cài ZIP không tự migrate hay nâng cấp máy nhà, Gate hoặc VP
81
87
  plugin/server/*.mjs → publisher (mod tự spawn bằng node)
82
88
  plugin/herdr-plugin.toml → nếu bạn dùng herdr TUI thay vì app
83
89
  windows/ → bộ cài và CLI publisher Windows
90
+ templates/herdgate-via.mjs → cầu SSH qua WSS/443
84
91
 
85
92
  ## Gỡ
86
93