@turmind/halo 2.2.3 → 2.2.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/README.md +2 -1
  2. package/admin-out/404.html +1 -1
  3. package/admin-out/_next/static/chunks/1989-a9d3b40139c1e9a7.js +1 -0
  4. package/admin-out/_next/static/chunks/4320-0fb65ed376b666a8.js +1 -0
  5. package/admin-out/_next/static/chunks/4362.115405260299210d.js +1 -0
  6. package/admin-out/_next/static/chunks/{7613.fe24922a6a839656.js → 7073.4081ed209e6748bd.js} +1 -1
  7. package/admin-out/_next/static/chunks/{7073.9d5f9dd7a77827b7.js → 7613.9c1657bb3b52d1b5.js} +1 -1
  8. package/admin-out/_next/static/chunks/app/{page-10a4afac9cbc27a2.js → page-7c5bdfc234c8ac16.js} +1 -1
  9. package/admin-out/_next/static/chunks/{webpack-9532ec7f1fa9a757.js → webpack-1970c5140e4019b7.js} +1 -1
  10. package/admin-out/_next/static/css/7eb37d975d4027da.css +1 -0
  11. package/admin-out/index.html +1 -1
  12. package/admin-out/index.txt +5 -5
  13. package/bundled-docs/design/a2a.md +3 -1
  14. package/bundled-docs/design/architecture.md +2 -2
  15. package/bundled-docs/design/canvas-extensions.md +2 -0
  16. package/bundled-docs/design/node.md +18 -39
  17. package/bundled-docs/design/observability.md +2 -2
  18. package/bundled-docs/design/relay.md +1 -1
  19. package/bundled-docs/design/session.md +2 -2
  20. package/bundled-docs/design/storage.md +8 -3
  21. package/bundled-docs/design/ws.md +3 -3
  22. package/bundled-docs/dev/add-model-provider.md +18 -0
  23. package/bundled-docs/dev/api.md +10 -3
  24. package/bundled-docs/dev/deploy.md +1 -1
  25. package/bundled-docs/dev/env.md +3 -2
  26. package/bundled-docs/dev/tools.md +10 -6
  27. package/bundled-docs/guide/README.md +1 -1
  28. package/bundled-docs/guide/agents.md +2 -2
  29. package/bundled-docs/guide/cli.md +2 -31
  30. package/bundled-docs/guide/node.md +57 -36
  31. package/bundled-docs/guide/skills.md +1 -1
  32. package/bundled-docs/requirements/settings.md +62 -2
  33. package/dist/evo-wrapper.js +323 -247
  34. package/dist/index.js +1860 -1965
  35. package/package.json +2 -1
  36. package/templates/agents/default/agent.yaml +1 -1
  37. package/templates/models/aws-bedrock-claude-invoke.yaml +62 -1
  38. package/templates/models/aws-bedrock-openai.yaml +81 -7
  39. package/admin-out/_next/static/chunks/1607-bf227fc5e4db45b7.js +0 -1
  40. package/admin-out/_next/static/chunks/1989-3b2faa2731d973c4.js +0 -1
  41. package/admin-out/_next/static/chunks/4362.9d2ee4f9b6facc49.js +0 -1
  42. package/admin-out/_next/static/css/87d4fc049f865389.css +0 -1
  43. /package/admin-out/_next/static/{bbPEaP5uh2KvLzSHGU2CO → 1969BAiACyey4NBiI4j44}/_buildManifest.js +0 -0
  44. /package/admin-out/_next/static/{bbPEaP5uh2KvLzSHGU2CO → 1969BAiACyey4NBiI4j44}/_ssgManifest.js +0 -0
@@ -1,10 +1,10 @@
1
- # Linked machines (`halo node`)
1
+ # Linked machines (`halo-node`)
2
2
 
3
- > User guide: [guide/node.md](../guide/node.md). Tool reference: [dev/tools.md → Node tools](../dev/tools.md#node-tools). API: [dev/api.md → Linked machines](../dev/api.md#linked-machines-halo-node). Code: `packages/server/src/nodes/` (wired in `index.ts`, `session-agent-builder.ts`), CLI `packages/cli/src/index.ts` (`cmdNode`), Go client `packages/node-go/`, admin `features/nodes/`.
3
+ > User guide: [guide/node.md](../guide/node.md). Tool reference: [dev/tools.md → Node tools](../dev/tools.md#node-tools). API: [dev/api.md → Linked machines](../dev/api.md#linked-machines-halo-node). Code: `packages/server/src/nodes/` (wired in `index.ts`, `session-agent-builder.ts`), machine client `packages/node-go/`, admin `features/nodes/`.
4
4
 
5
5
  ## Problem
6
6
 
7
- The server's agents work on the server's filesystem. Users also want them to touch files on *their own* machine — a laptop or a box behind NAT the server can't reach. `halo node` is a small outbound client: it dials the server over WebSocket, and the workspace's full-access agents get `node_*` tools that run the local file tools (and, opt-in, `shell_exec`) **on that machine**. No inbound port, no model keys, no `halo setup` on the machine.
7
+ The server's agents work on the server's filesystem. Users also want them to touch files on *their own* machine — a laptop or a box behind NAT the server can't reach. `halo-node` is a small outbound client: it dials the server over WebSocket, and the workspace's full-access agents get `node_*` tools that run the local file tools (and, opt-in, `shell_exec`) **on that machine**. No inbound port, no model keys, no `halo setup` on the machine.
8
8
 
9
9
  ## Pieces
10
10
 
@@ -14,51 +14,28 @@ The server's agents work on the server's filesystem. Users also want them to tou
14
14
  | Hub | `nodes/hub.ts` | Server side. `noServer` `WebSocketServer` on `/api/nodes/connect`; one live connection per account; request/response correlation, timeouts, cancel, heartbeat; `nodes:changed` broadcast. Process singleton via `setNodeHub` / `getNodeHub` |
15
15
  | Tools | `nodes/tools.ts` | `buildNodeTools(workspaceRoot)` → the fixed 8 tools. Dispatch: hub (in-process) → `NODES_UNAVAILABLE` |
16
16
  | Routes | `nodes/routes.ts` | Admin CRUD (`/api/nodes/accounts`) |
17
- | Client | `nodes/node-client.ts` | `runNode()` — the `halo node` process: connect, reconnect with backoff, execute requests |
18
- | Executor | `nodes/node-exec.ts` + `node-path.ts` | Machine side. Wraps the **same** `createWorkspaceTools(base, { accessLevel: 'full' })` the server's agents use; adds the root check |
19
- | Go client | `packages/node-go/` | `halo-node`: a standalone Go binary (Linux amd64 / arm64 / armv7 and Windows amd64, no cgo) speaking the same protocol v1 with the same flags. It ports the client, the executor and the parts of `workspace-tools.ts` / `sandbox.ts` it needs (the seven ops, POSIX rm guard, process-group / Windows Job Object kill), with JS value, UTF-16 and V8 regex semantics (`internal/jsv`, `internal/jsregex` over regexp2). Ships as release assets via `.github/workflows/node-go.yml`, not npm |
20
- | Conformance | `packages/server/test/node-go-conformance.test.ts` + `packages/node-go/conformance/` | The TS executor is the oracle: a real hub connects TS `runNode` (`test/helpers/ts-node-main.ts`) and the Go binary side by side, and every result, file and log line is compared. Skipped unless `HALO_NODE_GO_BIN` is set. node-go.yml runs it, along with rm-guard vector `--check` and a 20k regex differential fuzz, on any PR touching either side, so a TS change goes red until Go follows. Accepted differences: [below](#go-client-known-differences-from-the-ts-client) |
17
+ | Client | `packages/node-go/` | `halo-node`: the machine side, a standalone Go binary (no cgo, cross-compiled for Linux amd64 / arm64 / armv7, macOS arm64 / amd64 and Windows amd64). Connects, reconnects with backoff and runs each request through its own port of the local file tools (the seven ops, POSIX rm guard, process-group / Windows Job Object kill) with JS value, UTF-16 and V8 regex semantics (`internal/jsv`, `internal/jsregex` over regexp2), so results read like the server's own tools. Ships as release assets via `.github/workflows/node-go.yml`, not npm |
21
18
 
22
19
  Not built in AgentCore mode (`nodeHub = AGENTCORE ? null : …`), the CLI / TUI or the evo sandbox — there the tools exist but return `NODES_UNAVAILABLE`.
23
20
 
24
- ### Go client: known differences from the TS client
21
+ ### Platforms
25
22
 
26
- Accepted gaps between `halo-node` and `halo node`. In the conformance suite D1 is normalized away, D4's prefix is rewritten before comparing, and D5 is a `known` case that also asserts TS still differs; D11 is pinned the same way by `known` vectors in `internal/jsregex/testdata/regex-regress.jsonl`; the rest have no case. The labels are the ones the test files use.
23
+ CI (`node-go.yml`) runs `go vet` / `go test` natively on Linux and Windows and `go test` for Linux arm64 / armv7 under qemu (binfmt) — including `cmd/halo-node/lifecycle*_test.go`, which run the binary against a fake hub (fatal 401 / 404 / 426 / 4001, retried 409 / 429 / 4009 / other HTTP, backoff, a running call killed on 4001 / drop / SIGTERM / cancel, a TERM-ignoring command escalated to SIGKILL), and `internal/protocol`'s drift test, which reads `server/src/nodes/protocol.ts` and fails when a mirrored constant (path, headers, close codes, frame / result caps, version, ops) differs — plus the rm-guard vector `--check` (`testgen/gen-rmguard-vectors.mjs`: the vectors must still equal what the server's `sandbox.ts` `assertRmSafe` says) and a 20k-pattern regex differential fuzz against Node's `RegExp` (`testgen/regex-fuzz.mjs`). Accepted regex gaps against V8 are pinned by `known` vectors in `internal/jsregex/testdata/regex-regress.jsonl` (D11: a capture inside a quantified group isn't reset per iteration; D10: regexp2 engine limits and its 30 s per-match timeout, which aborts the grep / glob call). macOS builds are only cross-compiled — no macOS runner, not tested on a Mac. They use the POSIX code paths (process groups, rm guard); `internal/jsv`'s errno table is Linux-only, so on macOS a filesystem error reads `Unknown system error -<n>: …` instead of Node's `ENOENT: no such file or directory, …` text.
27
24
 
28
- | # | Difference | Why accepted |
29
- |---|---|---|
30
- | D1 | The text in `could not connect (…)` is Go's `net` error, not Node's (`dial tcp …: connection refused` vs `connect ECONNREFUSED …`). Also log-only: a frame over 8 MiB from the server is `connection lost (1006 …)` in Go, `(1009 …)` in TS; a reset before the welcome has no `(read ECONNRESET)` | log only |
31
- | D2 | A multi-byte character split across two `node_run` output chunks: TS decodes each chunk alone (two U+FFFD), Go joins them | Go is right; cases cover single-chunk output |
32
- | D3 | Killed by an outside signal mid-run, TS waits 500 ms more (`shutdownWithin`), Go doesn't | timing only, same output |
33
- | D4 | CLI errors start `halo-node:` instead of `halo node:` and HELP names the binary; unknown flags / positional args use Go `flag`'s wording. `--server` values WHATWG still parses but Go's `net/url` doesn't take as a host (`https:/h`, `http:h`, a leading / trailing space) are `Invalid URL` in Go; TS connects | local stderr only; these are typos of `https://h` |
34
- | D5 | `node_write` content as an array with an array-like plain-object chunk (`[{length:2}]`): TS writes `0000` and succeeds, Go errors `…Received an instance of Object` | off-schema input (case `write D5 array-like object chunk`) |
35
- | D6, D7 | — (removed: `edit` with `old_string: []` and array `include` in `grep` now match TS) | |
36
- | D8 | `~/.halo/secrets/{config,settings}.yaml` aren't read: shell timeout only from `HALO_SHELL_TIMEOUT`, grep's 50 matches and the 5 MiB output cap fixed | matters only on a machine that also runs a halo server with those changed |
37
- | D9 | `mkdir -p` where the parent exists but mkdir keeps failing ENOENT (procfs, e.g. `/proc/sys/x/y.txt`): Node retries forever and TS never replies; Go returns the ENOENT | Go is right; whole-machine writes to procfs only; no case (TS hangs) |
38
- | D10 | Regex engine limits: V8 throws at `test()` (backtrack stack, `Regular expression too large`), regexp2 matches on; nested nullable quantifiers can backtrack exponentially in regexp2 where V8 prunes. Go caps group nesting at 10000 (`Invalid regular expression: /…/: Stack overflow` at compile, so grep's `Invalid regex pattern` warning; V8 compiles it and throws the same text at `test()` past ~11000 capture / lookaround levels; from 32768 capture groups (`(` × 32768) V8 already refuses at compile with `…: Too many captures` — both warn, the suffix differs). One `test()` over 30s aborts the whole grep / glob with `regex match timed out after 30s on <at> — call aborted …`, `<at>` being `<file>:<line>` for grep's line match and `<file>` for an include / glob match; V8 has no limit, so TS runs into the hub's 120s cancel and keeps spinning. Any other regexp2 match-time failure aborts the same way as `regex engine error on <at> — call aborted` | pathological patterns on both sides; the fuzz only requires Go to compile them |
39
- | D11 | A capture inside a quantified group isn't reset per iteration: `/^(?:(a)\|b)+\1$/` (also `*`, `{2}`) — `"ab"` is true in V8, false in Go; `"aba"` the reverse. regexp2 keeps the last iteration's capture where ECMAScript clears it | backreference to an alternation's capture under a quantifier only; pinned by `known` vectors |
40
- | D12 | A lone surrogate escape in a request's JSON (`"\ud83d"`) arrives in Go as U+FFFD (`encoding/json`), in TS as the surrogate: `edit`'s `old_string` and grep's `pattern` can then match a U+FFFD the file decoded to, where TS finds nothing. A NUL byte in a path is `EINVAL: invalid argument, stat '…'` in Go, `ERR_INVALID_ARG_VALUE` (`…without null bytes`) in TS | off-schema input; same outcome (no match / error), different text |
41
- | D13 | `fs.readFile` limits by bytes, not decoded length: a file over 536870888 bytes (V8's string max) is `Invalid string length` in Go even when it is mostly multi-byte text Node could still hold; over 2 GiB both say `File size (<n>) is greater than 2 GiB`. The cap is on bytes read; decoded, invalid UTF-8 grows to at most 3× (each byte a 3-byte U+FFFD). grep's plain literal patterns search valid UTF-8 directly without widening the line. Other patterns still feed regexp2 one rune per UTF-16 unit (4 bytes each, sized to the actual unit count): a 256 MB ASCII single line still needs ≈1 GB of match input, past what an armv7 process has | ≥512 MiB of text through a node tool; byte count is a safe bound |
42
- | D14 | `node_run`'s stdin is a pipe (FIFO), TS's a socketpair: reads and writes behave the same; only a test like `[ -S /dev/stdin ]` tells them apart | nothing writes to it either way |
43
- | D15 | Built with `CGO_ENABLED=0`, user lookup reads `/etc/passwd` only, not NSS: for an LDAP / SSSD user the name in the `--allow-shell` log line and the home used when `HOME` is unset can differ (TS asks libc) | log text / an unset `HOME` only |
44
- | D16 | A `res` goes out in 32 KiB WebSocket fragments, TS sends one frame: the server's `ws` reassembles before parsing, the 8 MiB cap is per message, and pings can interleave so a slow uplink doesn't starve the pong | wire framing only; pinned by the fragment-count proto check |
45
- | D17 | Windows cancellation / timeout stops the whole Job Object. Go also observes shell exit independently of output EOF and immediately closes the job, terminating background descendants before draining stdout / stderr. TS's win32 `execAsync` kills only cmd.exe, so workers can survive or keep output pipes open. Go preserves `Command timed out after <ms>ms` and `Command failed: <command>` rather than Node's wrapped `chcp …` error text, and decodes failure output as UTF-8 / GBK too (TS only decodes successful output) | Intentional Windows lifecycle / output correctness; native cancel, timeout, shell-exit and decoder tests. The real TS fix belongs in sandbox.ts's win32 exec branch |
46
- | D18 | Windows node paths using `\\?\` / `\\.\` namespace or device prefixes (including slash variants), or any component ending in a dot or space other than `.` / `..`, are refused rather than emulated. The same restrictions apply to resolved paths, including shared roots and junction targets, in folder and whole-machine modes | Ordinary Win32 opens can normalize these spellings to a different object after realpath validation. Go rejects them with a node refusal (the existing outside-root message in folder mode); TS does not. Native read/write regressions cover distinct `inside.` / `inside ` directories beside an outward `inside` junction |
47
-
48
- Windows implementation: the shell starts suspended, is assigned to a `KILL_ON_JOB_CLOSE` job, then resumes. Cancel and timeout call `TerminateJobObject` (no graceful SIGTERM equivalent for hidden cmd.exe), retaining the shared 2 s escalation / 200 ms force-settle ceiling. The byte-capped stdout / stderr streams decode as strict UTF-8, falling back to GBK. Folder checks use case-insensitive Windows `filepath.Rel`, reject other volumes, and resolve the longest existing prefix with `GetFinalPathNameByHandle` (including junctions). A bootstrap cmd.exe gets a new hidden console, runs `chcp 65001`, then starts the command's cmd.exe so it initializes in UTF-8 even without an interactive desktop. The Unicode command passes through `HALO_NODE_COMMAND` in the child environment (delayed expansion in the bootstrap only; the inner `/d /s /c` shell inherits the machine/user `DelayedExpansion` default, like TS), not the bootstrap's legacy-codepage parser. Win32 filesystem errors map through libuv's Node-style errno names. `~/` uses `USERPROFILE`; `~\` stays literal, matching TS.
49
-
50
- CI keeps the Linux TS-oracle conformance / rm-guard vectors unchanged and adds native Windows vet, tests and exe build. The POSIX conformance fixtures (sh commands, chmod, Unix paths and symlink assumptions) are not run on Windows; native Windows tests cover those platform-specific boundaries instead. The POSIX rm-guard tests are build-excluded there because neither client calls the guard on Windows.
25
+ Windows implementation: the shell starts suspended, is assigned to a `KILL_ON_JOB_CLOSE` job, then resumes. Cancel and timeout call `TerminateJobObject` (no graceful SIGTERM equivalent for hidden cmd.exe), retaining the shared 2 s escalation / 200 ms force-settle ceiling. The byte-capped stdout / stderr streams decode as strict UTF-8, falling back to GBK. Folder checks use case-insensitive Windows `filepath.Rel`, reject other volumes, and resolve the longest existing prefix with `GetFinalPathNameByHandle` (including junctions). A bootstrap cmd.exe gets a new hidden console, runs `chcp 65001`, then starts the command's cmd.exe so it initializes in UTF-8 even without an interactive desktop. The Unicode command passes through `HALO_NODE_COMMAND` in the child environment (delayed expansion in the bootstrap only; the inner `/d /s /c` shell inherits the machine/user `DelayedExpansion` default), not the bootstrap's legacy-codepage parser. Win32 filesystem errors map through libuv's Node-style errno names. `~/` uses `USERPROFILE`; `~\` stays literal.
26
+
27
+ The POSIX rm-guard tests are build-excluded on Windows: the guard isn't called there.
51
28
 
52
29
  ## Two modes
53
30
 
54
31
  - **Folder mode** (`--root <dir>`): `root` is realpath'd at start. Every path argument and node_run's `cwd` must resolve inside it, else the call is refused before any I/O (`"<p>" is outside this machine's shared folder (<root>) — …`; machine log `[Node] refused <op> <path>: outside root`). Relative paths resolve against `root`.
55
- - **Whole-machine mode** (no `--root`, or `--root /`): no prefix check — the same reach as a full-access local session, i.e. whatever the machine user can access. Relative paths and node_run's default cwd resolve against `base` = the directory `halo node` was started in. A startup line says so. `node_list` reports the machine as `root: null, wholeMachine: true, base: "<dir>"`; the admin row shows "Whole machine".
32
+ - **Whole-machine mode** (no `--root`, or `--root /`): no prefix check — the same reach as a full-access local session, i.e. whatever the machine user can access. Relative paths and node_run's default cwd resolve against `base` = the directory `halo-node` was started in. A startup line says so. `node_list` reports the machine as `root: null, wholeMachine: true, base: "<dir>"`; the admin row shows "Whole machine".
56
33
 
57
34
  `--allow-shell` is independent of the mode and off by default. Commands run **unsandboxed** as the machine user — in folder mode only their `cwd` is held inside `root`, the command itself can reach anything (v1 accepted trade-off; the startup warning and the guide say so). `{{ns.params.x}}` placeholders are never substituted on the machine (empty namespace set), so no settings secret reaches a command.
58
35
 
59
- ### Path check (`resolveInRoot`)
36
+ ### Path check (`nodepath.InRoot`)
60
37
 
61
- Resolve against root, then walk up with **`lstat`** to the longest existing prefix, `realpath` it, re-append the missing tail, and test containment with `path.relative` (so `/x/root-other` is not inside `/x/root`; drive letters compare right on Windows). `lstat`, not `existsSync`: `existsSync` follows a dangling symlink and reports it missing, so `root/link → /outside/new` would stay unresolved, pass the prefix test, and the write would follow the link out of root. With `lstat` the link is found, its `realpath` fails, and the call is refused. A narrow TOCTOU window (a component swapped for a symlink between check and syscall) remains — same as the local tools.
38
+ Resolve against root, then walk up with **`lstat`** to the longest existing prefix, `realpath` it, re-append the missing tail, and test containment with `filepath.Rel` (so `/x/root-other` is not inside `/x/root`; drive letters compare right on Windows). `lstat`, not a following `stat`: that follows a dangling symlink and reports it missing, so `root/link → /outside/new` would stay unresolved, pass the prefix test, and the write would follow the link out of root. With `lstat` the link is found, its `realpath` fails, and the call is refused. A narrow TOCTOU window (a component swapped for a symlink between check and syscall) remains — same as the local tools.
62
39
 
63
40
  When the checked path equals the plain resolution, the agent's own string is passed through, so results read exactly like the local tool's (`File written: notes/a.md`).
64
41
 
@@ -74,6 +51,8 @@ When the checked path equals the plain resolution, the agent's own string is pas
74
51
  | `x-halo-node-info` | base64url JSON `{ root \| null, base, shell, shellTimeoutMs, os, hostname, version }` |
75
52
  | any `--header` | passed through (auth proxies) |
76
53
 
54
+ Forward proxy: `--proxy <http(s)://[user:pass@]host:port>` is used alone; without it `HTTPS_PROXY` / `HTTP_PROXY` / `NO_PROXY` apply (Go's `http.ProxyFromEnvironment`: loopback servers are never proxied). A `wss://` server goes through `CONNECT`, so the token stays inside TLS; `user:pass@` becomes `Proxy-Authorization: Basic` and is never printed (the startup line shows `Redacted()`). Redirects are never followed: a 3xx is `server refused the connection (HTTP 3xx)` and the token never goes to the `Location`.
55
+
77
56
  `verifyClient` answers in this order: **429** IP locked out (`middleware/brute-force.ts`, own `node-token` bucket: 5 bad tokens in 15 min) → **426** protocol mismatch → **401** unknown token (recorded as a failure) → **404** bound workspace gone → **400** info / instance missing or malformed → **409** account already connected from a *different* instance. The path is in no auth middleware; the token is the auth.
78
57
 
79
58
  **Frames** (JSON text, ≤ `MAX_FRAME_BYTES` = 8 MiB both ways):
@@ -87,7 +66,7 @@ node → server { t: 'res', id, ok: true, text } | { t: 'res', id, ok: false, e
87
66
 
88
67
  `args` are the local tool's input minus `node` (node_run: `{ command, cwd }`). Results longer than `NODE_RESULT_MAX_CHARS` (256K) are cut on the machine with `[Content truncated on the machine: N chars total]`; the agent loop's own caps (8000 to the LLM, 64K to the UI) then apply as for any local tool. A request over 8 MiB is refused before sending.
89
68
 
90
- **Close codes**: `4001` token revoked (DELETE in admin) → node exits 1. `4009` replaced: a reconnect from the **same** instance supersedes the old socket (half-open link recovery); a **different** instance is refused (`409` at handshake, or `4009 already connected from another process` if it raced) and keeps retrying with backoff — so starting a second `halo node` with the same token never kicks the first. `1001` server shutting down → reconnect.
69
+ **Close codes**: `4001` token revoked (DELETE in admin) → node exits 1. `4009` replaced: a reconnect from the **same** instance supersedes the old socket (half-open link recovery); a **different** instance is refused (`409` at handshake, or `4009 already connected from another process` if it raced) and keeps retrying with backoff — so starting a second `halo-node` with the same token never kicks the first. `1001` server shutting down → reconnect.
91
70
 
92
71
  **Liveness**: server pings every 20 s and terminates after 2 missed pongs; the node's watchdog terminates after 60 s without any traffic. Either side dropping fails all in-flight calls.
93
72
 
@@ -102,13 +81,13 @@ Results are never replayed. When the link drops mid-call:
102
81
  - server side: each pending call rejects with `machine "<n>" went offline during the call (connection lost). The operation may or may not have completed on the machine — check its state before retrying.`
103
82
  - machine side: every running call is aborted (node_run's process group gets SIGTERM, SIGKILL 2 s later) and the log reads `connection lost (…) — stopped N running call(s); reconnecting in Xs`.
104
83
 
105
- When `halo node` itself exits — Ctrl-C (`stop`) or a fatal refusal such as a revoked token (4001) — the same abort runs, but `done` (and so `process.exit`) waits until the aborted calls have settled, capped at 2.5 s. The SIGKILL follow-up is a timer in the node process and the command's group is detached, so exiting first would leave a TERM-ignoring command running on the machine.
84
+ When `halo-node` itself exits — Ctrl-C or a fatal refusal such as a revoked token (4001) — the same abort runs, but the process exits only after the aborted calls have settled, capped at 2.5 s. The SIGKILL follow-up is a timer in the node process and the command's group is detached, so exiting first would leave a TERM-ignoring command running on the machine.
106
85
 
107
86
  Offline vs unknown are distinct errors: `machine "<n>" is offline (…). Do not retry in a loop — tell the user to start it.` vs `unknown machine "<n>" — machines linked to this workspace: a, b (see node_list)`.
108
87
 
109
88
  ## Cron runs
110
89
 
111
- Cron runs execute in the server process since 2.2 ([cron.md](cron.md#execution-runjob)), so their node tools reach the hub directly like any other server-side session. The loopback relay the old `halo cli` child used (`POST /api/nodes/call`, a per-boot secret in `HALO_NODE_RELAY_URL` / `HALO_NODE_RELAY_TOKEN`) is gone. `child-env.ts` still strips `HALO_NODE_TOKEN` from every child it builds, so a `halo node`'s own `node_run` commands don't see its pairing token — env hygiene, not a boundary: a full-level command runs unsandboxed as the same user and can read that user's processes and files anyway.
90
+ Cron runs execute in the server process since 2.2 ([cron.md](cron.md#execution-runjob)), so their node tools reach the hub directly like any other server-side session. The loopback relay the old `halo cli` child used (`POST /api/nodes/call`, a per-boot secret in `HALO_NODE_RELAY_URL` / `HALO_NODE_RELAY_TOKEN`) is gone. `child-env.ts` still strips `HALO_NODE_TOKEN` from every child the server builds, and `halo-node` drops it from its own environment at start and from every `node_run` command's — env hygiene, not a boundary: a full-level command runs unsandboxed as the same user and can read that user's processes and files anyway.
112
91
 
113
92
  ## Prompt-cache invariant
114
93
 
@@ -118,4 +97,4 @@ Cost: the 8 descriptions measure ≈ 460 tokens (Claude CountTokens); with schem
118
97
 
119
98
  ## Admin
120
99
 
121
- `features/nodes/` adds **Channels → Linked machines** (after Web): a row per token with online dot, name, workspace, `root | Whole machine · shell on/off · os`, token copy and delete. The add dialog takes workspace + name and shows two ready-to-copy commands (token in the env, not a flag). **Linux** is `curl -fsSL <base>/install.sh | sh`, then `PATH="$HOME/.local/bin:$PATH" HALO_NODE_TOKEN=<t> halo-node --server <origin> --root <folder>` — a non-root install lands in `~/.local/bin`, which Debian / Ubuntu / Raspberry Pi OS put on `PATH` only at a login after it exists. **macOS / Windows** runs `HALO_NODE_TOKEN=<t> halo node --server <origin> --root <folder>` (npm). `install.sh` (`packages/node-go/install.sh`, POSIX sh) is uploaded per release with that release's version stamped in. It maps `uname -m`, downloads the binary, checks it against `SHA256SUMS` and runs its `--version`, then installs to `/usr/local/bin` (root) or `~/.local/bin` by a rename in that directory, so a running `halo-node` is replaced without "Text file busy". No token, no service. The base comes from `/api/health`'s `version`: a plain `x.y.z` pins `releases/download/v<x.y.z>`, so the machine and the server come from the same tag; a dev build (`x.y.z-<sha>`) or a failed fetch uses `releases/latest/download`. Online state comes from the list response, then `nodes:changed` (full snapshot, global broadcast — the channels page is a global view); a WS reconnect drops the pushed snapshot and re-fetches the list. No polling.
100
+ `features/nodes/` adds **Channels → Linked machines** (after Web): a row per token with online dot, name, workspace, `root | Whole machine · shell on/off · os`, token copy and delete. The add dialog takes workspace + name and shows two ready-to-copy commands (token in the env, not a flag). **Linux / macOS** is `curl -fsSL <base>/install.sh | sh`, then `PATH="$HOME/.local/bin:$PATH" HALO_NODE_TOKEN=<t> halo-node --server <origin> --root <folder>` — a non-root install lands in `~/.local/bin`, which Debian / Ubuntu / Raspberry Pi OS put on `PATH` only at a login after it exists. **Windows** (PowerShell) downloads `<base>/halo-node-windows-amd64.exe` with `Invoke-WebRequest`, sets `$env:HALO_NODE_TOKEN` and runs `.\halo-node-windows-amd64.exe --server <origin> --root <folder>`. `install.sh` (`packages/node-go/install.sh`, POSIX sh) is uploaded per release with that release's version stamped in. It maps `uname -s` (Linux / Darwin) and `uname -m` (Darwin: `arm64` → arm64, `x86_64` → amd64), downloads the binary, checks it against `SHA256SUMS` and runs its `--version`, then installs to `/usr/local/bin` (root) or `~/.local/bin` by a rename in that directory, so a running `halo-node` is replaced without "Text file busy". No token, no service. The base comes from `/api/health`'s `version`: a plain `x.y.z` pins `releases/download/v<x.y.z>`, so the machine and the server come from the same tag; a dev build (`x.y.z-<sha>`) or a failed fetch uses `releases/latest/download`. Online state comes from the list response, then `nodes:changed` (full snapshot, global broadcast — the channels page is a global view); a WS reconnect drops the pushed snapshot and re-fetches the list. No polling.
@@ -82,7 +82,7 @@ One per turn — the root span, opened by `beginTurn` and closed by `endTurn`.
82
82
  | `gen_ai.task.output` | | final assistant text |
83
83
  | `gen_ai.system_instructions` | | system prompt, capped 8 KB — once per turn here, not on every `chat` |
84
84
 
85
- On error: status `ERROR` + the error message, plus a low-cardinality `error.type` attribute (parsed `<ErrName>Error/Exception` prefix, else `_OTHER`). A turn that ends in a model refusal (`stop_reason: "refusal"`) counts as an error too: ERROR span with `error.type=_OTHER`, and `halo.turn.duration` records `outcome=error`.
85
+ On error: status `ERROR` + the error message, plus a low-cardinality `error.type` attribute (parsed `<ErrName>Error/Exception` prefix, else `_OTHER`). A turn that ends in a model refusal (stop reason `refusal` — Anthropic's refusal or an OpenAI-format provider's content-filter stop) counts as an error too: ERROR span with `error.type=_OTHER`, and `halo.turn.duration` records `outcome=error`.
86
86
 
87
87
  ### `chat <modelId>`
88
88
 
@@ -94,7 +94,7 @@ Child of `invoke_agent`, one per model call — created retroactively on the `us
94
94
  | `gen_ai.request.model` | ✓ | |
95
95
  | `gen_ai.usage.input_tokens` | ✓ | |
96
96
  | `gen_ai.usage.output_tokens` | ✓ | |
97
- | `gen_ai.response.finish_reasons` | `[<stop reason>]` — that call's own stop reason, already normalized by the provider: `end_turn`, `tool_use`, `max_tokens`, `stop_sequence` or `refusal` (a truncated or refused call is no longer exported as `end_turn`); omitted if the call reported none | |
97
+ | `gen_ai.response.finish_reasons` | `[<stop reason>]` — that call's own stop reason, already normalized by the provider: `end_turn`, `tool_use`, `max_tokens`, `stop_sequence` or `refusal` (a truncated or refused call is no longer exported as `end_turn`). `refusal` covers Anthropic's `stop_reason: "refusal"` and the OpenAI-format providers' content-filter stops (finish_reason `content_filter` / `sensitive`, Responses `incomplete_details.reason: "content_filter"` or a `refusal` content part); omitted if the call reported none | |
98
98
  | `halo.ttft_ms` | streamed calls only — ms from request start to the first delta (text / thinking / tool-args), same number as the admin `ttft` badge | |
99
99
  | `gen_ai.input.messages` | | semconv JSON (see below) — **delta only**: messages appended since this turn's previous `chat` span (first call → the user message; later calls → that cycle's `tool_result`s) |
100
100
  | `gen_ai.output.messages` | | semconv JSON, only the trailing assistant message |
@@ -22,7 +22,7 @@ Relay rides the same "deliver a session's wrap-up to some *other* session when i
22
22
 
23
23
  **Hard interrupt** adds one step: if `sendUserMessage` returned `queued` (target was busy), call `target.interruptSession(session_id)`. Order is enqueue-then-abort, same as `querySession(interrupt=true)` — the aborted turn's finally sees a non-empty queue, so it drains into the message instead of firing a relay report for the half-done turn.
24
24
 
25
- **Delivery** (`deliverRelayReport(host, session)`, runs at *every* root turn end in every workspace): cheap `reply_to` column read first (no-op for the 99.9 % of sessions without one) → subtree-quiet gate → body = `finalOutput || output`, `[RELAY TARGET ABORTED …]` prefix when `turnError` is set (before the cap, so the marker survives; for an account error — 401/402/403, bad key, no balance — the closing hint says to fix the target's model configuration and not to resume it with `relay_send`, since a re-send fails the same way; for a model refusal — `turnErrorKind === 'refusal'`, the target's history tripped the safety classifier — it says not to resume it and to `relay_send` to a NEW session id with a clean brief instead) → cap at `limits.autoReportMax` with a `relay_read(...)` pointer → header `[Relay report · workspace <ws> · session <id> · status: completed]` (`status: aborted` when `turnError` is set) → `clearReplyTo` → `registry.getOrCreate(reply_to.workspace)` → `appendUserMessage` + `sendUserMessage` on the caller. Append-then-send is the same pair the run-ledger nudge uses: `sendUserMessage` alone never writes the UI transcript, so the secretary's admin view would show a reply to nothing.
25
+ **Delivery** (`deliverRelayReport(host, session)`, runs at *every* root turn end in every workspace): cheap `reply_to` column read first (no-op for the 99.9 % of sessions without one) → subtree-quiet gate → body = `finalOutput || output`, `[RELAY TARGET ABORTED …]` prefix when `turnError` is set (before the cap, so the marker survives; for an account error — 401/402/403, bad key, no balance — the closing hint says to fix the target's model configuration and not to resume it with `relay_send`, since a re-send fails the same way; for a model refusal — `turnErrorKind === 'refusal'`, the target's history tripped the provider's safety / content filter — it says not to resume it and to `relay_send` to a NEW session id with a clean brief instead) → cap at `limits.autoReportMax` with a `relay_read(...)` pointer → header `[Relay report · workspace <ws> · session <id> · status: completed]` (`status: aborted` when `turnError` is set) → `clearReplyTo` → `registry.getOrCreate(reply_to.workspace)` → `appendUserMessage` + `sendUserMessage` on the caller. Append-then-send is the same pair the run-ledger nudge uses: `sendUserMessage` alone never writes the UI transcript, so the secretary's admin view would show a reply to nothing.
26
26
 
27
27
  ## Invariants and why
28
28
 
@@ -156,7 +156,7 @@ Hierarchical encoding: `root_id>child_segment>grandchild_segment`.
156
156
 
157
157
  Both true → sets `stoppedAt` + emits `agent_done` + `querySession`s the result back to the parent. The parent's `querySession` clears its own `stoppedAt`, handles the report, and may trigger its own `tryReportToParent` — bubbling up.
158
158
 
159
- **Abnormal-termination marker**: when the reported turn was killed by an unrecoverable error (retry budget exhausted / account-level failure), the session's `turnError` field holds the error text and the auto-report is prefixed with an explicit `[SUB-AGENT ABORTED: … Error: <text> … Re-dispatch with query_session(…) to resume.]` block. An **account** error (`turnErrorKind === 'account'`: 401/402/403, bad key, no balance) replaces the trailing hint with "this is a model account / credential / balance / permission problem — retrying or re-dispatching will fail the same way; fix the model configuration first (or tell the user), and do not resume it with `query_session` until then"; `RELAY TARGET ABORTED` ([relay.md](relay.md)) and goal mode's `WORKER ABORTED` get the same swap. A **model refusal** (`stop_reason: "refusal"`, see **Refusal stop** under [Resilient execution loop](#resilient-execution-loop)) also sets `turnError` (`Model declined to respond (<category>): <explanation>`) with `turnErrorKind === 'refusal'`, whose hint reads "this session's conversation history tripped the model's safety classifier — resuming it with `query_session` or re-sending will be declined the same way; do not resume it — start a NEW session with start_session and a clean brief instead, rewording the part that tripped the classifier" (relay / goal / A2A word it for their own context). Without it the parent LLM consumed mid-turn fragments (or a literal "(no output)") as completed reports — the cron-era "sub-agent reported without finishing" incident, root-caused to a Bedrock h2 hang (`TimeoutError: http2 request did not get a response`, now also in the transient-transport retry list). Prefix, don't suppress: skipping the report would leave the parent waiting forever, the partial trace has diagnostic value, and `stoppedAt` is stamped as usual so the child never shows as falsely running. The marker is prepended *before* the truncation cap so it can never be sliced off.
159
+ **Abnormal-termination marker**: when the reported turn was killed by an unrecoverable error (retry budget exhausted / account-level failure), the session's `turnError` field holds the error text and the auto-report is prefixed with an explicit `[SUB-AGENT ABORTED: … Error: <text> … Re-dispatch with query_session(…) to resume.]` block. An **account** error (`turnErrorKind === 'account'`: 401/402/403, bad key, no balance) replaces the trailing hint with "this is a model account / credential / balance / permission problem — retrying or re-dispatching will fail the same way; fix the model configuration first (or tell the user), and do not resume it with `query_session` until then"; `RELAY TARGET ABORTED` ([relay.md](relay.md)) and goal mode's `WORKER ABORTED` get the same swap. A **model refusal** (stop reason `refusal` — Anthropic's or an OpenAI-format content-filter stop, see **Refusal stop** under [Resilient execution loop](#resilient-execution-loop)) also sets `turnError` (`Model declined to respond (<category>): <explanation>`) with `turnErrorKind === 'refusal'`, whose hint reads "this session's conversation history tripped the provider's safety / content filter — resuming it with `query_session` or re-sending will be declined the same way; do not resume it — start a NEW session with start_session and a clean brief instead, rewording the part that tripped the filter" (relay / goal / A2A word it for their own context). Without it the parent LLM consumed mid-turn fragments (or a literal "(no output)") as completed reports — the cron-era "sub-agent reported without finishing" incident, root-caused to a Bedrock h2 hang (`TimeoutError: http2 request did not get a response`, now also in the transient-transport retry list). Prefix, don't suppress: skipping the report would leave the parent waiting forever, the partial trace has diagnostic value, and `stoppedAt` is stamped as usual so the child never shows as falsely running. The marker is prepended *before* the truncation cap so it can never be sliced off.
160
160
 
161
161
  The full finally-chain order is `tryReportToParent → deliverGoalRound → deliverRelayReport → releaseSession`. `deliverGoalRound` is a retained legacy hook (goal mode's round delivery): it keys off the row's `goal_session_id` and works as before for any existing goal binding. After the parent-report check, a root whose row carries a `reply_to` back-pointer (it was dispatched from another workspace via `relay_send`) gets its wrap-up appended + sent into the caller session in that workspace (same subtree-quiet gate, `reply_to` cleared before sending — see [relay.md](relay.md)), then the session is released. Each hook decides on its own db column (`parent_id` / `goal_session_id` / `reply_to`), so the three never fire for the same session.
162
162
 
@@ -402,7 +402,7 @@ Frontend network issues don't affect the backend:
402
402
  | **4xx multimodal rejection** (`Multimodal data is corrupted` / `Could not process image`) | Replace all image blocks in history with text placeholders, persist, retry — see [Multimodal 4xx degrade](#multimodal-4xx-degrade) below |
403
403
  | Unrecoverable error | Report to user, stop |
404
404
 
405
- **Refusal stop.** `stop_reason: "refusal"` (Anthropic models on Bedrock invoke + generic anthropic) is an HTTP 200 with `stop_details {category, explanation}` — not an error, so it never enters the retry matrix above. `agent-loop.ts` handles it explicitly: no assistant message is pushed, any partial `tool_use` is **not** executed, the loop ends; `session-manager.ts` emits a `system` event `⚠️ [<agent>] Model declined to respond (<category>): <explanation>` suggesting `/session new`. Previously it fell through the `end_turn` path and looked like a silent stall. It also sets the turn's `turnError` (kind `refusal`), so every abnormal-termination marker sees it — the parent's `[SUB-AGENT ABORTED …]`, `RELAY TARGET ABORTED` / `status: aborted`, goal mode's `WORKER ABORTED`, A2A `FAILED` and an AgentCore invocation's `error` — each with a "start a new session" hint instead of "resume it"; the turn sends no interim report, and a `continue_task` armed in it does not kick (see **Kick position** under [Message queue and drain](#message-queue-and-drain)). Before, a refused sub-agent reported `(no output)` (or the last narration) as a finished result, and the parent kept resuming it into the same refusal.
405
+ **Refusal stop.** `stop_reason: "refusal"` (Anthropic-format providers) is an HTTP 200 with `stop_details {category, explanation}` — not an error, so it never enters the retry matrix above. The OpenAI-format layers map a provider's documented content-filter stop to the same `stopReason: 'refusal'`: `chatCompletionResult` (`openai-chat-format.ts`, every `chat/completions` provider) for finish_reason `content_filter` (OpenAI, DeepSeek, Doubao, Hunyuan TokenHub) or `sensitive` (Zhipu), with `category` = that raw value and no explanation; `MantleAgent` (Responses API) for `incomplete_details.reason: "content_filter"` (category `content_filter`, also with an empty `output[]`, which then skips the `MantleEmptyResponse` retry) or a `refusal` content part (category `refusal`, its text as the explanation). Any other unknown finish value stays `end_turn`. `agent-loop.ts` handles it explicitly: no assistant message is pushed, any partial `tool_use` is **not** executed, the loop ends; `session-manager.ts` emits a `system` event `⚠️ [<agent>] Model declined to respond (<category>): <explanation>` suggesting `/session new`. Previously it fell through the `end_turn` path and looked like a silent stall. It also sets the turn's `turnError` (kind `refusal`), so every abnormal-termination marker sees it — the parent's `[SUB-AGENT ABORTED …]`, `RELAY TARGET ABORTED` / `status: aborted`, goal mode's `WORKER ABORTED`, A2A `FAILED` and an AgentCore invocation's `error` — each with a "start a new session" hint instead of "resume it"; the turn sends no interim report, and a `continue_task` armed in it does not kick (see **Kick position** under [Message queue and drain](#message-queue-and-drain)). Before, a refused sub-agent reported `(no output)` (or the last narration) as a finished result, and the parent kept resuming it into the same refusal.
406
406
 
407
407
  **Classification order: structured signal first, message text last.** The interrupt branch is decided by the attempt's own `AbortController` signal (every interrupt path we own goes through `abortReason()` on it), never by the words `cancelled` / `aborted` in the message — an upstream error body can legitimately read `"The operation was aborted due to …"` (Bedrock), and matching that swallowed a real failure as a user interrupt: silent `break`, no retry, no error event, an empty reply. Likewise the account-level branch trusts `httpStatus` when one was recovered: 401/402/403 is terminal, any *other* status is not, whatever the body says — an OpenAI-compatible 503 whose body reads `"authentication service temporarily unavailable"` used to be classified as a dead key and never retried. The keyword list is only consulted when no status could be extracted. Pinned by `turn-retry-idempotent.test.ts` (seven classification cases).
408
408
 
@@ -18,7 +18,7 @@ Defines the persisted-data format for every Halo surface. Format changes must re
18
18
  │ │ └── AGENT.md # Agent personality
19
19
  │ ├── models/ # Model registry, bundled — one file per provider, overwritten on template refresh
20
20
  │ │ └── <providerId>.yaml # e.g. aws-bedrock-claude-invoke.yaml
21
- │ ├── models.d/ # Hub-installed provider yamls (`/extension models`); never touched by seeding
21
+ │ ├── models.d/ # Hub-installed provider yamls (`/extension models`, Settings → Sync latest models); never touched by seeding
22
22
  │ │ └── <providerId>.yaml # merged with models/ by revision (see Model registry format)
23
23
  │ ├── skills/<id>/
24
24
  │ │ └── SKILL.md # Skill definition (frontmatter + body)
@@ -441,6 +441,9 @@ A value of the form `<<ENV_NAME>>` is replaced at read time with `process.env.EN
441
441
  | `general.logging.level` | warn | `debug` / `info` / `warn` / `error`. Restart required |
442
442
  | `general.a2a.public_url` | `''` | Global-only. Origin peers use to reach this server for A2A (no trailing slash): the agent card's interface URL and the default push-webhook base for `a2a_send`. Empty = derived from the request host (no outbound dispatch without it or a remote's `push_base`). Env `HALO_A2A_PUBLIC_URL` overrides. See [a2a.md](a2a.md) |
443
443
  | `general.a2a.url_allowlist` | `100.64.0.0/10,*.ts.net` | Global-only. CIDRs / host patterns A2A egress may reach besides public https (push webhooks, remote cards and RPC, inbound image URLs). Private / loopback only when listed; plain http only to listed hosts / ranges; link-local always refused. See [a2a.md](a2a.md#push-delivery-and-url-policy) |
444
+ | `general.outbound_proxy.enabled` | `false` | Global-only. Outbound proxy on/off: proxy env vars for spawned children, an undici `ProxyAgent` for web_fetch and A2A requests to remotes. Read live |
445
+ | `general.outbound_proxy.url` | `''` | Global-only. `http(s)://host:port`, no user-info (PUT / PATCH 400). Empty = off |
446
+ | `general.outbound_proxy.token` | `''` | **Secret**, global-only. `user:password` or a bare token; Basic auth (URL user-info for children, `Proxy-Authorization` in-process). See [requirements/settings.md](../requirements/settings.md#outbound-proxy) |
444
447
  | `general.observability.endpoint` | `''` | OTLP base URL of an OpenTelemetry collector (e.g. `http://localhost:4318`); empty = off. Restart required (server, cron runs included); evolution runs use a new value on their next run. See [observability.md](observability.md) |
445
448
  | `general.observability.service_name` | `halo` | OTel resource `service.name`. Restart required (server, cron included); evolution: next run |
446
449
  | `general.observability.headers` | `''` | **Secret.** Extra OTLP request headers, comma-separated `k=v`. Restart required (server, cron included); evolution: next run |
@@ -460,12 +463,14 @@ Values present in `settings.yaml` whose namespace isn't currently declared by an
460
463
 
461
464
  ## Model registry format
462
465
 
463
- Paths: `~/.halo/global/models/<providerId>.yaml` (bundled, seeded from `templates/models/` and overwritten on every template refresh) and `~/.halo/global/models.d/<providerId>.yaml` (hub-installed by `halo models install` / `/extension models`; seeding and refresh never touch it) — one file per provider. Both dirs are read on first access and merged into the in-memory registry. The same yaml also declares the provider's required server-side credentials (`secrets:` section), which the Settings page renders.
466
+ Paths: `~/.halo/global/models/<providerId>.yaml` (bundled, seeded from `templates/models/` and overwritten on every template refresh) and `~/.halo/global/models.d/<providerId>.yaml` (hub-installed by `halo models install` / `/extension models` or Settings → Sync latest models; seeding and refresh never touch it) — one file per provider. Both dirs are read on first access and merged into the in-memory registry. The same yaml also declares the provider's required server-side credentials (`secrets:` section), which the Settings page renders; below that form the provider page shows a read-only browse of its `models:` (context / output, capabilities, regions — see [requirements/settings.md](../requirements/settings.md#models-browse-provider-pages)).
464
467
 
465
468
  **Merge rule** (one implementation, `packages/server/src/models/registry.ts` `loadProviders`, behind the server registry, the Settings secrets forms, `halo setup` and the hub installer): per provider `id` the copy with the higher `revision` wins; on a tie the `models.d` copy wins; a yaml without `revision` counts as 0 (users' own custom yamls in `models/`). A `models.d` copy whose `runtime:` this build doesn't know is skipped with a `[Config]` log, so the bundled copy stays in effect. `halo setup` merges the package's `templates/models/` with `models.d/` the same way.
466
469
 
467
470
  **Hub install** (`halo models install <dir> [--yes]`, `models/install.ts`): plan, then write `models.d/<id>.yaml` via tmp + rename. A yaml is refused (others still go ahead) for an unknown `runtime:` ("needs a newer halo"), a missing / non-integer `revision`, a `secrets[].default` other than empty / `<<ENV_NAME>>`, or bad model fields — `contextWindow` / `maxContextWindow` not positive integers, `contextWindow` > `maxContextWindow`, `compressAt` outside (0, 1], `endpoints` not a list of `https://` URLs (reason shaped `models.<id>.<field> …`). An older revision than the copy in effect is skipped, an identical one is up to date. **Consent**: the whole run writes nothing and needs `--yes` (exit 3) when an installed provider is new, its `defaultEndpoint` / `endpointPresets` change, or a model's `endpoints:` gains a URL the provider didn't use anywhere before (not in its old `defaultEndpoint`, presets or any model's list). Removed URLs, and added ones the provider already used, are listed but need no consent.
468
471
 
472
+ **Two writers, one rule set.** `models.d/` is written by `halo models install` (what `/extension models` runs after `ext.sh` downloads the release) and by Settings → Sync latest models (`POST /api/models/update`, `models/hub-update.ts`). The latter fetches the GitHub hub's newest `models-v*` release zip itself (≤ 10 MB), unpacks its `*.yaml` flat by basename into a temp dir, then calls the same `planModelsInstall` / `applyModelsInstall`, so refusals, skips, consent and the tmp + rename write are identical; consent is a re-POST with `{confirm: true}` instead of `--yes`, and nothing is cached between the two calls. GitHub hubs only — other platforms stay with `/extension models`. API: [dev/api.md → Model sync](../dev/api.md#canvas-preview-extensions).
473
+
469
474
  **Live reload**: `models/watcher.ts` watches `models.d/` (non-recursive, 300 ms debounce; created at boot), drops the registry cache and broadcasts `models:changed` when the merged view changed; admin pages re-fetch `/agent-configs/models` and the settings schema. `models/` isn't watched — only template refresh writes it, before the registry is first read.
470
475
 
471
476
  ```yaml
@@ -671,7 +676,7 @@ All channel types (telegram, web, wechat, slack, feishu, wecom) share one table,
671
676
  | Column | Type | Notes |
672
677
  |---|---|---|
673
678
  | account_id | TEXT PK | Channel-specific ID (e.g. `halo_agent_bot`, `abc-im-bot`, `e718bb7b`) |
674
- | channel_type | TEXT NOT NULL | `'telegram'`, `'web'`, `'wechat'`, `'slack'`, `'feishu'`, `'wecom'`, or `'node'` (a `halo node` pairing token) |
679
+ | channel_type | TEXT NOT NULL | `'telegram'`, `'web'`, `'wechat'`, `'slack'`, `'feishu'`, `'wecom'`, or `'node'` (a `halo-node` pairing token) |
675
680
  | workspace_path | TEXT NOT NULL | Absolute path of the bound workspace |
676
681
  | label | TEXT | User-chosen name |
677
682
  | enabled | INTEGER | 1 = active, 0 = disabled |
@@ -120,7 +120,7 @@ Sub-agent text is root-tree noise on this wire: the listener is the root's (sub
120
120
 
121
121
  `chat:thinking` / `chat:stream` / `chat:followup` / `agent:tool_call` / `agent:tool_result` additionally carry `replay: true` (plus `sessionId`) when synthesized by the reattach path (never on live events) — see [Reconnect flow](#reconnect-flow) step 6.
122
122
 
123
- `chat:system` producers (`session-manager.ts`'s `stop` event handling): a `max_tokens` stop emits `⚠️ [<agent>] Response truncated: output token limit reached.`; a `refusal` stop (Anthropic `stop_reason: "refusal"`, HTTP 200 — the model declined, not an error) emits `⚠️ [<agent>] Model declined to respond (<category>): <explanation> — …` suggesting `/session new` (see [session.md](session.md#resilient-execution-loop)).
123
+ `chat:system` producers (`session-manager.ts`'s `stop` event handling): a `max_tokens` stop emits `⚠️ [<agent>] Response truncated: output token limit reached.`; a `refusal` stop (Anthropic `stop_reason: "refusal"` or an OpenAI-format content-filter stop such as finish_reason `content_filter`, HTTP 200 — the model or the provider's filter declined, not an error) emits `⚠️ [<agent>] Model declined to respond (<category>): <explanation> — …` suggesting `/session new` (see [session.md](session.md#resilient-execution-loop)).
124
124
 
125
125
  Server-internal flags on `AgentSessionEvent` that are **not** carried into the WS frame: `stream.final` (marks the turn's wrap-up text vs. pre-tool filler — consumed by channel responders and the cli, see [session.md](session.md#message-queue-and-drain)), `stream.streamed` / `thinking.streamed` (the text already went out as deltas — decides whether the whole event is forwarded at all), `system.outOfBand` (a `notifySystem` notice from outside the turn — the evo auto-apply line; still a plain `chat:system` here, only the cron run and `halo cli` leave it out of their printed text). The admin renders every streamed block, so none is needed on the wire.
126
126
 
@@ -145,11 +145,11 @@ Server-internal flags on `AgentSessionEvent` that are **not** carried into the W
145
145
  | `compact:started` / `compact:summarizing` / `compact:done` | compact handler; `ws/event-processor.ts` | Compaction progress, `{sessionId}`. An auto-compact whose LLM summary failed falls back to a local compact and closes with a `system` event carrying `compactEnd`, which also emits `compact:done` (the success path closes via `compacted`). |
146
146
  | `cron:job_changed` / `cron:run_changed` | `cron/runner.ts` + `routes/cron.ts` (broadcast) | Cron job/run state changed — the Triggers (cron) tab re-fetches instead of polling. `cron:job_changed`: `{jobId, kind?}` (`created` / `updated` / `deleted` / `reconciled` / `missed`; a run's finalize sends `lastRunStatus` + `lastRunAt` instead), plus — whenever the job still exists, except `missed` — `workingDir` (workspace-relative, `null` = root), `stateless` (boolean) and `sessionAgentId` (agent of the job's existing session, `null` when stateless / not created / unreadable; the job's agent is locked to it). `cron:run_changed`: `{jobId, runId, status}`; `status: 'deleted'` = that run row was withdrawn (a webhook fire's re-check found the session taken by another owner) — clients just re-fetch the list. See [cron.md](cron.md). |
147
147
  | `evolution:run_changed` / `evolution:apply_changed` | `evolution/ticker.ts` + `routes/evolution.ts` (broadcast) | Evolution run/apply state changed — drives the Evolution tab. Includes the L2/L3 auto path (`running → approved` plus the new `approved_by='auto'` apply, pushed by the ticker's diff) and `rolled_back` (pushed by the rollback route). See [evolution.md](evolution.md). |
148
- | `models:changed` | `models/watcher.ts` `reloadAndBroadcast` (broadcast to all clients) | `~/.halo/global/models.d/` changed (hub provider yamls installed by `halo models install` / `/extension models`, or removed by hand) and the merged registry differs from the last one — the server has already dropped its registry cache. No payload: the admin bumps its models bus, and the Agents panel / agent form, the composer's image-capability check and the Settings page re-fetch `/agent-configs/models` / the settings schema. Also bumped on WS reconnect (a frame lost while down). Global, no workspace marker. See [storage.md](storage.md#model-registry-format). |
148
+ | `models:changed` | `models/watcher.ts` `reloadAndBroadcast` (broadcast to all clients) | `~/.halo/global/models.d/` changed (hub provider yamls installed by `halo models install` / `/extension models` / Settings → Sync latest models, or removed by hand) and the merged registry differs from the last one — the server has already dropped its registry cache. No payload: the admin bumps its models bus, and the Agents panel / agent form, the composer's image-capability check and the Settings page re-fetch `/agent-configs/models` / the settings schema. Also bumped on WS reconnect (a frame lost while down). Global, no workspace marker. See [storage.md](storage.md#model-registry-format). |
149
149
  | `skills:changed` | `ws/skills-changed.ts` `notifySkillsChanged` (REST `POST` / `DELETE /skills`, `PATCH /skills/:id/toggle`, `/skill disable` · `enable` · `delete`) · `ws/watcher-pool.ts` (the workspace watcher saw `.halo/skills/**` change) | A skill was created / deleted / toggled, or a workspace skill folder changed on disk (an agent's `file_write`, a hand edit). No payload: the admin bumps its skill bus, and the chat slash-command popup, the Skills page and the agent form's skill picker re-fetch. Scoped on the server: a workspace skill or a toggle (the disable flag lives in that workspace's db) goes via `broadcastToWorkspace`, a global skill via `broadcast`. Only real changes emit: a no-op mutation (409 / 404 / already disabled) sends nothing, and the watcher path compares a per-workspace key (folder ids + each SKILL.md's frontmatter), so body / resource edits and the echo of a mutation that already notified stay silent. The watcher debounce is 300 ms on top of the file watcher's own 300 ms flush. `~/.halo/global/skills/` has no watcher, so a hand edit there shows on the Skills page's window-focus refetch. Also bumped on WS reconnect while the Skills page is open. |
150
150
  | `extension:changed` | `extensions/watcher.ts` `rescanAndBroadcast` (broadcast to all clients) | `~/.halo/global/extensions/` changed (install / upgrade / uninstall / on-off switch, from the admin upload, the one-click update, the `extension` skill, or a manual `mv`/`rm`; a switch rewrites `.state.json`, which trips the same watcher) — carries the **full** `ExtensionsSnapshot` (`{extensions, disabled, errors}`: switched on, switched off, refused), not a diff, and only fires when the snapshot key differs from the last one. The admin replaces its extension list wholesale (a missing `disabled`, from an older server, reads as `[]`): the preview registry re-resolves open tabs from `extensions` only (an upgraded extension remounts, or shows a banner if the tab is dirty; an uninstalled or switched-off one falls back to the "no built-in preview" page, or keeps a dirty tab alive with a banner) and the Settings → Extensions panel re-renders. Global, no workspace marker — extensions are server-wide. See [canvas-extensions.md](canvas-extensions.md). |
151
151
  | `updates:changed` | `updates/core-check.ts` · `updates/extension-check.ts` (broadcast to all clients) | An update check finished, or (extensions scope) a watcher rescan changed a cached verdict — `{scope: 'core' \| 'extensions', status}`, `status` = the same `CoreUpdateStatus` / `ExtensionsUpdateStatus` that `GET /api/updates?scope=…` returns. Checks start only from that GET (cache ≥ 24 h old) or `POST /api/updates/check`, never on a timer. The Settings sidebar foot (`core`) and Settings → Extensions (`extensions`) replace their status with it; nothing else listens. Global, no workspace marker. See [canvas-extensions.md → Updates](canvas-extensions.md#updates-updatejson). |
152
- | `nodes:changed` | `nodes/hub.ts` `announce` (broadcast to all clients) | A `halo node` connected or disconnected — carries the **full** list of online machines, `{online: [{accountId, root, shell, os, since}]}` (`root: null` = whole-machine mode), not a diff. The admin's Channels → Linked machines page replaces its online map wholesale and re-fetches the account list on WS reconnect. Global, no workspace marker (the channels page is a global view). See [node.md](node.md). |
152
+ | `nodes:changed` | `nodes/hub.ts` `announce` (broadcast to all clients) | A `halo-node` connected or disconnected — carries the **full** list of online machines, `{online: [{accountId, root, shell, os, since}]}` (`root: null` = whole-machine mode), not a diff. The admin's Channels → Linked machines page replaces its online map wholesale and re-fetches the account list on WS reconnect. Global, no workspace marker (the channels page is a global view). See [node.md](node.md). |
153
153
 
154
154
  ### `error` frames: optional `code`
155
155
 
@@ -198,6 +198,24 @@ models:
198
198
  - `capabilities.thinking.effortPresets` → renders the Thinking effort dropdown
199
199
  - Omit either block → the UI hides that control for this model
200
200
 
201
+ **What users see in Settings.** A provider's Settings page lists its models read-only below the credential fields ([requirements/settings.md → Models browse](../requirements/settings.md#models-browse-provider-pages)). The view only displays — it changes nothing at runtime — but it is where users compare models, so these fields are user-visible:
202
+
203
+ | yaml field | Shown as |
204
+ |---|---|
205
+ | `runtime`, `revision` | provider line: "Runtime · Revision" |
206
+ | `defaultModelId` | provider line "Default model" + the card's Default tag; absent → the first model |
207
+ | `models[].displayName`, `models[].id` | card title (absent → the id) and the copyable mono id under it |
208
+ | `models[].contextWindow`, `maxContextWindow` | "Context 272K (max 1M)" — the max only when it differs |
209
+ | `models[].compressAt` | tooltip on the context: "Compacts at 90% of the context" |
210
+ | `models[].maxOutputTokens` | "Output 128K" |
211
+ | `capabilities.image` / `video` / `audio` | Image / Video / Audio chips; all three false → "Text only" |
212
+ | `capabilities.thinking` | "Thinking low / … / max" from `effortPresets[].value` with `default` bold; `mode: manual` → "Thinking budget 4K" from `defaultBudgetTokens` (8192 when absent); `defaultEnabled: true` → "· on by default" |
213
+ | `capabilities.promptCaching` | "Cache 5m / 1h" from `ttlPresets[].value`, `default` bold |
214
+ | `capabilities.verbosity` | "Verbosity low / medium / high" from `presets[].value`, `default` bold |
215
+ | `models[].endpoints`, `endpointPresets`, `defaultEndpoint` | endpoint chips via `resolveModelEndpoints` (the default first if the presets omit it): `bedrock-runtime.<region>.amazonaws.com` / `bedrock-mantle.<region>.api.aws` → the region, any other URL → its host; the default highlighted |
216
+
217
+ Chips show preset `value`s, not their `label`s. Sizes read as K / M (a power-of-two size binary: 262144 → 256K). The provider's `displayName` / `description` head the page from the settings schema, and `secrets:` are the form above.
218
+
201
219
  **Context fields** — `contextWindow` is the *recommended* window, written into a new agent's `context.maxTokens`: the official figure, capped at 272000 (1M-class models bill input past 272K at long-context rates); `maxContextWindow` is the model's true limit; `compressAt` the new agent's compaction ratio. Read through `resolveModelContext()` / `resolveEffectiveContext()` in [config.ts](../../../packages/server/src/config.ts). At runtime a session's budget (compaction threshold, token ring) is agent.yaml `context.maxTokens` / `compressAt`, else the global defaults (`config.model.maxContextTokens` 272K capped at the model's max, `general.compact.compress_at` 0.9), then clamped to the model's max — the same chain for live sessions (`buildModelRuntime`, session-agent-builder.ts) and cold ones (`getContextConfig()`, session-manager.ts). The registry's recommended `contextWindow` / `compressAt` and `capabilities.verbosity.default` are **scaffold-only**: written into agent.yaml when an agent is created or the form switches model, never read at runtime for an existing agent. A new agent's compressAt is the explicitly set global `compress_at`, else the model's `compressAt`, else 0.9. Declare all three on every model entry; without `maxContextWindow` / `contextWindow` a small-window model (e.g. 128K) runs at 272K and overflows before compaction kicks in. Details: [design/storage.md#model-registry-format](../design/storage.md#model-registry-format).
202
220
 
203
221
  Shape source: [packages/server/templates/models/aws-bedrock-claude-invoke.yaml](../../../packages/server/templates/models/aws-bedrock-claude-invoke.yaml) and [design/storage.md#model-registry-format](../design/storage.md#model-registry-format).
@@ -121,6 +121,12 @@ Install / update / switch / uninstall don't broadcast themselves — the resulti
121
121
 
122
122
  Types: `packages/core/src/protocol/update-types.ts`.
123
123
 
124
+ **Model sync** (`routes/updates.ts` + `models/hub-update.ts`; cookie; not mounted in AgentCore mode). Backs Settings → Model providers → Sync latest models. Plans and writes with the same `planModelsInstall` / `applyModelsInstall` as `halo models install` — rules: [design/storage.md → Model registry format](../design/storage.md#model-registry-format). No frame of its own: a write trips the `models.d/` watcher, which pushes `models:changed`.
125
+
126
+ | Method | Path | Auth | Purpose |
127
+ |---|---|---|---|
128
+ | POST | `/api/models/update` | cookie | Body `{ confirm?: boolean }` — only `true` counts as consent. The hub is `extension.params.hub_repo` (empty = turmind/halo-hub; `owner/repo` or `https://github.com/<owner>/<repo>` only). Lists its releases (`api.github.com/repos/<owner>/<repo>/releases?per_page=100`, anonymous: **60 requests/hour per IP**, no token is sent), takes the newest non-draft, non-prerelease `models-v<digit>…` release with a `.zip` asset, unpacks its `*.yaml` (basename only, flat) into a temp dir and plans it against the copies in effect. 200 `{hub, release, needsConfirm, written: string[], summary: string \| null, entries}`, one entry per yaml `{file, id?, action: 'install' \| 'up-to-date' \| 'skip' \| 'refuse', reason?, revision?, current?: {source: 'bundled' \| 'hub', revision}, endpointChanges: string[], needsConfirm?}`. `needsConfirm: true` = an install adds an endpoint and `confirm` wasn't `true`: **nothing is written**, `written: []`, `summary: null`; re-POST `{ confirm: true }` fetches and plans again (nothing cached between calls), then writes. Otherwise `written` = the ids written to `models.d/` and `summary` = the CLI's summary line. 400 `{error}` `hub_repo` isn't GitHub (use `/extension models`); 409 `{error: 'A model sync is already running'}` (one at a time, in-process); 502 `{error}` the hub / network is at fault — release API non-200 (rate-limited, private or wrong repo), invalid JSON, no matching release, download failure, not a zip, two yamls with one basename, no yaml in it, over the size cap (list 2 MB, zip 10 MB), timeout / network error; 500 otherwise (e.g. `models.d/` not writable) |
129
+
124
130
  ## Session Logs (unified)
125
131
 
126
132
  File: `packages/server/src/routes/sessions.ts`
@@ -343,7 +349,7 @@ SSE responses (`/api/web/chat`, `/api/web/subscribe`) carry an SSE comment line
343
349
 
344
350
  See [design/web.md](../design/web.md).
345
351
 
346
- ## Linked machines (`halo node`)
352
+ ## Linked machines (`halo-node`)
347
353
 
348
354
  File: `packages/server/src/nodes/routes.ts` (+ `hub.ts` for the WebSocket). Not mounted in AgentCore mode. Design: [design/node.md](../design/node.md).
349
355
 
@@ -930,6 +936,7 @@ Resolves declared schema (from `models/<id>.yaml` `secrets:`, `skills/<id>/confi
930
936
  "description": "...",
931
937
  "description_zh": "...",
932
938
  "default": "...", // schema-default placeholder; omitted when the schema declares none
939
+ "placeholder": "...", // general.* only: example for an empty input, never a fallback value (e.g. outbound_proxy.url)
933
940
  "secret": true, // UI masks input + value
934
941
  "globalOnly": false, // general.* only: workspace overrides ignored, workspace writes 400
935
942
  "restartRequired": false, // general.* only: read once at boot — UI shows a restart notice after saving
@@ -966,7 +973,7 @@ Resolves declared schema (from `models/<id>.yaml` `secrets:`, `skills/<id>/confi
966
973
  { "ok": true }
967
974
  ```
968
975
 
969
- PUT replaces the full scope; PUT / PATCH / DELETE with `scope: "workspace"` reject a `globalOnly` key with 400 (`<key> is global-only and cannot be set per workspace`) and an unresolvable `projectId` with 404 `Project not found`. DELETE takes `{scope, projectId, key}` and removes the key. PUT / PATCH / DELETE fire the server's `onSettingsChange` listeners (e.g. the sandbox hidden-paths reload); a DELETE of an already-absent key is a no-op — `{ ok: true }`, no file write, no notify. The Settings page uses DELETE for both Reset (current scope removed → falls back to lower scope / unset) and orphan Remove.
976
+ PUT replaces the full scope; PUT / PATCH / DELETE with `scope: "workspace"` reject a `globalOnly` key with 400 (`<key> is global-only and cannot be set per workspace`) and an unresolvable `projectId` with 404 `Project not found`. PUT / PATCH also 400 a value outside a field's `min` / `max`, and a `general.outbound_proxy.url` that isn't `http(s)://` or carries `user:pass@` (`general.outbound_proxy.url: credentials are not allowed in the URL — put them in Token (general.outbound_proxy.token)`; the message never echoes the URL), file untouched. DELETE takes `{scope, projectId, key}` and removes the key. PUT / PATCH / DELETE fire the server's `onSettingsChange` listeners (e.g. the sandbox hidden-paths reload); a DELETE of an already-absent key is a no-op — `{ ok: true }`, no file write, no notify. The Settings page uses DELETE for both Reset (current scope removed → falls back to lower scope / unset) and orphan Remove.
970
977
 
971
978
  ### POST `/api/wechat/login/start`
972
979
 
@@ -1107,6 +1114,6 @@ It does not remove session files. The full delete (rows + JSON + archive segment
1107
1114
  | `listener:released` | `sessionId` | Event listener reclaimed — re-send `subscribe` (see above) |
1108
1115
  | `exchange:deleted` | `sessionId`, `deletedIds`, `userOrdinal`, `userRows` | An `exchange:delete` this connection sent landed: ids of every soft-deleted row of the active log, the ordinal of the turn just deleted, and the log's user-turn count (without `taskId`) that ordinal runs over (the sender's own reply only — no full `state:snapshot` follows) |
1109
1116
  | `skills:changed` | — | A skill was created / deleted / toggled, or a workspace skill folder changed on disk — re-fetch skill lists and `/api/commands`. Workspace-scoped changes reach only that workspace's sockets (see [Skills](#skills)) |
1110
- | `nodes:changed` | `online: [{accountId, root \| null, shell, os, since}]` | A `halo node` connected / disconnected — full snapshot of online machines (global broadcast; `root: null` = whole machine). See [Linked machines](#linked-machines-halo-node) |
1117
+ | `nodes:changed` | `online: [{accountId, root \| null, shell, os, since}]` | A `halo-node` connected / disconnected — full snapshot of online machines (global broadcast; `root: null` = whole machine). See [Linked machines](#linked-machines-halo-node) |
1111
1118
 
1112
1119
  Events with `taskId` set belong to sub-agent turns (nested sessions); events without are the root agent's. Session-scoped frames carry `sessionId` (the root session they belong to); the admin routes each one to the tab holding that session, and a frame without one goes to the tab on screen. Full list in [design/ws.md](../design/ws.md).
@@ -323,7 +323,7 @@ Use only for a new public release; the retained details here are not extra prepa
323
323
  5. **npm**: publish from `packages/cli/dist-pub/` **once**, using the intended dist-tag (`--tag alpha` for an alpha, not `latest`). Confirm `npm view @turmind/halo@x.y.z version` and the intended tag via `npm view @turmind/halo dist-tags.alpha` (or `dist-tags.latest` for stable); both must equal the requested version. Registry replication can lag ~2–3 minutes: wait/recheck, never republish. `dist-pub/` is release-only; desktop staging uses `dist-dev/`. Historical reason: a second v1.3.4 publish put `1.3.4-<sha>` on `latest` for ~90s.
324
324
  6. **GitHub**: `gh release create vx.y.z --notes-from-tag` (or the short CHANGELOG summary); use `--prerelease` for an alpha.
325
325
  7. **Windows exe**: `(cd packages/desktop && HALO_STAGE_FULL=1 CI=true pnpm dist:win)`, then `gh release upload vx.y.z "packages/desktop/dist/Halo Setup x.y.z.exe"`. Verify it appears in `gh release view vx.y.z --json assets`; v1.3.0 and v1.3.1 missed this asset. macOS dmg is the user's step unless requested.
326
- 8. **halo-node binaries**: nothing to build by hand. Publishing the release triggers `.github/workflows/node-go.yml`'s release job (`go vet` + `go test`, then `build.sh`), which uploads `halo-node-linux-{amd64,arm64,armv7}`, `halo-node-windows-amd64.exe`, `SHA256SUMS` and `install.sh` (with the version stamped in; it isn't listed in `SHA256SUMS`). Wait for it to go green, then check that all six appear in `gh release view vx.y.z --json assets` — the admin's Linux connect command runs this tag's `install.sh`. If the job failed, rerun it with `gh workflow run node-go.yml -f tag=vx.y.z` (`workflow_dispatch`; `--clobber` makes reruns safe). The trigger needs the release created by a person (`gh release create` as yourself), because a release created with `GITHUB_TOKEN` doesn't start other workflows.
326
+ 8. **halo-node binaries**: nothing to build by hand. Publishing the release triggers `.github/workflows/node-go.yml`'s release job (`go vet` + `go test`, then `build.sh`), which uploads eight assets: `halo-node-linux-amd64`, `halo-node-linux-arm64`, `halo-node-linux-armv7`, `halo-node-darwin-arm64`, `halo-node-darwin-amd64`, `halo-node-windows-amd64.exe`, `SHA256SUMS` (the six binaries) and `install.sh` (with the version stamped in; it isn't listed in `SHA256SUMS`). Wait for it to go green, then check that all eight appear in `gh release view vx.y.z --json assets` — the admin's Linux / macOS connect command runs this tag's `install.sh`, the Windows one downloads this tag's exe. If the job failed, rerun it with `gh workflow run node-go.yml -f tag=vx.y.z` (`workflow_dispatch`; `--clobber` makes reruns safe). The trigger needs the release created by a person (`gh release create` as yourself), because a release created with `GITHUB_TOKEN` doesn't start other workflows.
327
327
 
328
328
  **npm token gotcha**: `npm publish` on this package needs a granular access token created with **"Bypass 2FA"** checked — scope / permission alone yields `403 Two-factor authentication or granular access token with bypass 2fa enabled is required`. `npm whoami` and `npm token list` succeed with a non-bypass token, so neither is a valid pre-flight; check `GET https://registry.npmjs.org/-/npm/v1/tokens` (with the token as bearer) and look for `"bypass_2fa": true` on the token in use before starting a release.
329
329
 
@@ -122,7 +122,8 @@ Source: `packages/server/src/config.ts`
122
122
  | `HALO_DEFAULT_PROVIDER` | (none) | — | Read by `halo setup --non-interactive`: binds every built-in agent to this provider id (and its default model) and, for each of its env-backed keys whose variable is set, writes the `<<NAME>>` placeholder into `settings.yaml` (Bedrock excepted). An unknown id only prints a warning |
123
123
  | `HALO_CLI` | `halo` (`halo.cmd` on Windows) | — | `halo` cli executable the evolution wrapper spawns — a dev override (cron runs in-process since 2.2 and no longer uses it) |
124
124
  | `HALO_VERSION` | `dev` | — | Version string reported by `GET /api/health` and the cli; stamped by the bundle / desktop launcher, so normally not set by hand |
125
- | `HALO_NODE_TOKEN` | (none) | — | `halo node` only (on the linked machine): pairing token, used when `--token` isn't given — preferred over the flag, which shows up in `ps` / shell history. Stripped from child-process env (see [design/node.md](../design/node.md)) |
125
+ | `HALO_NODE_TOKEN` | (none) | — | `halo-node` only (on the linked machine): pairing token, used when `--token` isn't given — preferred over the flag, which shows up in `ps` / shell history. Stripped from child-process env (see [design/node.md](../design/node.md)) |
126
+ | `HTTPS_PROXY` / `HTTP_PROXY` / `NO_PROXY` | (none) | — | `halo-node` only (on the linked machine): forward proxy for an `https://` / `http://` server and the hosts that bypass it (Go `ProxyFromEnvironment`; lowercase names too; `localhost` / loopback never proxied). Ignored when `--proxy <url>` is given — the flag wins (see [guide/node.md](../guide/node.md#through-a-forward-proxy)) |
126
127
  | `HALO_PROMPT` | (none) | — | `plain` makes `halo setup` use numbered-menu prompts instead of the raw-mode TUI |
127
128
  | `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` | (none) | `settings.yaml general.observability.capture_content` | `true` / `false` / `1` / `0` (any case) overrides `capture_content` either way; unset or any other value → the setting. Read at process start (see [design/observability.md](../design/observability.md)) |
128
129
 
@@ -155,7 +156,7 @@ When a provider manifest declares a `default: <<NAME>>` for a secret, `halo setu
155
156
  - `tavily-web-search.params.api_key` — example only; declared by whichever skill needs it
156
157
  - `web-search.params.fast_region` / `web-search.params.deep_region` — AWS regions for web-search's fast gear (Nova grounding, US only; empty = us-east-1) and deep gear (Bedrock Mantle; empty = us-west-2 for luna, us-east-1 for sol, and sol set to a region it isn't offered in falls back to us-east-1)
157
158
  - `extension.params.hub_base` — https base URL holding `<id>/update.json` for every hub extension, read by `/extension install <id>` and the admin's update check (extensions without their own `source`); empty = the GitHub raw `extensions/` directory on `main` of `hub_repo`, and a non-GitHub `hub_repo` needs it set. See [design/canvas-extensions.md → Updates](../design/canvas-extensions.md#updates-updatejson)
158
- - `extension.params.hub_repo` — hub `/extension models` pulls the newest `models-v*` release from, and the source of the default `hub_base` (empty = `https://github.com/turmind/halo-hub`); for `models` the skill's `ext.sh` also reads the optional `GITHUB_TOKEN` / `GITLAB_TOKEN` / `GITEA_TOKEN` from the server environment and sends them to that hub's release API only. See [design/canvas-extensions.md](../design/canvas-extensions.md#extension-skill-and-halo-hub)
159
+ - `extension.params.hub_repo` — hub `/extension models` and Settings → Sync latest models (GitHub hubs only, no token) pull the newest `models-v*` release from, and the source of the default `hub_base` (empty = `https://github.com/turmind/halo-hub`); for `models` the skill's `ext.sh` also reads the optional `GITHUB_TOKEN` / `GITLAB_TOKEN` / `GITEA_TOKEN` from the server environment and sends them to that hub's release API only. See [design/canvas-extensions.md](../design/canvas-extensions.md#extension-skill-and-halo-hub)
159
160
 
160
161
  Hardcoded (config.ts, no override):
161
162
  - `auth.tokenMaxAge` — 14 days
@@ -80,6 +80,8 @@ Run a shell command. Full shell access.
80
80
 
81
81
  Timeout 120 s (`HALO_SHELL_TIMEOUT`). Max output 5 MB. The tool description surfaces the effective timeout to the agent and advises backgrounding (`nohup … &` + log polling) for longer tasks.
82
82
 
83
+ **Outbound proxy.** With Settings → General → `outbound_proxy.enabled` on and a `url` set, every access level's command gets `http_proxy` / `https_proxy` / `HTTP_PROXY` / `HTTPS_PROXY` (credential in the URL's user-info) plus `no_proxy` / `NO_PROXY` (`localhost,127.0.0.1,::1` + the server's own), so curl / git / pip go through it. It is read at each command, no restart. bwrap / Seatbelt sessions get the same vars (in bwrap only the command, not the file tools' helpers). Their network isn't isolated, so a proxy on the host's 127.0.0.1 is reachable. The credential is masked as `***` in the tool result — its URL and base64 forms at any length, the bare token from 4 characters. See [requirements/settings.md → Outbound proxy](../requirements/settings.md#outbound-proxy).
84
+
83
85
  **Server shutdown** (server only — the CLI / TUI never enter shutdown). A command killed by a signal halo didn't send while the server is shutting down (a systemd stop signals the whole cgroup, the command with it) returns `[Command cut off by a server restart — it was still running when the server shut down, so it may or may not have taken effect. Check before re-running it.]` followed by `Output so far:` and the partial stdout (`STDERR:` appended when there is any), instead of `Command failed: …`. A command issued after shutdown began is not run: `Error: command not run — the server is shutting down. Safe to re-issue after the restart if still needed.` Normal exits read as before. See [session.md → Shutdown](../design/session.md#boot-reconcile-of-crash-orphans-reconcileorphansonboot).
84
86
 
85
87
  **Windows output encoding.** The Windows path (`sandbox.ts`) prepends `chcp 65001` so cmd built-ins (`echo`, …) emit UTF-8, then captures raw bytes (`encoding: 'buffer'`) and decodes them strict-UTF-8 with a GBK fallback. This is because native Win32 console tools (`ipconfig`, `systeminfo`, …) ignore `chcp` and still emit the OEM code page (GBK/CP936 on zh-CN); decoding such bytes as UTF-8 produced mojibake. The strict-UTF-8 attempt passes genuine UTF-8 through untouched and only falls back to GBK when the bytes aren't valid UTF-8 (GBK double-byte sequences almost always aren't). mac/Linux are unaffected (UTF-8 throughout).
@@ -120,6 +122,8 @@ HTTP request.
120
122
 
121
123
  Timeout 10 s. Max body 50 KB (truncated if larger). Returns status + content-type + response body.
122
124
 
125
+ With the outbound proxy on (Settings → General → `outbound_proxy.*`), the request goes through it, the credential sent as `Proxy-Authorization`. Hosts in no_proxy (`localhost` / `127.0.0.1` / `::1` + the server's `NO_PROXY`, matched like curl: domain + subdomains, IPs, CIDRs) connect directly, decided per redirect hop. Model provider calls are not affected. A caller-supplied `Proxy-Authorization` header (any case) is refused with a tool error while the proxy is on, no_proxy targets included.
126
+
123
127
  ## Security
124
128
 
125
129
  ### Access level (per-session, dynamic)
@@ -379,7 +383,7 @@ Returns `{ "code": 0, "workspace": "<realpath>", "session_id": "…", "state": "
379
383
  <target's finalOutput || output>
380
384
  ```
381
385
 
382
- Body capped at `limits.autoReportMax` (head-kept) with a `[Report truncated: N chars total. Use relay_read("<ws>", "<id>") for the full text.]` marker. A turn killed by an unrecoverable error carries `· status: aborted]` in the header and is prefixed `[RELAY TARGET ABORTED: … Error: <text> … Re-send with relay_send to let it resume.]` **before** the cap so it can't be sliced off. When the error is an account problem (401/402/403, bad key, no balance), the closing hint instead says it is a model account / credential / balance / permission problem in the target workspace: retrying or re-sending fails the same way, so fix its model configuration (or tell the user to) and do not resume it with `relay_send` until then. For a model refusal (the target's history tripped the safety classifier) it says not to resume it — `relay_send` to a NEW session id with a clean brief, rewording the part that tripped it. `reply_to` is cleared **before** sending: one dispatch → one report, a failed delivery can't double-fire on the next turn end, and a user chatting directly in the department workspace afterwards never pings the secretary.
386
+ Body capped at `limits.autoReportMax` (head-kept) with a `[Report truncated: N chars total. Use relay_read("<ws>", "<id>") for the full text.]` marker. A turn killed by an unrecoverable error carries `· status: aborted]` in the header and is prefixed `[RELAY TARGET ABORTED: … Error: <text> … Re-send with relay_send to let it resume.]` **before** the cap so it can't be sliced off. When the error is an account problem (401/402/403, bad key, no balance), the closing hint instead says it is a model account / credential / balance / permission problem in the target workspace: retrying or re-sending fails the same way, so fix its model configuration (or tell the user to) and do not resume it with `relay_send` until then. For a model refusal (the target's history tripped the provider's safety / content filter) it says not to resume it — `relay_send` to a NEW session id with a clean brief, rewording the part that tripped it. `reply_to` is cleared **before** sending: one dispatch → one report, a failed delivery can't double-fire on the next turn end, and a user chatting directly in the department workspace afterwards never pings the secretary.
383
387
 
384
388
  **Interim report**: if a busy target answers a follow-up `relay_send` in a turn that another turn follows — it called `continue_task` to resume what it was doing, or a further relay message was already queued when the answer ended (also on the opening turn), or the target was waiting on its own sub-agents when asked and the run ended on the answer with a sub-agent still running — the answer is delivered right away as
385
389
 
@@ -530,7 +534,7 @@ Returns `{ "code": 0, "remotes": [{ name, agent, description, skills }], "pendin
530
534
 
531
535
  File: `packages/server/src/nodes/tools.ts` (`buildNodeTools`). Design notes in [design/node.md](../design/node.md); user guide [guide/node.md](../guide/node.md).
532
536
 
533
- Act on the user's **linked machines** — computers running `halo node`, paired with this workspace in Admin → Channels → Linked machines. Each node tool mirrors one local workspace tool and runs **that same tool** on the machine (same arguments, same output), plus a required `node` argument naming the machine:
537
+ Act on the user's **linked machines** — computers running `halo-node`, paired with this workspace in Admin → Channels → Linked machines. Each node tool mirrors one local workspace tool and runs **that same tool** on the machine (same arguments, same output), plus a required `node` argument naming the machine:
534
538
 
535
539
  | Tool | Mirrors | Extra / different args |
536
540
  |---|---|---|
@@ -547,17 +551,17 @@ Act on the user's **linked machines** — computers running `halo node`, paired
547
551
 
548
552
  **Paths**: a machine shares one folder (`--root`; `root` in `node_list`) — relative paths resolve against it, paths that resolve outside it (symlinks included) are refused: `"<p>" is outside this machine's shared folder (<root>) — node tools can only reach paths inside it`. A whole-machine node (`root: null, wholeMachine: true`) has no such check; relative paths resolve against its `base`.
549
553
 
550
- **Runtime**: in `halo server` (cron runs included) calls go over the machine's WebSocket. In the standalone CLI / TUI, the evo sandbox and AgentCore mode the tools are listed but every one — `node_list` included — returns `linked machines are not available in this runtime — only \`halo server\` accepts \`halo node\` connections. This is permanent for this runtime, not a temporary outage: do not retry. …`.
554
+ **Runtime**: in `halo server` (cron runs included) calls go over the machine's WebSocket. In the standalone CLI / TUI, the evo sandbox and AgentCore mode the tools are listed but every one — `node_list` included — returns `linked machines are not available in this runtime — only \`halo server\` accepts \`halo-node\` connections. This is permanent for this runtime, not a temporary outage: do not retry. …`.
551
555
 
552
556
  **Errors** (returned as the tool error text):
553
557
 
554
558
  | Case | Message |
555
559
  |---|---|
556
- | Machine linked but not connected | `machine "<n>" is offline (\`halo node\` is not connected). Do not retry in a loop — tell the user to start it.` |
560
+ | Machine linked but not connected | `machine "<n>" is offline (\`halo-node\` is not connected). Do not retry in a loop — tell the user to start it.` |
557
561
  | Name not linked to this workspace | `unknown machine "<n>" — machines linked to this workspace: <a, b \| (none)> (see node_list)` |
558
562
  | Link dropped mid-call | `machine "<n>" went offline during the call (connection lost). The operation may or may not have completed on the machine — check its state before retrying.` |
559
563
  | No answer in time (120 s; node_run: shell timeout + 15 s) | `machine "<n>" did not answer within <s>s — the call was cancelled; it may or may not have taken effect.` |
560
- | node_run without `--allow-shell` | `node_run is disabled on this machine — the user must restart \`halo node\` with --allow-shell to allow commands` |
564
+ | node_run without `--allow-shell` | `node_run is disabled on this machine — the user must restart \`halo-node\` with --allow-shell to allow commands` |
561
565
  | Request over 8 MB | `request too large to send to the machine (<x> MB > 8 MB limit)` |
562
566
 
563
567
  Results over 256K chars are cut on the machine (`[Content truncated on the machine: N chars total]`); the usual tool-result budget then applies.
@@ -571,7 +575,7 @@ Returns `{ "code": 0, "nodes": [...] }`, one entry per linked machine:
571
575
  - offline: `{ "name": "nas", "online": false }`
572
576
  - online: `{ "name": "mbp", "online": true, "root": "/Users/a/proj", "shell": true, "shellTimeoutSec": 120, "os": "darwin", "since": "<ISO>" }` — whole-machine: `"root": null, "wholeMachine": true, "base": "<start dir>"`; `shellTimeoutSec` only when `shell` is true.
573
577
 
574
- No machines at all → `"nodes": []` plus `"hint": "No machines linked — create a token in Admin → Channels → Linked machines, then run \`halo node\` on the machine."`.
578
+ No machines at all → `"nodes": []` plus `"hint": "No machines linked — create a token in Admin → Channels → Linked machines, then run \`halo-node\` on the machine."`.
575
579
 
576
580
  ### node_run
577
581
 
@@ -26,7 +26,7 @@ Aimed at Halo end-users. Pick the doc that matches your situation.
26
26
 
27
27
  ## Linked machines
28
28
 
29
- [node.md](node.md) — run `halo node` on your own computer so a workspace's agents can work on files there.
29
+ [node.md](node.md) — run `halo-node` on your own computer so a workspace's agents can work on files there.
30
30
 
31
31
  ## Related docs
32
32
 
@@ -42,7 +42,7 @@ Click an agent in the sidebar to open the right-hand editor. Two views:
42
42
  Data-driven form for name / description / model / tools / skills / thinking. Changes auto-save about half a second after you stop editing — there's no Save button. The agent's AGENT.md is shown below the form as a read-only preview.
43
43
 
44
44
  **Model section** has four fields:
45
- - `provider` — dropdown populated from `~/.halo/global/models/*.yaml` plus hub-installed `~/.halo/global/models.d/*.yaml` (`/extension models`; updates the list live)
45
+ - `provider` — dropdown populated from `~/.halo/global/models/*.yaml` plus hub-installed `~/.halo/global/models.d/*.yaml` (`/extension models`; updates the list live — Settings → Model providers → Sync latest models does the same in one click, GitHub hubs only)
46
46
  - `id` — combobox (input + datalist); preset options filtered by current provider, also accepts manual input of any model ID
47
47
  - `endpoint` — combobox (input + datalist); full endpoint URL (e.g. `https://bedrock-runtime.ap-northeast-1.amazonaws.com`), supports custom proxy URLs. When a model is only served in some regions the list narrows to those, switching to it moves the endpoint there, and an endpoint off its list shows a warning
48
48
  - `maxTokens` — optional; defaults to `maxOutputTokens` in the provider yaml
@@ -227,6 +227,6 @@ With `export NANO_BANANA_KEY=sk-xxx` in the env, the agent's system prompt gets
227
227
 
228
228
  ## Model registry — adding a new provider
229
229
 
230
- Halo ships 13 providers (`aws-bedrock-claude-invoke`, `aws-bedrock-mantle`, `aws-bedrock-openai`, `anthropic`, `openai`, `kimi`, `deepseek`, `minimax`, `qwen`, `hunyuan`, `doubao`, `zhipu`, `mimo-token-plan-china`); each one is a yaml under `~/.halo/global/models/` (refreshed between releases by `/extension models`, which installs newer copies into `~/.halo/global/models.d/`) whose `runtime:` field names the implementation in [packages/server/src/agents/model-runtime.ts](../../../packages/server/src/agents/model-runtime.ts) (e.g. `anthropic-messages`, `openai-chat`, `bedrock-invoke`). For any OpenAI- or Anthropic-compatible endpoint you don't need new code: pick `openai` or `anthropic` as the provider and fill in the endpoint and model id in Form view.
230
+ Halo ships 13 providers (`aws-bedrock-claude-invoke`, `aws-bedrock-mantle`, `aws-bedrock-openai`, `anthropic`, `openai`, `kimi`, `deepseek`, `minimax`, `qwen`, `hunyuan`, `doubao`, `zhipu`, `mimo-token-plan-china`); each one is a yaml under `~/.halo/global/models/` (refreshed between releases by `/extension models`, which installs newer copies into `~/.halo/global/models.d/`; Settings → Model providers → Sync latest models does it in one click, GitHub hubs only) whose `runtime:` field names the implementation in [packages/server/src/agents/model-runtime.ts](../../../packages/server/src/agents/model-runtime.ts) (e.g. `anthropic-messages`, `openai-chat`, `bedrock-invoke`). For any OpenAI- or Anthropic-compatible endpoint you don't need new code: pick `openai` or `anthropic` as the provider and fill in the endpoint and model id in Form view. To see what a provider offers, open Settings → Model providers → that provider: below its API key fields it lists every model with its context / output size, capabilities and the regions (AWS) or hosts it can be called on — read-only.
231
231
 
232
232
  A provider that speaks an existing wire protocol needs only a yaml manifest with that `runtime:`; a new protocol also needs a runtime class. The step-by-step is in [dev/add-model-provider.md](../dev/add-model-provider.md). A provider yaml without `runtime:` (or with a name Halo doesn't know) makes session spawn fail with an error naming the yaml; for a bundled provider that means the seed is stale — run `halo setup` or restart the Halo server.