@turmind/halo 1.5.11 → 1.6.0

package/bundled-docs/design/agentcore.md

@@ -1,135 +1,180 @@
 # AgentCore Runtime Mode
 
-Halo server as an **Amazon Bedrock AgentCore Runtime** container — a fourth
-way to run halo (server / CLI / desktop / AgentCore). One env var flips it:
-`HALO_RUNTIME_MODE=agentcore`. Demo package: `packages/agentcore-demo/`
-(Dockerfile, chat frontend, CDK stack, auth Lambdas — its README carries the
-operational gotchas; this doc covers how the mode works inside the server).
+Halo server as an **Amazon Bedrock AgentCore Runtime** container speaking the
+**A2A protocol** (AgentCore `serverProtocol: A2A`) — a fourth way to run halo
+(server / CLI / desktop / AgentCore). `halo agentcore --workspace <path>` sets
+`HALO_RUNTIME_MODE=agentcore-a2a` and runs the server in the foreground; the
+implementation is `packages/server/src/a2a/agentcore.ts`. The repo ships no
+deploy package (Dockerfile / IaC): you build the image and create the runtime
+yourself. The former HTTP-protocol mode (`/invocations`, WS `/ws`, per-user
+workspaces) and its demo package were removed 2026-10-09 (git history `7e98369`).
 
 ```
-Browser (static frontend, S3 + CloudFront)
-   │  /api/login, /api/verify  → CloudFront /api/* behavior → API GW → auth Lambda (DDB users, JWT)
-   │  /api/ws-presign          → presign Lambda (SigV4-signs the AgentCore WS URL)
-   │  wss (presigned URL, 5-min TTL)
+Caller (any halo server, or any A2A client that SigV4-signs)
+   │  POST …/runtimes/<escaped ARN>/invocations/   (A2A JSON-RPC, SigV4)
+   │  GET  …/invocations/.well-known/agent-card.json
+   │  X-Amzn-Bedrock-AgentCore-Runtime-Session-Id = fixed id
    ▼
-AgentCore Runtime (per-session microVM, auth terminated here)
-   │  X-Amzn-Bedrock-AgentCore-Runtime-Session-Id = user UUID
+AgentCore Runtime (per-session microVM, verifies the signed request)
    ▼
-halo server :8080  (HALO_RUNTIME_MODE=agentcore)
-   ├── GET  /ping          Healthy | HealthyBusy (any running agent session)
-   ├── POST /invocations   {"input":{"prompt"}} → full assistant message
-   └── WS   /ws            streaming frames (primary path)
+halo agentcore :9000  (HALO_RUNTIME_MODE=agentcore-a2a)
+   ├── GET  /ping                          Healthy | HealthyBusy
+   ├── POST /                              A2A JSON-RPC
+   └── GET  /.well-known/agent-card.json   card (no securitySchemes)
+        one fixed workspace (EFS) — <ws>/.halo/{a2a.db, runs.db, agentcore.lease}
 ```
 
+## Running it
+
+`halo agentcore [-w <path>] [-p N]`; the workspace is required and must exist.
+
+`HALO_HOME` must be initialized first: run `halo setup --non-interactive` in
+the image build (seeds `~/.halo/global` — agents, skills, models). `halo
+agentcore` skips the CLI's setup gate, but the server still exits at boot with
+"~/.halo/global/ not initialized" without it. Edits to the seeded files (e.g.
+pointing the agents' Bedrock endpoint at the runtime's region) go after it.
+
+| Setting | Meaning |
+|---|---|
+| `-w, --workspace` / `HALO_WORKSPACE` | The one workspace served — an EFS mount in practice. |
+| `-p, --port` | Listen port, default `9000` (the AgentCore contract). Always overrides `HALO_PORT`. |
+| `HALO_A2A_PUBLIC_URL` | The runtime's invoke URL (`…/runtimes/<escaped ARN>/invocations`); becomes the card's interface URL (`<url>/`). Unset → warning, and the card advertises the container's own origin. |
+| `HALO_A2A_ACCESS` | Access level of every session: `workspace` (default) \| `full` \| `readonly`. Anything else is a boot error. |
+| `HALO_LOG_LEVEL=info` | Needed for CloudWatch — see the ops notes. |
+
 ## What the mode changes (packages/server/src/index.ts)
 
-`config.server.runtimeMode === 'agentcore'` (env `HALO_RUNTIME_MODE`) skips:
-
-- **Password/JWT gate** — no admin password needed at startup, `/ws` skips the
-  cookie check. AgentCore terminates auth upstream (SigV4 presign / OAuth);
-  the container is only reachable through the runtime.
-- **Single-instance lock** — one microVM per session; many server processes
-  coexist by design.
-- **Channels, cron, evolution, archive daemon, extensions watcher** —
-  meaningless in an ephemeral per-session microVM; sessions are driven only
-  through the AgentCore surface.
-
-Everything else (agent loop, tools, skills, sqlite persistence) is the normal
-server. The adapter itself is `packages/server/src/routes/agentcore.ts`,
-mounted at the root (the AgentCore contract paths live outside `/api`).
-
-## Per-user workspace isolation
-
-`userWorkspace(base, runtimeSessionId)` maps each runtime session id to
-`<HALO_WORKSPACE>/users/<sanitized-id>/` — an isolated workspace with its own
-`.halo/` (sqlite + session files). The id chain:
-
-DDB `agentcore-demo-users` UUID → login response → frontend uses it as
-sessionId → presign Lambda signs it into
-`X-Amzn-Bedrock-AgentCore-Runtime-Session-Id` → adapter routes the workspace.
-
-So **1 user = 1 runtime session id = 1 workspace**: users never see each
-other's history; reconnecting with the same id resumes the same conversation.
-`HALO_WORKSPACE` points at an EFS mount (`/mnt/workspace` in the CDK stack,
-`/mnt/efs` by the Dockerfile default) — microVMs are ephemeral, EFS makes
-the workspaces survive session termination and image rollouts.
-`~/.halo/global/runs.db` (the [run ledger](session.md#run-ledger--restart-nudge-for-interrupted-roots-halo-globalrunsdb)) lives under the container's `HALO_HOME`, which is **not** on EFS — it dies with the microVM, so the restart nudge doesn't apply in agentcore mode (the writes themselves are harmless, just moot).
-
-## WS protocol quirks (why /init exists)
-
-The AgentCore WS proxy only forwards **client frames containing `inputText`**
-and cannot push server frames on connect. Hence:
-
-- Frontend sends `{"inputText":"/init"}` after open; the server special-cases
-  it (also accepts `{"type":"init"}`) and replies with a `history` frame —
-  a full snapshot, which the frontend renders by rebuild-from-scratch (not
-  append), keyed by a signature so identical snapshots skip re-rendering.
-- Frames with no `inputText` text and no `imageRefs` are silently ignored
-  server-side — the frontend uses `{type:'ping'}` every 30s purely to keep
-  the proxy from cutting the socket.
-- `/session switch` sends a `{type:'switch'}` frame → frontend clears, then
-  the follow-up history frame rebuilds.
-
-Slash commands run through the shared `dispatchCommand` with a module-level
-`activeOverrides` map keyed by **`runtimeSessionId`** (one entry per runtime
-session, holding "which halo session is this socket's current one"). The entry
-lives only as long as the socket: `ws.on('close')` deletes it alongside the event
-listener, since a reconnect re-resolves from the session id anyway — without the
-delete the map grew one permanent entry per runtime session for the container's
-whole lifetime (audit B-L3). Every command is `accessLevel: 'full'` by design:
-auth terminated upstream at AgentCore, so whoever reached this socket already
-owns the isolated per-user workspace behind it.
-
-## WS image upload protocol (chunking)
-
-The AgentCore WS proxy hard-caps a single frame at **64KB** — send more and
-the connection is cut with close code 1009 ("Policy violated: message size
-limit of 64 KB for a message frame is exceeded", verified against the Tokyo
-prod runtime). An inline base64 image blows past that instantly, so images
-ride as chunked `image_chunk` frames, assembled server-side, then referenced
-by id from the actual message frame:
-
-```jsonc
-// 1..N chunk frames per image (inputText empty string is required — the
-// proxy only forwards client frames that carry an inputText key at all)
-{"inputText":"", "type":"image_chunk", "uploadId":"u1", "seq":0, "total":8,
- "mimeType":"image/jpeg", "data":"<base64 slice, ≤48KB>"}
-
-// message frame referencing the finished upload(s)
-{"inputText":"describe this", "imageRefs":["u1"]}
+`config.server.runtimeMode === 'agentcore-a2a'` skips:
+
+- **Password/JWT gate** and the **single-instance lock** — AgentCore verifies
+  the signed request before forwarding; many microVMs coexist by design.
+- **Channels, cron, evolution, archive daemon, extensions watcher, admin WS
+  and transcribe proxy** (every WS upgrade gets the router's 400).
+- **The whole `/api/*` surface** (auth middleware included) **and the admin
+  static bundle / SPA fallback.** The container answers only `/ping`, `POST /`
+  and the card (the only paths AgentCore passes through); everything else is a
+  plain JSON 404, no HTML.
+- The normal `/a2a/<path>` mount and the `/a2a-push` receiver. There is no
+  outbound A2A from inside the container (`a2a_send` reports itself
+  unavailable): a remote's webhook can't reach a microVM.
+
+Everything else (agent loop, tools, skills, sqlite) is the normal server. The
+card is `<ws>/.halo/agent-card.json`, seeded (name `Halo`, `skills: []`) if
+missing, never overwritten — edit it to describe the agent.
+
+## Workspace and lease
+
+Sessions live in the workspace's `.halo/` and A2A task state in
+`<ws>/.halo/a2a.db`, so tasks survive microVM recycling. `a2a.db` holds push
+tokens, so it (and `-wal` / `-shm`) is hidden from non-full sessions
+(`tools/sandbox.ts`).
+
+Each runtime session id gets its own microVM, all mounting the same EFS, so a
+single-writer lease guards it — `<ws>/.halo/agentcore.lease`, plain NFSv4 file
+ops only:
+
+- **Activation is lazy.** AgentCore boots microVMs ahead of time (a warm pool)
+  and mounts the filesystem only when a session is assigned, at its first
+  invocation — at boot `HALO_WORKSPACE` is still the image's empty dir. So
+  nothing touches the workspace at boot: the first request other than `/ping`
+  creates `.halo/` and seeds the card.
+- **Acquire on request.** Each `POST /` makes one acquisition attempt unless the
+  lease is held (concurrent POSTs share one attempt). There is no background
+  retry: a refused microVM never polls, so only the microVM callers are actually
+  talking to competes. AgentCore can leave microVMs it no longer routes to
+  (see gotchas), and a polling one would take the workspace away from the
+  fixed session id.
+- The holder rewrites a heartbeat every **10 s**; a lease older than **45 s**
+  is stale and taken over by the next request (rename, then a 2 s inline
+  recheck that it still names us). The owner is a per-process random id, so a
+  successor takes over after release or staleness, whatever its session id.
+- `a2a.db`, the SessionManager and the push sender start only once the lease
+  is held; `runtime.lock` is then deleted (its pid probe can't see other microVMs).
+- Without the lease: `POST /` answers JSON-RPC `-32603` "workspace is in use by
+  another runtime session — call with the fixed runtime session id", and
+  `/ping` reports `Healthy` (never busy) so AgentCore can reap that microVM.
+- **Handoff takes ~45 s in practice, not a graceful release.** Graceful stop
+  does try to delete the lease, but on AgentCore the EFS mount is usually
+  already detached when SIGTERM arrives (`StopRuntimeSession`, idle reap). The
+  next session — the same id too, which comes back on a fresh microVM — waits
+  out the stale window and gets `-32603` meanwhile. Measured: 42–47 s
+  typically, 1.6 s once when the release did land.
+- On **losing** the lease (heartbeat stalled past the stale window and someone
+  took over) the process exits immediately, skipping the graceful flush that
+  would write into the new holder's workspace.
+
+### Resume after a mid-task kill
+
+The [run ledger](session.md#run-ledger--restart-nudge-for-interrupted-roots-haloglobalrunsdb) lives on the workspace in this mode — `<ws>/.halo/runs.db` (hidden from non-full sessions, like `a2a.db`), not the container's `HALO_HOME`, which dies with the microVM. So a task cut off mid-run (the 8 h `maxLifetime`, a crash, a recycle) resumes on the next microVM:
+
+- `onAcquired` opens the ledger (`setRunsDb(createRunsDb(<ws>/.halo))`) **before** `registry.getOrCreate`; `index.ts` opens none in this mode. The SessionManager constructor then runs the server's boot chain — claim `runtime.lock`, orphan reconcile, goal sweep, run-ledger sweep — which drains the rows the dead microVM left and nudges each interrupted root.
+- The `POST /` that acquired the lease waits for those nudges to reach their sessions (`nudgesSettled`) before it is handled, so a GetTask on the interrupted task sees its root running → `WORKING`, not idle.
+- Nothing wakes a new microVM by itself: the resume happens on the **next invocation** — the caller's `a2a_read` / GetTask, or a new send. No caller-side polling.
+- `ownsRuntimes` stays `false`, so the lazy FAILED in `routes.ts` `reconcileStale` remains the fallback for a task the sweep didn't resume (root idle, quiet subtree): "Interrupted by a server restart on the remote. Send again on the same context to resume."
+- Verified locally: `kill -9` mid-task (2 of 6 `sleep 30` calls done) → new process refused for ~45 s, then the first accepted GetTask read `WORKING` and the task completed with all six results.
+
+## Access model
+
+"Upstream" strategy: AgentCore verifies the caller's SigV4 before forwarding,
+so the container does no token auth and the card carries no `securitySchemes`.
+Every call is one fixed caller (`accountId: agentcore`) on the fixed workspace,
+mounted at `/` (any sub-path is 404), with the session level from
+`HALO_A2A_ACCESS` — default `workspace`, not `full`. See
+[guide/delegation-and-access.md](../guide/delegation-and-access.md).
+
+## Caller side (any halo server)
+
+In the caller server's `~/.halo/secrets/a2a-remotes.yaml` (server-wide, full-only):
+
+```yaml
+remotes:
+  agentcore:
+    card: https://bedrock-agentcore.ap-northeast-1.amazonaws.com/runtimes/<escaped ARN>/invocations/.well-known/agent-card.json
+    auth: sigv4
+    push_base: http://<caller's VPC-reachable host>:<port>
 ```
 
-Limits (all enforced in `acceptChunk` / `claimImageRefs`): ≤4 images per
-message, ≤8MB of base64 per image, ≤4 concurrent pending uploads per
-connection, ≤256 chunks per upload. Assembly state (`Map<uploadId,
-PendingUpload>`) is **per-connection, in-memory only** — a dropped socket
-discards any partial uploads and the client just re-sends; there is no
-cross-reconnect persistence by design. The server doesn't ack individual
-chunks (that'd add N round-trips for no benefit since the proxy forwards
-mid-stream frames fine) — it only replies with an `error` frame when a chunk
-or `imageRefs` fails validation.
-
-On a completed reference, the image is decoded and persisted via
-`saveInboundMedia` into the user's workspace — the same storage path
-telegram uses — and the message text gets a `[图片已保存: <path>]` marker
-appended so history replay can show the image was there without re-sending
-the bytes.
-
-## Session lifecycle (the part everyone gets wrong)
-
-- Idle timeout (`idleRuntimeSessionTimeout`) terminates a session whose
-  `/ping` reports `Healthy`; `maxLifetime` (8h) force-terminates even busy
-  ones; failed health checks kill immediately.
-- **An open WebSocket = an in-flight invocation** — the session never counts
-  as idle while a socket is open. The 30s frontend keepalive therefore keeps
-  the whole session warm; effective idle timeout ≈ "after the tab closes".
-- `/ping` returns `HealthyBusy` whenever any agent session is running
-  (`registry.list().some(({ sm }) => sm.hasRunningSessions())`) — the official
-  keep-alive for long tool chains with no open connection.
-- Termination is cheap: data is on EFS, next connect cold-starts (~3s) and
-  history reloads. `stop-runtime-session` kills one session on demand; there
-  is **no list/get-runtime-sessions API** (observe via CloudWatch `Sessions`
-  metric + runtime log filtering).
+- `auth: sigv4` signs the card GET and every RPC (service `bedrock-agentcore`,
+  region from the host name, SDK default credential chain) — `a2a/outbound.ts`.
+- It sends a fixed `X-Amzn-Bedrock-AgentCore-Runtime-Session-Id` =
+  `halo-` + sha256(caller workspace realpath + remote name), 69 chars: the same
+  (workspace, remote) always lands on the same microVM, so card, RPCs and
+  follow-ups share it.
+- A `-32054` "retry" error (AgentCore's RetryableConflict — the session's
+  microVM is being provisioned or torn down) is retried with backoff
+  0.5 → 1 → 2 → 4 → 8 s, re-signed each time; the plain ConflictException
+  shares the code but isn't retried.
+- `push_base` is where the container posts task results (the webhook is
+  `<push_base>/a2a-push/<id>`); default is the caller's own public URL, which
+  must be reachable from the runtime's network. A failed push is retried with
+  backoff and given up after 10 attempts or **1 h** of age, whichever comes
+  first (the age cap is 24 h elsewhere), because pending pushes keep `/ping`
+  busy.
+- The container checks the push URL against `general.a2a.url_allowlist` in its
+  own settings.yaml (default `100.64.0.0/10,*.ts.net`): a private-IP or
+  plain-http `push_base` is refused unless listed there.
+- Only `halo server` calls out: `a2a_send` is available at every access level
+  except the A2A read-only profile, and unavailable inside the AgentCore
+  container itself and in CLI/TUI.
+
+## Session lifecycle
+
+- `/ping` returns `HealthyBusy` while the lease is held and an agent session is
+  running (`hasRunningSessions()`) or a push is pending — the keep-alive for
+  long tool chains with no open request.
+- Idle timeout (`idleRuntimeSessionTimeout`, **60 s**) terminates a session
+  whose `/ping` reports `Healthy` — the microVM exits on its own once idle (no
+  self-stop, no extra IAM). So the 8 h `maxLifetime`, which force-terminates
+  even busy ones, bounds one busy stretch rather than accumulating across
+  tasks; a task cut off by it resumes (see above). Failed health checks kill
+  immediately.
+- Termination is cheap: state is on EFS; the next call cold-starts a microVM
+  and takes the lease once the old one is stale (~45 s, see above). A call in
+  that gap gets the lease-busy `-32603`; the caller side (`a2a/outbound.ts`)
+  retries it at 5 / 10 / 15 / 20 s, matched on the code plus the `LEASE_BUSY`
+  message prefix — keep that text stable.
+  There is **no list/get-runtime-sessions API** (observe via the CloudWatch
+  `Sessions` metric + runtime log filtering).
 
 ## Ops crib sheet
 
@@ -142,8 +187,35 @@ the bytes.
   newer runtimes).
 - VPC mode has no public IP: private subnets need `0.0.0.0/0 → NAT` or all
   egress (Bedrock included) hangs → opaque 502s.
-- Full operational detail + deploy walkthrough:
-  `packages/agentcore-demo/README.md`.
+
+## Gotchas
+
+Found deploying `halo_a2a` (ap-northeast-1, 2026-10-09):
+
+- **EFS is mounted only at a session's first invocation**, not at microVM
+  boot (warm pool) — hence lazy activation. Anything touching the workspace
+  at startup writes into the image's empty dir (first card fetch: 500, ENOENT).
+- **StopRuntimeSession stops one microVM.** Concurrent first calls on a new
+  session id can land on several; the others linger until the idle timeout (60 s).
+  Harmless now that refused microVMs don't poll the lease.
+- **EFS is gone by SIGTERM** — graceful lease release usually can't land
+  (`missing` / `Unknown system error -512`); handoff = stale window.
+- **curl `--aws-sigv4` can't sign the ARN-form invoke URL** (curl 8.5 doesn't
+  double-encode the escaped `%3A` / `%2F` in the canonical path → 403). Use
+  the id form `…/runtimes/<runtime-id>/invocations?accountId=<acct>` for
+  JSON-RPC (`accountId` required). The card is served only on the ARN form,
+  so sign that with botocore or the server's own `sigv4Headers` (smithy), which
+  both get it right.
+- **aws-cli `create-agent-runtime` has no `metadataConfiguration`** (2.36);
+  only `update-agent-runtime` takes `requireMMDSV2` — create, then update.
+- **`pnpm deploy --prod` rewrites the repo's
+  `node_modules/.pnpm-workspace-state-v1.json`** as a prod-only install; the
+  next `pnpm <script>` in that checkout then tries `install --production` and
+  aborts without a TTY. Back the file up around `pnpm deploy` (or run builds
+  with `pnpm_config_verify_deps_before_run=warn` — `pnpm_config_*`, not
+  `npm_config_*`).
+- `-32054` RetryableConflict never showed up in practice (concurrent cold
+  first calls, bursts after a stop); the caller retry stays as cheap insurance.
 
 ## Observability
 

package/bundled-docs/design/architecture.md

@@ -41,7 +41,7 @@
 │    /api/wecom/*       (wecom descriptor)                            │
 │    /ws                → setupWebSocketHandler({wss,registry})       │
 │    /*                 → static frontend (Next.js out/)              │
-│    /ping, /invocations → createAgentCoreRoutes() (agentcore mode)   │
+│    /ping, POST / (A2A) → a2a/agentcore.ts (agentcore-a2a mode)      │
 └─────────────────────────────────────────────────────────────────────┘
 ```
 
@@ -188,7 +188,7 @@ File: `ws/terminal-manager.ts`. Spawns a shell via node-pty. On disconnect, deta
 Files: `evolution/{ticker,evo-wrapper,enqueue,spawn,archive}.ts`, `db/evo-db.ts`, `routes/evolution.ts`. Internal agents `__evo_agent__` / `__score__` / `__apply_agent__` (in `templates/agents/`) drive a 12-phase orchestration: snapshot → evo drafts → wrapper dry-runs → scorer grades → reviewer approves → apply agent merges → wrapper history-snapshots + cps to main. Per-task wrapper Node child process owns all sub-cli calls; ticker is stateless and lives in the server. State in `~/.halo/global/evo.db`. See [evolution.md](evolution.md).
 
 ### Run ledger — restart nudge for interrupted roots
-Files: `agents/run-ledger.ts` (sweep + skip rules) + `db/runs-db.ts` (the global `~/.halo/global/runs.db` table `running_sessions`, same singleton pattern as `evo.db`/`cron.db`). `runSession` inserts its session id on entry and deletes it in the finally (server-only, gated on `reconcileOrphansOnBoot`), so whatever is left at boot is exactly what the previous process was mid-run on; the sweep drains those rows and nudges each interrupted root. `index.ts` runs this eagerly: right after constructing the `SessionManagerRegistry` and before `bootChannels`, it loops `listRunningWorkspaces()` and claims+builds a `SessionManager` for each one that isn't already owned by another live server, rather than waiting for someone to open the workspace. See [design/session.md](session.md#run-ledger--restart-nudge-for-interrupted-roots-halo-globalrunsdb).
+Files: `agents/run-ledger.ts` (sweep + skip rules) + `db/runs-db.ts` (the global `~/.halo/global/runs.db` table `running_sessions`, same singleton pattern as `evo.db`/`cron.db`). `runSession` inserts its session id on entry and deletes it in the finally (server-only, gated on `reconcileOrphansOnBoot`), so whatever is left at boot is exactly what the previous process was mid-run on; the sweep drains those rows and nudges each interrupted root. `index.ts` runs this eagerly: right after constructing the `SessionManagerRegistry` and before `bootChannels`, it loops `listRunningWorkspaces()` and claims+builds a `SessionManager` for each one that isn't already owned by another live server, rather than waiting for someone to open the workspace. See [design/session.md](session.md#run-ledger--restart-nudge-for-interrupted-roots-haloglobalrunsdb).
 
 ## REST routes
 

package/bundled-docs/design/canvas-extensions.md

@@ -31,7 +31,7 @@ The admin editor's preview set was compile-time: eight plugins registered in `pr
 | `extensions` | ✓ | non-empty; each `^\.[a-z0-9]+$` (dot, lowercase); matched against the lowercased file ext |
 | `entry` | ✓ | relative HTML path, no leading `/`, no `..`; file must exist |
 | `priority` | – | `default` (default; opens the file directly) \| `option` (open-with menu only) |
-| `capabilities` | – | `"save"` \| `"media"` (iframe gets `allow="microphone; display-capture; clipboard-write"`) \| `"transcribe"` (may open the server's [transcription proxy](#streaming-transcription-proxy)); **unknown value = manifest error** (reject rather than let the host silently drop a declared capability) |
+| `capabilities` | – | `"save"` \| `"media"` (iframe gets `allow="microphone; display-capture; clipboard-write"`) \| `"transcribe"` (may open the server's [transcription proxy](#streaming-transcription-proxy)) \| `"fs-read"` (read-only `fs` over the workspace and the whole machine + the host file picker, bundle or not — see [Scoped file access and the picker](#scoped-file-access-and-the-picker-fs-read)); **unknown value = manifest error** (reject rather than let the host silently drop a declared capability) |
 | `bundle` | – | boolean, default `false`; `true` = every suffix names a **directory** (see [Bundle extensions](#bundle-extensions)). Non-boolean, or `true` together with `save` (`bundle extensions cannot declare save`) = manifest error |
 | `platforms` | – | non-empty array of `web` \| `desktop-mac` \| `desktop-win` \| `desktop-linux` (deduped); omitted = everywhere. Empty / unknown value = manifest error |
 | `settings` | – | `{ params?: [field…], secrets?: [field…] }`, field = `{ key, description?, description_zh?, default?, type?, options? }` (skill `config.yaml` format). Validated **strictly** (a skill's config.yaml is parsed leniently): `key` `^[a-z][a-z0-9_]{0,63}$`, unique across both lists, `type` ∈ string\|int\|float\|boolean\|enum (enum needs `options`), scalar `default` stringified; anything else = manifest error. `ExtensionInfo.settings` carries the declarations, never values. See [Extension settings](#extension-settings) |
@@ -113,12 +113,14 @@ Types in `packages/core/src/protocol/extension-frames.ts` (`EXTENSION_PROTOCOL_V
 
 | Dir | `type` | Fields | Semantics |
 |---|---|---|---|
-| ext→host | `ready` | `protocol` | listener installed; host sends nothing before it. v1 host doesn't read `protocol` (field reserved for negotiation) |
-| host→ext | `init` | `protocol`, `file{name,path,size,ext}`, `capabilities`, `theme`, `themeVars`, `bundle`, `platform`, `lang`, `export` | once, right after `ready`; `capabilities` = what the host grants; for a bundle `file` is the directory; `platform` = host platform, `lang` = `zh`\|`en` admin UI language; `theme` / `themeVars` see [Theme tokens](#theme-tokens); `export` = this host accepts `export` frames (see [Export](#export)) — older hosts omit it, so show export UI only when it is `true` |
+| ext→host | `ready` | `protocol` | listener installed; host sends nothing before it. v1 host doesn't read `protocol` (field reserved for negotiation). A second `ready` (the extension reloaded itself) is not ignored: the host re-sends `init` (+ `load`, except to a bundle) and clears the tab's dirty flag, since the reloaded document holds nothing |
+| host→ext | `init` | `protocol`, `file{name,path,size,ext}`, `capabilities`, `theme`, `themeVars`, `bundle`, `platform`, `lang`, `export` | right after each `ready`; `capabilities` = what the host grants; for a bundle `file` is the directory; `platform` = host platform, `lang` = `zh`\|`en` admin UI language; `theme` / `themeVars` see [Theme tokens](#theme-tokens); `export` = this host accepts `export` frames (see [Export](#export)) — older hosts omit it, so show export UI only when it is `true` |
 | host→ext | `load` | `buffer`, `mtime` | file bytes (transferred); resent when the file changes on disk and the doc isn't dirty → "replace current document". **Never sent to a bundle** |
 | ext→host | `dirty` | `dirty` | needs `save` (or `bundle`, where it means "busy"); otherwise ignored + `console.warn` |
-| ext→host | `fs` | `id`, `op: read\|write\|append\|list\|stat`, `path`, `buffer?` | bundle only (else `fs-result{denied}` + warn); `path` bundle-relative POSIX, validated before any request |
+| ext→host | `fs` | `id`, `op: read\|write\|append\|list\|stat`, `path`, `scope?: workspace\|system`, `buffer?` | no `scope`: bundle only (else `fs-result{denied}` + warn), `path` bundle-relative POSIX, all ops. With `scope` (needs `fs-read`, bundle or not; else `denied` + warn): `read` / `stat` / `list` only (`write` / `append` → `denied`), `workspace` = workspace-relative POSIX, `system` = absolute normalized path (POSIX `/…`, Windows `C:/…`, forward slashes, no `..`). Paths are validated before any request — see [Scoped file access](#scoped-file-access-and-the-picker-fs-read) |
 | host→ext | `fs-result` | `id`, `ok`, `buffer?` \| `entries?` \| `size?`+`mtime?` — or `code: not-found\|invalid-path\|denied\|io`, `error` | exactly one per `fs`, same `id`; `read` buffer transferred |
+| ext→host | `pick` | `id`, `accept: string[]`, `start?` | `fs-read` only (else `pick-result{denied}` + warn): open the host's file picker. `accept` = lower-case suffixes with the dot (`['.zip']`), `[]` = any file; `start` = workspace-relative or absolute dir to open at (default: the bundle dir for a bundle, else the open file's dir) |
+| host→ext | `pick-result` | `id`, `ok: true`, `path`, `name`, `size` — or `ok: false`, `code: cancelled\|denied`, `error` | exactly one per `pick`, same `id`. `path` is workspace-relative when the file's realpath is inside the workspace, else absolute (read it with `scope: 'system'`). A second `pick` while a picker is open → `cancelled` |
 | host→ext | `save-request` | – | Save button pressed and doc dirty; ext answers `save` |
 | ext→host | `save` | `buffer` | no `save` capability → `save-error{denied}`, no network; a save while one is in flight is dropped |
 | host→ext | `saved` | `mtime` | PUT succeeded; ext clears dirty, `mtime` is the new baseline |
@@ -170,20 +172,39 @@ An extension that can render its document as an image (draw.io *File ▸ Export
 
 ### MRU, keep-alive, upgrade, uninstall
 
-- `editor-panel` keeps the last 5 preview tabs mounted (`PREVIEW_CACHE_SIZE`); "heavy" previews mount active-only. `isHeavyPreview` is true for an extension **without** `save` and not `bundle`: nothing to lose on unmount, and each viewer may hold a WebGL context (Chrome caps ~16 per page). An extension **with** `save` stays in the MRU because its dirty state lives only in the iframe; a dirty preview is never evicted past the cap. The active heavy preview's wrapper is keyed by path: `FilePreview`'s `Dispatch` is keyed only by file extension and a host posts the file bytes once per mount, so switching between two heavy tabs of the same type (two `.ipynb`) used to keep showing the first file.
+- `editor-panel` keeps the last 5 preview tabs mounted (`PREVIEW_CACHE_SIZE`); "heavy" previews mount active-only. `isHeavyPreview` is true for an extension **without** `save` and not `bundle`: nothing to lose on unmount, and each viewer may hold a WebGL context (Chrome caps ~16 per page). An extension **with** `save` stays in the MRU because its dirty state lives only in the iframe; a dirty preview is never evicted past the cap. The mounted previews render in **path-sorted** order, not MRU order: promoting a tab reorders the MRU, React would then move the keyed nodes with `insertBefore`, and moving an iframe in the DOM reloads it — a running emulator or editor would start over on every tab switch. With a stable order, insertions and removals never move the siblings. The active heavy preview's wrapper is keyed by path: `FilePreview`'s `Dispatch` is keyed only by file extension and a host posts the file bytes once per mount, so switching between two heavy tabs of the same type (two `.ipynb`) used to keep showing the first file.
 - **Upgrade while mounted** (`extension:changed` gives the tab a new `info.version`): not dirty → remount with the new URL; dirty → non-blocking banner "updated to vX" with a Reload link (`reload` asks to discard first).
 - **Uninstall while mounted**: not dirty → `resolve()` drops the candidate and the tab falls to the next one (built-in plugin if the ext has one, else the fallback page). Dirty → `Dispatch` keeps rendering the last-shown extension (`uninstalled` banner) so the edits can still be saved; the loaded iframe already holds its assets, though a fresh load would 404. Once saved, the tab falls through the same way.
 - **Fallback page** (`UnsupportedPreview`): "This file type has no built-in preview. Extensions for more file types are available at halo-hub ↗" + Open as Text / Download. The link is a constant — no lookup, no per-ext URL, no online recommendation (would need a network call or a bundled index that goes stale).
 - **Settings → Extensions** (`features/settings/extensions-view.tsx`): upload zip, list (errors in red), remove with `confirmAction`. It never re-fetches after a mutation — `extension:changed` updates it.
 
+### Immersive maximize
+
+Maximizing the canvas on a **viewer** extension (an emulator, a model viewer) hides every bit of chrome around it. The predicate is `isImmersiveViewer(info)` in `registry.ts`: any extension **without** `save`, bundles included (a bundle's state lives in its own directory, not behind a Save button). It is *derived* — `editorStore.maximized` stays the only persisted state, and leaving immersive is `setMaximized(false)`; the extension host decides (it knows which candidate actually rendered), publishing itself through `claimImmersive()` (`features/editor/immersive.ts`) while `maximized` is on and the pane's active tab (`ImmersivePane` context, provided by `EditorPanel` for the focused pane only) is its path.
+
+- **Pure CSS.** While immersive, the host's banners + iframe region become a `fixed inset-0 z-[70]` overlay, and `EditorPanel` / `workspace-layout` class-hide the panel header, tab bar, non-focused panes and the floating bottom panel (`hidden`, never unmounted). The iframe is never reparented or remounted, so the viewer keeps its state.
+- **Real fullscreen** only from the maximize click: the button calls `armImmersiveFullscreen()` (valid for that click's task), and the host that goes immersive takes the arm inside the click's user activation and requests fullscreen on `<html>`. A `maximized` restored from localStorage has no activation and stays in-page. The browser or Electron ending fullscreen ends immersive too (one Esc, one exit) — unless the viewer had its own fullscreen (an emulator's canvas), which keeps that Esc.
+- **Exit**: **Esc** — the iframe is same-origin, so its keydown is watched inside each loaded document (the viewer's own fullscreen takes that Esc first) — or the top pill (`editor.immersive.exit`, "× Exit · Esc"). Mouse: top-centre, shown for 2.5 s on entering, pinned while the pointer is within 48 px of the viewport top (watched by listeners, no hot zone over the iframe), hidden 1.5 s after it leaves. Touch: always on screen as a compact × at top-left, where viewers keep their title label; a tap in the top band widens it for 3 s, and the pill exits only for a press that started on it. On exit the host hands focus back to the iframe, except for "Back to chat".
+- **Reply toast.** The chat is hidden, so when any loaded chat tab's turn settles while immersive (`workspace-layout`, the same hook as the finish notification) an in-page toast (`z-[80]`, ~8 s, newest session first) says "Agent replied · <title>" (or "N sessions replied") with **Back to chat**: it exits immersive without refocusing the iframe, opens that session's tab, switches the bottom panel to Chat and dispatches `halo:focus-chat-input` so the caret lands in the composer (`message-input.tsx` listens). The toast is dropped as soon as immersive ends.
+- The host file picker (`pick`, below) renders in a body portal at `z-[90]`, above the overlay and its toast.
+
 ### Bundle extensions
 
 A `bundle: true` extension opens a **directory** whose name ends in one of its suffixes (`Standup.htrans/`) — a package the extension owns, first user `htrans` (meeting recorder). Routing is separate from files: `resolveBundle(ext)` lists bundle extensions only, `resolve(ext)` never does (a *file* `x.htrans` is untouched), and `isBundleName(name)` = a `default` bundle extension that runs here claims the suffix.
 
-- **Explorer**: such a directory gets a package icon; clicking (or Enter on) the row opens it as a preview tab like a file, double-click too; only the chevron expands it, so its contents stay browsable. Context menu adds Open to the Side. "New File" with a bundle suffix runs `mkdir` and opens the new bundle. The tab has no Open as Text / Download. Persisted tabs carry `bundle: true`; on restore the tab reopens only if a bundle extension still claims it, otherwise it is dropped (the directory is never read as text).
+- **Explorer**: such a directory gets a package icon; clicking (or Enter on) the row opens it as a preview tab like a file, double-click too; only the chevron expands it, so its contents stay browsable. Context menu adds Open to the Side. "New File" with a bundle suffix runs `mkdir` and opens the new bundle — both the tree's inline New File and the Explorer header's New File button (which would otherwise create an empty file no extension can open). The tab has no Open as Text / Download. Persisted tabs carry `bundle: true`; on restore the tab reopens only if a bundle extension still claims it, otherwise it is dropped (the directory is never read as text).
 - **Access**: no `load`; the extension reads and writes inside its directory with `fs` frames. The host maps them to `GET /files/download?inline=1` (read), `GET /files/tree` (list), `GET /files/stat` (stat) and `PUT /files/raw?create=1&root=<bundle>[&append=1]` (write / append; a vanished bundle → `not-found`), and serializes write / append **per path** (`createKeyedQueue`), so appends land in request order; other ops run concurrently. `file:changed` never reloads a bundle.
 - **Busy**: `dirty: true` = "recording, don't drop me" → tab dot, close confirm (also when the same bundle is open in the other split pane — each pane runs its own iframe), MRU pin, page-unload warning: the browser always shows its generic beforeunload prompt; the desktop shell shows a native confirm, which the admin only triggers while a tab is busy or unsaved. Deleting, renaming or drag-moving a busy bundle (or an ancestor directory) asks the same close confirm first, then closes the tab **before** the file operation. Ctrl+S / Save does nothing on a bundle tab. Bundle extensions are never heavy (they stay in the MRU like `save` extensions). fs results that arrive after the iframe was remounted (retry / upgrade) are dropped.
 
+### Scoped file access and the picker (`fs-read`)
+
+A viewer extension (emulator, model viewer) otherwise sees only its own file or bundle. `fs-read` lets it read elsewhere on the machine and lets the user choose what to read. UX gating in the host, **not** an isolation boundary: the iframe is same-origin with the admin cookie and could call `/api/fs/raw` itself, which is why the guard below sits on the routes.
+
+- **`fs` with `scope`** (`extension-host-logic.ts` `onFsFrame`): `workspace` → the same `/files/*` read routes with a workspace-relative path; `system` → admin-cookie routes by absolute path: `read` → `GET /fs/raw` (streamed bytes), `stat` → `GET /fs/stat`, `list` → `GET /fs/browse?files=1` (name + type per entry). Validation happens before any request: `isSystemPath` (POSIX `/…` or `C:/…`, forward slashes, no NUL, no empty / `.` / `..` segment); unknown scope or op, a missing capability, or `write` / `append` → `fs-result{denied}`; a bad path → `invalid-path`.
+- **`pick`**: opens `FilePicker` (`shared/components/file-picker.tsx`) — a whole-machine browser with Workspace and Home shortcuts, a path box, a breadcrumb and the `accept` filter (directories always show; keyboard: arrows / Enter / Backspace for up / Esc cancels, the capture listener swallowing Esc before maximize sees it). One picker at a time, tied to the iframe attempt that asked; a reply for a since-remounted iframe is dropped. If the viewer is in its own fullscreen the host first drops back one level, so the picker isn't covered. On pick, the host `stat`s both the file and the workspace and compares **realpaths**, so a symlinked workspace or ROM dir lands on the right side: inside → workspace-relative `path`, else the absolute one. Closing it answers `cancelled`.
+- **Hidden-path guard** (`routes/files.ts`): `/fs/stat`, `/fs/raw` and `/fs/browse?files=1` all refuse the sandbox hidden lists (`~/.halo/secrets`, `~/.ssh`, `.git-credentials`, the global dbs …) via `isHiddenHostPath` (exported from `tools/sandbox.ts`; no second list), judged on the **realpath** so a symlink can't reach them. A refused path answers with the same 404 and message as a missing file, and a listing silently omits hidden entries. `/fs/browse?files=1` lists the same noise-filtered tree as `/files/tree` but with dot-entries shown and symlinks counted as what they point at; `sizes=1` adds file sizes. The default folder-picker response is unchanged. See [dev/api.md → Filesystem browse](../dev/api.md#filesystem-browse-not-project-scoped--for-the-workspace-picker).
+- Settings → Extensions shows a "Reads files on this machine" tag for an extension that declares it.
+
 ### Platforms
 
 `currentPlatform()` (`registry.ts`, computed once): UA contains `Electron/` → `desktop-mac|win|linux` by OS, else `web`. An extension whose `platforms` excludes it is skipped by every resolve (routes as if not installed; a bundle directory is a plain folder), the fallback page says `"<name>" only runs on: …`, and Settings → Extensions shows a platforms badge plus a gray "not supported here" tag.
@@ -194,7 +215,7 @@ An installed extension with manifest `settings` gets one Settings section (`sett
 
 ### Streaming transcription proxy
 
-WS `/api/transcribe/stream?ext=<id>&lang=<auto|xx-XX|xx-XX,yy-YY,…>` (`routes/transcribe-ws.ts`) relays an extension's live audio to **Amazon Transcribe streaming** so AWS credentials never reach the browser. `index.ts` runs both WS servers in `noServer` mode behind one http `upgrade` dispatcher (`/ws` → admin socket, unchanged incl. AgentCore; `/api/transcribe/stream` → proxy, not mounted in AgentCore mode; any other path → socket destroyed) — a `ws` server attached with `{ server, path }` would 400 every other path's handshake.
+WS `/api/transcribe/stream?ext=<id>&lang=<auto|xx-XX|xx-XX,yy-YY,…>` (`routes/transcribe-ws.ts`) relays an extension's live audio to **Amazon Transcribe streaming** so AWS credentials never reach the browser. `index.ts` runs both WS servers in `noServer` mode behind one http `upgrade` dispatcher (`/ws` → admin socket and `/api/transcribe/stream` → proxy, neither mounted in AgentCore mode, where every upgrade is refused; any other path → socket destroyed) — a `ws` server attached with `{ server, path }` would 400 every other path's handshake.
 
 - **Handshake**: login cookie (`isAuthenticated`, same as `/ws`) else 401; `ext` must be installed and declare `transcribe` else 403. The extension iframe is same-origin, so its WS carries the cookie.
 - **Upstream**: `StartStreamTranscription` (`pcm`, 16 kHz). `lang=auto` (default) → `IdentifyMultipleLanguages` + `LanguageOptions` = `ext-<id>.params.auto_languages` → manifest default → `zh-CN,zh-HK,en-US` (Cantonese must be a candidate, otherwise it is transcribed as wrong Mandarin text); one code (`^[a-z]{2}-[A-Z]{2}$`) → `LanguageCode`; a comma-separated list of 2–5 distinct codes → `IdentifyMultipleLanguages` + `LanguageOptions` = exactly that list (htrans sends `zh-CN,en-US` / `zh-HK,en-US` for its Mandarin + English / Cantonese + English options); anything else → `error bad-request`, upstream never called. "One dialect per language" in a list is left to Transcribe (its BadRequest maps to `bad-request`). Region `ext-<id>.params.region` → manifest default → `us-east-1`. Credentials: all three of `ext-<id>.secrets.access_key_id` / `secret_access_key` / `session_token` empty → the SDK default chain (`defaultProvider()`); both keys set → static keys (+ `session_token` if set); any other partial set → `error credentials` naming the missing key(s), upstream never called (no silent switch to the machine's identity). All read per connection.
@@ -225,9 +246,9 @@ The hub repo keeps one directory per extension (`glb` / `ipynb` commit their ven
 
 ## Extension capability boundary
 
-An extension **can**: receive one file's bytes and metadata; render in its own iframe; load static assets from its own directory via the path token; report `error`; receive `theme` and `lang`; with `save`, report `dirty` and hand back bytes for the host to write to **that same file**, and (on `init.export` hosts) hand back export bytes the host writes as a **new sibling file** after an overwrite confirm; with `bundle`, read / write / append / list / stat **inside its bundle directory** via `fs`; with `media`, use the microphone and screen capture; with `transcribe`, stream audio to the server's transcription proxy (results only — credentials stay server-side).
+An extension **can**: receive one file's bytes and metadata; render in its own iframe; load static assets from its own directory via the path token; report `error`; receive `theme` and `lang`; with `save`, report `dirty` and hand back bytes for the host to write to **that same file**, and (on `init.export` hosts) hand back export bytes the host writes as a **new sibling file** after an overwrite confirm; with `bundle`, read / write / append / list / stat **inside its bundle directory** via `fs`; with `fs-read`, **read** (read / stat / list) any file in the workspace or on the machine outside the sandbox hidden lists, and ask the user to pick one; with `media`, use the microphone and screen capture; with `transcribe`, stream audio to the server's transcription proxy (results only — credentials stay server-side).
 
-An extension **cannot** (sandbox + protocol): open popups, downloads, forms or navigate the top window; use camera / autoplay, or mic / screen capture without `media` (`allow=""`); add admin UI (toolbar, commands, sidebar); talk to other iframes. The **protocol** gives it no way to read or write any workspace file except the open one, or outside its bundle directory. Since the host grants `allow-same-origin` (see Host), the sandbox no longer isolates it from the admin's origin: a hostile extension *could* script `parent.document` or call the cookie-authed `/api/*` (measured: `fetch('/api/extensions')` from inside → 200). That is the accepted trust model — installing an extension is like installing a skill — not a gap to patch per route. **Outbound network is not blocked** — no CSP is injected, so `fetch('https://…')` works where the remote allows CORS; trust model is "code the user chose to install" (same as a skill), and hub policy is offline-capable extensions. A hard block would be one `Content-Security-Policy` header on the asset route, no protocol change.
+An extension **cannot** (sandbox + protocol): open popups, downloads, forms or navigate the top window; use camera / autoplay, or mic / screen capture without `media` (`allow=""`); add admin UI (toolbar, commands, sidebar); talk to other iframes. The **protocol** gives it no way to write any workspace file except the open one (or inside its bundle directory), and no way to read others without `fs-read`. Since the host grants `allow-same-origin` (see Host), the sandbox no longer isolates it from the admin's origin: a hostile extension *could* script `parent.document` or call the cookie-authed `/api/*` (measured: `fetch('/api/extensions')` from inside → 200). That is the accepted trust model — installing an extension is like installing a skill — not a gap to patch per route. **Outbound network is not blocked** — no CSP is injected, so `fetch('https://…')` works where the remote allows CORS; trust model is "code the user chose to install" (same as a skill), and hub policy is offline-capable extensions. A hard block would be one `Content-Security-Policy` header on the asset route, no protocol change.
 
 Adding a capability: (1) extend `ExtensionCapability` + the `CAPABILITIES` whitelist in `registry.ts`; (2) add the frame type in `extension-frames.ts`; (3) gate it in `extension-host-logic.ts` (undeclared → ignore + warn, like `save`); (4) if it needs the server (e.g. reading a sibling file), the **host** calls `/api/files/*` — an extension never touches the API directly. Install, scan, watcher and registry stay untouched.
 

package/bundled-docs/design/observability.md

@@ -2,7 +2,7 @@
 
 ## Overview
 
-A vendor-neutral observability layer built on the official OpenTelemetry JS SDK. One setting, `general.observability.endpoint`, turns on OTLP http/protobuf export of traces, metrics and logs to whatever collector the operator points at — empty means fully off. Principle: zero AWS (or any vendor) code in the server — SigV4 signing, backend endpoints, and resource enrichment all live in the collector, not in halo.
+A vendor-neutral observability layer built on the official OpenTelemetry JS SDK. One setting, `general.observability.endpoint`, turns on OTLP http/protobuf export of traces, metrics and logs to whatever collector the operator points at — empty means fully off. It applies to every process that runs the agent loop: the server, `halo cli` and `halo tui` — and so the cron jobs and evolution runs, which are spawned `halo cli` children. Principle: zero AWS (or any vendor) code in the server — SigV4 signing, backend endpoints, and resource enrichment all live in the collector, not in halo.
 
 ## Configuration
 
@@ -13,7 +13,9 @@ A vendor-neutral observability layer built on the official OpenTelemetry JS SDK.
 | `general.observability.headers` | string, **secret** | `''` | Extra headers for every OTLP request, comma-separated `k=v` (e.g. `authorization=Bearer …`). Restart required. |
 | `general.observability.capture_content` | boolean | `false` | Put prompt / completion / tool argument+result text on spans (`gen_ai.*.messages` etc.). Off = metadata only (model, tokens, latency, tool names). Restart required. |
 
-All four keys are `globalOnly` and take effect on server restart — `otel.ts` reads them once at boot, before anything else touches the OTel API.
+All four keys are `globalOnly` and take effect on server restart — `otel.ts` reads them once at boot, before anything else touches the OTel API. A `halo cli` / `halo tui` process reads them the same way at its own start, so a change reaches the next run.
+
+**Which process exported.** Every process shares `service.name`, so the resource also carries `halo.process` = `server` \| `cli` \| `tui` (`HaloProcess` in `otel.ts`; `initObservability()` defaults to `server`). Cron and evolution children are `halo cli` runs, so they report `cli`; their `session.id` (`cron-<jobId>`, or the `__evo_agent__`-style agent name) tells them apart from a hand-run cli. The startup log line names it: `[Observability] OTLP export → … (service.name=…, halo.process=cli, …)`.
 
 **Env-var mapping.** `initObservability()` maps settings onto the standard `OTEL_*` env vars with `??=`, so an operator who already exports them (or `--require`s an external distro) wins:
 
@@ -35,17 +37,23 @@ No `OTEL_EXPORTER_OTLP_ENDPOINT` after the mapping → `enabled=false`, `initObs
 | module | role |
 |---|---|
 | `packages/server/src/observability/otel.ts` | Bootstrap. `initObservability()` / `shutdownObservability()`, the `enabled` gate, `tracer` / `getMeter()` / `otelLogger` / `captureContent()` exports. |
-| `packages/server/src/observability/otel-sdk.ts` | The ONE module that imports the SDK packages and OTLP exporters. |
+| `packages/server/src/observability/otel-sdk.ts` | The ONE module that loads the SDK packages and OTLP exporters (`import()`ed inside `registerSdk`, see below). |
 | `packages/server/src/observability/genai-spans.ts` | The span model: `beginTurn` / `onAgentEvent` / `recordRetry` / `endTurn`. |
 | `packages/server/src/agents/session-manager.ts` | Calls the four hooks above from the turn loop (see below). |
 | `packages/server/src/logger.ts` | Bridges every `logger.*` call into an OTel LogRecord when `enabled`. |
 | `packages/server/src/index.ts` | `await initObservability()` (before `initLogger()`) and `await shutdownObservability()` (in `gracefulShutdown`). |
+| `packages/cli/src/harness.ts` · `index.ts` | `initRuntime('cli' \| 'tui')` calls `initObservability(<process>)` before `initLogger()` (the list-only `agents` / `sessions` commands pass nothing, so stay off); the exit flushes below. The server package exports `./observability/otel` for this. |
 
 **Hook call sites in `session-manager.ts`** (all in `runAgentTurn`): `beginTurn(session, message)` at the start, `onAgentEvent(session, event)` inside the agent event loop (one call per `AgentEvent`), `recordRetry(kind)` at the 7 retry catch sites, `endTurn(session, {error?})` once after the retry loop (the turn's single exit point, success or failure).
 
 **The `enabled` gate.** `otel.ts` exports a single `enabled: boolean`, set once by `initObservability()`. Every hook in `genai-spans.ts` and the `logger.ts` bridge checks it first — an unconfigured server pays nothing beyond one boolean read per hook call.
 
-**Why `otel-sdk.ts` is a dynamic import.** `logger.ts` is shared with the CLI (`halo cli`), which never wants observability. Eagerly importing the SDK module graph would cost ~100ms on every `halo cli` start. `otel.ts` only `import()`s `otel-sdk.ts` after confirming an endpoint is configured, so the unconfigured path (server or CLI) never loads it.
+**Why the SDK is loaded by dynamic import.** The SDK module graph costs ~120ms, which every unconfigured `halo` start (`--version` included) would otherwise pay. `otel.ts` only `import()`s `otel-sdk.ts` after confirming an endpoint is configured, and `otel-sdk.ts` itself `import()`s the SDK / exporter packages inside `registerSdk` rather than at the top: the npm bundle (esbuild) inlines `otel-sdk.ts`, which would hoist top-level imports of those external packages into the bundle's static imports and load them on every start. So the unconfigured path (server, `halo cli`, `halo tui`) never loads the SDK.
+
+**Flush on exit (cli / tui).** A short-lived process must push its batched spans / metrics / logs before it goes. `shutdownObservability()` force-flushes and shuts down every provider, is idempotent (a signal can land while the end-of-run flush is in flight), and is **capped at 3 s** (`SHUTDOWN_CAP_MS`) so a dead collector delays an exit but never hangs it; it resolves `false` when the cap fired, and is instant when observability is off. The 3 s sits inside the cron runner's 30 s and the evolution wrapper's 10 s SIGTERM→SIGKILL grace. Three exit paths:
+- **natural end** (one-shot run done, TUI quit, a thrown error): a `beforeExit` hook registered by `initRuntime` flushes; when the cap fired the exporters' retry timers still hold the event loop, so the hook then calls `process.exit()` itself (stdout has drained by then). `beforeExit` never fires on `process.exit()`.
+- **SIGINT / SIGTERM**: the handlers stop the harness first (ending the turn, so its spans are closed and in the batch), then `exitAfterFlush(130 | 143)` (`harness.ts`) flushes and exits. A second SIGINT still hard-exits at once.
+- Any other `process.exit()` skips the flush. On Windows `child.kill('SIGTERM')` delivers no catchable signal, so a timed-out cron run there loses its last batch.
 
 **Why the meter is lazy.** The `@opentelemetry/api` tracer and logger are proxies — they stay no-op until a real provider is registered, then transparently start forwarding, so `tracer` / `otelLogger` can be module-level consts. The metrics API has **no proxy**: a `Meter` fetched via `metrics.getMeter()` before `setGlobalMeterProvider()` runs is a permanent no-op forever after, even once a provider registers later. `getMeter()` in `otel.ts` is therefore a function, not a const, and `genai-spans.ts` calls it lazily on first metric emission (`instruments()`, cached after).
 

package/bundled-docs/design/relay.md

@@ -61,7 +61,7 @@ No quiet gate (the session is by definition not done) and **`reply_to` is kept**
 
 ## What it deliberately doesn't do
 
-- **No cross-server reach.** Same `SessionManagerRegistry`, same process. Different server → not supported yet (A2A is planned).
+- **No cross-server reach.** Same `SessionManagerRegistry`, same process. Different server → [A2A](a2a.md), which reuses the same `reply_to` column (`{ a2a: taskId }`), quiet gate and interim doors.
 - **No session discovery beyond a flat root list.** `relay_list` returns a workspace's root sessions (id / agent / title / status, newest 100) so the secretary can reuse an existing conversation; it doesn't walk trees or search transcripts — that's the admin's job.
 - **No broadcast / fan-out helper.** The secretary calls `relay_send` once per department; the reports come back individually with the workspace + session id in the header, so it can tell them apart without extra bookkeeping.
 - **No ACL beyond full access.** A full-access secretary can reach every workspace on the box — that's the deployment's trust boundary already (it can `shell_exec` into them anyway).

package/bundled-docs/design/session.md

@@ -182,6 +182,8 @@ The orphan reconcile above fixes the *children*'s `stoppedAt`; nobody told the *
 
 **Eager boot** (`index.ts`): rather than waiting for someone to open a workspace, right after constructing `SessionManagerRegistry` (before `bootChannels`) the server calls `listRunningWorkspaces()` and for each workspace with leftover rows whose `.halo` dir still exists: `claimWorkspaceRuntime(ws)` first — if another live server owns it, `continue` (rows stay for a later boot, and no SessionManager is built, since caching a non-owner SM would freeze "not owner" for the whole process lifetime) — otherwise `registry.getOrCreate(ws)`, which fires the same constructor chain (claim → reconcile → retained `sweepActiveGoals` → `sweepInterruptedRuns`). An interrupted root with nobody around is exactly the one that must nudge itself without a human opening the tab first.
 
+**AgentCore A2A mode** (`a2a/agentcore.ts`): the ledger is `<HALO_WORKSPACE>/.halo/runs.db` on the EFS mount instead of `~/.halo/global/runs.db` (a microVM's home dies with it), opened only once the workspace lease is held; the sweep runs on lease acquisition rather than at boot, and the first `POST /` waits for its nudges to start (`nudgesSettled`) before it is served — see [agentcore.md](agentcore.md). That file joins the workspace-relative hidden list (`WORKSPACE_HIDDEN_FILES`).
+
 `runs.db`/`-wal`/`-shm` join the built-in sandbox hidden-files list (`DEFAULT_HIDDEN_FILES`, always included; `general.sandbox.hidden_files` only adds to it) alongside `evo.db` / `cron.db` (see [storage.md](storage.md)). Known gap carried from the design doc: `<ISO>` in the nudge is sweep time, not the actual interruption time — for a workspace that sat locked by another server before this boot could claim it, that's a real but accepted drift.
 
 ### Message queue and drain
@@ -190,7 +192,7 @@ The orphan reconcile above fixes the *children*'s `stoppedAt`; nobody told the *
 
 A `QueuedMessage` is `{ text, sourceSessionId?, images? }`: `sourceSessionId` is set for agent→agent entries (drives the `(from: session X)` prefix and the sibling-status suffix) and absent for user messages; `images` is the user multimodal payload (agent entries have none).
 
-**runSession loop** (`runSession(sessionId, message)`, `message: string | ContentBlock[]`): runs the opening turn when there is one, then drains. An empty **string** message means "the work is already in `messageQueue`" (the `querySession` idle path) — it skips the opening turn and goes straight to drain. The per-turn reset (fresh `toolCallLog` / `warnedToolHashes`, `interruptRequested = false`) runs before the opening turn; `drainQueue` repeats the same reset before each merged batch so a stale interrupt flag never leaks across turns. Server-only: right before `session.promise = runFn()` it inserts the session id into the run ledger, and in the `finally` (right after `session.promise = null`, before emitting `complete`) it deletes it — see [Run ledger](#run-ledger--restart-nudge-for-interrupted-roots-halo-globalrunsdb) above.
+**runSession loop** (`runSession(sessionId, message)`, `message: string | ContentBlock[]`): runs the opening turn when there is one, then drains. An empty **string** message means "the work is already in `messageQueue`" (the `querySession` idle path) — it skips the opening turn and goes straight to drain. The per-turn reset (fresh `toolCallLog` / `warnedToolHashes`, `interruptRequested = false`) runs before the opening turn; `drainQueue` repeats the same reset before each merged batch so a stale interrupt flag never leaks across turns. Server-only: right before `session.promise = runFn()` it inserts the session id into the run ledger, and in the `finally` (right after `session.promise = null`, before emitting `complete`) it deletes it — see [Run ledger](#run-ledger--restart-nudge-for-interrupted-roots-haloglobalrunsdb) above.
 
 **Observability.** `runAgentTurn` is wrapped by `beginTurn` / `endTurn` from `observability/genai-spans.ts` — one `invoke_agent` span per turn, with `chat` / `execute_tool` child spans created from the `onAgentEvent` hook inside the event loop; see [design/observability.md](observability.md).