zcode-acp-server 0.1.0 → 0.3.0

package/docs/ARCHITECTURE.md

@@ -14,7 +14,7 @@ application-client (Zed / JetBrains)
        |
        v
 zcode-acp-server (stdio JSON-RPC ACP)
-  |-- handlers/     session, extensions, dispatch, server-requests, io, slash
+  |-- handlers/     session, extensions, dispatch, server-requests, io, slash, account
   |-- translators/    event-translator, projection-differ, tool-helpers
   |-- interaction/    adapter
   |-- config/         options, runtime-model, model-cache
@@ -25,18 +25,58 @@ zcode-acp-server (stdio JSON-RPC ACP)
 zcode app-server --stdio (line-delimited JSON)
 ```
 
+## ACP Handshake
+
+The `initialize` request (`server.ts`) negotiates the protocol version and
+declares the agent's shape to the editor:
+
+- **`protocolVersion`** — pinned to `PROTOCOL_VERSION` (currently 1).
+- **`agentInfo`** — name/title/version from `AGENT_INFO` in `utils.ts`.
+- **`agentCapabilities`** — `loadSession`, plus `sessionCapabilities.list /
+resume / fork`. The prompt capabilities (image/audio/embeddedContext) and
+  MCP capabilities are all off.
+- **`authMethods`** — a single agent-type entry (`zcode-credentials`). The
+  bridge reads the GLM API key itself from `~/.zcode/v2/config.json` and
+  forwards it to the ZCode subprocess via `ANTHROPIC_API_KEY`; the editor
+  never supplies credentials. Omitting the `type` field defaults to `"agent"`,
+  which the ACP registry's auth-check accepts as "agent self-handles auth".
+
+`initialize` does **not** spawn the backend, and neither does `session/new`:
+the backend is lazily created on the first backend RPC — for a fresh session
+that is the first `session/create` at its first use (prompt / config change /
+extension method). This keeps the handshake succeeding even in an environment
+without `~/.zcode/v2/config.json` (e.g. the registry CI runs `initialize` with
+an isolated `HOME`).
+
+Client capabilities advertised at `initialize` are recorded on the server
+(`clientCapabilities`) and drive later behaviour: `supportsElicitationForm()`
+gates form-based elicitation, and `supportsTerminalOutput()` gates Zed's Bash
+terminal UI.
+
 ## Core Data Flow
 
 ### 1. Session lifecycle
 
 ```
-session/new → session/create → register EventListener
+session/new → placeholder id (backend session NOT created yet)
+     |
+first use: prompt / set_config_option / extension method
+     |
+  session/create → register EventListener
      |
 prompt request → session/send → EventTranslator translates → dispatchEvent
      |                                  |
   end_turn / cancelled         session/update notification
 ```
 
+Sessions are materialized lazily (`ensureRealSession`): an editor startup that
+never sends a message leaves no empty session in the backend or the App's task
+index. The placeholder → backend-session mapping is persisted to
+`~/.zcode/v2/acp-lazy-sessions.json` (`src/lazy-sessions.ts`), so a `session/
+resume` / `session/load` of a placeholder from a previous bridge lifetime still
+resolves: with a recorded backend id the real session is resumed, without one a
+fresh (empty) session is materialized — never "Session not found".
+
 ### 2. Event stream subscription
 
 ```
@@ -52,11 +92,13 @@ pollEvent() consumes → EventTranslator.translate()
 ### 3. Dual-path event handling
 
 #### Real-time path (EventTranslator)
+
 - Listens to zcode `session/event` pushes
 - Translates each event to an ACP `session/update` in real time
 - Maintains `seenToolIds` to avoid duplicates
 
 #### Snapshot path (ProjectionDiffer)
+
 - On turn completion, builds a snapshot from `session/messages` + `session/read`
 - Diffs two snapshots to produce new events (PlanUpdate / TextDelta / ToolCallNew, etc.)
 - Used for turn-completion triage and stall recovery
@@ -82,6 +124,7 @@ Key: **`seenToolIds` synchronization**
 
 In `session.ts:629`, after the event path finishes processing, the state is
 synced to the differ:
+
 ```typescript
 for (const seenId of translator.seenToolIds) {
   differ.markToolSeen(seenId);
@@ -92,49 +135,96 @@ This ensures the snapshot diff does not re-emit tools already handled by the
 event path, preventing Bash terminal output from being overwritten by a
 content-less ToolCallNew.
 
+## Data & Privacy
+
+**No telemetry, no analytics, no third-party network calls.** The server is a
+local relay: prompts, code, and tool outputs pass through process memory on
+their way between the editor and the ZCode subprocess, but reach the GLM cloud
+API only because the ZCode backend itself sends them for inference.
+
+| Concern     | Detail                                                                                                                                                           |
+| ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Network     | One outbound request in the whole codebase — `src/quota/client.ts` GET to the quota API, Bearer token only, no body                                              |
+| Credentials | API key from `~/.zcode/v2/config.json` (authenticates the subprocess + quota request), never logged. OAuth handled by the ZCode subprocess, not this server      |
+| Disk        | No new files. Writes only to the existing `~/.zcode/v2/tasks-index.sqlite` — syncs sessions into the ZCode app's history & search (session title + first prompt) |
+| Logging     | `log()`/`warn()` → stderr only for troubleshooting; even with `ZCODE_ACP_DEBUG=1`, no prompts/code/keys are logged                                               |
+
 ## Module Responsibilities
 
 ### `backend/` — ZCode process communication
 
-| File | Responsibility |
-|------|------|
-| `client.ts` | Spawn/manage the zcode subprocess, reader-loop, request/response multiplexing, process watchdog |
-| `listener.ts` | EventStreamListener (subscribe/consume the event stream) and TurnMonitor (snapshot polling) |
-| `types.ts` | ZCode JSON-RPC message type definitions |
+| File          | Responsibility                                                                                  |
+| ------------- | ----------------------------------------------------------------------------------------------- |
+| `client.ts`   | Spawn/manage the zcode subprocess, reader-loop, request/response multiplexing, process watchdog |
+| `listener.ts` | EventStreamListener (subscribe/consume the event stream) and TurnMonitor (snapshot polling)     |
+| `types.ts`    | ZCode JSON-RPC message type definitions                                                         |
 
 ### `translators/` — Event translation
 
-| File | Responsibility |
-|------|------|
-| `event-translator.ts` | Translate zcode events to InternalEvent (real-time path) |
-| `projection-differ.ts` | Diff two snapshots to produce InternalEvent (snapshot path) |
-| `tool-helpers.ts` | Tool-related pure functions: title generation, output rendering, diff parsing, location extraction |
-| `types.ts` | InternalEvent union type and plan entry builders |
+| File                   | Responsibility                                                                                     |
+| ---------------------- | -------------------------------------------------------------------------------------------------- |
+| `event-translator.ts`  | Translate zcode events to InternalEvent (real-time path)                                           |
+| `projection-differ.ts` | Diff two snapshots to produce InternalEvent (snapshot path)                                        |
+| `tool-helpers.ts`      | Tool-related pure functions: title generation, output rendering, diff parsing, location extraction |
+| `types.ts`             | InternalEvent union type and plan entry builders                                                   |
 
 ### `handlers/` — ACP method handling
 
-| File | Responsibility |
-|------|------|
-| `session.ts` | session/new/list/resume/load/prompt/set_config_option/cancel |
-| `extensions.ts` | fork/rewind/rewindCascade/goal/compact/steer/cancelBackgroundTask/setModel/setMode/setThoughtLevel |
-| `dispatch.ts` | dispatchEvent single exit point: InternalEvent → ACP session/update |
-| `server-requests.ts` | Handle zcode interaction/* requests (tool auth, ExitPlanMode, AskUserQuestion), protocol negotiation routing |
-| `io.ts` | ACP notification helpers (including `sendAvailableCommandsDeferred` deferred notification) |
-| `slash.ts` | Interception of `/`-prefixed commands (/compact /goal /fork /rewind /steer /model /mode /thought) |
+| File                  | Responsibility                                                                                                                                                                                          |
+| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `session.ts`          | session/new/list/resume/load/prompt/set_config_option/cancel                                                                                                                                            |
+| `extensions.ts`       | fork/rewind/rewindCascade/goal/compact/steer/cancelBackgroundTask/setModel/setMode/setThoughtLevel                                                                                                      |
+| `dispatch.ts`         | dispatchEvent single exit point: InternalEvent → ACP session/update                                                                                                                                     |
+| `background-tasks.ts` | Session-scoped `BackgroundTaskListener` — forwards background sub-agent status (`session.updated` taskId) + completion-notification turns to the client OUTSIDE request handlers (lives across prompts) |
+| `server-requests.ts`  | Handle zcode interaction/* requests (tool auth, ExitPlanMode, AskUserQuestion), protocol negotiation routing                                                                                            |
+| `io.ts`               | ACP notification helpers (including `sendAvailableCommandsDeferred` deferred notification)                                                                                                              |
+| `slash.ts`            | Interception of `/`-prefixed commands (/compact /goal /fork /rewind /steer /model /mode /thought); non-advertised `/x` prompts are neutralized into plain text (`neutralizeSlashText`)                  |
+| `account.ts`          | `account/usage_stats` — account-level plan quota for remote clients (Proposal 0002; quota pipeline + graceful error)                                                                                    |
 
 ### `interaction/` — Interaction bridging
 
-| File | Responsibility |
-|------|------|
+| File         | Responsibility                                                                                   |
+| ------------ | ------------------------------------------------------------------------------------------------ |
 | `adapter.ts` | Conversion adapter from zcode interaction requests to ACP (requestPermission + elicitation form) |
 
 ### `config/` — Configuration management
 
-| File | Responsibility |
-|------|------|
-| `options.ts` | configOptions / modes construction, set_config_option dispatch |
-| `runtime-model.ts` | runtimeModel overlay construction and application |
-| `model-cache.ts` | Model ID cache and usage initialization |
+| File               | Responsibility                                                 |
+| ------------------ | -------------------------------------------------------------- |
+| `options.ts`       | configOptions / modes construction, set_config_option dispatch |
+| `runtime-model.ts` | runtimeModel overlay construction and application              |
+| `model-cache.ts`   | Model ID cache and usage initialization                        |
+
+### `remote/` — Remote access (opt-in via `ZCODE_ACP_REMOTE=1`)
+
+| File            | Responsibility                                                                                                                                               |
+| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `broadcast.ts`  | ClientRegistry + broadcast proxy: notify fans out to all clients; request is first-response-wins with loser `$/cancel_request`                               |
+| `config.ts`     | ENV parsing (gate, mandatory token, hub/bridge ports)                                                                                                        |
+| `endpoint.ts`   | Loopback ACP endpoint (SDK AcpServer transport, port auto-increment) + hub registration/heartbeat                                                            |
+| `hub-server.ts` | The hub singleton: token auth, instance discovery, byte-level WS proxying, heartbeat pruning, on-demand `?probe=1` liveness, idle exit, version self-upgrade |
+
+When enabled, the same `AgentApp` serves the stdio editor and a loopback
+WebSocket endpoint. Every connection (editor or remote) joins the broadcast
+registry via `trackConnections`, so one turn's notifications reach all clients
+regardless of who prompted. The bridge registers itself with the machine-level
+`zcode-acp-hub` (`bin/hub.ts`), which is the only public entry point and holds
+no session state (see `docs/adr/0002`). The bridge's lifetime still follows the
+stdio client (ADR-0001); the listener is `unref()`'d so remote clients alone
+never keep the process alive.
+
+Hub upgrades are self-managing: each heartbeat carries the bridge's package
+version, and a hub that sees a NEWER bridge replies `{ok, restarting}`, exits,
+and is re-spawned by that bridge from its own (upgraded) `dist/` within a few
+seconds. Equal, older, or absent versions never trigger a restart — downgrades
+and mixed-version fleets are fine. Without this handshake a long-lived hub
+would keep running pre-upgrade code until its 10-minute idle exit.
+
+Discovery liveness has two layers: the heartbeat TTL (30s, pruned every 5s)
+drops bridges that stopped registering — the fallback for hard kills — and
+`GET /api/instances?probe=1` actively TCP-probes each registered loopback port
+on demand, so a client refresh gets an immediately-honest list with no
+background probing cost.
 
 ## Key State Machines
 
@@ -222,12 +312,12 @@ mirroring the Python bridge's `_pending_post_notifs` queue +
 The session mode can change through four entry points, all of which must
 notify the editor UI:
 
-| Trigger | Path | Notifies UI |
-|------|------|:---:|
-| `session/setMode` request | `extensions.ts:setMode` | yes |
-| `session/set_config_option` (mode) | `session.ts:setConfigOptionHandler` → `emitConfigOptionUpdate` | yes |
-| `/mode` slash command | `slash.ts` → `emitConfigOptionUpdate` | yes |
-| In-turn `EnterPlanMode`/`ExitPlanMode` | reconciled at turn completion | yes |
+| Trigger                                | Path                                                           | Notifies UI |
+| -------------------------------------- | -------------------------------------------------------------- | :---------: |
+| `session/setMode` request              | `extensions.ts:setMode`                                        |     yes     |
+| `session/set_config_option` (mode)     | `session.ts:setConfigOptionHandler` → `emitConfigOptionUpdate` |     yes     |
+| `/mode` slash command                  | `slash.ts` → `emitConfigOptionUpdate`                          |     yes     |
+| In-turn `EnterPlanMode`/`ExitPlanMode` | reconciled at turn completion                                  |     yes     |
 
 The in-turn path bypasses the bridge entirely, so `prompt()` runs
 `emitModeIfChanged` (`session.ts`) at turn completion: it re-reads the
@@ -277,12 +367,12 @@ zcode process exits, so a normal shutdown leaves no lingering watchdog.
 
 ### Why a dual path?
 
-| Scenario | Real-time path | Snapshot path |
-|------|---------|----------|
-| Normal streaming | Low latency | Must wait for turn end |
-| Lost events | Loses data | Recovers from snapshot |
-| Deduplication | seenToolIds | seenMessageIds + markToolSeen() |
-| Turn-completion triage | Not triggered | PlanUpdate / usage_update |
+| Scenario               | Real-time path | Snapshot path                   |
+| ---------------------- | -------------- | ------------------------------- |
+| Normal streaming       | Low latency    | Must wait for turn end          |
+| Lost events            | Loses data     | Recovers from snapshot          |
+| Deduplication          | seenToolIds    | seenMessageIds + markToolSeen() |
+| Turn-completion triage | Not triggered  | PlanUpdate / usage_update       |
 
 ### Why no polling fallback?
 

package/docs/BACKLOG.md

@@ -0,0 +1,59 @@
+# Protocol Backlog
+
+Backend RPC methods and event types exposed by the ZCode CLI (`zcode app-server`)
+that are **not yet wired into the bridge**, tracked for potential future support.
+
+Last audited against **app-server 0.15.2** (bundled in ZCode desktop 3.4.2,
+2026-07-22). Method names were extracted from the bundled `zcode.cjs`.
+
+## Candidate methods (optional enhancements)
+
+These are available in the backend but have no ACP-side counterpart yet. Pick
+them up when a concrete ACP/editor need appears.
+
+| Method | Purpose | Current bridge behavior |
+|--------|---------|-------------------------|
+| `session/subagents` | Query the list of sub-agents for a session | Sub-agent info is parsed from the `Agent` tool result (`_meta.subagent`); sufficient for now |
+| `session/events` | Pull-mode event history (complement to `session/subscribe`) | Not used; could support event replay/gap-fill |
+| `session/usage` | Per-session token usage | Context bar uses `session.updated` usage payload instead |
+| `session/close` | Explicitly close a session (vs `session/stop` which ends a turn) | Not used; sessions close on process exit |
+
+### New event types (undocumented in PROTOCOL.md)
+
+| Event | Status field | Notes |
+|-------|--------------|-------|
+| `turn.steerQueued` | — | Emitted when a `session/steer` is queued behind a running turn. Enhances steer UI feedback |
+| `turn.steerDrained` | — | Emitted when queued steer instructions are drained into the turn |
+| `turn.terminal` | `status: success \| interrupted \| failed`, `resultType?`, `durationMs`, `...usage` | Terminal turn lifecycle event; currently the bridge relies on `turn.completed`/`turn.failed` |
+
+## Not planned (client/config layer)
+
+These methods belong to the desktop client or workspace configuration layer and
+have no ACP equivalent. Listed for completeness only — the bridge does not
+intend to surface them.
+
+`automation/create`, `automation/list`, `automation/delete` (scheduled tasks),
+`usage/stats` (token analytics; the account-level plan quota it does NOT cover
+is exposed via the bridge's own `account/usage_stats` — see Proposal 0002),
+`workspace/readState`, `workspace/upsertModelProvider`,
+`workspace/removeModelProvider`, `workspace/updateProviderRegistry`,
+`workspace/setDefaultModel`, `workspace/setDefaultThoughtLevel`,
+`workspace/setDefaultMode`, `workspace/generateText`, `mcp/list`,
+`plugins/list`, `plugins/setEnabled`, `plugins/overview`, `plugins/describe`,
+`plugins/marketplace/*`.
+
+## Verification method
+
+The bundled CLI is minified, so a literal `grep "session/rewind"` returns 0
+hits even when the method is fully supported — method names are split across
+variable references. To audit reliably:
+
+```sh
+cd /Applications/ZCode.app/Contents/Resources/glm
+# Full RPC enum: search for the method-dispatch table (Key:"ns/name" pairs)
+python3 -c "import re; ..."   # see commit that added this file
+# Word-level presence is more reliable for verifying a method still exists:
+grep -oiF "rewind" zcode.cjs | wc -l   # 357 → definitely present
+```
+
+Never conclude a method was removed from a single string-literal search.

package/docs/DEVELOPMENT.md

@@ -90,6 +90,32 @@ describe("backend", () => {
 });
 ```
 
+### Method 4: Remote access (hub + endpoint)
+
+Run the bridge with the remote env vars on scratch ports (keep stdin open —
+the bridge exits when its stdio client disconnects):
+
+```bash
+ZCODE_ACP_REMOTE=1 \
+ZCODE_ACP_REMOTE_TOKEN=dev-token \
+ZCODE_ACP_HUB_PORT=18377 \
+ZCODE_ACP_REMOTE_PORT=18378 \
+ZCODE_ACP_DEBUG=1 \
+tail -f /dev/null | node dist/index.js
+```
+
+The first bridge spawns `dist/bin/hub.js` on its own. Verify discovery:
+
+```bash
+curl -H "Authorization: Bearer dev-token" http://127.0.0.1:18377/api/instances
+curl -H "Authorization: Bearer dev-token" "http://127.0.0.1:18377/api/instances?probe=1"
+```
+
+Integration tests live in `tests/hub.test.ts` (hub API, probe, proxy, idle
+exit) and `tests/remote-endpoint.test.ts` (registration + end-to-end proxied
+initialize). The remote features are opt-in and self-contained — remote
+failures warn and disable remote only, so stdio testing works without them.
+
 ## Debugging Tips
 
 ### Enable verbose logging
@@ -117,6 +143,11 @@ cat ~/.zcode/v2/config.json
 # confirm a provider is enabled and has models
 ```
 
+The bridge reads the GLM API key from this file and forwards it to the ZCode
+subprocess — no editor-side API key or environment variable is required. If
+`config.json` is missing or no provider is enabled, install and log into the
+ZCode desktop app so it writes a valid file.
+
 ### Test session/subscribe
 
 ```bash