@oar-sh/oar 0.9.13 → 0.9.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. package/.github/extensions/web-relay/extension.mjs +9 -0
  2. package/.github/extensions/web-relay/server-lifecycle/managed-server.mjs +15 -1
  3. package/.github/extensions/web-relay/skills/question-bridge.mjs +4 -1
  4. package/CHANGELOG.md +302 -0
  5. package/README.md +12 -12
  6. package/bin/oar.js +155 -94
  7. package/package.json +5 -5
  8. package/server/README.md +296 -149
  9. package/server/claude-cloud-worker/claude-cloud-session-process.mjs +10 -4
  10. package/server/claude-worker/claude-session-process.mjs +39 -13
  11. package/server/claude-worker/claude-turn-publisher.mjs +20 -1
  12. package/server/claude-worker/subagent-timeline-recorder.mjs +75 -8
  13. package/server/copilot-worker/copilot-ask-user-tool.mjs +1 -1
  14. package/server/copilot-worker/copilot-prompt-context.mjs +3 -1
  15. package/server/copilot-worker/copilot-question-bridge.mjs +11 -6
  16. package/server/copilot-worker/copilot-sdk-adapter.mjs +11 -0
  17. package/server/copilot-worker/copilot-sdk-session-process.mjs +14 -1
  18. package/server/cursor-worker/cursor-ask-user-tool.mjs +1 -1
  19. package/server/cursor-worker/cursor-sdk-adapter.mjs +22 -7
  20. package/server/cursor-worker/cursor-session-worker.mjs +9 -2
  21. package/server/cursor-worker/cursor-turn-runner.mjs +18 -3
  22. package/server/grok-worker/grok-sdk-adapter.mjs +14 -0
  23. package/server/grok-worker/grok-turn-runner.mjs +13 -3
  24. package/server/public/app/api-client.js +3 -1
  25. package/server/public/app/ask-user-view.js +29 -4
  26. package/server/public/app/conversation-provider-indicator.mjs +18 -8
  27. package/server/public/app/question-answer-rule.mjs +15 -0
  28. package/server/public/app/question-multi-select.mjs +9 -2
  29. package/server/public/app/relay-event-view.mjs +5 -1
  30. package/server/public/index.html +9 -5
  31. package/server/relay-exit-codes.mjs +6 -0
  32. package/server/relay-self-restart.mjs +6 -1
  33. package/server/relay-tools.md +14 -11
  34. package/server/repositories/message-repository.mjs +29 -28
  35. package/server/repositories/subagent-timeline-statements.mjs +2 -0
  36. package/server/routes/ask-user-routes.mjs +41 -7
  37. package/server/routes/messages-routes.mjs +65 -10
  38. package/server/routes/preview-routes.mjs +73 -7
  39. package/server/routes/sessions-routes.mjs +4 -2
  40. package/server/server-runtime.mjs +156 -94
  41. package/server/services/cursor-session-root-service.mjs +70 -6
  42. package/server/services/data-dir-resolution.mjs +208 -19
  43. package/server/services/earlier-database-service.mjs +631 -0
  44. package/server/services/oar-cli-helpers.mjs +6 -1
  45. package/server/services/oar-state-layout.mjs +107 -0
  46. package/server/services/oar-state-migration-service.mjs +387 -96
  47. package/server/services/package-data-rescue-preinstall.mjs +22 -0
  48. package/server/services/package-data-rescue.mjs +137 -0
  49. package/server/services/preview-registry-service.mjs +3 -2
  50. package/server/services/push-dispatch-service.mjs +33 -6
  51. package/server/services/relay-event-service.mjs +183 -5
  52. package/server/services/relay-restart-queue.mjs +10 -3
  53. package/server/services/relay-singleton-guard.mjs +300 -36
  54. package/server/services/remote-relay-caller-context.mjs +7 -2
  55. package/server/services/remote-relay-dispatcher.mjs +30 -9
  56. package/server/services/session-activity-tracker.mjs +52 -15
  57. package/server/services/session-worker-launch-service.mjs +31 -1
  58. package/server/services/stale-turn-recovery.mjs +99 -0
  59. package/server/services/subagent-timeline-service.mjs +4 -1
  60. package/server/services/windows-autostart-service.mjs +329 -31
  61. package/server/services/windows-boot-autostart-service.mjs +188 -60
  62. package/shared/preview-instructions.mjs +17 -12
  63. package/shared/preview-tool-core.mjs +32 -14
  64. package/shared/question-answer-rule.mjs +63 -0
  65. package/shared/relay-events.mjs +54 -4
  66. package/shared/remote-relay-contract.mjs +20 -10
  67. package/shared/subagent-timeline.mjs +4 -0
  68. package/shared/windows-system-tools.mjs +25 -0
  69. package/shared/worker-runtime/config-loader.mjs +54 -8
  70. package/shared/worker-runtime/prompt-context.mjs +3 -1
  71. package/shared/worker-runtime/working-directory.mjs +65 -0
  72. package/server/uploads/image_input.md +0 -540
package/server/README.md CHANGED
@@ -278,8 +278,8 @@ own, never the relay-wide default:
278
278
  level. OpenAI, Claude, Cursor and Grok conversations take the conversation's
279
279
  effort the same way; with no effort stored, or one that neither matches nor
280
280
  clamps (an `ultracode` on a model without it), they keep the provider's own
281
- default as before (`none`, which for Claude, Cursor and Grok means the
282
- model's default). A Claude Cloud conversation always runs with `none`: the
281
+ default as before (`none`, `low` for an OpenAI o-series model; for Claude,
282
+ Cursor and Grok `none` means the model's default). A Claude Cloud conversation always runs with `none`: the
283
283
  cloud takes no effort. A board action's follow-up takes the conversation's
284
284
  effort as well.
285
285
  - **Mode**: a send that names no `relayMode`/`mode` runs in the
@@ -298,19 +298,25 @@ longer counts: sending it again queues it.
298
298
 
299
299
  ### OpenAI API key
300
300
 
301
- Open **Settings → OpenAI API key** to save an OpenAI key and exact model ID
302
- (`gpt-4o` by default). Use the provider toggle to switch new conversations
303
- between OpenAI and GitHub Copilot without re-entering the saved key. While
304
- enabled, newly created extension-managed conversation workers start with Copilot
305
- CLI BYOK environment variables pointing to `https://api.openai.com/v1`.
306
- Zero-message conversations follow provider changes; started conversations keep
307
- their original provider and lock the composer to their assigned model.
308
-
309
- GPT-5 and OpenAI reasoning-model sessions use the Responses API automatically;
310
- other OpenAI models retain the broader-compatible Chat Completions wire format.
311
- OpenAI BYOK sessions currently use reasoning effort `none`; GitHub Copilot
312
- reasoning choices remain separate even when both providers expose the same model
313
- ID.
301
+ Open **Settings → Providers → OpenAI** to save an OpenAI API key, the default model ID (`gpt-4o`
302
+ if none) and an optional **Base URL** (`https://api.openai.com/v1` when empty; any
303
+ OpenAI-compatible endpoint, or the capture proxy `server/tools/capture-proxy.mjs`). **Enable OpenAI
304
+ API key for New Chat model selection** makes OpenAI selectable in New Chat without re-entering the
305
+ key. An OpenAI conversation runs on the Copilot engine the relay has selected, extension or SDK:
306
+ the worker is started with `COPILOT_PROVIDER_TYPE=openai`, `COPILOT_PROVIDER_BASE_URL` (the saved
307
+ base URL), `COPILOT_PROVIDER_API_KEY`, `COPILOT_PROVIDER_WIRE_API` and `COPILOT_MODEL`
308
+ (`applyOpenAIProviderEnvironment`); the CLI reads them on the extension engine, the SDK worker
309
+ turns them into the session's provider (`copilot-worker/copilot-byok-provider.mjs`). Started
310
+ conversations keep their provider and lock the composer to their model.
311
+
312
+ GPT-5, `codex-*` and the o1/o3/o4 models use the Responses API; other OpenAI models use Chat
313
+ Completions (`resolveOpenAIWireApi`). The reasoning efforts follow the model family
314
+ (`shared/openai-reasoning.mjs`): GPT-5 and `codex` offer `none`, `low`, `medium`, `high` and
315
+ `xhigh`, the o-series `low`, `medium` and `high`, other models only `none` (image models take
316
+ `auto`, `low`, `medium` and `high` as their quality). An effort the model does not offer is refused
317
+ with `400`; with none named the conversation's own applies (clamped to the model's levels), else
318
+ `none`, or `low` on a model without `none`. GitHub Copilot's reasoning choices stay separate even when both providers list the
319
+ same model ID.
314
320
 
315
321
  Saving or re-enabling the key refreshes OpenAI's `/v1/models` list and adds
316
322
  candidate OpenAI model IDs to the composer picker. The Settings model-variant
@@ -322,11 +328,13 @@ the Settings model is only the default. Starting a chat from a locked OpenAI
322
328
  conversation opens an independent model chooser without changing the active
323
329
  conversation.
324
330
 
325
- Disabling OpenAI retains the key and cached model list. Removing the key is a
326
- separate destructive action and makes subsequent conversations use GitHub
327
- Copilot.
328
- An existing OpenAI-assigned conversation can continue while its worker remains
329
- running, but cannot launch a replacement worker until an OpenAI key is configured.
331
+ Disabling OpenAI retains the key and cached model list. **Remove saved key** deletes the key and the
332
+ cached list and switches OpenAI off, and it is refused with `409 openai-key-removal-blocked`
333
+ (`activeConversationCount`, `startedConversationCount`, `activeQueueConversationCount`) while any
334
+ OpenAI conversation that is not deleted, image conversations and archived ones included, has a
335
+ message or a pending, processing or parked queue row (`setOpenAIProviderSettings` in
336
+ `server-runtime.mjs`). Disable OpenAI instead, or delete those conversations first. A removal that
337
+ goes through moves the OpenAI conversations without a message to GitHub Copilot.
330
338
 
331
339
  ### Claude models
332
340
 
@@ -541,12 +549,20 @@ runs in the background, and who writes its timeline.
541
549
  outlives the turn that spawned it, so its frames arrive between turns and during later turns;
542
550
  they go to its own record, never to a later turn's live lanes. A background launch's receipt
543
551
  (`tool_use_result.status: 'async_launched'`) leaves the run running; its `task_notification`
544
- ends it. Spawns carry `timelineOwner: 'worker'`.
552
+ ends it. Spawns carry `timelineOwner: 'worker'`. A finished background agent that the session
553
+ sends a new message (SendMessage) runs again as the same task: `task_started` names its task id
554
+ again, with a new `run_id` and the SendMessage call as `tool_use_id`. The recorder leads that id
555
+ back to the agent's first run, so the record goes on under the same run (a note marks where it
556
+ resumed, and the run reads `running` again until its next `task_notification`); the task panel
557
+ row, the settle notice and a targeted stop name that run too. Every timeline post also names the
558
+ run's `taskId`, for a worker that did not see the first run (see the route below).
545
559
  - **Copilot and Cursor** publish their sub-agents on the live lanes only (`/api/activity`,
546
560
  `/api/thought`, `/api/stream` with a `subagentRunId`); the relay copies each frame into the run's
547
561
  timeline, uncut. Fields they do not report (model, tokens) stay blank.
548
562
  - `POST /api/background-tasks` copies model, type, task id and tokens onto the run whose id is the
549
- task's `toolUseId`, so the row keeps them after the in-memory task set is cleared.
563
+ task's `toolUseId`, so the row keeps them after the in-memory task set is cleared. A sub-agent
564
+ task whose `toolUseId` is no run (a resumed agent from a worker that did not see its first run)
565
+ gets the id of the conversation's run that carries its task id, so the row opens that record.
550
566
  - Every `subagent_status` event carries the whole header and the run's own turn (`messageId`). A
551
567
  turn that fails closes its foreground runs; background runs are closed by their worker, or by the
552
568
  relay when the worker dies.
@@ -704,13 +720,15 @@ transaction as the conversation. Nothing is created at Anthropic here. `POST /ap
704
720
  `providerType: 'claude-cloud'` on a conversation of another provider is `409
705
721
  CLAUDE_CLOUD_REQUIRES_NEW_CONVERSATION`.
706
722
 
707
- `POST /api/message` accepts another model of the list until the first message has created the
708
- cloud session, and stores it on the runtime session; after that a different model is `409
709
- CLAUDE_CLOUD_MODEL_REQUIRES_NEW_CONVERSATION`. The worker, however, creates the session with the
710
- model it was launched with: `resolveModel` prefers `CLAUDE_CLOUD_RELAY_MODEL` over the delivered
711
- `providerModel`, so a change made after the worker started only takes effect if the worker is
712
- launched again before the first message. Reasoning effort is always `none` and the context tier
713
- always `default`, whatever the request says.
723
+ `POST /api/message` accepts any model of the list, before and after the first message, and stores
724
+ it on the runtime session; a model outside the list is `400 CLAUDE_CLOUD_MODEL_UNAVAILABLE`. The
725
+ delivery carries the conversation's model as `providerModel`, which the worker's `resolveModel`
726
+ prefers over the message's own model and over `CLAUDE_CLOUD_RELAY_MODEL` (the launch model). The
727
+ first message creates the session with it; before a later message whose model is not the one the
728
+ worker last gave the sandbox, the worker sends a `set_model` control request (`applyModel`; after a
729
+ worker restart once, since it cannot know). A `set_model` the API refuses costs the switch, not the
730
+ turn. Reasoning effort is always `none` and the context tier always `default`, whatever the request
731
+ says.
714
732
 
715
733
  ### Delivery, binding and resume
716
734
 
@@ -784,8 +802,10 @@ Per-turn behavior:
784
802
  as a terminal error result even though the API key is still valid, so the worker recreates the
785
803
  handle and retries a single time; a second auth failure is treated as a real key problem.
786
804
  - **Session root** — the worker keeps its per-session agent store at
787
- `$CURSOR_AGENT_STORE_DIR` or `server/data/cursor-agents/<sdkSessionId>`, created on the first turn.
788
- The explorer's Session root resolves to it via `resolveCursorSessionRoot`.
805
+ `$CURSOR_AGENT_STORE_DIR/<sdkSessionId>`, else `cursor-agents/<sdkSessionId>` in the relay's data
806
+ dir (`~/.oar/data` on a global install, `server/data` in a checkout), created on the first turn. A
807
+ session with a store at the earlier default `server/data/cursor-agents` keeps using it there when
808
+ it has none in the data dir, or when its `agent.db` there changed last. The explorer's Session root resolves to it via `resolveCursorSessionRoot`.
789
809
  - Cursor turns are skipped by `fetchUsageSummary` like OpenAI and Claude turns — no Copilot usage
790
810
  line is attached.
791
811
 
@@ -800,12 +820,27 @@ carries its own expiry.
800
820
  began. A turn that legitimately runs for hours of tool calls keeps pushing the cutoff forward.
801
821
  `processing_at` is only the fallback for rows with no owner (session-worker routing disabled).
802
822
  While the CLI is offline the window shrinks to 45 s.
803
- 2. **Absolute ceiling (Settings → Max turn duration, default 60 min).** Measured from `processing_at`.
804
- `0` disables it; the slider range is 0–600 minutes in 5-minute steps. This is a backstop for a
805
- worker that has hung while still heartbeating, and is stored in the DB app setting
806
- `turn_ceiling_minutes` — not in `config.json`. Shared bounds and formatting live in
823
+ 2. **Absolute ceiling (Settings → Max turn duration, default 60 min).** Measured from the later of
824
+ `processing_at` and the turn's last answered question (`relay_questions.answered_at`): answering
825
+ a question restarts the clock, so a long interview is not cut off the moment its last answer
826
+ arrives. `0` disables it; the slider range is 0–600 minutes in 5-minute steps. This is a
827
+ backstop for a worker that has hung while still heartbeating, and is stored in the DB app
828
+ setting `turn_ceiling_minutes` — not in `config.json`. Shared bounds and formatting live in
807
829
  `shared/turn-ceiling.mjs`.
808
830
 
831
+ Both guards are one SQL rule in `message-repository.mjs`, shared by the listing
832
+ (`listRecoverableProcessing`) and the requeue (`recoverProcessingBefore`, one row at a time,
833
+ re-checking the rule as a safeguard; the sweep runs synchronously, so nothing changes between
834
+ the two). The sweep (`services/stale-turn-recovery.mjs`) applies the retry budget to the listed
835
+ rows, requeues the rest and announces each one it requeued with `message_status`.
836
+
837
+ The sweep has no provider filter, so Grok rows are requeued by the rule above too. The Grok worker
838
+ also enforces the ceiling itself (delivered with each queue message, `grok-worker/acp-client.mjs`
839
+ `sessionPrompt`): counted from when it sent the prompt, it cancels the turn and fails it as
840
+ `grok.turn-stalled`. That check does not pause for question cards (on Grok: remote-relay approval
841
+ cards through the OAR MCP server), and an answer does not restart its clock. Whichever trips first
842
+ ends the turn.
843
+
809
844
  ## Subagent runs and live streaming
810
845
 
811
846
  - `POST /api/stream` and `POST /api/activity` / `POST /api/thought` accept an optional
@@ -845,6 +880,9 @@ be on, as for every local action; a paired relay: see [Across paired relays](#ac
845
880
  watched. The first subscribe of a conversation shows the user an approval card naming the
846
881
  sessions; a subscription to `all` is always named on a card (asked again after an Allow that did
847
882
  not cover it). The Allow is remembered as unlock rows `self:watch` / `self:watch-all`.
883
+ `events: ["relay_restarted"]` alone needs no sessions, watches none and shows no card (see
884
+ [relay_restarted](#relay_restarted-woken-when-the-relay-runs-again)); on a paired relay it is
885
+ refused, by the tool and by that relay's subscribe route.
848
886
  - `unsubscribe {subscription_id | "all"}`, `list_subscriptions {watcher?}` (another conversation's
849
887
  subscriptions, to inspect before a transfer).
850
888
  - `transfer_subscriptions {from}` — moves every active subscription of another conversation, with
@@ -863,7 +901,7 @@ turn or task runs and no question waits for an answer), `all_idle`
863
901
  `subagent_finished` is no event of its own but a narrower `task_finished` a subscription can name
864
902
  (`RELAY_EVENT_SUBSCRIBABLE_TYPES`): the end of a task that is not a shell command, delivered as a
865
903
  `task_finished` line. The default (`events` left out) is every type but `task_started` and
866
- `task_finished`, with `subagent_finished`: a shell command that ended well wakes only a watcher
904
+ `task_finished` (and `relay_restarted`), with `subagent_finished`: a shell command that ended well wakes only a watcher
867
905
  that names `task_finished`, which takes every task end. A busy worker ends a shell command every
868
906
  minute or so, and each was a wake-up, a model turn on the watcher's whole context. A subscription
869
907
  whose stored list is exactly the old default (every type but `task_started`, in any order) named no
@@ -919,6 +957,49 @@ touched for a day is not counted as running meanwhile.
919
957
  and the background-only worker death (worker_died), `/api/relay-question` and the approval card's
920
958
  notify (question_asked), the activity tracker's task edges, and the archive and delete routes.
921
959
 
960
+ ### relay_restarted: woken when the relay runs again
961
+
962
+ `relay_restarted` is an event of this relay, not of a session (`RELAY_EVENT_RELAY_LEVEL_TYPES`):
963
+ a subscription gets it only when it names it, never by default, and one that names nothing else
964
+ needs no sessions. An agent that is about to request a restart subscribes with `once: true` first:
965
+ `POST /api/relay/shutdown` has no caller identity (agents call it with the token, `requestedBy` is
966
+ free text, and agents do not know their conversation id), while the tool knows the calling conversation.
967
+
968
+ - **How the run before ended.** A clean end of the runtime writes one `app_settings` row
969
+ (`relay_last_exit`: `action`, `reason`, `requested`, `requestedBy`, `requestedAt`, `stoppedAt`,
970
+ `noWake`): the queued (or `immediate`) request from the restart queue's `startShutdown`, else
971
+ `shutdownRuntime` (a signal, the owner process gone). The first note of a runtime wins; a runtime
972
+ stopped before it listened writes none, so the record of the run before stays for the next start.
973
+ - **At startup** (`noteRelayStarted`, once the HTTP server listens: a start that fails to bind
974
+ leaves the row for the next one) the relay reads and deletes that row in the same transaction
975
+ that records one `relay_restarted` event for every active subscription of this relay's live
976
+ watchers that names it (not a paired relay's, not a mirror), so each start is told once; a
977
+ `once` subscription ends with it. Without a row the line says the relay did not record how it
978
+ stopped (for example a crash, a forced stop or a reboot). On Windows a stop by the supervisor or
979
+ the CLI extension ends the process without its signal handler, so an ordinary stop there may read
980
+ like that too. A row older than 30 days (`RELAY_LAST_EXIT_MAX_AGE_MS`) is not taken as the run
981
+ just before (an older OAR, which neither writes nor takes it, may have run in between): it is
982
+ dropped, its `noWake` with it. A younger one is used; the line shows its stop time, so a relay
983
+ stopped over a weekend still says how. Delivery is that of every event: coalesced for 30 s (the workers
984
+ report back first), counted toward the hourly cap, written into an event row that is still
985
+ pending.
986
+ - **Loop guard.** On every `POST /api/relay/shutdown` (`noteRestartRequested`) the relay notes the
987
+ conversations whose processing turn answers a `relay_restarted` event (its queue row's
988
+ `relay_events`); they go into `noWake` and are not woken by that restart (logged), so a woken
989
+ turn that restarts again ends the chain. That holds for any restart requested while such a turn
990
+ runs (the button, an update, another agent): the API cannot tell who asked. The line says so up
991
+ front. A cancelled (or never queued) request forgets them. A held-back `once` subscription ends
992
+ (`held-back`, logged): that restart was the one it was for, and a later one would wake it with a
993
+ stale "carry on". If it also watches sessions, that ends with it: their events stop too. One
994
+ without `once` stays, and a later restart requested while no such turn
995
+ runs wakes it again. `heartbeat_minutes` on a subscription without sessions is refused (nothing
996
+ runs in an empty set).
997
+ - The line: `[relay event] relay_restarted: the relay runs again since <time>, after a restart
998
+ requested by "<requestedBy>" at <time> (reason "<reason>"). Timers and jobs your CLI scheduled
999
+ before may be gone: check what you were waiting for and carry on. A restart requested while this
1000
+ turn runs does not wake you again.` A stop by signal reads `after it was stopped (SIGTERM) at
1001
+ <time>` (a process id in the runtime's own reason is left out).
1002
+
922
1003
  ### Across paired relays
923
1004
 
924
1005
  The watching relay (**A**, it holds the watcher) and the watched one (**B**) each keep a row of the
@@ -985,8 +1066,11 @@ finished tasks and starts the clock at boot. Running tasks come back when the wo
985
1066
  sets again (see [Background tasks and workflow progress](#background-tasks-and-workflow-progress)):
986
1067
  a task that started before this runtime did is not announced as started a second time, and the end
987
1068
  of a task this runtime never knew is still an event when the settle notice describes the task
988
- (`description`, `taskType`, `startedAt`). `all_idle` waits out the first 45 seconds after a relay
989
- start, like the suspend check.
1069
+ (`description`, `taskType`, `startedAt`). A task that ended stays ended when a later set lists it
1070
+ again, unless the set names another run of it (`taskRunId`, the SDK's `run_id`): a resumed Claude
1071
+ sub-agent keeps its task id, so its new run is a start (and later an end) of its own, and a settle
1072
+ notice of the earlier run that arrives late does not end it. `all_idle` waits out the first 45
1073
+ seconds after a relay start, like the suspend check.
990
1074
 
991
1075
  `GET /api/conversations` and `GET /api/conversation/:id` carry `sessionState` (`running_turn`: a
992
1076
  turn processing, or queued and not held by a usage-limit pause; `background_only`: tasks running;
@@ -1108,7 +1192,8 @@ The composer also includes a per-message mode picker:
1108
1192
 
1109
1193
  Mode is stored with each queued message so the relay can change behavior per turn.
1110
1194
  Clarification prompts from the CLI are forwarded back into the browser as question
1111
- cards with a reply box.
1195
+ cards with a reply box; a card created with `allowFreeform: false` (approval and
1196
+ permission cards) has none and is answered with its choices.
1112
1197
 
1113
1198
  `plan` mode behavior:
1114
1199
  - Draft a concise implementation plan only (no implementation output).
@@ -1188,60 +1273,92 @@ Question bridge rule:
1188
1273
 
1189
1274
  ## Session Worker Management
1190
1275
 
1191
- Session workers are long-running CLI processes that execute turns for Copilot agent sessions. The relay manages a pool of workers to enable:
1192
- - Parallel execution (multiple sessions can run simultaneously)
1193
- - Session persistence (a worker stays alive across turns for a given session)
1194
- - Graceful restarts (degraded workers are replaced without losing queued work)
1195
-
1196
- ### Worker Lifecycle States
1197
-
1198
- 1. **Spawned** — Worker process created, initializing CLI connection
1199
- 2. **Ready** — Worker opened WebSocket to relay, registered in worker pool, ready to accept turns
1200
- 3. **Processing** — Worker actively executing a turn (streaming activity to browser)
1201
- 4. **Degraded** — Worker failed heartbeat or reported errors; relay moves pending work to other workers or restarts
1202
- 5. **Dead/Exited** — Worker process terminated; polling loop will spawn replacement if turns still pending
1203
-
1204
- ### Worker Heartbeat & Health Check
1205
-
1206
- - Workers send keep-alive pings every 30 seconds (default) to `/api/workers/{id}/heartbeat`
1207
- - Relay tracks last heartbeat time for each worker
1208
- - If a worker misses 2+ consecutive heartbeats (~60 seconds), it's marked "degraded"
1209
- - Degraded workers don't receive new turns; existing work is reassigned
1210
- - Polling loop detects degraded workers and respawns them if they're still needed
1211
-
1212
- ### Supervisor Service
1213
-
1214
- The **session-worker-supervisor-service** tracks all active workers:
1215
-
1216
- - Maintains registry: `workers[sdkSessionId] = { id, process, sdkSessionId, status, lastHeartbeat }`
1217
- - Monitors process health: checks if process is still running, collects exit codes
1218
- - Handles restarts: when a worker fails, spawns a replacement with exponential backoff
1219
- - Cleans up: removes dead workers from registry after TTL (default 5 minutes)
1220
-
1221
- ### Environment & Workspace
1222
-
1223
- Session workers inherit:
1224
-
1225
- - `COPILOT_WORKSPACE_ROOT` — Target workspace directory (from turn metadata or session config)
1226
- - `INIT_CWD` — Initial working directory (fallback for workspace resolution)
1227
- - `COPILOT_REMOTE_*` — All relay config via environment variables
1228
- - CLI session state from the last known good checkpoint
1229
-
1230
- ### Restart Orchestration
1231
-
1232
- When a worker needs restart:
1233
-
1234
- 1. **Graceful shutdown** — Send SIGTERM, wait up to 5 seconds for clean exit
1235
- 2. **Kill if needed** — Force SIGKILL if SIGTERM timeout
1236
- 3. **Backoff** — Wait before respawn (start at 1s, double up to 30s max)
1237
- 4. **Reassign work** — Any pending turns for that session are placed back in queue for the new worker
1238
- 5. **Report status** — Relay broadcasts worker status to browser UI (shows "reconnecting..." briefly)
1239
-
1240
- ### Platform-Specific Behavior
1241
-
1242
- - **macOS/Linux:** Workers prefer `tmux` sessions for better isolation and debugging (can attach with `tmux attach -t {sessionId}`)
1243
- - **Windows:** Workers launch in a new Terminal window (TTY-Console if available, otherwise cmd)
1244
- - Both: Worker inherits relay's environment and runs from relay repo root so project-local extensions are discoverable
1276
+ Each conversation has its own worker process, keyed by the conversation's SDK session id: one
1277
+ worker per session, and a session's turns never move to another worker. A Copilot conversation on
1278
+ the *Extension* engine runs the `copilot` CLI with the web-relay extension; every other one
1279
+ (Claude, Claude Cloud, Cursor, Grok, and Copilot on the *SDK* engine; OpenAI conversations run on
1280
+ whichever Copilot engine is selected) runs a plain Node worker (`NODE_WORKER_DESCRIPTORS` in
1281
+ `services/session-worker-launch-service.mjs`). The supervisor
1282
+ (`services/session-worker-supervisor-service.mjs`) keeps the lifecycle, health and restart limits,
1283
+ the launch service spawns, `services/session-worker-stop-service.mjs` stops, and the dead-worker
1284
+ recovery lives in `server-runtime.mjs`. Most of it acts only while the
1285
+ `SESSION_WORKER_ROUTING_ENABLED` feature flag is on.
1286
+
1287
+ ### Lifecycle and health
1288
+
1289
+ - Registry status: `starting` → `ready` ⇄ `processing`, and `error`. A send, the primer for queued
1290
+ messages and `POST /api/session-worker/:sdkSessionId/launch` call `ensureWorker`, which reuses a
1291
+ live process of the session (found in the process list, or the tmux pane) and launches one
1292
+ otherwise.
1293
+ - Heartbeats: every worker posts `POST /api/heartbeat` every 10 s, and any frame on its worker
1294
+ socket counts as one too. The supervisor uses them only at the start: a launched worker that has
1295
+ sent none within 90 s (`heartbeatTimeoutMs`; a Copilot CLI takes about 35 s to boot) is degraded
1296
+ as `startup-heartbeat-timeout`. After its first heartbeat a worker is judged by its pid, not by
1297
+ missed beats.
1298
+ - Health (`uiState`, in `/api/status` with the worker list): `green` healthy, `red` a question card
1299
+ waits, `yellow` degraded, `white` nothing known. `degradedReason` is `startup-heartbeat-timeout`,
1300
+ `stale-pid` (the pid is gone while a turn or a question waits for the worker), `spawn-failed`,
1301
+ `restart-exhausted` or the worker's own last error. Yellow clears after 10 s of a healthy worker
1302
+ without a new failure (`degradedRecoveryGraceMs`).
1303
+ - An idle (`ready`) worker whose process is gone and that has not heartbeated for 90 s leaves the
1304
+ list; the next message launches a new one. A live idle worker is never evicted by time
1305
+ (`idleEvictionMs` is 0).
1306
+
1307
+ ### A worker that dies
1308
+
1309
+ - When a worker's socket closes while its session owns a processing row, holds the composer or has
1310
+ background sub-agents running, the relay checks 15 s later (`WORKER_DEATH_GRACE_MS`) whether the
1311
+ process is still alive. A sweep every 30 s checks the owners of all processing rows the same way.
1312
+ - `recoverDeadWorkerProcessingRows` then drops a continuation row, fails a row the worker had
1313
+ already consumed or that is over `maxRequeueRetries` (5), and puts the others back to `pending`
1314
+ (2 s later, same owner session, one retry more). The session counts a failure (`markError`), the
1315
+ dead worker's sub-agent runs are closed, and the primer launches a new worker for the waiting rows.
1316
+ - A worker that is alive but has had no socket for 60 s while a message waits is replaced (see
1317
+ [Worker link reconnects](#worker-link-reconnects)).
1318
+
1319
+ ### Launch failures and backoff
1320
+
1321
+ A launch that throws, a worker that died or a queue row it failed counts a failure: the next launch
1322
+ of that session waits 1 s, doubling per failure (1, 2, 4, 8, 16 s; capped at 30 s), for 5 retries (`maxRestartRetries`).
1323
+ Then the session is `restart-exhausted`: one more launch is allowed 60 s after the last failure,
1324
+ for up to 10 minutes after the first exhaustion, and after that only when a launch is asked for
1325
+ (the conversation's Retry, the launch or kill route, a relay restart). While a session's worker
1326
+ cannot be started the conversation shows a note (`services/worker-launch-note-service.mjs`). A
1327
+ successful launch resets the count.
1328
+
1329
+ ### Stopping a worker
1330
+
1331
+ `stopSessionWorkerProcesses` serves **☠️ Kill session** (`POST /api/session-worker/:sdkSessionId/kill`),
1332
+ the CWD relaunch, a provider rebind and the replacement of a worker without a socket. It reads the
1333
+ session's processes from a fresh process list and checks that they are gone: on Linux and macOS
1334
+ SIGTERM to each and `tmux kill-session`, up to 3 s, then SIGKILL to the survivors and up to 2 s
1335
+ more; on Windows `Stop-Process -Force` on the process tree, leaf first; processes still alive after 3 s
1336
+ are stopped once more, with any new ones on a fresh list, and get up to 2 s more. A process list that cannot be read stops nothing (`process-list-unreadable`), and processes
1337
+ that survive are an error (`409` from the kill route). The kill route fails the rows the worker
1338
+ owned and blocks launches of the session for 3 s (`killBlockGraceMs`), so the primer does not
1339
+ start it again at once; the launch route clears that block.
1340
+
1341
+ ### Launch modes and environment
1342
+
1343
+ - **Linux/macOS with `tmux`:** a detached tmux session named after the SDK session id
1344
+ (`tmux attach -t <sdk-session-id>`), running `sh -lc`: the extension engine's CLI under `script`
1345
+ for a pseudo-TTY, a Node worker with its output appended to its worker log.
1346
+ - **Linux/macOS without tmux:** a detached child process, a Node worker's output into its log.
1347
+ - **Windows:** `cmd /c start` opens one console window per worker, titled `<Kind> Worker <first 8
1348
+ characters of the session id>`; the extension engine runs `gh copilot -- --allow-all
1349
+ --session-id <id>`.
1350
+
1351
+ The launch environment is the relay's own, taken once at boot (`buildSessionWorkerLaunchEnv`), plus
1352
+ per launch the conversation's working directory as `COPILOT_WORKSPACE_ROOT` and `INIT_CWD`,
1353
+ `SESSION_ID`, `COPILOT_ALLOW_ALL`, and the bound provider's worker kind, model and credentials
1354
+ (`buildSessionWorkerLaunchEnvForSession`). The detached and Windows launches pass all of it. Under
1355
+ tmux the worker's login shell gets only the variables `buildTmuxWorkerShellCommand` exports, and
1356
+ the secrets (`COPILOT_PROVIDER_API_KEY`, `CURSOR_API_KEY`, `CLAUDE_CODE_OAUTH_TOKEN` for cloud
1357
+ workers) through an owner-only file it reads and deletes; any other variable, such as
1358
+ `OAR_TURN_STALL_*_MS`, `CLAUDE_CODE_AUTO_COMPACT_WINDOW` or `XAI_API_KEY`, only from the tmux
1359
+ server's environment or the login profile (see [Session workers under
1360
+ tmux](../docs/configuration.md#environment-variables)). `COPILOT_WEB_RELAY_DISABLE_CLI_SPAWN` makes
1361
+ every launch fail (`cli-spawn-disabled`), for test relays.
1245
1362
 
1246
1363
  ## How It Works
1247
1364
 
@@ -1330,11 +1447,11 @@ If a conversation is still unbound when the CLI first sees it, the extension now
1330
1447
 
1331
1448
  ### Worker Launcher
1332
1449
 
1333
- - Workers on Windows launch via a **stable Terminal window** (when available).
1334
- - If TTY-Console (optional dependency) is installed, workers use an interactive terminal for debugging.
1335
- - Otherwise, workers spawn in a new `cmd` or PowerShell window.
1336
- - Window name is workspace-based so repeated worker restarts reuse the same window instead of opening duplicates.
1337
- - Claude workers open a `Claude Worker <id>` window running `node claude-session-worker.mjs` directly, instead of the `gh copilot` launch path.
1450
+ - Each worker opens its own console window through `cmd /c start`, titled `<Kind> Worker <first 8
1451
+ characters of the session id>` (`Copilot Worker`, `Claude Worker`, `Cursor Worker`, `Grok Worker`,
1452
+ `Claude Cloud Worker`, `Copilot SDK Worker`); a relaunch opens a new window.
1453
+ - Node workers run `node <kind>-session-worker.mjs --session-id <id>` directly; the extension engine
1454
+ runs `gh copilot -- --allow-all --session-id <id>`.
1338
1455
 
1339
1456
  ### Path Handling
1340
1457
 
@@ -1420,8 +1537,8 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
1420
1537
  | POST | `/api/relay/shutdown/cancel` | Withdraw a queued (not yet started) shutdown or restart; held messages go out at once |
1421
1538
  | GET | `/api/relay-questions` | (CLI/UI) List relay questions by `status` (for example `pending` or `answered`) |
1422
1539
  | GET | `/api/relay-question/:id` | (CLI) Fetch a single relay question |
1423
- | POST | `/api/relay-question` | (CLI) Create a relay question for the browser |
1424
- | POST | `/api/relay-question/:id/answer` | (UI) Submit an answer for a relay question |
1540
+ | POST | `/api/relay-question` | (CLI) Create a relay question for the browser. `allowFreeform` defaults to `false` when `choices` are given (to `true` without): such a card has no reply box and takes only its choices |
1541
+ | POST | `/api/relay-question/:id/answer` | (UI) Submit an answer for a relay question. A card created with `allowFreeform: false` and choices (approval and permission cards) takes only its choices, matched trimmed and in any case and stored as the card's label (several, `", "`-joined, on a multi-select card); anything else is refused with `400` and `choices`, and the card stays open |
1425
1542
  | POST | `/api/relay-question/:id/timeout` | (CLI/UI) Mark a relay question timed out |
1426
1543
  | POST | `/api/heartbeat` | (CLI) Keep CLI status alive. A session worker may send `backgroundTasks: { count, digest }`; the answer carries `resyncBackgroundTasks: true` when the relay's set for that worker differs |
1427
1544
  | GET | `/api/status` | Overall status |
@@ -1452,9 +1569,9 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
1452
1569
  | POST | `/api/claude-cloud-session` | (Claude Cloud worker) Report the cloud session a conversation is bound to: `{ conversationId, cloudSessionId, sessionUrl?, lastSequence, pushedBranch?, costUsd?, model? }` → `{ ok, cloud }`. Sent after the session was created, after every `result` and on a push; emits `claude_cloud_session`. `404` without a runtime session, `409` for a conversation that is not a cloud one |
1453
1570
  | POST | `/api/cursor-plan-usage` | (Cursor worker) Report the agent's cumulative billed usage; the relay diffs it against a per-agent checkpoint and books the increase into the current billing cycle under the pool implied by the turn's model |
1454
1571
  | POST | `/api/subagent-run` | (Worker) Register/update a subagent run. A new run needs the spawning turn's `messageId`; an update to a known run may omit it (a background agent's end arrives between turns). Optional header fields: `subagentType`, `model`, `modelInherited`, `taskId`, `background`, `timelineOwner`; a post without `status` leaves the run's state alone. Emits `subagent_status` with the whole header |
1455
- | POST | `/api/subagent-timeline` | (Worker) `{ conversationId, subagentRunId, entries: [{ key, kind, text, done?, toolName?, toolUseId?, metadata? }], header? }` → `{ ok, stored, truncated }`. Kinds: `thought`, `tool_call`, `tool_result`, `text`, `child`, `note`. Fenced by the run row, not by an open turn; an unknown run whose `header.parentSubagentId` is known is created under the parent's turn, anything else is `404 unknown_run`. Sends `subagent_timeline` `{ conversationId, messageId, subagentRunId, entries, truncated }`, volatile, only to the pages watching the run (`subagent_viewer_watch`) |
1572
+ | POST | `/api/subagent-timeline` | (Worker) `{ conversationId, subagentRunId, taskId?, entries: [{ key, kind, text, done?, toolName?, toolUseId?, metadata? }], header? }` → `{ ok, stored, truncated }`. Kinds: `thought`, `tool_call`, `tool_result`, `text`, `child`, `note`. Fenced by the run row, not by an open turn; an unknown run whose `header.parentSubagentId` is known is created under the parent's turn, an unknown run whose `taskId` (or `header.taskId`) a run of the conversation carries is written to that run (a resumed agent; when the post sets that finished run `running` again, the relay adds a note where it resumed), anything else is `404 unknown_run`. `header.status: 'running'` clears the run's end time. Sends `subagent_timeline` `{ conversationId, messageId, subagentRunId, entries, truncated }`, volatile, only to the pages watching the run (`subagent_viewer_watch`) |
1456
1573
  | GET | `/api/conversation/:conversationId/subagent/:subagentRunId` | One run's record for the viewer: `?after=<seq>&limit=<n>` (default 500, max 2000) → `{ run, entries, children, nextAfter, complete }`. `entries` are those after `after` plus any still open; `run.legacy` marks a record rebuilt from the lane rows |
1457
- | POST | `/api/conversation/:conversationId/subagent/:subagentRunId/cancel` | Request cancellation of one subagent run (unsupported by the Claude runtime) |
1574
+ | POST | `/api/conversation/:conversationId/subagent/:subagentRunId/cancel` | Queue an `abort_subagent` control for the session's worker (`{ parentMessageId?, clientId? }` → `{ ok, queued, acknowledgement }`; `not-found`, `message-mismatch`, `already-cancelled` and `already-finished` queue nothing). The Claude worker stops a background sub-agent (`stopTask`), the Copilot SDK worker a background agent (`rpc.tasks.cancel`), the extension engine tries the SDK session's abort methods; any other run, or worker, answers "not supported". Workers read controls only while a turn runs. Whole-turn **Stop** always works |
1458
1575
  | PATCH | `/api/conversation/:id/message/:messageId/share-visibility` | Hide or unhide a single message from shared views |
1459
1576
  | PATCH | `/api/conversation/:id/message/:messageId/pin` | `{ pinned }` → pin or unpin a single message; answers with the conversation's pin list and broadcasts `conversation_pins_updated` |
1460
1577
  | POST | `/api/conversation/:id/share` | The conversation's read-only share link, made if it has none (one active share per conversation): `{ token, shareUrl, path, created }`; a new one sends `share_created` |
@@ -1481,7 +1598,7 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
1481
1598
  | DELETE | `/api/relay-events/subscriptions/:id` | The user's Unsubscribe (a paired relay's subscription too: it learns with its next fetch); refused to other relays' agents |
1482
1599
  | POST | `/api/relay-events/remote/subscribe` | (Relay to relay) A paired relay's agent subscribes to sessions here: `{ watcher: { conversationId, title }, subscriber: { name }, scope, sessions, events?, once?, match?, stalled_minutes?, heartbeat_minutes?, dry_run? }` → `{ ok, subscription }`. Another relay's request only, inbound switch on; see [Across paired relays](#across-paired-relays) |
1483
1600
  | POST | `/api/relay-events/remote/pull` | (Relay to relay) That relay fetches its events: `{ after, subscriptions: [{ id, watcher }], ended: [ids] }` → `{ ok, events, cursor, more, subscriptions, ended, dropped }` |
1484
- | POST | `/api/background-task-settled` | (Worker) How a background task ended: `{ conversationId, taskId, toolUseId?, status, summary?, reason?, description?, taskType?, startedAt? }` (Claude worker, from `task_notification`; the last three describe a task that was in its published set) |
1601
+ | POST | `/api/background-task-settled` | (Worker) How a background task ended: `{ conversationId, taskId, toolUseId?, taskRunId?, status, summary?, reason?, description?, taskType?, startedAt? }` (Claude worker, from `task_notification`; the last three describe a task that was in its published set) |
1485
1602
  | GET | `/api/remote-relays/inflight` | (Worker) Number of `remote_relay` calls running for `conversationId` |
1486
1603
  | GET/POST | `/api/settings/remote-relays` | This relay's `publicUrl` and `inboundEnabled` (accept prompts from other relays' agents) |
1487
1604
  | GET | `/api/settings/agent-sessions` | Agent sessions: `{ enabled, maxWaitSeconds, limits: { minWaitSeconds: 120, maxWaitSeconds: 3600, stepSeconds: 60 }, maxActiveSessions: 4 }`. `enabled` (default `false`) lets this relay's agents start and use sessions on it through `remote_relay`; `maxWaitSeconds` (default `600`) is the longest wait of one `remote_relay` call, for this relay and paired ones |
@@ -1513,6 +1630,7 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
1513
1630
  - `requestedAt`, `reason`, `requestedBy`
1514
1631
  - `queue`: current `pendingCount` / `processingCount` / `parkedCount`
1515
1632
  - `waitingFor`, `held`, `backgroundTasks`, `survival`, `maxWaitMs`, `deadlineAt`, `immediate` (below)
1633
+ - A conversation that subscribed to `relay_restarted` is woken when the relay runs again, with who requested it, when and why ([relay_restarted](#relay_restarted-woken-when-the-relay-runs-again)); one whose turn answered that event when the request came is not woken by it.
1516
1634
  - The relay waits until no turn is processing before acting. While the request is queued no pending message is handed to a worker (a turn started meanwhile would postpone the restart again); pending and parked rows stay in the database and run after the restart.
1517
1635
  - `immediate: true` (boolean-ish, like `restart`) starts it at once, running turns or not: the banner's **Restart now**. The answer still names what it did not wait for.
1518
1636
  - Maximum wait (`services/relay-restart-queue.mjs`): after **Ask after a queued relay restart has waited** (`relay_restart_max_wait_minutes`, 30 by default, 5–240 or `0` = no limit; `OAR_RELAY_RESTART_MAX_WAIT_MS` overrides it in ms, for test relays) the request stays queued and sets `question: { askedAt }`; the relay pushes one notification (titled *Relay restart waiting*; switch **Queued relay restart waited its maximum time**, `relayRestart`) and never restarts while turns run; it still goes ahead as soon as the last turn ends, question or not. A raised maximum (or none) that moves the deadline into the future withdraws the question; `/keep-waiting` without a question does nothing (`extended: false`). An `immediate` request may name the queued request it was shown (`expectRequestedAt`, its `requestedAt`); when that request is gone the answer is `accepted: false, stale: true` and nothing starts. Agents are told never to send `immediate` (`relay-tools.md`). The user answers with `immediate`, `/keep-waiting` (another full period from now; counted in `keptWaiting`) or `/cancel`. With no limit the request, and the messages it holds, wait until the turns end or someone decides.
@@ -1556,7 +1674,16 @@ follow-up, or a second time in the same runtime. The browser renders it as a com
1556
1674
 
1557
1675
  ### Upload storage model
1558
1676
 
1559
- - Physical blobs are stored in `server/uploads/<sha256>` (content-addressed).
1677
+ - Physical blobs are stored in `<data dir>/uploads/<sha256>` (content-addressed) when
1678
+ `COPILOT_WEB_RELAY_DATA_DIR` or the config's `dataDir` chose the data dir, else in
1679
+ `server/uploads/<sha256>` (`resolveUploadDir` in `services/data-dir-resolution.mjs`).
1680
+ - Blobs earlier versions left elsewhere are read in place: `server/uploads` (a config
1681
+ `dataDir` did not move uploads before 0.9.14) and `uploads/` in the folder that holds the
1682
+ data dir (state migrations before 0.9.14 copied a checkout's uploads there).
1683
+ `resolveUploadReadDirs` lists them after the upload dir; a blob is read from the first
1684
+ copy whose size is the recorded one (two of that size that are not alike, i.e. differ in
1685
+ their first or last 64 KiB: the one whose content hashes to its name, decided once per blob). New uploads are written to the upload dir only, and
1686
+ the unreferenced-upload sweep and a cache wipe delete only there.
1560
1687
  - Metadata (`original_name`, `mime_type`, `size_bytes`) is stored in SQLite.
1561
1688
  - Message/conversation references are tracked in SQLite; when a conversation is deleted,
1562
1689
  unreferenced blobs are garbage-collected from disk automatically.
@@ -1580,16 +1707,18 @@ follow-up, or a second time in the same runtime. The browser renders it as a com
1580
1707
  - Images over ~2 MB are downscaled client-side to a 2560 px longest edge and re-encoded
1581
1708
  to **JPEG** before upload. GIF and SVG are never re-encoded, and a re-encode that fails,
1582
1709
  grows the file, or yields anything other than JPEG/PNG falls back to the original bytes.
1583
- - The output format is deliberately restricted to JPEG/PNG: Copilot's image-input docs call
1584
- these "the most widely supported formats". WebP compresses better and previews correctly
1585
- in the browser, but arrives at the model unreadable.
1710
+ - The output format is deliberately restricted to JPEG/PNG (`image-downscale.mjs`): Copilot's
1711
+ image-input docs call these "the most widely supported formats". The composer converts for
1712
+ every provider alike, although the Claude workers, local and cloud, would take a WebP as it is
1713
+ (`claude-attachments.mjs` inlines JPEG, PNG, GIF and WebP).
1586
1714
  - Re-encoding therefore has two triggers, tracked by `reencodeReason()`:
1587
1715
  - `'size'` — the image is over the byte threshold. Abandoned if the result is no smaller.
1588
1716
  - `'format'` — the source is not PNG/JPEG/GIF (e.g. WebP, AVIF, HEIC), so it is converted
1589
1717
  at its original dimensions **regardless of size**, and kept even if the result is larger:
1590
1718
  a bigger image the model can read beats a smaller one it cannot.
1591
1719
  - Without the `'format'` trigger a WebP small enough to skip the size threshold would reach
1592
- the model invisible.
1720
+ the model in a format outside that pair. An image that does not come through the composer
1721
+ (`POST /api/upload` from a script) is stored as it is.
1593
1722
  - Attachments upload **eagerly** on selection rather than on Send. Chips show an
1594
1723
  uploading spinner, and a failed upload becomes a clickable retry.
1595
1724
  - Send is disabled while any upload is in flight. Failed uploads are deliberately not
@@ -1651,7 +1780,7 @@ follow-up, or a second time in the same runtime. The browser renders it as a com
1651
1780
  - Use `/api/repo/list?path=<repo-relative-dir>&includeHidden=0|1&includeHeavy=0|1` for lazy-loaded workspace directory browsing.
1652
1781
  - Use `/api/drives/roots` + `/api/drives/list?path=<path>&includeHidden=0|1` for lazy-loaded drive/root browsing (separate from workspace heavy mode).
1653
1782
  - Use `/api/drives/file?path=<path>` and `/api/drives/files-preview?path=<path>` for drive/root file raw/preview access.
1654
- - Use `/api/session-root/list?path=<sessionRootPath>&includeHidden=0|1` for the explorer's Session root. Child directories below it lazy-load through `/api/drives/list` as usual. The Claude Agent SDK creates a session's directory lazily — only once the session writes `subagents/` or `tool-results/` files — so the root is served as an empty folder until then rather than 404ing, and the transcript that lives one level up in the project directory is listed as a child of it. Cursor sessions resolve their Session root the same lazy way: the worker's per-session agent store (`$CURSOR_AGENT_STORE_DIR` or `server/data/cursor-agents/<sdkSessionId>`) exists only after the session's first turn, and the Session button stays disabled until it does.
1783
+ - Use `/api/session-root/list?path=<sessionRootPath>&includeHidden=0|1` for the explorer's Session root. Child directories below it lazy-load through `/api/drives/list` as usual. The Claude Agent SDK creates a session's directory lazily — only once the session writes `subagents/` or `tool-results/` files — so the root is served as an empty folder until then rather than 404ing, and the transcript that lives one level up in the project directory is listed as a child of it. Cursor sessions resolve their Session root the same lazy way: the worker's per-session agent store (`$CURSOR_AGENT_STORE_DIR` or `cursor-agents/<sdkSessionId>` in the relay's data dir) exists only after the session's first turn, and the Session button stays disabled until it does.
1655
1784
  - Requests are auth-protected and restricted to files inside the workspace root.
1656
1785
  - **Platform split** — the drives API adapts automatically based on the server OS:
1657
1786
  - **Windows:** drive roots are discovered via `fsutil.exe`; paths use Windows drive-letter format (`C:/foo/bar`); directory listing uses PowerShell.
@@ -1835,8 +1964,18 @@ The migration `server/migrations/0001-add-structured-answer.mjs` creates the `st
1835
1964
 
1836
1965
  ## Relay Tool Guidance
1837
1966
 
1838
- The CLI extension (`.github/extensions/web-relay/extension.mjs`) loads `relay-tools.md`
1839
- for shared tool guidance.
1967
+ The CLI extension (`.github/extensions/web-relay/extension.mjs`) and the Copilot SDK worker
1968
+ (`copilot-worker/copilot-prompt-context.mjs`) load `relay-tools.md` (`COPILOT_WEB_RELAY_TOOLS`
1969
+ overrides the path) for shared tool guidance.
1970
+
1971
+ How to embed media in a reply (a markdown image naming the file's absolute host path; the page
1972
+ renders images and turns video and audio into players) is one text,
1973
+ `shared/media-embed-instructions.mjs`. The Claude worker appends it to the system prompt, the
1974
+ Copilot SDK worker to its relay guidance, the Cursor and Grok workers put it in front of the first
1975
+ prompt each worker process sends. The
1976
+ Copilot extension engine's prompt builder (`shared/worker-runtime/prompt-context.mjs`) does not add
1977
+ it and `relay-tools.md` does not contain it, and a Claude Cloud session gets no relay guidance at
1978
+ all, so those two are not told.
1840
1979
 
1841
1980
  The browser UI keeps the usage button in the sidebar header, and that button continues to
1842
1981
  call `/api/usage` directly. It now renders the unified plan-usage report — one card per
@@ -1852,7 +1991,6 @@ service).
1852
1991
  "authToken": "<your-token>",
1853
1992
  "port": 3333,
1854
1993
  "localhostOnly": true,
1855
- "pollIntervalMs": 3000,
1856
1994
  "processingTimeoutMs": 600000,
1857
1995
  "conversationSessionMode": "isolated",
1858
1996
  "contextIndicatorMode": "default",
@@ -1882,17 +2020,31 @@ service).
1882
2020
 
1883
2021
  | Key | Default | Description |
1884
2022
  |---|---|---|
1885
- | `authToken` | *(required)* | Token for all API / UI access |
1886
- | `port` | `3333` | HTTP/WebSocket listen port |
2023
+ | `authToken` | generated if empty | Token for all API / UI access. Empty or missing: a random token on every start, printed to the log and not saved (`--token <value>` sets one for the run) |
2024
+ | `port` | `3333` | HTTP/WebSocket listen port (`--port <n>` for the run) |
1887
2025
  | `localhostOnly` | `true` | Bind only to loopback (`127.0.0.1`) and block LAN/WAN access |
1888
- | `pollIntervalMs` | `3000` | CLI poll interval (ms) |
2026
+ | `host` | `127.0.0.1` | Read by the workers only: the address they connect to the relay on (`0.0.0.0`, `::` and `localhost` mean `127.0.0.1`) |
2027
+ | `dataDir` | `data/` beside the server | Database, the relay's lock file, uploads (`uploads/` inside it) and Cursor agent stores (`cursor-agents`); relative to the config file's folder; `COPILOT_WEB_RELAY_DATA_DIR` wins. Without either, a checkout keeps its uploads in `server/uploads`; uploads made there before `dataDir` was set stay there and are still read |
2028
+ | `pollIntervalMs` | — | Read by nothing; new config files still get `3000` |
1889
2029
  | `processingTimeoutMs` | `600000` | Worker-inactivity window before a turn is treated as stale (ms) — see [Turn recovery](#turn-recovery-and-the-max-turn-duration) |
1890
- | `conversationSessionMode` | `isolated` | SDK session strategy (`isolated` or `shared`) |
1891
- | `contextIndicatorMode` | `default` | Input context indicator style (`default` shimmer line or `bar` fill indicator) |
2030
+ | `maxRequeueRetries` | `5` | Retries of a recovered queue row before it fails |
2031
+ | `conversationSessionMode` | `isolated` | `isolated` or `shared`; recorded as `runtime_sessions.strategy` and reported in `/api/status`, with no effect on behaviour |
2032
+ | `contextIndicatorMode` | `default` | Composer context indicator: `default`, a line across the composer's top edge whose colour runs from green to red as the context fills, or `bar`, a line as long as the share in use; anything else is `default`. Sent to the page in `/api/status` |
2033
+ | `remotePath` | `""` | URL path prefix when the relay is served under a subpath (PWA URLs, socket.io path) |
2034
+ | `trustProxy` | `"loopback"` | Express `trust proxy`: which proxies may set `X-Forwarded-*` |
2035
+ | `workspaceRootAllowList` | unset | Folders conversations may start in; see [Changing the launch CWD](#changing-the-launch-cwd) |
2036
+ | `publicHostnames` | `[]` | The relay's own public hostnames; the preview lane refuses them |
2037
+ | `previews.enabled`, `.port`, `.bindHost`, `.allowPublicBind`, `.publicBaseUrl`, `.allowedTargetHosts`, `.maxLive` | `false`, `port + 1`, `127.0.0.1`, `false`, —, `[]`, `8` | The preview listener (`services/preview-registry-service.mjs`, [docs/preview-servers.md](../docs/preview-servers.md)) |
2038
+ | `pushVapidSubject` | `mailto:copilot-remote@example.com` | Contact URI sent to push services |
2039
+ | `sharedPresenceMaxPerConversation`, `sharedPresenceMaxGlobal` | `200`, `5000` | Caps of the shared-view watcher count |
2040
+ | `cliBinaries` | *(written by the relay)* | Provider CLI paths resolved by an install from Settings, applied at startup |
2041
+ | `cliPort` | `4445` | Port of the standalone `relay.mjs`; the preview lane refuses it |
2042
+ | `features` | — | Legacy: moved into the database once at startup, then the **Features** tab |
1892
2043
  | `sdkVersion` | *(latest detected)* | Optional semver pin (for example `1.0.63`) used by relay SDK auto-detection |
1893
2044
  | `sdkPath` | *(auto-detected)* | Optional absolute override for Copilot SDK entry path (`.../copilot-sdk/index.js`) |
1894
2045
  | `cliPath` | *(auto-detected)* | Optional absolute override for Copilot CLI app path (`.../app.js`) |
1895
2046
  | `restartGracefulTimeoutMs` | `8000` | Graceful shutdown wait before force fallback |
2047
+ | `restartReadyCooldownMs` | `1000` | How long the worker restart orchestrator still holds dequeue after a restart transaction is ready |
1896
2048
  | `restartShutdownTimeoutMs` | `45000` | Drain timeout while waiting for active queue jobs |
1897
2049
  | `restartSpawnTimeoutMs` | `18000` | Max wait for resume process to leave online state |
1898
2050
  | `restartRebindTimeoutMs` | `20000` | Max wait for session-sync rebind confirmation |
@@ -2059,23 +2211,18 @@ reported as `relayShutdown`.
2059
2211
 
2060
2212
  ## Database Migrations
2061
2213
 
2062
- The relay uses a simple SQL migration system to evolve the SQLite schema over time.
2063
-
2064
- **Migration File Location:** `server/migrations/`
2065
-
2066
- **Migration Lifecycle:**
2067
-
2068
- 1. On server startup, relay checks the database for applied migrations
2069
- 2. Unapplied migrations (in filename order) are executed automatically
2070
- 3. Each migration file exports a `up()` function that creates/alters tables
2071
- 4. After successful execution, migration is recorded in the database metadata
2072
- 5. Migrations are idempotent: re-running a migration is safe (no duplicates)
2073
-
2074
- **Currently Deployed Migration:**
2075
-
2076
- - **`0001-add-structured-answer.mjs`** — Adds `structured_answer` and `request_schema` columns to the `relay_questions` table to support multi-field elicitation forms (see [Structured Answers and Multi-Field Elicitation](#structured-answers-and-multi-field-elicitation) section).
2077
- - `request_schema` (TEXT/JSON) — JSON schema defining form fields and validation
2078
- - `structured_answer` (TEXT/JSON) — User's submitted multi-field response object
2214
+ There is no migration runner and no record of applied migrations. At every start
2215
+ `applySchema(db)` (`server/db-schema.mjs`, the same code the tests build their databases with) runs
2216
+ `SCHEMA_SQL` (`CREATE TABLE IF NOT EXISTS` for a fresh database), then the upgrade sequence:
2217
+ guarded `ALTER TABLE … ADD COLUMN` statements (each runs only when `PRAGMA table_info` lacks the
2218
+ column) and, among them, the functions of `server/migrations/0002-…` to `0018-…` in order. Each of those checks the
2219
+ schema itself and does nothing when its change is already there, so all of them run on every boot.
2220
+ From 0004 on a failure is logged (`migration 00NN failed; retrying on next boot`) and the boot goes
2221
+ on. Every file can also be run by hand against a database: `node server/migrations/<file>.mjs
2222
+ [path/to/copilot.db]`. `0001-add-structured-answer.mjs` (the `structured_answer` and
2223
+ `request_schema` columns of `relay_questions`, see [Structured Answers and Multi-Field
2224
+ Elicitation](#structured-answers-and-multi-field-elicitation)) is only such a script: the boot adds
2225
+ those columns with its own guarded `ALTER TABLE`.
2079
2226
 
2080
2227
  **In-place column additions:**
2081
2228
 
@@ -2139,26 +2286,26 @@ one:
2139
2286
  - `conversations.cloud_source_json` (TEXT/JSON) — `{ repoUrl, branch, environmentId, sessionUrl,
2140
2287
  pushedBranches: [{ branch, at }] }`
2141
2288
 
2142
- New tables are likewise created at startup with `CREATE TABLE IF NOT EXISTS` in both the fresh-schema
2143
- and migration blocks — most recently `workflow_runs` (final workflow digest per assistant message,
2289
+ Older tables are created at startup with `CREATE TABLE IF NOT EXISTS` in `SCHEMA_SQL` (`db-schema.mjs`),
2290
+ for example `workflow_runs` (final workflow digest per assistant message,
2144
2291
  keyed on `response_message_id`, cascade-deleted with the conversation; see
2145
- [Background tasks and workflow progress](#background-tasks-and-workflow-progress)).
2292
+ [Background tasks and workflow progress](#background-tasks-and-workflow-progress)); newer ones, such as
2293
+ `background_task_rows`, come from a migration only (see below).
2146
2294
 
2147
2295
  Statements that depend on these columns are prepared conditionally, so an older database keeps
2148
2296
  working with the corresponding feature inert rather than crashing at startup.
2149
2297
 
2150
- **To Deploy New Migrations:**
2151
-
2152
- 1. Create file `server/migrations/000N-description.mjs` (increment the number)
2153
- 2. Export an async `up()` function that receives the database connection
2154
- 3. Write SQL alterations:
2155
- ```javascript
2156
- export async function up(db) {
2157
- await db.exec(`ALTER TABLE some_table ADD COLUMN new_col TEXT;`);
2158
- }
2159
- ```
2160
- 4. Restart the relay; migration runs automatically
2161
- 5. Verify: check relay logs for `[migrations] Applying 000N-description`
2298
+ **Adding a migration:**
2299
+
2300
+ 1. Create `server/migrations/00NN-description.mjs` with the next number, exporting a synchronous
2301
+ `migrateSomething(db)` that checks the schema (`sqlite_master`, `PRAGMA table_info`) and changes
2302
+ only what is missing, plus a `migrate(dbPath)` and an `isMain` block for running it by hand
2303
+ (`0018-background-task-rows.mjs` is a small example).
2304
+ 2. Call it from `applySchema` after the previous one, inside `try`/`catch` with a
2305
+ `migration 00NN failed; retrying on next boot` warning. It runs on a fresh database too, so
2306
+ what it creates needs no entry in `SCHEMA_SQL`.
2307
+ 3. Prepare statements that need the change conditionally, so a database the migration has not
2308
+ reached keeps the feature inert instead of failing at startup.
2162
2309
 
2163
2310
  ## Files
2164
2311