@oar-sh/oar 0.9.12 → 0.9.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/extensions/web-relay/extension.mjs +9 -0
- package/.github/extensions/web-relay/server-lifecycle/managed-server.mjs +15 -1
- package/.github/extensions/web-relay/skills/question-bridge.mjs +4 -1
- package/CHANGELOG.md +508 -0
- package/README.md +12 -12
- package/bin/oar.js +155 -94
- package/package.json +5 -5
- package/server/README.md +479 -157
- package/server/claude-cloud-worker/claude-cloud-session-process.mjs +10 -4
- package/server/claude-worker/claude-session-process.mjs +92 -15
- package/server/claude-worker/claude-turn-publisher.mjs +20 -1
- package/server/claude-worker/subagent-timeline-recorder.mjs +75 -8
- package/server/claude-worker/task-output-discovery.mjs +28 -2
- package/server/copilot-worker/copilot-ask-user-tool.mjs +1 -1
- package/server/copilot-worker/copilot-continuation-signals.mjs +11 -5
- package/server/copilot-worker/copilot-prompt-context.mjs +3 -1
- package/server/copilot-worker/copilot-question-bridge.mjs +11 -6
- package/server/copilot-worker/copilot-sdk-adapter.mjs +11 -0
- package/server/copilot-worker/copilot-sdk-session-process.mjs +24 -1
- package/server/cursor-worker/cursor-ask-user-tool.mjs +1 -1
- package/server/cursor-worker/cursor-sdk-adapter.mjs +22 -7
- package/server/cursor-worker/cursor-session-worker.mjs +9 -2
- package/server/cursor-worker/cursor-turn-runner.mjs +18 -3
- package/server/db-schema.mjs +10 -0
- package/server/grok-worker/acp-host-services.mjs +6 -3
- package/server/grok-worker/grok-background-tasks.mjs +7 -0
- package/server/grok-worker/grok-sdk-adapter.mjs +14 -0
- package/server/grok-worker/grok-turn-runner.mjs +13 -3
- package/server/migrations/0018-background-task-rows.mjs +101 -0
- package/server/public/app/action-confirmations.js +12 -15
- package/server/public/app/api-client.js +60 -4
- package/server/public/app/ask-user-view.js +29 -4
- package/server/public/app/background-tasks-view.mjs +56 -7
- package/server/public/app/bootstrap.js +7 -0
- package/server/public/app/conversation-preferences.mjs +10 -4
- package/server/public/app/conversation-provider-indicator.mjs +18 -8
- package/server/public/app/conversation-view.js +33 -9
- package/server/public/app/host-suspend-ui.js +45 -44
- package/server/public/app/journal-view.js +9 -3
- package/server/public/app/push-settings.js +1 -0
- package/server/public/app/question-answer-rule.mjs +15 -0
- package/server/public/app/question-multi-select.mjs +9 -2
- package/server/public/app/relay-event-view.mjs +5 -1
- package/server/public/app/relay-prompt-sanitizer.mjs +9 -2
- package/server/public/app/relay-restart-note-view.mjs +36 -0
- package/server/public/app/relay-restart-ui.js +568 -0
- package/server/public/app/settings-modal.js +52 -0
- package/server/public/app/socket-handlers.js +1 -1
- package/server/public/app/task-console-render.mjs +171 -7
- package/server/public/app/task-console.mjs +111 -11
- package/server/public/app/task-output-text.mjs +15 -6
- package/server/public/app/transcript-jump-bottom-view.mjs +5 -1
- package/server/public/app/transcript-jump-bottom.mjs +20 -0
- package/server/public/index.html +76 -5
- package/server/relay-exit-codes.mjs +6 -0
- package/server/relay-self-restart.mjs +6 -1
- package/server/relay-tools.md +17 -9
- package/server/repositories/message-repository.mjs +32 -28
- package/server/repositories/remote-relay-repository.mjs +4 -2
- package/server/repositories/subagent-timeline-statements.mjs +2 -0
- package/server/routes/ask-user-routes.mjs +41 -7
- package/server/routes/messages-routes.mjs +220 -37
- package/server/routes/preview-routes.mjs +73 -7
- package/server/routes/relay-board-routes.mjs +19 -2
- package/server/routes/relay-restart-routes.mjs +73 -0
- package/server/routes/sessions-routes.mjs +79 -5
- package/server/server-runtime.mjs +335 -218
- package/server/services/about-service.mjs +13 -12
- package/server/services/background-task-persistence.mjs +133 -0
- package/server/services/background-task-store.mjs +240 -13
- package/server/services/conversation-model-fallback.mjs +57 -0
- package/server/services/cursor-session-root-service.mjs +70 -6
- package/server/services/data-dir-resolution.mjs +208 -19
- package/server/services/earlier-database-service.mjs +631 -0
- package/server/services/host-suspend-command.mjs +2 -1
- package/server/services/oar-cli-helpers.mjs +6 -1
- package/server/services/oar-state-layout.mjs +107 -0
- package/server/services/oar-state-migration-service.mjs +387 -96
- package/server/services/package-data-rescue-preinstall.mjs +22 -0
- package/server/services/package-data-rescue.mjs +137 -0
- package/server/services/preview-proxy-server.mjs +14 -1
- package/server/services/preview-registry-service.mjs +3 -2
- package/server/services/process-list-service.mjs +2 -1
- package/server/services/provider-model-selection.mjs +195 -0
- package/server/services/push-dispatch-service.mjs +42 -8
- package/server/services/relay-event-service.mjs +210 -14
- package/server/services/relay-prompt-sanitizer.mjs +7 -0
- package/server/services/relay-restart-picture.mjs +209 -0
- package/server/services/relay-restart-queue.mjs +387 -0
- package/server/services/relay-shutdown-gate.mjs +4 -1
- package/server/services/relay-singleton-guard.mjs +300 -36
- package/server/services/remote-relay-caller-context.mjs +7 -2
- package/server/services/remote-relay-dispatcher.mjs +217 -28
- package/server/services/restart-resume-service.mjs +303 -0
- package/server/services/session-activity-tracker.mjs +52 -15
- package/server/services/session-worker-launch-service.mjs +31 -1
- package/server/services/session-worker-process-service.mjs +6 -4
- package/server/services/stale-turn-recovery.mjs +99 -0
- package/server/services/subagent-timeline-service.mjs +4 -1
- package/server/services/task-output-service.mjs +217 -18
- package/server/services/windows-autostart-service.mjs +329 -31
- package/server/services/windows-boot-autostart-service.mjs +188 -60
- package/shared/preview-instructions.mjs +17 -12
- package/shared/preview-tool-core.mjs +32 -14
- package/shared/question-answer-rule.mjs +63 -0
- package/shared/relay-events.mjs +106 -3
- package/shared/remote-relay-contract.mjs +32 -15
- package/shared/subagent-timeline.mjs +4 -0
- package/shared/task-output-text.mjs +196 -83
- package/shared/windows-system-tools.mjs +25 -0
- package/shared/worker-runtime/config-loader.mjs +54 -8
- package/shared/worker-runtime/prompt-context.mjs +3 -1
- package/shared/worker-runtime/worker-websocket-link.mjs +96 -20
- package/shared/worker-runtime/working-directory.mjs +65 -0
- package/server/uploads/image_input.md +0 -540
package/server/README.md
CHANGED
|
@@ -216,7 +216,8 @@ reports both the actual suspend and such a drop. For the first 45 seconds after
|
|
|
216
216
|
a relay start the check lists one more blocker, "the relay just started,
|
|
217
217
|
sessions are still reporting": the workers that outlived the previous runtime
|
|
218
218
|
may not have reported their background work yet. A queued **🌄 Restart web relay**
|
|
219
|
-
gets
|
|
219
|
+
gets a banner of its own, with the sessions it waits for, **Restart now** and
|
|
220
|
+
**Cancel restart** (see [Manual relay shutdown / self-restart](#manual-relay-shutdown--self-restart)).
|
|
220
221
|
|
|
221
222
|
API: `GET /api/host/suspend` (state plus current blockers), `POST /api/host/suspend`,
|
|
222
223
|
`POST /api/host/suspend/cancel`, and `POST /api/relay/shutdown/cancel`
|
|
@@ -254,21 +255,68 @@ Behavior notes:
|
|
|
254
255
|
- In extension-managed mode, the relay still switches model per message and
|
|
255
256
|
reports the active model used in the response.
|
|
256
257
|
|
|
257
|
-
###
|
|
258
|
+
### A message without a model, an effort or a mode
|
|
259
|
+
|
|
260
|
+
The composer always sends all three. A client that leaves one out of
|
|
261
|
+
`POST /api/message` (a script, an agent on this relay) gets the conversation's
|
|
262
|
+
own, never the relay-wide default:
|
|
263
|
+
|
|
264
|
+
- **Model**, for an existing Copilot conversation: the conversation's saved
|
|
265
|
+
`preferred_model`, then the runtime session's `provider_model`, then its
|
|
266
|
+
`model`, then the model of the latest user message (`auto` skipped in each);
|
|
267
|
+
only a conversation with none of these runs on the catalog's current model.
|
|
268
|
+
A saved model the catalog no longer lists is refused with
|
|
269
|
+
`409 CONVERSATION_MODEL_UNAVAILABLE` (`model`, `supportedModels`) instead of
|
|
270
|
+
being replaced; the composer shows that text in its banner. An explicit
|
|
271
|
+
`auto` is unchanged (new conversations only). Claude, Cursor, Grok, Claude
|
|
272
|
+
Cloud and OpenAI conversations keep their pinned `provider_model`, as before.
|
|
273
|
+
- **Effort**: for Copilot, an effort the model's variant id carries, else the
|
|
274
|
+
conversation's `preferred_reasoning_effort`, clamped to the nearest level the
|
|
275
|
+
model offers (`nearestSupportedReasoningEffort`, the composer's rule); with
|
|
276
|
+
nothing to keep, the model's lowest level that is not `none`, so `none` is
|
|
277
|
+
chosen only when the conversation asked for it or the model has no other
|
|
278
|
+
level. OpenAI, Claude, Cursor and Grok conversations take the conversation's
|
|
279
|
+
effort the same way; with no effort stored, or one that neither matches nor
|
|
280
|
+
clamps (an `ultracode` on a model without it), they keep the provider's own
|
|
281
|
+
default as before (`none`, `low` for an OpenAI o-series model; for Claude,
|
|
282
|
+
Cursor and Grok `none` means the model's default). A Claude Cloud conversation always runs with `none`: the
|
|
283
|
+
cloud takes no effort. A board action's follow-up takes the conversation's
|
|
284
|
+
effort as well.
|
|
285
|
+
- **Mode**: a send that names no `relayMode`/`mode` runs in the
|
|
286
|
+
conversation's saved `preferred_relay_mode`, not the relay default.
|
|
287
|
+
- **Preferences**: a send to an existing conversation saves only what it named.
|
|
288
|
+
A model it named is saved (with its effort only when that was named too, else
|
|
289
|
+
the stored effort stays); an effort alone is saved against the stored model,
|
|
290
|
+
and a conversation with no saved model runs with that effort but saves
|
|
291
|
+
nothing; a send that named neither writes nothing. The mode saved with them
|
|
292
|
+
is the one the turn runs in, so a send without a mode leaves the stored mode
|
|
293
|
+
as it is. A send from another relay's agent never writes them, as before.
|
|
294
|
+
|
|
295
|
+
A text repeat inside the duplicate window answers `200` with `duplicate: true`
|
|
296
|
+
and `queued: false`, and nothing runs. A message whose turn **failed** no
|
|
297
|
+
longer counts: sending it again queues it.
|
|
258
298
|
|
|
259
|
-
|
|
260
|
-
(`gpt-4o` by default). Use the provider toggle to switch new conversations
|
|
261
|
-
between OpenAI and GitHub Copilot without re-entering the saved key. While
|
|
262
|
-
enabled, newly created extension-managed conversation workers start with Copilot
|
|
263
|
-
CLI BYOK environment variables pointing to `https://api.openai.com/v1`.
|
|
264
|
-
Zero-message conversations follow provider changes; started conversations keep
|
|
265
|
-
their original provider and lock the composer to their assigned model.
|
|
299
|
+
### OpenAI API key
|
|
266
300
|
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
OpenAI
|
|
270
|
-
|
|
271
|
-
|
|
301
|
+
Open **Settings → Providers → OpenAI** to save an OpenAI API key, the default model ID (`gpt-4o`
|
|
302
|
+
if none) and an optional **Base URL** (`https://api.openai.com/v1` when empty; any
|
|
303
|
+
OpenAI-compatible endpoint, or the capture proxy `server/tools/capture-proxy.mjs`). **Enable OpenAI
|
|
304
|
+
API key for New Chat model selection** makes OpenAI selectable in New Chat without re-entering the
|
|
305
|
+
key. An OpenAI conversation runs on the Copilot engine the relay has selected, extension or SDK:
|
|
306
|
+
the worker is started with `COPILOT_PROVIDER_TYPE=openai`, `COPILOT_PROVIDER_BASE_URL` (the saved
|
|
307
|
+
base URL), `COPILOT_PROVIDER_API_KEY`, `COPILOT_PROVIDER_WIRE_API` and `COPILOT_MODEL`
|
|
308
|
+
(`applyOpenAIProviderEnvironment`); the CLI reads them on the extension engine, the SDK worker
|
|
309
|
+
turns them into the session's provider (`copilot-worker/copilot-byok-provider.mjs`). Started
|
|
310
|
+
conversations keep their provider and lock the composer to their model.
|
|
311
|
+
|
|
312
|
+
GPT-5, `codex-*` and the o1/o3/o4 models use the Responses API; other OpenAI models use Chat
|
|
313
|
+
Completions (`resolveOpenAIWireApi`). The reasoning efforts follow the model family
|
|
314
|
+
(`shared/openai-reasoning.mjs`): GPT-5 and `codex` offer `none`, `low`, `medium`, `high` and
|
|
315
|
+
`xhigh`, the o-series `low`, `medium` and `high`, other models only `none` (image models take
|
|
316
|
+
`auto`, `low`, `medium` and `high` as their quality). An effort the model does not offer is refused
|
|
317
|
+
with `400`; with none named the conversation's own applies (clamped to the model's levels), else
|
|
318
|
+
`none`, or `low` on a model without `none`. GitHub Copilot's reasoning choices stay separate even when both providers list the
|
|
319
|
+
same model ID.
|
|
272
320
|
|
|
273
321
|
Saving or re-enabling the key refreshes OpenAI's `/v1/models` list and adds
|
|
274
322
|
candidate OpenAI model IDs to the composer picker. The Settings model-variant
|
|
@@ -280,11 +328,13 @@ the Settings model is only the default. Starting a chat from a locked OpenAI
|
|
|
280
328
|
conversation opens an independent model chooser without changing the active
|
|
281
329
|
conversation.
|
|
282
330
|
|
|
283
|
-
Disabling OpenAI retains the key and cached model list.
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
331
|
+
Disabling OpenAI retains the key and cached model list. **Remove saved key** deletes the key and the
|
|
332
|
+
cached list and switches OpenAI off, and it is refused with `409 openai-key-removal-blocked`
|
|
333
|
+
(`activeConversationCount`, `startedConversationCount`, `activeQueueConversationCount`) while any
|
|
334
|
+
OpenAI conversation that is not deleted, image conversations and archived ones included, has a
|
|
335
|
+
message or a pending, processing or parked queue row (`setOpenAIProviderSettings` in
|
|
336
|
+
`server-runtime.mjs`). Disable OpenAI instead, or delete those conversations first. A removal that
|
|
337
|
+
goes through moves the OpenAI conversations without a message to GitHub Copilot.
|
|
288
338
|
|
|
289
339
|
### Claude models
|
|
290
340
|
|
|
@@ -375,10 +425,14 @@ cached in the worker after the server accepts it, so a failed persist is retried
|
|
|
375
425
|
The worker keeps one persistent CLI process per conversation, so background tasks survive the reply
|
|
376
426
|
that started them. The live task set is posted to `POST /api/background-tasks` (REPLACE semantics)
|
|
377
427
|
and broadcast as the `background_tasks` socket event; the composer's task panel renders it and can
|
|
378
|
-
stop an individual task over a `worker.control` push.
|
|
428
|
+
stop an individual task over a `worker.control` push. The conversation payload carries the set too
|
|
429
|
+
(`backgroundTasks`, `endedBackgroundTasks`); the page's live poll and refresh leave it out when a
|
|
430
|
+
`background_tasks` event for the conversation arrived while their request was on its way, as the
|
|
431
|
+
event's set is at least as new.
|
|
379
432
|
|
|
380
433
|
The relay holds the task sets in memory (`services/background-task-store.mjs`), so a relay restart
|
|
381
|
-
starts without them while the workers and their tasks keep running
|
|
434
|
+
starts without them while the workers and their tasks keep running (the console rows are stored;
|
|
435
|
+
see below). The worker is the source of
|
|
382
436
|
truth, and the relay asks:
|
|
383
437
|
|
|
384
438
|
- **Digest and resync.** A worker's heartbeat carries `backgroundTasks: { count, digest }`, a digest
|
|
@@ -407,6 +461,43 @@ truth, and the relay asks:
|
|
|
407
461
|
- **Ends.** The Claude worker keeps a settle notice whose post failed (the newest 20, for 10
|
|
408
462
|
minutes) and sends it again after the next heartbeat the relay answers; such a post is given up
|
|
409
463
|
after 10 s without an answer.
|
|
464
|
+
- **The console rows.** What the task console needs of a shell or Monitor task (its `outputPath`
|
|
465
|
+
and `outputKind`, the ended row with status, exit code and `endedAt`, how a task ended when the
|
|
466
|
+
settle came before the set dropped it) is kept in `background_task_rows` (migration 0018,
|
|
467
|
+
`services/background-task-persistence.mjs`), written only when that part of a set changes, not on
|
|
468
|
+
every progress post. At boot the store restores the ended rows still inside their hour, and holds
|
|
469
|
+
the live ones (`restored`): the output route and the console's live room find a held task (the
|
|
470
|
+
route answers it with `status: 'held'`, `held: true` and `stoppable: false`; the console says it
|
|
471
|
+
waits for its worker), but `get()` and with it the panel, the session state and the suspend check
|
|
472
|
+
do not count it as running, and the stop route only pushes the stop to the worker. The worker's next set settles it (a task in the set is live again, with the
|
|
473
|
+
path it had; one not in it becomes an ended row, with the status the runtime before was told), a
|
|
474
|
+
settle sent again for it ends it, and while a key has held tasks its heartbeat digest never
|
|
475
|
+
counts as in step, so the relay asks for the set. Tasks of a worker that sends no set within twice
|
|
476
|
+
the boot reporting window (at least 15 s; 90 s by default) are ended with status `unknown` and the
|
|
477
|
+
pages get the `background_tasks` event (`endUnconfirmed`); a later set that lists one brings it
|
|
478
|
+
back. A held task's end time is when the runtime before last knew it running: a stored live row's
|
|
479
|
+
`updated_at` is written with the set and confirmed at most every 30 minutes while the worker's
|
|
480
|
+
digest agrees, and a live row not confirmed for 6 hours (a worker that went away in an earlier
|
|
481
|
+
runtime without an empty set) is deleted at boot instead of restored. Health readings are not
|
|
482
|
+
stored: the poller reads the files again within 10 s, the ended rows once. A runner's counts come
|
|
483
|
+
from the last 64 KB when its closing summary is there. A live log that outgrew that window is read
|
|
484
|
+
whole by a per-task runner scan (`services/task-output-service.mjs`): it looks for a runner's
|
|
485
|
+
header (a test run after a long install or build prints its header once) and counts from there,
|
|
486
|
+
reads on from where it stopped as the file grows, and starts over on a new file under the path
|
|
487
|
+
(shorter, another inode, an older mtime). The scans live in memory and share 4 MB per tick, each
|
|
488
|
+
tick starting at another task, so after a restart a long log is caught up over a few ticks (the
|
|
489
|
+
window's reading stands until then) and one busy log cannot keep the others waiting. An ended row
|
|
490
|
+
never starts a scan, so the rows a restart brings back are read in their window only. What a
|
|
491
|
+
worker says about a task (description, summary, last tool call, and a shell row's `command`, which
|
|
492
|
+
the Claude worker takes from the Bash or Monitor call that started the task, Copilot from the
|
|
493
|
+
shell tool's arguments or the task registry, Grok from the terminal) goes through `cleanTaskText`
|
|
494
|
+
on the publish and settle routes: control codes and colours out, credentials scrubbed as in the
|
|
495
|
+
output, then cut (a command at 2,000 characters; a Grok description is its command line). It is
|
|
496
|
+
stored with the console rows, so a held or ended row keeps it; share links carry no task rows. The
|
|
497
|
+
`none` answers of the output route use the page's texts (`TASK_OUTPUT_NONE_MESSAGES` in
|
|
498
|
+
`public/app/task-console-render.mjs`). Rows go with their hour, the 20-per-key limit, a set that
|
|
499
|
+
drops them or the conversation's delete. Without the table (a failed migration) the store works in
|
|
500
|
+
memory as before.
|
|
410
501
|
- **Run rows left `running`.** A background sub-agent's row whose end was never written would count
|
|
411
502
|
as running work for a day. For a Claude worker that has reported its task set in this runtime, a
|
|
412
503
|
`running` background row that the set does not list and that nothing has updated for 15 minutes
|
|
@@ -458,12 +549,20 @@ runs in the background, and who writes its timeline.
|
|
|
458
549
|
outlives the turn that spawned it, so its frames arrive between turns and during later turns;
|
|
459
550
|
they go to its own record, never to a later turn's live lanes. A background launch's receipt
|
|
460
551
|
(`tool_use_result.status: 'async_launched'`) leaves the run running; its `task_notification`
|
|
461
|
-
ends it. Spawns carry `timelineOwner: 'worker'`.
|
|
552
|
+
ends it. Spawns carry `timelineOwner: 'worker'`. A finished background agent that the session
|
|
553
|
+
sends a new message (SendMessage) runs again as the same task: `task_started` names its task id
|
|
554
|
+
again, with a new `run_id` and the SendMessage call as `tool_use_id`. The recorder leads that id
|
|
555
|
+
back to the agent's first run, so the record goes on under the same run (a note marks where it
|
|
556
|
+
resumed, and the run reads `running` again until its next `task_notification`); the task panel
|
|
557
|
+
row, the settle notice and a targeted stop name that run too. Every timeline post also names the
|
|
558
|
+
run's `taskId`, for a worker that did not see the first run (see the route below).
|
|
462
559
|
- **Copilot and Cursor** publish their sub-agents on the live lanes only (`/api/activity`,
|
|
463
560
|
`/api/thought`, `/api/stream` with a `subagentRunId`); the relay copies each frame into the run's
|
|
464
561
|
timeline, uncut. Fields they do not report (model, tokens) stay blank.
|
|
465
562
|
- `POST /api/background-tasks` copies model, type, task id and tokens onto the run whose id is the
|
|
466
|
-
task's `toolUseId`, so the row keeps them after the in-memory task set is cleared.
|
|
563
|
+
task's `toolUseId`, so the row keeps them after the in-memory task set is cleared. A sub-agent
|
|
564
|
+
task whose `toolUseId` is no run (a resumed agent from a worker that did not see its first run)
|
|
565
|
+
gets the id of the conversation's run that carries its task id, so the row opens that record.
|
|
467
566
|
- Every `subagent_status` event carries the whole header and the run's own turn (`messageId`). A
|
|
468
567
|
turn that fails closes its foreground runs; background runs are closed by their worker, or by the
|
|
469
568
|
relay when the worker dies.
|
|
@@ -621,13 +720,15 @@ transaction as the conversation. Nothing is created at Anthropic here. `POST /ap
|
|
|
621
720
|
`providerType: 'claude-cloud'` on a conversation of another provider is `409
|
|
622
721
|
CLAUDE_CLOUD_REQUIRES_NEW_CONVERSATION`.
|
|
623
722
|
|
|
624
|
-
`POST /api/message` accepts
|
|
625
|
-
|
|
626
|
-
|
|
627
|
-
|
|
628
|
-
|
|
629
|
-
|
|
630
|
-
|
|
723
|
+
`POST /api/message` accepts any model of the list, before and after the first message, and stores
|
|
724
|
+
it on the runtime session; a model outside the list is `400 CLAUDE_CLOUD_MODEL_UNAVAILABLE`. The
|
|
725
|
+
delivery carries the conversation's model as `providerModel`, which the worker's `resolveModel`
|
|
726
|
+
prefers over the message's own model and over `CLAUDE_CLOUD_RELAY_MODEL` (the launch model). The
|
|
727
|
+
first message creates the session with it; before a later message whose model is not the one the
|
|
728
|
+
worker last gave the sandbox, the worker sends a `set_model` control request (`applyModel`; after a
|
|
729
|
+
worker restart once, since it cannot know). A `set_model` the API refuses costs the switch, not the
|
|
730
|
+
turn. Reasoning effort is always `none` and the context tier always `default`, whatever the request
|
|
731
|
+
says.
|
|
631
732
|
|
|
632
733
|
### Delivery, binding and resume
|
|
633
734
|
|
|
@@ -701,8 +802,10 @@ Per-turn behavior:
|
|
|
701
802
|
as a terminal error result even though the API key is still valid, so the worker recreates the
|
|
702
803
|
handle and retries a single time; a second auth failure is treated as a real key problem.
|
|
703
804
|
- **Session root** — the worker keeps its per-session agent store at
|
|
704
|
-
`$CURSOR_AGENT_STORE_DIR
|
|
705
|
-
|
|
805
|
+
`$CURSOR_AGENT_STORE_DIR/<sdkSessionId>`, else `cursor-agents/<sdkSessionId>` in the relay's data
|
|
806
|
+
dir (`~/.oar/data` on a global install, `server/data` in a checkout), created on the first turn. A
|
|
807
|
+
session with a store at the earlier default `server/data/cursor-agents` keeps using it there when
|
|
808
|
+
it has none in the data dir, or when its `agent.db` there changed last. The explorer's Session root resolves to it via `resolveCursorSessionRoot`.
|
|
706
809
|
- Cursor turns are skipped by `fetchUsageSummary` like OpenAI and Claude turns — no Copilot usage
|
|
707
810
|
line is attached.
|
|
708
811
|
|
|
@@ -717,12 +820,27 @@ carries its own expiry.
|
|
|
717
820
|
began. A turn that legitimately runs for hours of tool calls keeps pushing the cutoff forward.
|
|
718
821
|
`processing_at` is only the fallback for rows with no owner (session-worker routing disabled).
|
|
719
822
|
While the CLI is offline the window shrinks to 45 s.
|
|
720
|
-
2. **Absolute ceiling (Settings → Max turn duration, default 60 min).** Measured from
|
|
721
|
-
`
|
|
722
|
-
|
|
723
|
-
`
|
|
823
|
+
2. **Absolute ceiling (Settings → Max turn duration, default 60 min).** Measured from the later of
|
|
824
|
+
`processing_at` and the turn's last answered question (`relay_questions.answered_at`): answering
|
|
825
|
+
a question restarts the clock, so a long interview is not cut off the moment its last answer
|
|
826
|
+
arrives. `0` disables it; the slider range is 0–600 minutes in 5-minute steps. This is a
|
|
827
|
+
backstop for a worker that has hung while still heartbeating, and is stored in the DB app
|
|
828
|
+
setting `turn_ceiling_minutes` — not in `config.json`. Shared bounds and formatting live in
|
|
724
829
|
`shared/turn-ceiling.mjs`.
|
|
725
830
|
|
|
831
|
+
Both guards are one SQL rule in `message-repository.mjs`, shared by the listing
|
|
832
|
+
(`listRecoverableProcessing`) and the requeue (`recoverProcessingBefore`, one row at a time,
|
|
833
|
+
re-checking the rule as a safeguard; the sweep runs synchronously, so nothing changes between
|
|
834
|
+
the two). The sweep (`services/stale-turn-recovery.mjs`) applies the retry budget to the listed
|
|
835
|
+
rows, requeues the rest and announces each one it requeued with `message_status`.
|
|
836
|
+
|
|
837
|
+
The sweep has no provider filter, so Grok rows are requeued by the rule above too. The Grok worker
|
|
838
|
+
also enforces the ceiling itself (delivered with each queue message, `grok-worker/acp-client.mjs`
|
|
839
|
+
`sessionPrompt`): counted from when it sent the prompt, it cancels the turn and fails it as
|
|
840
|
+
`grok.turn-stalled`. That check does not pause for question cards (on Grok: remote-relay approval
|
|
841
|
+
cards through the OAR MCP server), and an answer does not restart its clock. Whichever trips first
|
|
842
|
+
ends the turn.
|
|
843
|
+
|
|
726
844
|
## Subagent runs and live streaming
|
|
727
845
|
|
|
728
846
|
- `POST /api/stream` and `POST /api/activity` / `POST /api/thought` accept an optional
|
|
@@ -762,6 +880,9 @@ be on, as for every local action; a paired relay: see [Across paired relays](#ac
|
|
|
762
880
|
watched. The first subscribe of a conversation shows the user an approval card naming the
|
|
763
881
|
sessions; a subscription to `all` is always named on a card (asked again after an Allow that did
|
|
764
882
|
not cover it). The Allow is remembered as unlock rows `self:watch` / `self:watch-all`.
|
|
883
|
+
`events: ["relay_restarted"]` alone needs no sessions, watches none and shows no card (see
|
|
884
|
+
[relay_restarted](#relay_restarted-woken-when-the-relay-runs-again)); on a paired relay it is
|
|
885
|
+
refused, by the tool and by that relay's subscribe route.
|
|
765
886
|
- `unsubscribe {subscription_id | "all"}`, `list_subscriptions {watcher?}` (another conversation's
|
|
766
887
|
subscriptions, to inspect before a transfer).
|
|
767
888
|
- `transfer_subscriptions {from}` — moves every active subscription of another conversation, with
|
|
@@ -773,10 +894,22 @@ be on, as for every local action; a paired relay: see [Across paired relays](#ac
|
|
|
773
894
|
**Events** (`shared/relay-events.mjs`): `question_asked` (prompt, options, question id; an approval
|
|
774
895
|
card is marked as one only the user can answer), `error` (`turn_failed`, `usage_limit`,
|
|
775
896
|
`worker_died`), `turn_end` (status `done` or `stopped`, reply preview), `task_finished` /
|
|
776
|
-
`task_failed` (background sub-agents and shell tasks: kind, status, exit code where known
|
|
777
|
-
`stalled` (no activity for `stalled_minutes`, default 15, while a
|
|
778
|
-
question waits for an answer), `all_idle`
|
|
779
|
-
(nothing runs in the watched set any more). `task_started` only when named in `events`.
|
|
897
|
+
`task_failed` (background sub-agents and shell tasks: kind, status, exit code where known; a
|
|
898
|
+
non-zero exit is `task_failed`), `stalled` (no activity for `stalled_minutes`, default 15, while a
|
|
899
|
+
turn or task runs and no question waits for an answer), `all_idle`
|
|
900
|
+
(nothing runs in the watched set any more). `task_started` only when named in `events`.
|
|
901
|
+
`subagent_finished` is no event of its own but a narrower `task_finished` a subscription can name
|
|
902
|
+
(`RELAY_EVENT_SUBSCRIBABLE_TYPES`): the end of a task that is not a shell command, delivered as a
|
|
903
|
+
`task_finished` line. The default (`events` left out) is every type but `task_started` and
|
|
904
|
+
`task_finished` (and `relay_restarted`), with `subagent_finished`: a shell command that ended well wakes only a watcher
|
|
905
|
+
that names `task_finished`, which takes every task end. A busy worker ends a shell command every
|
|
906
|
+
minute or so, and each was a wake-up, a model turn on the watcher's whole context. A subscription
|
|
907
|
+
whose stored list is exactly the old default (every type but `task_started`, in any order) named no
|
|
908
|
+
events and is read as the new default (`effectiveRelayEventTypes`, at read time, the row is never
|
|
909
|
+
rewritten, so an older relay reading the same database sees what it stored); any other list,
|
|
910
|
+
including the old default plus more, is the watcher's choice and stays. A subscribe that names
|
|
911
|
+
exactly the old default list stores `subagent_finished` with it, so it is never read as the default. A mirror shows the paired
|
|
912
|
+
relay's list as that relay returned it. With
|
|
780
913
|
`heartbeat_minutes` a `heartbeat` event comes at that interval while the watched set has work
|
|
781
914
|
running, none while it is idle. `match` (a case-insensitive regular expression) filters `turn_end`
|
|
782
915
|
and task events by their text; other events pass. `once` ends a subscription after its first event,
|
|
@@ -824,6 +957,49 @@ touched for a day is not counted as running meanwhile.
|
|
|
824
957
|
and the background-only worker death (worker_died), `/api/relay-question` and the approval card's
|
|
825
958
|
notify (question_asked), the activity tracker's task edges, and the archive and delete routes.
|
|
826
959
|
|
|
960
|
+
### relay_restarted: woken when the relay runs again
|
|
961
|
+
|
|
962
|
+
`relay_restarted` is an event of this relay, not of a session (`RELAY_EVENT_RELAY_LEVEL_TYPES`):
|
|
963
|
+
a subscription gets it only when it names it, never by default, and one that names nothing else
|
|
964
|
+
needs no sessions. An agent that is about to request a restart subscribes with `once: true` first:
|
|
965
|
+
`POST /api/relay/shutdown` has no caller identity (agents call it with the token, `requestedBy` is
|
|
966
|
+
free text, and agents do not know their conversation id), while the tool knows the calling conversation.
|
|
967
|
+
|
|
968
|
+
- **How the run before ended.** A clean end of the runtime writes one `app_settings` row
|
|
969
|
+
(`relay_last_exit`: `action`, `reason`, `requested`, `requestedBy`, `requestedAt`, `stoppedAt`,
|
|
970
|
+
`noWake`): the queued (or `immediate`) request from the restart queue's `startShutdown`, else
|
|
971
|
+
`shutdownRuntime` (a signal, the owner process gone). The first note of a runtime wins; a runtime
|
|
972
|
+
stopped before it listened writes none, so the record of the run before stays for the next start.
|
|
973
|
+
- **At startup** (`noteRelayStarted`, once the HTTP server listens: a start that fails to bind
|
|
974
|
+
leaves the row for the next one) the relay reads and deletes that row in the same transaction
|
|
975
|
+
that records one `relay_restarted` event for every active subscription of this relay's live
|
|
976
|
+
watchers that names it (not a paired relay's, not a mirror), so each start is told once; a
|
|
977
|
+
`once` subscription ends with it. Without a row the line says the relay did not record how it
|
|
978
|
+
stopped (for example a crash, a forced stop or a reboot). On Windows a stop by the supervisor or
|
|
979
|
+
the CLI extension ends the process without its signal handler, so an ordinary stop there may read
|
|
980
|
+
like that too. A row older than 30 days (`RELAY_LAST_EXIT_MAX_AGE_MS`) is not taken as the run
|
|
981
|
+
just before (an older OAR, which neither writes nor takes it, may have run in between): it is
|
|
982
|
+
dropped, its `noWake` with it. A younger one is used; the line shows its stop time, so a relay
|
|
983
|
+
stopped over a weekend still says how. Delivery is that of every event: coalesced for 30 s (the workers
|
|
984
|
+
report back first), counted toward the hourly cap, written into an event row that is still
|
|
985
|
+
pending.
|
|
986
|
+
- **Loop guard.** On every `POST /api/relay/shutdown` (`noteRestartRequested`) the relay notes the
|
|
987
|
+
conversations whose processing turn answers a `relay_restarted` event (its queue row's
|
|
988
|
+
`relay_events`); they go into `noWake` and are not woken by that restart (logged), so a woken
|
|
989
|
+
turn that restarts again ends the chain. That holds for any restart requested while such a turn
|
|
990
|
+
runs (the button, an update, another agent): the API cannot tell who asked. The line says so up
|
|
991
|
+
front. A cancelled (or never queued) request forgets them. A held-back `once` subscription ends
|
|
992
|
+
(`held-back`, logged): that restart was the one it was for, and a later one would wake it with a
|
|
993
|
+
stale "carry on". If it also watches sessions, that ends with it: their events stop too. One
|
|
994
|
+
without `once` stays, and a later restart requested while no such turn
|
|
995
|
+
runs wakes it again. `heartbeat_minutes` on a subscription without sessions is refused (nothing
|
|
996
|
+
runs in an empty set).
|
|
997
|
+
- The line: `[relay event] relay_restarted: the relay runs again since <time>, after a restart
|
|
998
|
+
requested by "<requestedBy>" at <time> (reason "<reason>"). Timers and jobs your CLI scheduled
|
|
999
|
+
before may be gone: check what you were waiting for and carry on. A restart requested while this
|
|
1000
|
+
turn runs does not wake you again.` A stop by signal reads `after it was stopped (SIGTERM) at
|
|
1001
|
+
<time>` (a process id in the runtime's own reason is left out).
|
|
1002
|
+
|
|
827
1003
|
### Across paired relays
|
|
828
1004
|
|
|
829
1005
|
The watching relay (**A**, it holds the watcher) and the watched one (**B**) each keep a row of the
|
|
@@ -890,8 +1066,11 @@ finished tasks and starts the clock at boot. Running tasks come back when the wo
|
|
|
890
1066
|
sets again (see [Background tasks and workflow progress](#background-tasks-and-workflow-progress)):
|
|
891
1067
|
a task that started before this runtime did is not announced as started a second time, and the end
|
|
892
1068
|
of a task this runtime never knew is still an event when the settle notice describes the task
|
|
893
|
-
(`description`, `taskType`, `startedAt`).
|
|
894
|
-
|
|
1069
|
+
(`description`, `taskType`, `startedAt`). A task that ended stays ended when a later set lists it
|
|
1070
|
+
again, unless the set names another run of it (`taskRunId`, the SDK's `run_id`): a resumed Claude
|
|
1071
|
+
sub-agent keeps its task id, so its new run is a start (and later an end) of its own, and a settle
|
|
1072
|
+
notice of the earlier run that arrives late does not end it. `all_idle` waits out the first 45
|
|
1073
|
+
seconds after a relay start, like the suspend check.
|
|
895
1074
|
|
|
896
1075
|
`GET /api/conversations` and `GET /api/conversation/:id` carry `sessionState` (`running_turn`: a
|
|
897
1076
|
turn processing, or queued and not held by a usage-limit pause; `background_only`: tasks running;
|
|
@@ -900,8 +1079,10 @@ turn processing, or queued and not held by a usage-limit pause; `background_only
|
|
|
900
1079
|
`completed` | `failed` | `stopped`, `exitCode`, `summary`, `endedAt`) and `lastActivityAt`.
|
|
901
1080
|
`remote_relay`'s `list_sessions` and `read_session` pass them on as `state`, `tasks` and
|
|
902
1081
|
`lastActivityAt`; `active` stays and is true for background work too. `read_session` adds to each
|
|
903
|
-
task its `taskId`, `output` (whether it has output to read)
|
|
904
|
-
|
|
1082
|
+
task its `taskId`, `output` (whether it has output to read), a shell task's `command` (its command
|
|
1083
|
+
line where the worker reports one, scrubbed of credentials and cut at 2,000 characters by the relay
|
|
1084
|
+
when the worker posts it, shown to at most 500) and, for a task with an output file, `health`
|
|
1085
|
+
(`minutesSinceOutput`, `quiet`, `quietAfterMinutes`, test-run `counts`);
|
|
905
1086
|
`read_task_output {session, task_id, lines?}` (read permission; default 200 lines, at most 500 or
|
|
906
1087
|
32 KB) reads the end of that output through `GET /api/conversation/:id/background-task/:taskId/output`,
|
|
907
1088
|
scrubbed of credentials.
|
|
@@ -1011,7 +1192,8 @@ The composer also includes a per-message mode picker:
|
|
|
1011
1192
|
|
|
1012
1193
|
Mode is stored with each queued message so the relay can change behavior per turn.
|
|
1013
1194
|
Clarification prompts from the CLI are forwarded back into the browser as question
|
|
1014
|
-
cards with a reply box
|
|
1195
|
+
cards with a reply box; a card created with `allowFreeform: false` (approval and
|
|
1196
|
+
permission cards) has none and is answered with its choices.
|
|
1015
1197
|
|
|
1016
1198
|
`plan` mode behavior:
|
|
1017
1199
|
- Draft a concise implementation plan only (no implementation output).
|
|
@@ -1091,60 +1273,92 @@ Question bridge rule:
|
|
|
1091
1273
|
|
|
1092
1274
|
## Session Worker Management
|
|
1093
1275
|
|
|
1094
|
-
|
|
1095
|
-
|
|
1096
|
-
|
|
1097
|
-
|
|
1098
|
-
|
|
1099
|
-
|
|
1100
|
-
|
|
1101
|
-
|
|
1102
|
-
|
|
1103
|
-
|
|
1104
|
-
|
|
1105
|
-
|
|
1106
|
-
|
|
1107
|
-
|
|
1108
|
-
|
|
1109
|
-
|
|
1110
|
-
|
|
1111
|
-
-
|
|
1112
|
-
|
|
1113
|
-
|
|
1114
|
-
|
|
1115
|
-
|
|
1116
|
-
|
|
1117
|
-
|
|
1118
|
-
|
|
1119
|
-
-
|
|
1120
|
-
|
|
1121
|
-
-
|
|
1122
|
-
|
|
1123
|
-
|
|
1124
|
-
|
|
1125
|
-
|
|
1126
|
-
|
|
1127
|
-
|
|
1128
|
-
-
|
|
1129
|
-
|
|
1130
|
-
- `
|
|
1131
|
-
|
|
1132
|
-
|
|
1133
|
-
|
|
1134
|
-
|
|
1135
|
-
|
|
1136
|
-
|
|
1137
|
-
|
|
1138
|
-
|
|
1139
|
-
|
|
1140
|
-
|
|
1141
|
-
|
|
1142
|
-
|
|
1143
|
-
|
|
1144
|
-
|
|
1145
|
-
|
|
1146
|
-
|
|
1147
|
-
|
|
1276
|
+
Each conversation has its own worker process, keyed by the conversation's SDK session id: one
|
|
1277
|
+
worker per session, and a session's turns never move to another worker. A Copilot conversation on
|
|
1278
|
+
the *Extension* engine runs the `copilot` CLI with the web-relay extension; every other one
|
|
1279
|
+
(Claude, Claude Cloud, Cursor, Grok, and Copilot on the *SDK* engine; OpenAI conversations run on
|
|
1280
|
+
whichever Copilot engine is selected) runs a plain Node worker (`NODE_WORKER_DESCRIPTORS` in
|
|
1281
|
+
`services/session-worker-launch-service.mjs`). The supervisor
|
|
1282
|
+
(`services/session-worker-supervisor-service.mjs`) keeps the lifecycle, health and restart limits,
|
|
1283
|
+
the launch service spawns, `services/session-worker-stop-service.mjs` stops, and the dead-worker
|
|
1284
|
+
recovery lives in `server-runtime.mjs`. Most of it acts only while the
|
|
1285
|
+
`SESSION_WORKER_ROUTING_ENABLED` feature flag is on.
|
|
1286
|
+
|
|
1287
|
+
### Lifecycle and health
|
|
1288
|
+
|
|
1289
|
+
- Registry status: `starting` → `ready` ⇄ `processing`, and `error`. A send, the primer for queued
|
|
1290
|
+
messages and `POST /api/session-worker/:sdkSessionId/launch` call `ensureWorker`, which reuses a
|
|
1291
|
+
live process of the session (found in the process list, or the tmux pane) and launches one
|
|
1292
|
+
otherwise.
|
|
1293
|
+
- Heartbeats: every worker posts `POST /api/heartbeat` every 10 s, and any frame on its worker
|
|
1294
|
+
socket counts as one too. The supervisor uses them only at the start: a launched worker that has
|
|
1295
|
+
sent none within 90 s (`heartbeatTimeoutMs`; a Copilot CLI takes about 35 s to boot) is degraded
|
|
1296
|
+
as `startup-heartbeat-timeout`. After its first heartbeat a worker is judged by its pid, not by
|
|
1297
|
+
missed beats.
|
|
1298
|
+
- Health (`uiState`, in `/api/status` with the worker list): `green` healthy, `red` a question card
|
|
1299
|
+
waits, `yellow` degraded, `white` nothing known. `degradedReason` is `startup-heartbeat-timeout`,
|
|
1300
|
+
`stale-pid` (the pid is gone while a turn or a question waits for the worker), `spawn-failed`,
|
|
1301
|
+
`restart-exhausted` or the worker's own last error. Yellow clears after 10 s of a healthy worker
|
|
1302
|
+
without a new failure (`degradedRecoveryGraceMs`).
|
|
1303
|
+
- An idle (`ready`) worker whose process is gone and that has not heartbeated for 90 s leaves the
|
|
1304
|
+
list; the next message launches a new one. A live idle worker is never evicted by time
|
|
1305
|
+
(`idleEvictionMs` is 0).
|
|
1306
|
+
|
|
1307
|
+
### A worker that dies
|
|
1308
|
+
|
|
1309
|
+
- When a worker's socket closes while its session owns a processing row, holds the composer or has
|
|
1310
|
+
background sub-agents running, the relay checks 15 s later (`WORKER_DEATH_GRACE_MS`) whether the
|
|
1311
|
+
process is still alive. A sweep every 30 s checks the owners of all processing rows the same way.
|
|
1312
|
+
- `recoverDeadWorkerProcessingRows` then drops a continuation row, fails a row the worker had
|
|
1313
|
+
already consumed or that is over `maxRequeueRetries` (5), and puts the others back to `pending`
|
|
1314
|
+
(2 s later, same owner session, one retry more). The session counts a failure (`markError`), the
|
|
1315
|
+
dead worker's sub-agent runs are closed, and the primer launches a new worker for the waiting rows.
|
|
1316
|
+
- A worker that is alive but has had no socket for 60 s while a message waits is replaced (see
|
|
1317
|
+
[Worker link reconnects](#worker-link-reconnects)).
|
|
1318
|
+
|
|
1319
|
+
### Launch failures and backoff
|
|
1320
|
+
|
|
1321
|
+
A launch that throws, a worker that died or a queue row it failed counts a failure: the next launch
|
|
1322
|
+
of that session waits 1 s, doubling per failure (1, 2, 4, 8, 16 s; capped at 30 s), for 5 retries (`maxRestartRetries`).
|
|
1323
|
+
Then the session is `restart-exhausted`: one more launch is allowed 60 s after the last failure,
|
|
1324
|
+
for up to 10 minutes after the first exhaustion, and after that only when a launch is asked for
|
|
1325
|
+
(the conversation's Retry, the launch or kill route, a relay restart). While a session's worker
|
|
1326
|
+
cannot be started the conversation shows a note (`services/worker-launch-note-service.mjs`). A
|
|
1327
|
+
successful launch resets the count.
|
|
1328
|
+
|
|
1329
|
+
### Stopping a worker
|
|
1330
|
+
|
|
1331
|
+
`stopSessionWorkerProcesses` serves **☠️ Kill session** (`POST /api/session-worker/:sdkSessionId/kill`),
|
|
1332
|
+
the CWD relaunch, a provider rebind and the replacement of a worker without a socket. It reads the
|
|
1333
|
+
session's processes from a fresh process list and checks that they are gone: on Linux and macOS
|
|
1334
|
+
SIGTERM to each and `tmux kill-session`, up to 3 s, then SIGKILL to the survivors and up to 2 s
|
|
1335
|
+
more; on Windows `Stop-Process -Force` on the process tree, leaf first; processes still alive after 3 s
|
|
1336
|
+
are stopped once more, with any new ones on a fresh list, and get up to 2 s more. A process list that cannot be read stops nothing (`process-list-unreadable`), and processes
|
|
1337
|
+
that survive are an error (`409` from the kill route). The kill route fails the rows the worker
|
|
1338
|
+
owned and blocks launches of the session for 3 s (`killBlockGraceMs`), so the primer does not
|
|
1339
|
+
start it again at once; the launch route clears that block.
|
|
1340
|
+
|
|
1341
|
+
### Launch modes and environment
|
|
1342
|
+
|
|
1343
|
+
- **Linux/macOS with `tmux`:** a detached tmux session named after the SDK session id
|
|
1344
|
+
(`tmux attach -t <sdk-session-id>`), running `sh -lc`: the extension engine's CLI under `script`
|
|
1345
|
+
for a pseudo-TTY, a Node worker with its output appended to its worker log.
|
|
1346
|
+
- **Linux/macOS without tmux:** a detached child process, a Node worker's output into its log.
|
|
1347
|
+
- **Windows:** `cmd /c start` opens one console window per worker, titled `<Kind> Worker <first 8
|
|
1348
|
+
characters of the session id>`; the extension engine runs `gh copilot -- --allow-all
|
|
1349
|
+
--session-id <id>`.
|
|
1350
|
+
|
|
1351
|
+
The launch environment is the relay's own, taken once at boot (`buildSessionWorkerLaunchEnv`), plus
|
|
1352
|
+
per launch the conversation's working directory as `COPILOT_WORKSPACE_ROOT` and `INIT_CWD`,
|
|
1353
|
+
`SESSION_ID`, `COPILOT_ALLOW_ALL`, and the bound provider's worker kind, model and credentials
|
|
1354
|
+
(`buildSessionWorkerLaunchEnvForSession`). The detached and Windows launches pass all of it. Under
|
|
1355
|
+
tmux the worker's login shell gets only the variables `buildTmuxWorkerShellCommand` exports, and
|
|
1356
|
+
the secrets (`COPILOT_PROVIDER_API_KEY`, `CURSOR_API_KEY`, `CLAUDE_CODE_OAUTH_TOKEN` for cloud
|
|
1357
|
+
workers) through an owner-only file it reads and deletes; any other variable, such as
|
|
1358
|
+
`OAR_TURN_STALL_*_MS`, `CLAUDE_CODE_AUTO_COMPACT_WINDOW` or `XAI_API_KEY`, only from the tmux
|
|
1359
|
+
server's environment or the login profile (see [Session workers under
|
|
1360
|
+
tmux](../docs/configuration.md#environment-variables)). `COPILOT_WEB_RELAY_DISABLE_CLI_SPAWN` makes
|
|
1361
|
+
every launch fail (`cli-spawn-disabled`), for test relays.
|
|
1148
1362
|
|
|
1149
1363
|
## How It Works
|
|
1150
1364
|
|
|
@@ -1163,6 +1377,7 @@ After launching the server, the CLI opens a worker WebSocket link:
|
|
|
1163
1377
|
- Identifies itself with the bound SDK session id and worker pid
|
|
1164
1378
|
- Receives queued turns over `ws(s)://.../api/session-worker/ws`
|
|
1165
1379
|
- Reconnects starting at 1 second, with exponential backoff capped at 8 seconds
|
|
1380
|
+
(see [Worker link reconnects](#worker-link-reconnects))
|
|
1166
1381
|
- Falls back to `GET /api/pending` only while the worker socket is unavailable
|
|
1167
1382
|
- Processes any message with full PC access (file system, commands, etc.)
|
|
1168
1383
|
- Posts the response back
|
|
@@ -1177,6 +1392,33 @@ After launching the server, the CLI opens a worker WebSocket link:
|
|
|
1177
1392
|
- Answered relay question cards stay visible in the conversation journal, including
|
|
1178
1393
|
the answer you submitted.
|
|
1179
1394
|
|
|
1395
|
+
### Worker link reconnects
|
|
1396
|
+
|
|
1397
|
+
Every session worker (Copilot SDK, Claude, Claude Cloud, Cursor, Grok) and the Copilot CLI extension
|
|
1398
|
+
use the same link, `shared/worker-runtime/worker-websocket-link.mjs`. Its reconnect rules:
|
|
1399
|
+
|
|
1400
|
+
- A connection counts as made at the relay's first `server.hello`, which the relay sends as soon as it
|
|
1401
|
+
accepts the socket. A socket still connecting, or open without that hello, after 15 seconds
|
|
1402
|
+
(`connectTimeoutMs`) is given up and retried: a relay hung in its shutdown can hold a socket in
|
|
1403
|
+
either state without ever ending it.
|
|
1404
|
+
- The link ends a socket on whichever comes first of an error, a close, that deadline, a stale
|
|
1405
|
+
connection (30 seconds without a server frame) and the relay's `server.draining`, and schedules
|
|
1406
|
+
the next attempt from there; later events from the same socket are ignored. It never waits for a
|
|
1407
|
+
close event, which Node 22 does not fire for a socket closed while connecting, and a socket whose
|
|
1408
|
+
constructor throws is retried too.
|
|
1409
|
+
- There is at most one pending reconnect and one socket at a time. The delay starts at 1 second and
|
|
1410
|
+
doubles up to 8 seconds, plus up to 250 ms of jitter; it starts over only after a completed
|
|
1411
|
+
handshake, so a relay that accepts and drops the socket is not retried every second.
|
|
1412
|
+
- Each failed attempt logs one `worker ws disconnected <reason> attempt=<n> retryInMs=<ms>` line in
|
|
1413
|
+
the worker log for the first three, then at most one a minute, with `notLogged=<n>` counting the
|
|
1414
|
+
ones in between.
|
|
1415
|
+
- `stop()` (worker shutdown) clears the pending reconnect and the deadline and closes the socket
|
|
1416
|
+
without scheduling another attempt.
|
|
1417
|
+
|
|
1418
|
+
The relay's detached-worker watch (`services/detached-worker-watch-service.mjs`) is the safety net
|
|
1419
|
+
behind this: it replaces a worker that has had no socket for 60 seconds while a message waits, but
|
|
1420
|
+
never while a turn or background work of that session runs, so the link has to reconnect by itself.
|
|
1421
|
+
|
|
1180
1422
|
### Session activation behavior (important)
|
|
1181
1423
|
|
|
1182
1424
|
In extension-managed mode, the web server can already be running while the CLI is still
|
|
@@ -1205,11 +1447,11 @@ If a conversation is still unbound when the CLI first sees it, the extension now
|
|
|
1205
1447
|
|
|
1206
1448
|
### Worker Launcher
|
|
1207
1449
|
|
|
1208
|
-
-
|
|
1209
|
-
|
|
1210
|
-
|
|
1211
|
-
-
|
|
1212
|
-
|
|
1450
|
+
- Each worker opens its own console window through `cmd /c start`, titled `<Kind> Worker <first 8
|
|
1451
|
+
characters of the session id>` (`Copilot Worker`, `Claude Worker`, `Cursor Worker`, `Grok Worker`,
|
|
1452
|
+
`Claude Cloud Worker`, `Copilot SDK Worker`); a relaunch opens a new window.
|
|
1453
|
+
- Node workers run `node <kind>-session-worker.mjs --session-id <id>` directly; the extension engine
|
|
1454
|
+
runs `gh copilot -- --allow-all --session-id <id>`.
|
|
1213
1455
|
|
|
1214
1456
|
### Path Handling
|
|
1215
1457
|
|
|
@@ -1290,10 +1532,13 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
|
|
|
1290
1532
|
| POST | `/api/relay/pause` | Pause dequeueing and drop currently queued messages |
|
|
1291
1533
|
| POST | `/api/relay/resume` | Resume dequeueing after pause |
|
|
1292
1534
|
| POST | `/api/relay/shutdown` | Queue an authenticated relay shutdown or self-restart |
|
|
1535
|
+
| GET | `/api/relay/shutdown` | The queued request's state (as `relay_shutdown_state`), plus `preview`: what a restart asked now would wait for and leave running (`?action=shutdown` for a shutdown). See [Manual relay shutdown / self-restart](#manual-relay-shutdown--self-restart) |
|
|
1536
|
+
| POST | `/api/relay/shutdown/keep-waiting` | Answer to the max-wait question: the queued request waits another full period, counted from now → `{ ok, extended, …state }` |
|
|
1537
|
+
| POST | `/api/relay/shutdown/cancel` | Withdraw a queued (not yet started) shutdown or restart; held messages go out at once |
|
|
1293
1538
|
| GET | `/api/relay-questions` | (CLI/UI) List relay questions by `status` (for example `pending` or `answered`) |
|
|
1294
1539
|
| GET | `/api/relay-question/:id` | (CLI) Fetch a single relay question |
|
|
1295
|
-
| POST | `/api/relay-question` | (CLI) Create a relay question for the browser |
|
|
1296
|
-
| POST | `/api/relay-question/:id/answer` | (UI) Submit an answer for a relay question |
|
|
1540
|
+
| POST | `/api/relay-question` | (CLI) Create a relay question for the browser. `allowFreeform` defaults to `false` when `choices` are given (to `true` without): such a card has no reply box and takes only its choices |
|
|
1541
|
+
| POST | `/api/relay-question/:id/answer` | (UI) Submit an answer for a relay question. A card created with `allowFreeform: false` and choices (approval and permission cards) takes only its choices, matched trimmed and in any case and stored as the card's label (several, `", "`-joined, on a multi-select card); anything else is refused with `400` and `choices`, and the card stays open |
|
|
1297
1542
|
| POST | `/api/relay-question/:id/timeout` | (CLI/UI) Mark a relay question timed out |
|
|
1298
1543
|
| POST | `/api/heartbeat` | (CLI) Keep CLI status alive. A session worker may send `backgroundTasks: { count, digest }`; the answer carries `resyncBackgroundTasks: true` when the relay's set for that worker differs |
|
|
1299
1544
|
| GET | `/api/status` | Overall status |
|
|
@@ -1314,6 +1559,8 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
|
|
|
1314
1559
|
| POST | `/api/settings/cursor` | Set/remove the Cursor API key, default model, or enabled model subset; key changes reset discovered models and effort tiers |
|
|
1315
1560
|
| GET | `/api/settings/turn-ceiling` | Read the max turn duration plus slider bounds (`minMinutes`, `maxMinutes`, `stepMinutes`, `defaultMinutes`) |
|
|
1316
1561
|
| POST | `/api/settings/turn-ceiling` | Set the max turn duration in minutes (`0` = no limit) |
|
|
1562
|
+
| GET | `/api/settings/relay-restart-max-wait` | How long a queued relay restart waits before it asks: `{ maxWaitMinutes, choices: [5, 10, 15, 30, 60, 120, 240, 0], defaultMinutes: 30, overrideMs }`; `0` = no limit; `overrideMs` is `OAR_RELAY_RESTART_MAX_WAIT_MS` when set |
|
|
1563
|
+
| POST | `/api/settings/relay-restart-max-wait` | `{ maxWaitMinutes }` → the GET body; anything but one of `choices` is `400`. A queued request measures its current period against the new value at once |
|
|
1317
1564
|
| GET | `/api/settings/subagent-record-retention` | How long sub-agent records keep tool inputs and output: `{ retentionDays, choices: [0, 7, 14, 30, 90], defaultDays: 90 }`; `0` = keep for good |
|
|
1318
1565
|
| POST | `/api/settings/subagent-record-retention` | `{ retentionDays }` → the GET body; anything but one of `choices` is `400`. Runs a retention sweep 5 s later |
|
|
1319
1566
|
| POST | `/api/claude-native-session` | (Claude worker) Persist the native Agent SDK session id for resume |
|
|
@@ -1322,9 +1569,9 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
|
|
|
1322
1569
|
| POST | `/api/claude-cloud-session` | (Claude Cloud worker) Report the cloud session a conversation is bound to: `{ conversationId, cloudSessionId, sessionUrl?, lastSequence, pushedBranch?, costUsd?, model? }` → `{ ok, cloud }`. Sent after the session was created, after every `result` and on a push; emits `claude_cloud_session`. `404` without a runtime session, `409` for a conversation that is not a cloud one |
|
|
1323
1570
|
| POST | `/api/cursor-plan-usage` | (Cursor worker) Report the agent's cumulative billed usage; the relay diffs it against a per-agent checkpoint and books the increase into the current billing cycle under the pool implied by the turn's model |
|
|
1324
1571
|
| POST | `/api/subagent-run` | (Worker) Register/update a subagent run. A new run needs the spawning turn's `messageId`; an update to a known run may omit it (a background agent's end arrives between turns). Optional header fields: `subagentType`, `model`, `modelInherited`, `taskId`, `background`, `timelineOwner`; a post without `status` leaves the run's state alone. Emits `subagent_status` with the whole header |
|
|
1325
|
-
| POST | `/api/subagent-timeline` | (Worker) `{ conversationId, subagentRunId, entries: [{ key, kind, text, done?, toolName?, toolUseId?, metadata? }], header? }` → `{ ok, stored, truncated }`. Kinds: `thought`, `tool_call`, `tool_result`, `text`, `child`, `note`. Fenced by the run row, not by an open turn; an unknown run whose `header.parentSubagentId` is known is created under the parent's turn, anything else is `404 unknown_run`. Sends `subagent_timeline` `{ conversationId, messageId, subagentRunId, entries, truncated }`, volatile, only to the pages watching the run (`subagent_viewer_watch`) |
|
|
1572
|
+
| POST | `/api/subagent-timeline` | (Worker) `{ conversationId, subagentRunId, taskId?, entries: [{ key, kind, text, done?, toolName?, toolUseId?, metadata? }], header? }` → `{ ok, stored, truncated }`. Kinds: `thought`, `tool_call`, `tool_result`, `text`, `child`, `note`. Fenced by the run row, not by an open turn; an unknown run whose `header.parentSubagentId` is known is created under the parent's turn, an unknown run whose `taskId` (or `header.taskId`) a run of the conversation carries is written to that run (a resumed agent; when the post sets that finished run `running` again, the relay adds a note where it resumed), anything else is `404 unknown_run`. `header.status: 'running'` clears the run's end time. Sends `subagent_timeline` `{ conversationId, messageId, subagentRunId, entries, truncated }`, volatile, only to the pages watching the run (`subagent_viewer_watch`) |
|
|
1326
1573
|
| GET | `/api/conversation/:conversationId/subagent/:subagentRunId` | One run's record for the viewer: `?after=<seq>&limit=<n>` (default 500, max 2000) → `{ run, entries, children, nextAfter, complete }`. `entries` are those after `after` plus any still open; `run.legacy` marks a record rebuilt from the lane rows |
|
|
1327
|
-
| POST | `/api/conversation/:conversationId/subagent/:subagentRunId/cancel` |
|
|
1574
|
+
| POST | `/api/conversation/:conversationId/subagent/:subagentRunId/cancel` | Queue an `abort_subagent` control for the session's worker (`{ parentMessageId?, clientId? }` → `{ ok, queued, acknowledgement }`; `not-found`, `message-mismatch`, `already-cancelled` and `already-finished` queue nothing). The Claude worker stops a background sub-agent (`stopTask`), the Copilot SDK worker a background agent (`rpc.tasks.cancel`), the extension engine tries the SDK session's abort methods; any other run, or worker, answers "not supported". Workers read controls only while a turn runs. Whole-turn **Stop** always works |
|
|
1328
1575
|
| PATCH | `/api/conversation/:id/message/:messageId/share-visibility` | Hide or unhide a single message from shared views |
|
|
1329
1576
|
| PATCH | `/api/conversation/:id/message/:messageId/pin` | `{ pinned }` → pin or unpin a single message; answers with the conversation's pin list and broadcasts `conversation_pins_updated` |
|
|
1330
1577
|
| POST | `/api/conversation/:id/share` | The conversation's read-only share link, made if it has none (one active share per conversation): `{ token, shareUrl, path, created }`; a new one sends `share_created` |
|
|
@@ -1346,12 +1593,12 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
|
|
|
1346
1593
|
| POST | `/api/remote-relays/:id/check` | Probe a paired relay now |
|
|
1347
1594
|
| POST | `/api/remote-relays/pair` | (Relay to relay) Another relay introduces itself after the user paired it there |
|
|
1348
1595
|
| GET | `/api/remote-relays/summary` | (Worker) `{ count, online, localEnabled }`: the number of paired relays, how many of them are online, and whether agent sessions are switched on. Workers register `remote_relay` when `count` is above zero or `localEnabled` is `true` |
|
|
1349
|
-
| POST | `/api/remote-relays/tool` | (Worker) One `remote_relay` tool call: `{ conversationId, action, args }`. For a paired relay the relay checks the mention unlock, the permission, the hop limit and (ask/plan mode) the user's approval, then forwards the call to the remote. For this relay itself (named by its name or `this`; refused with `REMOTE_RELAY_LOCAL_DISABLED` while agent sessions are off) it checks instead that the call does not aim at the caller's own conversation (`REMOTE_RELAY_OWN_SESSION`) and, for `create_session`, that the caller was not created by an agent (`REMOTE_RELAY_NESTED_SESSION`), that fewer than 4 sessions it created have a turn queued or running (`REMOTE_RELAY_SESSION_LIMIT`) and that the user allowed it once for the conversation; the call then runs against this relay's own routes in-process. `wait_seconds` is cut to the `maxWaitSeconds` setting for every target |
|
|
1596
|
+
| POST | `/api/remote-relays/tool` | (Worker) One `remote_relay` tool call: `{ conversationId, action, args }`. For a paired relay the relay checks the mention unlock, the permission, the hop limit and (ask/plan mode) the user's approval, then forwards the call to the remote. For this relay itself (named by its name or `this`; refused with `REMOTE_RELAY_LOCAL_DISABLED` while agent sessions are off) it checks instead that the call does not aim at the caller's own conversation (`REMOTE_RELAY_OWN_SESSION`) and, for `create_session`, that the caller was not created by an agent (`REMOTE_RELAY_NESTED_SESSION`), that fewer than 4 sessions it created have a turn queued or running (`REMOTE_RELAY_SESSION_LIMIT`) and that the user allowed it once for the conversation; the call then runs against this relay's own routes in-process. A `model` in `create_session` or `send` is checked first, before any approval card, against the list New Chat offers for the provider on the target (`/api/models` for Copilot and OpenAI, the provider settings routes for Claude, Cursor, Grok and Claude Cloud): one it does not list is refused with `REMOTE_RELAY_UNKNOWN_MODEL`, `suggestions` (the close matches) and `models`, and nothing is created or sent. Casing follows the list; a Claude alias (`opus`, `sonnet`, `haiku`, with an optional `[1m]`) resolves to the newest listed model of its family by version number (a date suffix ignored, the undated id first, `[1m]` ids only when the alias asks and they are listed); a `[1m]` id with only its base listed runs as the base, with a note; a Copilot id with its effort (`gpt-5.4-high`) goes to the bootstrap and the first prompt as the listed id plus `reasoningEffort` (an explicit `effort` wins, and a note says so), but only when the catalogue lists that effort for the id (`reasoningByModel`): `gpt-5.1-codex-max` with only `gpt-5.1-codex` listed is unknown, with the base suggested; `auto` passes. `send` also takes the provider's whole discovered list (`availableModels`) and the session's own model, as the message route does. Without a list (discovery has not run, the route did not answer; `auto` alone is no list) the model goes on unchecked and `modelNote` says so, as it does when the bootstrap binds another model than the one asked. A provider not set up there answers `REMOTE_RELAY_PROVIDER_UNAVAILABLE` before its model is looked at. The bootstrap route itself checks a Copilot model from an agent against Copilot's own models: an unknown one is refused (`COPILOT_MODEL_UNAVAILABLE`, with `supportedModels` and `suggestedModels`; another provider's model gets that provider's answer), a listed one is bound in the list's casing, an effort-suffixed one as the listed id plus that effort when `reasoningByModel` lists it; the browser's New Chat keeps its fallback to the current model. `wait_seconds` is cut to the `maxWaitSeconds` setting for every target |
|
|
1350
1597
|
| GET | `/api/relay-events/subscriptions` | Every active [relay event subscription](#relay-event-subscriptions) (`{ available, subscriptions }`); refused to other relays' agents |
|
|
1351
1598
|
| DELETE | `/api/relay-events/subscriptions/:id` | The user's Unsubscribe (a paired relay's subscription too: it learns with its next fetch); refused to other relays' agents |
|
|
1352
1599
|
| POST | `/api/relay-events/remote/subscribe` | (Relay to relay) A paired relay's agent subscribes to sessions here: `{ watcher: { conversationId, title }, subscriber: { name }, scope, sessions, events?, once?, match?, stalled_minutes?, heartbeat_minutes?, dry_run? }` → `{ ok, subscription }`. Another relay's request only, inbound switch on; see [Across paired relays](#across-paired-relays) |
|
|
1353
1600
|
| POST | `/api/relay-events/remote/pull` | (Relay to relay) That relay fetches its events: `{ after, subscriptions: [{ id, watcher }], ended: [ids] }` → `{ ok, events, cursor, more, subscriptions, ended, dropped }` |
|
|
1354
|
-
| POST | `/api/background-task-settled` | (Worker) How a background task ended: `{ conversationId, taskId, toolUseId?, status, summary?, reason?, description?, taskType?, startedAt? }` (Claude worker, from `task_notification`; the last three describe a task that was in its published set) |
|
|
1601
|
+
| POST | `/api/background-task-settled` | (Worker) How a background task ended: `{ conversationId, taskId, toolUseId?, taskRunId?, status, summary?, reason?, description?, taskType?, startedAt? }` (Claude worker, from `task_notification`; the last three describe a task that was in its published set) |
|
|
1355
1602
|
| GET | `/api/remote-relays/inflight` | (Worker) Number of `remote_relay` calls running for `conversationId` |
|
|
1356
1603
|
| GET/POST | `/api/settings/remote-relays` | This relay's `publicUrl` and `inboundEnabled` (accept prompts from other relays' agents) |
|
|
1357
1604
|
| GET | `/api/settings/agent-sessions` | Agent sessions: `{ enabled, maxWaitSeconds, limits: { minWaitSeconds: 120, maxWaitSeconds: 3600, stepSeconds: 60 }, maxActiveSessions: 4 }`. `enabled` (default `false`) lets this relay's agents start and use sessions on it through `remote_relay`; `maxWaitSeconds` (default `600`) is the longest wait of one `remote_relay` call, for this relay and paired ones |
|
|
@@ -1365,6 +1612,7 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
|
|
|
1365
1612
|
- `reason` (optional string)
|
|
1366
1613
|
- `requestedBy` (optional string)
|
|
1367
1614
|
- `restart` (optional boolean-ish flag; `true` queues self-restart instead of plain shutdown)
|
|
1615
|
+
- `immediate` (optional boolean-ish flag; `true` does not wait for the running turns)
|
|
1368
1616
|
- Example restart request:
|
|
1369
1617
|
|
|
1370
1618
|
```json
|
|
@@ -1381,14 +1629,61 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
|
|
|
1381
1629
|
- `restart`: boolean mirror of the action
|
|
1382
1630
|
- `requestedAt`, `reason`, `requestedBy`
|
|
1383
1631
|
- `queue`: current `pendingCount` / `processingCount` / `parkedCount`
|
|
1384
|
-
-
|
|
1632
|
+
- `waitingFor`, `held`, `backgroundTasks`, `survival`, `maxWaitMs`, `deadlineAt`, `immediate` (below)
|
|
1633
|
+
- A conversation that subscribed to `relay_restarted` is woken when the relay runs again, with who requested it, when and why ([relay_restarted](#relay_restarted-woken-when-the-relay-runs-again)); one whose turn answered that event when the request came is not woken by it.
|
|
1634
|
+
- The relay waits until no turn is processing before acting. While the request is queued no pending message is handed to a worker (a turn started meanwhile would postpone the restart again); pending and parked rows stay in the database and run after the restart.
|
|
1635
|
+
- `immediate: true` (boolean-ish, like `restart`) starts it at once, running turns or not: the banner's **Restart now**. The answer still names what it did not wait for.
|
|
1636
|
+
- Maximum wait (`services/relay-restart-queue.mjs`): after **Ask after a queued relay restart has waited** (`relay_restart_max_wait_minutes`, 30 by default, 5–240 or `0` = no limit; `OAR_RELAY_RESTART_MAX_WAIT_MS` overrides it in ms, for test relays) the request stays queued and sets `question: { askedAt }`; the relay pushes one notification (titled *Relay restart waiting*; switch **Queued relay restart waited its maximum time**, `relayRestart`) and never restarts while turns run; it still goes ahead as soon as the last turn ends, question or not. A raised maximum (or none) that moves the deadline into the future withdraws the question; `/keep-waiting` without a question does nothing (`extended: false`). An `immediate` request may name the queued request it was shown (`expectRequestedAt`, its `requestedAt`); when that request is gone the answer is `accepted: false, stale: true` and nothing starts. Agents are told never to send `immediate` (`relay-tools.md`). The user answers with `immediate`, `/keep-waiting` (another full period from now; counted in `keptWaiting`) or `/cancel`. With no limit the request, and the messages it holds, wait until the turns end or someone decides.
|
|
1637
|
+
- What it waits for, holds and leaves running (`services/relay-restart-picture.mjs`), in the answer, in `GET /api/relay/shutdown`, in `/api/status` → `relayShutdown` and in the `relay_shutdown_state` socket event (sent on every change and on connect):
|
|
1638
|
+
- `waitingFor`: `[{ conversationId, title, turns }]`, the sessions with a processing turn
|
|
1639
|
+
- `held`: `{ count, conversations: [{ conversationId, title, count }] }`, pending and parked messages
|
|
1640
|
+
- `backgroundTasks`: `[{ conversationId, title, count, detail, survives, why }]`, sessions with running background work of a live worker (the task panel's sets and running sub-agent runs, through `background-work-view.mjs`), then sessions with console tasks held since the last restart (`background-task-store.mjs` `listRestored`), which no worker has confirmed yet: `survives: null`
|
|
1641
|
+
- `survival`: `{ action, workersSurvive, summary }`. Workers are separate processes (tmux, detached, a console window) and outlive a restart, which the relay's supervisor or the CLI extension takes; they outlive a shutdown too, except under a systemd service (detected from `INVOCATION_ID` and a `.service` cgroup), whose stop may take the workers with it (`survives: false`, shown as "may stop"; a worker in a tmux session the service did not start survives, which the relay cannot tell). A Claude Cloud session runs at Anthropic and always survives
|
|
1642
|
+
- while queued also `holdsNewTurns`, `maxWaitMs`, `waitingSince` (start of the current period), `deadlineAt` (`null` without a limit), `question`, `keptWaiting`
|
|
1385
1643
|
- Ownership after an intentional restart follows the argv role, so exactly one supervisor acts on exit 75:
|
|
1386
1644
|
- `server.js --supervised` exits 75 and relaunches under `.github/extensions/web-relay/server-lifecycle/managed-server.mjs`
|
|
1387
1645
|
- bare `node server/server.js` keeps its supervisor attached, respawns the `--relay-runtime` child, and keeps the same terminal session alive
|
|
1388
1646
|
|
|
1647
|
+
### Interrupted turns after a restart
|
|
1648
|
+
|
|
1649
|
+
`services/restart-resume-service.mjs` implements **After a relay restart, tell interrupted sessions
|
|
1650
|
+
to carry on** (`app_settings` `restart_resume_notes`, `'1'` on, off when absent; `GET/POST
|
|
1651
|
+
/api/settings/restart-resume` `{ enabled }`, a boolean, 400 otherwise). At boot it notes the queue
|
|
1652
|
+
rows that are `processing` (the turns the runtime before left) and follows them through the
|
|
1653
|
+
recovery paths, which report each outcome (`noteRestartTurnOutcome`):
|
|
1654
|
+
|
|
1655
|
+
| Path | `cut` (cannot run again) | `requeued` | `failed` |
|
|
1656
|
+
| --- | --- | --- | --- |
|
|
1657
|
+
| stale sweep (`recoverProcessingOlderThan`) | stale continuation dropped; consumed row failed | requeued | over the retry budget |
|
|
1658
|
+
| dead worker (`recoverDeadWorkerProcessingRows`) | continuation dropped; consumed row failed | requeued | over the retry budget |
|
|
1659
|
+
| owner heartbeat idle (`recoverOwnedProcessingRowsForSession`) | continuation dropped; consumed row failed; a started turn failed as `turn-replay-blocked` | requeued | |
|
|
1660
|
+
|
|
1661
|
+
A heartbeat that lists a row (`activeQueueMessageId(s)`) claims it: its worker lived through the
|
|
1662
|
+
restart and has the turn, so whatever ends it later is not put down to the restart, and the
|
|
1663
|
+
session gets no note. A row that leaves `processing` another way is read from the queue: back in `pending` or `parked` is
|
|
1664
|
+
`requeued`, anything else (a reply from a worker that lived through the restart) is settled. Once
|
|
1665
|
+
none of a conversation's rows of the boot is processing, it gets a note when one was `cut` and none
|
|
1666
|
+
was `requeued`, `claimed` or `failed`; the stale sweep's 15 s tick decides the ones that settled on their own,
|
|
1667
|
+
and the watch ends 30 minutes after the boot. The note is a message of the relay's (role `user`,
|
|
1668
|
+
`kind = 'relay-restart-resume'`) with a pending queue row on the interrupted turn's model, mode and
|
|
1669
|
+
owner, queued at the time it is written (behind a message the user queued meanwhile), as the
|
|
1670
|
+
usage-limit follow-up is; the primer then starts the worker. No note when the setting is off at
|
|
1671
|
+
that moment, for a deleted or archived conversation, while the conversation holds a usage-limit
|
|
1672
|
+
follow-up, or a second time in the same runtime. The browser renders it as a compact relay row
|
|
1673
|
+
(`relay-restart-note-view.mjs`).
|
|
1674
|
+
|
|
1389
1675
|
### Upload storage model
|
|
1390
1676
|
|
|
1391
|
-
- Physical blobs are stored in
|
|
1677
|
+
- Physical blobs are stored in `<data dir>/uploads/<sha256>` (content-addressed) when
|
|
1678
|
+
`COPILOT_WEB_RELAY_DATA_DIR` or the config's `dataDir` chose the data dir, else in
|
|
1679
|
+
`server/uploads/<sha256>` (`resolveUploadDir` in `services/data-dir-resolution.mjs`).
|
|
1680
|
+
- Blobs earlier versions left elsewhere are read in place: `server/uploads` (a config
|
|
1681
|
+
`dataDir` did not move uploads before 0.9.14) and `uploads/` in the folder that holds the
|
|
1682
|
+
data dir (state migrations before 0.9.14 copied a checkout's uploads there).
|
|
1683
|
+
`resolveUploadReadDirs` lists them after the upload dir; a blob is read from the first
|
|
1684
|
+
copy whose size is the recorded one (two of that size that are not alike, i.e. differ in
|
|
1685
|
+
their first or last 64 KiB: the one whose content hashes to its name, decided once per blob). New uploads are written to the upload dir only, and
|
|
1686
|
+
the unreferenced-upload sweep and a cache wipe delete only there.
|
|
1392
1687
|
- Metadata (`original_name`, `mime_type`, `size_bytes`) is stored in SQLite.
|
|
1393
1688
|
- Message/conversation references are tracked in SQLite; when a conversation is deleted,
|
|
1394
1689
|
unreferenced blobs are garbage-collected from disk automatically.
|
|
@@ -1412,16 +1707,18 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
|
|
|
1412
1707
|
- Images over ~2 MB are downscaled client-side to a 2560 px longest edge and re-encoded
|
|
1413
1708
|
to **JPEG** before upload. GIF and SVG are never re-encoded, and a re-encode that fails,
|
|
1414
1709
|
grows the file, or yields anything other than JPEG/PNG falls back to the original bytes.
|
|
1415
|
-
- The output format is deliberately restricted to JPEG/PNG: Copilot's
|
|
1416
|
-
these "the most widely supported formats".
|
|
1417
|
-
|
|
1710
|
+
- The output format is deliberately restricted to JPEG/PNG (`image-downscale.mjs`): Copilot's
|
|
1711
|
+
image-input docs call these "the most widely supported formats". The composer converts for
|
|
1712
|
+
every provider alike, although the Claude workers, local and cloud, would take a WebP as it is
|
|
1713
|
+
(`claude-attachments.mjs` inlines JPEG, PNG, GIF and WebP).
|
|
1418
1714
|
- Re-encoding therefore has two triggers, tracked by `reencodeReason()`:
|
|
1419
1715
|
- `'size'` — the image is over the byte threshold. Abandoned if the result is no smaller.
|
|
1420
1716
|
- `'format'` — the source is not PNG/JPEG/GIF (e.g. WebP, AVIF, HEIC), so it is converted
|
|
1421
1717
|
at its original dimensions **regardless of size**, and kept even if the result is larger:
|
|
1422
1718
|
a bigger image the model can read beats a smaller one it cannot.
|
|
1423
1719
|
- Without the `'format'` trigger a WebP small enough to skip the size threshold would reach
|
|
1424
|
-
the model
|
|
1720
|
+
the model in a format outside that pair. An image that does not come through the composer
|
|
1721
|
+
(`POST /api/upload` from a script) is stored as it is.
|
|
1425
1722
|
- Attachments upload **eagerly** on selection rather than on Send. Chips show an
|
|
1426
1723
|
uploading spinner, and a failed upload becomes a clickable retry.
|
|
1427
1724
|
- Send is disabled while any upload is in flight. Failed uploads are deliberately not
|
|
@@ -1483,7 +1780,7 @@ Queue metrics include `parkedCount` for turns deferred behind restart/rebind gat
|
|
|
1483
1780
|
- Use `/api/repo/list?path=<repo-relative-dir>&includeHidden=0|1&includeHeavy=0|1` for lazy-loaded workspace directory browsing.
|
|
1484
1781
|
- Use `/api/drives/roots` + `/api/drives/list?path=<path>&includeHidden=0|1` for lazy-loaded drive/root browsing (separate from workspace heavy mode).
|
|
1485
1782
|
- Use `/api/drives/file?path=<path>` and `/api/drives/files-preview?path=<path>` for drive/root file raw/preview access.
|
|
1486
|
-
- Use `/api/session-root/list?path=<sessionRootPath>&includeHidden=0|1` for the explorer's Session root. Child directories below it lazy-load through `/api/drives/list` as usual. The Claude Agent SDK creates a session's directory lazily — only once the session writes `subagents/` or `tool-results/` files — so the root is served as an empty folder until then rather than 404ing, and the transcript that lives one level up in the project directory is listed as a child of it. Cursor sessions resolve their Session root the same lazy way: the worker's per-session agent store (`$CURSOR_AGENT_STORE_DIR` or `
|
|
1783
|
+
- Use `/api/session-root/list?path=<sessionRootPath>&includeHidden=0|1` for the explorer's Session root. Child directories below it lazy-load through `/api/drives/list` as usual. The Claude Agent SDK creates a session's directory lazily — only once the session writes `subagents/` or `tool-results/` files — so the root is served as an empty folder until then rather than 404ing, and the transcript that lives one level up in the project directory is listed as a child of it. Cursor sessions resolve their Session root the same lazy way: the worker's per-session agent store (`$CURSOR_AGENT_STORE_DIR` or `cursor-agents/<sdkSessionId>` in the relay's data dir) exists only after the session's first turn, and the Session button stays disabled until it does.
|
|
1487
1784
|
- Requests are auth-protected and restricted to files inside the workspace root.
|
|
1488
1785
|
- **Platform split** — the drives API adapts automatically based on the server OS:
|
|
1489
1786
|
- **Windows:** drive roots are discovered via `fsutil.exe`; paths use Windows drive-letter format (`C:/foo/bar`); directory listing uses PowerShell.
|
|
@@ -1667,8 +1964,18 @@ The migration `server/migrations/0001-add-structured-answer.mjs` creates the `st
|
|
|
1667
1964
|
|
|
1668
1965
|
## Relay Tool Guidance
|
|
1669
1966
|
|
|
1670
|
-
The CLI extension (`.github/extensions/web-relay/extension.mjs`)
|
|
1671
|
-
|
|
1967
|
+
The CLI extension (`.github/extensions/web-relay/extension.mjs`) and the Copilot SDK worker
|
|
1968
|
+
(`copilot-worker/copilot-prompt-context.mjs`) load `relay-tools.md` (`COPILOT_WEB_RELAY_TOOLS`
|
|
1969
|
+
overrides the path) for shared tool guidance.
|
|
1970
|
+
|
|
1971
|
+
How to embed media in a reply (a markdown image naming the file's absolute host path; the page
|
|
1972
|
+
renders images and turns video and audio into players) is one text,
|
|
1973
|
+
`shared/media-embed-instructions.mjs`. The Claude worker appends it to the system prompt, the
|
|
1974
|
+
Copilot SDK worker to its relay guidance, the Cursor and Grok workers put it in front of the first
|
|
1975
|
+
prompt each worker process sends. The
|
|
1976
|
+
Copilot extension engine's prompt builder (`shared/worker-runtime/prompt-context.mjs`) does not add
|
|
1977
|
+
it and `relay-tools.md` does not contain it, and a Claude Cloud session gets no relay guidance at
|
|
1978
|
+
all, so those two are not told.
|
|
1672
1979
|
|
|
1673
1980
|
The browser UI keeps the usage button in the sidebar header, and that button continues to
|
|
1674
1981
|
call `/api/usage` directly. It now renders the unified plan-usage report — one card per
|
|
@@ -1684,7 +1991,6 @@ service).
|
|
|
1684
1991
|
"authToken": "<your-token>",
|
|
1685
1992
|
"port": 3333,
|
|
1686
1993
|
"localhostOnly": true,
|
|
1687
|
-
"pollIntervalMs": 3000,
|
|
1688
1994
|
"processingTimeoutMs": 600000,
|
|
1689
1995
|
"conversationSessionMode": "isolated",
|
|
1690
1996
|
"contextIndicatorMode": "default",
|
|
@@ -1714,17 +2020,31 @@ service).
|
|
|
1714
2020
|
|
|
1715
2021
|
| Key | Default | Description |
|
|
1716
2022
|
|---|---|---|
|
|
1717
|
-
| `authToken` |
|
|
1718
|
-
| `port` | `3333` | HTTP/WebSocket listen port |
|
|
2023
|
+
| `authToken` | generated if empty | Token for all API / UI access. Empty or missing: a random token on every start, printed to the log and not saved (`--token <value>` sets one for the run) |
|
|
2024
|
+
| `port` | `3333` | HTTP/WebSocket listen port (`--port <n>` for the run) |
|
|
1719
2025
|
| `localhostOnly` | `true` | Bind only to loopback (`127.0.0.1`) and block LAN/WAN access |
|
|
1720
|
-
| `
|
|
2026
|
+
| `host` | `127.0.0.1` | Read by the workers only: the address they connect to the relay on (`0.0.0.0`, `::` and `localhost` mean `127.0.0.1`) |
|
|
2027
|
+
| `dataDir` | `data/` beside the server | Database, the relay's lock file, uploads (`uploads/` inside it) and Cursor agent stores (`cursor-agents`); relative to the config file's folder; `COPILOT_WEB_RELAY_DATA_DIR` wins. Without either, a checkout keeps its uploads in `server/uploads`; uploads made there before `dataDir` was set stay there and are still read |
|
|
2028
|
+
| `pollIntervalMs` | — | Read by nothing; new config files still get `3000` |
|
|
1721
2029
|
| `processingTimeoutMs` | `600000` | Worker-inactivity window before a turn is treated as stale (ms) — see [Turn recovery](#turn-recovery-and-the-max-turn-duration) |
|
|
1722
|
-
| `
|
|
1723
|
-
| `
|
|
2030
|
+
| `maxRequeueRetries` | `5` | Retries of a recovered queue row before it fails |
|
|
2031
|
+
| `conversationSessionMode` | `isolated` | `isolated` or `shared`; recorded as `runtime_sessions.strategy` and reported in `/api/status`, with no effect on behaviour |
|
|
2032
|
+
| `contextIndicatorMode` | `default` | Composer context indicator: `default`, a line across the composer's top edge whose colour runs from green to red as the context fills, or `bar`, a line as long as the share in use; anything else is `default`. Sent to the page in `/api/status` |
|
|
2033
|
+
| `remotePath` | `""` | URL path prefix when the relay is served under a subpath (PWA URLs, socket.io path) |
|
|
2034
|
+
| `trustProxy` | `"loopback"` | Express `trust proxy`: which proxies may set `X-Forwarded-*` |
|
|
2035
|
+
| `workspaceRootAllowList` | unset | Folders conversations may start in; see [Changing the launch CWD](#changing-the-launch-cwd) |
|
|
2036
|
+
| `publicHostnames` | `[]` | The relay's own public hostnames; the preview lane refuses them |
|
|
2037
|
+
| `previews.enabled`, `.port`, `.bindHost`, `.allowPublicBind`, `.publicBaseUrl`, `.allowedTargetHosts`, `.maxLive` | `false`, `port + 1`, `127.0.0.1`, `false`, —, `[]`, `8` | The preview listener (`services/preview-registry-service.mjs`, [docs/preview-servers.md](../docs/preview-servers.md)) |
|
|
2038
|
+
| `pushVapidSubject` | `mailto:copilot-remote@example.com` | Contact URI sent to push services |
|
|
2039
|
+
| `sharedPresenceMaxPerConversation`, `sharedPresenceMaxGlobal` | `200`, `5000` | Caps of the shared-view watcher count |
|
|
2040
|
+
| `cliBinaries` | *(written by the relay)* | Provider CLI paths resolved by an install from Settings, applied at startup |
|
|
2041
|
+
| `cliPort` | `4445` | Port of the standalone `relay.mjs`; the preview lane refuses it |
|
|
2042
|
+
| `features` | — | Legacy: moved into the database once at startup, then the **Features** tab |
|
|
1724
2043
|
| `sdkVersion` | *(latest detected)* | Optional semver pin (for example `1.0.63`) used by relay SDK auto-detection |
|
|
1725
2044
|
| `sdkPath` | *(auto-detected)* | Optional absolute override for Copilot SDK entry path (`.../copilot-sdk/index.js`) |
|
|
1726
2045
|
| `cliPath` | *(auto-detected)* | Optional absolute override for Copilot CLI app path (`.../app.js`) |
|
|
1727
2046
|
| `restartGracefulTimeoutMs` | `8000` | Graceful shutdown wait before force fallback |
|
|
2047
|
+
| `restartReadyCooldownMs` | `1000` | How long the worker restart orchestrator still holds dequeue after a restart transaction is ready |
|
|
1728
2048
|
| `restartShutdownTimeoutMs` | `45000` | Drain timeout while waiting for active queue jobs |
|
|
1729
2049
|
| `restartSpawnTimeoutMs` | `18000` | Max wait for resume process to leave online state |
|
|
1730
2050
|
| `restartRebindTimeoutMs` | `20000` | Max wait for session-sync rebind confirmation |
|
|
@@ -1891,23 +2211,18 @@ reported as `relayShutdown`.
|
|
|
1891
2211
|
|
|
1892
2212
|
## Database Migrations
|
|
1893
2213
|
|
|
1894
|
-
|
|
1895
|
-
|
|
1896
|
-
|
|
1897
|
-
|
|
1898
|
-
|
|
1899
|
-
|
|
1900
|
-
|
|
1901
|
-
|
|
1902
|
-
|
|
1903
|
-
|
|
1904
|
-
|
|
1905
|
-
|
|
1906
|
-
**Currently Deployed Migration:**
|
|
1907
|
-
|
|
1908
|
-
- **`0001-add-structured-answer.mjs`** — Adds `structured_answer` and `request_schema` columns to the `relay_questions` table to support multi-field elicitation forms (see [Structured Answers and Multi-Field Elicitation](#structured-answers-and-multi-field-elicitation) section).
|
|
1909
|
-
- `request_schema` (TEXT/JSON) — JSON schema defining form fields and validation
|
|
1910
|
-
- `structured_answer` (TEXT/JSON) — User's submitted multi-field response object
|
|
2214
|
+
There is no migration runner and no record of applied migrations. At every start
|
|
2215
|
+
`applySchema(db)` (`server/db-schema.mjs`, the same code the tests build their databases with) runs
|
|
2216
|
+
`SCHEMA_SQL` (`CREATE TABLE IF NOT EXISTS` for a fresh database), then the upgrade sequence:
|
|
2217
|
+
guarded `ALTER TABLE … ADD COLUMN` statements (each runs only when `PRAGMA table_info` lacks the
|
|
2218
|
+
column) and, among them, the functions of `server/migrations/0002-…` to `0018-…` in order. Each of those checks the
|
|
2219
|
+
schema itself and does nothing when its change is already there, so all of them run on every boot.
|
|
2220
|
+
From 0004 on a failure is logged (`migration 00NN failed; retrying on next boot`) and the boot goes
|
|
2221
|
+
on. Every file can also be run by hand against a database: `node server/migrations/<file>.mjs
|
|
2222
|
+
[path/to/copilot.db]`. `0001-add-structured-answer.mjs` (the `structured_answer` and
|
|
2223
|
+
`request_schema` columns of `relay_questions`, see [Structured Answers and Multi-Field
|
|
2224
|
+
Elicitation](#structured-answers-and-multi-field-elicitation)) is only such a script: the boot adds
|
|
2225
|
+
those columns with its own guarded `ALTER TABLE`.
|
|
1911
2226
|
|
|
1912
2227
|
**In-place column additions:**
|
|
1913
2228
|
|
|
@@ -1951,6 +2266,13 @@ any length. A conversation with a row whose timestamp is not the relay's ISO for
|
|
|
1951
2266
|
(`YYYY-MM-DDTHH:MM:SS.mmmZ`) or whose id is not lowercase letters, digits and dashes is paged the
|
|
1952
2267
|
old way: every row loaded, the page cut afterwards.
|
|
1953
2268
|
|
|
2269
|
+
`0018-background-task-rows.mjs` adds `background_task_rows` (`store_key`, `task_id`, `state`
|
|
2270
|
+
`live` | `ended`, `row_json`, `ended_at`, `updated_at`; one row per key and task): the task
|
|
2271
|
+
console's rows across a relay restart (see
|
|
2272
|
+
[Background tasks and workflow progress](#background-tasks-and-workflow-progress)). Additive and
|
|
2273
|
+
idempotent; a failure is logged and retried on the next boot, and the store keeps its rows in
|
|
2274
|
+
memory meanwhile. An older build ignores the table.
|
|
2275
|
+
|
|
1954
2276
|
`0007-claude-cloud.mjs` adds the storage of [Claude Cloud](#claude-cloud-provider) conversations.
|
|
1955
2277
|
It is additive and idempotent, runs on every boot, and a failure is logged and retried on the next
|
|
1956
2278
|
one:
|
|
@@ -1964,26 +2286,26 @@ one:
|
|
|
1964
2286
|
- `conversations.cloud_source_json` (TEXT/JSON) — `{ repoUrl, branch, environmentId, sessionUrl,
|
|
1965
2287
|
pushedBranches: [{ branch, at }] }`
|
|
1966
2288
|
|
|
1967
|
-
|
|
1968
|
-
|
|
2289
|
+
Older tables are created at startup with `CREATE TABLE IF NOT EXISTS` in `SCHEMA_SQL` (`db-schema.mjs`),
|
|
2290
|
+
for example `workflow_runs` (final workflow digest per assistant message,
|
|
1969
2291
|
keyed on `response_message_id`, cascade-deleted with the conversation; see
|
|
1970
|
-
[Background tasks and workflow progress](#background-tasks-and-workflow-progress))
|
|
2292
|
+
[Background tasks and workflow progress](#background-tasks-and-workflow-progress)); newer ones, such as
|
|
2293
|
+
`background_task_rows`, come from a migration only (see below).
|
|
1971
2294
|
|
|
1972
2295
|
Statements that depend on these columns are prepared conditionally, so an older database keeps
|
|
1973
2296
|
working with the corresponding feature inert rather than crashing at startup.
|
|
1974
2297
|
|
|
1975
|
-
**
|
|
1976
|
-
|
|
1977
|
-
1. Create
|
|
1978
|
-
|
|
1979
|
-
|
|
1980
|
-
|
|
1981
|
-
|
|
1982
|
-
|
|
1983
|
-
|
|
1984
|
-
|
|
1985
|
-
|
|
1986
|
-
5. Verify: check relay logs for `[migrations] Applying 000N-description`
|
|
2298
|
+
**Adding a migration:**
|
|
2299
|
+
|
|
2300
|
+
1. Create `server/migrations/00NN-description.mjs` with the next number, exporting a synchronous
|
|
2301
|
+
`migrateSomething(db)` that checks the schema (`sqlite_master`, `PRAGMA table_info`) and changes
|
|
2302
|
+
only what is missing, plus a `migrate(dbPath)` and an `isMain` block for running it by hand
|
|
2303
|
+
(`0018-background-task-rows.mjs` is a small example).
|
|
2304
|
+
2. Call it from `applySchema` after the previous one, inside `try`/`catch` with a
|
|
2305
|
+
`migration 00NN failed; retrying on next boot` warning. It runs on a fresh database too, so
|
|
2306
|
+
what it creates needs no entry in `SCHEMA_SQL`.
|
|
2307
|
+
3. Prepare statements that need the change conditionally, so a database the migration has not
|
|
2308
|
+
reached keeps the feature inert instead of failing at startup.
|
|
1987
2309
|
|
|
1988
2310
|
## Files
|
|
1989
2311
|
|