@acryldev/plugin-context 0.0.0-stage → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +12 -2
- package/docs/README.md +160 -0
- package/docs/delivery/local-live.md +113 -0
- package/docs/delivery/marketplace.md +16 -0
- package/docs/docs.json +2836 -0
- package/docs/extending/agent-preset.md +37 -0
- package/docs/extending/chat-command.md +28 -0
- package/docs/extending/client-slot.md +135 -0
- package/docs/extending/config-schema.md +7 -0
- package/docs/extending/cordis-core.md +63 -0
- package/docs/extending/desktop-app.md +53 -0
- package/docs/extending/desktop-main.md +25 -0
- package/docs/extending/event-hook.md +14 -0
- package/docs/extending/host-route.md +33 -0
- package/docs/extending/llm-adapter.md +38 -0
- package/docs/extending/packaging.md +53 -0
- package/docs/extending/prompt-contribution.md +9 -0
- package/docs/extending/service.md +13 -0
- package/docs/extending/skill-provider.md +9 -0
- package/docs/extending/state-and-persistence.md +71 -0
- package/docs/extending/three-role-capability.md +33 -0
- package/docs/extending/tool-plugin.md +24 -0
- package/docs/extending/tui-command.md +8 -0
- package/docs/extending/tui-components.md +61 -0
- package/docs/extending/ui-branding.md +43 -0
- package/docs/extending/ui-components.md +56 -0
- package/docs/extending/ui-customization.md +41 -0
- package/docs/extending/ui-library.md +66 -0
- package/docs/extending/ui-theme.md +55 -0
- package/docs/extending/workspace-tab.md +50 -0
- package/docs/maps/events.md +108 -0
- package/docs/maps/mount-points.md +154 -0
- package/docs/maps/slot-contracts.md +3376 -0
- package/docs/maps/taxonomy.md +306 -0
- package/docs/maps/theme-tokens.md +115 -0
- package/docs/maps/ui-components.md +26 -0
- package/docs/reference/cookbook/adding-a-package.md +174 -0
- package/docs/reference/cookbook/adding-a-remote-api.md +199 -0
- package/docs/reference/cookbook/adding-a-settings-card.md +74 -0
- package/docs/reference/cookbook/adding-a-tool.md +103 -0
- package/docs/reference/cookbook/adding-an-llm-adapter.md +45 -0
- package/docs/reference/cookbook/extension-cookbook.md +134 -0
- package/docs/reference/cordis-api/context.md +366 -0
- package/docs/reference/cordis-api/events.md +209 -0
- package/docs/reference/cordis-api/fiber.md +377 -0
- package/docs/reference/cordis-api/inherited.md +40 -0
- package/docs/reference/cordis-api/registry.md +154 -0
- package/docs/reference/cordis-api/service.md +104 -0
- package/docs/reference/cordis-guides/cordis-usage-cheatsheet.md +167 -0
- package/docs/reference/cordis-guides/cordis_system_guide_for_coding_agents.md +4666 -0
- package/docs/reference/cordis-guides/development-canvas-plugin.md +109 -0
- package/docs/reference/cordis-guides/hello-world-plugin-guide.md +517 -0
- package/docs/reference/harness/agent-lifecycle.md +93 -0
- package/docs/reference/harness/architecture.md +168 -0
- package/docs/reference/harness/capability-seams.md +675 -0
- package/docs/reference/harness/config-catalog.md +4523 -0
- package/docs/reference/harness/cordis-primer.md +47 -0
- package/docs/reference/harness/defensive-patterns.md +35 -0
- package/docs/reference/harness/event-producer-consumer.md +111 -0
- package/docs/reference/harness/glossary.md +47 -0
- package/docs/reference/harness/tool-catalog.md +2721 -0
- package/docs/reference/harness/tool-execution-pipeline.md +67 -0
- package/docs/reference/subsystems/README.md +75 -0
- package/docs/reference/subsystems/agent-team.md +233 -0
- package/docs/reference/subsystems/approval.md +172 -0
- package/docs/reference/subsystems/attachment.md +363 -0
- package/docs/reference/subsystems/boot.md +324 -0
- package/docs/reference/subsystems/browser-use.md +69 -0
- package/docs/reference/subsystems/client-modules.md +186 -0
- package/docs/reference/subsystems/client-resources.md +94 -0
- package/docs/reference/subsystems/commands.md +225 -0
- package/docs/reference/subsystems/compaction.md +286 -0
- package/docs/reference/subsystems/computer-use.md +58 -0
- package/docs/reference/subsystems/conversation.md +287 -0
- package/docs/reference/subsystems/core.md +1107 -0
- package/docs/reference/subsystems/credentials.md +497 -0
- package/docs/reference/subsystems/deliverables.md +180 -0
- package/docs/reference/subsystems/extensions.md +386 -0
- package/docs/reference/subsystems/feedback.md +402 -0
- package/docs/reference/subsystems/filesystem.md +532 -0
- package/docs/reference/subsystems/goal.md +315 -0
- package/docs/reference/subsystems/invariants.md +90 -0
- package/docs/reference/subsystems/jobs.md +501 -0
- package/docs/reference/subsystems/llm-streaming.md +1131 -0
- package/docs/reference/subsystems/lsp.md +204 -0
- package/docs/reference/subsystems/mcp.md +141 -0
- package/docs/reference/subsystems/office-to-pdf.md +89 -0
- package/docs/reference/subsystems/otel.md +46 -0
- package/docs/reference/subsystems/permission-presets.md +169 -0
- package/docs/reference/subsystems/persistence.md +418 -0
- package/docs/reference/subsystems/plan.md +89 -0
- package/docs/reference/subsystems/product-telemetry.md +81 -0
- package/docs/reference/subsystems/ptc-runtime.md +234 -0
- package/docs/reference/subsystems/sandbox.md +223 -0
- package/docs/reference/subsystems/schedule.md +545 -0
- package/docs/reference/subsystems/scope.md +61 -0
- package/docs/reference/subsystems/session-projection.md +358 -0
- package/docs/reference/subsystems/session-query.md +511 -0
- package/docs/reference/subsystems/session-reference.md +226 -0
- package/docs/reference/subsystems/session-telemetry.md +214 -0
- package/docs/reference/subsystems/session-title.md +206 -0
- package/docs/reference/subsystems/session.md +1300 -0
- package/docs/reference/subsystems/settings.md +157 -0
- package/docs/reference/subsystems/shell.md +316 -0
- package/docs/reference/subsystems/sidebar-right.md +160 -0
- package/docs/reference/subsystems/skills.md +354 -0
- package/docs/reference/subsystems/slots.md +201 -0
- package/docs/reference/subsystems/spill.md +126 -0
- package/docs/reference/subsystems/ssh.md +137 -0
- package/docs/reference/subsystems/storage.md +262 -0
- package/docs/reference/subsystems/subagent.md +762 -0
- package/docs/reference/subsystems/subprocess.md +323 -0
- package/docs/reference/subsystems/system-prompt.md +226 -0
- package/docs/reference/subsystems/terminal.md +186 -0
- package/docs/reference/subsystems/todo.md +34 -0
- package/docs/reference/subsystems/token-meter.md +108 -0
- package/docs/reference/subsystems/tools.md +749 -0
- package/docs/reference/subsystems/typert.md +489 -0
- package/docs/reference/subsystems/user-questions.md +309 -0
- package/docs/reference/subsystems/voice-input.md +166 -0
- package/docs/reference/subsystems/web-client.md +102 -0
- package/docs/reference/subsystems/web-server.md +228 -0
- package/docs/reference/subsystems/web.md +208 -0
- package/docs/reference/subsystems/webhook.md +72 -0
- package/docs/reference/subsystems/workflow.md +280 -0
- package/docs/reference/subsystems/workspace.md +661 -0
- package/docs/start-here/blueprints.md +33 -0
- package/docs/start-here/this-runtime.md +84 -0
- package/docs/start-here/troubleshooting.md +29 -0
- package/docs/start-here/trust-and-safety.md +39 -0
- package/docs/start-here/verify-before-done.md +46 -0
- package/example-plugins/README.md +45 -0
- package/example-plugins/packages/agent-preset-reviewer/agent.cordis.yml +18 -0
- package/example-plugins/packages/agent-preset-reviewer/preset.yml +3 -0
- package/example-plugins/packages/capability-swap-consumer/cordis.patch.yml +3 -0
- package/example-plugins/packages/capability-swap-consumer/index.js +14 -0
- package/example-plugins/packages/capability-swap-consumer/package.json +25 -0
- package/example-plugins/packages/capability-swap-provider-loud/cordis.patch.yml +3 -0
- package/example-plugins/packages/capability-swap-provider-loud/index.js +20 -0
- package/example-plugins/packages/capability-swap-provider-loud/package.json +25 -0
- package/example-plugins/packages/capability-swap-provider-quiet/cordis.patch.yml +3 -0
- package/example-plugins/packages/capability-swap-provider-quiet/index.js +20 -0
- package/example-plugins/packages/capability-swap-provider-quiet/package.json +25 -0
- package/example-plugins/packages/chat-command-basic/cordis.patch.yml +3 -0
- package/example-plugins/packages/chat-command-basic/index.js +26 -0
- package/example-plugins/packages/chat-command-basic/package.json +25 -0
- package/example-plugins/packages/client-chat-message-action/client.js +46 -0
- package/example-plugins/packages/client-chat-message-action/cordis.patch.yml +3 -0
- package/example-plugins/packages/client-chat-message-action/index.js +9 -0
- package/example-plugins/packages/client-chat-message-action/package.json +31 -0
- package/example-plugins/packages/client-slot-header-action/client.js +57 -0
- package/example-plugins/packages/client-slot-header-action/cordis.patch.yml +3 -0
- package/example-plugins/packages/client-slot-header-action/index.js +13 -0
- package/example-plugins/packages/client-slot-header-action/package.json +31 -0
- package/example-plugins/packages/client-slot-sidebar-tab/client.js +57 -0
- package/example-plugins/packages/client-slot-sidebar-tab/cordis.patch.yml +3 -0
- package/example-plugins/packages/client-slot-sidebar-tab/index.js +9 -0
- package/example-plugins/packages/client-slot-sidebar-tab/package.json +32 -0
- package/example-plugins/packages/client-theme-override/client.js +52 -0
- package/example-plugins/packages/client-theme-override/cordis.patch.yml +3 -0
- package/example-plugins/packages/client-theme-override/index.js +9 -0
- package/example-plugins/packages/client-theme-override/package.json +30 -0
- package/example-plugins/packages/client-tool-view/client.js +47 -0
- package/example-plugins/packages/client-tool-view/cordis.patch.yml +3 -0
- package/example-plugins/packages/client-tool-view/index.js +24 -0
- package/example-plugins/packages/client-tool-view/package.json +34 -0
- package/example-plugins/packages/client-ui-components/client.js +77 -0
- package/example-plugins/packages/client-ui-components/cordis.patch.yml +3 -0
- package/example-plugins/packages/client-ui-components/index.js +9 -0
- package/example-plugins/packages/client-ui-components/package.json +31 -0
- package/example-plugins/packages/client-ui-library/client.js +715 -0
- package/example-plugins/packages/client-ui-library/cordis.patch.yml +3 -0
- package/example-plugins/packages/client-ui-library/index.js +9 -0
- package/example-plugins/packages/client-ui-library/package.json +37 -0
- package/example-plugins/packages/client-workspace-tab/client.js +72 -0
- package/example-plugins/packages/client-workspace-tab/cordis.patch.yml +3 -0
- package/example-plugins/packages/client-workspace-tab/index.js +9 -0
- package/example-plugins/packages/client-workspace-tab/package.json +30 -0
- package/example-plugins/packages/config-schema-basic/cordis.patch.yml +3 -0
- package/example-plugins/packages/config-schema-basic/index.js +19 -0
- package/example-plugins/packages/config-schema-basic/package.json +25 -0
- package/example-plugins/packages/desktop-main-profile-info/cordis.patch.yml +3 -0
- package/example-plugins/packages/desktop-main-profile-info/index.js +22 -0
- package/example-plugins/packages/desktop-main-profile-info/package.json +25 -0
- package/example-plugins/packages/event-hook-basic/cordis.patch.yml +3 -0
- package/example-plugins/packages/event-hook-basic/index.js +17 -0
- package/example-plugins/packages/event-hook-basic/package.json +22 -0
- package/example-plugins/packages/generated-capability-template/cordis.patch.yml +3 -0
- package/example-plugins/packages/generated-capability-template/index.js +16 -0
- package/example-plugins/packages/generated-capability-template/package.json +42 -0
- package/example-plugins/packages/host-route-basic/cordis.patch.yml +3 -0
- package/example-plugins/packages/host-route-basic/index.js +20 -0
- package/example-plugins/packages/host-route-basic/package.json +29 -0
- package/example-plugins/packages/lifecycle-function-basic/cordis.patch.yml +3 -0
- package/example-plugins/packages/lifecycle-function-basic/index.js +12 -0
- package/example-plugins/packages/lifecycle-function-basic/package.json +15 -0
- package/example-plugins/packages/lifecycle-function-hot-shim/cordis.patch.yml +3 -0
- package/example-plugins/packages/lifecycle-function-hot-shim/impl.js +5 -0
- package/example-plugins/packages/lifecycle-function-hot-shim/index.js +18 -0
- package/example-plugins/packages/lifecycle-function-hot-shim/package.json +23 -0
- package/example-plugins/packages/lifecycle-hooks-observer/cordis.patch.yml +3 -0
- package/example-plugins/packages/lifecycle-hooks-observer/index.js +23 -0
- package/example-plugins/packages/lifecycle-hooks-observer/package.json +22 -0
- package/example-plugins/packages/llm-adapter-echo/cordis.patch.yml +3 -0
- package/example-plugins/packages/llm-adapter-echo/index.js +42 -0
- package/example-plugins/packages/llm-adapter-echo/package.json +25 -0
- package/example-plugins/packages/prompt-assemble-hook/cordis.patch.yml +3 -0
- package/example-plugins/packages/prompt-assemble-hook/index.js +22 -0
- package/example-plugins/packages/prompt-assemble-hook/package.json +22 -0
- package/example-plugins/packages/prompt-contribution-basic/cordis.patch.yml +3 -0
- package/example-plugins/packages/prompt-contribution-basic/index.js +18 -0
- package/example-plugins/packages/prompt-contribution-basic/package.json +22 -0
- package/example-plugins/packages/service-consumer-greeter/cordis.patch.yml +3 -0
- package/example-plugins/packages/service-consumer-greeter/index.js +14 -0
- package/example-plugins/packages/service-consumer-greeter/package.json +22 -0
- package/example-plugins/packages/service-provider-greeter/cordis.patch.yml +3 -0
- package/example-plugins/packages/service-provider-greeter/index.js +22 -0
- package/example-plugins/packages/service-provider-greeter/package.json +25 -0
- package/example-plugins/packages/settings-section-basic/cordis.patch.yml +3 -0
- package/example-plugins/packages/settings-section-basic/index.js +34 -0
- package/example-plugins/packages/settings-section-basic/package.json +26 -0
- package/example-plugins/packages/skill-provider-basic/cordis.patch.yml +3 -0
- package/example-plugins/packages/skill-provider-basic/index.js +31 -0
- package/example-plugins/packages/skill-provider-basic/package.json +22 -0
- package/example-plugins/packages/state-host-store/client.js +57 -0
- package/example-plugins/packages/state-host-store/cordis.patch.yml +3 -0
- package/example-plugins/packages/state-host-store/index.js +90 -0
- package/example-plugins/packages/state-host-store/package.json +31 -0
- package/example-plugins/packages/state-workspace-file/cordis.patch.yml +3 -0
- package/example-plugins/packages/state-workspace-file/index.js +52 -0
- package/example-plugins/packages/state-workspace-file/package.json +25 -0
- package/example-plugins/packages/tool-basic/cordis.patch.yml +3 -0
- package/example-plugins/packages/tool-basic/index.js +25 -0
- package/example-plugins/packages/tool-basic/package.json +25 -0
- package/example-plugins/packages/tool-policy-hook/cordis.patch.yml +3 -0
- package/example-plugins/packages/tool-policy-hook/index.js +28 -0
- package/example-plugins/packages/tool-policy-hook/package.json +25 -0
- package/example-plugins/packages/tui-command-basic/cordis.patch.yml +3 -0
- package/example-plugins/packages/tui-command-basic/index.js +24 -0
- package/example-plugins/packages/tui-command-basic/package.json +22 -0
- package/example-plugins/packages/tui-overlay-themed/cordis.patch.yml +3 -0
- package/example-plugins/packages/tui-overlay-themed/index.js +65 -0
- package/example-plugins/packages/tui-overlay-themed/package.json +25 -0
- package/example-plugins/packages/tui-ui-library/cordis.patch.yml +3 -0
- package/example-plugins/packages/tui-ui-library/index.js +73 -0
- package/example-plugins/packages/tui-ui-library/package.json +29 -0
- package/example-plugins/packages/web-page-branding/cordis.patch.yml +3 -0
- package/example-plugins/packages/web-page-branding/index.js +31 -0
- package/example-plugins/packages/web-page-branding/package.json +22 -0
- package/example-plugins/scenarios.json +11 -0
- package/index.js +384 -0
- package/lib/blend-apply.js +107 -0
- package/lib/blend-capture.js +249 -0
- package/lib/blend-ledger.js +76 -0
- package/lib/contract.js +5 -0
- package/lib/install.js +516 -0
- package/lib/lookup.js +67 -0
- package/lib/manifest.js +56 -0
- package/lib/pack-root.js +22 -0
- package/lib/provenance.js +110 -0
- package/lib/publish.js +64 -0
- package/lib/reconcile.js +83 -0
- package/lib/router.js +55 -0
- package/lib/scopes.js +82 -0
- package/lib/skills.js +54 -0
- package/lib/stage.js +157 -0
- package/lib/verify.js +77 -0
- package/package.json +58 -3
- package/scripts/lib/indexes.mjs +45 -0
- package/scripts/lib/manifest.mjs +127 -0
- package/scripts/lib/maps.mjs +482 -0
- package/skills/acryl-add-provider-or-capability/SKILL.md +12 -0
- package/skills/acryl-add-ui/SKILL.md +18 -0
- package/skills/acryl-build-plugin/SKILL.md +21 -0
- package/skills/acryl-change-plugin/SKILL.md +16 -0
- package/skills/acryl-desktop-shell/SKILL.md +12 -0
- package/skills/acryl-diagnose-plugin/SKILL.md +10 -0
- package/skills/acryl-fix-plugin/SKILL.md +12 -0
- package/skills/acryl-improve-ui/SKILL.md +10 -0
- package/skills/acryl-remove-plugin/SKILL.md +11 -0
- package/skills/acryl-restyle-ui/SKILL.md +14 -0
- package/skills/acryl-share-plugin/SKILL.md +23 -0
- package/skills/acryl-tui-ui/SKILL.md +12 -0
|
@@ -0,0 +1,309 @@
|
|
|
1
|
+
<!-- Synced from deepseek-harness:docs/subsystems/user-questions.md @ 639ed01539. Do not edit; run scripts/sync-corpus.mjs. Relative links inside may not resolve here. -->
|
|
2
|
+
|
|
3
|
+
# User Interaction
|
|
4
|
+
|
|
5
|
+
English | [中文](user-questions.zh.md)
|
|
6
|
+
|
|
7
|
+
The user-questions seam of [dsh-user-questions](../../packages/interaction/user-questions). It is the provider-neutral vocabulary a tool or permission plugin uses when it needs the human to answer before the agent can continue. Agent-scoped waterfall listeners compose the available UI surfaces, including listeners relayed to a connected client.
|
|
8
|
+
|
|
9
|
+
Source: [`packages/interaction/user-questions/src/index.ts`](../../packages/interaction/user-questions/src/index.ts)
|
|
10
|
+
|
|
11
|
+
## Question options
|
|
12
|
+
|
|
13
|
+
`AskUserQuestionOption` contains one selectable choice. `label` is the user-facing option text and also the model-facing selected value; `description` is optional UI help text.
|
|
14
|
+
|
|
15
|
+
```ts type-equiv
|
|
16
|
+
/** One selectable answer offered to the user. */
|
|
17
|
+
interface AskUserQuestionOption {
|
|
18
|
+
/** User-facing label. */
|
|
19
|
+
label: string
|
|
20
|
+
/** Optional extra context rendered by capable UIs. */
|
|
21
|
+
description?: string
|
|
22
|
+
}
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
## Presentation intent
|
|
26
|
+
|
|
27
|
+
`AskUserQuestionIntent` optionally declares a known decision kind. It is tagged on `kind` so intents can be added; a UI that does not recognise a tag renders the generic option list. An intent changes presentation only - a UI honouring it answers with the same option labels a generic UI would send, so the caller reads the same answer fields either way. `approve` names the affirmative option instead of relying on option order. `ask()` rejects the two assertions no type can carry: an `approve` naming none of its own question's options, and an intent on a question with no `detail`.
|
|
28
|
+
|
|
29
|
+
```ts type-equiv
|
|
30
|
+
/**
|
|
31
|
+
* A caller-declared presentation intent: the question IS this kind of
|
|
32
|
+
* decision, so a UI that recognises the tag may present it as such instead of as a
|
|
33
|
+
* generic option list. Tagged so further intents can be added; a UI that does
|
|
34
|
+
* not know a tag renders the generic flow, and the answer encoding is identical
|
|
35
|
+
* either way - an intent changes presentation only, never the protocol.
|
|
36
|
+
*/
|
|
37
|
+
type AskUserQuestionIntent = {
|
|
38
|
+
/** A plan submitted for review: `detail` is the plan markdown `ask()` requires, and the decision approves or declines it. */
|
|
39
|
+
kind: 'plan-review'
|
|
40
|
+
/**
|
|
41
|
+
* The option label that approves the plan; every other option declines it.
|
|
42
|
+
* Named rather than positional so no UI infers the verdict from option order.
|
|
43
|
+
* An `approve` naming no option of its own question is rejected at `ask()`.
|
|
44
|
+
*/
|
|
45
|
+
approve: string
|
|
46
|
+
/** Logged tool invocation whose arguments contain the reviewed plan. */
|
|
47
|
+
callId?: ToolCallId
|
|
48
|
+
}
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
## Question item
|
|
52
|
+
|
|
53
|
+
`AskUserQuestionItem` is one question in a request. The caller supplies a stable `id`, which is echoed back with the answer so batched questions remain routable. Optional `detail` carries supporting text that providers render with the question but keep out of selectable option labels.
|
|
54
|
+
|
|
55
|
+
```ts type-equiv
|
|
56
|
+
/** One question in a user-questions request. */
|
|
57
|
+
interface AskUserQuestionItem {
|
|
58
|
+
/** Stable caller-provided question id, echoed in the answer. */
|
|
59
|
+
id: string
|
|
60
|
+
/** The question to display. */
|
|
61
|
+
question: string
|
|
62
|
+
/** Optional supporting detail rendered with the question but kept out of option labels. */
|
|
63
|
+
detail?: string
|
|
64
|
+
/** Optional short heading/group label. */
|
|
65
|
+
header?: string
|
|
66
|
+
/** Optional choices the UI can render as a menu. */
|
|
67
|
+
options?: AskUserQuestionOption[]
|
|
68
|
+
/** Whether more than one option may be selected. Defaults to single-select. */
|
|
69
|
+
multiSelect?: boolean
|
|
70
|
+
/** Optional presentation intent for capable UIs; absent asks for the generic option list. */
|
|
71
|
+
intent?: AskUserQuestionIntent
|
|
72
|
+
}
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
## Ask request
|
|
76
|
+
|
|
77
|
+
`AskUserQuestionRequest` is the cross-package request. `questions` is an array so a UI can present related prompts in one flow while preserving a stable id per answer. When present, `agent` is the exact live caller; the interaction seam admits it only while the live registry identifies that instance as a runtime root.
|
|
78
|
+
|
|
79
|
+
```ts type-equiv
|
|
80
|
+
/** Request for a human answer. */
|
|
81
|
+
interface AskUserQuestionRequest extends AskUserQuestionRequestEvent {}
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
## Answer
|
|
85
|
+
|
|
86
|
+
Providers return one answer item per question id. `selected` contains selected option labels, and `custom` carries a free-form "Other" answer when the user typed one. For a single-select question, `custom` overrides the selected choice and `selected` is empty. For a multi-select question, `custom` may supplement the labels in `selected`. A UI may also use an item with empty `selected` and no `custom` to preserve a skipped question in an otherwise completed batch.
|
|
87
|
+
|
|
88
|
+
```ts type-equiv
|
|
89
|
+
/** Answer to one question. */
|
|
90
|
+
interface AskUserQuestionAnswerItem {
|
|
91
|
+
/** The answered question id. */
|
|
92
|
+
id: string
|
|
93
|
+
/** Selected option labels. May accompany custom text for a multi-select question. */
|
|
94
|
+
selected: string[]
|
|
95
|
+
/** Optional free-text "Other" answer. */
|
|
96
|
+
custom?: string
|
|
97
|
+
}
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
```ts type-equiv
|
|
101
|
+
/** The human's answer. */
|
|
102
|
+
interface AskUserQuestionAnswer {
|
|
103
|
+
/** Structured answers keyed by question id. */
|
|
104
|
+
answers: AskUserQuestionAnswerItem[]
|
|
105
|
+
}
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Timed answerers receive `wait: { callId, timed: true }`. The call id identifies the card and the foreground wait. The Client calls `attachWait` to claim the wait and obtain its remaining duration; transport forwards the original event without inspecting or rewriting these fields. A `wait` without `timed` identifies an indefinite card-keyed request. An absent `wait` identifies a blocking request without a card key.
|
|
109
|
+
|
|
110
|
+
```ts type-equiv
|
|
111
|
+
/** Client-safe payload declared for the user-question answerer waterfall. */
|
|
112
|
+
interface AskUserQuestionRequestEvent {
|
|
113
|
+
/** Questions to display. */
|
|
114
|
+
questions: AskUserQuestionItem[]
|
|
115
|
+
/** Agent identity projected to the corresponding Client Context in transit. */
|
|
116
|
+
agent?: Agent
|
|
117
|
+
/** Cancellation lifetime of the pending request. */
|
|
118
|
+
signal?: AbortSignal
|
|
119
|
+
/**
|
|
120
|
+
* Tool call the Client card is keyed by. Timed answerers attach to the
|
|
121
|
+
* business wait stream before starting their local countdown.
|
|
122
|
+
*/
|
|
123
|
+
wait?: {
|
|
124
|
+
/** Tool call the Client card is keyed by. */
|
|
125
|
+
callId: ToolCallId
|
|
126
|
+
/** True for a foreground wait that requires a Client claim; absent for indefinite waits. */
|
|
127
|
+
timed?: boolean
|
|
128
|
+
}
|
|
129
|
+
}
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
## Timed questions and late replies
|
|
133
|
+
|
|
134
|
+
A timed question waits for a bounded foreground window and then lets the agent continue independent work. If the user answers inside the window, the tool returns the answer batch; otherwise `TimedUserQuestionResult` is `{ pending: true, callId }`. Pending means the question is still answerable; it is not an empty answer, a refusal, a confirmation, or permission.
|
|
135
|
+
|
|
136
|
+
```ts type-equiv
|
|
137
|
+
/** Timed ask result returned when the foreground answer window closes. */
|
|
138
|
+
type TimedUserQuestionResult = AskUserQuestionAnswer | { pending: true; callId: ToolCallId }
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
A Client claim is a business Remote stream, not a Gateway delivery. `TimedQuestionWait` holds the original Host deadline and runs its timer only while no claim exists. Its `attachWait` stream yields the current remaining duration, and the Client counts down with its own clock. Focus, editing, and Take time stay local. Stream cancellation releases that claim; after the last claim leaves, the Host resumes the original deadline. An unattended timeout aborts the foreground waterfall's private signal and returns pending without cancelling the Turn. Gateway and `api-remotes` contain no question-specific timing policy.
|
|
142
|
+
|
|
143
|
+
The `userQuestions` Session projection folds existing events into the answerable question set and the settled one; no wait state, focus state, or deadline is logged. Each `request/header` records the exact tool schemas the model saw. Native calls are tracked only when that header declares the timed `ask_user_question` schema with a `timeout` parameter; the blocking legacy tool never declares it. A native `tool/call` opens a question. A `tool/result` marks it `continued` only when it carries the pending payload or is the synthetic `TOOL_OUTCOME_UNKNOWN` result Session resume repair appends; an answer batch moves it into `settled` with that batch, and any failure drops it. A `run_code` PTC sub-call enters the projection when its `tool/ptc-dispatch` result is pending, even though the request header exposes only `run_code`. A late reply moves a continued question into `settled` when its message is admitted as `user/message`, together with the answer batch read out of that message: the timed call's own result recorded the timeout, so a transcript row has nowhere else to read what the user finally chose. A queued reply can be canceled before admission and leaves the question answerable. A reply whose text carries no readable batch settles the call with no answers; one naming a call that is not active changes nothing.
|
|
144
|
+
|
|
145
|
+
```ts type-equiv
|
|
146
|
+
/** `open` while the tool call may still return the answer; `continued` once the answer can only arrive as a new turn. */
|
|
147
|
+
type UserQuestionState = 'open' | 'continued'
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
```ts type-equiv
|
|
151
|
+
/** One unanswered timed `ask_user_question` call reconstructed from the Session log. */
|
|
152
|
+
interface PendingUserQuestion {
|
|
153
|
+
readonly callId: ToolCallId
|
|
154
|
+
readonly questions: readonly AskUserQuestionItem[]
|
|
155
|
+
readonly state: UserQuestionState
|
|
156
|
+
}
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
```ts type-equiv
|
|
160
|
+
/**
|
|
161
|
+
* One timed `ask_user_question` call and the answers it settled with: the
|
|
162
|
+
* batch its own result carried when the user answered inside the window,
|
|
163
|
+
* otherwise the batch its late reply carried, because that call's own result
|
|
164
|
+
* recorded the timeout. A transcript row reads what the user finally answered
|
|
165
|
+
* from here, and only a call listed here was a timed one.
|
|
166
|
+
*/
|
|
167
|
+
interface SettledUserQuestion {
|
|
168
|
+
/** The settled call. */
|
|
169
|
+
readonly callId: ToolCallId
|
|
170
|
+
/** The recorded batch, one entry per question; empty when the late reply carried none. */
|
|
171
|
+
readonly answers: readonly AskUserQuestionAnswerItem[]
|
|
172
|
+
}
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
```ts type-equiv
|
|
176
|
+
/**
|
|
177
|
+
* Both halves of one Session's timed `ask_user_question` state, as every
|
|
178
|
+
* Client reads them. A call made while the blocking legacy tool was in
|
|
179
|
+
* effect appears in neither half.
|
|
180
|
+
*/
|
|
181
|
+
interface UserQuestionProjectionView {
|
|
182
|
+
/** Calls that can still take an answer, in ask order. */
|
|
183
|
+
readonly active: readonly PendingUserQuestion[]
|
|
184
|
+
/** Calls an answer settled, in settlement order. */
|
|
185
|
+
readonly settled: readonly SettledUserQuestion[]
|
|
186
|
+
}
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
An open question accepts answers only through the waterfall. Once it is `continued`, the `answer` Remote method queues a user message for the owning agent and wakes it if idle. The message has source `user-question-reply`, outcome `answered`, and an `answer_to_pending_question` payload. The method returns `false` for an open or unknown call, rejects a second queued reply with `REPLY_QUEUED` even after a Host restart, and rejects incomplete or duplicate answer batches with `BAD_ANSWER`.
|
|
190
|
+
|
|
191
|
+
Closing the Client panel sends nothing. A foreground timeout ends only the tool wait. Continued questions have no deadline; they stay answerable until a reply is admitted.
|
|
192
|
+
|
|
193
|
+
After a Host restart, the Remote layer resumes the root agent when the Session reopens. A late reply then enters a new turn.
|
|
194
|
+
|
|
195
|
+
## Errors
|
|
196
|
+
|
|
197
|
+
`UserQuestionError` extends `HarnessError`, so `ctx.tools.execute()` preserves `{ name, code }` for model-facing tool failures such as `EMPTY_QUESTIONS`, `BAD_INTENT`, `NO_PROVIDER`, `ASK_ABORTED`, or UI-side cancellation.
|
|
198
|
+
|
|
199
|
+
```ts type-equiv
|
|
200
|
+
/** Stable error taxonomy for user-questions failures. */
|
|
201
|
+
class UserQuestionError extends HarnessError {
|
|
202
|
+
constructor(message: string, code: string, options?: ErrorOptions) {
|
|
203
|
+
super(message, code, options)
|
|
204
|
+
this.name = 'UserQuestionError'
|
|
205
|
+
}
|
|
206
|
+
}
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) - do not edit between markers -->
|
|
210
|
+
|
|
211
|
+
<a id="cordis-surface"></a>
|
|
212
|
+
|
|
213
|
+
## Cordis API
|
|
214
|
+
|
|
215
|
+
Generated from source by `scripts/gen-cordis-catalog.ts` (verified fresh by `pnpm run verify-cordis-catalog` in doc-sync; regenerate with `pnpm run gen-cordis-catalog`) - the language sides differ only in locale-specific paired document paths. Signature blocks use a `ts cordis-catalog` fence and keep the original source JSDoc; dispatch modes are defined in the [primer](../cordis-primer.md#dispatch-modes), and the framework-inherited `ctx` API lives in [cordis-api/inherited.md](../cordis-api/inherited.md).
|
|
216
|
+
|
|
217
|
+
<a id="ctxuserquestions--userquestionservice"></a>
|
|
218
|
+
|
|
219
|
+
### `ctx.userQuestions` - `UserQuestionService`
|
|
220
|
+
|
|
221
|
+
`ctx.userQuestions`: validation plus the scoped answerer waterfall.
|
|
222
|
+
|
|
223
|
+
```ts cordis-catalog
|
|
224
|
+
/**
|
|
225
|
+
* Answer a continued question. The reply is steered into the agent as a
|
|
226
|
+
* user message whose source names the call; that message is also the
|
|
227
|
+
* record that closes the question in the projection.
|
|
228
|
+
* @param agent - Live root agent for the owning Session.
|
|
229
|
+
* @param callId - Continued question identity.
|
|
230
|
+
* @param answer - Complete structured answer batch, one item per question of the call.
|
|
231
|
+
* @returns Whether the question is still continued; an accepted reply stays
|
|
232
|
+
* queued until the agent admits its user message.
|
|
233
|
+
* @throws {UserQuestionError} `BAD_ANSWER` when the batch does not name each
|
|
234
|
+
* question of the call exactly once, or `REPLY_QUEUED` when a reply is
|
|
235
|
+
* already waiting for admission.
|
|
236
|
+
*/
|
|
237
|
+
@Remote answer(agent: Agent, callId: ToolCallId, answer: AskUserQuestionAnswer): boolean
|
|
238
|
+
|
|
239
|
+
/**
|
|
240
|
+
* Let one answer UI hold a live timed wait. Closing the stream releases its claim.
|
|
241
|
+
* @param agent - Live root agent owning the question.
|
|
242
|
+
* @param callId - Foreground tool call to attach to.
|
|
243
|
+
* @param signal - Remote stream cancellation, including Client disconnect.
|
|
244
|
+
* @returns One Host-computed remaining duration, or no frames once the wait ended.
|
|
245
|
+
*/
|
|
246
|
+
@Remote({ mode: 'stream' }) async *attachWait(agent: Agent, callId: ToolCallId, signal: AbortSignal): AsyncIterable<{ remainingMs: number }>
|
|
247
|
+
|
|
248
|
+
/**
|
|
249
|
+
* Foreground wait whose first settlement the Client decides: the Client
|
|
250
|
+
* rejects with `ASK_TIMED_OUT` when its countdown ends, and this method maps
|
|
251
|
+
* that code to the pending result.
|
|
252
|
+
* @param request - Questions, live owner agent, and abort signal.
|
|
253
|
+
* @param callId - Tool call identity the Client card is keyed by.
|
|
254
|
+
* @param timeoutMs - Positive foreground wait in milliseconds.
|
|
255
|
+
* @returns The answer when it arrives inside the window, otherwise a pending
|
|
256
|
+
* result, also when no connected Client claimed the request by the deadline.
|
|
257
|
+
* @throws {UserQuestionError} `BAD_TIMEOUT` for a non-integer, non-positive,
|
|
258
|
+
* or oversized wait.
|
|
259
|
+
*/
|
|
260
|
+
async askTimed( request: AskUserQuestionRequest & { agent: Agent }, callId: ToolCallId, timeoutMs: number, ): Promise<TimedUserQuestionResult>
|
|
261
|
+
|
|
262
|
+
/**
|
|
263
|
+
* Ask the scoped answerer waterfall and wait for the user's answer.
|
|
264
|
+
*
|
|
265
|
+
* When a caller supplies an agent, human interaction is valid only for the
|
|
266
|
+
* exact live runtime root. Runtime ownership, not durable session lineage,
|
|
267
|
+
* decides this boundary: an owned child has no human answerer and would
|
|
268
|
+
* block forever, while a lineage-bearing session resumed as a new runtime
|
|
269
|
+
* root may ask normally.
|
|
270
|
+
*
|
|
271
|
+
* @param request Questions, owner agent, and abort signal.
|
|
272
|
+
* @returns The answer chosen or typed by the human.
|
|
273
|
+
* @throws {UserQuestionError} code `ASK_ABORTED` when the supplied signal
|
|
274
|
+
* is already or becomes aborted, `CALLER_NOT_LIVE` when a supplied agent
|
|
275
|
+
* is not the registry's exact live instance, or `DELEGATED_CALLER` when
|
|
276
|
+
* that live agent is owned by another agent.
|
|
277
|
+
*/
|
|
278
|
+
async ask(request: AskUserQuestionRequest): Promise<AskUserQuestionAnswer>
|
|
279
|
+
```
|
|
280
|
+
|
|
281
|
+
Types: [Agent](core.md) · [ToolCallId](core.md)
|
|
282
|
+
|
|
283
|
+
Source: [`packages/interaction/user-questions/src/index.ts`](../../packages/interaction/user-questions/src/index.ts)
|
|
284
|
+
|
|
285
|
+
<a id="user-questions-events"></a>
|
|
286
|
+
|
|
287
|
+
### `user-questions/*` events
|
|
288
|
+
|
|
289
|
+
<a id="user-questionsrequest--waterfall"></a>
|
|
290
|
+
|
|
291
|
+
#### `user-questions/request` - waterfall
|
|
292
|
+
|
|
293
|
+
Ask composed answerers for structured user input. Return an answer to claim the request or call `next()` to delegate. Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.
|
|
294
|
+
|
|
295
|
+
```ts cordis-catalog
|
|
296
|
+
/**
|
|
297
|
+
* Ask composed answerers for structured user input. Return an answer to
|
|
298
|
+
* claim the request or call `next()` to delegate. Scope-filtered dispatch
|
|
299
|
+
* (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.
|
|
300
|
+
* @param request - pending user-question request.
|
|
301
|
+
* @mode waterfall
|
|
302
|
+
*/
|
|
303
|
+
'user-questions/request'( this: Scoped<Agent>, request: AskUserQuestionRequestEvent, next: () => Promise<AskUserQuestionAnswer>, ): Promise<AskUserQuestionAnswer>
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
Types: [Agent](core.md) · [Scoped](scope.md)
|
|
307
|
+
|
|
308
|
+
Source: [`packages/interaction/user-questions/src/types.ts`](../../packages/interaction/user-questions/src/types.ts)
|
|
309
|
+
<!-- END GENERATED cordis-surface -->
|
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
<!-- Synced from deepseek-harness:docs/subsystems/voice-input.md @ 639ed01539. Do not edit; run scripts/sync-corpus.mjs. Relative links inside may not resolve here. -->
|
|
2
|
+
|
|
3
|
+
# Voice input
|
|
4
|
+
|
|
5
|
+
English | [中文](voice-input.zh.md)
|
|
6
|
+
|
|
7
|
+
Experimental speech recognition has three roles: the [Service Definition](../../packages/experimental/speech-to-text/README.md) routes named providers, the [SenseVoice provider](../../packages/experimental/speech-to-text-sensevoice/README.md) owns local inference, and the [Remote consumer](../../packages/experimental/api-speech-to-text/README.md) serves the browser. The [optional bundle](../../packages/experimental/voice-input-bundle/README.md) composes them with the microphone UI.
|
|
8
|
+
|
|
9
|
+
## Provider selection
|
|
10
|
+
|
|
11
|
+
`SpeechProviderId` brands the registration identity. `SpeechProviderInfo` carries a display name, accepted `languages` hints and `host-local` or `cloud` processing location. `SpeechRequest` contains WAV bytes and optional provider/language selection; `resolve()` produces `SpeechSpec` with one captured provider instance. A missing provider or unsupported language fails; replacement or withdrawal invalidates resolved work. Audio never falls back to a different provider.
|
|
12
|
+
|
|
13
|
+
`SpeechProvider.transcribe()` accepts one complete `SpeechInput` and an `AbortSignal`. `Transcript` returns plain `text`, `audioSeconds` and `inferenceSeconds`. Providers honor cancellation and settle their work before deregistration completes. The local provider serializes calls, bounds its queue and manages its worker lifetime independently of Sessions.
|
|
14
|
+
|
|
15
|
+
## Browser and Host ownership
|
|
16
|
+
|
|
17
|
+
The browser owns microphone tracks and the unsent draft. `TranscriptionRequest` sends canonical base64 PCM16 WAV through the authenticated Remote. `SpeechCatalog` advertises available providers, the default selection and recording byte/duration limits. The API validates this process input before recognition.
|
|
18
|
+
|
|
19
|
+
The input facade captures a revision-bearing selection before recording. `InputActions.insertText()` inserts one undoable plain-text edit only while the selection revision is current and submission permits editing. A rejected insertion leaves the transcript available for explicit insertion. Switching Sessions or disposing the plugin invalidates late results. Recognition itself writes no Session event; ordinary user submission owns the final model-visible text.
|
|
20
|
+
|
|
21
|
+
## Preparation and settings
|
|
22
|
+
|
|
23
|
+
`SpeechProviderInfo.downloadSources` advertises preparation origins. `SpeechPreparationOptions.downloadSource` selects one for a task; omission preserves provider policy. SenseVoice validates the advertised choice, pins a manual source without fallback, and rejects source changes during active preparation. The UI keeps a choice for retries in the current card; it is not a persisted recognition preference.
|
|
24
|
+
|
|
25
|
+
The Host provider owns one preparation task across page and Session changes. The Client shares one `follow()` subscription across the composer, setup prompt and bundle details. Optional `SpeechSetupEstimate` metadata supplies provider-specific planning hints, separate from measured progress. `SpeechPreparationStepKind` identifies ordered resource operations; `SpeechPreparationStep` records each status and start time. The complete `SpeechPreparationState` retains these steps after cancellation or failure. The collapsed UI shows the current operation; expanding lists all steps, with byte progress or elapsed time only for the running step. Closing an observer never cancels preparation. Verified files survive retries and idle worker reclamation.
|
|
26
|
+
|
|
27
|
+
Failed preparation can include `SpeechDownloadFailure` with the asset, source origin, classified reason and optional diagnostic code or HTTP status. The Client localizes recovery advice; raw download causes remain on the Host.
|
|
28
|
+
|
|
29
|
+
The microphone occupies `conversation.input.activity`, between the model selector and Send. Clicking starts capture and expands the toolbar; Stop transcribes and inserts into the draft. The activity preserves the editor and submit action, owns local feedback, and releases expansion on unmount. Cancel, Escape or hiding the page discards capture. Waveform history displays measured microphone amplitude. Bundle details contain recognition preferences, preparation state and progress. Explicit enablement uses `plugins.bundle.activation` to guide users with missing models to setup; the list shows only the bundle description and switch.
|
|
30
|
+
|
|
31
|
+
## Design rationale
|
|
32
|
+
|
|
33
|
+
The [voice input decision](../../.agents/notes/implemented/architecture/2026-09-16-experimental-voice-input.md) explains transient audio, explicit provider selection and lazy local preparation.
|
|
34
|
+
|
|
35
|
+
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) - do not edit between markers -->
|
|
36
|
+
|
|
37
|
+
<a id="cordis-surface"></a>
|
|
38
|
+
|
|
39
|
+
## Cordis API
|
|
40
|
+
|
|
41
|
+
Generated from source by `scripts/gen-cordis-catalog.ts` (verified fresh by `pnpm run verify-cordis-catalog` in doc-sync; regenerate with `pnpm run gen-cordis-catalog`) - the language sides differ only in locale-specific paired document paths. Signature blocks use a `ts cordis-catalog` fence and keep the original source JSDoc; dispatch modes are defined in the [primer](../cordis-primer.md#dispatch-modes), and the framework-inherited `ctx` API lives in [cordis-api/inherited.md](../cordis-api/inherited.md).
|
|
42
|
+
|
|
43
|
+
<a id="ctxspeechcontroller--speechcontroller"></a>
|
|
44
|
+
|
|
45
|
+
### `ctx.speechController` - `SpeechController`
|
|
46
|
+
|
|
47
|
+
Speech calls never activate or submit to an Agent.
|
|
48
|
+
|
|
49
|
+
```ts cordis-catalog
|
|
50
|
+
/**
|
|
51
|
+
* Read provider choices without preparing a recognizer.
|
|
52
|
+
* @returns available providers, resolved default, and recording limits.
|
|
53
|
+
*/
|
|
54
|
+
@Remote catalog(): SpeechCatalog
|
|
55
|
+
|
|
56
|
+
/**
|
|
57
|
+
* Follow provider readiness independently of Session and preparation lifetimes.
|
|
58
|
+
* @param signal - Client observation lifetime.
|
|
59
|
+
* @returns initial and subsequent complete readiness snapshots.
|
|
60
|
+
*/
|
|
61
|
+
@Remote({ mode: 'stream' }) async *follow(signal: AbortSignal): AsyncIterable<SpeechCatalog>
|
|
62
|
+
|
|
63
|
+
/**
|
|
64
|
+
* Persist the user's recognition preferences.
|
|
65
|
+
* @param patch - changed preference fields.
|
|
66
|
+
* @returns after preferences are saved.
|
|
67
|
+
*/
|
|
68
|
+
@Remote configure(patch: SpeechSelectionPatch): Promise<void>
|
|
69
|
+
|
|
70
|
+
/**
|
|
71
|
+
* Start or join one Host-owned preparation task.
|
|
72
|
+
* @param providerId - selected recognizer.
|
|
73
|
+
* @param options - task-local source selection validated by the provider.
|
|
74
|
+
*/
|
|
75
|
+
@Remote prepare(providerId: SpeechProviderId, options?: SpeechPreparationOptions): void
|
|
76
|
+
|
|
77
|
+
/**
|
|
78
|
+
* Explicitly cancel resource preparation.
|
|
79
|
+
* @param providerId - selected recognizer.
|
|
80
|
+
* @returns after the preparation task settles.
|
|
81
|
+
*/
|
|
82
|
+
@Remote cancelPreparation(providerId: SpeechProviderId): Promise<void>
|
|
83
|
+
|
|
84
|
+
/**
|
|
85
|
+
* Validate and transcribe one recording through the explicit provider selection.
|
|
86
|
+
* @param request - canonical WAV encoded as base64, provider id and language hint.
|
|
87
|
+
* @param signal - Client cancellation or Remote contribution disposal.
|
|
88
|
+
* @returns final transcript without adding a Session event.
|
|
89
|
+
*/
|
|
90
|
+
@Remote async transcribe(request: TranscriptionRequest, signal: AbortSignal): Promise<Transcript>
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
Source: [`packages/experimental/api-speech-to-text/src/index.ts`](../../packages/experimental/api-speech-to-text/src/index.ts)
|
|
94
|
+
|
|
95
|
+
<a id="ctxspeechtotext--speechtotext"></a>
|
|
96
|
+
|
|
97
|
+
### `ctx.speechToText` - `SpeechToText`
|
|
98
|
+
|
|
99
|
+
Registry shared by all transcription consumers in one Host composition.
|
|
100
|
+
|
|
101
|
+
```ts cordis-catalog
|
|
102
|
+
/**
|
|
103
|
+
* Register one recognizer; duplicate ids fail without replacing the original.
|
|
104
|
+
* @param provider - recognizer owned by the contributing fiber.
|
|
105
|
+
* @returns idempotent disposer which rejects admission, cancels, and joins accepted work.
|
|
106
|
+
*/
|
|
107
|
+
register(provider: SpeechProvider): () => Promise<void>
|
|
108
|
+
|
|
109
|
+
/**
|
|
110
|
+
* Read the current recognizer roster.
|
|
111
|
+
* @returns available provider facts in registration order.
|
|
112
|
+
*/
|
|
113
|
+
listProviders(): readonly SpeechProviderInfo[]
|
|
114
|
+
|
|
115
|
+
/**
|
|
116
|
+
* Observe complete readiness snapshots; a slow reader coalesces intermediate progress.
|
|
117
|
+
* @param caller - observer lifetime, independent of any preparation task.
|
|
118
|
+
* @returns an initial snapshot followed by the latest provider states.
|
|
119
|
+
*/
|
|
120
|
+
async *follow(caller: AbortSignal): AsyncIterable<SpeechSnapshot>
|
|
121
|
+
|
|
122
|
+
/**
|
|
123
|
+
* Read provider readiness and current user preferences together.
|
|
124
|
+
* @returns one detached complete observation.
|
|
125
|
+
*/
|
|
126
|
+
snapshot(): SpeechSnapshot
|
|
127
|
+
|
|
128
|
+
/**
|
|
129
|
+
* Persist changed selection fields into this plugin's profile entry; the resulting language must be accepted by the selected provider.
|
|
130
|
+
* @param patch - explicit provider or language changes.
|
|
131
|
+
* @returns after the profile write and the live update it applies.
|
|
132
|
+
*/
|
|
133
|
+
async configure(patch: SpeechSelectionPatch): Promise<void>
|
|
134
|
+
|
|
135
|
+
/**
|
|
136
|
+
* Start or join provider-owned preparation.
|
|
137
|
+
* @param id - exact registered provider identity.
|
|
138
|
+
* @param options - task-local source selection validated by the provider.
|
|
139
|
+
*/
|
|
140
|
+
prepare(id: SpeechProviderId, options?: SpeechPreparationOptions): void
|
|
141
|
+
|
|
142
|
+
/**
|
|
143
|
+
* Explicitly cancel provider preparation without tying it to a browser connection.
|
|
144
|
+
* @param id - exact registered provider identity.
|
|
145
|
+
* @returns after the preparation task settles.
|
|
146
|
+
*/
|
|
147
|
+
async cancelPreparation(id: SpeechProviderId): Promise<void>
|
|
148
|
+
|
|
149
|
+
/**
|
|
150
|
+
* Apply composition defaults and capture the selected provider. Missing providers and unsupported languages fail explicitly.
|
|
151
|
+
* @param request - complete recording and optional selection.
|
|
152
|
+
* @returns provider-pinned input for transcribe().
|
|
153
|
+
*/
|
|
154
|
+
resolve(request: SpeechRequest): SpeechSpec
|
|
155
|
+
|
|
156
|
+
/**
|
|
157
|
+
* Execute exactly the resolved provider; no fallback sends audio elsewhere.
|
|
158
|
+
* @param spec - resolved input; a withdrawn or replaced registration is rejected.
|
|
159
|
+
* @param signal - caller cancellation.
|
|
160
|
+
* @returns final transcript after provider settlement.
|
|
161
|
+
*/
|
|
162
|
+
async transcribe(spec: SpeechSpec, signal: AbortSignal): Promise<Transcript>
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Source: [`packages/experimental/speech-to-text/src/index.ts`](../../packages/experimental/speech-to-text/src/index.ts)
|
|
166
|
+
<!-- END GENERATED cordis-surface -->
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
<!-- Synced from deepseek-harness:docs/subsystems/web-client.md @ 639ed01539. Do not edit; run scripts/sync-corpus.mjs. Relative links inside may not resolve here. -->
|
|
2
|
+
|
|
3
|
+
# Web Client architecture
|
|
4
|
+
|
|
5
|
+
English | [中文](web-client.zh.md)
|
|
6
|
+
|
|
7
|
+
The Web Client is a browser-side Cordis application assembled from independently loaded plugins. Its architecture has four reusable foundations: [Client Modules](client-modules.md) loads the plugin graph, the [API Gateway](../api-gateway.md) provides typed Host communication, [Slots](slots.md) composes React UI, and [Conversation](conversation.md) turns a Session history window into target-owned views. This page connects those systems and defines where Client models and feature packages belong.
|
|
8
|
+
|
|
9
|
+
[Keyboard shortcuts](../../packages/client/shortcuts/README.md) owns window-local command registration and physical-key dispatch; [the shortcut reference](../../packages/client/ui-shortcuts/README.md) presents available commands and local input actions. Command owners declare each runtime/platform default and share their existing actions with mouse controls. The shared modal primitive arbitrates top-layer Escape and restores focus.
|
|
10
|
+
|
|
11
|
+
## Layers and ownership
|
|
12
|
+
|
|
13
|
+
| Layer | Main owners | Responsibility |
|
|
14
|
+
|---|---|---|
|
|
15
|
+
| Host application | business services and `packages/api/*-controller` Host entries | Own authoritative state, persistence, mutation ordering, access policy, and stream production. |
|
|
16
|
+
| Transport and API assembly | `client/connection`, `api/gateway`, `api/remotes` | Establish a Client generation, expose generated `ctx.remote` methods and streams, forward selected Cordis events, and carry cancellation and results. |
|
|
17
|
+
| Client models | `api/session-controller/client`, `api/workspace-controller/client` | Maintain React-free mirrors of Host state, resolve stream/unary races, own object identities and subscriptions, and expose narrow command services. |
|
|
18
|
+
| UI adapters | `client/ui-session`, `client/ui-workspace` | Convert model observables into root or Provider-bound Session Slot sources and own view-level navigation and status policy. |
|
|
19
|
+
| Conversation data | `client/ui-conversation`, target packages such as `ui-chat` and `ui-trajectory` | Assemble standard events and compact historical Assistant runs into independent target snapshots and own the shared conversation shell and input flow. |
|
|
20
|
+
| Composition and rendering | `client/ui-slots`, `client/ui-renderer`, `client/ui-layout`, feature UI packages | Declare extension locations, derive component props, bind observables to React hooks, and mount the final tree. |
|
|
21
|
+
|
|
22
|
+
The dependency direction is Host state → Remote transport → Client model → UI adapter → Conversation or presentation → Slots → React. User actions travel back through callbacks that close over an injected Client service or generated Remote namespace. A presentation component never receives Cordis `ctx`, a transport object, or another feature plugin's implementation.
|
|
23
|
+
|
|
24
|
+
## Browser boot
|
|
25
|
+
|
|
26
|
+
The Host writes the composed `WebBootGraph` to `window.__DSH_BOOT__` and installs the browser module-loader facade before parser-preloaded scripts execute. The module system is a lazy CommonJS table: loading a bundle registers its factory, while materializing an entry runs the factory with synchronous `require` over platform modules and declared dynamic dependencies.
|
|
27
|
+
|
|
28
|
+
The Web boot kernel creates the module system, prefetches `immediately` entries, mounts the vendored Cordis Loader, and creates every graph entry. Cordis service injection determines activation; module graph order determines only whether synchronous imports can be materialized. After the complete roster reaches a settled state, `ui-renderer` hydrates the framework-free boot DOM and calls the sole context-level `renderSlot('root')` operation. [Client Modules](client-modules.md) owns the graph, bundle route, cache revision, and loader details.
|
|
29
|
+
|
|
30
|
+
## Remote communication
|
|
31
|
+
|
|
32
|
+
Host business services annotate callable methods with Typert Remote decorators. Host generation emits strict descriptors, runtime codecs, declaration merges, and source maps. The Client-side `api-remotes` assembly selects those generated contributions and mounts concrete methods under `ctx.remote.<namespace>` and Session-scoped `agentCtx.remote.<namespace>`. Feature packages depend on the generated service face, not the Gateway implementation or a Host package's runtime entry.
|
|
33
|
+
|
|
34
|
+
The Connection owns request URL resolution, correlation, the `/api` carrier, trust checks, exact Fetch routes, and connection generations. API Gateway owns Remote dispatch, cancellation, logical streams, and selected Host event forwarding. Controller operations belong on generated Remote methods or explicit Remote streams; feature-owned downloads register exact Fetch routes. The [API Gateway reference](../api-gateway.md) defines generation and invocation, while the [Connection README](../../packages/client/connection/README.md) defines the physical carrier and trust policy.
|
|
35
|
+
|
|
36
|
+
The internal `$events` logical stream is the Connection generation source. Its opening `ready` frame carries the Host home used for path display and establishes the generation after Host listeners are attached, before any controller begins a baseline read. `ctx.remote.$on()` delivers allowlisted ordinary events to the root Client Context and scoped waterfall events to the resolved Session Context; a waterfall listener returns a result, calls `next()`, or rejects.
|
|
37
|
+
|
|
38
|
+
## Client models
|
|
39
|
+
|
|
40
|
+
Each API controller package owns a paired Host and Client face. The Host side owns authoritative mutation and stream production. The Client side owns an identity-stable, React-free model over the same generated wire types and exposes observable snapshots plus commands. UI packages consume these Client services and do not reproduce transport state in component stores.
|
|
41
|
+
|
|
42
|
+
### Sessions
|
|
43
|
+
|
|
44
|
+
[`api/session-controller`](../../packages/api/session-controller/README.md) exposes Host commands for list, search, creation, prompt, queue, cancellation, pagination, and follow/control streams. Its Client side is organized as `ClientSessions → SessionManager → Session`:
|
|
45
|
+
|
|
46
|
+
- `ClientSessions` provides `ctx.sessions`, owns references, source counts, Session scopes, and stable `SessionBinding` objects, and projects catalog state without selecting a global current Session.
|
|
47
|
+
- `SessionManager` owns the list baseline, live list/control updates, lazy Session instances, projection stores, subagent catalogs, and conflict ordering between pulls and later updates.
|
|
48
|
+
- Each `Session` owns one contiguous logical-event window represented by `SessionEventLikeEntry` values, paging, follow, prompt/control state, and the observable snapshot consumed by adapters.
|
|
49
|
+
|
|
50
|
+
The durable event path opens `follow()`, whose first frame contains the current header, tail page, cursor, and complete projection baseline. History records have an explicit `event` or `chunks` discriminator and an aligned inner `event`; the journal validates each inclusive logical sequence range before the Client retains the records as `SessionEventLikeEntry` values without per-record conversion. Each physical generation atomically replaces the retained window from that snapshot; standard live events then append by sequence. `page()` is reserved for older history and gap repair. The transient control stream starts every generation with a complete baseline and then applies projection updates.
|
|
51
|
+
|
|
52
|
+
### Workspaces
|
|
53
|
+
|
|
54
|
+
[`api/workspace-controller`](../../packages/api/workspace-controller/README.md) keeps Workspace mutation policy and the authoritative follow feed on the Host. `ClientWorkspaceModel` owns browser rows, Workspace order, archived and pinned Session id arrays, command echoes, and stream/unary race resolution. Every stream generation starts with a complete baseline followed by `upsert`, `remove`, `order`, `archived`, and `pinned` increments; reconnect replaces the model from the new baseline. `WorkspaceController` exposes that model as `ctx.workspaces`, while `ui-workspace` contributes `useWorkspaces` and navigation callbacks. The sidebar's `ArchivedFilter` controls default-hidden, shown, or archived-only rows in lists and search. Archived rows retain their ordering slots, render grayed, and cannot open until restored through the row or search-result Unarchive action. Restoration calls `workspace.unarchiveSession`, and its complete archive set reaches Clients through the unary response and `archived` increment. Session display order stays browser-local and includes hidden archives; pinning moves a Session within that full order, while unpinning does not restore its earlier position.
|
|
55
|
+
|
|
56
|
+
This pairing is not a second source of business truth. Host controllers decide durable state and mutation outcomes; Client models maintain the latest usable local projection, preserve object identity where useful to rendering, and encode how delayed responses and replacement baselines merge.
|
|
57
|
+
|
|
58
|
+
## Conversation and presentation
|
|
59
|
+
|
|
60
|
+
Web and desktop share the [Coding Tools preference](../../packages/client/ui-settings/README.md#use-this-package). It controls diagnostic Views, new-session preset selection, changed-file cards, and the builtin HTML preview policy without changing Session records.
|
|
61
|
+
|
|
62
|
+
`ui-session` installs the Session scope adapter and publishes `useSessions`, `useSessionStatus`, `useSessionRetainInfo`, `useSession`, `sessionId`, and `useProjection`. `SessionProvider` inherits an outer binding or binds an explicit `SessionReference`, so concurrent subtrees can target different Sessions. Domain adapters add further standard sources without putting React hooks on the model objects.
|
|
63
|
+
|
|
64
|
+
`ui-conversation` binds once to each `SessionBinding.eventSource`. Its event registry correlates durable Session events and Client-only `assistant/live-chunk` updates into stable business Contexts, and its view registry materializes target snapshots. Chat Assistant, Trajectory Assistant, and Turn Tail interpret both live chunks and the compact streams embedded in durable settlements, so reconnect and paged history reproduce the same Assistant state without durable token rows. `ui-chat` and `ui-trajectory` register separate Definitions and builders: they may interpret the same event family, but they do not import or share each other's final display model. The shell selects a registered view and passes its snapshot through standard hooks and Slots. [Conversation](conversation.md) defines Context identity, replay, Location data, target builders, and keyed renderers.
|
|
65
|
+
|
|
66
|
+
`ui-slots` provides the typed registry and lifecycle ledger; `ui-renderer` is the only package that binds bare observables through `useSyncExternalStore`, owns React contexts, and renders the root tree. Feature components receive framework hooks, owner props, store actions, and explicit injection through their derived props. [Web Client Slots](slots.md) lists those inputs, extension APIs, and the current Slot hierarchy.
|
|
67
|
+
|
|
68
|
+
## Data paths
|
|
69
|
+
|
|
70
|
+
| Path | Sequence |
|
|
71
|
+
|---|---|
|
|
72
|
+
| durable Session display | Host Session log → packed Remote `follow`/`page` history → Client `SessionEventLikeEntry` window → Conversation Contexts → target snapshot (`chat`, `trajectory`, or another registered target) → Slot view → React |
|
|
73
|
+
| transient Session control | Host control baseline → Remote snapshot stream → `SessionManager` projection stores → Session and list snapshots → standard hooks → components |
|
|
74
|
+
| Background jobs | Host job registry → `job.list` / `job.follow` → [`ClientJobs`](../../packages/api/job-controller/README.md) roster and output views → job list and panels |
|
|
75
|
+
| Workspace state | Host Workspace baseline and increments → `ClientWorkspaceModel` → `ctx.workspaces.list` → `useWorkspaces` → sidebar, hero, and navigation entries |
|
|
76
|
+
| scoped interaction | Host Cordis waterfall → API Remotes `$events` → `ctx.remote.$on()` on the Session Context → owning UI package → result or `next()` |
|
|
77
|
+
| user command | component callback → registration inject face or Slot owner → `ctx.sessions`, `ctx.workspaces`, or generated scoped Remote → Host Controller → authoritative update → stream or event projection back to the Client |
|
|
78
|
+
|
|
79
|
+
## Reconnection
|
|
80
|
+
|
|
81
|
+
Physical and logical recovery are separate. Gateway mux restores the physical WebSocket; each `RemoteStream` reopens its own logical source when the Connection publishes a usable generation. A carrier failure is retryable, while a business error, malformed opening item, or protocol violation is terminal for the owning logical stream.
|
|
82
|
+
|
|
83
|
+
Recovery follows the data's semantics:
|
|
84
|
+
|
|
85
|
+
- A durable Session journal validates logical sequence ranges and replaces its window from every generation's opening snapshot; `page()` supplies older history and repairs any later range gap.
|
|
86
|
+
- Session control and Workspace streams retain the last published value while disconnected, then atomically replace it from a fresh opening baseline.
|
|
87
|
+
- Ordinary forwarded notifications are not replayed. Stateful domains need a baseline, cursor, or explicit query; scoped waterfalls retain their own request lifetime.
|
|
88
|
+
|
|
89
|
+
There is no monolithic Client `Runtime`, `HostFrame`, `events.mux`, `events.host`, or universal `resync()` API. The Connection exposes generation state, Gateway owns logical stream supervision, and each Client model defines replacement or resume semantics appropriate to its data.
|
|
90
|
+
|
|
91
|
+
## Package boundaries
|
|
92
|
+
|
|
93
|
+
Feature plugin packages may share declarations through `import type`; they do not runtime-import or re-export another feature plugin's values. Cross-package behavior uses injected Cordis services, and cross-package UI uses Slots. Target-specific Conversation Definitions, projection helpers, and final view data stay with their target package even when Chat and Trajectory intentionally implement parallel logic.
|
|
94
|
+
|
|
95
|
+
Shared runtime values need a narrow static owner with no feature lifecycle, such as `client/store`, `ui-primitives`, or a browser-safe utility package. Transport and generated API assembly may import runtime contributions because assembling one protocol is their explicit responsibility. A feature package does not add `dsh.client.external` merely to bypass this rule.
|
|
96
|
+
|
|
97
|
+
Use the four detailed references according to the extension being added:
|
|
98
|
+
|
|
99
|
+
- [Client Modules](client-modules.md) for package discovery, loading, shared module identities, and boot order.
|
|
100
|
+
- [API Gateway](../api-gateway.md) for Host methods, generated Remote contributions, streams, and forwarded events.
|
|
101
|
+
- [Web Client Slots](slots.md) for components, hooks, stores, injection, and placement.
|
|
102
|
+
- [Conversation](conversation.md) for durable event correlation, target snapshots, and Chat or Trajectory view contributions.
|