@elizaos/plugin-agent-orchestrator 2.0.0-alpha.9 → 2.0.11-beta.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (75) hide show
  1. package/CHANGELOG.md +60 -0
  2. package/LICENSE +21 -0
  3. package/README.md +232 -0
  4. package/docs/SUBAGENT_FLOW_AND_PARITY.md +138 -0
  5. package/docs/default-eliza-skills-and-agent-bridge-plan.md +225 -0
  6. package/docs/economics-goal-runbook.md +140 -0
  7. package/docs/multi-account-orchestration.md +76 -0
  8. package/docs/orchestrator-buildout-followups.md +124 -0
  9. package/docs/orchestrator-dashboard-task-widget-secrets-assessment.md +174 -0
  10. package/docs/orchestrator-dashboard-task-widget-secrets-design.md +301 -0
  11. package/docs/research/orchestrator-view-research-report.md +273 -0
  12. package/docs/sub-agent-routing.md +227 -0
  13. package/package.json +197 -38
  14. package/dist/index.d.ts +0 -21
  15. package/dist/index.d.ts.map +0 -1
  16. package/dist/index.js +0 -21120
  17. package/dist/index.js.map +0 -109
  18. package/dist/src/actions/messaging.d.ts +0 -24
  19. package/dist/src/actions/messaging.d.ts.map +0 -1
  20. package/dist/src/actions/peek-subagent.d.ts +0 -3
  21. package/dist/src/actions/peek-subagent.d.ts.map +0 -1
  22. package/dist/src/actions/subagent-management.d.ts +0 -7
  23. package/dist/src/actions/subagent-management.d.ts.map +0 -1
  24. package/dist/src/actions/task-management.d.ts +0 -9
  25. package/dist/src/actions/task-management.d.ts.map +0 -1
  26. package/dist/src/config.d.ts +0 -4
  27. package/dist/src/config.d.ts.map +0 -1
  28. package/dist/src/providers/orchestrator-config.d.ts +0 -80
  29. package/dist/src/providers/orchestrator-config.d.ts.map +0 -1
  30. package/dist/src/providers/task-context.d.ts +0 -3
  31. package/dist/src/providers/task-context.d.ts.map +0 -1
  32. package/dist/src/services/agent-orchestrator-service.d.ts +0 -59
  33. package/dist/src/services/agent-orchestrator-service.d.ts.map +0 -1
  34. package/dist/src/services/messaging-service.d.ts +0 -111
  35. package/dist/src/services/messaging-service.d.ts.map +0 -1
  36. package/dist/src/services/sandbox-service.d.ts +0 -103
  37. package/dist/src/services/sandbox-service.d.ts.map +0 -1
  38. package/dist/src/services/subagent-service.d.ts +0 -140
  39. package/dist/src/services/subagent-service.d.ts.map +0 -1
  40. package/dist/src/sub-agents/adapter.d.ts +0 -13
  41. package/dist/src/sub-agents/adapter.d.ts.map +0 -1
  42. package/dist/src/sub-agents/claude-agent-sdk-sub-agent.d.ts +0 -18
  43. package/dist/src/sub-agents/claude-agent-sdk-sub-agent.d.ts.map +0 -1
  44. package/dist/src/sub-agents/codex-sdk-sub-agent.d.ts +0 -18
  45. package/dist/src/sub-agents/codex-sdk-sub-agent.d.ts.map +0 -1
  46. package/dist/src/sub-agents/eliza-sub-agent.d.ts +0 -27
  47. package/dist/src/sub-agents/eliza-sub-agent.d.ts.map +0 -1
  48. package/dist/src/sub-agents/elizaos-native-sub-agent.d.ts +0 -61
  49. package/dist/src/sub-agents/elizaos-native-sub-agent.d.ts.map +0 -1
  50. package/dist/src/sub-agents/index.d.ts +0 -10
  51. package/dist/src/sub-agents/index.d.ts.map +0 -1
  52. package/dist/src/sub-agents/opencode-sub-agent.d.ts +0 -44
  53. package/dist/src/sub-agents/opencode-sub-agent.d.ts.map +0 -1
  54. package/dist/src/sub-agents/registry.d.ts +0 -3
  55. package/dist/src/sub-agents/registry.d.ts.map +0 -1
  56. package/dist/src/sub-agents/sweagent-sub-agent.d.ts +0 -19
  57. package/dist/src/sub-agents/sweagent-sub-agent.d.ts.map +0 -1
  58. package/dist/src/sub-agents/tools.d.ts +0 -15
  59. package/dist/src/sub-agents/tools.d.ts.map +0 -1
  60. package/dist/src/sub-agents/types.d.ts +0 -170
  61. package/dist/src/sub-agents/types.d.ts.map +0 -1
  62. package/dist/src/types/index.d.ts +0 -12
  63. package/dist/src/types/index.d.ts.map +0 -1
  64. package/dist/src/types/messaging.d.ts +0 -202
  65. package/dist/src/types/messaging.d.ts.map +0 -1
  66. package/dist/src/types/sandbox.d.ts +0 -228
  67. package/dist/src/types/sandbox.d.ts.map +0 -1
  68. package/dist/src/types/subagent.d.ts +0 -232
  69. package/dist/src/types/subagent.d.ts.map +0 -1
  70. package/dist/src/types.d.ts +0 -138
  71. package/dist/src/types.d.ts.map +0 -1
  72. package/dist/src/utils/index.d.ts +0 -7
  73. package/dist/src/utils/index.d.ts.map +0 -1
  74. package/dist/src/utils/session.d.ts +0 -184
  75. package/dist/src/utils/session.d.ts.map +0 -1
@@ -0,0 +1,273 @@
1
+ # Orchestrator View Research Report
2
+
3
+ Date: 2026-05-30
4
+
5
+ ## High-Level Goal
6
+
7
+ Build an `/orchestrator` experience for ElizaOS that treats work as durable tasks, not disposable chats. The main orchestrator room should remain open indefinitely, create and fork task rooms from natural language or explicit UI controls, provision Claude/Codex/OpenCode/Eliza sub-agents using persistent `/goal`-style prompts, and never present a task as done until acceptance criteria, verification, and final handoff are complete. The UI must expose task status, room messages, sub-agent activity, model/provider/subscription selection, token spend, pause/resume/archive/delete controls, secure and local-insecure secret collection, voice/chat-controllable view actions, searchable paginated history, remote/mobile access, and route/action-specific automated plus screenshot/manual verification.
8
+
9
+ ## Current Status And Remaining Work
10
+
11
+ This report started as a research snapshot. The current branch now has a first durable `/api/orchestrator/*` task surface, `/orchestrator` plugin view registration, task-thread client methods backed by real routes, mandatory goal wrappers for direct coding-agent spawn/send paths, usage aggregation from ACP result usage, and dispatcher-level credential bridge wiring. Treat older unsupported-feature statements below as historical context when they conflict with this section.
12
+
13
+ Remaining work:
14
+
15
+ - Replace caller-supplied validation with registered verification hooks that collect durable evidence before `done`.
16
+ - Add end-to-end coverage for every declared orchestrator view capability, TUI endpoint, and task lifecycle mutation.
17
+ - Add screenshot/manual review for the orchestrator UI and any shared cloud frontend changes before declaring UI work complete.
18
+ - Finish provider/subscription settings cleanup and internationalize every new orchestrator string.
19
+ - Prove remote/mobile access against a local desktop agent and against cloud-connected agents.
20
+
21
+ ## Executive Findings
22
+
23
+ The repository already has three partial layers that should be joined rather than rebuilt from scratch:
24
+
25
+ - `plugins/plugin-agent-orchestrator` owns ACP subprocess sessions, agent selection, routing, workspace lifecycle, sub-agent progress, and task actions.
26
+ - `plugins/plugin-task-coordinator` owns a plugin view bundle at `/task-coordinator` plus task/session panels, but it is currently wired to mostly stubbed task-thread client methods.
27
+ - `packages/ui` owns the app shell, dynamic plugin views, built-in `/tasks` page, chat sidebar widgets, task-coordinator slots, route resolution, i18n, and local smoke coverage.
28
+
29
+ The main blocker is that the desired product model is "task thread with room, messages, sub-agents, goal, telemetry, and lifecycle", while the implemented backend model exposed to UI is mostly "ACP session list". Typed task-thread shapes exist in `packages/ui/src/api/client-types-cloud.ts`, and `CodingAgentTasksPanel` already expects thread APIs, but `ElizaClient` returns empty/null for task-thread list/detail/archive/reopen. The `/api/coding-agents/metrics` route returns `{}`, and no route currently exposes per-agent token usage.
30
+
31
+ ## Current Backend State
32
+
33
+ `plugin-agent-orchestrator` registers raw HTTP routes for coding agents, workspace, and issues, but not task-thread coordinator APIs. The route table includes `/api/coding-agents`, `/api/coding-agents/spawn`, `/api/coding-agents/metrics`, per-agent send/stop/output, parent-context/memory/active-workspaces, workspace, and issue routes in `plugins/plugin-agent-orchestrator/src/setup-routes.ts:81`.
34
+
35
+ The active session routes can list sessions, spawn sessions, send input, stop sessions, and read output. Relevant gaps:
36
+
37
+ - `/api/coding-agents/metrics` currently returns `{}` in `plugins/plugin-agent-orchestrator/src/api/agent-routes.ts:274`.
38
+ - `/api/coding-agents/workspace-files` returns an empty file list in `plugins/plugin-agent-orchestrator/src/api/agent-routes.ts:337`.
39
+ - `/api/coding-agents/settings` does expose preferred framework, configured subscription provider, and framework availability in `plugins/plugin-agent-orchestrator/src/api/agent-routes.ts:391`.
40
+ - `/api/coding-agents` lists raw ACP sessions from `ctx.acpService.listSessions()` in `plugins/plugin-agent-orchestrator/src/api/agent-routes.ts:441`.
41
+ - `/api/coding-agents/spawn` passes `initialTask: taskText` directly in `plugins/plugin-agent-orchestrator/src/api/agent-routes.ts:553`; it does not enforce a goal wrapper on direct API spawns.
42
+ - `/api/coding-agents/:id/send` passes user input directly to `sendToSession` in `plugins/plugin-agent-orchestrator/src/api/agent-routes.ts:610`; it does not enforce a goal wrapper on direct sends.
43
+ - Credential bridge routes exist, but the main dispatcher does not import or call `handleBridgeRoutes`; the bridge module itself still says to hook it from `routes.ts` in `plugins/plugin-agent-orchestrator/src/api/bridge-routes.ts:230`. This blocks secure credential retrieval from the current route surface.
44
+
45
+ The action path is richer. `TASKS_SPAWN_AGENT` constructs task-room/worktree-room metadata, route hints, initial task metadata, and swarming instructions. It tells agents to keep working until finished or blocked in `plugins/plugin-agent-orchestrator/src/actions/tasks.ts:375`, spawns with `initialTask: taskWithRouteHints` in `plugins/plugin-agent-orchestrator/src/actions/tasks.ts:723`, and records session metadata including task room, worktree route, label, source, and `initialTask` in `plugins/plugin-agent-orchestrator/src/actions/tasks.ts:729`. However, it intentionally returns `continueChain: false` immediately after spawn in `plugins/plugin-agent-orchestrator/src/actions/tasks.ts:762` so the parent turn ends while the sub-agent works asynchronously. That is the opposite of the requested "orchestrator does not return until goal completion" behavior unless the new task/goal runner adds a durable task status contract outside the single chat turn.
46
+
47
+ `TASKS_SEND_TO_AGENT` also sends direct follow-up text unless it is reacting to a routed incomplete completion, where it builds a stronger "continue original task" follow-up in `plugins/plugin-agent-orchestrator/src/actions/tasks.ts:856`. This should become a mandatory `/goal` envelope for Claude/Codex/OpenCode/Eliza prompts, not a best-effort retry phrase.
48
+
49
+ ## Current UI State
50
+
51
+ `plugin-task-coordinator` already declares a GUI view at `/task-coordinator`, an XR view at `/task-coordinator`, and a TUI view at `/task-coordinator/tui` in `plugins/plugin-task-coordinator/src/index.ts:6`. The GUI exports `CodingAgentTasksPanel`; the TUI exposes capabilities for listing sessions, listing task threads, opening a thread, stopping a session, and refreshing.
52
+
53
+ `packages/ui` can route dynamic plugin views by matching `ViewRegistryEntry.path` and loading their `bundleUrl` through `DynamicViewLoader` in `packages/ui/src/App.tsx:451`. Built-in static views include `tasks`, which renders `TasksPageView` in `packages/ui/src/App.tsx:542`. The navigation type and path map include `tasks`, but not `orchestrator`, in `packages/ui/src/navigation/index.ts:44` and `packages/ui/src/navigation/index.ts:318`.
54
+
55
+ `packages/ui/src/slots/task-coordinator-slots.tsx` deliberately keeps app-core from importing the plugin directly and instead lets plugins register task-coordinator components into slots. This is the right pattern to preserve if `/orchestrator` remains a frontend plugin view, but it also means `plugin-agent-orchestrator` cannot simply add a React file without introducing a browser build and a view-registration story.
56
+
57
+ The current task panel is a useful seed, but not enough for the requested view. `CodingAgentTasksPanel` polls `client.listCodingAgentTaskThreads` every 5 seconds in `plugins/plugin-task-coordinator/src/CodingAgentTasksPanel.tsx:680`, loads selected thread details, and has archive/reopen handlers in `plugins/plugin-task-coordinator/src/CodingAgentTasksPanel.tsx:802`. The client methods behind those calls currently return empty/null/false in `packages/ui/src/api/client-agent.ts:3403`, so local task rooms and history are not truly available.
58
+
59
+ Chat sidebar widgets are registered for app runs and activity under `agent-orchestrator` in `packages/ui/src/widgets/registry.ts:67`, but they are not the requested full-room orchestration surface.
60
+
61
+ There is no first-class `/orchestrator` route in desktop or cloud today. Desktop has a built-in `tasks` tab and `/apps/tasks` surface, while `plugin-task-coordinator` registers `/task-coordinator`. Cloud is further behind for this specific feature: inline cloud agent chat is intentionally not wired yet, and public cloud chat is character-room text streaming rather than orchestrator/task-room state.
62
+
63
+ Slash-chat support is also not built in for this product yet. Existing slash behavior is tied to saved/custom action expansion, not built-in `/orchestrator`, `/task`, `/spawn`, or `/tasks` commands. The first version should decide whether slash commands call the orchestrator API directly or translate into planner-visible task requests.
64
+
65
+ ## Data Model Required
66
+
67
+ Add a durable orchestrator task model owned by `plugin-agent-orchestrator`:
68
+
69
+ - `orchestrator_tasks`: id, title, goal, status, priority, owner, worldId, mainRoomId, taskRoomId, archive/delete flags, created/updated/closed timestamps, acceptance criteria, current plan, parent/fork source, provider policy, pause state.
70
+ - `orchestrator_task_sessions`: taskId, sessionId, agentType, providerSource, model, goalPrompt, workdir, status, active tool, lastActivityAt, spawnedAt, stoppedAt, completionSummary, retry count, token totals.
71
+ - `orchestrator_task_events`: taskId, sessionId, eventType, summary, data, timestamp.
72
+ - `orchestrator_task_messages`: taskId, roomId, messageId, sender kind, content refs, searchable text, timestamp. Prefer referencing existing runtime memories where possible instead of duplicating message bodies.
73
+ - `orchestrator_task_usage`: taskId, sessionId, provider, model, input/output/reasoning/cache tokens, cost estimate, source event id, timestamp.
74
+ - `orchestrator_task_artifacts`: taskId, sessionId, path/URI, artifact type, title, verification status.
75
+ - `orchestrator_task_decisions`: taskId, decision type, action selected, prompt excerpt, reasoning summary, timestamp.
76
+
77
+ The existing UI thread types in `packages/ui/src/api/client-types-cloud.ts:987` are close, but they need room/message fields, token usage, provider/model usage, pause state, and task fork lineage. The existing `CodingAgentTaskThreadDetail` already includes sessions, decisions, events, artifacts, transcripts, and pending decisions in `packages/ui/src/api/client-types-cloud.ts:1104`; extend this instead of inventing an incompatible parallel shape.
78
+
79
+ Usage capture needs backend work. ACP usage events are currently treated as informational and not surfaced in `plugins/plugin-agent-orchestrator/src/services/acp-service.ts:1639`, so token/cost UI cannot be accurate until those events are persisted and aggregated.
80
+
81
+ ## Prompt And Goal Semantics
82
+
83
+ The requested behavior should be implemented as a task runner contract, not by expecting one HTTP response or one chat turn to remain open forever. The orchestrator should create a durable goal record, spawn agents with a mandatory goal wrapper, and continue dispatching, verifying, retrying, pausing, or escalating until the goal status is terminal.
84
+
85
+ Required prompt policy:
86
+
87
+ - Every `spawnSession` initial task for Claude, Codex, OpenCode, ElizaOS, and Pi Agent must pass through a single `buildGoalPrompt(...)` function.
88
+ - Every `sendToSession` follow-up from direct API, UI, or planner action must pass through `buildGoalFollowUp(...)`.
89
+ - The wrapper must include: goal, acceptance criteria, room IDs, workdir, allowed capabilities, "do not finish until complete or genuinely blocked", "verify before final", "report token/tool status when available", and "return structured completion fields".
90
+ - Direct API spawns in `agent-routes.ts` must get the same wrapper as `TASKS_SPAWN_AGENT`; currently they bypass route hints and goal semantics.
91
+ - Direct API sends in `agent-routes.ts` must get the same wrapper as `TASKS_SEND_TO_AGENT`; currently they pass raw input.
92
+ - When a sub-agent claims completion, the router should transition task state to `validating`, run verification hooks, and only mark `done` after proof passes.
93
+ - The durable task runner should keep a structured objective, blocked reason, retry/resume budget, validation state, and completion summary. Today goal persistence is prompt text plus ACP session metadata, not a true `/goal` record.
94
+
95
+ The parent orchestrator should not "return done" while a task remains active. In UI terms, this means the chat can acknowledge "task created and running", but task status remains active until validation completes. In agent terms, any final answer about the task must be gated on the durable task state, not just the sub-agent's `task_complete` event.
96
+
97
+ ## API Surface Needed
98
+
99
+ Add coordinator routes under `/api/coding-agents/coordinator/*` or alias them to `/api/orchestrator/*`:
100
+
101
+ - `GET /api/orchestrator/status`: aggregate active tasks, sessions, provider usage, token spend, paused state.
102
+ - `GET /api/orchestrator/tasks?cursor=&status=&search=&includeArchived=`
103
+ - `POST /api/orchestrator/tasks`: create task from structured form or inferred chat action.
104
+ - `GET /api/orchestrator/tasks/:taskId`
105
+ - `PATCH /api/orchestrator/tasks/:taskId`: update title, goal, acceptance criteria, priority.
106
+ - `POST /api/orchestrator/tasks/:taskId/pause`
107
+ - `POST /api/orchestrator/tasks/:taskId/resume`
108
+ - `POST /api/orchestrator/tasks/:taskId/archive`
109
+ - `DELETE /api/orchestrator/tasks/:taskId`
110
+ - `POST /api/orchestrator/tasks/:taskId/fork`
111
+ - `POST /api/orchestrator/tasks/:taskId/messages`: user joins a task room.
112
+ - `GET /api/orchestrator/tasks/:taskId/messages?cursor=&limit=`
113
+ - `GET /api/orchestrator/tasks/:taskId/events?cursor=&limit=`
114
+ - `GET /api/orchestrator/tasks/:taskId/usage`
115
+ - `POST /api/orchestrator/tasks/:taskId/agents`: add sub-agent.
116
+ - `POST /api/orchestrator/tasks/:taskId/agents/:sessionId/stop`
117
+ - `POST /api/orchestrator/pause-all`
118
+ - `POST /api/orchestrator/resume-all`
119
+
120
+ Keep compatibility by mapping `client.listCodingAgentTaskThreads`, `getCodingAgentTaskThread`, `archiveCodingAgentTaskThread`, and `reopenCodingAgentTaskThread` to these real routes instead of returning stubs.
121
+
122
+ Also register and dispatch credential bridge routes if they are part of provider setup or sub-agent secret retrieval. Leaving `bridge-routes.ts` present but unmounted creates a false sense of security coverage.
123
+
124
+ Normalize the coding-agent preflight contract at the same time. Current frontend code paths expect an object with `installed`/`available` style fields, while the backend preflight route returns an array of preflight rows. This can make the code button and task affordances appear unreliably.
125
+
126
+ ## Route And View Registration Recommendation
127
+
128
+ Use `/orchestrator` as the primary product route.
129
+
130
+ Preferred implementation path:
131
+
132
+ 1. Rename or duplicate `plugin-task-coordinator` view registration so GUI path `/orchestrator` loads the new orchestrator component, with `/task-coordinator` retained as a compatibility alias.
133
+ 2. Keep `plugin-agent-orchestrator` as the backend owner and add all durable task/coordinator routes there.
134
+ 3. Keep `packages/ui` slots for shared app-shell embedding, but move the full orchestrator product UI into the plugin view bundle to avoid hardcoding a large new static page in `packages/ui`.
135
+ 4. Add `orchestrator` to the UI navigation type/path map only if it needs a first-class built-in tab. If it is a plugin view with `desktopTabEnabled`, dynamic view routing can load it by path without adding a built-in tab.
136
+
137
+ This preserves the existing package boundary: Node orchestration backend in `plugin-agent-orchestrator`, React view bundle in a browser-capable plugin package, and generic shell/view infrastructure in `packages/ui`.
138
+
139
+ Do not start with cloud `/orchestrator` as the primary implementation path. Build the desktop/local route and backend task contract first, then bridge the same API shape into cloud once cloud-side agent chat/task containers expose durable task-room state.
140
+
141
+ ## UX Requirements
142
+
143
+ The `/orchestrator` screen should be an operational workbench:
144
+
145
+ - Left rail: ongoing tasks with status, priority, latest activity, active sub-agent count, token spend, provider icons, pause/archive affordances, search, filters, archived toggle, and plus button.
146
+ - Center: selected room timeline with user, orchestrator, and sub-agent messages; system events collapsed by default; paginated search and jump-to-time.
147
+ - Right rail or inspector: sub-agent roster, current tool/activity, workdir/repo, active branch, artifacts, verification checklist, acceptance criteria, token/cost breakdown, provider/subscription state.
148
+ - Composer: sends to selected room, can also create/fork/update tasks via chat intent.
149
+ - Task creation: plus button opens a compact structured form, but any field can be inferred from chat.
150
+ - Controls: pause task, pause all agents, resume, archive, delete, fork, add agent, stop agent, change provider/model, copy/share task link.
151
+ - Voice/chat actions: every clickable action must have a view capability and natural-language route so "pause this task", "fork it", "show Codex only", or "add Claude to this room" works while the view is open.
152
+ - Settings: provider/subscription management must show Eliza Cloud, local Eliza, Claude, Codex/OpenAI, OpenCode, and any configured cloud routes; copy must be fully internationalized and avoid leaking tokens.
153
+
154
+ Avoid a marketing layout. This is an operator console: dense, stable, quiet, readable, and optimized for scanning.
155
+
156
+ ## Secret And OAuth Flow
157
+
158
+ The codebase already has sensitive request infrastructure for secure inline owner-app secret requests and local route submission. `owner-app-inline-adapter` builds owner-only secret forms, and `sensitive-request-routes.ts` supports create/get/submit/cancel flows. Extend this for orchestrator provider setup:
159
+
160
+ - Secure cloud path: OAuth and API secrets stored in Eliza Cloud or scoped cloud vault, never pasted into public chat.
161
+ - Secure local path: local vault/secret manager when available.
162
+ - Insecure local fallback: explicit "stored locally/insecure" mode for fully local use where cloud is unavailable.
163
+ - Form inference: chat can start an OAuth/secret request, the form opens prefilled with inferred provider/scope, and user can submit in UI.
164
+ - Audit: every secret request emits redacted audit events.
165
+ - Tests: verify redaction, route auth, submission, cancellation, cloud unavailable fallback, and no accidental secret echo in messages/events/transcripts.
166
+
167
+ ## Remote And Mobile Access
168
+
169
+ The requested "phone as remote to desktop agent" mode needs explicit scope:
170
+
171
+ - Local desktop runtime exposes a remote pairing/tunnel endpoint with auth and revocation.
172
+ - Mobile connects as a remote client to the desktop agent, not as a separate agent owner.
173
+ - Cloud mode connects all clients to the cloud-hosted state.
174
+ - The orchestrator task state must sync through the same API shape in both local-remote and cloud modes.
175
+ - Long-running task updates should use websocket/SSE where available, with polling fallback matching current 5-second polling.
176
+
177
+ ## Test And Verification Plan
178
+
179
+ Backend unit coverage:
180
+
181
+ - Route registration includes every new `/api/orchestrator/*` and coordinator compatibility route.
182
+ - Direct HTTP handler tests for `/api/coding-agents` list/spawn/get/send/stop/output, including missing service, invalid body, spawn workdir rejection, concurrency limit, and output fallback.
183
+ - Task create/list/detail/search/pagination/archive/delete/fork/pause/resume.
184
+ - Session spawn and send always call goal-wrapper builders.
185
+ - Direct API spawn/send and planner action spawn/send produce equivalent goal envelopes.
186
+ - Sub-agent completion does not mark done until validation passes.
187
+ - Token usage aggregation handles missing provider usage, partial usage, duplicate events, and multiple providers.
188
+ - Credential bridge routes are mounted, authorized, redacted, and unavailable in modes that should not expose them.
189
+ - Store fallback works for SQL, file, and memory session stores.
190
+ - Access policy rejects unauthorized remote/mobile clients.
191
+
192
+ UI component coverage:
193
+
194
+ - Empty state, loading state, error state, active task, blocked task, validating task, done task, archived task.
195
+ - Search, pagination, task selection, task creation form, plus button, fork, pause, pause all, resume, archive, delete confirmation, add sub-agent, stop sub-agent.
196
+ - Token/cost visualizations with zero/unknown/large values.
197
+ - Provider settings and i18n keys for all visible strings.
198
+ - Voice/chat capability calls for every action.
199
+ - Preflight response normalization so provider/setup affordances render consistently.
200
+ - `mapAcpSessionsToCodingAgentSessions`, `getCodingAgentStatus`, `stopCodingAgent`, scratch workspace actions, and PTY subscribe/send/resize/buffer client helpers.
201
+
202
+ End-to-end coverage:
203
+
204
+ - Create task by plus button.
205
+ - Create task by chat message.
206
+ - Fork task from chat.
207
+ - Add Claude/Codex/OpenCode/Eliza sub-agent where available, with mocked providers in CI.
208
+ - Pause one task stops/suspends all attached agents.
209
+ - Pause all affects all running tasks.
210
+ - Archive/delete removes from active list and preserves/cleans history according to mode.
211
+ - Search and paginate main orchestrator room history and task room history.
212
+ - Secret request: secure cloud, secure local, insecure local fallback.
213
+ - Remote phone connects to desktop agent and controls an active task.
214
+ - Cloud client and desktop client see the same task state.
215
+ - Slash commands `/task`, `/tasks`, `/spawn`, and `/stop` route to the intended task APIs or planner requests with destructive-action confirmation.
216
+ - Validation failure triggers retry/reopen, not `done`.
217
+ - Token spend updates per sub-agent and aggregate.
218
+ - Terminal/output management: buffered output loads, live `pty-output` appends, interrupt sends Ctrl-C, terminal input sends a line, and stop posts to the session stop endpoint.
219
+ - Failure handling: `/api/coding-agents` 503, thread list failure, detail failure, output fetch failure, and stop failure all render actionable UI without swallowing the error silently.
220
+
221
+ Existing coverage to extend:
222
+
223
+ - `plugins/plugin-agent-orchestrator/__tests__/unit/register-routes.test.ts` verifies route loader registration.
224
+ - `plugins/plugin-agent-orchestrator/__tests__/unit/sub-agent-router.test.ts` verifies task-complete synthetic memory routing.
225
+ - `packages/app/test/ui-smoke/plugin-views-visual.spec.ts:76` already includes `/task-coordinator`.
226
+ - `packages/app/test/ui-smoke/all-pages-clicksafe.spec.ts:477` stubs coding-agent preflight/status routes.
227
+ - `packages/cloud-frontend/AGENTS.md` requires `bun run --cwd packages/cloud-frontend audit:cloud` plus manual review for any `packages/cloud-frontend` UI change.
228
+
229
+ Manual visual review:
230
+
231
+ - For `packages/ui` / plugin view changes, run local app UI smoke and plugin-view visual specs, capture desktop and mobile screenshots of `/orchestrator`, `/orchestrator?archived=1`, task detail, settings/provider setup, and secret request forms.
232
+ - Until an app-wide manual-review protocol exists, require screenshots for `/task-coordinator`, `/task-coordinator/tui`, chat with active sub-agent chip, and terminal drawer/side-panel as compatibility surfaces.
233
+ - If any `packages/cloud-frontend` files are touched, follow the mandatory cloud visual audit loop exactly and leave no affected route at `needs-work` or `broken`.
234
+ - For meaningful redesign, iterate screenshots at least five times, tracking issues fixed per loop.
235
+
236
+ ## Highest Risks
237
+
238
+ - Treating ACP sessions as tasks will lose room history, fork lineage, and durable status. A real task store is required.
239
+ - Letting direct API routes bypass goal-wrapper semantics will create inconsistent worker behavior.
240
+ - Marking completion from `task_complete` alone will violate the "certain complete" requirement.
241
+ - Token accounting may not be available uniformly across Claude/Codex/OpenCode/Eliza; the UI must distinguish measured, estimated, and unavailable usage.
242
+ - Existing archive, reopen, and pause action paths report unsupported ACP-only mode, so current UI archive/reopen affordances cannot work until task-thread lifecycle exists.
243
+ - Putting the full UI in `plugin-agent-orchestrator` without a browser build will break current package boundaries.
244
+ - Building cloud UI before cloud task-room APIs exist will produce a decorative shell, not a working orchestrator.
245
+ - Preflight response mismatch can hide or misstate provider availability.
246
+ - Existing Playwright coverage proves plugin-view renderability, not the management workflows; without dedicated mocked workflow specs, regressions in stop, archive, thread detail, terminal output, and error states will slip through.
247
+ - "100% e2e coverage" is aspirational unless scoped to reachable user flows and backed by mocked provider harnesses; live Claude/Codex tests must stay gated.
248
+ - Secrets can leak through transcripts, task messages, or debug logs unless every sensitive request path redacts by default.
249
+
250
+ ## Recommended Build Sequence
251
+
252
+ 1. Backend task store and routes: create task/thread persistence, expose coordinator APIs, wire compatibility client methods.
253
+ 2. Goal wrapper: centralize spawn/send prompt wrapping and enforce it on both action and HTTP routes.
254
+ 3. Usage telemetry: capture token/cost events where providers expose them and surface estimated/unavailable states otherwise.
255
+ 4. View registration: ship `/orchestrator` plugin view, keep `/task-coordinator` alias.
256
+ 5. UI workbench: task rail, room timeline, sub-agent inspector, controls, create/fork flows, i18n.
257
+ 6. Chat/voice capabilities: add view actions for every UI control and natural-language task commands.
258
+ 7. Secret/OAuth forms: integrate sensitive-request flows into provider setup and task execution blockers.
259
+ 8. Remote/mobile: unify cloud/local-remote state APIs and websocket/polling updates.
260
+ 9. Tests and visual audits: build the e2e matrix, mocked providers, live gated smokes, screenshot review artifacts, and regression docs.
261
+
262
+ ## Acceptance Criteria
263
+
264
+ - `/orchestrator` opens as a first-class view and `/task-coordinator` remains compatible or redirects.
265
+ - A user can create, fork, pause, resume, archive, delete, search, and inspect tasks by UI and chat/voice.
266
+ - Each task has a durable room, message history, event history, sub-agent roster, status, goal, acceptance criteria, artifacts, and usage.
267
+ - Claude/Codex/OpenCode/Eliza workers always receive goal-wrapped prompts and follow-ups.
268
+ - The orchestrator never reports a task as `done` until validation passes or a human explicitly overrides.
269
+ - Provider defaults follow configured subscription readiness: user-owned Claude/OpenAI where available, otherwise Eliza Cloud/local policy.
270
+ - Secure and local-insecure secret setup paths are clear, tested, and redacted.
271
+ - Desktop, mobile remote, and cloud clients observe consistent task state.
272
+ - Every visible string is internationalized.
273
+ - Every button/flow/input/view has automated coverage or a documented impossible-to-automate reason plus manual screenshot review.
@@ -0,0 +1,227 @@
1
+ # Sub-agent routing
2
+
3
+ > Canonical orchestration path for ACP sub-agents. ACP-spawned sessions route
4
+ > through `AcpService` and `SubAgentRouter`.
5
+
6
+ ## Goals
7
+
8
+ 1. **Origin tracking.** When the main agent spawns a sub-agent in response to
9
+ a user message in room R, the sub-agent's terminal output (`task_complete`,
10
+ `error`, `blocked`) lands back in room R, addressed appropriately.
11
+ 2. **Main-agent-in-the-loop.** When a sub-agent reports done, the **main
12
+ agent** — not a separate coordinator — decides whether to:
13
+ - reply to the user (sub-agent finished, here's the result),
14
+ - reply to the sub-agent via `SEND_TO_AGENT` (proof not satisfying; keep
15
+ going),
16
+ - or both in one turn.
17
+ 3. **Cache friendliness.** Sub-agent updates should not invalidate the
18
+ stable provider/system prefix on every event.
19
+
20
+ ## Components
21
+
22
+ ### `AcpService`
23
+
24
+ Spawn surface. TASKS op=create records origin context in
25
+ `session.metadata` at spawn time:
26
+
27
+ ```ts
28
+ {
29
+ messageId: message.id, // parent message UUID
30
+ roomId: taskRoomId, // task-owner room, defaults to message.roomId
31
+ originRoomId: message.roomId,
32
+ taskRoomId,
33
+ worktreeRoomId,
34
+ swarmRooms: [
35
+ { roomId: taskRoomId, roles: ["task"] },
36
+ { roomId: worktreeRoomId, roles: ["worktree"] },
37
+ ],
38
+ worldId: message.worldId,
39
+ userId: message.entityId,
40
+ label,
41
+ source: content.source,
42
+ }
43
+ ```
44
+
45
+ ### `SubAgentRouter` (`services/sub-agent-router.ts`)
46
+
47
+ Subscribes to `AcpService.onSessionEvent`. On `task_complete`, `error`, or
48
+ `blocked` (boundary events only — not streaming chunks), it:
49
+
50
+ 1. Reads `session.metadata` for origin and swarm-room keys.
51
+ 2. Constructs a synthetic `Memory` with:
52
+ - `entityId` = a deterministic per-session sub-agent UUID derived locally
53
+ via SHA1 of `<runtime.agentId>:acpx:sub-agent:<sessionId>` (no runtime
54
+ dependency on `@elizaos/core`'s `createUniqueUuid` so the router stays
55
+ type-only on core),
56
+ - `agentId` = `runtime.agentId`,
57
+ - `roomId` = the selected swarm target room,
58
+ - `content.source` = `"sub_agent"`,
59
+ - `content.inReplyTo` = origin `messageId`,
60
+ - `content.metadata.subAgent*` carries the structured event
61
+ (`subAgentSessionId`, `subAgentLabel`, `subAgentEvent`,
62
+ `subAgentStatus`, `subAgentAgentType`, `subAgentRoundTrip`,
63
+ `subAgentRoundTripCap`, `subAgentCapExceeded`, `originUserId`,
64
+ `originMessageId`, `originSource`).
65
+ - `content.metadata.subAgentRoutingKind`,
66
+ `subAgentTargetRoomId`, `subAgentTargetRoomRole`,
67
+ `subAgentTargetRoomRoles`, `taskRoomId`, `worktreeRoomId`, and
68
+ `subAgentSwarmRooms` tell the main agent why this memory landed in
69
+ this room and which other swarm room(s) exist for the task.
70
+ 3. Delivers via `runtime.messageService.handleMessage(runtime, memory, callback)`,
71
+ which also persists the memory. If `messageService` is unavailable, it
72
+ falls back to `runtime.createMemory(..., "messages")` plus
73
+ `MESSAGE_RECEIVED`.
74
+
75
+ For platform-originated tasks, the router builds a short-lived callback from
76
+ `runtime.sendMessageToTarget` so the planner's answer can return to the same
77
+ selected swarm room.
78
+
79
+ #### Why only boundary events
80
+
81
+ Streaming events (`agent_message_chunk`, `tool_running`, `ready`) would
82
+ re-fire the planner constantly and burn the prompt cache. Live status is
83
+ exposed via the provider instead. The router is the channel for events
84
+ that warrant an action decision.
85
+
86
+ #### Dedup / idempotency
87
+
88
+ Events are deduped in-memory by
89
+ `<sessionId>|<event>|<status>|<short hash of payload>`. Same sub-agent
90
+ re-emitting the same `task_complete` payload posts once. A different
91
+ response payload posts again — that's "the sub-agent did more work and
92
+ reported a new state".
93
+
94
+ Swarm target rooms are also normalized before posting. `taskRoomId` is first,
95
+ `worktreeRoomId` is second, duplicate room IDs collapse into one target, and
96
+ the collapsed target keeps both roles (`["task", "worktree"]`). That means a
97
+ task room that is also the worktree coordination room gets one useful message
98
+ rather than two identical messages with ambiguous purpose.
99
+
100
+ #### Routing kinds
101
+
102
+ Most terminal events use `subAgentRoutingKind: "TASK_STATUS"` and fan out to
103
+ the normalized task/worktree swarm rooms. Two explicit coordination events are
104
+ targeted:
105
+
106
+ - `QUESTION_FOR_TASK_CREATOR` routes only to the task room and carries
107
+ `subAgentTargetRoomRole: "task"`.
108
+ - `AGENT_COORDINATION` routes to the worktree room when present, otherwise the
109
+ task room, and carries the selected target role.
110
+
111
+ `blocked` events default to `QUESTION_FOR_TASK_CREATOR` because the sub-agent
112
+ is waiting on human or parent-agent input.
113
+
114
+ #### Disable switch
115
+
116
+ `ACPX_SUB_AGENT_ROUTER_DISABLED=1` keeps the service registered but unbound
117
+ (useful for tests, headless backfills, or staging where you want spawning
118
+ without runtime injection).
119
+
120
+ #### Round-trip cap
121
+
122
+ To prevent ping-pong loops where the main agent and a sub-agent endlessly
123
+ ask each other to keep going, the router tracks per-session inject count.
124
+ When the count exceeds `ACPX_SUB_AGENT_ROUND_TRIP_CAP` (default 32) the
125
+ router force-stops the session and emits a single
126
+ `round_trip_cap_exceeded` memory carrying `subAgentRoundTrip`,
127
+ `subAgentRoundTripCap`, and `subAgentCapExceeded: true`. Subsequent events
128
+ from the same capped session are suppressed.
129
+
130
+ Set `ACPX_SUB_AGENT_ROUND_TRIP_CAP=N` in the runtime config to override.
131
+ The default of 32 is generous; a typical sub-agent task hits 1–5
132
+ round-trips before terminal completion.
133
+
134
+ ### `activeSubAgentsProvider` (`providers/active-sub-agents.ts`)
135
+
136
+ Cache-friendly view of live sub-agent sessions. Filters to:
137
+
138
+ - sessions whose `metadata.roomId` is set (i.e. routed by `createTaskAction`),
139
+ - sessions not in a terminal status (`stopped`, `completed`, `error`,
140
+ `errored`, `cancelled`).
141
+
142
+ The text is **structural only** — id, label, agentType, bucketed status,
143
+ last two workdir segments. No timestamps, no message excerpts. Sorted by
144
+ `sessionId` so the rendered text is byte-stable across turns when the
145
+ active set is unchanged.
146
+
147
+ Status bucketing: `ready`, `running`, `busy`, `tool_running`, and
148
+ `authenticating` all collapse to the literal string `"active"` in the
149
+ provider text. `blocked` is preserved as a distinct value (the planner
150
+ needs to know a session is waiting for input). Terminal statuses
151
+ (`stopped`, `completed`, `error`, `errored`, `cancelled`) cause the
152
+ session to be filtered out entirely. This keeps the cached provider
153
+ segment byte-identical across transient status flips like
154
+ `ready → tool_running → ready`, which would otherwise invalidate the
155
+ prefix cache on every tool call.
156
+
157
+ This is the live status channel. The synthetic Memory posted by the router
158
+ is the per-event channel.
159
+
160
+ ### Action set
161
+
162
+ The main agent's planner sees:
163
+
164
+ - **`REPLY`** (from the bootstrap action set) — replies to the user in
165
+ current room.
166
+ - **`SEND_TO_AGENT { sessionId, text }`** — pushes a follow-up to a
167
+ live sub-agent. Use when the sub-agent's proof is unsatisfying or it
168
+ asked a clarifying question.
169
+ - **`STOP_AGENT { sessionId }`** — terminates. Use when the sub-agent's
170
+ output is clearly final and you don't want it idling.
171
+ - **`ACPX_CREATE_TASK`** — spawn additional sub-agents.
172
+
173
+ Multi-action plans (e.g. `[REPLY, SEND_TO_AGENT]`) are supported by the
174
+ planner and execute sequentially in one turn.
175
+
176
+ ## Cache discipline
177
+
178
+ Anthropic prompt caching breaks at segment boundaries (see
179
+ `plugin-anthropic/models/text.ts`). The plugin marks providers and action
180
+ examples as `stable: true`. Sub-agent flow is designed around this:
181
+
182
+ - **Stable prefix (cached):** system prompt, character bio, action examples,
183
+ active-sub-agents provider text (structural only, sorted, deterministic).
184
+ - **Volatile suffix (re-tokenized):** the sub-agent's synthetic message
185
+ text (the per-event narration).
186
+
187
+ Each new sub-agent event invalidates only the message tail. The provider
188
+ text changes only when a session enters or leaves the active set, not on
189
+ every chunk.
190
+
191
+ A per-session router invocation is one cache-miss tail; everything before
192
+ the most recent turn stays warm.
193
+
194
+ ## Loop safety
195
+
196
+ - The router emits **inbound** memories with `entityId` set to the
197
+ sub-agent's pseudo-UUID (not `runtime.agentId`), so the runtime processes
198
+ them as messages from another entity, not as the agent's own outputs.
199
+ - The main agent's reply via `SEND_TO_AGENT` does not directly trigger a
200
+ new `task_complete`. The sub-agent has to actually do work first, which
201
+ bounds re-entry.
202
+ - Dedup prevents accidental double-injection from event re-emission.
203
+ - The round-trip cap (above) is the hard ceiling for ping-pong loops.
204
+
205
+ ## ACP Boundary
206
+
207
+ `plugin-agent-orchestrator` has a single task-agent transport: `AcpService`.
208
+ There is no PTY/coordinator fallback path.
209
+
210
+ Autonomous follow-up decisions are handled by the main agent's normal action
211
+ selection over the synthetic Memory emitted by `SubAgentRouter`.
212
+
213
+ ## Testing
214
+
215
+ - `__tests__/unit/sub-agent-router.test.ts` — origin tracking, dedup,
216
+ streaming-event filtering, disable switch, error narration, fallback
217
+ emit, unsubscribe.
218
+ - `__tests__/unit/active-sub-agents.test.ts` — origin filtering, terminal
219
+ exclusion, deterministic sort, no volatile fields, action-hint text.
220
+
221
+ ## Related files
222
+
223
+ - [src/services/sub-agent-router.ts](../src/services/sub-agent-router.ts)
224
+ - [src/providers/active-sub-agents.ts](../src/providers/active-sub-agents.ts)
225
+ - [src/services/acp-service.ts](../src/services/acp-service.ts)
226
+ - [src/actions/create-task.ts](../src/actions/create-task.ts)
227
+ - [src/actions/send-to-agent.ts](../src/actions/send-to-agent.ts)