agentic-orchestrator 0.1.28 → 0.2.0

package/spec-files/outstanding/agentic_orchestrator_request_more_context_and_dashboard_human_input_spec.md

@@ -0,0 +1,456 @@
+# Feature Spec: Request-More-Context Recovery and Dashboard Human Input (AOP)
+
+> **Purpose of this document**: Fix the current defect where worker `REQUEST.action=more_context` is treated as handled without actually unblocking execution, and define the first-class user-input and dashboard flows required to resolve that class of stall deterministically.
+
+**Version:** 1.0
+**Date:** 2026-03-16
+**Status:** Draft
+**Roadmap Mapping:** M52
+**Depends On:**
+
+- `spec-files/outstanding/agentic_orchestrator_human_input_interaction_protocol_spec.md`
+- `spec-files/outstanding/agentic_orchestrator_dashboard_quick_launch_and_control_surface_spec.md`
+- `spec-files/outstanding/agentic_orchestrator_feature_focus_view_ux_redesign_spec.md`
+
+---
+
+## 0. Scope
+
+This spec addresses the specific live defect class:
+
+1. `REQUEST.action=more_context` is currently accepted as progress even though it only triggers a local `feature.get_context` refresh.
+2. Builder and QA turns can therefore complete with no file edits, no human handoff, and no durable question state.
+3. Operators have no explicit dashboard path to see, answer, or clear those blocking questions.
+
+This spec defines:
+
+- strict semantics for `REQUEST.action=more_context`
+- explicit split between repo-context refresh and human-input request flows
+- deterministic retry / pause behavior after a request
+- dashboard-native pending-question surfaces and answer submission
+- regression coverage for the current no-op builder loop defect
+
+Out of scope:
+
+- general-purpose chat between operators and workers
+- external ticketing approval workflows
+- non-dashboard UI surfaces beyond minimal parity CLI hooks
+
+### 0.1 Required Standards
+
+All implementation derived from this spec MUST preserve:
+
+- deterministic orchestration behavior and feature state transitions
+- atomic state mutation and append-only artifact semantics under `.aop/**`
+- MCP/in-process tool parity
+- role-based authorization and `operation_id` idempotency for mutating actions
+- interactive-only worker execution contract
+- dashboard use of orchestrator service/tool seams rather than direct state writes
+
+### 0.2 Required Inputs
+
+Implementing agents MUST read:
+
+- `apps/control-plane/src/supervisor/worker-decision-loop.ts`
+- `apps/control-plane/src/supervisor/build-wave-executor.ts`
+- `apps/control-plane/src/supervisor/qa-wave-executor.ts`
+- `apps/control-plane/src/supervisor/planning-wave-executor.ts`
+- `apps/control-plane/src/supervisor/run-coordinator.ts`
+- `apps/control-plane/src/core/constants.ts`
+- `apps/control-plane/src/core/error-codes.ts`
+- `apps/control-plane/src/application/services/execution-control-service.ts`
+- `packages/web-dashboard/src/app/page.tsx`
+- `packages/web-dashboard/src/app/feature/[id]/page.tsx`
+- `packages/web-dashboard/src/components/dashboard/triage-drawer.tsx`
+- `packages/web-dashboard/src/lib/types.ts`
+- `agentic/orchestrator/tools/catalog.json`
+- `agentic/orchestrator/schemas/state.schema.json`
+- `config/agentic/orchestrator/prompts/builder.system.md`
+- `config/agentic/orchestrator/prompts/qa.system.md`
+
+### 0.3 OOP and Architecture Guardrails (Mandatory)
+
+Implementation for this spec MUST follow:
+
+- **KISS**: do not treat every `REQUEST` as a new subsystem; split only into context refresh, human input, and plan amendment.
+- **SRP**: request classification, question persistence, wave pause/resume behavior, and dashboard rendering MUST remain separate responsibilities.
+- **SOLID-oriented design**:
+  - open/closed for future request types without rewriting wave executors
+  - interface segregation between question store/query ports and dashboard adapters
+  - dependency inversion so dashboard and CLI depend on question/runtime contracts, not file storage details
+- **Hexagonal alignment**: worker-output parsing and supervisor policy stay in control-plane code; dashboard uses API/tool contracts only.
+- **Composition over inheritance**: compose request classifiers, question services, and dashboard presenters from small focused modules.
+- **High cohesion / low coupling**: `more_context` repair must not be buried inside unrelated gate, merge, or review flows.
+
+---
+
+## 1. Problem Statement
+
+Today the supervisor recognizes `REQUEST.action=more_context`, but the implementation is incomplete:
+
+1. `WorkerDecisionLoop.routeRequest()` calls `feature.get_context`.
+2. It marks `requestHandled = true`.
+3. That counts as progress for the turn.
+4. The worker is **not** immediately given another turn with the refreshed context.
+5. No question artifact is created and no user input is requested.
+
+Result:
+
+- builder and QA turns can finish with zero file edits and still avoid normal no-progress handling for that iteration
+- repeated `REQUEST more_context` turns can create silent no-op loops
+- operators cannot tell whether a feature needs repo context, plan changes, or human clarification
+- dashboard users have no structured way to answer the question even when the worker truly needs a human decision
+
+This is a defect, not merely a missing enhancement.
+
+---
+
+## 2. Design Goals
+
+### 2.1 Must-Have Outcomes
+
+1. `REQUEST.action=more_context` is no longer treated as “handled” unless the supervisor actually performs a same-iteration recovery step.
+2. The system distinguishes between:
+   - repo-context refresh
+   - plan amendment request
+   - human-input request
+3. Human-input requests create durable question artifacts and move the feature into an explicit paused state.
+4. The dashboard exposes pending questions, answer controls, and resume state clearly.
+5. Builder/QA no-op loops caused by repeated `more_context` requests become observable and blockable.
+
+### 2.2 Non-Goals
+
+- allowing workers to free-form message humans without schema
+- replacing `aop send` with a conversational UI
+- making dashboard real-time transport mandatory for correctness
+
+---
+
+## 3. Normative Behavioral Changes
+
+### 3.1 `REQUEST.action=more_context` Must Be Reclassified
+
+Normative rule:
+
+- `more_context` MUST mean “re-run this worker with refreshed repo/runtime context once.”
+
+It MUST NOT mean:
+
+- “the turn succeeded and should simply continue later”
+- “the worker requires human input”
+
+Required behavior:
+
+1. Supervisor fetches fresh context.
+2. Supervisor re-invokes the same worker once, in the same phase/iteration, with the refreshed context included in `last_tool_results` or equivalent refreshed bundle.
+3. If the second turn still emits `more_context`, that counts as an unresolved blocker, not successful progress.
+
+### 3.2 Human Input Requires a Different Action
+
+Add explicit worker request action:
+
+- `ask_user_input`
+
+Normative rule:
+
+- human clarification, approval, missing business choice, or policy exception MUST use `ask_user_input`, not `more_context`
+
+Examples:
+
+- approve editing outside allowed paths
+- choose one of several implementation tradeoffs
+- provide a missing environment or product decision
+- resolve contradictory spec requirements
+
+### 3.3 Repeated `more_context` Must Become a Deterministic Stall Class
+
+If a worker emits `more_context` twice in the same phase attempt with no material worktree diff, the supervisor MUST:
+
+- classify the attempt as unresolved context stall
+- record a machine-readable status reason
+- either:
+  - block the feature with a dedicated error code, or
+  - convert to `awaiting_input` if the worker’s second-turn payload escalates to `ask_user_input`
+
+It MUST NOT silently continue as ordinary progress.
+
+---
+
+## 4. Request Taxonomy
+
+### 4.1 Supported Worker Request Actions
+
+The worker runtime MUST support the following semantics:
+
+1. `context_refresh`
+   - immediate same-turn refresh and retry
+   - non-human
+2. `more_context`
+   - alias of `context_refresh` for backward compatibility in v1
+   - same-turn refresh and retry
+3. `amend_plan`
+   - planner-only accepted plan mutation
+4. `ask_user_input`
+   - durable blocking question
+5. `lock_acquire`
+6. `lock_release`
+7. `prioritize`
+
+### 4.2 Backward Compatibility Rule
+
+For backward compatibility, workers may still emit `more_context`.
+
+However:
+
+- prompts MUST be updated to prefer `ask_user_input` for human needs
+- repeated `more_context` with no diff MUST no longer count as successful progress
+
+---
+
+## 5. State and Artifact Model
+
+### 5.1 Explicit Awaiting-Input State
+
+This spec adopts the explicit paused state introduced by the human-input protocol:
+
+- `awaiting_input`
+
+Rules:
+
+- wave executors MUST skip features in `awaiting_input`
+- paused features retain enough metadata to resume the correct phase deterministically
+
+### 5.2 Durable Question Artifact
+
+Add or reuse:
+
+- `.aop/features/<feature_id>/questions.json`
+
+Each open question MUST include:
+
+- `question_id`
+- `status`
+- `blocking`
+- `role`
+- `phase`
+- `request_action`
+- `prompt`
+- `details`
+- `expected_answer`
+- `created_at`
+- `expires_at`
+- `resume_status`
+- `resume_phase`
+- `answer`
+- `answered_at`
+- `answered_by`
+
+### 5.3 Context-Stall Tracking
+
+Add supervisor-visible metadata for repeated refresh loops:
+
+- `context_request_count`
+- `last_context_request_at`
+- `last_context_request_role`
+
+This MAY live in transient supervisor state or persisted frontmatter, but the blocking disposition and reason MUST be durable.
+
+---
+
+## 6. Supervisor and Runtime Changes
+
+### 6.1 WorkerDecisionLoop
+
+Required changes:
+
+1. When `more_context` or `context_refresh` is received:
+   - fetch fresh context
+   - re-run the worker once immediately
+   - merge the second-turn result into the phase decision
+2. If the second turn again requests context and no diff exists:
+   - mark unresolved context stall
+3. `requestHandled` MUST NOT be set to true solely because a refresh was fetched; it is true only if the refresh-retry path completed successfully.
+
+### 6.2 Build and QA Wave Executors
+
+Required changes:
+
+1. Builders and QA workers that end in unresolved context stall must be treated as no-progress/blocking policy inputs.
+2. Executors MUST preserve current malformed-output and checkpoint validation rules.
+3. `ask_user_input` requests MUST transition the feature to `awaiting_input` instead of remaining in `building` or `qa`.
+
+### 6.3 Run Coordinator
+
+Required changes:
+
+1. Features in `awaiting_input` are omitted from planning/build/qa scheduling.
+2. Answering a question re-admits the feature into the correct phase cursor.
+3. The coordinator MUST surface pending-question counts in its execution summaries.
+
+---
+
+## 7. Dashboard Integration (Mandatory)
+
+### 7.1 Root Dashboard Surfaces
+
+The dashboard root MUST display pending human-input questions in a non-intrusive operator surface.
+
+Minimum requirements:
+
+- pending-question badge/count in the root command center
+- triage drawer section for open questions on the selected feature
+- a CTA to open the feature focus view question panel
+
+### 7.2 Feature Focus View
+
+The feature detail/focus route MUST expose a dedicated human-input section when a question is open.
+
+Required contents:
+
+- role that asked the question
+- phase and timestamp
+- prompt text
+- structured details payload rendered readably
+- allowed answer options or answer schema hint
+- answer composer
+- submit action
+
+### 7.3 Dashboard Answer Submission
+
+Add dashboard API support for:
+
+- list pending questions
+- answer question
+- optionally dismiss/expire only if policy permits
+
+Suggested routes:
+
+- `GET /api/runtime/questions`
+- `GET /api/features/[id]/questions`
+- `POST /api/features/[id]/questions/[questionId]/answer`
+
+Normative rule:
+
+- dashboard answer submission MUST call orchestrator tool/service contracts; it MUST NOT write `.aop/**` directly
+
+### 7.4 Dashboard Resume Feedback
+
+After a successful answer:
+
+- the UI MUST show that the question is closed
+- the feature status MUST update from `awaiting_input` to its resume phase/status
+- the page MUST surface that the feature is queued to resume execution
+
+### 7.5 Empty-Diff Context Loops in the Dashboard
+
+If a feature is blocked for repeated unresolved context refreshes, the dashboard MUST show:
+
+- explicit reason: repeated context-refresh with no progress
+- last role/phase that requested context
+- suggested operator actions:
+  - answer human-input question if present
+  - reopen planning / retry
+  - inspect raw worker output
+
+---
+
+## 8. Prompt Contract Changes
+
+### 8.1 Builder and QA Prompts
+
+Builder and QA prompts MUST explicitly distinguish:
+
+- use `more_context` / `context_refresh` only when refreshed repo/runtime state may immediately unblock the work
+- use `ask_user_input` when a human decision or clarification is required
+
+They MUST say:
+
+- repeated context-refresh without file progress is treated as a stall
+- `NOTE` is valid only after real planned file edits
+
+### 8.2 Planner Prompt
+
+Planner prompt MUST continue preferring:
+
+- `PLAN_SUBMISSION`
+- `REQUEST.action=amend_plan`
+
+But if the planner needs a human business/product choice, it MUST be allowed to emit `ask_user_input`.
+
+---
+
+## 9. Error Codes and Policy
+
+Add dedicated error/disposition support for this defect class:
+
+- `provider_context_stall`
+- `question_awaiting_input`
+- `question_answer_invalid`
+- `question_already_closed`
+
+Policy requirements:
+
+- repeated unresolved context-refresh loops MUST be independently configurable from generic `provider_no_progress`
+- question timeout / expiry behavior MUST remain policy-driven
+
+---
+
+## 10. Acceptance Criteria
+
+1. A worker emitting `REQUEST.action=more_context` triggers an immediate same-phase context refresh and one retry in the same orchestration turn.
+2. A second `more_context` response with no material diff is not counted as successful progress.
+3. Workers can emit `REQUEST.action=ask_user_input` with structured question metadata.
+4. The kernel/runtime persists open questions durably and moves the feature into `awaiting_input`.
+5. Planning/build/qa waves skip `awaiting_input` features until an answer is submitted.
+6. The dashboard root and feature focus view both surface open questions and allow answers.
+7. Answering a question through the dashboard uses orchestrator APIs/tools and resumes the feature deterministically.
+8. The dashboard shows explicit reasoning for blocked repeated-context stalls.
+9. Regression tests prove the current defect no longer occurs:
+   - `more_context` alone does not count as progress unless the refresh-retry path succeeds
+   - repeated `more_context` with empty diffs blocks or stalls deterministically
+   - `ask_user_input` transitions to `awaiting_input`
+   - dashboard answer submission closes the question and re-admits the feature
+
+---
+
+## 11. Implementation Slices
+
+### Slice M52.1
+
+Refactor `WorkerDecisionLoop` so `more_context` performs same-turn refresh-and-retry instead of a bare handled request.
+
+### Slice M52.2
+
+Add question lifecycle tools/artifacts/status support by extending the human-input protocol implementation.
+
+### Slice M52.3
+
+Update build/QA/planner prompts and request classification rules.
+
+### Slice M52.4
+
+Expose dashboard question list + answer APIs and root/focus view question UI.
+
+### Slice M52.5
+
+Add regressions for context-stall classification, awaiting-input transitions, and dashboard answer-driven resume.
+
+---
+
+## 12. Risks
+
+1. Treating every `more_context` as human input would over-block execution and reduce autonomy; this spec explicitly forbids that.
+2. Treating every `more_context` as handled progress recreates the current silent no-op loop; this spec explicitly forbids that.
+3. Same-turn refresh-and-retry could create recursion or runaway loops if not capped at one retry.
+4. Dashboard answer flows can drift from CLI/service semantics if they bypass orchestrator tools.
+5. Question rendering could expose overly raw JSON unless the dashboard includes a structured details presenter.
+
+---
+
+## 13. Success Metric
+
+This spec is successful when:
+
+- builders and QA workers no longer silently “succeed” after `more_context` without either making changes, amending the plan, or producing a durable human-input question
+- operators can resolve blocking questions directly in the dashboard
+- blocked/no-progress features clearly show whether the issue is malformed provider output, repeated context stall, or pending human input