stageflow 0.8.0 → 0.12.0

package/dist/ui/index.html

@@ -13,8 +13,8 @@
     <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Figtree:wght@400;500;600;700&display=swap" />
     <link rel="icon" type="image/svg+xml" href="/stageflow-icon.svg" />
     <title>Stageflow</title>
-    <script type="module" crossorigin src="/assets/index-CSoGDA3A.js"></script>
-    <link rel="stylesheet" crossorigin href="/assets/index-C3N7MXAC.css">
+    <script type="module" crossorigin src="/assets/index-CXGz4nL7.js"></script>
+    <link rel="stylesheet" crossorigin href="/assets/index-uVHiae8s.css">
   </head>
   <body>
     <div id="root"></div>

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "stageflow",
-  "version": "0.8.0",
+  "version": "0.12.0",
   "description": "Stageflow — CLI pipeline runtime for configurable stages on Pi",
   "type": "module",
   "bin": {
@@ -60,6 +60,7 @@
     "vitest": "^4.1.10"
   },
   "dependencies": {
+    "@anthropic-ai/claude-agent-sdk": "^0.3.263",
     "@earendil-works/pi-ai": "^0.84.1",
     "@earendil-works/pi-coding-agent": "^0.84.1",
     "@modelcontextprotocol/node": "^2.0.0",

package/skills/stageflow/references/control-surface.md

@@ -11,7 +11,7 @@ node ../scripts/detect-host.mjs
 node ../scripts/detect-host.mjs --base-url http://127.0.0.1:3847
 ```
 
-The script `GET`s `{baseUrl}/api/health` (default `http://127.0.0.1:3847`, 1500 ms timeout). **up** means HTTP 200 and parseable JSON. Non-200, non-JSON, or timeout is **down**.
+The script `GET`s `{baseUrl}/api/health` (default `http://127.0.0.1:3847`, 1500 ms timeout). **up** means HTTP 200 and parseable JSON. Non-200, non-JSON, or timeout is **down**. This probe is Stageflow host up/down only — it does not detect a coding-agent question UI. Gate presentation lives in [`../../stageflow-run/references/native-question-ui.md`](../../stageflow-run/references/native-question-ui.md).
 
 Stdout is one line: `up <baseUrl>` or `down <baseUrl>`. Exit `0` when up, `1` when down, `2` on usage error.
 
@@ -23,4 +23,6 @@ Use MCP tools over the Streamable HTTP endpoint at `{baseUrl}/mcp`. Tool names a
 
 ## When the host is down
 
-Use the `sf` CLI. Command names and flags live in [docs/cli-reference.md](../../../docs/cli-reference.md). Typical talking-job commands: `sf run`, `sf validate`, `sf envelope get`, `sf artifact read`, `sf providers`.
+Use the `sf` CLI. Command names and flags live in [docs/cli-reference.md](../../../docs/cli-reference.md). Typical talking-job commands: `sf run`, `sf runs waiting`, `sf runs answer`, `sf runs wait`, `sf validate`, `sf envelope get`, `sf artifact read`, `sf providers`.
+
+Probe before each mutating `sf runs` verb. If the probe is **up** or the command refuses because a host is up, continue that gate via MCP — do not start a second mutating writer, and do not start `sf mcp` as a disposable bridge.

package/skills/stageflow-author/references/catalog-mapping.md

@@ -16,18 +16,19 @@ Collision handling lives in [`catalog-write-conventions.md`](catalog-write-conve
 
 ## Sequencing
 
-`needs` is a single parent stage id. Fan-in (one stage needing two parents) is out of catalog shape.
+`needs` is a single parent stage id, or an array of at least two parent ids when one step waits for several earlier steps. Strings default to `on: [succeeded]`. `{ id, on }` accepts a non-empty unique subset of `succeeded` | `failed` | `skipped`.
 
 | Human says | Pipeline shape |
 |---|---|
 | Steps in order | Each later stage `needs` the previous id. No `fork` field. |
 | Steps happen together | Sibling stages share one `needs` (the same parent). No `fork` field. |
+| A later step waits for two or more earlier steps | That stage `needs` an array of those parent ids (length ≥ 2). |
 | Exactly one branch runs | Deciding stage gets `fork: { select: one }`. Each branch `needs` the decider. |
 | Either, both, or a subset may run | Deciding stage gets `fork: { select: subset }`. Each branch `needs` the decider. |
 
 Default `select` is `one` unless the human says more than one branch can run.
 
-Map linear chains, sibling fan-out, and single-level `fork`. A runtime clone count is not knowable from a static description — stay on those three shapes. Leave `clonable` and `clone_forks` unset.
+Map linear chains, sibling fan-out, generic fan-in (`needs` array), and single-level `fork`. A runtime clone count is not knowable from a static description — leave `clonable` and `clone_forks` unset. Clone-list joins (one catalog parent, `priorEnvelopes`) are not this shape.
 
 A review, approval, or sign-off step is a gated stage: put `gate_kinds` on that stage file and follow [`stage-prompt-template.md`](stage-prompt-template.md).
 
@@ -110,6 +111,29 @@ stages:
 
 No `fork` field. Both siblings run.
 
+### Diamond join
+
+"After research and validation both finish, synthesize":
+
+```yaml
+stages:
+  - id: clarify
+    uses: ./clarify.yaml
+  - id: research
+    uses: ./research.yaml
+    needs: clarify
+  - id: validation
+    uses: ./validation.yaml
+    needs: clarify
+  - id: synthesize
+    uses: ./synthesize.yaml
+    needs:
+      - research
+      - validation
+```
+
+`needs` array length is ≥ 2. The join reads `priorEnvelopesByStage`, not clone-list `priorEnvelopes`.
+
 ### Fork, select subset
 
 "Email, post, or both could go out":

package/skills/stageflow-author/references/catalog-write-conventions.md

@@ -20,7 +20,7 @@ One directory (or the project root) holds:
 | `<pipeline-id>.pipeline.yaml` | `id` matching the filename stem; `stages:` object entries |
 | `<stage-id>.yaml` | `id`, `system_prompt`, `model`; `gate_kinds` when the step is gated |
 
-Each pipeline stage entry has `id` and `uses: ./<id>.yaml` (path relative to the pipeline file). Non-root stages add `needs: <parent-id>`. A deciding stage adds `fork:`. Filename stem matches `id` on every file.
+Each pipeline stage entry has `id` and `uses: ./<id>.yaml` (path relative to the pipeline file). Non-root stages add `needs: <parent-id>`, or `needs:` as an array of at least two parent ids (strings or `{ id, on }`) when the stage waits for several parents. A deciding stage adds `fork:`. Filename stem matches `id` on every file.
 
 ## Collisions
 

package/skills/stageflow-run/SKILL.md

@@ -2,19 +2,20 @@
 name: stageflow-run
 description: >-
   Starts a catalog pipeline from the harness, reports progress, and answers
-  every HITL gate in this chat. Triggers: run a pipeline, start <pipeline>,
-  check on my run, answer the pending question.
+  every HITL gate on the host native question UI when it can, otherwise in
+  this chat. Triggers: run a pipeline, start <pipeline>, check on my run,
+  answer the pending question.
 compatibility: Requires the sf CLI on PATH. An MCP host (sf ui or sf mcp) is optional.
 disable-model-invocation: true
 ---
 
 # Stageflow run
 
-Start a catalog pipeline, keep every HITL gate in this chat, and report the outcome. Author and session-capture own catalog YAML; this job owns the run and any throwaway `*.task.yaml`.
+Start a catalog pipeline, present HITL on the host native question UI when it can, and report the outcome. Author and session-capture own catalog YAML; this job owns the run and any throwaway `*.task.yaml`.
 
 Talking jobs cite [`../stageflow/references/control-surface.md`](../stageflow/references/control-surface.md). Probe with [`../stageflow/scripts/detect-host.mjs`](../stageflow/scripts/detect-host.mjs) only. Do not write a second probe.
 
-MCP tool shapes: [`docs/mcp.md`](../../docs/mcp.md). CLI flags and exit codes: [`docs/cli-reference.md`](../../docs/cli-reference.md). Direct tool calls: [`references/mcp-call.md`](references/mcp-call.md). Selection: [`references/task-and-pipeline-selection.md`](references/task-and-pipeline-selection.md).
+MCP tool shapes: [`docs/mcp.md`](../../docs/mcp.md). CLI flags and exit codes: [`docs/cli-reference.md`](../../docs/cli-reference.md). Direct tool calls: [`references/mcp-call.md`](references/mcp-call.md). Selection: [`references/task-and-pipeline-selection.md`](references/task-and-pipeline-selection.md). Gate presentation: [`references/native-question-ui.md`](references/native-question-ui.md).
 
 ## Preconditions
 
@@ -38,15 +39,15 @@ node ../stageflow/scripts/detect-host.mjs --base-url http://127.0.0.1:3847
 
 Read [`references/task-and-pipeline-selection.md`](references/task-and-pipeline-selection.md). Follow it until you have a pipeline filesystem path and a task (catalog path, MCP inline object, or CLI throwaway file).
 
-If this chat already has a `runId` and the request is check, answer, or continue: skip start. Host up → [Wait](#wait). Host down and the human wants to answer → [Bridge](#bridge).
+If this chat already has a `runId` and the request is check, answer, or continue: skip start. Host up → [Wait](#wait). Host down → [CLI wait](#cli-wait).
 
 **Done when** the target is named, or a live `runId` is in hand.
 
 ## MCP tools
 
-Prefer this harness's native Stageflow MCP tools when their names are already in the tool list. When they are not, call [`scripts/mcp-call.mjs`](scripts/mcp-call.mjs) — see [`references/mcp-call.md`](references/mcp-call.md). Always use `mcp-call.mjs --stateless` for a bridge this skill started.
+Prefer this harness's native Stageflow MCP tools when their names are already in the tool list. When they are not, call [`scripts/mcp-call.mjs`](scripts/mcp-call.mjs) — see [`references/mcp-call.md`](references/mcp-call.md). Use `--stateless` only when the **user** started the host with `--mcp-stateless`. Do not start `sf mcp` from this skill.
 
-Call only `list_pipelines`, `list_tasks`, `start_run`, `get_run`, `wait_run`, `list_waiting`, `answer_gate`, `get_health`.
+Call only these Stageflow MCP tools: `list_pipelines`, `list_tasks`, `start_run`, `get_run`, `wait_run`, `list_waiting`, `answer_gate`, `get_health`. Host question tools already in this harness's tool list (`AskQuestion`, `AskUserQuestion`, `ask_user`) are for [Gate](#gate) presentation, not Stageflow MCP.
 
 ## MCP path
 
@@ -78,9 +79,14 @@ Call only `list_pipelines`, `list_tasks`, `start_run`, `get_run`, `wait_run`, `l
 
 ### Gate
 
+Read [`references/native-question-ui.md`](references/native-question-ui.md) before presenting a pending prompt.
+
 1. `list_waiting` with `{ "runId" }`.
 2. Print the pending prompt text **verbatim** (and artifacts / sub-questions when present).
-3. Collect the human's reply in this chat. Map it to the prompt `kind`:
+3. Present the decision as that reference directs:
+   - If a host question tool is already in this harness's tool list and the gate is representable, invoke that picker. For `multi_question`, one picker call with one question per sub-item when **every** sub-question is representable, then one `answer_gate`; otherwise the **whole** gate in this chat.
+   - Otherwise collect the reply in this chat.
+4. Map the reply to the prompt `kind` (picker Accept/Reject → `accept`/`reject`):
 
 | kind | `answer` |
 |---|---|
@@ -89,12 +95,12 @@ Call only `list_pipelines`, `list_tasks`, `start_run`, `get_run`, `wait_run`, `l
 | `artifact_backed` | `{ "promptId", "kind": "artifact_backed", "decision": "accept" \| "reject" }` |
 | `multi_question` | `{ "promptId", "kind": "multi_question", "answers": { "<id>": { "kind", "text" \| "decision" } } }` |
 
-`promptId` is `pending_prompt.id` or `waiting_prompt_id`. Map yes/y/accept/approve to `accept`; no/n/reject/deny to `reject`. Ask once when the kind is `multi_question` and any sub-question is unanswered, or when confirm/artifact_backed is not a clear decision.
+`promptId` is `pending_prompt.id` or `waiting_prompt_id`. Map yes/y/accept/approve/Accept to `accept`; no/n/reject/deny/Reject to `reject`. Each `multi_question` sub-answer keeps that sub's `kind` (`confirm` → `decision`, `free_text` → `text`). Collect every sub-answer before `answer_gate`.
 
-4. `answer_gate` with `{ "runId", "stageId", "answer" }`.
-5. Return to [Wait](#wait).
+5. `answer_gate` with `{ "runId", "stageId", "answer" }`.
+6. Return to [Wait](#wait).
 
-**Done when** `answer_gate` returns `{ "ok": true }` and Wait is re-entered. On `isError` (400 / 404 / 409), print the payload and ask for a corrected reply — do not open another surface.
+**Done when** `answer_gate` returns `{ "ok": true }` and Wait is re-entered. On `isError` (400 / 404 / 409), print the payload and collect a corrected reply the same way (picker if still representable; otherwise this chat).
 
 ## CLI path
 
@@ -113,24 +119,49 @@ Parse the single JSON document.
 | `0` | `succeeded` | [Report](#report) |
 | `1` | `failed` | [Report](#report) `reason` verbatim |
 | `1` | `busy` | [Report](#report) `busy_capacity` / `busy_checkout` and the included fields. Stop. |
-| `2` | `waiting` | [Bridge](#bridge) with this `runId` / `runDir` |
+| `2` | `waiting` | [CLI wait](#cli-wait) with this `runId` / `runDir` |
+
+**Done when** the outcome is reported, or a waiting exit has handed `runId` to CLI wait.
+
+## CLI wait
+
+Host down after `sf run` exit `2`, or when this chat already has a `runId` and the host is down. Do not start `sf mcp`. Presentation follows [`references/native-question-ui.md`](references/native-question-ui.md). Submit with `sf runs answer --json`.
+
+1. Probe again with [`../stageflow/scripts/detect-host.mjs`](../stageflow/scripts/detect-host.mjs) (the host may have appeared). Probe before each `sf runs answer`.
+2. **Up:** use that host — native tools if present, otherwise `mcp-call.mjs` (omit `--stateless` unless the user started the host with `--mcp-stateless`). Continue at [Gate](#gate).
+3. **Down:**
 
-**Done when** the outcome is reported, or a waiting exit has handed `runId` and `runDir` to Bridge.
+```
+sf runs waiting --run <runId> --json
+```
+
+Print the pending prompt **verbatim**. Present it as native-question-ui directs (picker or chat). Map the reply with the [Gate](#gate) kind table.
 
-## Bridge
+```
+sf runs answer --run <runId> --stage <stageId> --answer '<json>' --json
+```
+
+Success is `{ "ok": true }` exit `0` even if the run parks again. Do not treat answer as terminal.
+
+```
+sf runs wait --run <runId> --json --until any
+```
+
+A shorter `--timeout-ms` is fine when the harness timeout is tight. Default `60000`, max `240000`. Branch on JSON `reason`, not wait exit `0`:
+
+| `reason` | next |
+|---|---|
+| `waiting` or `already` with a waiting snapshot | `sf runs waiting` then answer |
+| `terminal` | [Report](#report) |
+| `timeout` | call `sf runs wait` again |
 
-A CLI waiting exit has no answer command. Keep the gate in this chat by joining a host on the same project root (git top-level when `git rev-parse --show-toplevel` succeeds, otherwise the current directory).
+Wait abort (exit `130`, `{ "error", "code": "aborted" }`) → [Report](#report): the run continues, resumable later. Stop. Do not reuse `sf run` exit `2` for wait.
 
-1. Probe again with [`../stageflow/scripts/detect-host.mjs`](../stageflow/scripts/detect-host.mjs) (the host may have appeared).
-2. **Up:** use that host — native tools if present, otherwise `mcp-call.mjs` (omit `--stateless`). Continue at [Gate](#gate).
-3. **Down:** start `sf mcp --mcp-stateless` as a background process on that same project root. This is a disposable HITL bridge, not setup auto-start.
-4. Probe until stdout is `up <baseUrl>`, at most 8 times. **Done when** the host is up. If it stays down, [Report](#report) the parked `runId` / `runDir` and the bind/stderr text. Stop.
-5. Drive the rest with `mcp-call.mjs --stateless` only: [Gate](#gate) → [Wait](#wait) until terminal. A second wait on the same run reuses this process.
-6. When the run is terminal, stop the bridge process.
+4. If `sf runs answer` refuses because the host came up, continue that gate via MCP [Gate](#gate). Do not dual-write.
 
 If this chat ends before terminal, say the run stays in `.stageflow/` and to continue by re-invoking this skill (or starting a host and using MCP).
 
-**Done when** the run is terminal and the bridge this skill started is stopped, or a parked-run failure is reported.
+**Done when** the run is terminal, aborted, or a hard error is printed.
 
 ## Report
 
@@ -142,10 +173,10 @@ Print one shape on every path:
 4. Run id.
 5. Run folder (`runDir` / `.stageflow/`).
 
-MCP `get_run` / `wait_run` carry live stage state. `sf run --json` reports start and finish only — say that in chat when the CLI path ran without a host. A succeeded report names id and folder and does not keep waiting language. A failure names `reason` in this same shape.
+MCP `get_run` / `wait_run` and `sf runs wait` / `sf runs show` `--json` carry live stage state. `sf run --json` reports start and finish only — say that in chat when the CLI path ran without a later wait/show. A succeeded report names id and folder and does not keep waiting language. A failure names `reason` in this same shape.
 
 **Done when** that five-part report is printed.
 
 ## Non-goals
 
-This job starts, watches, and answers runs. It does not require `sf ui`. The operator console is not the answer path. It does not register or invent an MCP tool.
+This job starts, watches, and answers runs. It does not require `sf ui`. The operator console is not the answer path. It does not register or invent an MCP tool. It does not start `sf mcp`. It does not retry, abandon, or rerun stages.

package/skills/stageflow-run/references/mcp-call.md

@@ -1,6 +1,6 @@
 # mcp-call
 
-Use [`../scripts/mcp-call.mjs`](../scripts/mcp-call.mjs) when this harness has no native Stageflow MCP tools, and always for a self-started `sf mcp --mcp-stateless` bridge.
+Use [`../scripts/mcp-call.mjs`](../scripts/mcp-call.mjs) when the Stageflow host is up and this harness has no native Stageflow MCP tools. Do not start `sf mcp` from this skill.
 
 ```
 node scripts/mcp-call.mjs --base-url <url> --tool <name> --args '<json>' [--stateless]
@@ -8,7 +8,7 @@ node scripts/mcp-call.mjs --base-url <url> --tool <name> --args '<json>' [--stat
 
 Default `--base-url` is `http://127.0.0.1:3847`. `--args` defaults to `{}`.
 
-`--stateless` sends one `tools/call` and no session header. Use it for hosts started with `sf mcp --mcp-stateless`.
+`--stateless` sends one `tools/call` and no session header. Use it for hosts the **user** started with `sf mcp --mcp-stateless`, not a skill-started process.
 
 Without `--stateless`: `initialize`, capture `Mcp-Session-Id`, reuse it for `tools/call`.
 

package/skills/stageflow-run/references/native-question-ui.md

@@ -0,0 +1,78 @@
+# Native question UI
+
+Present a waiting HITL gate on the coding-agent host's structured question tool when that tool can represent the gate. Otherwise collect the reply in this chat. Submit with `answer_gate` when the Stageflow host is up, or `sf runs answer --json` when it is down. This file is presentation only — it does not change MCP tools, `detect-host.mjs`, or the operator console.
+
+Read `pending_prompt` from `list_waiting` or `sf runs waiting --json` (not only `waiting_questions` strings).
+
+## Detect a picker
+
+Inspect **this harness's current tool list**. A picker is a tool that collects a structured choice (question text plus labeled options), not a free-form chat reply.
+
+Do not run `detect-host.mjs` for this. That script is Stageflow host up/down (MCP vs CLI). It does not detect a question UI.
+
+| If this name is in the tool list | Use it |
+|---|---|
+| `AskQuestion` | Cursor question cards |
+| `AskUserQuestion` | Claude Code questions |
+| `ask_user` | Pi questions |
+
+If none of those (or another options-based question tool already in the list) is present, collect in chat. Do not invent a tool name or call shape. Codex and OpenCode stay in chat until an options-based question tool appears in the list.
+
+Pass the pending message as the question text and the mapped labels as that tool's options. One question per call except `multi_question`: one call with one question per representable sub-item when the tool accepts a questions array (Cursor `AskQuestion` does). Do not invent placeholder options such as Type in Other — host Other is the custom-text escape. If the picker returns a custom or Other value, map that text the same as a chat reply.
+
+**Done when** you know picker or chat for this gate.
+
+## Representable vs chat
+
+| Gate | Picker | Chat |
+|---|---|---|
+| `confirm` | Accept and Reject | Host has no picker |
+| `artifact_backed` | Accept and Reject (decision only) | Host has no picker |
+| `free_text` with a harvested closed set | Those names as options | Open-ended, or harvest is ambiguous |
+| `free_text` with no closed set | — | Always |
+| `multi_question` | One picker call with one question per sub-item when **every** sub-question is representable; then one submit | Any sub-question is open or unmappable — the **whole** gate in chat |
+
+Map picker Accept / yes / approve → `accept`. Reject / no / deny → `reject`. Always include `promptId` (`pending_prompt.id` or `waiting_prompt_id`).
+
+| Item | Submit |
+|---|---|
+| Top-level or sub `confirm` | `{ "kind": "confirm", "decision": "accept" \| "reject" }` |
+| Top-level `artifact_backed` | `{ "kind": "artifact_backed", "decision": "accept" \| "reject" }` plus optional `text` after a reject follow-up |
+| Top-level or sub `free_text` (including harvested) | `{ "kind": "free_text", "text": "<selected label>" }` |
+
+Never submit a `confirm` item as `free_text`. For `multi_question`, wrap sub-answers in the Gate `answers` object and submit once (`answer_gate` or `sf runs answer --json`).
+
+When in doubt, chat.
+
+**Done when** the path (picker or chat) is chosen from this table.
+
+## Harvest
+
+Harvest only from `pending_prompt.message` (or a `multi_question` item `message`) when it names a **hard closed set**:
+
+- `reply with exactly one of:`
+- `choose one of:`
+- `exactly one of`
+
+Split the names after that directive (commas, `or`, newlines). Those literal names are the picker options.
+
+Do **not** harvest loose example lists (`you might try`, `e.g.`, `for example` without `exactly one of`). A listed example set that is not a hard closed directive stays in chat.
+
+Submit the selected label as `text` for `free_text` items. If the prompt also lists aliases for the same choice (for example `minimal` and `prototype~1`), keep every listed name on the card; do not rewrite the label at submit — the stage maps synonyms.
+
+**Done when** the option list is the prompt's literal closed set, or the gate stayed in chat.
+
+## Artifact-backed
+
+1. Print the pending message and artifact paths (from `list_waiting`) in this chat. Do not put artifact bodies on the card. Do not call `read_artifact` unless the human asked to open a file.
+2. Open the picker with Accept and Reject.
+3. Accept → submit immediately (`decision: "accept"`).
+4. Reject → one chat follow-up for optional notes, then submit `decision: "reject"` and `text` when they typed notes.
+
+**Done when** the operator has seen the paths in chat and the decision is on the picker (or chat if there is no picker).
+
+## Errors
+
+On MCP `answer_gate` `isError` (400 / 404 / 409) or CLI `sf runs answer` exit `1`, print the payload. Collect a corrected reply on the picker if the gate is still representable; otherwise chat. Then submit again.
+
+**Done when** `{ "ok": true }` or the error payload has been printed and a corrected reply is in hand.