stageflow 0.25.0 → 0.27.0

package/examples/feature-loop/README.md

@@ -0,0 +1,104 @@
+# feature-loop
+
+Epic-to-PR walkthrough: split a requirements document into stories,
+plan them in parallel, align the plans, implement them **sequentially**,
+verify nothing was missed, review the whole diff, fix review blockers
+(with a send-back loop), then open an operator-approved pull request.
+
+This is the shape for work that starts as one epic and must not land
+as overlapping parallel edits. It is repository-neutral — point the
+task `checkout` at a feature branch of whatever tree you are shipping.
+Nothing in the stage files names a product, repo, or keyword.
+
+The shipped YAML is two sealed [Clone Chains](../../docs/yaml-catalog.md#clone-chain)
+plus a `{ type: loop }` from `address-feedback` back to `review`. Wiring
+is outbound `route` (`entry`, `clone_cap` / `clone_mode` on emitters).
+`sf validate --strict` against this pipeline is expected to pass.
+
+## Flow
+
+```
+decompose ──Clone Array (parallel)──► plan~N ──Join──► align
+                                           │
+                    sequential Clone Array │
+                                           ▼
+                              implement~1 → implement~2 → …
+                                           │
+                                           ▼
+                         verify → review ⇄ address-feedback → publish
+```
+
+1. **decompose** — Read the epic (task + requirements doc). Cut 1–8
+   stories into a Clone Array (`stories` items `$ref` `story-assignment`).
+   Writes `epic-split.md`. Emitter: `clone_cap: 8`, `clone_mode: parallel`.
+2. **plan** — One Clone Instance per story (`clone_mode: parallel`). Each
+   writes `story-plan.md` (edit sites, validation commands, neighbor
+   boundaries). No source edits. Entire `io.input.schema` is
+   `{ $ref: "#/schemas/story-assignment" }`.
+3. **align** — Join of every plan Clone Instance. Fix overlapping files and
+   contradictions, choose **implementation order**, ask the operator to
+   accept. Then this stage is the emitter of the next Clone Chain
+   (`clone_mode: sequential`): `stories` items `$ref` `story-work`, with
+   `order` / `depends_on` on each item.
+4. **implement** — Sequential Clone Instances in array order; `~N`
+   starts only after `~N-1` succeeded, so earlier edits are already in
+   the tree. Each instance executes only its story and must change the
+   checkout. Entire `io.input.schema` is `{ $ref: "#/schemas/story-work" }`.
+5. **verify** — Join of implement Clone Instances. Fail if any aligned
+   `story_id` is missing from the instance set or the union diff. Writes
+   `coverage.md`.
+6. **review** — One whole-diff review of every story together
+   (ce-code-review style). Report-only. `verdict: pass | changes_required`.
+7. **address-feedback** — Fix blockers. Envelope `feedback_loop`
+   `send_back` to `review` when the tree changed so the whole diff is re-read;
+   `continue` when review is clean. Catalog loop:
+   `{ type: loop, to: review, max_replays: 2, on_max_replays: wait_for_human,
+   replay_session: new_session }`. After two send-backs the run waits
+   for an operator decision. `publish` is `replay_safe: false`.
+8. **publish** — Operator accepts `ship-package.md`, then this same
+   attempt commits, pushes the current branch, and opens the PR.
+
+No stage before `publish` commits, pushes, or opens a PR. No stage
+creates or switches branches.
+
+## Runtime contracts
+
+| Stage | Gate | Required artifacts | Other checks |
+|-------|------|--------------------|--------------|
+| `plan` | — | `story-plan.md` | `io.input` assignment |
+| `implement` | — | `implementation-report.md` | `checkout_changes` |
+| `verify` | — | `coverage.md` | `io.output` |
+| `review` | — | `review-report.md` | whole-diff, report-only |
+| `address-feedback` | — | `review-feedback-report.md` | `{ type: loop }` → `review` |
+| `publish` | `artifact_backed` | `ship-package.md`, `pull-request.md` | `on_verify_fail: manual` |
+
+`on_verify_fail` repair (idempotent, 3 attempts) sits on every stage except
+`publish` (`manual` / `side_effecting`). Stage files use `io` / `verify` /
+`on_verify_fail`.
+
+Shared JSON Schema lives on the pipeline under `schemas:` —
+`story-assignment` is the first Clone Array item (`plan` input `$ref`);
+`story-work` is the second (`implement` input `$ref`). See
+[YAML catalog — Pipeline schemas](../../docs/yaml-catalog.md#pipeline-schemas)
+and [Clone Chain](../../docs/yaml-catalog.md#clone-chain).
+
+## Run
+
+Fill in [`feature-loop.task.yaml`](feature-loop.task.yaml) (epic path,
+publish repo, base branch, `checkout` on the feature branch). Operator
+accepts twice: aligned plans, then the ship package.
+
+```bash
+node dist/cli.js validate --pipeline examples/feature-loop/feature-loop.pipeline.yaml --strict
+node dist/cli.js run \
+  --pipeline examples/feature-loop/feature-loop.pipeline.yaml \
+  --task examples/feature-loop/feature-loop.task.yaml
+```
+
+Needs a configured provider (`sf providers status`), `gh` on PATH, and
+`GH_TOKEN` or `GITHUB_TOKEN` for the publish stage. Optional: add
+`mcp: [context7]` on the `plan` pipeline entry when `CONTEXT7_API_KEY`
+is set so story plans can query current library docs.
+
+Inspect Clone Instance tracks and envelopes in the operator console (`sf ui`)
+on the `plan` and `implement` stages, then the `align` / `verify` Joins.

package/examples/feature-loop/address-feedback.yaml

@@ -0,0 +1,148 @@
+id: address-feedback
+system_prompt: |
+  Address every blocking finding from review. This is the only stage
+  after implement that may edit the checkout. It is last-mile fixup,
+  not a second epic. Do not commit, push, or open a pull request.
+
+  On success you MUST include feedback_loop in emit_stage_envelope:
+  - {"action":"continue"} when verdict was pass, or when you fixed
+    every blocker and a further review pass is not needed (no remaining
+    blocking risk).
+  - {"action":"send_back","target":"review"} when you changed the
+    checkout to address blockers — review must re-read the whole diff
+    before publish. Do not combine send_back with fork_choice or
+
+  Reading upstream work: the user prompt contains "Factory run
+  workspace: <absolute path>". review-report.md and review payload
+  (verdict, blocking_findings) are authoritative. If Feedback Loop
+  Context is present, that is a prior send-back: fix what the latest
+  review still blocks.
+
+  Definition of done — every line must hold before you emit success:
+  - review-report.md and the review envelope were read.
+  - If verdict is pass and blocking_findings is empty: the checkout is
+    untouched, fixed_files is empty, addressed_all_blocking is true,
+    feedback_loop is continue.
+  - If any blocker exists: every blocking_findings entry is fixed, then
+    search for other instances of the same category and fix those too.
+    git diff is non-empty. Every cited failing check was re-run and
+    now passes. addressed_all_blocking is true. feedback_loop is
+    send_back to review unless a Feedback Loop Context remaining-replay
+    count is already exhausted — then continue only if the latest
+    review is actually clean; otherwise emit failure.
+  - Every fixed_files entry is a repo-relative checkout path in
+    git diff --name-only.
+  - review-feedback-report.md was written through write_stage_artifact
+    and appears in the emitted artifacts list.
+
+  1. Read review-report.md. If every verdict is pass, skip to the
+     report step with nothing to fix and continue.
+  2. Fix each blocker at the cited location, then sweep the rest of
+     the diff for the same category. No unrelated refactors. Do not
+     weaken or delete tests to go green.
+  3. Re-run every command a blocker cited. If a fix cannot complete,
+     emit failure — do not present partial remediation as success.
+  4. Call write_stage_artifact with path review-feedback-report.md
+     listing each blocker, the fix, the passing re-run, or that there
+     was nothing to address.
+  5. Emit exactly once, including the report and inherited evidence,
+     with a payload matching the schema and feedback_loop as above.
+
+  Success requires real checkout edits when blockers existed. Put
+  checkout-relative paths in payload.fixed_files. Do not put
+  stages/.../artifacts paths there.
+
+  Contract violations — emit failure rather than presenting these as
+  success:
+  - addressed_all_blocking: true while any blocker remains unverified.
+  - continue after checkout edits that have not been re-reviewed, when
+    a send_back replay remains available.
+  - send_back when the tree was not changed.
+  - A fixed_files entry under stages/.../artifacts/.
+
+  summary and payload are handed to the next stage verbatim. State
+  what was fixed, whether review must run again, and remaining risk.
+  Do not repeat this stage's prohibitions in summary.
+
+  When finished, call emit_stage_envelope exactly once. Your last tool
+  call in this attempt must be emit_stage_envelope. An empty final
+  message is a stage failure.
+model: cursor/auto
+timeout_ms: 3600000
+io:
+  input:
+    schema:
+      type: object
+      required:
+        - verdict
+        - findings
+        - blocking_findings
+        - suggestions
+        - evidence
+        - reviewed_files
+      properties:
+        verdict:
+          type: string
+          enum:
+            - pass
+            - changes_required
+        findings:
+          type: array
+          items:
+            type: string
+        blocking_findings:
+          type: array
+          items:
+            type: string
+        suggestions:
+          type: array
+          items:
+            type: string
+        evidence:
+          type: array
+          items:
+            type: string
+        reviewed_files:
+          type: array
+          minItems: 1
+          items:
+            type: string
+  output:
+    schema:
+      type: object
+      required:
+        - addressed_all_blocking
+        - fixed_files
+        - verification_commands
+        - remaining_risks
+        - sent_back
+      properties:
+        addressed_all_blocking:
+          type: boolean
+        fixed_files:
+          type: array
+          items:
+            type: string
+        verification_commands:
+          type: array
+          items:
+            type: string
+        remaining_risks:
+          type: array
+          items:
+            type: string
+        sent_back:
+          type: boolean
+verify:
+  - id: review-feedback-report-declared
+    type: artifact
+    basename: review-feedback-report.md
+    when:
+      - emit
+  - id: review-feedback-report
+    type: artifact
+    path: review-feedback-report.md
+    nonempty: true
+    when:
+      - after
+gate_kinds: []

package/examples/feature-loop/align.yaml

@@ -0,0 +1,116 @@
+id: align
+system_prompt: |
+  You are the join after plan. Read all story plans together,
+  fix overlaps and contradictions in the plan artifacts, choose the
+  sequential implementation order, and get operator acceptance before any
+  source edit. Do not implement, commit, push, or open a pull request.
+
+  Prior plan envelopes' payload (story_id, planned_files, validation_commands) and
+  story-plan.md are authoritative over any prose summary.
+
+  Reading upstream work: the user prompt contains "Factory run workspace:
+  <absolute path>". Read each prior artifact at <Factory run workspace>/
+  <artifact path>.
+
+  Definition of done — every line must hold before you emit success:
+  - Every plan envelope and story-plan.md was read.
+  - Overlapping planned_files, contradictory approaches, and missing
+    epic coverage were resolved by editing the affected story plans via
+    write_stage_artifact (new aligned copies named
+    story-plan-<story_id>.md in this attempt) — not by hoping implement
+    will notice.
+  - aligned-plans.md was written through write_stage_artifact, appears
+    in the emitted artifacts list, and records the final story set, the
+    implementation order, what changed in which plan, and remaining
+    risks.
+  - payload.stories is a Clone Array of story-work objects, one per
+    aligned story, in implementation order. Each item has story_id,
+    title, plan_artifact, order (1-based), depends_on, planned_files, and
+    validation_commands. The first item has no unmet neighbor
+    dependency; later items may depend only on earlier story_ids.
+    Encode order in each item; do not emit a sibling implementation_order
+    or story_ids array.
+  - remaining_risks is a string. approved is a boolean. alignment_notes
+    is a string.
+  - The operator accepted aligned-plans.md through artifact_backed, and
+    that accepted decision is recorded inside the artifact.
+
+  1. Read every plan envelope and story-plan.md. Diff planned_files across stories. Flag overlap, gaps
+     against epic-split.md, and validation commands that do not exist
+     in this repo.
+  2. Rewrite any story plan that needs a fix into
+     story-plan-<story_id>.md via write_stage_artifact. Unchanged plans
+     are still copied so implement reads from this attempt.
+  3. Choose a sequential order that respects neighbor_dependencies and
+     file coupling (shared types and contracts first). Put that order
+     on each stories item as order and depends_on.
+  4. Call write_stage_artifact with path aligned-plans.md covering the
+     matrix of stories, order, diffs you made, and why sequential
+     implement will not thrash shared files.
+  5. Call ask_operator with kind artifact_backed referencing
+     aligned-plans.md. Ask whether this set and order are ready to
+     implement. On reject or change text: revise artifacts and ask
+     again in this same stage. Never emit before accept.
+  6. After explicit acceptance, rewrite aligned-plans.md to include an
+     operator-decision section (accepted wording and any conditions).
+     Then emit the stories Clone Array plus approved, alignment_notes,
+     and remaining_risks as scalars.
+
+  Required files must be created with write_stage_artifact. Checkout
+  write/edit does not satisfy after-phase verify.
+
+  this same attempt. Do not pause after accept. Do not ask_operator for
+  emit/schema errors.
+
+  summary and payload are handed to the next stage verbatim. State the
+  approved story set, order, and artifact pointers only. Do not repeat
+  this stage's prohibitions in summary.
+
+  When finished, call emit_stage_envelope exactly once. Your last tool
+  call in this attempt must be emit_stage_envelope (or ask_operator if
+  waiting on HITL). An empty final message is a stage failure.
+model: cursor/auto
+timeout_ms: 3600000
+io:
+  input:
+    schema:
+      type: object
+  output:
+    schema:
+      type: object
+      required:
+        - stories
+        - approved
+        - alignment_notes
+        - remaining_risks
+      properties:
+        stories:
+          type: array
+          items:
+            $ref: "#/schemas/story-work"
+        approved:
+          type: boolean
+        alignment_notes:
+          type: string
+        remaining_risks:
+          type: string
+verify:
+  - id: plans-approved
+    type: gate
+    kind: artifact_backed
+    when:
+      - emit
+      - after
+  - id: aligned-plans-declared
+    type: artifact
+    basename: aligned-plans.md
+    when:
+      - emit
+  - id: aligned-plans
+    type: artifact
+    path: aligned-plans.md
+    nonempty: true
+    when:
+      - after
+gate_kinds:
+  - artifact_backed

package/examples/feature-loop/decompose.yaml

@@ -0,0 +1,86 @@
+id: decompose
+system_prompt: |
+  Split the task's requirements document (the epic) into independent
+  implementation stories for the next stage to plan. Do not write story plans here, edit source, commit, push, or
+  open a pull request.
+
+  There is no prior stage. The task goal, context, and constraints name
+  the epic: a requirements path, pasted epic, or equivalent. Read that
+  document from the checkout when a path is given, plus the files and
+  neighbors the epic implies. A split that restates the epic without
+  grounding it in the real tree is not done.
+
+  Definition of done — every line must hold before you emit success:
+  - epic-split.md was written through write_stage_artifact and appears in
+    the emitted artifacts list.
+  - payload.stories is an array of 1–8 story-assignment objects. Each item
+    has story_id, title, epic_slice, acceptance, neighboring_story_ids, and
+    constraints. Do not emit a parallel story_ids string array.
+  - Each story is a slice the next stage can plan in isolation: one
+    story_id (stable kebab-case), a title, an epic_slice that quotes or
+    paraphrases only the epic text that slice owns, acceptance the
+    implementer can falsify, neighboring_story_ids it must not steal
+    work from, and constraints that clone must honor.
+  - Stories do not overlap files or acceptance. Together they cover the
+    epic; leftover epic text with no owner is a contract violation.
+  - Emit 1–8 stories. Never zero stories, never more than eight.
+
+  1. Read the task and the named requirements document in the checkout.
+     Map the epic onto real files before cutting stories.
+  2. Cut 1–8 stories. Prefer seams the checkout already has (modules,
+     APIs, UI vs runtime, tests vs prod) over arbitrary chapter splits.
+     If the epic is one tight change, emit one story — do not invent filler
+     stories.
+  3. Call write_stage_artifact with path epic-split.md listing each
+     story_id, title, owned epic text, likely files, acceptance, and why
+     the slices can be planned in parallel.
+  4. Emit exactly one success envelope with payload matching the schema,
+     artifacts containing epic-split.md, and stories as those objects in
+     deterministic order. Sibling fields requirements_source and
+     split_rationale are strings only.
+
+  Required files must be created with write_stage_artifact. Checkout
+  write/edit does not satisfy after-phase verify.
+
+  summary and payload are handed to the next stage verbatim. State
+  outcomes and artifact pointers only. Do not repeat this stage's
+  prohibitions in summary — downstream will obey them literally.
+
+  When finished, call emit_stage_envelope exactly once with status,
+  this attempt must be emit_stage_envelope. An empty final message is a
+  stage failure.
+model: cursor/auto
+timeout_ms: 3600000
+io:
+  input:
+    schema:
+      type: object
+  output:
+    schema:
+      type: object
+      required:
+        - stories
+        - requirements_source
+        - split_rationale
+      properties:
+        stories:
+          type: array
+          items:
+            $ref: "#/schemas/story-assignment"
+        requirements_source:
+          type: string
+        split_rationale:
+          type: string
+verify:
+  - id: epic-split-declared
+    type: artifact
+    basename: epic-split.md
+    when:
+      - emit
+  - id: epic-split
+    type: artifact
+    path: epic-split.md
+    nonempty: true
+    when:
+      - after
+gate_kinds: []

package/examples/feature-loop/default-model-hierarchy.task.yaml

@@ -0,0 +1,58 @@
+id: default-model-hierarchy
+goal: >
+  Implement default model resolution across Stageflow YAML layers so a
+  model can be set once on stageflow.yaml or a pipeline and inherited by
+  stages that omit model. Resolution priority must be stage first, then
+  pipeline, then the global stageflow.yaml default. Split the work into
+  stories, plan each story, align the plans, implement them in the agreed
+  order, verify coverage, review the whole diff, address review feedback,
+  and open an operator-approved pull request.
+context: |
+  This repository is Stageflow itself (tejasghutukade/stageflow). The
+  checkout for this run is a dedicated git worktree already on branch
+  scratch/extra-workspace, tracking origin/main.
+
+  Epic (pasted — treat this block as the requirements document):
+
+  Today every stage must declare model (string). Omitting it fails load.
+  There is no pipeline-level or stageflow.yaml default for model. The
+  agent backend already resolves as stage > pipeline > global > "pi";
+  model needs the same shape of hierarchy, without reusing the agent key.
+
+  Required behavior:
+  1. Optional top-level model on stageflow.yaml (global default).
+  2. Optional top-level model on *.pipeline.yaml (pipeline default for
+     all stages in that pipeline).
+  3. Stage model remains allowed on inline stage bodies and external
+     stage YAML, but becomes optional when a higher default can fill it.
+  4. Effective resolution for each stage at load or run time:
+     stage.model ?? pipeline.model ?? stageflow.yaml model.
+     If still unset after that chain, load/run must fail with a clear
+     error — never invent a silent hardcoded model string.
+  5. Priority is always stage override, then pipeline, then global.
+
+  Likely touch points (verify against the tree; do not assume line
+  numbers): src/types/stageflowManifest.ts, src/types/pipeline.ts,
+  src/types for StageConfig, src/config/loadStageflowManifest.ts,
+  src/config load/merge pipeline helpers, src/config/loadStage.ts,
+  resolveAgentPort.ts (pattern to mirror for model, not to conflate),
+  stage worker / adapters that read stage.model, docs/yaml-catalog.md,
+  fixtures under tests/fixtures/, and any create/init surfaces that
+  currently require stage model.
+
+  Keep model (LLM id string) distinct from agent (pi | claude backend).
+
+  - Publish target: tejasghutukade/stageflow
+  - Base branch: main
+  - Head branch: the checkout's current branch (do not create or switch)
+constraints: |
+  Do not create, rename, or switch branches — the checkout is already
+  on the correct feature branch (scratch/extra-workspace).
+  Do not commit, push, or open a pull request except in publish, and
+  only after the operator accepts ship-package.md.
+  Do not expand scope past the epic. Story plans and sequential
+  implement clones must stay inside their assigned slices.
+  Do not change agent-backend resolution semantics except where
+  documentation must clarify that model and agent are separate.
+  publish_repo and base_branch must be the values named here, never guessed.
+checkout: /Users/tejasghutukade/Projects/software-factory/.worktrees/scratch/extra-workspace

package/examples/feature-loop/feature-loop.pipeline.yaml

@@ -0,0 +1,140 @@
+id: feature-loop
+schemas:
+  story-assignment:
+    type: object
+    required:
+      - story_id
+      - title
+      - epic_slice
+      - acceptance
+      - neighboring_story_ids
+      - constraints
+    properties:
+      story_id:
+        type: string
+      title:
+        type: string
+      epic_slice:
+        type: string
+      acceptance:
+        type: string
+      neighboring_story_ids:
+        type: array
+        items:
+          type: string
+      constraints:
+        type: array
+        items:
+          type: string
+  story-work:
+    type: object
+    required:
+      - story_id
+      - title
+      - plan_artifact
+      - order
+      - depends_on
+      - planned_files
+      - validation_commands
+    properties:
+      story_id:
+        type: string
+      title:
+        type: string
+      plan_artifact:
+        type: string
+      order:
+        type: integer
+        minimum: 1
+      depends_on:
+        type: array
+        items:
+          type: string
+      planned_files:
+        type: array
+        items:
+          type: string
+      validation_commands:
+        type: array
+        items:
+          type: string
+stages:
+  - id: decompose
+    uses: ./decompose.yaml
+    entry: true
+    clone_cap: 8
+    clone_mode: parallel
+    route:
+      - to: plan
+    on_verify_fail:
+      mode: repair
+      max_attempts: 3
+      retry_safety: idempotent
+      include_failed_checks: true
+  - id: plan
+    uses: ./plan.yaml
+    route:
+      - to: align
+    on_verify_fail:
+      mode: repair
+      max_attempts: 3
+      retry_safety: idempotent
+      include_failed_checks: true
+  - id: align
+    uses: ./align.yaml
+    clone_cap: 8
+    clone_mode: sequential
+    route:
+      - to: implement
+    on_verify_fail:
+      mode: repair
+      max_attempts: 3
+      retry_safety: idempotent
+      include_failed_checks: true
+  - id: implement
+    uses: ./implement.yaml
+    route:
+      - to: verify
+    on_verify_fail:
+      mode: repair
+      max_attempts: 3
+      retry_safety: idempotent
+      include_failed_checks: true
+  - id: verify
+    uses: ./verify.yaml
+    route:
+      - to: review
+    on_verify_fail:
+      mode: repair
+      max_attempts: 3
+      retry_safety: idempotent
+      include_failed_checks: true
+  - id: review
+    uses: ./review.yaml
+    route:
+      - to: address-feedback
+    on_verify_fail:
+      mode: repair
+      max_attempts: 3
+      retry_safety: idempotent
+      include_failed_checks: true
+  - id: address-feedback
+    uses: ./address-feedback.yaml
+    route:
+      - to: publish
+      - type: loop
+        to: review
+        max_replays: 2
+        on_max_replays: wait_for_human
+        replay_session: new_session
+    on_verify_fail:
+      mode: repair
+      max_attempts: 3
+      retry_safety: idempotent
+      include_failed_checks: true
+  - id: publish
+    uses: ./publish.yaml
+    on_verify_fail:
+      mode: manual
+      retry_safety: side_effecting
+    replay_safe: false

package/examples/feature-loop/feature-loop.task.yaml

@@ -0,0 +1,25 @@
+id: feature-loop
+goal: >
+  Implement the epic in the requirements document named in context:
+  split it into stories, plan each story, align the plans, implement
+  them in the agreed order, verify coverage, review the whole diff,
+  address review feedback, and open an operator-approved pull request.
+context: |
+  Point this task at a real checkout already on the feature branch.
+  Name the requirements/epic path (checkout-relative), the publish
+  repository (owner/name), and the base branch.
+
+  Replace the placeholders below before running:
+
+  - Requirements/epic: <path-to-requirements.md>
+  - Publish target: <owner/repo>
+  - Base branch: main
+  - Head branch: the checkout's current branch (do not create or switch)
+constraints: |
+  Do not create, rename, or switch branches — the checkout is already
+  on the correct feature branch.
+  Do not commit, push, or open a pull request except in publish, and
+  only after the operator accepts ship-package.md.
+  Do not expand scope past the epic. Story plans and sequential
+  implement clones must stay inside their assigned slices.
+  publish_repo and base_branch must be the values named here, never guessed.