stageflow 0.25.0 → 0.30.0

package/examples/compliance-review-demo/poll-bucket-config.yaml

@@ -0,0 +1,103 @@
+id: poll-bucket-config
+system_prompt: |
+  Read bucket, check_id, and scenario from the prior stage's payload.
+  bucket is the bare bucket name, no gs:// or s3:// prefix — add the right
+  prefix yourself for whichever CLI you end up using.
+
+  The four output fields (block_public_acls, ignore_public_acls,
+  block_public_policy, restrict_public_buckets) are cloud-agnostic — they
+  mean "is public access meaningfully blocked", not literally AWS S3
+  setting names. Try real polls in this order, first one that works wins:
+
+  1. GCP: if `gcloud` is on PATH and `gcloud auth list
+     --filter=status:ACTIVE --format="value(account)"` returns a account,
+     poll the real bucket:
+       gcloud storage buckets describe gs://<bucket> --format=json
+       gcloud storage buckets get-iam-policy gs://<bucket> --format=json
+     Read these exact fields from the describe output:
+       public_access_prevention  ("enforced" or "inherited")
+       uniform_bucket_level_access  (boolean)
+     From the IAM policy output, check every binding's members list for
+     "allUsers" or "allAuthenticatedUsers" — call this has_public_iam_binding.
+     Map to the four fields:
+       block_public_acls    = uniform_bucket_level_access
+       ignore_public_acls   = uniform_bucket_level_access
+       block_public_policy  = (public_access_prevention == "enforced")
+       restrict_public_buckets = NOT has_public_iam_binding
+     Set source = "real-gcp". Ignore scenario in this case.
+
+  2. AWS: else if `aws` CLI is on PATH and `aws sts get-caller-identity`
+     succeeds, run `aws s3api get-public-access-block --bucket <bucket>`
+     and use that real response directly (the field names already match).
+     Set source = "real-aws". Ignore scenario in this case.
+
+  3. Simulated: otherwise — no working cloud CLI — produce a SIMULATED
+     response and set source = "simulated". Never present simulated data
+     as real; label it "simulated" wherever it's written down. Use
+     scenario to pick the fixture:
+
+       scenario "fail":
+         { block_public_acls: true, ignore_public_acls: true,
+           block_public_policy: false, restrict_public_buckets: true }
+
+       scenario "fixed":
+         { block_public_acls: true, ignore_public_acls: true,
+           block_public_policy: true, restrict_public_buckets: true }
+
+  Write the raw response via write_stage_artifact. For a real GCP poll,
+  include both the raw gcloud fields (public_access_prevention,
+  uniform_bucket_level_access, has_public_iam_binding) and the four mapped
+  booleans, so the translation is visible, not hidden. Clearly mark the
+  artifact "simulated", "real-aws", or "real-gcp".
+
+  summary and payload are handed to the next stage verbatim. State outcomes
+  and artifact pointers only. Do not repeat this stage's prohibitions in
+  summary — downstream will obey them literally.
+
+  When finished, call emit_stage_envelope exactly once with status, summary,
+  artifacts, and a payload matching io.output.schema: bucket, check_id,
+  scenario carried forward, plus source and the four boolean fields. Your
+  last tool call in this attempt must be emit_stage_envelope. An empty
+  final message is a stage failure.
+model: anthropic/claude-sonnet-4-5
+io:
+  input:
+    schema:
+      type: object
+      required: [bucket, check_id, scenario]
+      properties:
+        bucket:
+          type: string
+        check_id:
+          type: string
+        scenario:
+          type: string
+  output:
+    schema:
+      type: object
+      required:
+        - bucket
+        - check_id
+        - scenario
+        - source
+        - block_public_acls
+        - ignore_public_acls
+        - block_public_policy
+        - restrict_public_buckets
+      properties:
+        bucket:
+          type: string
+        check_id:
+          type: string
+        scenario:
+          type: string
+        source:
+          type: string
+        block_public_acls:
+          type: boolean
+        ignore_public_acls:
+          type: boolean
+        block_public_policy:
+          type: boolean
+        restrict_public_buckets:
+          type: boolean

package/examples/compliance-review-demo/publish-report.yaml

@@ -0,0 +1,79 @@
+id: publish-report
+system_prompt: |
+  Read the evidence log at
+  examples/compliance-review-demo/data/evidence-log.json (checkout-relative,
+  use the Read tool) and the current run's payload.
+
+  Write a short markdown report via write_stage_artifact (for example
+  report.md) showing the timeline for this check_id + resource: every
+  recorded verdict with its timestamp, and — for any row with
+  reviewed = true — the reviewer. Make the human-in-the-loop review visible:
+  call out explicitly which rows were auto-approved (no flip) versus which
+  required a person's confirmation (a flip), since that distinction is the
+  whole point of this pipeline.
+
+  When finished, call emit_stage_envelope exactly once with status, summary,
+  and artifacts pointing at the report. Your last tool call in this attempt
+  must be emit_stage_envelope. An empty final message is a stage failure.
+model: anthropic/claude-sonnet-4-5
+io:
+  input:
+    schema:
+      type: object
+      required:
+        - bucket
+        - check_id
+        - scenario
+        - source
+        - block_public_acls
+        - ignore_public_acls
+        - block_public_policy
+        - restrict_public_buckets
+        - verdict
+        - resource
+        - failed_fields
+        - reviewed
+        - auto_approved
+        - reviewer
+        - control_id
+        - changed_files
+      properties:
+        bucket:
+          type: string
+        check_id:
+          type: string
+        scenario:
+          type: string
+        source:
+          type: string
+        block_public_acls:
+          type: boolean
+        ignore_public_acls:
+          type: boolean
+        block_public_policy:
+          type: boolean
+        restrict_public_buckets:
+          type: boolean
+        verdict:
+          type: string
+        resource:
+          type: string
+        failed_fields:
+          type: array
+          items:
+            type: string
+        reviewed:
+          type: boolean
+        auto_approved:
+          type: boolean
+        reviewer:
+          type: string
+        control_id:
+          type: string
+        changed_files:
+          type: array
+          items:
+            type: string
+  output:
+    schema:
+      type: object

package/examples/compliance-review-demo/review-verdict.yaml

@@ -0,0 +1,122 @@
+id: review-verdict
+system_prompt: |
+  This stage never pauses for a human — it has to be safe to run unattended
+  on a schedule (cron, a nightly GitHub Action, Cloud Scheduler). It still
+  tells the difference between "nothing changed" and "this just flipped,"
+  it just never blocks on that difference.
+
+  1. Read examples/compliance-review-demo/data/evidence-log.json if it
+     exists (checkout-relative path, use the Read tool). Find the most
+     recent entry for this run's check_id + resource, if any.
+  2. No prior entry, or the prior entry's verdict matches this run's
+     verdict: set reviewer = "auto".
+  3. Prior entry exists and its verdict differs from this run's verdict (a
+     flip): set reviewer = "unreviewed-flip" and write a short artifact via
+     write_stage_artifact noting the flip (prior verdict, new verdict, the
+     raw field values from the prior stage's payload), so a person can spot
+     it later in the evidence log or the published report — without the
+     run ever waiting on them.
+  4. In both cases set reviewed = false and auto_approved = true. Never
+     call ask_operator — this stage has no gate_kinds. Always emit an
+     advancing success status; a flip is information, not a failure.
+
+  summary and payload are handed to the next stage verbatim. State outcomes
+  and artifact pointers only. Do not repeat this stage's prohibitions in
+  summary — downstream will obey them literally.
+
+  When finished, call emit_stage_envelope exactly once with status, summary,
+  artifacts, and a payload matching io.output.schema: everything carried
+  forward from the prior stage, plus reviewed, auto_approved, and reviewer
+  as described above. Your last tool call in this attempt must be
+  emit_stage_envelope. An empty final message is a stage failure.
+model: anthropic/claude-sonnet-4-5
+io:
+  input:
+    schema:
+      type: object
+      required:
+        - bucket
+        - check_id
+        - scenario
+        - source
+        - block_public_acls
+        - ignore_public_acls
+        - block_public_policy
+        - restrict_public_buckets
+        - verdict
+        - resource
+        - failed_fields
+      properties:
+        bucket:
+          type: string
+        check_id:
+          type: string
+        scenario:
+          type: string
+        source:
+          type: string
+        block_public_acls:
+          type: boolean
+        ignore_public_acls:
+          type: boolean
+        block_public_policy:
+          type: boolean
+        restrict_public_buckets:
+          type: boolean
+        verdict:
+          type: string
+        resource:
+          type: string
+        failed_fields:
+          type: array
+          items:
+            type: string
+  output:
+    schema:
+      type: object
+      required:
+        - bucket
+        - check_id
+        - scenario
+        - source
+        - block_public_acls
+        - ignore_public_acls
+        - block_public_policy
+        - restrict_public_buckets
+        - verdict
+        - resource
+        - failed_fields
+        - reviewed
+        - auto_approved
+        - reviewer
+      properties:
+        bucket:
+          type: string
+        check_id:
+          type: string
+        scenario:
+          type: string
+        source:
+          type: string
+        block_public_acls:
+          type: boolean
+        ignore_public_acls:
+          type: boolean
+        block_public_policy:
+          type: boolean
+        restrict_public_buckets:
+          type: boolean
+        verdict:
+          type: string
+        resource:
+          type: string
+        failed_fields:
+          type: array
+          items:
+            type: string
+        reviewed:
+          type: boolean
+        auto_approved:
+          type: boolean
+        reviewer:
+          type: string

package/examples/compliance-review-demo/run-compliance-check.yaml

@@ -0,0 +1,96 @@
+id: run-compliance-check
+system_prompt: |
+  Read the four public-access-block field values, bucket, and check_id from
+  the prior stage's payload.
+
+  Apply the rule for bucket-level public access block (the same rule
+  Prowler's SOC2-labeled AWS S3 check uses, applied here to whichever
+  cloud the prior stage actually polled): the check passes only if
+  block_public_acls, ignore_public_acls, block_public_policy, and
+  restrict_public_buckets are all true. If any field is false, the check
+  fails; name which field(s) failed in failed_fields. Use an empty list
+  when nothing failed.
+
+  Write the verdict via write_stage_artifact (for example verdict.md),
+  including the field values and which failed, if any.
+
+  summary and payload are handed to the next stage verbatim. State outcomes
+  and artifact pointers only. Do not repeat this stage's prohibitions in
+  summary — downstream will obey them literally.
+
+  When finished, call emit_stage_envelope exactly once with status, summary,
+  artifacts, and a payload matching io.output.schema: everything carried
+  forward from the prior stage, plus verdict ("PASS" or "FAIL"), resource
+  (the bucket name), and failed_fields. Your last tool call in this attempt
+  must be emit_stage_envelope. An empty final message is a stage failure.
+model: anthropic/claude-sonnet-4-5
+io:
+  input:
+    schema:
+      type: object
+      required:
+        - bucket
+        - check_id
+        - scenario
+        - source
+        - block_public_acls
+        - ignore_public_acls
+        - block_public_policy
+        - restrict_public_buckets
+      properties:
+        bucket:
+          type: string
+        check_id:
+          type: string
+        scenario:
+          type: string
+        source:
+          type: string
+        block_public_acls:
+          type: boolean
+        ignore_public_acls:
+          type: boolean
+        block_public_policy:
+          type: boolean
+        restrict_public_buckets:
+          type: boolean
+  output:
+    schema:
+      type: object
+      required:
+        - bucket
+        - check_id
+        - scenario
+        - source
+        - block_public_acls
+        - ignore_public_acls
+        - block_public_policy
+        - restrict_public_buckets
+        - verdict
+        - resource
+        - failed_fields
+      properties:
+        bucket:
+          type: string
+        check_id:
+          type: string
+        scenario:
+          type: string
+        source:
+          type: string
+        block_public_acls:
+          type: boolean
+        ignore_public_acls:
+          type: boolean
+        block_public_policy:
+          type: boolean
+        restrict_public_buckets:
+          type: boolean
+        verdict:
+          type: string
+        resource:
+          type: string
+        failed_fields:
+          type: array
+          items:
+            type: string

package/examples/compliance-review-demo/store-evidence.yaml

@@ -0,0 +1,143 @@
+id: store-evidence
+system_prompt: |
+  Append one row to the checkout-relative evidence log at
+  examples/compliance-review-demo/data/evidence-log.json.
+
+  Read that file with the Read tool if it exists and parse it as a JSON
+  array; if it does not exist yet, start from an empty array. Append a new
+  object with: timestamp (ISO 8601, now), check_id, resource, control_id,
+  verdict, reviewed, auto_approved, reviewer (use JSON null in this file
+  when reviewer is "") — all from the prior stage's payload. Write the full
+  updated array back to that same path with the checkout write/edit tool,
+  pretty-printed.
+
+  Success requires real checkout edits. Put
+  ["examples/compliance-review-demo/data/evidence-log.json"] in
+  payload.changed_files. Do not put stages/.../artifacts paths there. A
+  report or map alone is a failure emit, not success.
+
+  summary and payload are handed to the next stage verbatim. State outcomes
+  and artifact pointers only. Do not repeat this stage's prohibitions in
+  summary — downstream will obey them literally.
+
+  When finished, call emit_stage_envelope exactly once with status, summary,
+  artifacts, and a payload matching io.output.schema: everything carried
+  forward from the prior stage, plus changed_files naming the evidence log
+  path. Your last tool call in this attempt must be emit_stage_envelope. An
+  empty final message is a stage failure.
+model: anthropic/claude-sonnet-4-5
+io:
+  input:
+    schema:
+      type: object
+      required:
+        - bucket
+        - check_id
+        - scenario
+        - source
+        - block_public_acls
+        - ignore_public_acls
+        - block_public_policy
+        - restrict_public_buckets
+        - verdict
+        - resource
+        - failed_fields
+        - reviewed
+        - auto_approved
+        - reviewer
+        - control_id
+      properties:
+        bucket:
+          type: string
+        check_id:
+          type: string
+        scenario:
+          type: string
+        source:
+          type: string
+        block_public_acls:
+          type: boolean
+        ignore_public_acls:
+          type: boolean
+        block_public_policy:
+          type: boolean
+        restrict_public_buckets:
+          type: boolean
+        verdict:
+          type: string
+        resource:
+          type: string
+        failed_fields:
+          type: array
+          items:
+            type: string
+        reviewed:
+          type: boolean
+        auto_approved:
+          type: boolean
+        reviewer:
+          type: string
+        control_id:
+          type: string
+  output:
+    schema:
+      type: object
+      required:
+        - bucket
+        - check_id
+        - scenario
+        - source
+        - block_public_acls
+        - ignore_public_acls
+        - block_public_policy
+        - restrict_public_buckets
+        - verdict
+        - resource
+        - failed_fields
+        - reviewed
+        - auto_approved
+        - reviewer
+        - control_id
+        - changed_files
+      properties:
+        bucket:
+          type: string
+        check_id:
+          type: string
+        scenario:
+          type: string
+        source:
+          type: string
+        block_public_acls:
+          type: boolean
+        ignore_public_acls:
+          type: boolean
+        block_public_policy:
+          type: boolean
+        restrict_public_buckets:
+          type: boolean
+        verdict:
+          type: string
+        resource:
+          type: string
+        failed_fields:
+          type: array
+          items:
+            type: string
+        reviewed:
+          type: boolean
+        auto_approved:
+          type: boolean
+        reviewer:
+          type: string
+        control_id:
+          type: string
+        changed_files:
+          type: array
+          items:
+            type: string
+verify:
+  - id: evidence-log-written
+    type: checkout_changes
+    path_fields: [changed_files]
+    when: [after]

package/examples/conditional-fork/README.md

@@ -0,0 +1,89 @@
+# conditional-fork
+
+Operator HITL, then exclusive Route `if`. The `decide` stage asks for **branch-a** or **branch-b**, emits that value on `payload.branch`, and listed successors stay on the DAG. After success, each `if` runs against that payload: the matching arm runs; the other is skipped. The completing agent does not pick YAML successors.
+
+See [YAML catalog — Route wiring](../../docs/yaml-catalog.md#route) (Route `if`). `if.field` must be a required output property — optional fields cannot be used in `if`.
+
+## Prerequisites
+
+- Node.js ≥ 20, Stageflow installed (`npm i -g stageflow`), **or** this repo built (`npm run build`)
+- Provider auth connected (`sf ui` → Settings → Providers, or `sf providers login …`)
+- Operator console for the gate: `sf ui` (default `http://127.0.0.1:3847`)
+
+## Manual test
+
+Commands run from the **repository git root**. Run state lives in `<repo>/.stageflow` even if you start `sf ui` from a subdirectory.
+
+### Terminal 1 — operator console
+
+```bash
+cd examples/conditional-fork
+sf ui
+```
+
+### Terminal 2 — validate and run (repo root)
+
+```bash
+sf validate --pipeline examples/conditional-fork/fork-demo.pipeline.yaml --strict
+sf run \
+  --pipeline examples/conditional-fork/fork-demo.pipeline.yaml \
+  --task examples/conditional-fork/fork-demo.task.yaml
+```
+
+### From the Stageflow repo (no global install)
+
+```bash
+npm run build
+npm run dev -- validate --pipeline examples/conditional-fork/fork-demo.pipeline.yaml --strict
+npm run dev -- run \
+  --pipeline examples/conditional-fork/fork-demo.pipeline.yaml \
+  --task examples/conditional-fork/fork-demo.task.yaml
+```
+
+### At the HITL gate
+
+When `decide` blocks, open the run in the console and answer the free-text prompt with exactly:
+
+- `branch-a`, or
+- `branch-b`
+
+Exit code `2` means waiting on operator input. See [docs/hitl.md](../../docs/hitl.md).
+
+### What to verify in the UI
+
+After you answer:
+
+1. The chosen branch stage (`branch-a` or `branch-b`) reaches **succeeded**
+2. The other branch is **skipped**
+3. The run overall **succeeded**
+
+### Re-test the other branch
+
+Start a **new** run and answer the opposite id at the gate.
+
+## How it works
+
+`decide` is `entry: true` with two `route` targets, each gated by `if` (`field: branch`, `op: eq`). After the operator answers, the agent emits `payload: { "branch": "branch-a" }` or `{ "branch": "branch-b" }`. Stageflow evaluates those predicates against the success payload: the matching successor runs; the miss is skipped. Both `to:` stay on the DAG.
+
+`io.output.schema` requires `branch` as a string so `if.field: branch` is valid. Branch stage `io.input` is an empty object (a subset of decide output).
+
+Validate may warn `pipeline.route_all_gated` because every forward `to:` on `decide` has `if`. That warning keeps `ok: true`; `--strict` does not promote it.
+
+## Layout
+
+```
+examples/conditional-fork/
+  fork-demo.pipeline.yaml
+  fork-demo.task.yaml
+  decide.yaml
+  branch-a.yaml
+  branch-b.yaml
+```
+
+## References
+
+- [YAML catalog — Route wiring](../../docs/yaml-catalog.md#route)
+- [YAML catalog — Upgrading older catalogs](../../docs/yaml-catalog.md#upgrading-older-catalogs)
+- [HITL](../../docs/hitl.md)
+- Fixtures: [`route-if-eq.pipeline.yaml`](../../tests/fixtures/pipelines/route-if-eq.pipeline.yaml)
+- Related walkthrough: [`../route-if-tour/`](../route-if-tour/)

package/examples/conditional-fork/branch-a.yaml

@@ -0,0 +1,13 @@
+id: branch-a
+system_prompt: |
+  You are branch-a. Call emit_stage_envelope exactly once with status success,
+  a short summary that acknowledges branch-a ran, and artifacts []. Do not call
+  ask_operator.
+model: anthropic/claude-sonnet-4-5
+io:
+  input:
+    schema:
+      type: object
+  output:
+    schema:
+      type: object

package/examples/conditional-fork/branch-b.yaml

@@ -0,0 +1,13 @@
+id: branch-b
+system_prompt: |
+  You are branch-b. Call emit_stage_envelope exactly once with status success,
+  a short summary that acknowledges branch-b ran, and artifacts []. Do not call
+  ask_operator.
+model: cursor/auto
+io:
+  input:
+    schema:
+      type: object
+  output:
+    schema:
+      type: object

package/examples/conditional-fork/decide.yaml

@@ -0,0 +1,35 @@
+id: decide
+gate_kinds:
+  - free_text
+system_prompt: |
+  You are the deciding stage for exclusive Route if branching. The operator
+  chooses which branch runs. Follow these steps exactly:
+
+  1. Call ask_operator with kind free_text. Ask which branch to take. Tell the
+     operator to reply with exactly one of: branch-a or branch-b.
+  2. Wait for the operator answer. Do not call emit_stage_envelope before you
+     have that answer.
+  3. After the operator answers, call emit_stage_envelope exactly once with:
+     - status: success
+     - a non-empty summary
+     - artifacts: []
+     - payload: { "branch": "branch-a" } or { "branch": "branch-b" }, matching
+       the operator reply exactly
+
+  Rules:
+  - Never emit before the operator answers.
+  - Do not emit fork_choice. Routing is deterministic from payload.branch.
+  - Never invent other branch values.
+  - Put only the id the operator chose in payload.branch.
+model: cursor/auto
+io:
+  input:
+    schema:
+      type: object
+  output:
+    schema:
+      type: object
+      required: [branch]
+      properties:
+        branch:
+          type: string