@manhquy/flow-skill 0.1.0-rc.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (76) hide show
  1. package/CHANGELOG.md +60 -0
  2. package/LICENSE +21 -0
  3. package/README.md +152 -0
  4. package/README_VN.md +134 -0
  5. package/SECURITY.md +69 -0
  6. package/bin/cli.mjs +382 -0
  7. package/package.json +52 -0
  8. package/skills/flow/SKILL.md +234 -0
  9. package/skills/flow/_templates/00-idea.md +16 -0
  10. package/skills/flow/_templates/00-inspect.md +42 -0
  11. package/skills/flow/_templates/01-research.md +59 -0
  12. package/skills/flow/_templates/02-scope.md +71 -0
  13. package/skills/flow/_templates/03-prd.md +57 -0
  14. package/skills/flow/_templates/04-adr.md +21 -0
  15. package/skills/flow/_templates/05-contract.md +59 -0
  16. package/skills/flow/_templates/card.md +26 -0
  17. package/skills/flow/_templates/constitution.md +22 -0
  18. package/skills/flow/eval/fixtures/f01a/flow/01-research.md +77 -0
  19. package/skills/flow/eval/fixtures/f01b/flow/01-research.md +75 -0
  20. package/skills/flow/eval/fixtures/f02a/flow/02-scope.md +56 -0
  21. package/skills/flow/eval/fixtures/f02b/flow/02-scope.md +65 -0
  22. package/skills/flow/eval/fixtures/fcda/.flow/.gitkeep +0 -0
  23. package/skills/flow/eval/fixtures/fcda/cards/C-001.md +46 -0
  24. package/skills/flow/eval/fixtures/fcdb/.flow/.gitkeep +0 -0
  25. package/skills/flow/eval/fixtures/fcdb/cards/C-001.md +35 -0
  26. package/skills/flow/eval/manifest.tsv +7 -0
  27. package/skills/flow/harness/README.md +117 -0
  28. package/skills/flow/harness/_db.py +228 -0
  29. package/skills/flow/harness/_domain.py +161 -0
  30. package/skills/flow/harness/_presence.py +109 -0
  31. package/skills/flow/harness/flow_harness.py +1103 -0
  32. package/skills/flow/harness/repo_map.py +93 -0
  33. package/skills/flow/harness/schema/001-init.sql +129 -0
  34. package/skills/flow/harness/schema/002-story-verify.sql +9 -0
  35. package/skills/flow/harness/schema/003-tool-registry.sql +15 -0
  36. package/skills/flow/harness/schema/004-intervention.sql +15 -0
  37. package/skills/flow/harness/schema/005-tool-extensions.sql +20 -0
  38. package/skills/flow/harness/schema/009-accessed-count.sql +11 -0
  39. package/skills/flow/harness/schema/010-usage-event.sql +42 -0
  40. package/skills/flow/harness/schema/011-usage-gate-reason.sql +8 -0
  41. package/skills/flow/harness/schema/012-usage-ephemeral.sql +10 -0
  42. package/skills/flow/law/CLAUDE.md +140 -0
  43. package/skills/flow/law/DESIGN.md +141 -0
  44. package/skills/flow/law/RETRO.md +13 -0
  45. package/skills/flow/playbooks/README.md +28 -0
  46. package/skills/flow/playbooks/cloudflare-ai-gateway-qwen.md +346 -0
  47. package/skills/flow/playbooks/docker-deploy-stale-cache.md +32 -0
  48. package/skills/flow/playbooks/rag-lite-postgres-cloudflare.md +192 -0
  49. package/skills/flow/playbooks/windows-gitbash-jsonl-telemetry.md +57 -0
  50. package/skills/flow/references/adversarial-review.md +197 -0
  51. package/skills/flow/references/agent-detection.md +88 -0
  52. package/skills/flow/references/agent-stage-mapping.md +114 -0
  53. package/skills/flow/references/antigravity-integration.md +92 -0
  54. package/skills/flow/references/auto-run.md +81 -0
  55. package/skills/flow/references/claudekit-skills.md +128 -0
  56. package/skills/flow/references/codex-integration.md +104 -0
  57. package/skills/flow/references/command-dispatch.md +54 -0
  58. package/skills/flow/references/debt-and-halts.md +41 -0
  59. package/skills/flow/references/design-review-checklist.md +42 -0
  60. package/skills/flow/references/gate-eval.md +186 -0
  61. package/skills/flow/references/gate-rules.md +171 -0
  62. package/skills/flow/references/ground-truth-gates.md +49 -0
  63. package/skills/flow/references/loop-harness-2026-principles.md +52 -0
  64. package/skills/flow/references/mode-work.md +37 -0
  65. package/skills/flow/references/project-types.md +42 -0
  66. package/skills/flow/references/stage-state-machine.md +47 -0
  67. package/skills/flow/references/ui-patterns-tcr.md +49 -0
  68. package/skills/flow/runner/flow.cmd +18 -0
  69. package/skills/flow/runner/flow.sh +3297 -0
  70. package/skills-manifest.json +70 -0
  71. package/src/constants.mjs +65 -0
  72. package/src/detect.mjs +36 -0
  73. package/src/help.mjs +42 -0
  74. package/src/installer.mjs +231 -0
  75. package/src/prompt.mjs +60 -0
  76. package/src/tty.mjs +11 -0
@@ -0,0 +1,128 @@
1
+ # claudekit skills — the per-stage capability map (the seam)
2
+
3
+ `/flow` already orchestrates claudekit at the **agent layer** (`agent-detection.md` +
4
+ `agent-stage-mapping.md`: 13 ck: agents, ck:→bmad→built-in degrade). This file extends that
5
+ same seam to the **skill layer** — the curated set of ck *skills* worth reaching for at each
6
+ stage, beyond the wired agents. It is a **whitelist, not a dump**: the kit has ~87 skills and
7
+ ~60% are marketing/content noise for a build harness. Only the skills below ever get surfaced.
8
+
9
+ This is the single source of truth for the skill map; `agent-stage-mapping.md` points here.
10
+
11
+ ## The two rules that bind every skill suggestion
12
+
13
+ 1. **A skill INFORMS a stage; the gate JUDGES it.** Same as the Codex/Antigravity rule
14
+ (`agent-detection.md`, `codex-integration.md`): a skill's output feeds your triage — it
15
+ **never auto-passes and never auto-fails** a gate. The `flow.sh` exit code + the
16
+ `gate-rules.md` challenge remain the only deciders. A skill that "looks good" is not a PASS.
17
+ 2. **Detection is Claude-side, and degrades silently.** You already know the host Skills list
18
+ (the available-skills registry) — check it; do not assume. `flow.sh` **cannot** see which
19
+ skills are loaded (bash has no view of the registry, and the 5 install homes — Claude,
20
+ Codex, Antigravity ×2, project — differ), so skill detection is never put in the runner. A
21
+ missing skill **never lowers a gate** — it only changes whether the enrichment is offered.
22
+ Rich where the skill exists, unbroken where it doesn't.
23
+
24
+ ## Cost gate — offer, don't impose
25
+
26
+ The high-value skills below are **opt-in-with-prompt**: at the matching gate you *offer* the
27
+ skill and the operator confirms — you do not auto-fire it on every stage (the constitution /
28
+ Codex cost-gate discipline: advisory, off the hot path, no token-tax on trivial work). Skip the
29
+ offer on a trivial decision/card. Never wire a skill into `cmd_next`/`cmd_check`.
30
+
31
+ ## The whitelist — what to use when
32
+
33
+ Each skill is pinned to the stage where it pays off and the **distinct verb** it adds beyond the
34
+ already-wired agent. Pure skill/agent twins are deliberately absent (see "Don't surface").
35
+
36
+ | Skill | Stage | Distinct verb it adds (why it's not a twin) |
37
+ |---|---|---|
38
+ | **ck-predict** | ADR (04) | 5-persona pre-decision debate — no agent equivalent |
39
+ | **ck-scenario** | Scope/PRD/**Contract** (02–05) | 12-dimension edge-case → acceptance criteria + contract tests |
40
+ | **repomix** | Assess | packed single-file repo snapshot for LLM context |
41
+ | **ck-graphify** | Assess/Build | queryable knowledge graph (code+docs) for impact analysis |
42
+ | **review-pr** | Review/Ship | PR-context review: AI-slop, CI-blocker, breaking-change, `--fix` |
43
+ | **ck-security** | Review (security-class cards) | STRIDE+OWASP threat-model with attacker personas |
44
+ | **security-scan** | Review/Deploy | mechanical secrets/deps/OWASP scan (tip-level, cheap) |
45
+ | **retro** | Retro | git-history numeric retrospective (vs narrative journal) |
46
+ | **xia** | Research/Build | port/compare/adapt a feature from another repo |
47
+ | **ghpm** | Verify/Deploy | bind work to GitHub Issues/Projects/CI status |
48
+ | **deploy** | Deploy | platform auto-detect publish |
49
+ | **web-testing** | Verify-live | Playwright/k6 e2e/load/a11y (distinct from `tester` unit verb) |
50
+ | **docs-seeker** | Research/Build | fetch current library docs (llms.txt/context7) — version-drift guard |
51
+ | **scout** | Assess/any | fast parallel file discovery (distinct from inline read) |
52
+ | **ck-loop** | Build/Verify (Implement→Test→Audit→Fix) | mechanical iterate-to-numeric-target with per-iteration git commit/revert — the loop-engineering primitive flow lacked natively |
53
+
54
+ > Graph tool: **ck-graphify** is the chosen impact-analysis tool (operator decision, 2026-06-23
55
+ > — polyglot/doc-heavy brownfield fit, e.g. CMC Odoo). `gkg` is NOT wired; do not surface both.
56
+
57
+ ## The 6 deep-wired skills (offered inside the gate ritual)
58
+
59
+ These are wired into the per-stage prose because each sits at a gate where a miss is most
60
+ expensive and adds a verb no wired agent provides. **Wired ≠ required**: each is offered
61
+ (opt-in-with-prompt), degrades silently, and only informs the gate. Wiring locations:
62
+ ck-predict/ck-scenario → `gate-rules.md` (stages 04/05); review-pr + ck-security →
63
+ `adversarial-review.md` (the Review gate); retro → `law/RETRO.md`; ck-loop → the Build/Verify
64
+ tail, plumbing via `flow.sh loop-prep`/`loop-log` (this file).
65
+
66
+ 1. **ck-predict @ ADR** — before locking a non-trivial architecture decision, offer a 5-persona
67
+ debate. Catches arch/security/perf/UX defects when reversal is cheapest. Output informs the
68
+ ADR challenge; it does not pass it.
69
+ 2. **ck-scenario @ Contract** (and Scope/PRD) — offer 12-dimension edge-case decomposition to
70
+ harden the seam (`flow/05-contract.md`): the cases become acceptance criteria + the per-type
71
+ no-drift checks. Complements `/flow consistency` (it *generates* cases; consistency checks
72
+ *coherence*).
73
+ 3. **review-pr @ Review/Ship** — when the change lives as a GitHub PR, offer PR-context review
74
+ (AI-slop, CI-blocker, breaking-change) on top of the wired `code-reviewer` diff review.
75
+ 4. **ck-security @ security-class cards** — on the exact card class flow already halts at
76
+ (auth, authz, tenancy, payments, data migration, removing validation), offer a STRIDE+OWASP
77
+ threat-model. It informs; the **Tier-C operator HALT is never auto-passed** by a clean scan.
78
+ 5. **retro @ Retro** — offer a git-history numeric retrospective so the Retro line is backed by
79
+ real commit/velocity numbers, not prose.
80
+ 6. **ck-loop @ Build/Verify (Implement→Test→Audit→Fix tail)** — when a fix needs more than one
81
+ experimental attempt against a single numeric target (failing-test count, lint errors, etc.),
82
+ offer `flow.sh loop-prep <card>` to set up an isolated worktree + Verify/Guard commands, then
83
+ invoke the `ck-loop` skill with the printed block (ck-loop stays the untouched execution
84
+ engine — flow supplies plumbing only). See "Loop vs two-strikes" below for when to reach for
85
+ this instead of the default repair path. The finished run is recorded via `flow.sh loop-log`
86
+ (NOT the `intervention` channel below — recording it twice would double-count the same event).
87
+
88
+ ## Loop vs two-strikes — the one "fix it" decision tree
89
+
90
+ flow already has a bounded repair mechanism (two-strikes: one fresh-subagent repair attempt on a
91
+ review deadlock, then escalate to a cross-model lens — `adversarial-review.md`). `ck-loop` is a
92
+ different tool for a different situation; do not let operators reach for the wrong one:
93
+
94
+ | Situation | Use | Why |
95
+ |---|---|---|
96
+ | Review BLOCKED twice, same model, no metric | two-strikes → cross-model lens | bounded disagreement, not a number |
97
+ | One numeric verify command, needs >1 experimental attempt | `flow.sh loop-prep` + ck-loop | open iterate-to-target, git-tracked/revertable |
98
+ | Drive failing-tests / lint / perf count to a threshold | ck-loop | Verify is a single number; Direction lower |
99
+ | Single obvious fix, one retry | default auto repair | cheaper than spinning up a worktree+loop |
100
+
101
+ ## Don't surface (cut on purpose — these add noise, not signal)
102
+
103
+ - **Skill/agent twins** — the wired agent already covers these; surfacing the duplicate skill
104
+ doubles the "what do I use" confusion: research/researcher, ck-code-review/code-reviewer,
105
+ ck-debug/debugger, test/tester, git/git-manager, ck-plan/planner, docs/docs-manager,
106
+ journal/journal-writer, ask, brainstorm. Prefer the **agent** for in-stage execution; reach
107
+ for the skill only for a distinct verb the agent lacks (already captured above).
108
+ - **Competing orchestrators (cook, vibe, ship, bootstrap)** — each is its own end-to-end
109
+ pipeline with its own gates; invoking one inside a flow stage double-gates and fights flow's
110
+ stage authority. Cherry-pick a sub-step (e.g. ship's merge/PR) at the matching stage only.
111
+ - **`worktree` skill** — flow already ships `flow.sh workspace` (git-worktree-per-agent). Pure
112
+ duplicate; never wire it.
113
+ - **`bmad-spec` as a gate** — overlaps flow's own `/flow consistency` cross-artifact audit.
114
+ Keep it only as the optional Contract drafter already named in `agent-stage-mapping.md`.
115
+ - **Marketing/content/media skills** (ads, seo, social, email, video, branding, copywriting,
116
+ the `ckm:*`/`bmad-*` marketing families, cti-expert) — never relevant to a build gate.
117
+ - **Stack adapters** (backend/frontend/db/auth/payment/mobile/ui-* …) — surface only via
118
+ `playbooks/<stack>.md` by project-type, never as a flat global menu.
119
+
120
+ ## Lazy capture (ON — only at the 5 wired gates)
121
+
122
+ Skill-invocation telemetry is **enabled, lazily**: after a deep-wired skill runs at its gate,
123
+ record the use via the existing `flow.sh harness intervention add` (the same durable-metric
124
+ channel the Codex/Antigravity lenses use — `adversarial-review.md` §S2). **Only at the 5 wired
125
+ gates** — never on every skill, never on the `cmd_next`/`cmd_check` hot path, no new runner verb.
126
+ Note whether the skill caught a class the wired agent missed; that signal lets a later phase rank
127
+ the whitelist by what this operator actually reaches for (the `accessed_count` signal flow already
128
+ uses). The whitelist stays hand-curated until then.
@@ -0,0 +1,104 @@
1
+ # Codex — cross-vendor second engine (the seam)
2
+
3
+ `/flow` reaches OpenAI **Codex (GPT-5.x)** through the `openai-codex` Claude Code plugin
4
+ (detected, never required). Codex is a *second engine*, not a replacement for the ck:→bmad→
5
+ built-in ladder: a genuinely different model used at the moments where that is worth more than
6
+ another Claude pass. This file is the single source of truth for the seam; `agent-detection.md`
7
+ (detection + priority), `agent-stage-mapping.md` (per-stage use), `adversarial-review.md`
8
+ (cross-model review), and `auto-run.md` (Tier-B escalation) point here for the shapes.
9
+
10
+ ## Why a second vendor (the measured gap)
11
+
12
+ A single-vendor harness makes the builder and the reviewer share one model → correlated blind
13
+ spots pass green gates (this project's own review-pass-#4 caught an auth-skip a same-model pass
14
+ missed — `docs/quality-metrics.md`). Cross-model review empirically catches markedly more (a
15
+ gemini-cli study: single-model review 43% merge-ready vs cross-model iteration 91%). A different
16
+ engine is the cheapest way to close a same-vendor gate without weakening any gate.
17
+
18
+ ## Detection (I1) — two states: INSTALLED vs USABLE (never route on "installed" alone)
19
+
20
+ Detection has **two states**, because "installed" does not mean "works" — a plugin can be present
21
+ with no valid auth (common in headless/CI — see Codex issue #9253) or a broken companion script.
22
+ Routing into Codex on mere presence would spend a billable repair/review attempt and then fail at
23
+ invocation — that violates "absence never breaks a run". So:
24
+
25
+ - **INSTALLED** — `codex:codex-rescue` is in the host agent registry, OR the plugin dir exists
26
+ (`~/.claude/plugins/cache/openai-codex` / `.../marketplaces/openai-codex`). Necessary, not sufficient.
27
+ - **USABLE** — INSTALLED **and** a cheap, non-billable liveness/auth check passes:
28
+ `codex-companion.mjs setup --json` reports `ready === true` AND `auth.loggedIn === true`
29
+ (or a confirmed prior successful Codex call this session). Run this check **before the first
30
+ time** you would route work to Codex. NOTE: use `setup --json`, NOT `status` — `status` reports
31
+ runtime/job state but carries **no auth field**, so it passes even when Codex is unauthenticated
32
+ (it would route then fail at the real call). `setup --json` is the auth-aware, non-billable probe.
33
+
34
+ **Selection rule:** only select the Codex tier when state = **USABLE**. If INSTALLED-but-not-USABLE
35
+ (or absent): **degrade** to ck:→bmad→built-in, announce `"codex tier unavailable (installed but
36
+ not authenticated / not reachable) — degraded to <path>"`, and record the reason (durable metric).
37
+ Never select Codex on INSTALLED alone. Degrading is silent-but-announced, never an error, never a
38
+ gate change. Downstream docs (`agent-detection.md`, `auto-run.md`, `adversarial-review.md`) all
39
+ mean **USABLE** wherever they say "eligible".
40
+
41
+ ## Cost gate (F-D) — when Codex is allowed to fire
42
+
43
+ Codex calls are **billable** external GPT-5.x. Fire them ONLY at high-value moments:
44
+ - a **two-strikes deadlock** (a same-model agent BLOCKED twice),
45
+ - a **security-class card review** (auth/authorization/tenancy/payments/data-migration), or
46
+ - an **explicit operator opt-in** (e.g. "draft this stage on Codex", "/flow ... codex").
47
+
48
+ Never call Codex on every stage by default. Default engine stays ck:.
49
+
50
+ ## Invocation surfaces (from flow/05-contract.md — do not improvise other shapes)
51
+
52
+ | Use | Invocation | Effects | Returns |
53
+ |---|---|---|---|
54
+ | **Handoff / rescue / primary-draft** | `Task(subagent_type="codex:codex-rescue", prompt=<ScopedBrief>)` | writes code (`--write` default), billable | stdout + status line |
55
+ | **Cross-model review** | `node "$CODEX/scripts/codex-companion.mjs" review\|adversarial-review [--wait\|--background] [--base <ref>] [--scope auto\|working-tree\|branch] [focus]` | read-only review, billable | `ReviewResult` JSON |
56
+ | **Task (non-interactive)** | `node "$CODEX/scripts/codex-companion.mjs" task [--background] [--write] [--model <m\|spark>] [--effort <none..xhigh>] <prompt>` | writes if `--write`, billable | stdout + job id |
57
+ | **Job control** | `... status [job-id] [--all] [--json]` · `result [job-id] [--json]` · `cancel [job-id]` | read-only / cancel | job/result JSON |
58
+
59
+ `$CODEX` = the plugin root (`~/.claude/plugins/cache/openai-codex/codex/<ver>`). Prefer the
60
+ `codex:codex-rescue` **subagent** for handoffs (it owns prompt-shaping + foreground/background
61
+ choice); use `codex-companion.mjs` directly only for `review`/`adversarial-review`/job-control.
62
+
63
+ ### ScopedBrief (context isolation — orchestration-protocol)
64
+ Give Codex ONLY: `task` (one goal) · `read_for_context` (contract, law, the card, playbook) ·
65
+ `files_to_modify` (the card's Allowed files ONLY) · `acceptance` (the stage gate / card Verify) ·
66
+ `constraints` (contract is the seam; Vietnamese user-facing copy; touch only allowed files) ·
67
+ `return` (artifact + status). **Never** the session history.
68
+
69
+ ### ReviewResult (verbatim from `schemas/review-output.schema.json`)
70
+ ```
71
+ { verdict: "approve"|"needs-attention", summary: string,
72
+ findings: [ { severity:"critical"|"high"|"medium"|"low", title, body,
73
+ file, line_start>=1, line_end>=1, confidence:0..1, recommendation } ],
74
+ next_steps: [ string ] }
75
+ ```
76
+
77
+ ### Status protocol (handoff return)
78
+ `DONE | DONE_WITH_CONCERNS | BLOCKED | NEEDS_CONTEXT` — handle BLOCKED/NEEDS_CONTEXT before retry.
79
+
80
+ ## Gate parity (absolute) & auth (delegated)
81
+
82
+ - **Gate parity.** Codex DRAFTS or CRITIQUES; the identical stage gate (`flow.sh` + `gate-rules.md`)
83
+ still judges. A cross-model review INFORMS triage — it never auto-passes or auto-fails a card.
84
+ - **Auth.** Delegated entirely to the plugin (`codex login` / `OPENAI_API_KEY` / ChatGPT
85
+ subscription). `/flow` never reads, stores, or logs Codex credentials.
86
+ - **Data boundary (the other half of trust).** Selecting Codex SENDS the ScopedBrief — the source
87
+ diff plus the contract / PRD / law excerpts — to OpenAI's API, governed by the operator's OpenAI
88
+ plan and its data-retention/training terms. This is distinct from the credential clause above:
89
+ even with perfect secret handling, the *code and specs* leave the machine. For a sensitive,
90
+ regulated, or NDA'd codebase, the operator must opt in knowingly; the cost gate (high-value
91
+ moments only) keeps the exposure surface small by default.
92
+
93
+ ## Durable metric (S2)
94
+
95
+ After a Codex tier use, log it so the quality loop can read the trend (no new schema):
96
+ `flow.sh harness intervention add` (e.g. cross-model review found a class same-model missed) or
97
+ `intake` / `decision add`. This is the "thông số" that drives skill upgrades — see
98
+ `docs/quality-metrics.md` (cross-model-catch rate, rescue deflection rate).
99
+
100
+ ## Announce the path
101
+
102
+ Always tell the operator which engine ran: `"review via Codex cross-model lens (needs-attention,
103
+ 2 findings)"` / `"rescue via codex:codex-rescue after two strikes"` / `"codex tier unavailable —
104
+ degraded to code-reviewer"`. A run must stay legible.
@@ -0,0 +1,54 @@
1
+ # Command dispatch
2
+
3
+ Exact mapping: user input -> runner call -> your (Claude) duties. Always run the runner
4
+ first, relay its output faithfully, then do the semantic part. `<skill>` = this skill's
5
+ install dir; run from the project root so `flow/` and `cards/` resolve.
6
+
7
+ | User input | Runner call | Your duties after |
8
+ |---|---|---|
9
+ | `/flow` | `bash <skill>/runner/flow.sh status` | Runner now prints its own `NEXT ->` line (same helper as `resume`) plus stage dwell and, past 10 cards, a compact done/in-flight/todo summary. Relay it; nothing to author. |
10
+ | `/flow resume` | `bash <skill>/runner/flow.sh resume` | Read-only, no lock. Relay all four sections verbatim (last session, in-flight + dwell, gate state, NEXT ->). Run this FIRST when entering a project mid-cycle with no prior context this session. Never expand on or guess at raw args it didn't print — it deliberately shows command names only. |
11
+ | `/flow next` | `bash <skill>/runner/flow.sh next` | If FAIL: relay the exact violations + line numbers; offer help, don't author/tick in teach mode. If PASS: run the stage's challenge in `gate-rules.md`; flag hollow content, let operator decide. |
12
+ | `/flow assess` | `bash <skill>/runner/flow.sh assess` | Brownfield only: fill `flow/00-inspect.md` from EVIDENCE (read the code) — functionality/UI/UX vs product, risks, test baseline. Reuse `scout`/`researcher`. Gate is operator-reviewed; then proceed to `/flow next`. |
13
+ | `/flow card` | `bash <skill>/runner/flow.sh card` | Confirm the new card id; remind: fill Scope (one thing) / Allowed files / Verify / Done-evidence per `law/CLAUDE.md`. |
14
+ | `/flow card start C-NNN` | `bash <skill>/runner/flow.sh card start C-NNN` | Optional: mark the card in flight (operator-visible in-progress, shown in `/flow status`). Portable `cards/.inflight` registry; does NOT touch the gated `status:` field. Coexists with hand-edit. |
15
+ | `/flow card done C-NNN` | `bash <skill>/runner/flow.sh card done C-NNN` | Optional convenience: CLI-owned flip to `done`, gated by the SAME done-rules as `check` (reverts on fail — never a hollow done). Then still apply the semantic card review (diff-vs-scope, contract shapes, real evidence) as with `check`. |
16
+ | `/flow check C-NNN` | `bash <skill>/runner/flow.sh check C-NNN` | If PASS mechanically: review diff-vs-scope, allowed-files drift, contract-shape match, DESIGN.md for UI, evidence = real world-state. |
17
+ | `/flow contract` | `bash <skill>/runner/flow.sh contract` | After the contract gate / before UI cards (web): flags client base-URL vs served-path prefix drift (double-`/api`, mixed-prefix) — the class spec-diff tools miss. Advisory; confirm on the running app. |
18
+ | `/flow tokens` | `bash <skill>/runner/flow.sh tokens` | On/after UI cards: flags DESIGN.md tokens the CSS never uses + VALUE mismatches (same name, drifted value) + orphan CSS vars (info). Advisory; if the swap is intentional, record a dated DESIGN.md amendment. |
19
+ | `/flow coherence` | `bash <skill>/runner/flow.sh coherence` | Flags version drift across declared version fields (package.json / pyproject / src app_version). The cheap doc-vs-code slice; semantic contradictions stay a human gate-challenge. |
20
+ | `/flow consistency` | `bash <skill>/runner/flow.sh consistency` | After the contract gate, before cards: every PRD `FRn` is claimed by a card (`implements:`) and served by a contract interface; numeric success metric; no leftover placeholders. Then apply the semantic cross-artifact passes in `gate-rules.md` (hollow coverage, conflicting requirements, terminology drift). Advisory. |
21
+ | `/flow constitution` | `bash <skill>/runner/flow.sh constitution` | At the scope/PRD/contract seam: checks `flow/constitution.md` is well-formed (no placeholder, every invariant has an ID) + scans declared grep-markers. **Advisory, NOT a `next` gate** — then apply the semantic challenge in `gate-rules.md`. |
22
+ | `/flow eval [--stage 01\|02\|card] [--fixture <id>] [--n 3] [--timeout <s>]` | `bash <skill>/runner/flow.sh eval [...]` | **Billable** (skips cleanly, zero calls, if `claude` CLI absent). Relay the scorecard verbatim. See `references/gate-eval.md` before running: what it measures (a fresh-judge lower bound, not the work-mode self-challenge), cost, and thresholds. Never wired into `next`/`check` gating — informative only. |
23
+ | `/flow eval --report` | `bash <skill>/runner/flow.sh eval --report` | Offline, zero calls. Relay the last complete batch's scorecard + drift vs the prior complete batch. |
24
+ | `/flow project-type <web\|cli\|library\|skill>` | `bash <skill>/runner/flow.sh project-type [t]` | Set/read the project type that selects the per-type gate lens + done-evidence definition (`references/project-types.md`). Confirm the type matches reality before planning. |
25
+ | `/flow usage [--global\|--prune]` | `bash <skill>/runner/flow.sh usage [...]` | Roll up the JSONL usage log and relay the analytics (cycle-time, gate fail-rate, per-stage + per-card dwell, command breakdown). Local-only; `--prune [--keep N]` caps the log. Read-only — nothing to author. |
26
+ | `/flow skip <stage> --reason ...` | `bash <skill>/runner/flow.sh skip <stage> --reason ...` | Advance past a gate that has a matching open `DEBT.md` line. **Security-class skips are operator-only and HALT** — never auto-skip; the operator accepts the exposure in writing first. |
27
+ | `/flow debt add\|list` | `bash <skill>/runner/flow.sh debt add\|list` | Record/list deliberate gate-skips in `DEBT.md`. Security-class debt is operator-authored only; surface it at retro and before any real user touches the surface. |
28
+ | `/flow design <file>` | `bash <skill>/runner/flow.sh design <file>` | Mechanical DESIGN.md check on a UI file (emoji / `{{}}` / engine-words / gradient). Advisory; pair with the semantic `design-review-checklist.md` for mock/frontend cards. |
29
+ | `/flow harness <args>` | `bash <skill>/runner/flow.sh harness <args>` | Passthrough to the durable-layer CLI (intake/story/trace/decision/backlog/query). Visible output + real exit code. Use to write the durable hook after a stage/card; read it back with `recall`. |
30
+ | `/flow doctor` | `bash <skill>/runner/flow.sh doctor` | Environment/install self-check (bash/python/grep/git across macOS/Linux/Windows, runner path, Git Bash). Relay any FAIL as the fix list. Read-only. |
31
+ | `/flow promote <file>` | `bash <skill>/runner/flow.sh promote <file>` | Copy a hard-won playbook into the cross-project KB (`~/.claude/flow/playbooks`) so its lesson is surfaced by `recall` in every project, not just this one. |
32
+ | `/flow mode teach` | `bash <skill>/runner/flow.sh mode teach` | Confirm; you only gatekeep, operator authors. |
33
+ | `/flow mode work` | `bash <skill>/runner/flow.sh mode work` | Interview once, draft 00-05, pause for scope sign-off, deliver card set; still pass every gate. |
34
+ | `/flow ready` | `bash <skill>/runner/flow.sh ready` | Relay buildable cards; confirm allowed-files truly don't overlap before suggesting parallel. Operator dispatches. |
35
+ | `/flow workspace <verb>` | `bash <skill>/runner/flow.sh workspace add\|list\|enter\|remove\|check\|doctor [...]` | Multi-agent worktree isolation (human-driven, cross-vendor). Relay the runner output faithfully: on `add`, hand the operator the printed cd/env block (one worktree per agent — Claude `--worktree`/`-w`, Codex CLI manual + `CODEX_HOME`, Antigravity = open the dir as a workspace); on `check`/`doctor` exit 1, surface the collision/drift, don't auto-`--force` a `remove`. git is the source of truth; the `.flow/workspaces.jsonl` side-file only adds vendor/card/port/task. Advisory — not a `next` gate. |
36
+ | `/flow auto` | `bash <skill>/runner/flow.sh auto` | On preflight PASS, drive the autonomous run per SKILL.md AUTO principles (subagent/card, planner review, worktree, Tier-C halt on security debt). |
37
+ | `/flow recall` | `bash <skill>/runner/flow.sh recall` | Read back prior knowledge (open debt, recent retro, previous-card scope, harness friction/backlog, playbooks). Run at the START of a stage/card; apply it, don't re-learn known pain. |
38
+ | `/flow unlock` | `bash <skill>/runner/flow.sh unlock` | Clear this project's concurrency lock after a crashed/abandoned session. Confirm the other session is really gone first. |
39
+ | `/flow retro` | `bash <skill>/runner/flow.sh retro` | Ask the 3 questions; the operator writes the RETRO.md line — never you. |
40
+
41
+ ## Behavioral invariants (all commands)
42
+ 1. The runner's exit code is ground truth. Don't override it with optimism.
43
+ 2. In `teach` mode never tick a box or write an artifact for the operator.
44
+ 3. Never set a card `done` without pasted world-state evidence.
45
+ 4. Read any file the runner just created before editing it.
46
+ 5. Never edit `_templates/` or `runner/flow.sh` during a project run.
47
+ 6. Relay failures verbatim (line numbers included) — they are the operator's to-do list.
48
+ 7. If the runner reports BLOCKED by another session's lock, STOP and coordinate — never `FLOW_FORCE` past a live session; concurrent runs corrupt the plan. Set `FLOW_SESSION_ID` per session for hard protection.
49
+ 8. At the start of a stage or build card, run `/flow recall` first — its output (prior debt / retro / friction / previous-card scope) is context to apply, not noise. `status` shows a one-line memory summary; `card` injects the previous card's scope automatically.
50
+
51
+ ## Install-path note
52
+ - Project install: `bash .claude/skills/flow/runner/flow.sh <cmd>`
53
+ - Global install: `bash ~/.claude/skills/flow/runner/flow.sh <cmd>`
54
+ - Override project root: `FLOW_PROJECT_ROOT=/path bash <skill>/runner/flow.sh <cmd>`
@@ -0,0 +1,41 @@
1
+ # Debt & halts
2
+
3
+ Reordering or skipping a gate is a legitimate OPERATOR call (demo-first, riskiest-first).
4
+ But a skipped gate is a loan, and loans get written down. (buildflow CLAUDE.md "Debt".)
5
+
6
+ ## The DEBT.md ledger
7
+ Every deliberate skip opens one line (create `DEBT.md` on first use). Use the runner:
8
+ ```
9
+ bash <skill>/runner/flow.sh debt add "<what was skipped>" "<the exposure, concretely>" "<close before: named condition>"
10
+ bash <skill>/runner/flow.sh debt list # open debts
11
+ ```
12
+ Line shape:
13
+ ```
14
+ - [ ] DEBT: <what was skipped> -- <exposure> -- close before: <named condition> -- opened <date> (cards: C-NNN...)
15
+ ```
16
+
17
+ ## Security-class skips (never silent, never planner-decided)
18
+ auth · authorization · admin-surface exposure · tenancy · payments · data loss/migration ·
19
+ removing/weakening validation.
20
+ - The OPERATOR explicitly accepts the exposure, in writing, in the DEBT line. You do not
21
+ decide it for them.
22
+ - In `/flow auto` this is a **Tier-C HALT** — stop and ask.
23
+ - Closing a run with open security debt requires explicit operator acknowledgment.
24
+ "Temporary" is one forgotten step from production.
25
+
26
+ ## Close conditions
27
+ Checked at every `/flow retro` and before ANY real user touches the build. A card blocked
28
+ by a debt stays `todo` with PARTIAL evidence naming the debt — never half-done, never
29
+ rounded up to done.
30
+
31
+ ## Halts (when the run stops itself)
32
+ A run HALTS and reports — never silently continues — on any of:
33
+ - A hard-stop cap exceeded (iterations/tokens/wall-clock). (`loop-harness-2026-principles.md`)
34
+ - A red ground-truth signal that can't be repaired in two strikes. (`adversarial-review.md`)
35
+ - A Tier-C security-class touch. (above)
36
+ - A merge conflict during parallel builds (the allowed-files overlap check was gamed —
37
+ stop and re-plan). (`auto-run.md`)
38
+ - `BLOCKED` / `NEEDS_CONTEXT` from a subagent that more context can't resolve.
39
+
40
+ On halt: state what stopped it, what's done so far, the open debt/blocker, and the 2-4
41
+ concrete options for the operator. Let them decide — don't patch around a regression.
@@ -0,0 +1,42 @@
1
+ # Design review checklist (UI cards)
2
+
3
+ Every mock/frontend card is reviewed against `law/DESIGN.md` the same way backend cards are
4
+ reviewed against the contract. Two parts: a **mechanical** part (`flow.sh design <file>`,
5
+ grep-able, deterministic) and a **semantic** part (you, against DESIGN.md's structure laws).
6
+
7
+ ## Mechanical (run `flow.sh design <htmlfile>` — catches the never-do list)
8
+ Flags, with line numbers:
9
+ - **Emoji** anywhere (DESIGN.md: "No emojis. Anywhere. Ever.").
10
+ - **Raw `{{ }}` templates or visible JSON** outside a power surface.
11
+ - **Gradients on inputs / table rows / body bg** (gradients are hero-surface only).
12
+ - **Engine words** in user-facing copy: workflow, trigger, queue, webhook, agent, prompt,
13
+ cron, payload, endpoint (users think in THEIR objects, not engine concepts).
14
+ - **Design comments left in HTML.**
15
+ A flag is a strong signal, not always a verdict (an engine word inside a `<code>` sample is
16
+ fine) — you confirm. Exit 1 if any flag fires.
17
+
18
+ ## Semantic (you review against DESIGN.md — the script can't judge these)
19
+ - **Object-first:** the home page of a thing IS the thing; tabs are lenses on one object,
20
+ not separate features.
21
+ - **WYSIWYG / edit-in-place:** 80% of edits inline on the object page; a separate edit page
22
+ only for structural 20%. No multi-step wizard for editing.
23
+ - **Defaults beat configuration:** creatable in <=6 visible fields; the rest behind "More options".
24
+ - **Plain language:** "4 days after it ends", a field-picker chip — never cron, never raw templates.
25
+ - **Power behind a door:** any power surface is a `Simple | Pro` toggle that loses no data,
26
+ with a visible path back to simple.
27
+ - **Affordance ladder:** the field uses the lightest rung its shape allows (inline text ->
28
+ inline control -> popover composite -> modal). Empty value renders as `+ Add {label}`.
29
+ - **Luma object page:** calm pulse strip (no stat-tile cards), <=3 gradient-tinted hero
30
+ action cards (one click, no kebab), tabs as lenses, modal-first sub-actions.
31
+ - **Tokens:** Editorial Minimal tokens (or a deliberate, documented, whole-cluster override).
32
+ Inter body / Fraunces only for h1+titles+prominent stats / JetBrains Mono only for
33
+ IDs/dates/counts. 1px borders, no stacked shadows.
34
+ - **VN conventions:** Vietnamese native copy, `₫750,000` (symbol leading, comma groups),
35
+ VietQR as default pay, Zalo as a first-class support channel.
36
+
37
+ ## Process
38
+ 1. The UI **mock card** (static HTML, real copy, no logic) IS the design review — render it,
39
+ operator approves in a browser. Iterate here (mock retries cost seconds; framework retries
40
+ cost deploys). No frontend code before the mock is approved.
41
+ 2. Run `flow.sh design <file>` (mechanical) + this semantic pass on the mock and on every
42
+ frontend card. A violation blocks the card the same as a failing `## Verify`.
@@ -0,0 +1,186 @@
1
+ # Gate eval — behavioral proof for the semantic layer (`flow.sh eval`)
2
+
3
+ `gate-rules.md` is the semantic layer: after the mechanical gate passes, an LLM applies a
4
+ per-stage challenge and can flag a mechanically-clean artifact as substantively hollow. Until
5
+ this verb existed, nothing measured whether that actually happens — a hollow-but-well-formed
6
+ artifact passes the mechanical gate BY DESIGN (`flow.sh:scan_gate`/`cmd_check`), and the
7
+ semantic layer's only proof was operator trust. `flow.sh eval` runs the real per-stage ritual
8
+ text against curated hollow/sound fixture pairs and majority-votes a nonce-protected verdict,
9
+ so gate erosion (CLI update, model swap, prose edit) shows up as a number, not a vibe.
10
+
11
+ ## What it measures — and the honest limit on what it does NOT
12
+
13
+ The eval judge is a **fresh** LLM call: it receives the gate-rules.md challenge text plus one
14
+ fixture's content, cold, with no memory of authoring anything. This proves a **necessary, not
15
+ sufficient**, lower bound: if a fresh judge can't catch an obvious hollow artifact when
16
+ explicitly asked to, the semantic layer has failed outright. A green scorecard proves that
17
+ floor holds — it does **not** prove the semantic layer works during a real build.
18
+
19
+ The gap: in a real `/flow` session, the SAME model that just authored a stage artifact is the
20
+ one applying the challenge to it (`gate-rules.md:11` — "in `work` mode you authored it, so you
21
+ self-challenge before presenting"). That is a self-review, with the sunk-cost/authorship bias a
22
+ fresh judge never has. This eval does not reproduce that condition, and closing that gap is out
23
+ of scope for v1 (recorded design boundary, not an oversight). Read a green scorecard as "the
24
+ rule text is enforceable by *a* judge," not "this build's self-review caught everything."
25
+
26
+ ## Fixtures and thresholds
27
+
28
+ Six shipped fixtures (`skills/flow/eval/fixtures/`, manifest `skills/flow/eval/manifest.tsv`):
29
+ sound/hollow pairs for Stage 01 (Research — fabricated quotes, vague "users"), Stage 02 (Scope —
30
+ grade laundering, a real C quietly graded B), and the card gate ("merge ≈ shipped" evidence).
31
+ Fixture content maps 1:1 to the failure modes already documented in `gate-rules.md`, not an
32
+ invented taxonomy. The v1 input set is **manifest-listed shipped fixtures only** — `eval` never
33
+ accepts a caller-supplied artifact path; widening that is a v2 change that reopens security
34
+ review (an unvetted artifact reaching an LLM judge is a different trust boundary).
35
+
36
+ Per fixture, N=3 judge calls (`--n`), majority vote: **healthy = hollow fixtures flagged at
37
+ ≥2/3 majority, sound fixtures passed at majority**. More than 1/3 of a fixture's runs coming
38
+ back unparseable/timed-out makes that fixture's result `UNRELIABLE` (an infra failure, reported
39
+ separately from a real mismatch — never silently counted as a gate pass or fail).
40
+
41
+ ## Cost
42
+
43
+ `eval` is **opt-in and billable** — it makes real `claude -p` API calls. Zero cost when the
44
+ `claude` CLI isn't on `PATH` (clean skip, exit 0). When present, exactly one minimal probe call
45
+ is made to confirm the CLI runs headless before any real judging starts; a probe failure means
46
+ one billable call was made, then a clean skip. A full default batch is 6 fixtures × N=3 = 18
47
+ judge calls + the 1 probe = 19 calls.
48
+
49
+ **Measured real cost** (this machine, 2026-07-10, default/unforced model, no `--bare` available
50
+ under an OAuth/subscription session — see the build's spike notes): roughly **$0.30–0.37 per
51
+ call**, driven almost entirely by this repo's large global `CLAUDE.md` + skill/agent/MCP
52
+ declarations loading as system-prompt context on every call (~50–60K cache-creation tokens),
53
+ not by the judging prompt itself. A full 19-call baseline batch is real money, order **$6–7**
54
+ on this setup — not the trivial "a token or two" a casual reader might assume from "opt-in".
55
+ Forcing a cheaper model (`--model haiku` on the underlying CLI, not currently plumbed as a flow
56
+ flag) cuts this roughly 5x, but changes what the eval measures (a cheaper, possibly weaker judge)
57
+ and was deliberately left as an operator choice rather than a default, since the point of a
58
+ fresh-judge eval is to reflect a realistic judge, not the cheapest one that passes.
59
+
60
+ `--report` (below) is the free, offline way to re-read a prior batch's numbers without spending
61
+ anything.
62
+
63
+ ## Running it
64
+
65
+ ```
66
+ flow.sh eval # full manifest, N=3 per fixture
67
+ flow.sh eval --stage card # only the card-gate fixtures
68
+ flow.sh eval --fixture fcdb --n 1 # one fixture, one run (cheap smoke check)
69
+ flow.sh eval --timeout 180 # override the per-call timeout (default 120s)
70
+ flow.sh eval --report # OFFLINE: last complete batch's scorecard + drift
71
+ ```
72
+
73
+ Exit code: `0` = every evaluated fixture majority-matched its expected verdict (or a clean
74
+ skip); `1` = any mismatch, any `UNRELIABLE` fixture, or nothing matched the given filters. This
75
+ exit code is **informative only** — it is never wired into `next`/`check` gating. A red `eval`
76
+ run does not block a build; it tells the operator the semantic layer needs attention.
77
+
78
+ ## Results, scorecard, and drift
79
+
80
+ Every batch appends to `$FLOW_PROJECT_ROOT/.flow/eval-results.jsonl` (per-fixture rows plus a
81
+ `batch:"start"`/`batch:"done"` header/trailer pair). A batch with no trailer — Ctrl-C mid-run,
82
+ the only signal handled is best-effort (see Limitations) — is simply excluded from `--report`
83
+ and drift comparisons; that absence of a trailer IS the completeness signal, not a positive
84
+ "torn" flag. Reruns append a new `run_id`; there is no in-place dedup or pruning (personal
85
+ volume, same stance as the existing global usage-log rotation backlog — documented, not built).
86
+
87
+ Each result row carries `cli_version` (from `claude --version` — the CLI's own version string,
88
+ **not** a model id) and `model` (the served model id, parsed from the JSON response's own
89
+ `model` field — falls back to `"unknown"` if that source is ever unavailable; the two are kept
90
+ strictly separate so a CLI update is never mistaken for a model swap or vice versa), plus
91
+ `flow_version` and a CRLF-normalized `gate_rules_sha` (so a checkout's line-ending convention
92
+ never produces a false "the rules changed" alarm).
93
+
94
+ `--report` prints the last complete batch's scorecard (per-fixture MATCH/MISMATCH/UNRELIABLE,
95
+ per-stage hollow-flag-rate/sound-pass-rate) and an advisory drift line against the prior
96
+ complete batch: changes in `cli_version`/`model`/`gate_rules_sha`, plus the per-stage flag-rate
97
+ delta. Drift is advisory text only, never an exit-code signal. If the two compared batches
98
+ evaluated different fixture sets for a stage (e.g. a quick `--fixture`-filtered check against an
99
+ earlier full baseline), the delta is flagged as "not directly comparable" rather than presented
100
+ as a clean number — a shrinking/growing denominator is not the same signal as a judge's answer
101
+ changing. With `model:"unknown"` on either side, drift is explicitly narrowed to the
102
+ `cli_version`/prose axes.
103
+
104
+ ## Why `eval` is not `read_only` in telemetry
105
+
106
+ `eval` is the only verb that makes billable network calls AND writes a results file
107
+ (`eval-results.jsonl`); every other read-only verb (`status`, `usage`, `doctor`, ...) only reads
108
+ existing state. Marking it `read_only:true` in the standard usage-log would be a telemetry lie,
109
+ so it is deliberately excluded from `_log_is_readonly`'s whitelist. `--report` alone genuinely
110
+ is read-only (zero calls, offline) but is not special-cased in telemetry, since the standard
111
+ event still correctly reflects "a build-affecting command ran" at the `eval`-verb level.
112
+
113
+ ## v2 engine seam (documented, not built)
114
+
115
+ `_eval_engine_run()` is the single call site that invokes the judge CLI. A second/third engine
116
+ (Codex, Antigravity) would implement the same nonce-in/verdict-out contract behind that one
117
+ function, without touching the fixture loop, majority-vote, or scorecard logic. Not built in v1
118
+ — Claude-only was the approved scope, and no other engine's headless contract has been spiked
119
+ yet (the same "measure, don't assume" discipline that gated this build in the first place).
120
+
121
+ ## Limitations (recorded, not hidden)
122
+
123
+ - **Fixture authorship bias**: the same people who wrote `gate-rules.md`'s challenges also wrote
124
+ the fixtures meant to test it. Mitigated by sourcing every hollow fixture's failure mode
125
+ verbatim from the rule text it's tested against (never an invented taxonomy); residual risk is
126
+ accepted for v1 and disclosed here.
127
+ - **Interrupt cleanup is best-effort, not guaranteed**, on at least this build's development
128
+ platform: a signal arriving while the runner is blocked on a foreground judge call was not
129
+ observed to preempt that call promptly. Cleanup between fixtures/calls fires reliably; a
130
+ normal (uninterrupted) run always cleans up its temp files immediately.
131
+ - **Small fixture corpus (six)**: enough to prove the mechanism and give a real number, not
132
+ enough for statistical confidence across every stage/failure-mode combination. Widening the
133
+ corpus is a natural v1.x follow-up, not a v1 blocker.
134
+
135
+ ## Failure modes and postmortem (v0.21)
136
+
137
+ ### INVALID storms — the 260710 case
138
+
139
+ The first real batch after v0.19 shipped came back **17/18 INVALID** (run `…-1783695631-…`). The
140
+ retry doubled cost without recovery — attempt 2 landed in the same window — and the CLI's
141
+ `session-lifecycle-hook cancelled` stderr line, the only diagnostic the storm actually produced,
142
+ was thrown away because the pre-v0.21 seam captured stdout only. A single-call retry cleared the
143
+ window; the mechanism was never confirmed at the time (rate-limit vs hook contention).
144
+
145
+ **v0.21 changes that make the next storm postmortemable and cost-safe:**
146
+
147
+ 1. **Raw capture on final-INVALID.** Both attempts' stdout, stderr, and rc are persisted to
148
+ `.flow/eval-raw/<run_id>/<fixture>-v<vote>-a<attempt>.{out,err,rc}` (git-ignored via
149
+ `_ignore_run_state`; envelope stripped down to `result`/`assistant`/`rate_limit_event`
150
+ records — cwd, session_id, plugin/memory paths are removed).
151
+ 2. **Injectable backoff.** `FLOW_EVAL_RETRY_BACKOFF` env (default 5s) delays the retry so it
152
+ doesn't hammer the same window. Tests set `0` to keep CI fast.
153
+ 3. **Retry skip on rate-limit.** If attempt 1 carried a non-`allowed` `rate_limit_info.status`,
154
+ the retry is skipped (retry is documented for a *formatting slip*, not infra; retrying into a
155
+ live rate-limit window just doubles spend).
156
+ 4. **Circuit breaker.** The FIRST evaluated fixture coming back UNRELIABLE (`invalid_count*3 > n`)
157
+ aborts the batch with a distinct nonzero exit code (2) and NO `done` trailer written — the
158
+ junk batch is invisible to `--report`/drift, so it cannot poison the baseline. `--keep-going`
159
+ forces the full batch; document worst case (≤ N_fixtures × n × 2 + 1 probe ≈ 37 calls at
160
+ default 6 × 3).
161
+
162
+ ### Rate-limit visibility (advisory, best-effort)
163
+
164
+ The rows persisted after v0.21 gain `retries` (0/1 per fixture) and `rate_limited`
165
+ (`true`/`false` per fixture). `rate_limited` is anchored to the `rate_limit_info` record's own
166
+ `status` value — this is important because on cli 2.1.201 a **healthy** `allowed` event carries
167
+ `"overageStatus":"rejected"` as a separate field in the same envelope, and a naive
168
+ `grep '"status":"' | grep -v allowed` false-positives on it. A genuinely-throttled shape has not
169
+ yet been captured; treat `rate_limited` as best-effort advisory (absence ≠ not-throttled) until a
170
+ real throttled sample is added to the corpus.
171
+
172
+ ### Concurrency caveat
173
+
174
+ `cmd_eval` takes **no** flow lock (it is a read-mostly verb, and the raw/prune writes are the
175
+ only cross-run writes it does). The pre-batch raw-prune keys off the epoch embedded in `run_id`
176
+ (deterministic, mount-independent, unforgeable by `touch`) and honors a `FLOW_LOCK_TTL`-second
177
+ guard so an in-flight or in-postmortem run's dir cannot be pruned under the operator's feet.
178
+
179
+ ### INVALID-storm playbook
180
+
181
+ 1. If `flow.sh eval` exits `2` and prints `ABORT after first fixture UNRELIABLE`, do NOT rerun
182
+ without `--keep-going` — read the named `raw capture` dir first.
183
+ 2. Attempt-1 `.err` is usually the diagnostic (hook-cancelled / rate-limit line lands there).
184
+ 3. If the storm was rate-limit: wait past the window (typically minutes), then rerun.
185
+ 4. If persistent: capture the `.err` line in a comment, escalate — this is when a throttled
186
+ `rate_limit_info` sample would land in the corpus and let `rate_limited` become authoritative.