@manhquy/flow-skill 0.7.3 → 0.7.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +30 -0
- package/README.md +12 -5
- package/README_VN.md +12 -5
- package/package.json +1 -1
- package/skills/flow/SKILL.md +2 -2
- package/skills/flow/eval/fixtures/f06_injected/flow/05-contract.md +28 -0
- package/skills/flow/eval/fixtures/f06_sound/flow/05-contract.md +24 -0
- package/skills/flow/eval/manifest.tsv +2 -0
- package/skills/flow/harness/flow_harness.py +153 -1
- package/skills/flow/manifest-hashes.json +3 -0
- package/skills/flow/references/adversarial-review.md +6 -5
- package/skills/flow/references/agent-detection.md +8 -4
- package/skills/flow/references/agent-stage-mapping.md +14 -12
- package/skills/flow/references/claudekit-skills.md +48 -18
- package/skills/flow/references/command-dispatch.md +7 -6
- package/skills/flow/references/flow-catalog.tsv +5 -5
- package/skills/flow/references/gate-04.md +2 -2
- package/skills/flow/references/gate-05.md +2 -2
- package/skills/flow/references/gate-consistency.md +12 -0
- package/skills/flow/references/gate-eval.md +34 -0
- package/skills/flow/references/host-agnostic-parallel.md +79 -0
- package/skills/flow/references/native-rituals.md +3 -3
- package/skills/flow/references/pin-v1.txt +5 -0
- package/skills/flow/runner/attestations.sh +1 -1
- package/skills/flow/runner/flow.sh +531 -65
- package/skills-manifest.json +5 -2
|
@@ -1,12 +1,13 @@
|
|
|
1
1
|
# claudekit skills — the per-stage capability map (the seam)
|
|
2
2
|
|
|
3
|
-
`/flow` already orchestrates
|
|
4
|
-
`agent-stage-mapping.md`:
|
|
5
|
-
same seam to the **skill layer** — the curated set of
|
|
6
|
-
|
|
7
|
-
~
|
|
8
|
-
|
|
9
|
-
This is the single source of truth for the skill map;
|
|
3
|
+
`/flow` already orchestrates specialists at the **agent layer** (`agent-detection.md` +
|
|
4
|
+
`agent-stage-mapping.md`: host-native / ck: agents, then bmad, then built-in). This file
|
|
5
|
+
extends that same seam to the **skill layer** — the curated set of build skills worth
|
|
6
|
+
reaching for at each stage. It is a **whitelist, not a dump**: ClaudeKit (~87) and
|
|
7
|
+
AgentKit (~100+) both ship marketing/content noise. Only the skills below ever get
|
|
8
|
+
surfaced. ClaudeKit `ck-*` and AgentKit `ak:*` names are the **same rows** (Name
|
|
9
|
+
resolution). This is the single source of truth for the skill map;
|
|
10
|
+
`agent-stage-mapping.md` points here.
|
|
10
11
|
|
|
11
12
|
## Standalone note (v0.22)
|
|
12
13
|
|
|
@@ -32,6 +33,33 @@ review natively.
|
|
|
32
33
|
missing skill **never lowers a gate** — it only changes whether the enrichment is offered.
|
|
33
34
|
Rich where the skill exists, unbroken where it doesn't.
|
|
34
35
|
|
|
36
|
+
|
|
37
|
+
## Name resolution (ClaudeKit + AgentKit)
|
|
38
|
+
|
|
39
|
+
Host registries spell the same skill three ways: `ck-<slug>`, `ak:<slug>` /
|
|
40
|
+
`ak-<slug>`, or bare `<slug>`. Strip `ck:` / `ck-` / `ak:` / `ak-` and match the
|
|
41
|
+
slug. Offer the **first listed** alias; never offer two names for one verb.
|
|
42
|
+
AgentKit is not a dependency — absence degrades to the native ritual.
|
|
43
|
+
|
|
44
|
+
| Whitelist row | Also detect |
|
|
45
|
+
|---|---|
|
|
46
|
+
| ck-predict | ak:predict, ak-predict, predict |
|
|
47
|
+
| ck-scenario | ak:scenario, ak-scenario, scenario |
|
|
48
|
+
| ck-security | ak:security, ak-security |
|
|
49
|
+
| ck-loop | ak:loop, ak-loop, loop |
|
|
50
|
+
| ck-graphify | ak:graphify, ak-graphify, graphify |
|
|
51
|
+
| review-pr | ak:review-pr, ak-review-pr |
|
|
52
|
+
| security-scan | ak:security-scan, ak-security-scan |
|
|
53
|
+
| retro | ak:retro, ak-retro |
|
|
54
|
+
| xia | ak:xia, ak-xia |
|
|
55
|
+
| deploy | ak:deploy, ak-deploy |
|
|
56
|
+
| web-testing | ak:web-testing, ak-web-testing |
|
|
57
|
+
| docs-seeker | ak:docs-seeker, ak-docs-seeker |
|
|
58
|
+
| scout | ak:scout, ak-scout |
|
|
59
|
+
| repomix | ak:repomix, ak-repomix |
|
|
60
|
+
|
|
61
|
+
`gkg` / `ak:gkg` stay unwired (single graph pick = ck-graphify / ak:graphify).
|
|
62
|
+
|
|
35
63
|
## Cost gate — offer, don't impose
|
|
36
64
|
|
|
37
65
|
The high-value skills below are **opt-in-with-prompt**: at the matching gate you *offer* the
|
|
@@ -91,10 +119,10 @@ tail, plumbing via `flow.sh loop-prep`/`loop-log` (this file).
|
|
|
91
119
|
6. **ck-loop @ Build/Verify (Implement→Test→Audit→Fix tail)** — when a fix needs more than one
|
|
92
120
|
experimental attempt against a single numeric target (failing-test count, lint errors, etc.),
|
|
93
121
|
offer `flow.sh loop-prep <card>` to set up an isolated worktree + Verify/Guard commands, then
|
|
94
|
-
invoke
|
|
95
|
-
engine — flow supplies plumbing only). See "Loop vs two-strikes" below
|
|
96
|
-
|
|
97
|
-
|
|
122
|
+
invoke `ck-loop` or `ak:loop` with the printed block (the loop skill stays the untouched
|
|
123
|
+
execution engine — flow supplies plumbing only). See "Loop vs two-strikes" below. The
|
|
124
|
+
finished run is recorded via `flow.sh loop-log` (NOT the `intervention` channel below —
|
|
125
|
+
recording it twice would double-count the same event).
|
|
98
126
|
|
|
99
127
|
## Loop vs two-strikes — the one "fix it" decision tree
|
|
100
128
|
|
|
@@ -105,20 +133,22 @@ different tool for a different situation; do not let operators reach for the wro
|
|
|
105
133
|
| Situation | Use | Why |
|
|
106
134
|
|---|---|---|
|
|
107
135
|
| Review BLOCKED twice, same model, no metric | two-strikes → cross-model lens | bounded disagreement, not a number |
|
|
108
|
-
| One numeric verify command, needs >1 experimental attempt | `flow.sh loop-prep` + ck-loop | open iterate-to-target, git-tracked/revertable |
|
|
109
|
-
| Drive failing-tests / lint / perf count to a threshold | ck-loop | Verify is a single number; Direction lower |
|
|
136
|
+
| One numeric verify command, needs >1 experimental attempt | `flow.sh loop-prep` + ck-loop / ak:loop | open iterate-to-target, git-tracked/revertable |
|
|
137
|
+
| Drive failing-tests / lint / perf count to a threshold | ck-loop / ak:loop | Verify is a single number; Direction lower |
|
|
110
138
|
| Single obvious fix, one retry | default auto repair | cheaper than spinning up a worktree+loop |
|
|
111
139
|
|
|
112
140
|
## Don't surface (cut on purpose — these add noise, not signal)
|
|
113
141
|
|
|
114
142
|
- **Skill/agent twins** — the wired agent already covers these; surfacing the duplicate skill
|
|
115
|
-
doubles the "what do I use" confusion: research/researcher, ck-code-review/code-
|
|
116
|
-
ck-debug/debugger, test/tester, git/git
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
143
|
+
doubles the "what do I use" confusion: research/researcher, ck-code-review / ak:code-review /
|
|
144
|
+
code-reviewer, ck-debug / ak:debug / debugger, test / ak:test / tester, git / ak:git /
|
|
145
|
+
git-manager, ck-plan / ak:plan / planner, docs / ak:docs / docs-manager, journal /
|
|
146
|
+
ak:journal / journal-writer, ask, brainstorm. Prefer the **agent** for in-stage execution;
|
|
147
|
+
reach for the skill only for a distinct verb the agent lacks (already captured above).
|
|
148
|
+
- **Competing orchestrators (cook, vibe, ship, bootstrap, agentkit, plan, orchestrate, team, handover, issue-to-plan, fix)** — each is its own end-to-end
|
|
120
149
|
pipeline with its own gates; invoking one inside a flow stage double-gates and fights flow's
|
|
121
150
|
stage authority. Cherry-pick a sub-step (e.g. ship's merge/PR) at the matching stage only.
|
|
151
|
+
`ak:agentkit` routes *away* from flow — never invoke it mid-gate.
|
|
122
152
|
- **`worktree` skill** — flow already ships `flow.sh workspace` (git-worktree-per-agent). Pure
|
|
123
153
|
duplicate; never wire it.
|
|
124
154
|
- **`bmad-spec` as a gate** — overlaps flow's own `/flow consistency` cross-artifact audit.
|
|
@@ -9,8 +9,8 @@ is no fragment loader (ADR-0001).
|
|
|
9
9
|
|
|
10
10
|
| User input | Runner call | Host duties after |
|
|
11
11
|
|---|---|---|
|
|
12
|
-
| `/flow` | `bash <skill>/runner/flow.sh status` | Relay prose `NEXT ->`, dwell, and (past 10 cards) the compact summary; nothing to author. Optional `status --json` prints one `flow_context/v1` object (`v,stage,gate,next_verb,card,dwell,load`) — **display-only**. Hosts MUST NOT dispatch `next_verb` (`next`/`card`/`card-start`/`check`/`fix-gate`/`auto`/`skip`). Default `/flow` stays prose status. |
|
|
13
|
-
| `/flow resume` | `bash <skill>/runner/flow.sh resume` | Read-only session-story; no lock. Optional `resume --json` same closed schema as status; display-only, never exec `next_verb`. FAIL/PASS/Never: SKILL.md STOP + Dispatch. |
|
|
12
|
+
| `/flow` | `bash <skill>/runner/flow.sh status` | Relay prose `NEXT ->`, PIN v1 after NEXT_VERB (display-only, do not exec), dwell, and (past 10 cards) the compact summary; nothing to author. Compact recovery is prose status/resume, not `--json`. Optional `status --json` prints one `flow_context/v1` object (`v,stage,gate,next_verb,card,dwell,load`) — **display-only**. Hosts MUST NOT dispatch `next_verb` (`next`/`card`/`card-start`/`check`/`fix-gate`/`auto`/`skip`). Default `/flow` stays prose status. |
|
|
13
|
+
| `/flow resume` | `bash <skill>/runner/flow.sh resume` | Read-only session-story; no lock. PIN v1 after NEXT_VERB, display-only, do not exec. Compact recovery is prose status/resume, not `--json`. Optional `resume --budget` caps in-flight cards at 5 with count line; `resume --json` same closed schema as status; display-only, never exec `next_verb`. FAIL/PASS/Never: SKILL.md STOP + Dispatch. |
|
|
14
14
|
| `/flow next` | `bash <skill>/runner/flow.sh next` | After PASS, compare to `gate-examples.md` (not a second always-on load). FAIL/Never: SKILL.md STOP + Dispatch. |
|
|
15
15
|
| `/flow assess` | `bash <skill>/runner/flow.sh assess` | Fill `flow/00-inspect.md` from EVIDENCE; Brownfield challenge in `gate-rules.md`. |
|
|
16
16
|
| `/flow card` | `bash <skill>/runner/flow.sh card` | Fill per `law/CLAUDE.md`; before coding, `law/CODING.md`. |
|
|
@@ -21,25 +21,26 @@ is no fragment loader (ADR-0001).
|
|
|
21
21
|
| `/flow contract` | `bash <skill>/runner/flow.sh contract` | Advisory client base-URL vs served-path drift; confirm on the running app. |
|
|
22
22
|
| `/flow tokens` | `bash <skill>/runner/flow.sh tokens` | Advisory DESIGN.md token unused/mismatch; dated amendment if intentional. |
|
|
23
23
|
| `/flow coherence` | `bash <skill>/runner/flow.sh coherence` | Advisory version-field drift; semantic contradictions stay a human challenge. |
|
|
24
|
-
| `/flow consistency` | `bash <skill>/runner/flow.sh consistency` | Advisory PRD FRn coverage;
|
|
24
|
+
| `/flow consistency` | `bash <skill>/runner/flow.sh consistency [--contract]` | Advisory PRD FRn coverage or contract endpoint audit (`--contract`); see `references/gate-consistency.md`. |
|
|
25
25
|
| `/flow constitution` | `bash <skill>/runner/flow.sh constitution` | Advisory constitution form + markers; then `gate-rules.md`. Not a `next` gate. |
|
|
26
26
|
| `/flow clarify` | `bash <skill>/runner/flow.sh clarify` | List leftover Open-decision boxes; write-back `references/clarify.md`. Not a `next` gate. |
|
|
27
27
|
| `/flow converge` | `bash <skill>/runner/flow.sh converge [--file <payload>]` | Assess vs plan per `references/converge.md`; present the findings table first. |
|
|
28
28
|
| `/flow eval [--stage 01\|02\|05\|card] [--fixture <id>] [--n 3] [--timeout <s>]` | `bash <skill>/runner/flow.sh eval [...]` | Billable; live mode skips if `claude` absent. `--replay` does not inherit that SKIP. See `references/gate-eval.md`. |
|
|
29
29
|
| `/flow eval --report` | `bash <skill>/runner/flow.sh eval --report` | Offline; relay last complete batch + drift. See `references/gate-eval.md`. |
|
|
30
30
|
| `/flow eval --replay` | `bash <skill>/runner/flow.sh eval --replay` | Keyless replay; missing/stale fixtures exit 1. Verdicts never count toward the eval floor. |
|
|
31
|
+
| `/flow eval --diff <old> <new>` | `bash <skill>/runner/flow.sh eval --diff <old> <new>` | Offline; fixture regression classifier vs baseline. See `references/gate-eval.md`. |
|
|
31
32
|
| `/flow project-type <web\|cli\|library\|skill>` | `bash <skill>/runner/flow.sh project-type [t]` | Set/read type per `references/project-types.md`. Confirm before planning. |
|
|
32
33
|
| `/flow usage [--global\|--prune]` | `bash <skill>/runner/flow.sh usage [...]` | Relay local JSONL analytics; `--prune` caps the log. Nothing to author. |
|
|
33
34
|
| `/flow skip <stage> --reason ...` | `bash <skill>/runner/flow.sh skip <stage> --reason ...` | Advance only with matching `DEBT.md`. Security-class: operator-only HALT. |
|
|
34
35
|
| `/flow debt add\|list` | `bash <skill>/runner/flow.sh debt add\|list` | Record/list skips in `DEBT.md`. Security-class is operator-authored only. |
|
|
35
36
|
| `/flow design <file>` | `bash <skill>/runner/flow.sh design <file>` | Advisory DESIGN.md mechanical check; pair `design-review-checklist.md` for UI cards. |
|
|
36
37
|
| `/flow harness <args>` | `bash <skill>/runner/flow.sh harness <args>` | Passthrough to the durable-layer CLI; visible output + real exit. Use `recall` to read back. |
|
|
37
|
-
| `/flow doctor` | `bash <skill>/runner/flow.sh doctor` | Relay FAIL as the fix list. Read-only. |
|
|
38
|
-
| `/flow promote <file>` | `bash <skill>/runner/flow.sh promote <file
|
|
38
|
+
| `/flow doctor` | `bash <skill>/runner/flow.sh doctor` | Relay FAIL as the fix list. Read-only. Also verifies runner integrity against the manifest hash (loud FAIL on drift; skipped on older installs without it). |
|
|
39
|
+
| `/flow promote <file>` | `bash <skill>/runner/flow.sh promote <file> \| --suggest` | Copy playbook into `~/.claude/flow/playbooks` for cross-project `recall`, or suggest draft playbooks (`--suggest`) whose actions are derived from recorded harness patterns (>=2 occurrences); no generic advice is invented. |
|
|
39
40
|
| `/flow mode teach` | `bash <skill>/runner/flow.sh mode teach` | Confirm; host gatekeeps, operator authors. |
|
|
40
41
|
| `/flow mode work` | `bash <skill>/runner/flow.sh mode work` | Interview once, draft 00-05, pause for scope sign-off; same gates. See `references/mode-work.md`. |
|
|
41
42
|
| `/flow ready` | `bash <skill>/runner/flow.sh ready` | Relay buildable cards; operator dispatches. Confirm allowed-files don't overlap. |
|
|
42
|
-
| `/flow workspace <verb>` | `bash <skill>/runner/flow.sh workspace add\|list\|enter\|remove\|check\|doctor [...]` | Relay worktree isolation output; don't auto-`--force` a `remove`. Advisory. |
|
|
43
|
+
| `/flow workspace <verb>` | `bash <skill>/runner/flow.sh workspace add\|list\|enter\|remove\|check\|doctor\|port [...]` | Relay worktree isolation output; don't auto-`--force` a `remove`. `workspace port` allocates deterministic per-card port (`references/host-agnostic-parallel.md`). Advisory. |
|
|
43
44
|
| `/flow auto` | `bash <skill>/runner/flow.sh auto` | Load `references/auto-run.md`. |
|
|
44
45
|
| `/flow auto stop` | `bash <skill>/runner/flow.sh auto stop` | Clear auto policy; return to warning-only manual path. |
|
|
45
46
|
| `/flow attest …` | `bash <skill>/runner/flow.sh attest semantic\|live-verify\|status\|recover …` | Mint/inspect receipts (`references/attestations.md`). Must-ask before mint/recover. |
|
|
@@ -7,14 +7,14 @@ brownfield-assess existing-codebase-no-flow-history assess brownfield current-st
|
|
|
7
7
|
kill-doubt-idea stage-00-or-01 status offers the forge-idea ritual (references/forge-idea.md) at Idea/Scope: persona-driven interrogation until it hardens, proves out, or dies cheaply - flow
|
|
8
8
|
review-ask cards-built-pending-review check card review path: diff-vs-scope, allowed-files drift, contract shapes, evidence - flow
|
|
9
9
|
deploy-ask review-passed status flow has no deploy verb; status shows the Deploy stage; execution stays operator/CI - flow
|
|
10
|
-
retro-ask stage-verify-or-later retro runs the native numeric-retro ritual first (git-log recipes), then offers the operator the line retro flow
|
|
10
|
+
retro-ask stage-verify-or-later retro runs the native numeric-retro ritual first (git-log recipes), then offers the operator the line retro|ak:retro flow
|
|
11
11
|
usage-metrics-ask any usage cycle-time, gate fail-rate, per-stage dwell, command breakdown - flow
|
|
12
12
|
parallel-multi-agent-ask any ready buildable + parallel-safety hint before offering workspace add - flow
|
|
13
13
|
help-confused any status ground truth first, then explain the gate concept in one plain sentence - flow
|
|
14
14
|
card-create-ask all-planning-gates-passed card create next build card; reminds Scope/Allowed-files/Verify/Done-evidence - flow
|
|
15
|
-
adr-decision-ask stage-04 next offers the native persona-debate ritual before locking a non-trivial ADR decision ck-predict flow
|
|
16
|
-
contract-hardening-ask stage-05 contract offers the native edge-case decomposition ritual to harden the seam before the Contract gate ck-scenario flow
|
|
17
|
-
security-review-ask security-class-card check runs the native STRIDE ritual on every security-class card (guaranteed baseline) ck-security flow
|
|
18
|
-
loop-fix-ask build-verify-numeric-gap status offers the native loop protocol (flow.sh loop-prep/loop-log) when a fix needs more than one attempt against a numeric target ck-loop flow
|
|
15
|
+
adr-decision-ask stage-04 next offers the native persona-debate ritual before locking a non-trivial ADR decision ck-predict|ak:predict flow
|
|
16
|
+
contract-hardening-ask stage-05 contract offers the native edge-case decomposition ritual to harden the seam before the Contract gate ck-scenario|ak:scenario flow
|
|
17
|
+
security-review-ask security-class-card check runs the native STRIDE ritual on every security-class card (guaranteed baseline) ck-security|ak:security flow
|
|
18
|
+
loop-fix-ask build-verify-numeric-gap status offers the native loop protocol (flow.sh loop-prep/loop-log) when a fix needs more than one attempt against a numeric target ck-loop|ak:loop flow
|
|
19
19
|
open-decision-ask stage-02-or-03-or-05 clarify lists ## Open decisions bullets; skill offers references/clarify.md write-back (opt-in, never a next prereq) - flow
|
|
20
20
|
close-the-gap-ask cards-done-verify-plan converge assess present code vs plan per references/converge.md, then append-only remainder cards (transactional; CONVERGED writes nothing) - flow
|
|
@@ -8,5 +8,5 @@ NOT-doing list honest about what's deferred?
|
|
|
8
8
|
debate (architect/security/ops/user-advocate/cost) that surfaces defects while reversal
|
|
9
9
|
is still cheap. Output INFORMS this challenge; it never passes the gate. Skip on a
|
|
10
10
|
trivial ADR.
|
|
11
|
-
- **If `ck-predict` is installed**, it is a richer alternative to the native ritual (same
|
|
12
|
-
INFORMS-only rule, opt-in-with-prompt — see `claudekit-skills.md`).
|
|
11
|
+
- **If `ck-predict` or `ak:predict` is installed**, it is a richer alternative to the native ritual (same
|
|
12
|
+
INFORMS-only rule, opt-in-with-prompt — see `claudekit-skills.md` Name resolution).
|
|
@@ -28,6 +28,6 @@ cli=command+flags+output/exit, library=public function+args+return, skill=comman
|
|
|
28
28
|
no-drift check, so the contract is exhaustive before any card trusts it. Complements
|
|
29
29
|
`/flow consistency` (it *generates* cases; consistency checks *coherence*). INFORMS
|
|
30
30
|
the gate; never auto-passes it.
|
|
31
|
-
- **If `ck-scenario` is installed**, it is a richer 12-dimension alternative to the
|
|
31
|
+
- **If `ck-scenario` or `ak:scenario` is installed**, it is a richer 12-dimension alternative to the
|
|
32
32
|
native ritual (same INFORMS-only rule, opt-in-with-prompt — see
|
|
33
|
-
`claudekit-skills.md`).
|
|
33
|
+
`claudekit-skills.md` Name resolution).
|
|
@@ -22,3 +22,15 @@ declared in the PRD's **`## Features`** section (ids in prose or the pain table
|
|
|
22
22
|
ignored so a legacy mention can't inflate the set). A blank success-metric body is the gate's job, not
|
|
23
23
|
this probe's (it only checks "if a metric exists, it carries a number"). When `consistency` reports
|
|
24
24
|
"no FR ids found", treat it as *coverage-unverified*, not *coverage-clean*.
|
|
25
|
+
|
|
26
|
+
## Contract-to-card endpoint coverage (`/flow consistency --contract`)
|
|
27
|
+
|
|
28
|
+
Mechanical audit verifying that every endpoint declared in `flow/05-contract.md` is covered across generated cards before building starts.
|
|
29
|
+
|
|
30
|
+
- **Extraction:** Parses `flow/05-contract.md`'s interface table (`Method` and `Path` columns; tolerates column order variations).
|
|
31
|
+
- **Matching Rule:** Scans `cards/C-*.md` `## Scope` sections for literal path token substring matches (e.g. `/healthz`, `/api/users/:id`).
|
|
32
|
+
- **Summary & Warnings:**
|
|
33
|
+
- Prints `N endpoint(s), M covered, K uncovered`.
|
|
34
|
+
- Uncovered endpoints produce an advisory warning block and exit 1 (matching `consistency` house convention for findings).
|
|
35
|
+
- Clean audit exits 0 (`PASS: all N endpoint(s) covered across cards/ scope.`).
|
|
36
|
+
- **Auto-skip:** If `flow/05-contract.md` is absent or declares no endpoint table, prints a one-line note and exits 0 (consistent with F3 non-REST auto-skip spirit).
|
|
@@ -295,3 +295,37 @@ guard so an in-flight or in-postmortem run's dir cannot be pruned under the oper
|
|
|
295
295
|
3. If the storm was rate-limit: wait past the window (typically minutes), then rerun.
|
|
296
296
|
4. If persistent: capture the `.err` line in a comment, escalate — this is when a throttled
|
|
297
297
|
`rate_limit_info` sample would land in the corpus and let `rate_limited` become authoritative.
|
|
298
|
+
|
|
299
|
+
## eval --diff
|
|
300
|
+
|
|
301
|
+
Offline longitudinal regression classifier across eval batches or model bumps.
|
|
302
|
+
Mutually exclusive with live eval or replay (`flow.sh eval --diff <old_results> <new_results>`).
|
|
303
|
+
Zero model calls, zero new runtime dependencies, deterministic.
|
|
304
|
+
|
|
305
|
+
If multiple batches exist in an input results file, `--diff` compares the latest complete batch
|
|
306
|
+
(falling back to the latest batch `run_id` present). Fixture expectations are resolved from each
|
|
307
|
+
row's own `expected` field, falling back to `$EVAL_MANIFEST` (field 4).
|
|
308
|
+
|
|
309
|
+
Classification is correctness-relative (`correct = verdict == expected`):
|
|
310
|
+
|
|
311
|
+
| Old State | New State | Classification | Description |
|
|
312
|
+
|-----------|-----------|----------------|-------------|
|
|
313
|
+
| Correct (`old == expected`) | Correct (`new == expected`) | `UNCHANGED` | Intended verdict preserved (PASS→PASS on sound, FLAG→FLAG on hollow) |
|
|
314
|
+
| Correct (`old == expected`) | Incorrect (`new != expected`) | `REGRESSED` | Previously correct fixture eroded (e.g. PASS→FLAG on sound, FLAG→PASS on hollow) |
|
|
315
|
+
| Incorrect (`old != expected`) | Correct (`new == expected`) | `IMPROVED` | Previously failing/misclassified fixture now matches expected |
|
|
316
|
+
| Incorrect (`old != expected`) | Incorrect (`new != expected`) | `SATURATED` | Neither run reached expected verdict (e.g. PASS→PASS on hollow, FLAG→INVALID) |
|
|
317
|
+
| present | absent | `removed` | Fixture present only in old results |
|
|
318
|
+
| absent | present | `added` | Fixture present only in new results |
|
|
319
|
+
| unknown expected | any | `unknown` | Fixture missing expected verdict in both records and manifest |
|
|
320
|
+
| `expected_old != expected_new` | any | `expectation-changed` | Expected verdict changed between runs; displayed as `OLD→NEW` in expected column; does not count as regressed or improved |
|
|
321
|
+
|
|
322
|
+
An `INVALID` or `UNRELIABLE` verdict represents an infra/parse/timeout failure and never matches an expected `PASS` or `FLAG`. An `INVALID` outcome on a previously correct fixture is always classified as `REGRESSED`. Unknown fixtures (`expected` unresolvable) show `unknown` and never flip the exit code.
|
|
323
|
+
|
|
324
|
+
### Output and exit codes
|
|
325
|
+
|
|
326
|
+
Prints a compact aligned table (`id`, `expected`, `old`, `new`, `classification`) followed by summary counts:
|
|
327
|
+
`N regressed, M improved, K saturated, U unchanged, E expectation-changed`.
|
|
328
|
+
|
|
329
|
+
- Exit `0`: clean comparison (0 regressed).
|
|
330
|
+
- Exit `1`: regression detected (≥1 fixture classified as `REGRESSED`).
|
|
331
|
+
- Exit `2`: missing, unreadable, or containing no fixture records.
|
|
@@ -52,3 +52,82 @@ evidence. No hook install into `~/.claude` or `~/.omp`. No long-lived socket sub
|
|
|
52
52
|
|
|
53
53
|
Own-runtime / own-PTY / own-wait is a successor ADR plus a separate product,
|
|
54
54
|
not a card in this skill.
|
|
55
|
+
|
|
56
|
+
## workspace port
|
|
57
|
+
|
|
58
|
+
Deterministic per-card port allocator for parallel worktree web verification (`flow workspace port <C-NNN>`).
|
|
59
|
+
|
|
60
|
+
- **Formula:** `port = FLOW_WORKSPACE_BASEPORT + port_offset` (where `port_offset` is the registered worktree offset from `.flow/workspaces.jsonl`). Stable per registration and matches `workspace enter`.
|
|
61
|
+
- **Port Bounds & Fail-Loud:** If `port = FLOW_WORKSPACE_BASEPORT + port_offset > 65535`, `workspace port` fails loudly with exit 2 (`FAIL - port range exhausted for '<card>' (baseport <b> + offset <o> = <raw> > 65535); lower FLOW_WORKSPACE_BASEPORT`) rather than wrapping or aliasing.
|
|
62
|
+
- **Environment Override:** `FLOW_WORKSPACE_BASEPORT` (defaults to `3000`; valid numeric range is `1..65535`). Out-of-range baseport exits 2.
|
|
63
|
+
- **Target File Contract:** Writes/overwrites strictly `<worktree>/.env.flow.local` inside the card's registered workspace root with:
|
|
64
|
+
```sh
|
|
65
|
+
PORT=<port>
|
|
66
|
+
FLOW_CARD=<C-NNN>
|
|
67
|
+
```
|
|
68
|
+
Mutates only this single file; zero locks needed beyond standard workspace mechanics.
|
|
69
|
+
- **Exit Codes:**
|
|
70
|
+
- `0`: Workspace resolved, port computed, `.env.flow.local` written.
|
|
71
|
+
- `2`: Card argument invalid shape, no registered workspace in `.flow/workspaces.jsonl`, missing `port_offset`, baseport out of range, or port range exhausted > 65535 (never creates worktrees implicitly).
|
|
72
|
+
|
|
73
|
+
## resume output contract
|
|
74
|
+
|
|
75
|
+
`flow resume` is the external-orchestrator re-entry surface. It is read-only for planning/card state (event-tail + registry reads via
|
|
76
|
+
`inflight_display`; no lock, no plan/card mutation, no `next_verb`
|
|
77
|
+
execution) — but the runner's EXIT trap (`_log_on_exit` -> `_log_event`)
|
|
78
|
+
still appends telemetry on every invocation (a row to the project's
|
|
79
|
+
`events.jsonl` plus a compact row to the device-global log), so expect
|
|
80
|
+
those files to grow. The documented elements below describe shipped
|
|
81
|
+
behavior; anything not listed here is not to be parsed.
|
|
82
|
+
|
|
83
|
+
**Output shape, in order:**
|
|
84
|
+
|
|
85
|
+
1. **Header.** `flow resume`, then indented `project:` (repo root) and
|
|
86
|
+
`auto:` (attestation auto state). If planning has not started
|
|
87
|
+
(`current_stage_idx < 0`), resume emits only `nothing to resume - <next
|
|
88
|
+
action>` + `NEXT_VERB=<verb>` and stops.
|
|
89
|
+
2. **Last-session telemetry block.** The most recent foreign session_id in
|
|
90
|
+
`events.jsonl` (string inequality vs the current session), emitting that
|
|
91
|
+
session's rows as `<ts> <command> ok|FAIL(n) [stage_from -> stage_to]`.
|
|
92
|
+
Command **NAMES only — never raw args** (free-text args can carry
|
|
93
|
+
unmasked secrets). Rows are capped `tail -5` (most recent wins). A
|
|
94
|
+
`ppid:`-scheme session whose latest foreign row is < 900s from the anchor
|
|
95
|
+
(this session's own first row if present, else wall-clock `now` — the
|
|
96
|
+
no-own-row-yet case is exactly when resume runs) is labeled
|
|
97
|
+
`recent activity` instead of `last session`, because PID reuse makes a
|
|
98
|
+
"last session" that may never have existed. Absolute timestamps shown
|
|
99
|
+
either way. If `events.jsonl` is empty/absent: `no telemetry - showing
|
|
100
|
+
gate state only`, then gate state + NEXT, and no in-flight section.
|
|
101
|
+
3. **In-flight section.** `in flight:` header, then one line per todo card
|
|
102
|
+
still registered in `$CARDS_DIR/.inflight` (`cards/.inflight`, per
|
|
103
|
+
`_inflight_file`): ` <id> (in flight, started
|
|
104
|
+
<Xm|Xh|Xd> ago)`. Stale ids (card file gone) self-skip; **completed cards
|
|
105
|
+
are never listed** (only `status: todo` rows qualify).
|
|
106
|
+
4. **Gate state.** Identical block to `flow status` (`_gate_state_brief`):
|
|
107
|
+
current stage only — `planning: at stage <s>` plus `gate: PASS|BLOCKED`.
|
|
108
|
+
(In the rare earlier-stage-blocked state, NEXT may point at an earlier
|
|
109
|
+
stage than this current-stage gate; that is by design.)
|
|
110
|
+
5. **Exactly one `NEXT -> <action>` + `NEXT_VERB=<verb>` line pair**, from
|
|
111
|
+
the same `_next_action` ladder as `status` — the two verbs can never
|
|
112
|
+
disagree.
|
|
113
|
+
|
|
114
|
+
**`--budget` semantics.** `resume --budget` caps the in-flight list at 5
|
|
115
|
+
lines plus one ` +M more in flight - see 'flow card'` summary line
|
|
116
|
+
(where M = hidden count). With ≤5 in-flight cards, `--budget` output is
|
|
117
|
+
**byte-identical to default resume output** (matching rule:
|
|
118
|
+
byte-identical-under-budget). No other section is affected; default resume
|
|
119
|
+
lists all in-flight cards and never emits the summary line. Extra arguments
|
|
120
|
+
to either form exit 2.
|
|
121
|
+
|
|
122
|
+
**`resume --json`.** Emits exactly one closed `flow_context/v1` object —
|
|
123
|
+
the same schema as `status --json`:
|
|
124
|
+
`{"v":"flow_context/v1","stage":...,"gate":"PASS|BLOCKED|none","next_verb":...,"card":...,"dwell":...,"load":...}`.
|
|
125
|
+
Display-only: it reports what to run next but never executes `next_verb`.
|
|
126
|
+
|
|
127
|
+
**Scope note (not a stability promise).** The elements above are parseable by external
|
|
128
|
+
orchestrators; parse only these, never ad-hoc prose. This documents shipped
|
|
129
|
+
behavior as of the current flow version (`version` in `skills/flow/SKILL.md`
|
|
130
|
+
— currently 0.34.0 — surfaced as `flow_version` in telemetry). The project
|
|
131
|
+
has not adopted a formal compatibility guarantee: when upgrading flow,
|
|
132
|
+
re-diff this contract against the runner (`cmd_resume`, `inflight_display`,
|
|
133
|
+
`_log_event`) before relying on it.
|
|
@@ -6,12 +6,12 @@ maintenance. Each was written fresh from public/generic patterns (persona-panel
|
|
|
6
6
|
generic edge-case taxonomy, STRIDE — public Microsoft SDL methodology, git-log
|
|
7
7
|
retrospection, iterate-to-numeric-target loops, evidence-backed harness improvement) —
|
|
8
8
|
none of this text is copied from claudekit-engineer (proprietary, All Rights Reserved)
|
|
9
|
-
or any other kit. Where a matching ck skill is installed, it is offered as a **richer
|
|
9
|
+
or any other kit. Where a matching ck or ak skill is installed, it is offered as a **richer
|
|
10
10
|
alternative**, never a requirement — see the seam files (`gate-rules.md`,
|
|
11
11
|
`adversarial-review.md`, `law/RETRO.md`) for the native-first wiring, and
|
|
12
|
-
`claudekit-skills.md` for the optional-enrichment annex.
|
|
12
|
+
`claudekit-skills.md` for the optional-enrichment annex (Name resolution).
|
|
13
13
|
|
|
14
|
-
Every ritual below follows the same rule as the ck skills it stands in for: it
|
|
14
|
+
Every ritual below follows the same rule as the ck/ak skills it stands in for: it
|
|
15
15
|
**informs** the gate; the gate (the `flow.sh` exit code + the semantic challenge) still
|
|
16
16
|
judges. A ritual "looks good" is not a PASS.
|
|
17
17
|
|
|
@@ -0,0 +1,5 @@
|
|
|
1
|
+
PIN v1
|
|
2
|
+
Never FLOW_FORCE a live session. Mechanical FAIL -> STOP.
|
|
3
|
+
Security-class skip/debt -> HALT until the operator accepts in DEBT.md.
|
|
4
|
+
Never mark a card done without pasted world-state evidence.
|
|
5
|
+
NEXT_VERB is advisory; never auto-exec next, card, card-start, check, fix-gate, auto, or skip.
|
|
@@ -233,7 +233,7 @@ _att_git_clean_for_attest() {
|
|
|
233
233
|
path="${path#\"}"; path="${path%\"}"
|
|
234
234
|
case "$path" in
|
|
235
235
|
.flow|.flow/*) continue ;;
|
|
236
|
-
flow/.lock|flow/.lock.d|flow/.lock.d/*) continue ;;
|
|
236
|
+
flow/.lock|flow/.lock.*|flow/.lock.d|flow/.lock.d/*) continue ;;
|
|
237
237
|
cards/C-*.md)
|
|
238
238
|
_att_card_projection_matches_head "$path" || return 1
|
|
239
239
|
continue
|