@manhquy/flow-skill 0.7.0 → 0.7.1-next.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -2,6 +2,16 @@
2
2
 
3
3
  All notable changes to `@manhquy/flow-skill`. Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning: [SemVer](https://semver.org/).
4
4
 
5
+ ## [0.7.1-next.0] — 2026-08-20 — ships skill v0.31.0 (eval receipts + host-agnostic parallel + value-first docs)
6
+
7
+ Skill-content release. Re-ships skill product **v0.31.0** (see root
8
+ `CHANGELOG.md` `## 0.31.0`). Installer CLI contract unchanged. Publishes to
9
+ dist-tag **`next`**.
10
+
11
+ ### Changed
12
+ - Bundled skill tree re-synced from monorepo `skills/flow/`.
13
+ - Install (prerelease): **`npx @manhquy/flow-skill@next`**.
14
+
5
15
  ## [0.7.0] — 2026-08-14 — ships skill v0.30.0 (discipline-layer identity + CI/eval hardening)
6
16
 
7
17
  Skill-content release. Ships skill product **v0.30.0**: identity ADR (gates and
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@manhquy/flow-skill",
3
- "version": "0.7.0",
3
+ "version": "0.7.1-next.0",
4
4
  "description": "Install the flow skill (buildflow gated build process) into your coding agent(s) via one npx command",
5
5
  "type": "module",
6
6
  "bin": {
@@ -8,7 +8,7 @@ keywords: [flow, buildflow, gate, build, ship, scope, prd, contract, card, deplo
8
8
  license: MIT
9
9
  metadata:
10
10
  author: flow-skill
11
- version: "0.30.0"
11
+ version: "0.31.0"
12
12
  attribution: "Methodology from ai20k-build-phase/buildflow (Tony, arealisticdreamer.com); durable-layer ancestry and improve-ritual spirit from repository-harness (protocol v1 EOL — flow-owned fork); agent patterns from claudekit-engineer, BMAD-METHOD. v0.22 concierge from BMAD bmad-help; forge-idea from bmad-forge-idea (MIT, BMad Code LLC)."
13
13
  ---
14
14
 
@@ -190,6 +190,8 @@ law/contract excerpts (no session history); each returns DONE/DONE_WITH_CONCERNS
190
190
  NEEDS_CONTEXT. After a delegation: run the gate, apply the semantic challenge, write the
191
191
  durable hook (`flow.sh harness ...`), announce which path ran.
192
192
 
193
+ Parallel cards ride the current host (in-process Task, extra terminals, or an already-inside mux); flow requires no named multiplexer — `references/host-agnostic-parallel.md`.
194
+
193
195
  Mode `work` (`references/mode-work.md`): interview once → draft 00-05 → one scope pause →
194
196
  deliver the card set as one summary. Gates bind the same as `teach`.
195
197
 
@@ -74,6 +74,7 @@ Manager / real `agy` terminal, paste the ReviewResult back); an empty Gemini res
74
74
  - **Same gate regardless of path.** The buildflow gate (`flow.sh` + `gate-rules.md`) is the
75
75
  contract. An agent fills an artifact; the gate still judges it. A missing agent never
76
76
  lowers a gate — it only changes who drafts.
77
+ - **No named mux required.** Absence of any multiplexer never fails a gate. Parallel occupancy is host-owned — `host-agnostic-parallel.md`.
77
78
  - **Context isolation (orchestration-protocol).** Give each subagent ONLY: the task, the
78
79
  specific files to read/modify, acceptance criteria, and relevant law/contract excerpts.
79
80
  Never the full session history. One card = one scoped brief.
@@ -101,6 +101,7 @@ and log the durable metric (`flow.sh harness intervention add`) per `codex-integ
101
101
  parallel, each in its own worktree/session. Merge back in card-number order, one at a time,
102
102
  running the merged app once between merges. A merge conflict means the overlap check was
103
103
  gamed — stop and re-plan. Cards needing the deployed app (contract-tests, e2e) are serial.
104
+ Placement and wait stay with the host; see `host-agnostic-parallel.md`.
104
105
 
105
106
  **Human-driven `/flow workspace` coexists with this auto loop.** `auto` provisions a worktree per
106
107
  card internally (`git worktree add ../<project>-C-NNN -b card/C-NNN`); `/flow workspace add card/C-NNN`
@@ -90,6 +90,54 @@ unbounded-billing risk. Install GNU coreutils (`gtimeout`) or export `FLOW_EVAL_
90
90
  accept that risk for one run. `--report` and `--replay` never hit this guard. `--record` is live
91
91
  and does.
92
92
 
93
+ ### Cheap isolated live eval (host recipe)
94
+
95
+ Live cost is dominated by **host** context (project `CLAUDE.md`/`AGENTS.md` walk + user
96
+ plugins/skills/MCP), not the inlined judge prompt. Isolation is cwd + env around the
97
+ existing one-shot — never extra engine argv, never `--bare`, never `--model`.
98
+
99
+ `FLOW_EVAL_ISOLATED_CWD` unset → today's cwd (byte-identical). Set to an existing directory
100
+ → probe and engine run in `( cd "$FLOW_EVAL_ISOLATED_CWD" && export CLAUDE_CODE_SAFE_MODE=1 &&
101
+ … )`. Set but not a directory → fail loud. `CLAUDE_CODE_SAFE_MODE=1` is exported only in
102
+ that subshell (official Claude env; keeps OAuth). Do **not** empty `HOME` or point
103
+ `CLAUDE_CONFIG_DIR` at the scratch dir (Linux credentials live there).
104
+
105
+ ```bash
106
+ # Auth must be live (`claude auth login` if `claude auth status` says not logged in).
107
+ ISOLATED="$(mktemp -d /tmp/flow-eval-XXXXXX)" # NOT an ancestor of the flow-skill tree
108
+ export FLOW_EVAL_ISOLATED_CWD="$ISOLATED"
109
+ export FLOW_PROJECT_ROOT="/path/to/real/project" # results stay in $ROOT/.flow/
110
+ bash /path/to/skills/flow/runner/flow.sh eval --n 1 --fixture fcda
111
+ ```
112
+
113
+ The first isolated batch is a **new baseline**, not comparable to the 2026-07-10
114
+ loaded-context floor ($0.30–0.37/call, ~50–60K cache-creation). A judge-model change is a
115
+ full re-baseline. Do not mix isolated and loaded-context batches in tripwire-2. Update the
116
+ measured dollars in this Cost section **after** the operator checkpoint publishes isolated
117
+ `total_cost_usd` + `cache_creation_input_tokens`. Replay still never counts.
118
+
119
+ Cost measurement against one captured `--output-format json` document (do not commit the
120
+ envelope — `session_id`/`cwd` leak):
121
+
122
+ ```bash
123
+ printf '%s' "$raw" | python3 -c '
124
+ import json,sys
125
+ d=json.load(sys.stdin)
126
+ if isinstance(d,list):
127
+ d=next(x for x in d if isinstance(x,dict) and x.get("type")=="result")
128
+ u=d.get("usage") or {}
129
+ print("usd", d.get("total_cost_usd"))
130
+ print("in", u.get("input_tokens"), "out", u.get("output_tokens"))
131
+ print("cache_create", u.get("cache_creation_input_tokens") or u.get("cache_creation"))
132
+ print("cache_read", u.get("cache_read_input_tokens"))
133
+ print("model", d.get("model"))
134
+ '
135
+ ```
136
+
137
+ Success signal: `cache_creation_input_tokens` collapses from ~50–60K toward the inlined
138
+ judge prompt size; `total_cost_usd` drops well below $0.30.
139
+
140
+
93
141
  ## Modes
94
142
 
95
143
  | Mode | Calls | What it proves |
@@ -0,0 +1,54 @@
1
+ # Host-agnostic parallel occupancy
2
+
3
+ Flow requires **no named multiplexer**. Parallel cards are a ready-gate +
4
+ worktrees + host-owned placement + world-state fan-in. Absence of Herdr, tmux,
5
+ in-process `Task`, or any other host **never fails a gate** and never changes
6
+ `flow.sh` exit.
7
+
8
+ Herdr and tmux appear below only as **examples, never as a dependency** or a
9
+ first-class adapter.
10
+
11
+ ## Degrade ladder
12
+
13
+ | Rung | Primitive already in the process | Who binds occupants | Who waits |
14
+ |---|---|---|---|
15
+ | **0** | In-process subagents / `Task` | Hosting agent; cwd = the card's worktree | Host `Task` join |
16
+ | **1** | Operator extra terminals / tabs | Human pastes the print-enter block | Human |
17
+ | **2** | Process is **already inside** a mux (`HERDR_ENV=1` / `TMUX` set) | That mux's **own** skill/CLI; flow does not wrap or detect it | That mux's own wait |
18
+ | **3** | None of the above, or `/flow ready` overlap | Serial: one card | Hosting agent, one at a time |
19
+
20
+ Rung 2 is opt-in by the mux's own environment injection, never by `PATH`.
21
+ Inside-mux probe is the mux's own env (`HERDR_ENV=1`, `TMUX`), never
22
+ `command -v herdr` from outside (a CLI can talk to a same-uid socket without
23
+ `HERDR_ENV`). Absence of rungs 0–2 is the default path, not a degrade footnote.
24
+
25
+ ## Gate parity
26
+
27
+ Host `idle` / `done` / `blocked` is **not a card pass**. `flow.sh check` still
28
+ judges, and it must run **in the card's worktree** (parent-tree check of main
29
+ is a false fail/pass). Screen output is not evidence.
30
+
31
+ Keep the four "blocked" senses distinct:
32
+
33
+ | Sense | Meaning | Home |
34
+ |---|---|---|
35
+ | **host-blocked** | mux/pane approval chrome or occupant stuck | the host |
36
+ | **subagent-BLOCKED** | scoped subagent status protocol | `agent-detection.md` |
37
+ | **ready-blocked** | deps unmet or allowed-files overlap | `flow.sh ready` |
38
+ | **Tier-C security halt** | security-class exposure; operator DEBT | `auto-run.md` |
39
+
40
+ Mixing them auto-answers a payments card or auto-halts a yes/no prompt.
41
+
42
+ ## File-is-the-wait
43
+
44
+ The **HOST waits** (Task join / human / the mux's own wait). `flow.sh` **never waits on or polls** another agent. Fan-in is world-state: re-invoke `check` in
45
+ the card worktree after the host's wait returns.
46
+
47
+ ## Forbidden (ADR-0001; not a nit)
48
+
49
+ `flow.sh` must not exec host control CLIs (`herdr agent start`, `herdr agent wait`, `herdr agent prompt`, `herdr server stop`, `tmux send-keys`, and equivalents).
50
+ Forbidden by name: `mux-up`, `mux-run-wave`, `mux-proxy`. No screen output as
51
+ evidence. No hook install into `~/.claude` or `~/.omp`. No long-lived socket subscribe (`events.subscribe`). `flow.sh` never spawns agents; placement is print-enter or the host's own Task/mux.
52
+
53
+ Own-runtime / own-PTY / own-wait is a successor ADR plus a separate product,
54
+ not a card in this skill.
@@ -2302,6 +2302,14 @@ cmd_doctor() {
2302
2302
  command -v cargo >/dev/null 2>&1 && echo " cargo: ok (optional Rust harness power-path)" || true
2303
2303
  [ -d "$TEMPLATE_DIR" ] && echo " templates: ok" || { echo " templates: MISSING ($TEMPLATE_DIR)"; ok=0; }
2304
2304
  [ -f "$HARNESS_PY" ] && echo " harness: present ($HARNESS_PY)" || echo " harness: absent (durable layer off)"
2305
+ if command -v timeout >/dev/null 2>&1 || command -v gtimeout >/dev/null 2>&1; then
2306
+ echo " timeout: full (timeout|gtimeout present)"
2307
+ else
2308
+ case "$(uname -s 2>/dev/null)" in
2309
+ Darwin) echo " timeout: absent (no timeout/gtimeout; live eval REFUSED)" ;;
2310
+ *) echo " timeout: partial (watchdog fallback; live eval not GNU-bounded)" ;;
2311
+ esac
2312
+ fi
2305
2313
  echo
2306
2314
  if [ "$ok" -eq 1 ]; then
2307
2315
  echo "READY. Gate engine works$([ -n "$py" ] && echo '; durable layer on.' || echo '; durable layer off (no python).')"
@@ -3199,6 +3207,25 @@ _eval_engine_run() { # $1=promptfile $2=timeout-seconds -> stdout=raw json; retu
3199
3207
  # command string above. Empirical: adding a `2> '$errfile'` inside the timed cmd breaks the
3200
3208
  # watchdog-fallback path in `_run_with_timeout` on the timeout-less-PATH lane (macOS DEBT lane).
3201
3209
  # Outer-scope redirection is exactly the same capture at zero cost to the timeout contract.
3210
+ # Opt-in isolated cwd for live eval (Approach D). Unset = today's cwd (byte-identical
3211
+ # call path). Set but not a directory = fail loud. Isolation is cd+env around existing
3212
+ # probe/engine calls — never argv, never a second engine. CLAUDE_CODE_SAFE_MODE=1 is
3213
+ # exported ONLY in the isolated subshell (official Claude env; keeps OAuth; blast radius
3214
+ # is the judge child). Do not invent a flow alias; do not set HOME or CLAUDE_CONFIG_DIR.
3215
+ _eval_isolated_guard() {
3216
+ [ -z "${FLOW_EVAL_ISOLATED_CWD:-}" ] && return 0
3217
+ [ -d "$FLOW_EVAL_ISOLATED_CWD" ] && return 0
3218
+ echo "FAIL: FLOW_EVAL_ISOLATED_CWD is set but is not a directory: $FLOW_EVAL_ISOLATED_CWD"
3219
+ return 1
3220
+ }
3221
+ _eval_isolated_run() {
3222
+ if [ -n "${FLOW_EVAL_ISOLATED_CWD:-}" ]; then
3223
+ ( cd "$FLOW_EVAL_ISOLATED_CWD" && export CLAUDE_CODE_SAFE_MODE=1 && "$@" )
3224
+ else
3225
+ "$@"
3226
+ fi
3227
+ }
3228
+
3202
3229
 
3203
3230
  # Build the judge prompt for one fixture. Returns 1 (writes nothing usable) if the gate-rules.md
3204
3231
  # section extraction came back empty.
@@ -3291,6 +3318,13 @@ _eval_gate_rules_sha() {
3291
3318
  tr -d '\r' < "$GATE_RULES_FILE" | cksum | awk '{print $1}'
3292
3319
  }
3293
3320
 
3321
+ # CRLF-normalized assembler-output hash (same recipe as _eval_gate_rules_sha). Covers
3322
+ # prompt-file bytes only — not host CLAUDE.md. $1=prompt file.
3323
+ _eval_prompt_sha() {
3324
+ [ -f "$1" ] || { printf 'unknown'; return; }
3325
+ tr -d '\r' < "$1" | cksum | awk '{print $1}'
3326
+ }
3327
+
3294
3328
  # `claude --version` is the CLI version (e.g. "2.1.201 (Claude Code)"), NOT a model id - kept
3295
3329
  # as its own axis distinct from `_eval_parse_model` (red-team finding #4: 3 reviewers
3296
3330
  # independently confirmed conflating the two is wrong). `< /dev/null`: this and every other
@@ -3317,9 +3351,10 @@ _eval_replay_meta_get() { # $1=key -> value (empty + return 1 if missing)
3317
3351
  return 1
3318
3352
  }
3319
3353
 
3320
- _eval_replay_write_meta() { # $1=nonce $2=gate_rules_sha $3=model $4=n
3354
+ _eval_replay_write_meta() { # $1=nonce $2=gate_rules_sha $3=model $4=n [$5=prompt_sha.<fid>= lines]
3321
3355
  mkdir -p "$EVAL_REPLAY_DIR" || return 1
3322
- printf 'nonce=%s\ngate_rules_sha=%s\nmodel=%s\nn=%s\n' "$1" "$2" "$3" "$4" > "$EVAL_REPLAY_DIR/meta"
3356
+ printf 'nonce=%s\ngate_rules_sha=%s\nmodel=%s\nn=%s\n' "$1" "$2" "$3" "$4" > "$EVAL_REPLAY_DIR/meta" || return 1
3357
+ [ -n "${5:-}" ] && printf '%s' "$5" >> "$EVAL_REPLAY_DIR/meta"
3323
3358
  }
3324
3359
 
3325
3360
  # Persist one stripped verdict line only — never a JSON envelope (session_id/cwd stay out).
@@ -3366,17 +3401,18 @@ _eval_emit_batch_marker() { # $1=run_id $2=start|done $3=n_expected-or-n_written
3366
3401
  # cli_version/flow_version/gate_rules_sha are passed in (computed ONCE per batch by the caller,
3367
3402
  # not per fixture row) - they never change mid-batch, and recomputing per-row was both wasteful
3368
3403
  # (an extra subprocess per fixture) and the original vector for the stdin-consumption bug above.
3369
- _eval_emit_result() { # $1..$14 as before + $15=retries $16=rate_limited (true|false)
3404
+ _eval_emit_result() { # $1..$14 as before + $15=retries $16=rate_limited $17=prompt_sha $18=timed_out
3370
3405
  local run_id="$1" fid="$2" stage="$3" expected="$4" verdict="$5" match="$6" flag="$7" pass="$8" invalid="$9" n="${10}" model="${11}" cliv="${12}" flowv="${13}" grsha="${14}"
3371
- local retries="${15:-0}" ratel="${16:-false}"
3406
+ local retries="${15:-0}" ratel="${16:-false}" psha="${17:-}" timedout="${18:-false}"
3372
3407
  case "$retries" in ''|*[!0-9]*) retries=0 ;; esac
3373
3408
  case "$ratel" in true|false) : ;; *) ratel=false ;; esac
3409
+ case "$timedout" in true|false) : ;; *) timedout=false ;; esac
3374
3410
  mkdir -p "$LOG_DIR" 2>/dev/null || true
3375
- printf '{"ts":"%s","epoch_s":%s,"run_id":"%s","fixture":"%s","stage":"%s","expected":"%s","verdict":"%s","match":"%s","votes":{"flag":%s,"pass":%s,"invalid":%s},"n":%s,"cli_version":"%s","model":"%s","flow_version":"%s","gate_rules_sha":"%s","retries":%s,"rate_limited":%s}\n' \
3411
+ printf '{"ts":"%s","epoch_s":%s,"run_id":"%s","fixture":"%s","stage":"%s","expected":"%s","verdict":"%s","match":"%s","votes":{"flag":%s,"pass":%s,"invalid":%s},"n":%s,"cli_version":"%s","model":"%s","flow_version":"%s","gate_rules_sha":"%s","retries":%s,"rate_limited":%s,"prompt_sha":"%s","timed_out":%s}\n' \
3376
3412
  "$(date -u +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || echo unknown)" "$(_now)" "$(_json_str "$run_id")" "$(_json_str "$fid")" \
3377
3413
  "$(_json_str "$stage")" "$(_json_str "$expected")" "$(_json_str "$verdict")" "$(_json_str "$match")" \
3378
3414
  "$flag" "$pass" "$invalid" "$n" "$(_json_str "$cliv")" "$(_json_str "$model")" \
3379
- "$(_json_str "$flowv")" "$(_json_str "$grsha")" "$retries" "$ratel" >> "$EVAL_RESULTS_FILE" 2>/dev/null || true
3415
+ "$(_json_str "$flowv")" "$(_json_str "$grsha")" "$retries" "$ratel" "$(_json_str "$psha")" "$timedout" >> "$EVAL_RESULTS_FILE" 2>/dev/null || true
3380
3416
  }
3381
3417
 
3382
3418
  # Strip the noisy `system`/`init` envelope from a raw --output-format json line before persisting
@@ -3660,17 +3696,18 @@ _eval_routing_parse_action() {
3660
3696
  [ -n "$action" ] && printf '%s' "$action" || echo INVALID
3661
3697
  }
3662
3698
 
3663
- _eval_routing_emit_result() { # $1=run_id $2=fid $3=expected $4=verdict $5=match $6..$9=votes-hit/miss/invalid/n $10=model $11=cliv $12=flowv $13=rulessha $14=retries $15=rate_limited
3699
+ _eval_routing_emit_result() { # $1=run_id $2=fid $3=expected $4=verdict $5=match $6..$9=votes $10=model $11=cliv $12=flowv $13=rulessha $14=retries $15=rate_limited $16=prompt_sha $17=timed_out
3664
3700
  local run_id="$1" fid="$2" expected="$3" verdict="$4" match="$5" hit="$6" miss="$7" invalid="$8" n="$9"
3665
- local model="${10}" cliv="${11}" flowv="${12}" rulessha="${13}" retries="${14:-0}" ratel="${15:-false}"
3701
+ local model="${10}" cliv="${11}" flowv="${12}" rulessha="${13}" retries="${14:-0}" ratel="${15:-false}" psha="${16:-}" timedout="${17:-false}"
3666
3702
  case "$retries" in ''|*[!0-9]*) retries=0 ;; esac
3667
3703
  case "$ratel" in true|false) : ;; *) ratel=false ;; esac
3704
+ case "$timedout" in true|false) : ;; *) timedout=false ;; esac
3668
3705
  mkdir -p "$LOG_DIR" 2>/dev/null || true
3669
- printf '{"ts":"%s","epoch_s":%s,"run_id":"%s","fixture":"%s","expected":"%s","verdict":"%s","match":"%s","votes":{"hit":%s,"miss":%s,"invalid":%s},"n":%s,"cli_version":"%s","model":"%s","flow_version":"%s","rules_sha":"%s","retries":%s,"rate_limited":%s}\n' \
3706
+ printf '{"ts":"%s","epoch_s":%s,"run_id":"%s","fixture":"%s","expected":"%s","verdict":"%s","match":"%s","votes":{"hit":%s,"miss":%s,"invalid":%s},"n":%s,"cli_version":"%s","model":"%s","flow_version":"%s","rules_sha":"%s","retries":%s,"rate_limited":%s,"prompt_sha":"%s","timed_out":%s}\n' \
3670
3707
  "$(date -u +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || echo unknown)" "$(_now)" "$(_json_str "$run_id")" "$(_json_str "$fid")" \
3671
3708
  "$(_json_str "$expected")" "$(_json_str "$verdict")" "$(_json_str "$match")" \
3672
3709
  "$hit" "$miss" "$invalid" "$n" "$(_json_str "$cliv")" "$(_json_str "$model")" \
3673
- "$(_json_str "$flowv")" "$(_json_str "$rulessha")" "$retries" "$ratel" >> "$EVAL_ROUTING_RESULTS_FILE" 2>/dev/null || true
3710
+ "$(_json_str "$flowv")" "$(_json_str "$rulessha")" "$retries" "$ratel" "$(_json_str "$psha")" "$timedout" >> "$EVAL_ROUTING_RESULTS_FILE" 2>/dev/null || true
3674
3711
  }
3675
3712
 
3676
3713
  _eval_routing_complete_run_ids() {
@@ -3770,7 +3807,9 @@ function cmd_eval_routing { # same arg shape as cmd_eval, called only when --sta
3770
3807
  return 1
3771
3808
  fi
3772
3809
 
3773
- local probe; probe="$(_eval_probe)"
3810
+ _eval_isolated_guard || return 1
3811
+
3812
+ local probe; probe="$(_eval_isolated_run _eval_probe)"
3774
3813
  case "$probe" in
3775
3814
  absent)
3776
3815
  echo "SKIP: 'claude' CLI not found on PATH - routing eval needs it to run the LLM judge. Zero calls made."
@@ -3838,14 +3877,18 @@ function cmd_eval_routing { # same arg shape as cmd_eval, called only when --sta
3838
3877
  continue
3839
3878
  fi
3840
3879
 
3880
+ local psha
3881
+ psha="$(_eval_prompt_sha "$promptfile")"
3882
+
3841
3883
  total_evaluated=$((total_evaluated + 1))
3842
- local hit_count=0 miss_count=0 invalid_count=0 i=1 model="unknown" retries_sum=0 vote_rate_limited=false
3884
+ local hit_count=0 miss_count=0 invalid_count=0 i=1 model="unknown" retries_sum=0 vote_rate_limited=false vote_timed_out=false
3843
3885
  while [ "$i" -le "$n" ]; do
3844
3886
  local raw rc v raw1="" rc1=0
3845
- raw="$(_eval_engine_run "$promptfile" "$timeout" 2>/dev/null)"; rc=$?
3887
+ raw="$(_eval_isolated_run _eval_engine_run "$promptfile" "$timeout" 2>/dev/null)"; rc=$?
3846
3888
  raw1="$raw"; rc1="$rc"
3847
3889
  if [ "$rc" -eq 124 ]; then
3848
3890
  v=INVALID
3891
+ vote_timed_out=true
3849
3892
  else
3850
3893
  v="$(_eval_routing_parse_action "$raw" "$nonce")"
3851
3894
  [ "$model" = "unknown" ] && model="$(_eval_parse_model "$raw")"
@@ -3857,9 +3900,9 @@ function cmd_eval_routing { # same arg shape as cmd_eval, called only when --sta
3857
3900
  echo " $fid: retrying vote $i after ${backoff}s (parse-INVALID on attempt 1)"
3858
3901
  sleep "$backoff" 2>/dev/null || true
3859
3902
  fi
3860
- raw="$(_eval_engine_run "$promptfile" "$timeout" 2>/dev/null)"; rc=$?
3903
+ raw="$(_eval_isolated_run _eval_engine_run "$promptfile" "$timeout" 2>/dev/null)"; rc=$?
3861
3904
  retries_sum=$((retries_sum + 1))
3862
- if [ "$rc" -eq 124 ]; then v=INVALID; else v="$(_eval_routing_parse_action "$raw" "$nonce")"; fi
3905
+ if [ "$rc" -eq 124 ]; then v=INVALID; vote_timed_out=true; else v="$(_eval_routing_parse_action "$raw" "$nonce")"; fi
3863
3906
  [ "$model" = "unknown" ] && [ "$rc" -ne 124 ] && model="$(_eval_parse_model "$raw")"
3864
3907
  local rl2; rl2="$(_eval_parse_rate_limited "$raw")"
3865
3908
  [ "$rl2" = "true" ] && vote_rate_limited=true
@@ -3897,7 +3940,7 @@ function cmd_eval_routing { # same arg shape as cmd_eval, called only when --sta
3897
3940
  echo " $fid: MISS - expected '$fexpected' (hit=$hit_count miss=$miss_count invalid=$invalid_count)"
3898
3941
  fi
3899
3942
 
3900
- _eval_routing_emit_result "$nonce" "$fid" "$fexpected" "$verdict" "$match" "$hit_count" "$miss_count" "$invalid_count" "$n" "$model" "$batch_cliv" "$batch_flowv" "$batch_rulessha" "$retries_sum" "$vote_rate_limited"
3943
+ _eval_routing_emit_result "$nonce" "$fid" "$fexpected" "$verdict" "$match" "$hit_count" "$miss_count" "$invalid_count" "$n" "$model" "$batch_cliv" "$batch_flowv" "$batch_rulessha" "$retries_sum" "$vote_rate_limited" "$psha" "$vote_timed_out"
3901
3944
  n_written=$((n_written + 1))
3902
3945
 
3903
3946
  if [ "$total_evaluated" -eq 1 ] && [ "$verdict" = "UNRELIABLE" ] && [ "$keep_going" -eq 0 ]; then
@@ -4005,15 +4048,16 @@ _eval_converge_parse_verdict() { # $1=raw $2=nonce -> GAP|CONVERGED|INVALID
4005
4048
 
4006
4049
  _eval_converge_emit_result() { # same field shape as _eval_routing_emit_result
4007
4050
  local run_id="$1" fid="$2" expected="$3" verdict="$4" match="$5" hit="$6" miss="$7" invalid="$8" n="$9"
4008
- local model="${10}" cliv="${11}" flowv="${12}" rulessha="${13}" retries="${14:-0}" ratel="${15:-false}"
4051
+ local model="${10}" cliv="${11}" flowv="${12}" rulessha="${13}" retries="${14:-0}" ratel="${15:-false}" psha="${16:-}" timedout="${17:-false}"
4009
4052
  case "$retries" in ''|*[!0-9]*) retries=0 ;; esac
4010
4053
  case "$ratel" in true|false) : ;; *) ratel=false ;; esac
4054
+ case "$timedout" in true|false) : ;; *) timedout=false ;; esac
4011
4055
  mkdir -p "$LOG_DIR" 2>/dev/null || true
4012
- printf '{"ts":"%s","epoch_s":%s,"run_id":"%s","fixture":"%s","expected":"%s","verdict":"%s","match":"%s","votes":{"hit":%s,"miss":%s,"invalid":%s},"n":%s,"cli_version":"%s","model":"%s","flow_version":"%s","rules_sha":"%s","retries":%s,"rate_limited":%s}\n' \
4056
+ printf '{"ts":"%s","epoch_s":%s,"run_id":"%s","fixture":"%s","expected":"%s","verdict":"%s","match":"%s","votes":{"hit":%s,"miss":%s,"invalid":%s},"n":%s,"cli_version":"%s","model":"%s","flow_version":"%s","rules_sha":"%s","retries":%s,"rate_limited":%s,"prompt_sha":"%s","timed_out":%s}\n' \
4013
4057
  "$(date -u +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || echo unknown)" "$(_now)" "$(_json_str "$run_id")" "$(_json_str "$fid")" \
4014
4058
  "$(_json_str "$expected")" "$(_json_str "$verdict")" "$(_json_str "$match")" \
4015
4059
  "$hit" "$miss" "$invalid" "$n" "$(_json_str "$cliv")" "$(_json_str "$model")" \
4016
- "$(_json_str "$flowv")" "$(_json_str "$rulessha")" "$retries" "$ratel" >> "$EVAL_CONVERGE_RESULTS_FILE" 2>/dev/null || true
4060
+ "$(_json_str "$flowv")" "$(_json_str "$rulessha")" "$retries" "$ratel" "$(_json_str "$psha")" "$timedout" >> "$EVAL_CONVERGE_RESULTS_FILE" 2>/dev/null || true
4017
4061
  }
4018
4062
 
4019
4063
  _eval_converge_emit_batch_marker() { # $1=run_id $2=start|done $3=n
@@ -4075,7 +4119,9 @@ function cmd_eval_converge { # same arg shape as cmd_eval, called only when --st
4075
4119
  return 1
4076
4120
  fi
4077
4121
 
4078
- local probe; probe="$(_eval_probe)"
4122
+ _eval_isolated_guard || return 1
4123
+
4124
+ local probe; probe="$(_eval_isolated_run _eval_probe)"
4079
4125
  case "$probe" in
4080
4126
  absent) echo "SKIP: 'claude' CLI not found on PATH - converge eval needs it to run the LLM judge. Zero calls made."; return 0 ;;
4081
4127
  fail) echo "SKIP: 'claude' CLI present but the sentinel probe did not come back clean (one minimal probe call was made). Retry once 'claude -p' runs headless."; return 0 ;;
@@ -4123,17 +4169,20 @@ function cmd_eval_converge { # same arg shape as cmd_eval, called only when --st
4123
4169
  echo " $fid: FAIL - could not build converge prompt (converge.md/repo-dir/challenge missing)"; total_miss=$((total_miss + 1)); rm -rf "$rundir" 2>/dev/null; continue
4124
4170
  fi
4125
4171
 
4172
+ local psha
4173
+ psha="$(_eval_prompt_sha "$promptfile")"
4174
+
4126
4175
  total_evaluated=$((total_evaluated + 1))
4127
- local hit_count=0 miss_count=0 invalid_count=0 i=1 model="unknown" retries_sum=0 vote_rate_limited=false
4176
+ local hit_count=0 miss_count=0 invalid_count=0 i=1 model="unknown" retries_sum=0 vote_rate_limited=false vote_timed_out=false
4128
4177
  while [ "$i" -le "$n" ]; do
4129
4178
  local raw rc v raw1="" rc1=0
4130
- raw="$(_eval_engine_run "$promptfile" "$timeout" 2>/dev/null)"; rc=$?; raw1="$raw"; rc1="$rc"
4131
- if [ "$rc" -eq 124 ]; then v=INVALID; else v="$(_eval_converge_parse_verdict "$raw" "$nonce")"; [ "$model" = "unknown" ] && model="$(_eval_parse_model "$raw")"; fi
4179
+ raw="$(_eval_isolated_run _eval_engine_run "$promptfile" "$timeout" 2>/dev/null)"; rc=$?; raw1="$raw"; rc1="$rc"
4180
+ if [ "$rc" -eq 124 ]; then v=INVALID; vote_timed_out=true; else v="$(_eval_converge_parse_verdict "$raw" "$nonce")"; [ "$model" = "unknown" ] && model="$(_eval_parse_model "$raw")"; fi
4132
4181
  local rl1; rl1="$(_eval_parse_rate_limited "$raw1")"; [ "$rl1" = "true" ] && vote_rate_limited=true
4133
4182
  if [ "$v" = "INVALID" ] && [ "$rl1" != "true" ] && [ "$rc1" -ne 124 ]; then
4134
4183
  if [ "$backoff" -gt 0 ]; then echo " $fid: retrying vote $i after ${backoff}s (parse-INVALID on attempt 1)"; sleep "$backoff" 2>/dev/null || true; fi
4135
- raw="$(_eval_engine_run "$promptfile" "$timeout" 2>/dev/null)"; rc=$?; retries_sum=$((retries_sum + 1))
4136
- if [ "$rc" -eq 124 ]; then v=INVALID; else v="$(_eval_converge_parse_verdict "$raw" "$nonce")"; fi
4184
+ raw="$(_eval_isolated_run _eval_engine_run "$promptfile" "$timeout" 2>/dev/null)"; rc=$?; retries_sum=$((retries_sum + 1))
4185
+ if [ "$rc" -eq 124 ]; then v=INVALID; vote_timed_out=true; else v="$(_eval_converge_parse_verdict "$raw" "$nonce")"; fi
4137
4186
  [ "$model" = "unknown" ] && [ "$rc" -ne 124 ] && model="$(_eval_parse_model "$raw")"
4138
4187
  local rl2; rl2="$(_eval_parse_rate_limited "$raw")"; [ "$rl2" = "true" ] && vote_rate_limited=true
4139
4188
  fi
@@ -4158,7 +4207,7 @@ function cmd_eval_converge { # same arg shape as cmd_eval, called only when --st
4158
4207
  total_miss=$((total_miss + 1)); echo " $fid: MISS - expected '$fexpected' (hit=$hit_count miss=$miss_count invalid=$invalid_count)"
4159
4208
  fi
4160
4209
 
4161
- _eval_converge_emit_result "$nonce" "$fid" "$fexpected" "$verdict" "$match" "$hit_count" "$miss_count" "$invalid_count" "$n" "$model" "$batch_cliv" "$batch_flowv" "$batch_rulessha" "$retries_sum" "$vote_rate_limited"
4210
+ _eval_converge_emit_result "$nonce" "$fid" "$fexpected" "$verdict" "$match" "$hit_count" "$miss_count" "$invalid_count" "$n" "$model" "$batch_cliv" "$batch_flowv" "$batch_rulessha" "$retries_sum" "$vote_rate_limited" "$psha" "$vote_timed_out"
4162
4211
  n_written=$((n_written + 1))
4163
4212
 
4164
4213
  if [ "$total_evaluated" -eq 1 ] && [ "$verdict" = "UNRELIABLE" ] && [ "$keep_going" -eq 0 ]; then
@@ -4297,7 +4346,7 @@ function cmd_eval {
4297
4346
 
4298
4347
  local nonce="" total_mismatch=0 total_unreliable=0 total_evaluated=0 rundir="" n_written=0 aborted=0
4299
4348
  # Computed ONCE per batch, not per fixture row - see _eval_emit_result's comment.
4300
- local batch_cliv="" batch_flowv="" batch_grsha=""
4349
+ local batch_cliv="" batch_flowv="" batch_grsha="" prompt_sha_acc="" prompt_sha_enforced=0
4301
4350
  if [ "$replay_mode" -eq 1 ]; then
4302
4351
  # Replay wrap: skip probe SKIP-exit and _eval_cli_version only. Still compute
4303
4352
  # batch_grsha (staleness hard-fail) and _flow_version. Nonce from fixture, never _eval_nonce.
@@ -4320,9 +4369,13 @@ function cmd_eval {
4320
4369
  echo " recorded=$rec_sha current=$batch_grsha"
4321
4370
  return 1
4322
4371
  fi
4372
+ if grep -q '^prompt_sha\.' "$EVAL_REPLAY_DIR/meta" 2>/dev/null; then
4373
+ prompt_sha_enforced=1
4374
+ fi
4323
4375
  echo "replay: keyless artifact batch (nonce from fixture, zero live calls)"
4324
4376
  else
4325
- local probe; probe="$(_eval_probe)"
4377
+ _eval_isolated_guard || return 1
4378
+ local probe; probe="$(_eval_isolated_run _eval_probe)"
4326
4379
  case "$probe" in
4327
4380
  absent)
4328
4381
  echo "SKIP: 'claude' CLI not found on PATH - eval needs it to run the LLM judge. Zero calls made."
@@ -4425,13 +4478,36 @@ function cmd_eval {
4425
4478
  continue
4426
4479
  fi
4427
4480
 
4481
+ local psha
4482
+ psha="$(_eval_prompt_sha "$promptfile")"
4483
+ if [ "$replay_mode" -eq 1 ] && [ "$prompt_sha_enforced" -eq 1 ]; then
4484
+ local rec_psha
4485
+ rec_psha="$(_eval_replay_meta_get "prompt_sha.$fid")" || rec_psha=""
4486
+ if [ -z "$rec_psha" ] || [ "$rec_psha" != "$psha" ]; then
4487
+ echo "eval --replay: fixtures stale — prompt assembly changed; re-record live per ADR re-baseline rule"
4488
+ echo " recorded=$rec_psha current=$psha fixture=$fid"
4489
+ rm -rf "$rundir" 2>/dev/null
4490
+ trap - INT TERM
4491
+ return 1
4492
+ fi
4493
+ fi
4494
+ if [ "$record_mode" -eq 1 ]; then
4495
+ prompt_sha_acc="${prompt_sha_acc}prompt_sha.${fid}=${psha}"$'\n'
4496
+ fi
4497
+
4428
4498
  total_evaluated=$((total_evaluated + 1))
4429
- local flag_count=0 pass_count=0 invalid_count=0 i=1 model="unknown" retries_sum=0 vote_rate_limited=false
4499
+ local flag_count=0 pass_count=0 invalid_count=0 i=1 model="unknown" retries_sum=0 vote_rate_limited=false vote_timed_out=false
4430
4500
  # Filename-safe fixture id for raw-capture paths - manifest fid is only tr -d '\r'-cleaned
4431
4501
  # (the v1 trust-boundary is READ-side only, but we are now WRITING keyed by it), so a
4432
4502
  # hand-edited or FLOW_EVAL_MANIFEST-overridden manifest cannot traverse out of eval-raw/.
4433
4503
  local fid_safe; fid_safe="$(printf '%s' "$fid" | tr -c 'A-Za-z0-9' '-' | sed -E 's/-+/-/g; s/^-//; s/-$//')"
4434
4504
  [ -z "$fid_safe" ] && fid_safe="_"
4505
+ if [ "$replay_mode" -eq 0 ]; then
4506
+ local rawdir_prompt="$LOG_DIR/eval-raw/$nonce"
4507
+ if mkdir -p "$rawdir_prompt" 2>/dev/null; then
4508
+ cp "$promptfile" "$rawdir_prompt/${fid_safe}.prompt" 2>/dev/null || true
4509
+ fi
4510
+ fi
4435
4511
  while [ "$i" -le "$n" ]; do
4436
4512
  local raw rc v raw1="" rc1=0 err1="" err2=""
4437
4513
  # Attempt 1 - stderr captured at the OUTER redirection so it never enters the sh -c
@@ -4442,11 +4518,12 @@ function cmd_eval {
4442
4518
  raw="$(_eval_replay_vote "$fid" "$i")" || { rm -rf "$rundir" 2>/dev/null; trap - INT TERM; return 1; }
4443
4519
  rc=0
4444
4520
  else
4445
- raw="$(_eval_engine_run "$promptfile" "$timeout" 2>"$err1")"; rc=$?
4521
+ raw="$(_eval_isolated_run _eval_engine_run "$promptfile" "$timeout" 2>"$err1")"; rc=$?
4446
4522
  fi
4447
4523
  raw1="$raw"; rc1="$rc"
4448
4524
  if [ "$rc" -eq 124 ]; then
4449
4525
  v=INVALID
4526
+ vote_timed_out=true
4450
4527
  else
4451
4528
  v="$(_eval_parse_verdict "$raw" "$nonce")"
4452
4529
  [ "$model" = "unknown" ] && model="$(_eval_parse_model "$raw")"
@@ -4477,10 +4554,10 @@ function cmd_eval {
4477
4554
  raw="$(_eval_replay_vote "$fid" "$i")" || { rm -rf "$rundir" 2>/dev/null; trap - INT TERM; return 1; }
4478
4555
  rc=0
4479
4556
  else
4480
- raw="$(_eval_engine_run "$promptfile" "$timeout" 2>"$err2")"; rc=$?
4557
+ raw="$(_eval_isolated_run _eval_engine_run "$promptfile" "$timeout" 2>"$err2")"; rc=$?
4481
4558
  fi
4482
4559
  retries_sum=$((retries_sum + 1))
4483
- if [ "$rc" -eq 124 ]; then v=INVALID; else v="$(_eval_parse_verdict "$raw" "$nonce")"; fi
4560
+ if [ "$rc" -eq 124 ]; then v=INVALID; vote_timed_out=true; else v="$(_eval_parse_verdict "$raw" "$nonce")"; fi
4484
4561
  [ "$model" = "unknown" ] && [ "$rc" -ne 124 ] && model="$(_eval_parse_model "$raw")"
4485
4562
  local rl2; rl2="$(_eval_parse_rate_limited "$raw")"
4486
4563
  [ "$rl2" = "true" ] && vote_rate_limited=true
@@ -4555,7 +4632,7 @@ function cmd_eval {
4555
4632
  fi
4556
4633
 
4557
4634
  if [ "$replay_mode" -eq 0 ]; then
4558
- _eval_emit_result "$nonce" "$fid" "$fstage" "$fexpected" "$verdict" "$match" "$flag_count" "$pass_count" "$invalid_count" "$n" "$model" "$batch_cliv" "$batch_flowv" "$batch_grsha" "$retries_sum" "$vote_rate_limited"
4635
+ _eval_emit_result "$nonce" "$fid" "$fstage" "$fexpected" "$verdict" "$match" "$flag_count" "$pass_count" "$invalid_count" "$n" "$model" "$batch_cliv" "$batch_flowv" "$batch_grsha" "$retries_sum" "$vote_rate_limited" "$psha" "$vote_timed_out"
4559
4636
  fi
4560
4637
  n_written=$((n_written + 1))
4561
4638
 
@@ -4587,7 +4664,7 @@ function cmd_eval {
4587
4664
  _eval_emit_batch_marker "$nonce" done "$n_written"
4588
4665
  fi
4589
4666
  if [ "$record_mode" -eq 1 ] && [ "$aborted" -eq 0 ]; then
4590
- _eval_replay_write_meta "$nonce" "$batch_grsha" "${model:-unknown}" "$n"
4667
+ _eval_replay_write_meta "$nonce" "$batch_grsha" "${model:-unknown}" "$n" "$prompt_sha_acc"
4591
4668
  fi
4592
4669
 
4593
4670
  echo
@@ -1,8 +1,8 @@
1
1
  {
2
2
  "source": "../skills/flow",
3
3
  "destRelative": "skills/flow",
4
- "fileCount": 102,
5
- "syncedAt": "2026-08-14T09:19:34.115Z",
4
+ "fileCount": 103,
5
+ "syncedAt": "2026-08-20T13:35:30.768Z",
6
6
  "fileList": [
7
7
  "SKILL.md",
8
8
  "_templates/00-idea.md",
@@ -97,6 +97,7 @@
97
97
  "references/gate-eval.md",
98
98
  "references/gate-rules.md",
99
99
  "references/ground-truth-gates.md",
100
+ "references/host-agnostic-parallel.md",
100
101
  "references/loop-harness-2026-principles.md",
101
102
  "references/mode-work.md",
102
103
  "references/native-rituals.md",