forgedock 1.30.0 → 1.32.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (162) hide show
  1. package/bin/engine/outcome-codes.mjs +3 -2
  2. package/bin/engine/runlog.mjs +82 -12
  3. package/bin/engine/telemetry-seam.mjs +11 -4
  4. package/bin/engine/telemetry-seam.test.mjs +18 -0
  5. package/bin/engine-cli.mjs +409 -38
  6. package/bin/engine.mjs +47 -37
  7. package/bin/forgedock.mjs +35 -3
  8. package/bin/gc.mjs +50 -17
  9. package/bin/hooks/reread-nudge.mjs +65 -0
  10. package/bin/labels.json +5 -0
  11. package/bin/lean/apishape.mjs +170 -0
  12. package/bin/lean/apishape.test.mjs +187 -0
  13. package/bin/lean/batch.mjs +32 -0
  14. package/bin/lean/batch.test.mjs +338 -2
  15. package/bin/lean/budget.mjs +89 -0
  16. package/bin/lean/budget.test.mjs +106 -0
  17. package/bin/lean/checks.mjs +32 -9
  18. package/bin/lean/checks.test.mjs +78 -1
  19. package/bin/lean/conformance.e2e.test.mjs +27 -5
  20. package/bin/lean/conformance.mjs +150 -15
  21. package/bin/lean/conformance.test.mjs +175 -3
  22. package/bin/lean/context.mjs +128 -44
  23. package/bin/lean/context.test.mjs +157 -5
  24. package/bin/lean/decompose.mjs +90 -0
  25. package/bin/lean/decompose.test.mjs +105 -0
  26. package/bin/lean/depcache.mjs +11 -7
  27. package/bin/lean/depcache.test.mjs +81 -0
  28. package/bin/lean/e2e.mjs +102 -0
  29. package/bin/lean/e2e.test.mjs +86 -0
  30. package/bin/lean/effects.mjs +48 -15
  31. package/bin/lean/explore.mjs +47 -0
  32. package/bin/lean/explore.test.mjs +61 -0
  33. package/bin/lean/facts.mjs +84 -0
  34. package/bin/lean/facts.test.mjs +66 -0
  35. package/bin/lean/fixround.mjs +90 -0
  36. package/bin/lean/fixround.test.mjs +117 -0
  37. package/bin/lean/heavy.mjs +35 -2
  38. package/bin/lean/heavy.test.mjs +32 -2
  39. package/bin/lean/lean.mjs +712 -134
  40. package/bin/lean/lean.test.mjs +1626 -37
  41. package/bin/lean/ledger.mjs +5 -3
  42. package/bin/lean/ledger.test.mjs +18 -0
  43. package/bin/lean/light.mjs +135 -0
  44. package/bin/lean/light.test.mjs +173 -0
  45. package/bin/lean/lightlane.mjs +219 -0
  46. package/bin/lean/lightlane.test.mjs +237 -0
  47. package/bin/lean/limits.test.mjs +36 -1
  48. package/bin/lean/orchestrate.mjs +303 -41
  49. package/bin/lean/pitfalls.mjs +136 -0
  50. package/bin/lean/pitfalls.test.mjs +117 -0
  51. package/bin/lean/postmerge.mjs +85 -0
  52. package/bin/lean/postmerge.test.mjs +79 -0
  53. package/bin/lean/precheck.mjs +138 -13
  54. package/bin/lean/precheck.test.mjs +118 -1
  55. package/bin/lean/prefilter.mjs +220 -5
  56. package/bin/lean/prefilter.test.mjs +260 -4
  57. package/bin/lean/preflight.mjs +108 -3
  58. package/bin/lean/preflight.test.mjs +48 -1
  59. package/bin/lean/probe.mjs +297 -0
  60. package/bin/lean/probe.test.mjs +210 -0
  61. package/bin/lean/prompts.mjs +104 -13
  62. package/bin/lean/render.mjs +7 -3
  63. package/bin/lean/resume.mjs +67 -0
  64. package/bin/lean/resume.test.mjs +84 -0
  65. package/bin/lean/review-trigger.mjs +12 -0
  66. package/bin/lean/review-trigger.test.mjs +17 -1
  67. package/bin/lean/routing.mjs +74 -0
  68. package/bin/lean/routing.test.mjs +67 -0
  69. package/bin/lean/sched.mjs +16 -1
  70. package/bin/lean/sched.test.mjs +175 -1
  71. package/bin/lean/schedd.mjs +55 -9
  72. package/bin/lean/schedd.test.mjs +170 -3
  73. package/bin/lean/selfreview.mjs +37 -0
  74. package/bin/lean/selfreview.test.mjs +51 -0
  75. package/bin/lean/seqlease.mjs +40 -0
  76. package/bin/lean/seqlease.test.mjs +101 -0
  77. package/bin/lean/sequence.mjs +296 -13
  78. package/bin/lean/sequence.test.mjs +263 -2
  79. package/bin/lean/toolchain.mjs +60 -15
  80. package/bin/lean/toolchain.test.mjs +120 -3
  81. package/bin/lean/toolstats.mjs +72 -0
  82. package/bin/lean/toolstats.test.mjs +47 -0
  83. package/bin/lean/verifier.mjs +232 -0
  84. package/bin/lean/verifier.test.mjs +163 -0
  85. package/bin/lean/verify.mjs +72 -10
  86. package/bin/lean/verify.test.mjs +68 -3
  87. package/bin/review-staging.mjs +533 -0
  88. package/bin/runner.mjs +82 -13
  89. package/bin/stats/aggregate.mjs +87 -2
  90. package/bin/stats/efficiency.mjs +335 -0
  91. package/bin/stats/efficiency.test.mjs +266 -0
  92. package/bin/stats/github-source.mjs +8 -3
  93. package/bin/stats/index.mjs +91 -6
  94. package/bin/stats/local-source.mjs +145 -8
  95. package/bin/stats/query.mjs +181 -3
  96. package/bin/stats/render.mjs +22 -0
  97. package/bin/stats/stats.test.mjs +403 -8
  98. package/commands/orchestrate.md +3 -1
  99. package/commands/review-pr-agents/protocols.md +1 -1
  100. package/commands/review-pr-staging.md +84 -831
  101. package/commands/review-pr.md +1 -1
  102. package/commands/work-on/build/implement.md +1 -0
  103. package/commands/work-on/build/validate.md +11 -1
  104. package/docs/CONFIG.md +80 -5
  105. package/docs/OUTCOME-CODES.md +2 -1
  106. package/docs/STATS.md +66 -1
  107. package/docs/TELEMETRY.md +1 -1
  108. package/docs/eval/corpus/lightlane-v1.json +55 -0
  109. package/docs/eval/lightlane-bench.md +91 -0
  110. package/docs/spec/review-protocol.md +1 -1
  111. package/package.json +1 -1
  112. package/scripts/bench-lightlane.mjs +286 -0
  113. package/scripts/ci-local/lib.mjs +20 -0
  114. package/scripts/ci-local/plan.mjs +2 -12
  115. package/scripts/ci-local/scoping.mjs +10 -1
  116. package/scripts/ci-local.test.sh +2 -0
  117. package/scripts/fixtures/apishape/other/go-change/base/x.go +3 -0
  118. package/scripts/fixtures/apishape/other/go-change/head/x.go +3 -0
  119. package/scripts/fixtures/apishape/python/generator-function/base/seq.py +3 -0
  120. package/scripts/fixtures/apishape/python/generator-function/head/seq.py +2 -0
  121. package/scripts/fixtures/apishape/python/ichunked/base/more.py +17 -0
  122. package/scripts/fixtures/apishape/python/ichunked/head/more.py +20 -0
  123. package/scripts/fixtures/apishape/python/private-edit/base/calc.py +6 -0
  124. package/scripts/fixtures/apishape/python/private-edit/head/calc.py +9 -0
  125. package/scripts/fixtures/apishape/python/removed/base/ops.py +6 -0
  126. package/scripts/fixtures/apishape/python/removed/head/ops.py +2 -0
  127. package/scripts/fixtures/apishape/python/renamed/base/text.py +2 -0
  128. package/scripts/fixtures/apishape/python/renamed/head/text.py +2 -0
  129. package/scripts/fixtures/apishape/python/signature/base/shapes.py +2 -0
  130. package/scripts/fixtures/apishape/python/signature/head/shapes.py +2 -0
  131. package/scripts/fixtures/apishape/python/sync-async/base/client.py +7 -0
  132. package/scripts/fixtures/apishape/python/sync-async/head/client.py +7 -0
  133. package/scripts/fixtures/apishape/ts/generator-function/base/seq.js +3 -0
  134. package/scripts/fixtures/apishape/ts/generator-function/head/seq.js +5 -0
  135. package/scripts/fixtures/apishape/ts/private-edit/base/calc.ts +7 -0
  136. package/scripts/fixtures/apishape/ts/private-edit/head/calc.ts +9 -0
  137. package/scripts/fixtures/apishape/ts/removed/base/ops.ts +7 -0
  138. package/scripts/fixtures/apishape/ts/removed/head/ops.ts +3 -0
  139. package/scripts/fixtures/apishape/ts/renamed/base/text.ts +3 -0
  140. package/scripts/fixtures/apishape/ts/renamed/head/text.ts +3 -0
  141. package/scripts/fixtures/apishape/ts/signature/base/shapes.ts +3 -0
  142. package/scripts/fixtures/apishape/ts/signature/head/shapes.ts +3 -0
  143. package/scripts/fixtures/apishape/ts/sync-async/base/client.mjs +5 -0
  144. package/scripts/fixtures/apishape/ts/sync-async/head/client.mjs +5 -0
  145. package/scripts/forge-root.test.sh +2 -2
  146. package/scripts/publish-staging-sync.test.sh +62 -4
  147. package/scripts/review-finalize.sh +17 -6
  148. package/scripts/review-finalize.test.sh +47 -0
  149. package/scripts/review-pr-arguments.test.sh +13 -1
  150. package/scripts/review-pr-delta-wiring.test.sh +2 -2
  151. package/scripts/review-pr-gate2.test.sh +6 -14
  152. package/scripts/review-pr-staging-finding-filter.test.sh +4 -4
  153. package/scripts/review-pr-staging-scope.test.sh +39 -33
  154. package/scripts/review-prepare.sh +73 -6
  155. package/scripts/review-prepare.test.sh +85 -0
  156. package/scripts/review-staging-post.sh +293 -0
  157. package/scripts/review-staging-post.test.sh +143 -0
  158. package/scripts/review-staging-pre.sh +401 -0
  159. package/scripts/review-staging-pre.test.sh +150 -0
  160. package/scripts/symbol-index.mjs +26 -7
  161. package/scripts/symbol-index.py +55 -9
  162. package/scripts/trusted-comments.test.sh +1 -1
@@ -655,7 +655,7 @@ bash "$RF" classify --pr "$PR_NUMBER" --repo "$REPO" --context "$CTX" --out-dir
655
655
  cat "$OUT_DIR/note-list.txt" 2>/dev/null
656
656
  ```
657
657
 
658
- `extract` reads the trusted synthesized block for the head, else its trusted agent bodies (`BASE_SYNC_ONLY`: the carried head; `DELTA`: this head plus the earlier reviewed one), parses every `<!-- FINDING:... -->` marker (an unparseable marker blocks the review), dedupes by file + line range ±5 and by title (3+ shared keywords, higher confidence wins), and applies the **§6B.4 gates** in order, each keeping the finding when its input is unreadable: (1) **head-SHA**: a cited line past the end of the file at `REVIEW_SHA` is stale and DROPPED (`findings_dropped_stale`); (2) **introduced-line**: a MEDIUM-or-lower finding whose range (±5) misses the PR's added lines is `pre-existing` (**CRITICAL/HIGH are never routed as pre-existing**, whatever their confidence, `protocols.md` §5; input comes from the diff, never a reviewer's `**Scope**` tag; removed files, old names of renames and patch-less files are scope `unknown`, never pre-existing and never introduced; a renamed file with a patch is scored by its added lines); (3) **concrete-failure**: a MEDIUM-or-lower finding without a `**Failure scenario**` becomes a NOTE (`notes_demoted_no_scenario`).
658
+ `extract` reads the trusted synthesized block for the head, else its trusted agent bodies (`BASE_SYNC_ONLY`: the carried head; `DELTA`: this head plus the earlier reviewed one), parses every `<!-- FINDING:... -->` marker (an unparseable marker blocks the review), dedupes by file + line range ±5 and by title (3+ shared keywords, higher confidence wins), and applies the **§6B.4 gates** in order, each keeping the finding when its input is unreadable: (1) **head-SHA**: a cited line past the end of the file at `REVIEW_SHA` is stale and DROPPED (`findings_dropped_stale`); (2) **introduced-line**: a MEDIUM-or-lower finding whose range (±5) misses the PR's added lines is `pre-existing` (**CRITICAL/HIGH are never routed as pre-existing**, whatever their confidence, `protocols.md` §5; input comes from the diff, never a reviewer's `**Scope**` tag; removed files, old names of renames and patch-less files are scope `unknown`, never pre-existing and never introduced; a renamed file with a patch is scored by its added lines); (3) **concrete-failure**: a MEDIUM-or-lower finding with no failure scenario in its prose (a `**Failure scenario**`, or a line starting with a bold or plain `Failure scenario`, `Scenario`, `Repro`, `Reproduction` or `Steps to reproduce` label) becomes a NOTE (`notes_demoted_no_scenario`); **CONFIRMED MEDIUM and CRITICAL/HIGH are never demoted** this way and are filed with the agent comment as evidence.
659
659
 
660
660
  Read `STRUCTURED_FINDINGS`, `FINDINGS_KEPT`, `FINDINGS_DROPPED_STALE`, `FINDINGS_PREEXISTING` and `NOTES_DEMOTED_NO_SCENARIO` for the record and summary. The caller's part of 6A: add the Phase 2.5 `INTEG-N` findings no reviewer covered (CONFIRMED HIGH, so ISSUE, filed in 6C); findings come only from `FINDING:` markers, never from free text. If `AGENTS_WITHOUT_BLOCK` is non-empty those agents are undelivered: re-dispatch them once (Phase 4 rule), else take the degraded-panel path; `CLEAN_STRUCTURED` is withheld while it is non-empty. On `CLEAN_STRUCTURED=true` and no `INTEG-N` the review is clean: skip to Phase 7. A finding that somehow has no marker is classified with `classify-finding.sh --unstructured` (empty or unparseable severity gives `NOTE unstructured`, never ISSUE). **Pre-existing route**: a pre-existing finding (MEDIUM or lower) is never filed as `review-finding`/`needs-validation` nor linked to this PR (it stays out of findings-per-PR and the amplification ratio); it still passes the §6B.5 classifier, is listed once in the summary, and at most 3 per review (highest severity first) are filed with the `pre-existing` label plus `priority:*`, citing this PR (`preexisting_filed`, not `findings_filed`).
661
661
 
@@ -167,6 +167,7 @@ Work in `{WORKTREE_PATH}`. Follow the contract deliverables table exactly — im
167
167
  - For each acceptance criterion in the contract: implement it, then mentally verify it's met
168
168
  - Do NOT add unrequested scope — contract out-of-scope items stay out of scope
169
169
  - **Library callback verification**: When writing a lambda or callable that will be passed to a library/framework parameter (e.g., `prepared_statement_name_func=lambda: ""`, `key=lambda x: ...`), you MUST verify the expected calling convention BEFORE writing it. Check the library's default value for that parameter, its documentation, or its source code. A lambda with wrong arity causes `TypeError` at runtime — this is invisible to static analysis and linting. The P0 incident from PR #14391 was caused by `lambda _: ""` (1 arg) passed where SQLAlchemy expects 0 args.
170
+ - **Self-check scope**: verify with the configured `forge.yaml → verification.commands` when present (a configured `typescript.lint` is the only lint you run). Otherwise lint and test only the changed files or packages; never lint a dependency closure (turbo `--filter=...[base]`, nx affected), and leave whole-repo checks to CI. <!-- Added: forge#4381 -->
170
171
  - **Scratch placement (venvs, caches)**: create virtualenvs, package caches and other large scratch data under `"$TMPDIR/forge-venv-{NUMBER}"` (the engine exports a per-run, disk-backed `TMPDIR`) or inside the worktree. Never use a literal `/tmp/<name>` path: a size-limited `/tmp` fails concurrent lanes with EDQUOT/ENOSPC. Remove what you created when the run finishes. <!-- Added: forge#3572 -->
171
172
  - **Worktree-aware path derivation**: When writing shell code that derives repository paths, ALWAYS use `git rev-parse --show-toplevel` for the repo root in regular checkouts, or `git rev-parse --git-common-dir` (then `dirname`) to get the shared `.git` directory when the context may be a linked worktree. For worktree cleanup code specifically, ALWAYS use `--git-common-dir` — `--show-toplevel` returns the worktree path itself, NOT the main repo root. NEVER use `pwd`, relative paths, or `dirname` chains on `--show-toplevel` output for repo root derivation. Test path logic for both regular checkouts and linked worktrees. (Ref: review-findings #104, #105 — 4 PATH_DERIVATION defects, 15% of all review findings)
172
173
  - **State machine completeness verification**: When implementing routing logic, state transition tables, or phase dispatch code (e.g., adding a `Skill()` call in a routing loop, adding a new phase to a state machine, creating a new subcommand file), you MUST run three checks BEFORE staging: (1) **Routing target existence** — for each `Skill("subcommand", ...)` call or file reference in routing logic, verify the target file exists at `commands/{subcommand-path}.md`; (2) **State reachability** — for each declared state or phase, verify at least one prior state or entry condition in the router routes to it; (3) **Subcommand invocation wiring** — for each `commands/work-on/*.md` file that declares its invocation condition (e.g., `Invoked by work-on.md routing loop, when X`), verify that condition is actually handled in the router. A missing target file, an unreachable state, or a declared-but-unwired subcommand will not surface until review. (Ref: review-findings #85, #116, #137 — 4 ROUTING defects, 15% of all review findings)
@@ -214,6 +214,7 @@ FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common
214
214
  TS_FORMAT=$(yq '.verification.commands.typescript.format // ""' "$FORGE_CFG" 2>/dev/null || echo '')
215
215
  TS_TYPECHECK=$(yq '.verification.commands.typescript.typecheck // ""' "$FORGE_CFG" 2>/dev/null || echo '')
216
216
  TS_BUILD=$(yq '.verification.commands.typescript.build // ""' "$FORGE_CFG" 2>/dev/null || echo '')
217
+ TS_LINT=$(yq '.verification.commands.typescript.lint // ""' "$FORGE_CFG" 2>/dev/null || echo '')
217
218
 
218
219
  if [ -n "$TS_FORMAT" ]; then
219
220
  run_verification_command "typescript.format" "$TS_FORMAT"
@@ -234,8 +235,17 @@ else
234
235
  SKIPPED_CHECKS="${SKIPPED_CHECKS:+$SKIPPED_CHECKS, }typescript.typecheck/build"
235
236
  TS_EXIT=0
236
237
  fi
238
+
239
+ if [ -n "$TS_LINT" ]; then
240
+ run_verification_command "typescript.lint" "$TS_LINT"
241
+ TS_LINT_EXIT=$?
242
+ else
243
+ echo "SKIPPED — typescript.lint not configured in verification.commands"
244
+ SKIPPED_CHECKS="${SKIPPED_CHECKS:+$SKIPPED_CHECKS, }typescript.lint"
245
+ TS_LINT_EXIT=0
246
+ fi
237
247
  ```
238
- Typecheck or build failures are BLOCKING — fix type errors before continuing.
248
+ Typecheck, build or lint (`TS_LINT_EXIT`) failures are BLOCKING — fix them before continuing. Configure `typescript.lint` as a changed-files, cached lint; it is the lane's only lint (forge#4381).
239
249
 
240
250
  **Shell scripts**: Verify service interactions — read target middleware files, document what was verified in the V5 summary.
241
251
 
package/docs/CONFIG.md CHANGED
@@ -554,17 +554,20 @@ engine:
554
554
  | `<key>_model` | string | No | Model alias (`sonnet`, `opus`, `haiku`) or full model id passed as `--model`. Invalid values are ignored. |
555
555
  | `<key>_max_turns` | integer | No | Positive integer passed as `--max-turns`. Invalid values are ignored. |
556
556
 
557
- | `path` | string | No | `core` (default), `full`, `lean` or `auto`. `core` is the lean work-on (`bin/lean/`): a planning call that writes a frozen contract, an implement call in the same session, one read-only conformance reviewer, GitHub as the register (see `conformance`). `lean` and `auto` are aliases of `core`; `full` runs the legacy multi-phase pipeline and is required for `--retry` and `--wait-lease`. The `--path` flag overrides. `lean_reviewer_model` sets the reviewer model (default `claude-sonnet-5-5`); the builder uses `--model` or the CLI default. `forgedock orchestrate <issues...> [--parallel K] [--lane L]` runs the core per issue (K default 3, lane default `staging`) and honours `**Depends on**` ordering among the given issues. |
557
+ | `path` | string | No | `core` (default), `full`, `lean` or `auto`. `core` is the lean work-on (`bin/lean/`): a planning call that writes a frozen contract, an implement call in the same session, one read-only conformance reviewer, GitHub as the register (see `conformance`). `lean` and `auto` are aliases of `core`; `full` runs the legacy multi-phase pipeline and is required for `--retry` and `--wait-lease`. The `--path` flag overrides. `lean_reviewer_model` sets the reviewer model (default `claude-sonnet-5-5`); the builder uses `--model` or the CLI default. `forgedock orchestrate <issues...> [--parallel K] [--lane L]` runs the core per issue (K default 3, lane default `staging`) and honours `**Depends on**` ordering among the given issues. A lane that reports an open dependency pulls it into the batch, unless that dependency is labelled `needs-human` or already ended `needs-human` in the batch: then the issue and its dependents end `skipped (dependency #N needs-human)` without a lane for it (forge#4387). |
558
558
  | `conformance` | string | No | Core work-on conformance gate. The builder first writes a contract (requirements quoted verbatim from the issue and maintainers' comments, deliverables, preserve constraints, validate commands), which the engine freezes and posts as `FORGE:CONTRACT`; changes go only through typed amendments (`FORGE:AMENDMENT`). A deterministic precheck (scope, validation, deferrals) and a row-based conformance review follow, and the engine computes the verdict. `shadow` (default): the verdict is posted in `FORGE:REVIEW` and logged (`CONFORMANCE_SHADOW` event) but only what the previous reviewer blocked on still blocks (a requirement found not implemented, a regression), plus the precheck's equivalent: any `conformance.unimplemented` precheck or amendment failure (a validate command that failed or has no recorded result, a deliver requirement the builder reported as not implemented, an invalid amendment; #4160). These block before any reviewer run. Scope creep and omissions are recorded only. A review still incomplete or inconsistent after one targeted re-ask (only the missing ids) ends `needs-human`, or with `review_incomplete: degraded` merges with a `review-degraded` PR label and a tracked follow-up. Accepted deferrals are filed and linked, as in enforce. `scripts/calibration.mjs --shadow` reports the per-repo shadow verdict mix and when to switch to enforce. `enforce`: every failure blocks (one same-session builder resume, then `needs-human` with a `conformance.*`/`review.*` code, see docs/OUTCOME-CODES.md); deferrals the reviewer accepts are filed in one batched follow-up issue (`scripts/file-deferred-followups.sh`) and linked in the PR body and a `FORGE:TRACKED` comment before merge. |
559
559
  | `cli_idle_minutes` | number | No | Core builder idle timeout (#4112): the builder CLI runs with `--output-format stream-json` and is killed (code `cli.idle`) when it shows no stream activity for this many minutes. The run log records the last tool call and the live child processes. Default 15. |
560
+ | `cli_busy_minutes` | number | No | Extended idle window for a core builder (#4405). When the idle timeout is reached but the session has a live heavy step (a heavy-shim step, test runner, linter or type checker child) or a queued or running heavy-queue entry of its own processes, the kill waits until `cli_idle_minutes` + this many minutes of silence. The run log records `CLI_IDLE_EXTENDED` once per silent stretch, and a later `CLI_IDLE_KILL` carries `heavyQueue` ({queued, running}) and `extended: true`. A silent session with no such child is still killed at `cli_idle_minutes`. Default 45 (the heavy queue's 30-minute maximum wait plus the 15-minute idle default). |
560
561
  | `cli_max_minutes` | number | No | Absolute ceiling for a core builder session (code `cli.timeout`), replacing the fixed 60-minute kill. A session that keeps making progress is never idle-killed. Default 180. On either kill, uncommitted work is committed as `wip(forge): checkpoint after <reason>` and the lane branch is pushed (with the push retry), the builder retries once from the checkpoint, and a second kill ends `deferred`; a rerun resumes from the branch. |
562
+ | `turn_budget_s` / `turn_budget_m` / `turn_budget_l` | number | No | Core implement turn budget per lane size (#4392). Size comes from a `size:S|M|L` issue label, else the frozen contract's distinct deliverable paths (S ≤ 2, M ≤ 5, else L). The build call and its context/timeout restarts share the budget; at the limit the session is stopped, checkpointed, and one fresh wrap-up session of max(5, budget/5) turns either finishes or writes `status: budget-exceeded` with a digest. A stop or overrun ends `needs-human` with code `budget.exceeded`, which `forgedock orchestrate` recovers once (continue from the checkpoint or split). Lane results carry `turns: {size, budget, build, total, exceeded}`, and the batch report prints `turns by size`. Defaults 25 / 60 / 100; 0 disables that size. |
561
563
  | `lane_limits` | map | No | Cap each core lane's whole process tree (#4115), under `pipeline:` or `engine:`: `memory_max` (for example `3G`) and `cpu_quota` (for example `150%`). When set and `systemd-run --user --scope` works (probed once), the builder CLI is spawned as `systemd-run --user --scope -q -p MemoryMax=... -p CPUQuota=... -- claude ...`; elsewhere the caps are ignored. Write it nested or flat (`lane_limits_memory_max`, `lane_limits_cpu_quota`). While a builder runs the engine polls host load and memory every 15 s against the `admission` thresholds: over them it SIGSTOPs the lane's process group, under them SIGCONT, and never pauses longer than 10 minutes. |
562
564
  | `admission` | map | No | Host-resource admission for the core work-on (#4105), checked before the lease, worktree or any GitHub call. When a check fails the run ends `deferred` (code `host.busy`, exit 75) with a last line such as `issue #N → deferred (host busy: mem 1.2G < 2G; retry in 60s)`; no lease is taken and nothing is posted. Keys: `enabled` (default `true`; `false` disables admission), `min_free_mem_mb` (default 2048; `MemAvailable` from `/proc/meminfo`, else `os.freemem()`), `min_free_tmp_mb` (default 1024; free space of the run TMPDIR filesystem, `~/.forge/tmp`, and of `/tmp`, via `statfs`), `max_load_per_core` (default 4; 1-minute load average divided by cores). Write it nested (`engine:` then `admission:` with the keys indented) or flat as `admission_enabled`, `admission_min_free_mem_mb`, `admission_min_free_tmp_mb`, `admission_max_load_per_core`; the parser reads one nested level into `engine`. `/orchestrate` requeues a host-busy lane and lowers K by 1 for 5 minutes. |
563
565
  | `merge_wait` | string | No | Core work-on merge gate. `required` (default): wait only for the status checks the base branch requires: the union of classic branch protection (`gh api repos/{o}/{r}/branches/{base}/protection/required_status_checks`, contexts and checks) and repository rulesets (`gh api repos/{o}/{r}/rules/branches/{base}`, `required_status_checks` rules); with neither (404 or empty) merge right after review PASS. A merge rejected by base branch policy (PR `MERGEABLE` but `BLOCKED`) is waited on and retried within the CI budget, then left `ci-pending`; it is never base-synced and never `base.conflict`. A failing required check is `needs-human`; a pending one is polled with backoff and never `needs-human`. `all`: wait for every check (`scripts/wait-ci-green.sh`). `none`: merge right after review. Always `--match-head-commit` at the reviewed head. The policy used is recorded in the `FORGE:WORK-ON` comment. |
564
- | `context_pack` | string | No | Context pack (#4275). Before the plan call the core compiles the issue's GitHub/git neighbourhood into a small, budgeted pack: linked issues and PRs (parent, `Depends on`, body links, same-repo cross-references, siblings), prior attempts, merged PRs on the issue's files that were reverted or whose decisions/findings name a symbol from the issue, and repo-fact file pointers. `shadow` (default): the pack is compiled, posted on the issue as a collapsed `FORGE:CONTEXT` comment and logged as a `CONTEXT_PACK` event; the builder prompt is unchanged. `on`: it is also appended to the plan prompt as hints. `off`: nothing is gathered, posted or logged. The pack never enters the text contract quotes are checked against. Values read from linked issues' `FORGE:STATE` are kept only when they are outcome codes. Compare with `forgedock stats --context-ab`. |
565
- | `context_pack_sources` | string | No | Comma list of the pack sources to use: `graph`, `attempts`, `ledger`, `facts`. Default: all four. |
566
+ | `context_pack` | string | No | Context pack (#4275). Before the plan call the core compiles the issue's GitHub/git neighbourhood into a small, budgeted pack: linked issues and PRs (parent, `Depends on`, body links, same-repo cross-references, siblings), prior attempts, merged PRs on the issue's files that were reverted or whose decisions/findings name a symbol from the issue, structured `FORGE:FACTS` records of earlier lanes on the issue's files (rejected approaches with why, hot spots, decisions, conventions, baselines; #4401), and repo-fact file pointers (found in the lane worktree or, for untracked caches, the main checkout; #4400). `shadow` (default): the pack is compiled, posted on the issue as a collapsed `FORGE:CONTEXT` comment and logged as a `CONTEXT_PACK` event; the builder prompt is unchanged. `on`: it is also appended to the plan prompt as hints. `off`: nothing is gathered, posted or logged. `ab` (#4444): the A/B experiment. Each issue gets a stable arm, `on` or `shadow`, from a hash of `repo#issue`, logged as that mode with `context_ab: true`; the default stays `shadow`. The pack never enters the text contract quotes are checked against. Values read from linked issues' `FORGE:STATE` are kept only when they are outcome codes. Compare with `forgedock stats --context-ab` (per lane type and mode: first-pass yield, turns and cache-read per merge, rework rate, and recovery lanes' implement attempt against the first run on the same issue). Unless `off`, a resumed or recovery lane also gets a bounded resume digest of its last run in the builder prompt (end reason, validate results per command, deliver requirements, files changed; logged as `RESUME_DIGEST`; #4400), whatever the mode. |
567
+ | `context_pack_sources` | string | No | Comma list of the pack sources to use: `graph`, `attempts`, `pitfalls`, `ledger`, `facts` (earlier lanes' `FORGE:FACTS` on the issue's files plus repo-fact file pointers). Default: all five. |
566
568
  | `context_pack_bytes` | integer | No | Hard byte cap of the pack; items that do not fit are dropped whole and counted. Default: 6144. |
567
569
  | `context_pack_days` | integer | No | Window of the merged-PR ledger, in days. Default: 30. |
570
+ | `lane_instructions` | string | No | Lane profile (#4395). Core lane sessions always run with Claude Code auto-memory off (`CLAUDE_CODE_DISABLE_AUTO_MEMORY=1`). Set this to a repo-relative file, for example `devdocs/lane-instructions.md`, that holds only the hard rules plus pointers into docs. It is read from the lane worktree, so it is versioned with the repo. Lanes then also skip CLAUDE.md discovery (`CLAUDE_CODE_DISABLE_CLAUDE_MDS=1`), and the file text is appended to the system prompt of every fresh builder, reviewer and verifier session. An absolute path, a `..` path, or a missing, empty or over-16 KB file is a trajectory anomaly; lanes then keep CLAUDE.md. Settings and hooks always load (no `--setting-sources`). `FORGEDOCK_FULL_SESSION=1` keeps a full session. Each `phase_attempt` telemetry record carries `lane_profile` (`full`, `memory-off` or `lane`) and `baseline_tokens`, the first turn's context of a fresh session, so baseline growth is visible. `forgedock stats --lane-profile [--issues <list>]` compares `baseline_tokens` per phase and the conformance rate per profile from local run logs, for the before/after check on the same issue set. |
568
571
  | `ci_pending_max_minutes` | integer | No | Minutes the engine waits for CI when the review phase ends BLOCKED with "checks still pending", or the remediate phase holds at its M7 re-gate on "CI pending" (the merge gate's wait window ended while CI was still running). Default: 90. Invalid values are ignored. This key is flat like the others but is not part of the `<key>_model` / `<key>_max_turns` pattern. |
569
572
 
570
573
  **CI still pending**: the engine polls `scripts/wait-ci-green.sh` (no model calls) one read per poll on a 60 s, 2 min, 4 min, 5 min (cap) backoff with +/-20% jitter, while the lease keeps renewing, then re-dispatches the phase that was waiting, review or remediate (idempotent at an unchanged head: review re-runs the merge gate and merges; remediate resumes at its M7 re-gate). Before each poll it reads `gh api rate_limit` (not itself rate-limited) and, below 10% of the REST budget, sleeps until the window resets instead of polling; a rate-limited or transient (5xx, network) poll is retried and never re-dispatches the phase. Red CI takes the normal ci-gate path. When the budget is exhausted the run ends `needs-human` with outcome code `review.ci_pending_timeout`; `--retry` resumes. Each wait emits a `ci_pending_wait` telemetry event (`docs/TELEMETRY.md`) with the minutes waited, polls made and rate-limit sleeps.
@@ -592,12 +595,54 @@ lean:
592
595
  repro_test: prefer # off | prefer (default)
593
596
  precheck:
594
597
  max_minutes: 10 # optional pre-PR check round budget in minutes (1-120)
598
+ fix_model: claude-sonnet-5-5 # optional model of fresh fix-round sessions (default: the reviewer model)
599
+ builder_routing: on # on (default) | off: route the build to a mid-tier model by the frozen contract
600
+ builder_mid_model: claude-sonnet-5-5 # optional mid-tier builder model (default claude-sonnet-5-5)
601
+ light:
602
+ mode: shadow # off | shadow (default) | on: light-lane eligibility (#4448; lane wiring #4450)
603
+ max_lines: 40 # upgrade to the full lane above this many non-test diff lines
604
+ max_files: 2 # max predicted / changed non-test files
605
+ sequence:
606
+ regenerate: pnpm --filter @x/db db:generate # optional snapshot generator after a deterministic renumber
595
607
  ```
596
608
 
597
609
  | Field | Type | Required | Description |
598
610
  |-------|------|----------|-------------|
599
611
  | `repro_test` | string | No | `prefer` (default): on an issue labelled `bug` or `type:bug`, the plan step must freeze a `repro_test` in the contract (`{"path","id","command"}`) when the bug is testable. Before review, the engine runs `scripts/acceptance-red-green.sh` for that command on the host in every verify mode: one targeted test, red on the base (head's changed test files overlaid) and green on head. A test that ran but is not red on the base and green on head fails the precheck as `conformance.unimplemented` (it blocks in shadow too) and goes back to the builder for one in-session fix round. A head the harness could not run at all (timeout, missing runner or dependencies) ends `needs-human` with the recoverable code `env.repro_unrunnable`, not a builder round: the batch runner's recovery lane reruns it through the repo's own toolchain (see docs/OUTCOME-CODES.md). The harness runs with the lane env (`TMPDIR`, toolchain) and the workspace dependency trees, and a repro test file that is new on head is run against the base sources. A bug that is not testable (specs, docs, workflows, environment) records `"repro_test":"none"` with a one-line `repro_test_reason` and the lane continues as before. `off` gives exactly the previous contract and precheck. Each proof emits a `REPRO_RED_GREEN` run-log event (`base`, `head`) and adds a trajectory decision. |
600
612
  | `precheck.max_minutes` | number | No | Total budget in minutes (1–120) of one round of pre-PR check commands (#4331). The flat form `lean.precheck_max_minutes` also works and wins. When unset, `verify.precheck_total_seconds` applies (default 600 s). Check commands run change-scoped (a learned command with a test-map entry runs only when its globs match the changed files; unmapped commands always run), concurrently up to the host worker cap under one heavy slot, each timed (`ms` in the `PRECHECK_COMMANDS` run-log event). Commands not started when the budget runs out are recorded `deferred-to-ci` (an anomaly, not a failure) and left to CI. |
613
+ | `fix_model` | string | No | Model of fresh fix-round sessions (#4363; `engine.lean_fix_model` is also read). Default: the reviewer model (`engine.lean_reviewer_model`, else Sonnet). Fix rounds for precheck, CI, docs, structure, enumerating-test and sequence-number failures, and conformance or review FAILs whose failures all cite a `file:line`, run in a fresh session (no resume of the builder's transcript) whose prompt carries only the round's failure output, the capped `git diff <base>...HEAD`, the frozen contract and the issue title and summary (under 60K characters). A fresh session that makes no commit gets one resume of the original builder session in the same round. Other rounds (block or contract corrections, CI unavailable, base-sync conflicts, failures without a file:line) resume the builder as before; round limits (`verify.max_rounds`, the one review retry) are unchanged. The run log's `PHASE_ATTEMPT_USAGE` records `mode: "fresh"\|"resume"` on each fix-round attempt. Plan and build sessions keep the builder model. |
614
+ | `builder_routing` | string | No | `on` (default) or `off` (#4394). After the contract is frozen, the lane is classed from its deliverable paths only: `risk` (any path with a built-in risk word such as auth, billing, payments, session, secret or migration, or matching a `review.risk_paths` glob), `docs-config-test` (every path is a test, doc or config file; lockfiles count as code), `small` (at most 2 non-test deliverables) or `large` (more, or none). `docs-config-test` and `small` build on the mid tier (`builder_mid_model`); `risk` and `large` build on the top model (`--model`, else the CLI default). The plan and contract-correction calls always run on the top model. A failed validate round on the mid tier (`precheck`, `ci_red`, `merge_ci`) escalates the lane once: that fix round and every later builder call, fresh fix sessions included, run on the top model. Escalation adds no round. Conformance or review fix rounds do not escalate and keep `fix_model`. `off` keeps every builder call on the top model, as before, and still records the class. The run log gets `BUILDER_ROUTE {mode, class, tier, model, risk}` and `BUILDER_ESCALATED {from, to, trigger}`, and `RUN_TERMINAL` gets `builder_tier`, `builder_class`, `builder_escalated` and `builder_model`. `forgedock stats --tiers` reports merge, conformance and post-merge-rework rates, first-pass yield and cost per tier (`mid`, `mid->top`, `top`) and class, from local run logs. Target-repo risk areas (for example detection code) belong in `review.risk_paths`. |
615
+ | `builder_mid_model` | string | No | Mid-tier builder model for `builder_routing` (#4394; `engine.lean_builder_mid_model` is also read). Default `claude-sonnet-5-5`. An invalid id is ignored. |
616
+ | `light.mode` | string | No | `off`, `shadow` (default) or `on` (#4448). `bin/lean/light.mjs` decides whether an issue qualifies for the light lane: a repro, steps to reproduce, expected or actual behaviour or a code block in the body; 1 to `max_files` predicted non-test files; no `size:M`/`size:L` label; no predicted path with a built-in risk word or matching `review.risk_paths`; not an `investigate(` issue, a post-deploy issue (`post-deploy`/`awaiting-promotion` label, or only post-deploy checklist items) or a decision issue (`decision`, `needs-decision`, `product-decision`, `rfc` or `adr` label, or a `decision(`/`rfc(`/`adr(` title). A missing repro or file prediction is `ambiguous`, left to a cheap classifier. `off` is never eligible; `shadow` reports the verdict without changing the lane; `on` lets an eligible issue run light. The lane wiring is #4450. A light lane posts a `FORGE:LITE` receipt on the issue (outcome light or upgraded, diff stat, red/green repro, gates, true cost from the #4407 per-session deltas, eligibility reason, upgrade reasons), and the batch efficiency report ends with a `lanes:` line: the light/upgraded/full split with cost and the upgrade rate, upgraded ÷ (light + upgraded) (#4452). The flat form `lean.light_mode` also works and wins. |
617
+ | `light.max_lines` | number | No | Upgrade trigger (#4448): a light lane whose non-test diff (added + deleted lines) exceeds this moves to the full lane. Default 40. The other triggers: more than `max_files` non-test changed files, a repro that is not red on the base, two failed test runs, a touched risk path, or a public-API-shape change (#4449). Flat `lean.light_max_lines` wins. |
618
+ | `light.max_files` | number | No | Max predicted non-test files for eligibility and max changed non-test files before an upgrade (#4448). Default 2. Flat `lean.light_max_files` wins. |
619
+ | `sequence.regenerate` | string | No | Command run in the lane worktree (`sh -c`) after the engine renumbers a schema-changing sequence-numbered migration (#4386), to regenerate its snapshot, e.g. `pnpm --filter @x/db db:generate`. The flat form `lean.sequence_regenerate` also works and wins. When a lane-added numbered file (`NNNN_*.sql`, a drizzle `meta/_journal.json` + `meta/NNNN_snapshot.json`, a numbered ADR) shares its number with the base, the engine renumbers it without a builder session: `git mv` to max(base)+1…, the document's own number in its heading, references to the old name repo-wide, the base journal plus the lane's entries, and a data-only migration's snapshot cloned from the base tip with a new `id`/`prevId`. A schema-changing migration without this command, or a migration in another language (`.py`, `.ts`), falls back to the builder renumber, and the trajectory records why. A base-sync whose conflicts are only in such files is resolved the same way, with no re-review. At the merge gate a lane with numbered files holds a host sequence lease per (repo, lane branch, directory) from its last renumber check through the merge, so only one numbered-file PR is in the renumber→CI→merge window at a time. |
620
+
621
+ ## `stats` (OPTIONAL)
622
+
623
+ Thresholds of the fail-soft alerts (#4354). `forgedock stats` prints them as an `ALERTS` section, and the `forgedock orchestrate` batch report ends with the alerts for the batch's repo. Steps that never fail a run (shared dependency cache, pre-PR check budget, toolchain detection) can degrade silently. The alerts compare their rates in the local run logs (`~/.forge/runs`, or `FORGE_RUNS_DIR` for `forgedock stats`) against these thresholds. They read no network source and use no model.
624
+
625
+ ```yaml
626
+ stats:
627
+ alerts:
628
+ window: 50 # last N lanes per repo
629
+ min_lanes: 10 # fewer lanes in the window: nothing is printed
630
+ deps_failed_pct: 20 # DEPS_PREPARED status failed, per ecosystem
631
+ deferred_to_ci_pct: 30 # lanes with a deferred-to-ci pre-PR check
632
+ toolchain_warning_pct: 50 # lanes repeating the same TOOLCHAIN_WARNING
633
+ outcome_share_factor: 2 # outcome code share vs the previous window
634
+ ```
635
+
636
+ | Field | Type | Required | Description |
637
+ |-------|------|----------|-------------|
638
+ | `alerts.window` | number | No | Rolling window: the last N lanes of the repo, ordered by `RUN_START`. Each run in a run log is one lane. Default 50. |
639
+ | `alerts.min_lanes` | number | No | With fewer lanes in the window, or no run logs, nothing is printed. Default 10. |
640
+ | `alerts.deps_failed_pct` | number | No | Alert when `DEPS_PREPARED` status `failed` is above this percentage of the lanes that prepared that ecosystem. Default 20. |
641
+ | `alerts.deferred_to_ci_pct` | number | No | Alert when more than this percentage of lanes left a pre-PR check `deferred-to-ci`. Default 30. |
642
+ | `alerts.toolchain_warning_pct` | number | No | Alert when one `TOOLCHAIN_WARNING` text is repeated on more than this percentage of lanes. Default 50. |
643
+ | `alerts.outcome_share_factor` | number | No | Alert when an outcome code other than `completed` has at least this factor of its share in the previous window. The code must be on at least 2 lanes, and the previous window must have at least `min_lanes` lanes. A code new in this window counts. Default 2. |
644
+
645
+ Each alert line names the event, the rate, the sample size (`count/sample` lanes) and one example reason. When no alert fires, nothing is printed. The flat form `stats.alerts_<key>` also works and wins. `forgedock stats --no-local` skips the alerts.
601
646
 
602
647
  ## `verify` (OPTIONAL)
603
648
 
@@ -613,24 +658,44 @@ verify:
613
658
  precheck_timeout_seconds: 300 # optional per-command timeout of the declared pre-PR checks (default 300)
614
659
  precheck_total_seconds: 600 # optional total budget of one pre-PR check round (default 600)
615
660
  check_run: true # optional; post the forgedock/verify check run (default false)
661
+ checks: "lint,test" # optional; the check names that gate CI verification (default: the required checks)
662
+ post_merge: pnpm -r typecheck # optional; run on the merged tree at base-sync before the push (default: the lane's scoped typecheck)
663
+ duplicate_exports: false # optional; turn off the same-app duplicate-export check (default true)
664
+ e2e: auto # optional e2e merge gate: auto (default) | always | never
665
+ e2e_label: full-ci # PR label that makes the repo's CI run e2e (label path)
666
+ e2e_checks: "e2e" # check names of the e2e jobs the label path waits on
667
+ # e2e_workflow: e2e.yml # or: a workflow_dispatch workflow that runs e2e (wins over the label)
668
+ # e2e_paths_input: specs # its input that receives the changed spec paths
669
+ # e2e_paths: "e2e/**,tests/e2e/**" # globs of e2e specs (default below)
670
+ # e2e_blocking: changed # changed (default) | all
616
671
  ```
617
672
 
618
673
  | Key | Type | Required | Description |
619
674
  |-----|------|----------|-------------|
620
675
  | `mode` | string | No | `local` (default): today's behaviour; the builder runs every validate command on the host and no CI is started before review. `ci`: the builder runs only formatters and linters on the files it changed (no type-checks, test suites or test containers) and records validate ids it left to CI as `{"id","command","ci":true}`; the engine verifies on CI. `hybrid`: the builder runs the targeted validate commands too, and CI also verifies. |
621
- | `workflow` | string | No | Workflow file name or id dispatched on the lane branch (`gh workflow run <workflow> --ref <branch>`). The engine waits for the `workflow_dispatch` run of the lane's head commit. Without it, the engine waits on the head commit's push-triggered check runs. |
676
+ | `workflow` | string | No | Workflow file name or id dispatched on the lane branch (`gh workflow run <workflow> --ref <branch>`). The engine waits for the `workflow_dispatch` run of the lane's head commit. Without it, the engine waits on the head commit's check runs. Under `engine.merge_wait: required` only the base branch's required checks gate (other checks are reported, not gating), and each check name counts at its latest attempt (the newest check run, so a green rerun or a later green run of the same job wins). With no required checks configured, or none of them reported within 5 minutes, every check gates as before. |
622
677
  | `paths_input` | string | No | Name of a `workflow_dispatch` input that receives the changed paths, so the workflow can scope its jobs to the change. |
623
678
  | `skip_paths` | string | No | Comma-separated globs. A glob without `/` matches the basename anywhere; `**` spans directories. When every changed file matches, the lane skips CI verification (a docs-only lane runs no CI matrix). |
624
679
  | `max_rounds` | integer | No | Builder fix rounds for red CI (0–10, default 2). Also bounds the merge-time fix rounds and the declared-check fix rounds below, in every mode. |
625
680
  | `precheck_timeout_seconds` | integer | No | Timeout of each declared check command (1–3600, default 300); a timed-out command counts as failed (exit 124). |
626
681
  | `precheck_total_seconds` | integer | No | Total budget of one round of pre-PR check commands (1–7200, default 600; it was 1800 before 1.30.0). `lean.precheck.max_minutes` takes precedence. Commands not started within the budget are deferred to CI, not failed. |
682
+ | `checks` | string | No | Comma-separated check names that gate CI verification without `workflow` (forge#4387). Overrides the required-check set; other checks are reported, not gating. |
627
683
  | `check_run` | boolean | No | Default `false` (no Checks API call, behaviour unchanged). `true`: after each push of a reviewed head (PR open, base-sync, merge-time fix round) the engine posts one completed `forgedock/verify` check run on that head (forge#4270), `success` iff every deterministic gate the lane already ran passed: declared check commands, undeclared package edges, docs freshness, contract scope, validate ids, requirement status, hotspot growth, amendments. The output carries each gate's id, pass/fail and evidence as JSON; the run log records a `VERIFY` event. Report only: ForgeDock never makes it required and never waits on it (it is ignored by its own CI waits); requiring it is the repository's branch-protection choice. Creating check runs needs a token that may (a GitHub App with `checks: write`); a Checks API failure is logged as an anomaly and never fails the lane. |
684
+ | `post_merge` | string | No | Command run in the lane worktree on the merged tree after each base-sync merge, before the push (forge#4406). Default: the typecheck rows of the lane's change-scoped checks (`bin/lean/checks.mjs`), in `local` and `hybrid` mode only; `ci` mode runs no host typecheck unless `post_merge` is set. A failure resumes the builder with the output, up to `max_rounds` rounds, and the fixed head is re-reviewed; still failing is a trajectory anomaly and CI stays the gate. |
685
+ | `duplicate_exports` | boolean | No | Default `true`: at hand-off and after each base-sync merge, names the lane newly exports that another non-test module of the same app or package (nearest `package.json`/`pyproject.toml`/`setup.py`/`go.mod`/`Cargo.toml`) already declares go back to the builder once ("rename or reuse", both paths); one left over is an anomaly (forge#4406). Re-exports, default exports and names 3+ other modules declare (framework conventions) are ignored. `false` turns the check off. |
686
+ | `e2e` | string | No | e2e merge gate (forge#4403). `auto` (default): run e2e when the diff touches `e2e_paths` or a contract requirement quote has `[type:e2e]`. `always`: on every lane except docs-only ones. `never`: today's behaviour. Without `e2e_label` or `e2e_workflow` no e2e can run, so every mode keeps today's behaviour (a lane that changed specs records a decision saying so). |
687
+ | `e2e_label` | string | No | Label added to the PR (`gh pr edit --add-label`) so tiered CI runs its e2e jobs. The label must already exist in the repo. |
688
+ | `e2e_checks` | string | No | Comma-separated check-run names of the e2e jobs the label path waits on. Default: every check on the head except `forgedock/verify`. A check run concluded `skipped` (the unlabelled tier) counts as not run. |
689
+ | `e2e_workflow` | string | No | Workflow file name or id dispatched on the lane branch instead of the label (`workflow_dispatch`); the engine waits for its run of the reviewed head. Wins over `e2e_label`. |
690
+ | `e2e_paths_input` | string | No | `workflow_dispatch` input of `e2e_workflow` that receives the changed spec paths (newline-separated). |
691
+ | `e2e_paths` | string | No | Comma-separated globs of e2e specs (same glob rules as `skip_paths`). Default `e2e/**,**/e2e/**,*.e2e.*,*.e2e-spec.*,cypress/**,playwright/**`. |
692
+ | `e2e_blocking` | string | No | `changed` (default): only failures of the lane's own specs block the merge; other e2e failures are reported as an anomaly. `all`: every e2e failure blocks. |
628
693
 
629
694
  **Flow** (`ci`/`hybrid`, `bin/lean/verify.mjs` `runCiVerify`):
630
695
 
631
696
  1. After the build and before every conformance review, the engine pushes the lane branch (a plain push, never forced) and runs CI on the head. A base-sync that moves the head is verified again.
632
697
  2. Green (or skipped by `skip_paths`): the review runs. The precheck accepts validate ids left to CI for that head, but a recorded non-zero exit still fails.
633
- 3. Red: the failed jobs' logs (`gh run view --log-failed`, tails capped at about 12 KB) go back to the builder session. The builder fixes and commits, and the engine pushes and verifies again. Still red after `max_rounds`, or a fix round with no new commit, ends as `needs-human` (`ci.red`) and nothing merges.
698
+ 3. Red: the failed jobs' logs (`gh run view --log-failed`, tails capped at about 12 KB) go back to the builder session. The builder fixes and commits, and the engine pushes and verifies again. Still red after `max_rounds` ends as `needs-human` (`ci.red`) and nothing merges. A fix round with no new commit (the builder's signal that the failure is not the change's) reruns the failed jobs once (`gh run rerun <id> --failed`) and verifies again: green continues to review, still red ends `ci.red`. Each rerun emits a `CI_FLAKE_RERUN` run-log event (`trigger`, `head`, `runs`, `jobs`, `result`).
634
699
  4. Slow CI: once the core CI wait budget (360 min, the same budget the merge wait uses) is used up, the lane ends `ci-pending` and can be resumed. Slow CI is never `needs-human`.
635
700
  5. CI unavailable (dispatch refused, or no run or check for the head within 5 minutes): one local fallback. The builder runs every validate command on the host and records real exit codes.
636
701
 
@@ -639,8 +704,16 @@ Each verification emits a `CI_VERIFY` run-log/telemetry event (`mode`, `round`,
639
704
  **Every mode, including the default `local` (forge#4240):**
640
705
 
641
706
  - **Declared check commands before the PR.** Each `learned.test_commands` entry (a block list under `learned:`) runs with `bash -c` in the lane worktree after the build commit, before review and before the PR, with the per-command timeout above. These run on the host and use no Actions minutes. If they all pass, there is no extra builder turn. A failure resumes the builder in its session with the command, its exit code and the output tail, then the commands run again, up to `max_rounds`. A round with no new commit, or exhausted rounds, is recorded as an anomaly and the lane continues, because CI stays the gate. The engine has no built-in command list: to run `scripts/ci-local.sh`, list it here. Each round emits a `PRECHECK_COMMANDS` run-log event (`round`, `results: [{command, rc}]`).
707
+ - **The same checks in the builder's session (#4364).** The builder runs `forgedock check` in the lane worktree before it writes its result: the contract's validate commands plus the same change-scoped pre-PR commands, budget and worker cap, locally (no GitHub, no Actions minutes). The engine hands it the settings above in `.forge-lean/check-plan.json`; outside a lane, `forgedock check --base REF` reads this file's `learned.test_commands`, `lean.precheck.max_minutes` and `verify:`. A builder result of `fixed` without a passing run on its final tree logs a `SELF_CHECK_MISSING` run-log event (`state`: `skipped`, `stale` or `red`), and a failed pre-PR round's `PRECHECK_COMMANDS` event carries `self_check`. Neither fails the lane: the engine's own checks and CI stay authoritative.
642
708
  - **Fix rounds at merge.** When a required check fails on the PR (`engine.merge_wait: required`), the failed jobs' logs (`gh run view --log-failed`, capped) go back to the builder session. The fixed head is re-reviewed and pushed, then the wait starts again. Still red after `max_rounds` (one budget shared with the `ci`/`hybrid` rounds above), or a round with no new commit, ends `needs-human` (`ci.red`). `max_rounds: 0` ends `ci.red` on the first failure. Each round emits a `CI_FIX_ROUND` run-log event (`trigger`, `round`, `jobs`). `merge_wait: all` is unchanged.
643
709
 
710
+ **e2e merge gate (forge#4403, every verify mode).** Tiered CI usually skips e2e on lane PRs, so required checks alone let a lane merge with its own e2e specs failing. When the gate runs (see `e2e` above), after the merge wait and before `gh pr merge`, once per reviewed head, the engine labels the PR with `e2e_label` and waits on the head's `e2e_checks` (or dispatches `e2e_workflow` and waits on its run), with the same CI wait budget.
711
+ - Green: the PR merges.
712
+ - Red: a failure is the lane's when its failed-job log names one of the spec files the lane changed (path or basename), when the log is empty, or when the lane changed no spec (a `[type:e2e]` contract). The lane's failures (all failures with `e2e_blocking: all`) go back to the builder like a failed required check: one fix round from the shared `max_rounds` budget, re-review, push, then the wait and the e2e run start again; `ci.red` when the rounds are spent. Other failures are recorded as an anomaly and the PR merges.
713
+ - Still pending at the budget: `ci-pending` (resumable). Label refused, dispatch refused, or no e2e run within 5 minutes: `needs-human`, nothing merged.
714
+
715
+ Each run emits an `E2E_VERIFY` run-log/telemetry event (`pr`, `head`, `reason`, `result`, `specs`, `failures`, `own`, `blocking`). It spends Actions minutes only after review, on lanes that touch e2e.
716
+
644
717
  **Cost:** `ci`/`hybrid` spend GitHub Actions minutes before review, so they are opt-in per repo. On the green path they add no LLM calls; each red CI round adds one builder resume. The merge gate (`engine.merge_wait`) is unchanged.
645
718
 
646
719
  **Commands that use this section**: core work-on (`forgedock run-issue`, `forgedock orchestrate`; `bin/engine-cli.mjs` `readVerify`)
@@ -1006,6 +1079,8 @@ verification:
1006
1079
  | `services[].container` | string | No | Docker container name for `docker exec` verification |
1007
1080
  | `services[].health_url` | string | No | Local URL for container-level health checks |
1008
1081
  | `ci_local` | object | No | Local CI gate settings (see below) |
1082
+ | `commands.<lang>.<step>` | string | No | Toolchain commands (`format`, `lint`, `typecheck`, `test`, `build`) per language key (`python`, `typescript`, ...); see `forge.yaml.example`. An absent key logs `SKIPPED — not configured` |
1083
+ | `commands.typescript.lint` | string | No | TypeScript lint. Validate runs it like `typescript.typecheck` (bounded by `FORGEDOCK_VERIFICATION_TIMEOUT_SECONDS`, default 120 s; a failure blocks). The core engine reads it from the trusted `forge.yaml` (main checkout or base ref, never the PR's tree), and `bin/lean/checks.mjs` prints it in place of the discovered lint. Without it, a turbo lint is scoped to the changed packages (`--filter=[base]`), not their dependents. Prefer a changed-files, cached lint |
1009
1084
  | `consumer_sweep.mode` | string | No | `advisory` (default): the validate consumer sweep (`scripts/consumer-sweep.sh`, V4.6) records unaddressed referencing sites and never fails. `blocking`: unaddressed sites fail validate with `consumer_unaddressed`, build runs one repair round, and sites still unaddressed are recorded as `CONSUMERS: unresolved` and passed to review as known risk (never `needs-human`). Flip to `blocking` only after `bash scripts/consumer-sweep.sh --eval-history 20` shows a mean of at most 2 flagged sites per merge and none above 10 on your repo |
1010
1085
  | `consumer_sweep.budget_seconds` | integer | No | Wall-clock budget for the sweep (default `60`). When it runs out the sweep stops with `BUDGET_EXCEEDED`, reports the partial result, and never fails, in either mode |
1011
1086
 
@@ -65,6 +65,7 @@ An absent or unknown code is recorded as `unclassified`, with the raw reason kep
65
65
  | `builder.block_untyped` | Core work-on: the builder stayed `blocked` without a valid typed reason (missing `block`, a non-human `why` such as prod data, or a dependency that is not open) after one corrective resume | lean | `result.json` status `blocked`, twice |
66
66
  | `builder.no_evidence` | Core work-on: the builder reported invalid/already-fixed without file:line evidence, so the issue was not closed | lean | `result.json` with empty `evidence` |
67
67
  | `builder.no_commit` | Core work-on: the builder made no commit for a fix | lean | No changed files after the builder session |
68
+ | `budget.exceeded` | Core work-on (forge#4392): the implement turn budget for the lane's size (forge.yaml `engine.turn_budget_s|m|l`, defaults 25/60/100) was spent and the one bounded wrap-up session did not finish (it wrote `status: budget-exceeded` or overran its grant). Work is checkpointed on the lane branch; the detail carries the builder's digest (else the diff stat). Ends `needs-human`; recoverable once by `forgedock orchestrate` (the recovery lane continues from the checkpoint or ends with a typed split-required block) | lean | Stream turn count (`CLI_TURNS` from `runCliBackendAsync` `turnMax`), wrap-up result.json status |
68
69
  | `review.failed` | Core work-on: the fix/regression review failed twice (or could not resume the builder) | lean | Reviewer verdict FAIL |
69
70
  | `review.no_verdict` | Core work-on: the reviewer returned no parseable verdict twice | lean | Unparsable reviewer output |
70
71
  | `contract.invalid` | Core work-on: the builder's contract.json stayed invalid after one correction (missing requirement rows, a quote not verbatim from the issue, a defer without a why, no validate commands) | lean | `validateContract` problems after the correction resume |
@@ -95,7 +96,7 @@ An absent or unknown code is recorded as `unclassified`, with the raw reason kep
95
96
  `forgedock orchestrate` recovers mechanical outcomes inside the batch:
96
97
 
97
98
  - **recoverable**:
98
- - needs-human with `ci.red`, `base.conflict`, `builder.block_untyped`, `env.tool_missing` or `env.repro_unrunnable`;
99
+ - needs-human with `ci.red`, `base.conflict`, `builder.block_untyped`, `env.tool_missing`, `env.repro_unrunnable` or `budget.exceeded`;
99
100
  - `contract.invalid` when the issue was edited mid-run (`issueEdited`);
100
101
  - `ci-pending` on a conflicting PR, recovered as `base.conflict`.
101
102
 
package/docs/STATS.md CHANGED
@@ -34,12 +34,77 @@ printed to stderr while crawling, so `--json` stdout stays parseable.
34
34
 
35
35
  With no local state at all, `forgedock stats --repo <r>` reproduces the report from GitHub alone.
36
36
 
37
+ ## Efficiency (`--efficiency`, forge#4402)
38
+
39
+ ```bash
40
+ npx forgedock stats --efficiency [--batch <id>] [--window 7d,30d] [--repo <owner/repo>] [--json]
41
+ ```
42
+
43
+ Per batch (`--batch`, a dir under `~/.forge/batches`, `FORGEDOCK_BATCH_DIR`) and over trailing windows (`--window`,
44
+ default `7d,30d`: the union of the repo's batches whose `plan.created` falls in the window, where an issue in several
45
+ batches counts once with its latest row). The orchestrate batch report prints the same lines for the batch it just ran.
46
+ The code is in `bin/stats/efficiency.mjs`.
47
+
48
+ - **Cohort and cost to merge.** A merged issue counts every `RUN_START..RUN_TERMINAL` slice of its run log
49
+ (`~/.forge/runs/<issue>.jsonl`) that started at or before its settle time (its latest batch `log.jsonl` row) and
50
+ belongs to the repo. Earlier failed lanes from other batches or ad-hoc runs count too. Attempt cost is
51
+ `cost_usd_delta`, else `cost_usd`.
52
+ - **$/merge, turns/merge, context/turn**: all-phase cost ÷ merged, with implement and review totals; implement turns ÷
53
+ merged; implement cache-read ÷ implement turns. These are also split by size (the merging run's `turns.size`, else a
54
+ `size:` label, else `unknown`) and by model (the implement attempt's `model`).
55
+ - **Retries/merge by cause.** For builder attempts, the trigger decides the cause:
56
+
57
+ | Cause | Triggers |
58
+ | --- | --- |
59
+ | env | `ci_unavailable` |
60
+ | engine | `timeout_restart`, `context_restart`, `budget_wrapup`, `recovery` |
61
+ | code | `precheck`, `structure`, `docs`, `enumerating`, `sequence`, `ci_red`, `conformance`, `merge_ci`, `block_correction`, `contract_correction` |
62
+ | review | `review` |
63
+
64
+ `plan`, `build` and `verifier` are not retries. A lane rerun (every run slice of a merged issue but the last) counts
65
+ by the code that ended it: `env.*` is env; `engine.*`, `run.*`, `followup.*`, `base.*`, `dependency.*` are engine;
66
+ `ci.*`, `conformance.*`, `precheck.*`, `budget.*`, `builder.*`, `contract.*` are code; `review.*` is review; anything
67
+ else is unknown.
68
+ - **No action**: rows parked by the pre-filter (`awaiting-promotion`, `awaiting-precondition`) and lanes that ended
69
+ `builder.blocked` with `Builder is blocked (promotion)`, with their count and cost.
70
+ - **Issue quality vs. cost**: each merged issue's body is tagged `full` (a root-cause heading, a `file:line` and a
71
+ fix/design/approach/proposal heading), `partial` (some of these) or `none`, and n, $/merge and turns/merge are
72
+ compared per tag.
73
+ - **Acceptance (7d after promotion)**: from trusted `FORGE:PROBE` comments (#4399). An issue whose first record is
74
+ under 7 days old is pending. After that it passes when every record within 7 days of the first one passed. The
75
+ first record is the observed promotion, because a probe runs only once the fix is on the default branch.
76
+
77
+ Limits: logs written before #4362 carry no trigger, so implement attempts beyond the plan and build calls count as
78
+ `unknown`. Logs written before #4407 sum running session totals. Lanes run outside a batch are in no window. `gh` (for
79
+ merged issues only) is the one network call; when it fails, quality and acceptance print `n/a` with the reason.
80
+
81
+ ## Explorer A/B (`--explore-ab`, forge#4418)
82
+
83
+ Compares lanes run with forge.yaml `lean.explore: on` against `off` (#4391), from the local run logs only (no GitHub
84
+ crawl). Every RUN_START slice of each log counts, so re-running an issue with the other mode adds a second lane.
85
+
86
+ - Rows per size bucket (`S`, `M`, `L` from the lane's turn record, else `unsized`) and mode: n, mean builder (plan +
87
+ build) turns, mean builder `cache_read_input_tokens`, merge rate (merged / finished) and conformance rate (lanes that
88
+ reached review with no conformance/review fix round and no shadow would-block, over reviewed; same as `--tiers`).
89
+ - A paired table restricted to issues with a finished lane in both modes.
90
+ - Finished lanes without an `explore` tag (before #4391) are counted and left out.
91
+
92
+ `--json` prints the same data as `query.explore_ab`.
93
+
37
94
  ## Sections (per version and repo)
38
95
 
39
96
  - **Cost**: cache-read tokens per issue (mean, median), per phase (mean over all runs, so phases add up to the per-issue
40
97
  mean), per merged issue, output tokens, USD where recorded.
41
98
  - **Time**: p50 and p90 duration per phase and wall time per issue.
42
- - **Flow**: merged %, needs-human %, remediation rate, outcome-code distribution (`docs/OUTCOME-CODES.md`).
99
+ - **Flow**: merged %, needs-human %, remediation rate, outcome-code distribution (`docs/OUTCOME-CODES.md`), and the
100
+ **first-pass rate** (#4362): the share of lanes that reached the build (a `build` attempt, or the `timeout_restart` of
101
+ one) with no rework attempt after it, with the count (`k/n lanes`) and how many lanes are `unknown` (their attempts
102
+ carry no `trigger`: logs written before the field). Rework triggers: `precheck`, `structure`, `docs`, `enumerating`,
103
+ `sequence`, `ci_red`, `conformance`, `review`, `merge_ci` (`bin/lean/fixround.mjs` `REWORK_TRIGGERS`); a timeout
104
+ restart, a base-sync `recovery`, a `ci_unavailable` fallback or a `block_correction` does not break first pass.
105
+ - **Rework by trigger** (#4362): count and USD of the builder (`implement`) attempts with a rework trigger, per trigger,
106
+ plus a total. Attempts without a `trigger` are counted as `unknown`, never dropped. JSON also carries every builder
107
+ attempt by trigger (`rework.attempts`).
43
108
  - **Review funnel**: findings per review by severity, then ISSUE / NOTE / in-PR fix, plus `validated` and `false-positive` issues.
44
109
  - **Gates**: block rate per gate (acceptance, diff size OVER, CI, liveness), malformed-check rate, COHESIVE and OVER counts.
45
110
  - **Issue flow**: issues created by source (review-finding, pipeline bot, human), closed by kind, net flow, findings per merged issue.
package/docs/TELEMETRY.md CHANGED
@@ -40,7 +40,7 @@ is dropped. Payload keys are `[A-Za-z0-9_]+` and cannot shadow the shared keys;
40
40
  | `ci_local` | `scripts/ci-local.sh run` (validate V4.8), one per run | `result` = `PASS` \| `FAIL` \| `DEGRADED`, `jobs_run`, `jobs_deferred`, `jobs_failed`, `duration_ms` (execution), `wait_ms` (time queued for a machine slot), `fix_iterations` (0 first run, 1 and 2 after fixes), `checks`, `cache_hits`, `narrowed` (checks run on a subset or skipped for no impact), `widened` (checks widened to the full command), `selected` / `total` (summed scope) |
41
41
  | `ci_local_check` | `scripts/ci-local.sh run`, one per scoped, widened or executed check (at most 20 per run) | `job`, `selector`, `selected`, `total`, `cache` = `hit` \| `miss` \| `dedup` \| `na`, `duration_ms`, `wait_ms` (job slot wait), `widen` (reason or `none`). A remote-CI failure at merge on a check recorded here as narrowed is a selector bug |
42
42
  | `liveness` | `scripts/remediate-liveness.sh` | `decision` = `PROCEED` \| `DEFER` \| `EXHAUSTED`, `why` |
43
- | `phase_attempt` | the engine (`bin/engine/telemetry-seam.mjs`, one per phase attempt; `phase` = the phase id) | `attempt`, `model`, `backend` (`cli`/`api`), `turns`, `duration_ms`, `cost_usd`, `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_creation_tokens` (any value the runner did not report is omitted) |
43
+ | `phase_attempt` | the engine (`bin/engine/telemetry-seam.mjs`, one per phase attempt; `phase` = the phase id) | `attempt`, `trigger` (#4362: why the attempt ran, one of `plan`, `build`, `block_correction`, `contract_correction`, `precheck`, `structure`, `docs`, `enumerating`, `sequence`, `ci_unavailable`, `ci_red`, `conformance`, `review`, `merge_ci`, `timeout_restart`, `recovery`; `restart_of` = the trigger a `timeout_restart` replaced), `model`, `backend` (`cli`/`api`), `turns`, `duration_ms`, `cost_usd` (the raw reported value: a running total when the attempt resumed a CLI session), `cost_usd_delta` (#4407: this attempt's cost; consumers sum it when present, else `cost_usd`), `cost_usd_session` (the CLI session's running total), `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_creation_tokens` (any value the runner did not report is omitted) |
44
44
  | `run_terminal` | the engine (`terminate()`, one per run segment) | `reason` (terminal reason), `code` (see `docs/OUTCOME-CODES.md`), `wall_ms` (this engine process, start to terminal) |
45
45
 
46
46
  The engine records carry `run` = `r_<issue>_<start epoch seconds>` (one per `forgedock run-issue` process) so `forgedock stats`
@@ -0,0 +1,55 @@
1
+ {
2
+ "version": "lightlane-v1",
3
+ "description": "Three-way light-lane benchmark (#4453, parent #4442): three more-itertools bugs reported after the model cutoff, graded by the upstream fix tests. Instructions: docs/eval/lightlane-bench.md.",
4
+ "upstream": "more-itertools/more-itertools",
5
+ "license": "MIT",
6
+ "clone_url": "https://github.com/more-itertools/more-itertools.git",
7
+ "base": "670b715b307adfe5d2fca08a87ca562cb8bfea2c",
8
+ "suite_command": "python3 -m unittest discover -q -s tests -t .",
9
+ "control_prompt": "Fix this issue in the repository in the current directory. Write a failing regression test first, then the fix, then run the targeted tests. Do not use web tools.\n\nIssue: {title}\n\n{body}",
10
+ "issues": [
11
+ {
12
+ "id": "lightlane-1293",
13
+ "upstream_issue": 1293,
14
+ "fix_pr": 1294,
15
+ "fix_sha": "1dd480a08bc2a4c399c84bf21cb1558f8b5fc838",
16
+ "title": "Repeated zero-cache seekable.peek() retains nested chain iterators",
17
+ "body": "On current master (`b5e3886`), the fix for #1268 preserves values but each `seekable(..., maxlen=0).peek()` prepends a new `chain` around the previous source. Repeated peeking without advancing retains an iterator chain proportional to the number of calls, despite having no history cache.\n\n```python\nimport tracemalloc\nfrom more_itertools import seekable\n\ns = seekable(range(3), maxlen=0)\ntracemalloc.start()\ns.peek()\nbaseline = tracemalloc.get_traced_memory()[0]\nfor _ in range(10000):\n assert s.peek() == 0\nprint(tracemalloc.get_traced_memory()[0] - baseline)\nprint(list(s.elements()), list(s))\n```\n\nOn CPython 3.12.13/macOS this retains 2,427,672 additional bytes; output remains `[] [0, 1, 2]`. A single lookahead slot should be sufficient regardless of the number of repeated `peek()`/`bool()` calls. I am checking a fix that reuses the existing `peekable` helper only in the zero-history case, with unchanged bounded/unbounded history behavior.\n\nInvestigation performed with AI assistance.\n",
18
+ "oracle": {
19
+ "files": [
20
+ "tests/test_more.py"
21
+ ],
22
+ "command": "python3 -m unittest -q tests.test_more.SeekableTest.test_repeated_zero_cache_peek_has_bounded_memory",
23
+ "note": "670b715 is the merge parent of the fix, but the fix branch also carries the #1268/#1269 seekable maxlen=0 value fix; on the base the oracle errors with StopIteration, so an arm passes only by bounding memory and keeping the peeked values."
24
+ }
25
+ },
26
+ {
27
+ "id": "lightlane-1304",
28
+ "upstream_issue": 1304,
29
+ "fix_pr": 1305,
30
+ "fix_sha": "1ea82a711c69f590054987b5cb194157f8ce8ac4",
31
+ "title": "ichunked(..., 0) raises islice's ValueError on a nonempty iterable",
32
+ "body": "`ichunked` is documented as being like `chunked`, but with lazy chunks. `chunked` treats `n=0` as \"yield nothing\". `ichunked` only does that when the source is already empty; with any remaining items it calls `islice(iterator, n - 1)` and blows up.\n\nI hit this switching a streaming pipeline from `chunked` to `ichunked` so batches would not be materialized as lists. Chunk size comes from the same config in both paths; `0` means \"this stage is disabled, emit nothing.\"\n\n```python\nfrom more_itertools import chunked, ichunked\n\nrows = [\"a\", \"b\", \"c\"] # e.g. leftover rows from a DB cursor\nchunk_size = 0 # config: skip this sink\n\nprint(list(chunked(rows, chunk_size)))\n# []\n\nprint([list(c) for c in ichunked(rows, chunk_size)])\n# ValueError: Stop argument for islice() must be None or an integer: 0 <= x <= sys.maxsize.\n```\n\n`IchunkedTests.test_zero` only uses `iterable = []`, which never enters the `for first in iterator` loop, so the `islice(..., n - 1)` path is untested.\n\nNegative `n` has the same leak: `chunked` raises `ValueError: n must be at least 0`, `ichunked` raises islice's \"Indices for islice() must be None or an integer...\".\n\nHappy to send a PR.",
33
+ "oracle": {
34
+ "files": [
35
+ "tests/test_more.py"
36
+ ],
37
+ "command": "python3 -m unittest -q tests.test_more.IchunkedTests.test_zero_nonempty tests.test_more.IchunkedTests.test_negative"
38
+ }
39
+ },
40
+ {
41
+ "id": "lightlane-1308",
42
+ "upstream_issue": 1308,
43
+ "fix_pr": 1309,
44
+ "fix_sha": "67fcf640477c78605580f5ee703a54c73c1e7a8a",
45
+ "title": "bucket ignores a supplied validator when the callable is falsey",
46
+ "body": "`bucket` replaces a supplied validator with its default when the callable has a false truth value. An empty callable set is one example:\n\n```python\nfrom more_itertools import bucket\n\nclass AllowedKeys(set):\n def __call__(self, key):\n return key in self\n\nallowed = AllowedKeys()\nsource = iter([1, 2, 3])\nbuckets = bucket(source, key=lambda x: x, validator=allowed)\nprint(list(buckets[1])) # [1]; expected []\n```\n\nThe validator rejects every key, so selecting a bucket should return an empty iterator without consuming `source`. Instead, `validator or (lambda x: True)` discards it. Membership checks and iteration over bucket keys also accept keys the validator rejects. On an infinite input, a lookup for an absent, rejected key can keep scanning rather than return immediately.\n\nI reproduced this on current main (11.1.0). Changing the default selection to `(lambda x: True) if validator is None else validator` preserves the supplied callable. My regression checks rejected selection, membership, source consumption, and key iteration; the full unittest suite passes with that change.\n\nI can send the small fix and regression test if this behavior is unintended.\n",
47
+ "oracle": {
48
+ "files": [
49
+ "tests/test_more.py"
50
+ ],
51
+ "command": "python3 -m unittest -q tests.test_more.BucketTests.test_falsey_validator_is_honoured"
52
+ }
53
+ }
54
+ ]
55
+ }
@@ -0,0 +1,91 @@
1
+ # Light-lane three-way benchmark (#4453)
2
+
3
+ This benchmark compares the light lane (#4450) with the full lean lane and with a plain single-session `claude -p` control. All three arms run on the same issue set, and each is graded by the **upstream oracle tests**: the test files of the real upstream fix, which fail on the base and pass on the fix. It reproduces the #4442 benchmark of 2026-10-22 and checks its acceptance targets.
4
+
5
+ - Harness: `scripts/bench-lightlane.mjs`
6
+ - Corpus: `docs/eval/corpus/lightlane-v1.json`. This is more-itertools @ `670b715` with upstream #1293, #1304 and #1308. Their fixes are PRs #1294, #1305 and #1309, and they were reported after the model's training cutoff.
7
+
8
+ ## Prerequisites
9
+
10
+ - `python3`, `git`, `gh` (authenticated, with access to the sandbox repos), `claude` and `forgedock` on PATH. Use the ForgeDock release under test.
11
+ - Two **private sandbox repos**, one for the light arm and one for the full arm. Each needs the upstream history up to the base, so the base SHA exists in it:
12
+ ```sh
13
+ git clone https://github.com/more-itertools/more-itertools.git mi && cd mi
14
+ git push git@github.com:<you>/mi-light.git 670b715b307adfe5d2fca08a87ca562cb8bfea2c:refs/heads/main
15
+ git push git@github.com:<you>/mi-full.git 670b715b307adfe5d2fca08a87ca562cb8bfea2c:refs/heads/main
16
+ ```
17
+ If a sandbox is a squashed snapshot instead, set `arms.<arm>.base` in the run config to its base commit.
18
+ - A `forge.yaml` in each sandbox. The light arm sets `lean:` → `light:` → `mode: on`, and the full arm sets `mode: off`. Commit it on `main`; the harness diffs from the base, so this commit doesn't count as an arm change.
19
+ - The corpus issues filed in each sandbox, with the title and body taken verbatim from the corpus. Labels follow the #4442 setup: `bug`.
20
+ ```sh
21
+ jq -c '.issues[]' docs/eval/corpus/lightlane-v1.json | while read -r i; do
22
+ gh issue create -R <you>/mi-light --title "$(jq -r .title <<<"$i")" --body "$(jq -r .body <<<"$i")" --label bug
23
+ done
24
+ ```
25
+
26
+ ## Run config (`runcfg.json`, not committed)
27
+
28
+ ```json
29
+ {
30
+ "arms": {
31
+ "light": { "repo": "<you>/mi-light", "issues": { "lightlane-1293": 1, "lightlane-1304": 2, "lightlane-1308": 3 } },
32
+ "full": { "repo": "<you>/mi-full", "issues": { "lightlane-1293": 1, "lightlane-1304": 2, "lightlane-1308": 3 } },
33
+ "control": { "model": "claude-opus-5-5" }
34
+ }
35
+ }
36
+ ```
37
+
38
+ Optional per arm:
39
+ - `command`: argv prefix. The default is `["forgedock","run-issue"]`, or `["claude"]` for the control.
40
+ - `args`: extra arguments, for example `["--model","claude-opus-5-5"]` for a lane, or the permission flags your `claude` needs for the control.
41
+ - `clone_url` and `base` for the lane arms.
42
+
43
+ Use the same model in every arm, and no web tools.
44
+
45
+ ## Steps
46
+
47
+ 1. **Calibrate** the oracles. Every issue must report `red_on_base` and `green_on_fix`, otherwise the command exits 2:
48
+ ```sh
49
+ node scripts/bench-lightlane.mjs calibrate --corpus docs/eval/corpus/lightlane-v1.json
50
+ ```
51
+ 2. **Run** the arms. This takes about 10 minutes per arm per issue, runs sequentially, and appends one JSON row per arm and issue:
52
+ ```sh
53
+ node scripts/bench-lightlane.mjs run --corpus docs/eval/corpus/lightlane-v1.json --config runcfg.json --out rows.jsonl
54
+ ```
55
+ Use `--arms light,control` or `--issues lightlane-1304` to rerun part of the set. Use `--keep` to keep the scratch clones.
56
+ 3. **Aggregate** the rows. This prints the metrics and targets, and exits 2 when a target fails:
57
+ ```sh
58
+ node scripts/bench-lightlane.mjs aggregate rows.jsonl
59
+ ```
60
+ 4. Post the aggregate JSON and the target table, plus `rows.jsonl`, on the measurement tracking issue. Note the ForgeDock version, the model and the date.
61
+
62
+ ## What each metric means
63
+
64
+ | Field | Meaning |
65
+ |---|---|
66
+ | `cost_usd` | **True** cost. For the lanes, it is the run result's `cost_usd`: the sum of the #4407 per-session deltas, not the naive sum of running totals. For the control, it is `total_cost_usd`. |
67
+ | `wall_ms` | Lane wall time inside the engine (`wall_ms`, #4451). For the control, it is `duration_ms`. `harness_wall_ms` adds CLI start-up. |
68
+ | `model_ms` | Summed session time (`model_ms`, #4451). For the control, it is `duration_ms`, since the control is one session. |
69
+ | `oracle_pass` | The oracle command passes after the fix's test files are overlaid on the arm's head (`git checkout <fix_sha> -- tests/test_more.py`). |
70
+ | `suite_pass` | The repo's own suite (`suite_command`) passes on the arm's head, before the overlay. |
71
+ | `claimed_fix` | The arm reported success and its diff is non-empty. For the lanes, success is a `merged` or `ci-pending` terminal reason; for the control, it is exit 0 with `is_error: false`. |
72
+ | false fix | `claimed_fix` and not `oracle_pass`. |
73
+ | `api_shape` | `bin/lean/apishape.mjs` from the base to the arm's head (signature, sync↔async, generator↔function, removed or renamed public symbol). |
74
+ | undetected API shape | A light-arm row that finished light (`light: true`, no upgrade) although its diff changed the public API shape. |
75
+
76
+ ## Targets (#4442)
77
+
78
+ | Target | Check |
79
+ |---|---|
80
+ | Light true cost ≤ 1.2× control | The cost sum of eligible light rows, divided by the control's cost on the same issues. |
81
+ | Light wall ≤ 1.3× model time | The `wall_ms` sum of eligible light rows, divided by their `model_ms` sum. |
82
+ | Light oracle pass rate ≥ control | Over all light rows. |
83
+ | Light false-fix rate ≤ control | Over all light rows. |
84
+ | 0 undetected public-API-shape changes | Over all light rows. |
85
+
86
+ "Eligible" means the row's `light_eligibility` (the run result's light-lane verdict) is `eligible`. A target is `null` when an arm it needs has no rows.
87
+
88
+ ## Corpus notes
89
+
90
+ - Oracle commands are scoped to the test ids each fix PR added or changed. Class-wide commands would include tests that fail on the base for reasons unrelated to the issue.
91
+ - `lightlane-1293`: `670b715` is the merge parent of the fix, but the fix branch also carries the #1268/#1269 seekable `maxlen=0` value fix. On the base the oracle errors with `StopIteration`, so an arm passes only if it bounds memory and also keeps the peeked values.
@@ -167,7 +167,7 @@ Append this block at the very end of your comment (after the `---` footer line,
167
167
  8. **HTML comments**: The block is invisible in rendered markdown but parseable by the review system
168
168
  9. **Agent marker**: Include exactly one `<!-- FORGE:REVIEW-AGENT:{domain} -->` marker in the persisted body, where `{domain}` is the marker domain the orchestrator passes you (the canonical value from the roster-to-marker table in `review-pr.md`, e.g. `infra` and `scraper`; never a lowercased display name such as `infrastructure` or `scraping`).
169
169
  10. **Reviewed SHA**: Include exactly one line `Reviewed-SHA: [REVIEW_SHA]` (the full 40-hex head you were pinned to) in the persisted body, outside the marker. `/review-pr` scopes its completion wait and idempotent re-entry to this line, so a comment without the current head does not count as a completed review. Do NOT put the SHA inside the `<!-- FORGE:REVIEW-AGENT:{domain} -->` marker.
170
- 11. **Failure scenario and scope (provenance inputs)**: every finding's body section (the prose above the findings block) MUST carry a `**Failure scenario**` line — concrete inputs leading to a wrong output or behaviour — and a `**Scope**: changed` or `**Scope**: pre-existing` line (`changed` = the cited line is added or modified by this PR; `pre-existing` = it was already there). `/review-pr` §6B.4 reads these: a finding without a failure scenario is demoted to a note (unless CRITICAL/HIGH, any confidence), and a MEDIUM or lower finding whose cited line is not in the PR diff is routed as pre-existing (CRITICAL/HIGH never are; see §5). The `**Scope**` tag is informational and never gates routing. Speculative hardening with no failure scenario is a note, not a finding. The one-line `<!-- FINDING:... -->` grammar above is unchanged. <!-- Added: forge#3452 -->
170
+ 11. **Failure scenario and scope (provenance inputs)**: every finding's body section (the prose above the findings block) MUST carry a `**Failure scenario**` line — concrete inputs leading to a wrong output or behaviour — and a `**Scope**: changed` or `**Scope**: pre-existing` line (`changed` = the cited line is added or modified by this PR; `pre-existing` = it was already there). `/review-pr` §6B.4 reads these: a finding without a failure scenario is demoted to a note (unless CRITICAL/HIGH, any confidence, or CONFIRMED MEDIUM; a plain `Failure scenario:`, `Scenario:`, `Repro:` or `Steps to reproduce:` line also counts as the scenario), and a MEDIUM or lower finding whose cited line is not in the PR diff is routed as pre-existing (CRITICAL/HIGH never are; see §5). The `**Scope**` tag is informational and never gates routing. Speculative hardening with no failure scenario is a note, not a finding. The one-line `<!-- FINDING:... -->` grammar above is unchanged. <!-- Added: forge#3452 -->
171
171
 
172
172
  ### Domain Prefixes
173
173
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "forgedock",
3
- "version": "1.30.0",
3
+ "version": "1.32.0",
4
4
  "description": "Autonomous development pipeline for coding agents. GitHub as institutional memory, with native Claude Code and OpenCode entrypoints.",
5
5
  "bin": {
6
6
  "forgedock": "bin/forgedock.mjs"