forgedock 1.17.0 → 1.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (60) hide show
  1. package/bin/engine/outcome-codes.mjs +24 -2
  2. package/bin/engine/phases.mjs +153 -102
  3. package/bin/engine/runlog.mjs +7 -0
  4. package/bin/engine-cli.mjs +5 -0
  5. package/bin/engine.mjs +84 -10
  6. package/bin/forgedock.mjs +1 -1
  7. package/bin/hooks/pre-tool-use.mjs +65 -32
  8. package/bin/runner.mjs +63 -4
  9. package/bin/stats/github-source.mjs +89 -18
  10. package/bin/stats/stats.test.mjs +89 -1
  11. package/commands/cleanup.md +10 -3
  12. package/commands/deploy-pr.md +64 -0
  13. package/commands/fix-ci.md +20 -6
  14. package/commands/milestone.md +34 -17
  15. package/commands/orchestrate/phase-4-execution.md +11 -4
  16. package/commands/review-pr-staging.md +142 -519
  17. package/commands/review-pr.md +490 -3128
  18. package/commands/test-gate.md +25 -10
  19. package/commands/work-on/build/architect.md +13 -1
  20. package/commands/work-on/build/implement.md +2 -1
  21. package/commands/work-on/build/validate.md +2 -0
  22. package/commands/work-on/build.md +107 -86
  23. package/commands/work-on/close.md +50 -15
  24. package/commands/work-on/decompose.md +72 -34
  25. package/commands/work-on/investigate.md +9 -12
  26. package/commands/work-on/remediate.md +27 -13
  27. package/commands/work-on/review.md +5 -3
  28. package/commands/work-on.md +1 -2
  29. package/docs/CONFIG.md +44 -25
  30. package/docs/OUTCOME-CODES.md +7 -2
  31. package/docs/RELEASE-NOTES.md +12 -0
  32. package/docs/STATS.md +2 -1
  33. package/docs/TELEMETRY.md +2 -2
  34. package/docs/WORK-ON-RUNTIME.md +6 -0
  35. package/docs/spec/forge-protocol-v1.md +2 -2
  36. package/package.json +1 -1
  37. package/packages/protocol/src/phases.js +5 -5
  38. package/scripts/cap-findings.sh +1 -1
  39. package/scripts/cap-findings.test.sh +1 -1
  40. package/scripts/check-spec-bash.sh +5 -1
  41. package/scripts/check-spec-bash.test.sh +20 -0
  42. package/scripts/close-parent-c3.test.sh +69 -0
  43. package/scripts/diff-size.sh +93 -4
  44. package/scripts/diff-size.test.sh +59 -2
  45. package/scripts/forge-root.test.sh +13 -11
  46. package/scripts/publish-staging-sync.test.sh +48 -0
  47. package/scripts/resolve-deploy-gate.sh +32 -6
  48. package/scripts/resolve-deploy-gate.test.sh +42 -1
  49. package/scripts/review-finalize.sh +1427 -0
  50. package/scripts/review-finalize.test.sh +833 -0
  51. package/scripts/review-pr-delta-wiring.test.sh +19 -72
  52. package/scripts/review-pr-domain-map.test.sh +18 -13
  53. package/scripts/review-pr-gate2.test.sh +9 -65
  54. package/scripts/review-pr-staging-scope.test.sh +14 -98
  55. package/scripts/review-prepare.sh +48 -12
  56. package/scripts/review-prepare.test.sh +19 -1
  57. package/scripts/review-roster.sh +16 -2
  58. package/scripts/review-roster.test.sh +22 -0
  59. package/scripts/trusted-comments.test.sh +9 -2
  60. package/scripts/wait-ci-green.test.sh +3 -3
package/docs/CONFIG.md CHANGED
@@ -50,7 +50,7 @@ If you have both a local `claude` CLI and `ANTHROPIC_API_KEY` set and want to pi
50
50
  | [`project_board`](#project_board-optional) | No | GitHub Projects v2 integration |
51
51
  | [`orchestration`](#orchestration-optional) | No | `/orchestrate` concurrency and cascade admission policy |
52
52
  | [`pipeline`](#pipeline-optional) | No | `/orchestrate` batch-engine tuning (stall detection, token budget, narration) |
53
- | [`build`](#build-optional) | No | `/work-on` build-phase diff-size gate (threshold, exclusion globs) |
53
+ | [`build`](#build-optional) | No | `/work-on` build-phase scope check (`build.scope.*`) and review depth (`review.depth.*`) |
54
54
  | [`services`](#services-optional) | No | External service URLs and IDs |
55
55
  | [`review`](#review-optional) | No | Context injected into review agents |
56
56
  | [`devdocs`](#devdocs-optional) | No | Devdocs knowledge tree path |
@@ -487,52 +487,69 @@ engine:
487
487
 
488
488
  **Turn exhaustion**: when `--max-turns` is exhausted the run ends `engine-error` (detail `max-turns`) and the phase is never committed, so `forgedock run-issue --retry` re-enters the same phase. Raise `<key>_max_turns` before retrying. Phase runlog events record `model` and `turns` next to `usage` so a limit can be sized from measured turn counts.
489
489
 
490
+ **Per-run scratch dir** (`tmp_root`): every engine child runs with `TMPDIR` and `CLAUDE_CODE_TMPDIR` set to `<tmp_root>/<issue>` so venvs and Claude session data stay off a size-limited `/tmp`. `engine.tmp_root` is an absolute or `~`-prefixed path; default `~/.forge/tmp`. When `tmp_root` is unset and the operator already exported `TMPDIR` or `CLAUDE_CODE_TMPDIR`, that choice is respected and never cleaned. The dir is removed on a terminal outcome and kept on `engine-error` so `--retry` resumes. A disk-full/quota failure (EDQUOT, ENOSPC, errno -28/-122) ends a retryable `engine-error` (outcome code `infra.disk_quota`), never `needs-human`.
491
+
490
492
  **Commands that use this section**: CLI backend (`runCommand` in `bin/runner.mjs`)
491
493
 
492
494
  ---
493
495
 
494
496
  ## `build` (OPTIONAL)
495
497
 
496
- Tuning for the `/work-on` build phase. Today it holds the diff-size gate: after the implement phase stages its changes and before validation commits them, the build measures the changed lines (added plus deleted) of the index and any earlier commits against `origin/{base}`, using `scripts/diff-size.sh`. Review findings per PR rise steeply with diff size, so an oversized build is split before it is validated.
498
+ Tuning for the `/work-on` build phase. Today it holds the **scope check** (forge#3862, replacing the diff-size gate): after the implement phase stages its changes and before validation commits them, the build compares the staged diff with the architect plan (the `PLANNED_FILES:` list and `ESTIMATED_DIFF_LINES` in the plan's `### Scope Baseline`), measures a deterministic `review_weight` with `scripts/diff-size.sh`, and records the outcome in `FORGE:DIFF_SIZE`. **Size never blocks, a finished build is never discarded, and a completed build is never routed to decompose.**
497
499
 
498
500
  ```yaml
499
501
  build:
500
- diff_size:
501
- # Changed-line ceiling. Default: 1000. 0 disables the gate.
502
- threshold: 1000
503
-
504
- # Pre-build estimate (investigate): an estimate above threshold x factor routes
505
- # straight to decompose without building. Default: 1.2. Skipped when threshold is 0.
506
- estimate_factor: 1.2
507
-
508
- # Pre-build estimate (investigate): a non-batch issue with at least this many
509
- # independent seams and an estimate over half the threshold is decomposed into
510
- # packed sub-issues (not one per seam). Review-finding batches are exempt.
511
- # Default: 3.
512
- batch_seam_min: 3
513
-
502
+ scope:
503
+ test_weight: 50 # percent weight of test lines in review_weight (default 50)
504
+ module_depth: 2 # path segments that define a module (default 2)
505
+ overrun_pct: 200 # whitespace-insensitive lines over the plan estimate, in percent, that count as scope creep (default 200)
506
+ trim_out_of_plan: false # move out-of-plan hunks into the batched follow-up checklist (default false: flag only)
507
+ trim_max_files: 3 # trim only when at most this many files are out of plan (default 3)
514
508
  # Extra exclusion globs, ADDITIVE to the built-in defaults below.
515
509
  exclude_globs:
516
510
  - "docs/api/*"
517
511
  - "*.pb.go"
512
+
513
+ review:
514
+ depth:
515
+ extended_weight: 1500 # review_weight at or above which review is EXTENDED (default 1500)
516
+ extended_modules: 6 # modules touched at or above which review is EXTENDED (default 6)
517
+ slice_files: 12 # changed files per slice in extended review (default 12)
518
518
  ```
519
519
 
520
520
  | Field | Type | Required | Description |
521
521
  |-------|------|----------|-------------|
522
- | `diff_size.threshold` | integer | No | Changed lines allowed before the gate trips. Default: 1000. `0` disables the gate |
523
- | `diff_size.estimate_factor` | number | No | Multiplier on `threshold` for the pre-build estimate: the investigator forces a decomposition when `ESTIMATED_DIFF_LINES` > `threshold` x `estimate_factor`. The investigator emits the effective value as an own-line `ESTIMATE_FACTOR` field, and the engine's fail-toward-decompose net (`isDecomposed`) uses it even if the verdict says NO (1.2 when the field is absent or invalid). That net is suppressed (treated as not decomposed) for a sub-issue (`**Parent**: #N`), an issue with a trusted `FORGE:DECOMPOSED` comment, or a justified `FORGE:SIZE_OVERRIDE`, because decompose Guard 1 refuses those. Default: 1.2. Skipped when `threshold` is `0` |
524
- | `diff_size.batch_seam_min` | integer | No | Independent seams at which an issue is decomposed when `ESTIMATED_DIFF_LINES` is over half of `threshold`. Does not apply to review-finding batches (`batch` label or `FORGE:BATCH_MEMBERS`), which decompose only above `threshold` x `estimate_factor`. Seams are packed into sub-issues, not one per seam. Default: 3. Skipped when `threshold` is `0` |
525
- | `diff_size.exclude_globs` | list of strings | No | Globs excluded from the measurement, in addition to the built-in defaults. Default: `[]` |
522
+ | `build.scope.test_weight` | integer | No | Percent of a test file's changed lines counted in `review_weight` (tests are easier to review). Default: 50 |
523
+ | `build.scope.module_depth` | integer | No | Leading path segments of a file's directory that identify a module for the `modules` count. Default: 2 |
524
+ | `build.scope.overrun_pct` | integer | No | The scope is flagged when whitespace-insensitive changed lines exceed the plan's estimate by more than this percent (200 = more than twice the estimate). Default: 200 |
525
+ | `build.scope.trim_out_of_plan` | boolean | No | Opt-in. Moves each out-of-plan file's hunks out of the PR into the batched follow-up issue (forge#3843); the hunks are saved in a patch file next to the git dir, so nothing is lost. Never triggered by an overrun alone. Default: `false` (flag only) |
526
+ | `build.scope.trim_max_files` | integer | No | Trim only when at most this many files are out of plan; more is flagged, not trimmed. Default: 3 |
527
+ | `build.scope.exclude_globs` | list of strings | No | Globs excluded from the measurement, in addition to the built-in defaults. Default: `[]` |
528
+ | `review.depth.extended_weight` | integer | No | `review_weight` at or above which the build records `review_depth: EXTENDED`. Default: 1500 |
529
+ | `review.depth.extended_modules` | integer | No | Modules touched at or above which the build records `review_depth: EXTENDED`. Default: 6 |
530
+ | `review.depth.slice_files` | integer | No | In extended review, each domain agent reviews slices of at most this many changed files. Default: 12 |
531
+
532
+ All thresholds are config values with generous defaults, to be calibrated from telemetry (`diff_size` events carry the plan estimate, the actual, `review_weight`, the scope outcome and the review depth; see `docs/TELEMETRY.md`).
533
+
534
+ **`review_weight`** (computed by `scripts/diff-size.sh`): changed lines with whitespace ignored (`git diff -w`, so a pure re-indent counts about 0), over the counted files only (generated and vendored code excluded), where test files count `test_weight` percent. `modules` counts the distinct directories (first `module_depth` segments) holding at least one non-whitespace change. It drives review depth, never blocking.
526
535
 
527
536
  **Built-in default exclusions** (always applied, defined once in `scripts/diff-size.sh`): lockfiles (`package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `Cargo.lock`, `poetry.lock`, `uv.lock`, `go.sum`, `Gemfile.lock`, `composer.lock`), `*.min.*`, `*.snap`, and the directories `dist/`, `build/`, `generated/`, `fixtures/`, `vendor/` (anchored to the repo root, so first-party directories such as `commands/work-on/build/` are counted) plus `__generated__/` and `__snapshots__/` (any depth). The diff is measured against the merge base of `HEAD` and `origin/<base>`, so commits that landed on the base after the branch point are not counted. A configured glob ending in `/` matches that directory at any depth; any other glob is matched against the file basename and the full path. Globs are treated as data, never evaluated.
528
537
 
529
- **Over the threshold**: the build posts a `FORGE:DIFF_SIZE` comment with a split proposal and exits `BUILD_RESULT: status: NEEDS_DECOMPOSE`; the router then runs `work-on:decompose`. To build the oversized diff anyway, post a `FORGE:SIZE_OVERRIDE` comment whose body has a non-empty justification on the line after the marker (only comments from trusted authors count). An issue that is already a decomposed child, or that already has `FORGE:DECOMPOSED`, is not decomposed again: the build blocks until an override is posted, unless the diff is cohesive (next paragraph). The staged diff is also persisted before the exit (pushed ref `forge/wip/{NUMBER}`, or a patch file when the push fails) and each split-proposal line links its hunks, so they survive worktree cleanup; decompose copies the link into each sub-issue. The measured `diff_lines` is recorded in the `FORGE:TRAJECTORY` comment.
538
+ **Scope-check outcomes** (`result:` in `FORGE:DIFF_SIZE`): `OK` (in plan, standard depth); `EXTENDED_REVIEW` (in plan, `review_weight` or `modules` at the extended threshold: review adds reviewers and per-file slices); `SCOPE_FLAGGED` (files outside the plan, or an estimate overrun: listed in the comment for review, review extended, optionally trimmed); `OVERRIDDEN` (scope flagged but a justified `FORGE:SIZE_OVERRIDE` is present). An architect skip marker (empty plan) means no plan baseline: the out-of-plan check is skipped, never blocking. A `FORGE:SIZE_OVERRIDE` comment (trusted author, non-empty justification on the line after the marker) suppresses the scope flag and the decompose loop guard. The measured `diff_lines` is still recorded in the `FORGE:TRAJECTORY` comment.
539
+
540
+ **Decomposition is a planning decision**: `work-on:investigate` answers `**YES**` only when the work is separable (independent modules or services, no shared files, no ordering dependency, each piece independently testable), with the reason recorded; a cohesive change stays one unit at any size. It emits `ESTIMATED_DIFF_LINES` (the scope baseline) and `INDEPENDENT_SEAMS` inside `### Decomposition Assessment`. The engine downgrades a `**YES**` to NO unless `INDEPENDENT_SEAMS` is at least 2 with an own-line `DECOMPOSE_JUSTIFICATION:`; it has no line-count rule. A sub-issue, an issue with a trusted `FORGE:DECOMPOSED` comment, or one with a justified `FORGE:SIZE_OVERRIDE` is never decomposed (decompose Guard 1).
530
541
 
531
- **Cohesive sub-issue overage**: a sub-issue (or an issue with `FORGE:DECOMPOSED`) that is over the threshold with no override continues instead of blocking when `scripts/diff-size.sh` reports `cohesive=true`: at least 70% of the counted lines are in files added by the diff or in test files (test directories `test/`, `tests/`, `__tests__/`, or `*.test.*`, `*.spec.*`, `test_*`, `*_test.*`), and `diff_lines` is at most `threshold` x 1.5. The build records `result: COHESIVE` with the cohesion figures and a reviewer note in `FORGE:DIFF_SIZE`, adds no `needs-human`, and continues to validation. Test lines are classified, not discounted. The 70% and 1.5x constants live in the script and are not configurable. A non-sub-issue over the threshold still routes to `NEEDS_DECOMPOSE`, and `FORGE:SIZE_OVERRIDE` still takes precedence. Each sub-issue is packed to at most `0.8 x threshold` and carries an own-line `ESTIMATED_DIFF_LINES: N` in its body, which `FORGE:DIFF_SIZE` records next to the actual `diff_lines` as `estimated_diff_lines`.
542
+ **Migration from `build.diff_size.*`** (deprecated; read as aliases for one release, then removed):
532
543
 
533
- **Pre-build estimate**: `work-on:investigate` emits `ESTIMATED_DIFF_LINES`, `INDEPENDENT_SEAMS`, `GATE_THRESHOLD` and `ESTIMATE_FACTOR` inside `### Decomposition Assessment`. An estimate above `threshold` x `estimate_factor` (or, for a non-batch issue, `batch_seam_min`+ seams whose estimate is over `threshold` / 2) answers `**YES**`, so the issue is decomposed before any build into `max(2, ceil(estimate / (0.8 x threshold)))` packed sub-issues (the parent is a terminal tracker). The engine downgrades a `**YES**` that neither condition requires to NO unless the report has an own-line `DECOMPOSE_JUSTIFICATION:` line (it assumes `batch_seam_min` 3 and cannot see the `batch` label, so it is lenient for batches). The estimate rule is skipped when `threshold` is `0`.
544
+ | Old key | New behaviour |
545
+ |---------|---------------|
546
+ | `build.diff_size.threshold` | No longer a gate. A positive value is read as `review.depth.extended_weight` (extended review instead of a block or split); `0` (gate disabled) now just means the defaults. The build records `deprecated_config` in `FORGE:DIFF_SIZE` when an alias supplied a value |
547
+ | `build.diff_size.exclude_globs` | Read as an alias of `build.scope.exclude_globs` (additive) |
548
+ | `build.diff_size.estimate_factor`, `build.diff_size.batch_seam_min` | Ignored. The investigator's decomposition rule is separability, not a line count |
534
549
 
535
- **Commands that use this section**: `work-on` (build phase B5.5, `decompose`, `close`)
550
+ `BUILD_RESULT: status: NEEDS_DECOMPOSE`, the `forge/wip/{NUMBER}` ref, the cohesive-overage path and the outcome code `build.size_over_gate` are retired. An old run whose build was handed to decompose can be resumed at build with `run-issue --retry` once a trusted `FORGE:DECOMPOSED:REVOKED` comment (newer than the `FORGE:DECOMPOSED` comment) and a justified `FORGE:SIZE_OVERRIDE` are on the issue; untrusted comments never reopen a run.
551
+
552
+ **Commands that use this section**: `work-on` (build phase B5.5, `close`), `review-pr` (Phase 3 review depth)
536
553
 
537
554
  ---
538
555
 
@@ -550,6 +567,8 @@ decomposition:
550
567
  |-------|------|----------|-------------|
551
568
  | `close_parent` | boolean | No | When `true`, `work-on:decompose` (Phase D6.5) closes the parent after the sub-issues exist, `FORGE:DECOMPOSED:COMPLETE` is posted and `workflow:decomposed` is set, with the comment "Decomposed into #a #b #c; tracked here until all complete". The label is kept. Every child body is also checked for the `**Parent**: #N` reference. Default: `false`. Any value other than `true`, or an unreadable config, leaves the parent open |
552
569
 
570
+ **Batch members**: when the parent carries a `FORGE:BATCH_MEMBERS` block, `work-on:decompose` (Phase D2.7) reopens and reuses a closed member that matches a planned sub-issue one to one, instead of filing a duplicate. Members with a merged PR or a `false-positive`, `needs-human` or `operator-only` label are never reopened, and any ambiguity falls back to creating a new issue. There is no config key for this.
571
+
553
572
  **With `close_parent: true`**: `work-on:close` Phase C3 keeps checking children off the closed parent's checklist. When the last child closes, the parent gets an "All sub-issues complete" comment and `workflow:merged`; it is not closed a second time and never reopened. `/pipeline-status` and `/pipeline-health` exclude `workflow:decomposed` from work counts and report them separately as `trackers (N)`. `/orchestrate` expands an explicitly requested closed decomposed parent to its open children.
554
573
 
555
574
  **With `close_parent: false`** (default): unchanged. The parent stays open and is closed by C3 when the last child closes.
@@ -637,7 +656,7 @@ review:
637
656
  | `tech_stack` | string | No | One-line tech stack summary. Injected into all review agent prompts |
638
657
  | `context` | string (multiline) | No | Freeform context about architecture, deploy model, and known pitfalls |
639
658
  | `domains.*` | string | No | Domain-specific notes keyed by review agent name (`billing`, `auth`, `database`, `frontend`, `api`, `infra`, `security`) |
640
- | `max_filed_findings` | integer >= 0 | No | Per-review cap on ISSUE dispositions (`scripts/cap-findings.sh`, applied by `review-pr` §6B.5 and `review-pr-staging` §7B.5 after classification). Default `2`; `1` when the reviewed PR is itself a fix for a `review-finding` issue, where every filed HIGH/CRITICAL consumes that budget. ISSUE findings are ranked by severity, then confidence, then safety-domain agent (security, auth, billing, concurrency, database), then input order; an unparseable severity or confidence ranks as MEDIUM/LIKELY. The overflow becomes `NOTE capped`: listed in the PR body and counted as `notes_capped` in `FORGE:NOTE_DISPOSITION`, never filed and never dropped. **HIGH/CRITICAL findings are never capped** and still block; `ISSUE fix-efficacy-*` findings (the linked issue is not fixed, or the PR re-introduces its defect) are never capped either and count against the review-finding budget like HIGH. `0` is valid (cap every MEDIUM/LOW finding); a non-numeric or negative value falls back to the default with a warning. An explicit value overrides both defaults. |
659
+ | `max_filed_findings` | integer >= 0 | No | Per-review cap on ISSUE dispositions (`scripts/cap-findings.sh`, applied by `review-pr` §6B.5 and `review-pr-staging` §7B (via `review-finalize.sh classify`) after classification). Default `2`; `1` when the reviewed PR is itself a fix for a `review-finding` issue, where every filed HIGH/CRITICAL consumes that budget. ISSUE findings are ranked by severity, then confidence, then safety-domain agent (security, auth, billing, concurrency, database), then input order; an unparseable severity or confidence ranks as MEDIUM/LIKELY. The overflow becomes `NOTE capped`: listed in the PR body and counted as `notes_capped` in `FORGE:NOTE_DISPOSITION`, never filed and never dropped. **HIGH/CRITICAL findings are never capped** and still block; `ISSUE fix-efficacy-*` findings (the linked issue is not fixed, or the PR re-introduces its defect) are never capped either and count against the review-finding budget like HIGH. `0` is valid (cap every MEDIUM/LOW finding); a non-numeric or negative value falls back to the default with a warning. An explicit value overrides both defaults. |
641
660
  | `key_paths` | map(string→list of strings) | No | Domain-to-file mapping used by `/work-on` investigation and `/review-pr` agents to quickly locate relevant files. Keys are domain names matching issue labels; values are lists of file path patterns (glob-style, relative to repo root). When present, agents use this table instead of inferring files from labels and issue body. |
642
661
 
643
662
  **Commands that use this section**: `review-pr`, `review-pr-agents` (all domain agents), `work-on` (Phase 1B investigation)
@@ -18,10 +18,12 @@ An absent or unknown code is recorded as `unclassified`, with the raw reason kep
18
18
  | `review.changes_requested` | Review asked for an in-PR fix and handed off to remediate | review | `REVIEW_RESULT` `status: NEXT`, `next: remediate` |
19
19
  | `review.escalated` | Review escalated the PR to a human or remediation | review | PR/issue carries `needs-human` or `workflow:remediating` |
20
20
  | `review.rereview_skipped` | Re-review was skipped | reserved | No structured evidence source yet |
21
+ | `review.blocking_issue` | Review left the PR open at an unchanged head with a filed blocking ISSUE and handed off to remediate | review | Trusted `FORGE:NOTE_DISPOSITION` on the PR with `findings_filed` > 0 and `reviewed_sha` equal to the live head |
22
+ | `review.pr_open` | Review ended with the PR open and unmerged and no handoff signal | review | Live PR open, no `REVIEW_RESULT` remediate handoff, no `needs-human`/`workflow:remediating` label, no blocking disposition |
21
23
  | `build.acceptance_failed` | Newest acceptance gate marker is blocked | build | `FORGE:ACCEPTANCE_GATE:FAILED`/`:BLOCKED` or `FORGE:BUILD_BLOCKED` |
22
24
  | `build.acceptance_malformed` | Acceptance spec was malformed | reserved | Indistinguishable from `build.acceptance_failed` by sentinel alone |
23
- | `build.size_over_gate` | Size gate routed the build to decompose | build | `FORGE:DIFF_SIZE` `result: OVER` with `### Split Proposal` |
24
- | `build.cohesive` | Size gate judged an oversized diff cohesive | reserved | `result: COHESIVE` is a pass, so no failure to code |
25
+ | `build.size_over_gate` | Retired (forge#3862): the old size gate routed a finished build to decompose; never emitted now, kept so historic runs still parse | reserved | A finished build is never routed to decompose |
26
+ | `build.cohesive` | Retired (forge#3862): the old size gate judged an oversized diff cohesive | reserved | No size gate remains |
25
27
  | `remediate.no_marker` | Remediation ended without its completion marker | remediate | No trusted `FORGE:REMEDIATION:COMPLETE` |
26
28
  | `remediate.deferred_live_run` | Remediation deferred to a live earlier run | remediate, engine | Trusted `FORGE:REMEDIATION_DEFERRED` record |
27
29
  | `remediate.re_escalated` | Remediation re-escalated to a human | remediate | `**Re-gate outcome**: RE-ESCALATED` |
@@ -31,7 +33,10 @@ An absent or unknown code is recorded as `unclassified`, with the raw reason kep
31
33
  | `base.conflict` | Base-branch conflict blocked the PR | reserved | `REVIEW_RESULT` carries no conflict field yet |
32
34
  | `rate_limited` | Session or usage limit outlasted the pause budget | engine | `err.usageLimit` / `err.resetAtEpochMs` |
33
35
  | `engine.error` | Engine or tool failure (missing key/SDK, CLI crash, max turns, unresolved args, runner threw every attempt) | engine | `err.code` |
36
+ | `infra.disk_quota` | Disk full or quota exhausted (EDQUOT/ENOSPC, errno -28/-122); retryable infra failure, never `needs-human` | engine | Runner error text |
34
37
  | `lease.held` | Another live agent holds the issue lease | engine | Foreign live lease on acquire |
38
+ | `dependency.open` | A trusted `**Depends on**: #N` dependency is still open, so the run was refused before any state write | engine | Trusted issue body `**Depends on**` line + the dependency issue's open state |
39
+ | `dependency.unreadable` | The issue body or a declared dependency could not be read, so the run was refused (fail closed) | engine | Failed or unparsable `gh api` read |
35
40
  | `invalid` | Issue is invalid or in a divergent closed state | investigate, engine | `INVESTIGATION:INVALID` marker, or live snapshot |
36
41
  | `decomposed` | Issue was decomposed into sub-issues | investigate, decompose | `FORGE:DECOMPOSED` or the decomposition assessment |
37
42
  | `completed` | Non-failure terminal (merged, phase-complete, investigation) | engine | Terminal reason |
@@ -2,6 +2,18 @@
2
2
 
3
3
  Operator-facing notes for behavior changes that need action or awareness. Newest first.
4
4
 
5
+ ## Review specs call the prepare and finalize scripts (#3841)
6
+
7
+ `commands/review-pr.md` is now under 100 KB: Phases 1-3B and 6 run through `scripts/review-prepare.sh` and `scripts/review-finalize.sh`, and `commands/review-pr-staging.md` reuses both (`review-prepare.sh --no-delta` in 0C, then `review-finalize.sh extract|classify|file-prepare|file-verify` with `--scope-files` and `--prior-prs` in Phase 7) instead of carrying its own template resolution, slicing, classifier and cap copies. A staging bundle whose API diff is too large is read from `origin/<base>...origin/<head>`. CI now runs `--fence-funcs` over `commands/review-pr*.md` and `--fence-state` over the staging spec. Engine: `close.detectOutcome` reports `merged` only for a CLOSED issue that also carries `workflow:merged` (a stale label on an open issue no longer counts).
8
+
9
+ Action: none. Staging reviews now stop with `REVIEW_RESULT: status: BLOCKED` when `review-prepare.sh` exits 3 (for example no template source, or an empty trusted `typescript.build` on the full-build lane) instead of continuing.
10
+
11
+ ## Post-build scope check replaces the diff-size gate (#3862)
12
+
13
+ The line-count size gate is gone. A finished build is never discarded and never routed to decompose; `BUILD_RESULT: status: NEEDS_DECOMPOSE`, the `forge/wip/{N}` ref, the `COHESIVE` path and the engine's size-based decompose routing (including the investigate estimate net) are removed. Decomposition is a planning decision: `work-on:investigate` says `**YES**` only for separable work (independent modules or services, no shared files, no ordering), and the engine downgrades a YES without `INDEPENDENT_SEAMS >= 2` and a `DECOMPOSE_JUSTIFICATION`. Build B5.5 is now a scope check: it compares the diff with the architect plan (`### Scope Baseline`: `PLANNED_FILES:` and `ESTIMATED_DIFF_LINES:`), flags files outside the plan (opt-in trim into the batched follow-up), and measures `review_weight` (whitespace-insensitive, generated/vendored excluded, tests weighted, modules counted). A high `review_weight` extends review (more reviewers, per-file slices) instead of blocking; the Security agent and the Phase 2 floor stay unconditional. `FORGE:SIZE_OVERRIDE`, the loop guards and the batch exemption keep working. `run-issue --retry` reopens a stale `decomposed` run at build when a trusted `FORGE:DECOMPOSED:REVOKED` comment and a justified `FORGE:SIZE_OVERRIDE` are present. Telemetry `diff_size` events now carry estimate, actual, `review_weight`, scope outcome and review depth.
14
+
15
+ Action: replace `build.diff_size.*` with `build.scope.*` / `review.depth.*` (see docs/CONFIG.md, "Migration"). The old keys are read as deprecated aliases for one release; `threshold` no longer blocks anything. Stacked-PR splitting and default calibration are tracked in #3907.
16
+
5
17
  ## Gate scripts emit structured telemetry (#3834)
6
18
 
7
19
  `diff-size.sh`, `review-acceptance-gate.sh`, `classify-finding.sh`, `wait-ci-green.sh`, `remediate-liveness.sh` and the build/review spec call sites (B5.5, B6.5, review panel) now append one JSONL record per decision to a trusted `<!-- FORGE:TELEMETRY v1 -->` comment on the issue or PR (shell emitter `scripts/lib/telemetry.sh`, Node twin `bin/engine/telemetry.mjs`). A failed write is buffered under `~/.forge/telemetry-buffer/` and flushed on the next emit. Catalogue and privacy note: `docs/TELEMETRY.md`.
package/docs/STATS.md CHANGED
@@ -47,7 +47,8 @@ Each group shows a delta against the previous version of the same repo and is fl
47
47
  They match `scripts/cost-report.sh`, so the two reports agree on the same logs: only finished runs count; hydrated runs
48
48
  and runs with no usage events are excluded; tokens are cache-read tokens; `PHASE_ATTEMPT_USAGE` supersedes the cumulative
49
49
  `PHASE_COMMIT` usage of the same phase. On GitHub a run is the set of `phase_attempt` records sharing a `run` id and it is
50
- finished once a `run_terminal` record exists. Wall time covers one engine process; a resumed run is a separate run.
50
+ finished once a `run_terminal` record exists. On GitHub stats count one run per issue: the latest by start timestamp wins and earlier attempts (for example before a
51
+ `--retry`) are discarded, as in `scripts/cost-report.sh`. Wall time covers the winning engine process only.
51
52
 
52
53
  ## Versions and n/a
53
54
 
package/docs/TELEMETRY.md CHANGED
@@ -31,7 +31,7 @@ is dropped. Payload keys are `[A-Za-z0-9_]+` and cannot shadow the shared keys;
31
31
 
32
32
  | Event | Emitted by | Payload |
33
33
  |-------|-----------|---------|
34
- | `diff_size` | `scripts/diff-size.sh`; `work-on/build.md` B5.5 (final result) | `lines`, `excluded`, `threshold`, `result` = `OK` \| `OVER` \| `COHESIVE` (`OVERRIDDEN` from the B5.5 call site) |
34
+ | `diff_size` | `scripts/diff-size.sh`; `work-on/build.md` B5.5 (final result) | `lines`, `excluded`, `ws_lines`, `review_weight`, `modules`, `result`; the script emits the legacy `threshold` and `OK` \| `OVER` \| `COHESIVE`, the B5.5 call site emits the final `result` = `OK` \| `EXTENDED_REVIEW` \| `SCOPE_FLAGGED` \| `OVERRIDDEN` plus `estimate` (plan), `actual` (whitespace-insensitive lines), `scope` = `ok` \| `out_of_plan` \| `no_baseline` and `review_depth` = `STANDARD` \| `EXTENDED` (calibration inputs, forge#3862) |
35
35
  | `acceptance_gate` | `work-on/build.md` B6.5 (`phase=build`); `scripts/review-acceptance-gate.sh` (`phase=review`) | `phase`, `verified`, `failed`, `malformed`, `deferred` |
36
36
  | `finding_disposition` | `scripts/classify-finding.sh`, one per finding | `severity`, `confidence`, `agent`, `disposition` = `ISSUE` \| `NOTE` \| `INPR_FIX`, `reason` |
37
37
  | `review_panel` | `review-pr.md` Phase 4 | `agents_selected`, `agents_completed`, `mode` = `FULL` \| `DELTA` \| `BASE_SYNC_ONLY` |
@@ -42,7 +42,7 @@ is dropped. Payload keys are `[A-Za-z0-9_]+` and cannot shadow the shared keys;
42
42
 
43
43
  The engine records carry `run` = `r_<issue>_<start epoch seconds>` (one per `forgedock run-issue` process) so `forgedock stats`
44
44
  (`docs/STATS.md`) can group the attempts of a run and tell a finished run from an unfinished one. They are the GitHub-side
45
- source for Cost, Time and outcome codes: without them a fresh machine can rebuild only the gate and review sections.
45
+ source for Cost, Time and outcome codes (`run` is per process, but stats count one run per issue, last wins): without them a fresh machine can rebuild only the gate and review sections.
46
46
 
47
47
  The classifier and the scripts emit only when a destination is known: callers export `FORGE_ISSUE` and/or `FORGE_PR`
48
48
  (`review-pr.md` exports them before classifying; the other scripts default `FORGE_REPO`/`FORGE_ISSUE`/`FORGE_PR` from
@@ -51,6 +51,12 @@ Only the router may invoke them, so they always run at router layer + 1 (L2 unde
51
51
  L1 solo). When another phase needs one, it returns a handoff (`status: NEXT`, `next: <phase>`) and
52
52
  the router invokes it. Concretely: review returns `next: remediate` instead of calling remediation.
53
53
 
54
+ The same rule gives the `PHASE_TRAIL_FAILED` re-dispatch a single owner: `work-on:review` Phase R4 alone
55
+ re-runs the missing phases. `/review-pr` (Phase 8, via `scripts/review-finalize.sh merge-gate`) and
56
+ `work-on:remediate` (Phase M6) only report the status with its `MISSING:` lines and never re-dispatch. When
57
+ `/review-pr` finds neither `Task` nor `Agent` ("neither tool available"), the usual cause is this depth limit
58
+ (F1/F5): the review was invoked from inside a fork instead of by the router, so `Agent` was withheld.
59
+
54
60
  **R2 — Non-dispatching forks may nest.** Phases and children that never spawn sub-agents may invoke
55
61
  further forks at any depth (F5): build → context/architect/implement/validate → quality-gate,
56
62
  review → quality-gate / missing-phase re-dispatch, remediate → close. Forking them keeps each
@@ -535,8 +535,8 @@ needed.
535
535
 
536
536
  | Marker | Category | Meaning |
537
537
  |--------|----------|---------|
538
- | `<!-- FORGE:DIFF_SIZE -->` | Lifecycle (no completion sentinel) | Measured diff size of the staged build: `diff_lines`, `excluded_lines`, `threshold`, `result: OK \| OVERRIDDEN \| COHESIVE \| OVER`, optional `estimated_diff_lines` (sub-issues), cohesion figures and a reviewer note on `COHESIVE`, and on `OVER` a `### Split Proposal`. Refreshed in place; readers take the latest trusted comment. Consumed by decompose (plan source) and close (records `diff_lines` in `FORGE:TRAJECTORY`) |
539
- | `<!-- FORGE:SIZE_OVERRIDE -->` | Control | Posted by a human to let an over-threshold build proceed. Valid only from a trusted author and with a non-empty justification on the line after the marker |
538
+ | `<!-- FORGE:DIFF_SIZE -->` | Lifecycle (no completion sentinel) | Record of the build's post-build scope check (B5.5): `diff_lines`, `excluded_lines`, `ws_lines`, `review_weight`, `modules`, `estimated_diff_lines`, `scope: ok \| out_of_plan \| no_baseline`, `review_depth: STANDARD \| EXTENDED`, `result: OK \| EXTENDED_REVIEW \| SCOPE_FLAGGED \| OVERRIDDEN` plus `out_of_plan:` / `overrun:` / `trimmed:` notes. Never a size gate and never a decompose signal (a finished build is not routed to decompose). Refreshed in place; readers take the latest trusted comment. Consumed by review-pr (review depth) and close (records `diff_lines` in `FORGE:TRAJECTORY`). Older records carrying `threshold` / `result: OVER \| COHESIVE` still parse |
539
+ | `<!-- FORGE:SIZE_OVERRIDE -->` | Control | Posted by a human to suppress the scope-check flag (out-of-plan files or estimate overrun) and the decompose loop guard. Valid only from a trusted author and with a non-empty justification on the line after the marker |
540
540
 
541
541
  Both lookups are anchored at the start of the comment body and trust-filtered
542
542
  (`scripts/trusted-comments.sh`); an unreadable lookup fails closed.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "forgedock",
3
- "version": "1.17.0",
3
+ "version": "1.18.0",
4
4
  "description": "Autonomous development pipeline for coding agents. GitHub as institutional memory, with native Claude Code and OpenCode entrypoints.",
5
5
  "bin": {
6
6
  "forgedock": "bin/forgedock.mjs"
@@ -57,8 +57,8 @@ export const PHASE_IDS = ['investigate', 'decompose', 'build', 'review', 'remedi
57
57
  * investigate.md emits it as the HTML comment `<!-- DECOMPOSE:YES -->`.
58
58
  * @property {string} [notDecomposedMarker] - (investigate only) the explicit not-decomposed
59
59
  * verdict, emitted as `<!-- DECOMPOSE:NO -->`.
60
- * @property {string} [sizeGateMarker] - (build only) bare annotation-opener of the diff-size gate record.
61
- * @property {string} [sizeOverrideMarker] - (build only) bare annotation-opener of the size-gate override.
60
+ * @property {string} [sizeGateMarker] - (build only) bare annotation-opener of the scope-check record (diff size, review weight, scope outcome).
61
+ * @property {string} [sizeOverrideMarker] - (build only) bare annotation-opener of the scope-check override.
62
62
  * @property {string} [investigationMarker] - (build only) bare marker of the durable
63
63
  * investigation-deliverables comment, posted as `<!-- FORGE:INVESTIGATION:DELIVERABLES -->`.
64
64
  * @property {string} [partialMarker] - sentinel for an interrupted/partial annotation.
@@ -108,9 +108,9 @@ export const PHASE_MARKERS = {
108
108
  build: {
109
109
  completionMarker: RESERVED_TYPES.BUILDER.completionSentinel, // 'FORGE:BUILDER:COMPLETE'
110
110
  header: 'FORGE:BUILDER',
111
- // Size-gate (B5.5) records. Neither is a completion sentinel: the gate's
112
- // NEEDS_DECOMPOSE exit posts no FORGE:BUILDER:COMPLETE, so the engine reads
113
- // these to route an oversized build to decompose instead of retrying it.
111
+ // Scope-check (B5.5) records. Neither is a completion sentinel and neither routes anywhere
112
+ // (forge#3862): a finished build is never sent to decompose. DIFF_SIZE is the record of the
113
+ // scope-check outcome; SIZE_OVERRIDE suppresses the scope flag and the decompose loop guard.
114
114
  sizeGateMarker: `FORGE:${RESERVED_TYPES.DIFF_SIZE.type}`, // 'FORGE:DIFF_SIZE'
115
115
  sizeOverrideMarker: `FORGE:${RESERVED_TYPES.SIZE_OVERRIDE.type}`, // 'FORGE:SIZE_OVERRIDE'
116
116
  // Investigation exit (implement.md I2): no code, no PR, so no FORGE:BUILDER:COMPLETE.
@@ -2,7 +2,7 @@
2
2
  # SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios
3
3
  # SPDX-License-Identifier: AGPL-3.0-or-later
4
4
  #
5
- # cap-findings.sh — PR-level cap on filed review findings (review-pr §6B.5, review-pr-staging §7B.5).
5
+ # cap-findings.sh — PR-level cap on filed review findings (review-pr §6B.5, review-pr-staging §7B).
6
6
  # classify-finding.sh decides one finding at a time; this is the aggregation
7
7
  # step after it: at most N ISSUE dispositions per review run are kept, the rest
8
8
  # become `NOTE capped` (listed in FORGE:NOTE_DISPOSITION, never dropped).
@@ -2,7 +2,7 @@
2
2
  # SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios
3
3
  # SPDX-License-Identifier: AGPL-3.0-or-later
4
4
  #
5
- # Fixtures for scripts/cap-findings.sh (review-pr §6B.5 / review-pr-staging §7B.5 per-review cap).
5
+ # Fixtures for scripts/cap-findings.sh (review-pr §6B.5 / review-pr-staging §7B per-review cap).
6
6
  # Run: bash scripts/cap-findings.test.sh (bash 3.2 compatible)
7
7
 
8
8
  set -u
@@ -33,6 +33,8 @@
33
33
  # `export VAR=`, `read VAR`, `for VAR in`) at or before its first read. Comment-only lines are ignored; `$REPO`
34
34
  # does not match `$REPO_FLAG`. Suppress a deliberate read by putting
35
35
  # `allowlist:fence-state` on the same line. Pseudo-tool-call fences are scanned too.
36
+ # A line that sources scripts/lib/fence-bootstrap.sh and ends in `# FENCE_BOOTSTRAP` binds the variables that
37
+ # library derives (PR_NUMBER ALL_PR_NUMBERS GH_REPO DEFAULT_BRANCH STAGING_BRANCH CONFIG_FILE _SD FORGE_RUN_KEY).
36
38
  # Output: `FAIL <file>:<line> fence-state ...` then `SPEC-FENCE-STATE: ...`.
37
39
  # --fence-funcs Different check, opt-in: shell functions are per-shell too, so a function defined
38
40
  # (`name() {`, `function name`) in one ```bash/sh fence is undefined in every other
@@ -113,15 +115,17 @@ if [ "$FENCESTATE" = 1 ]; then
113
115
  }
114
116
  }
115
117
  function reset( i) { for (i = 1; i <= nv; i++) { asg[vn[i]] = 0; first[vn[i]] = ""; firsttxt[vn[i]] = "" } }
116
- BEGIN { nv = split(vars, vn, " ") }
118
+ BEGIN { nv = split(vars, vn, " "); nb = split("PR_NUMBER ALL_PR_NUMBERS GH_REPO DEFAULT_BRANCH STAGING_BRANCH CONFIG_FILE _SD FORGE_RUN_KEY", bv, " ") }
117
119
  !inb && /^[ \t]*```(bash|sh)[ \t]*$/ { inb = 1; reset(); next }
118
120
  !inb && /^[ \t]*```/ { other = !other; next }
119
121
  other { next }
120
122
  inb && /^[ \t]*```[ \t]*$/ { flush(); inb = 0; next }
121
123
  inb {
122
124
  if ($0 ~ /^[ \t]*#/) next
125
+ boot = ($0 ~ /fence-bootstrap\.sh/ && $0 ~ /# FENCE_BOOTSTRAP[ \t]*$/)
123
126
  for (i = 1; i <= nv; i++) {
124
127
  v = vn[i]
128
+ if (boot && first[v] == "") for (j = 1; j <= nb; j++) if (bv[j] == v) asg[v] = 1
125
129
  if (first[v] == "" && ($0 ~ ("(^|[;&|( \t])(export[ \t]+|local[ \t]+|declare[ \t]+(-[A-Za-z]+[ \t]+)?|readonly[ \t]+)?" v "=") ||
126
130
  $0 ~ ("(^|[;&|( \t])read[ \t]+(-[A-Za-z]+[ \t]+)*([A-Za-z0-9_]+[ \t]+)*" v "([ \t;]|$)") ||
127
131
  $0 ~ ("(^|[;&|( \t])for[ \t]+" v "[ \t]+in[ \t]"))) asg[v] = 1
@@ -165,6 +165,26 @@ check 0 "violations=0" "--fence-state passes declared, loop, e
165
165
  check 2 "Usage" "--fence-state without -- is a usage error" -- --fence-state PR_NUMBER fs-good.md
166
166
  check 2 "Usage" "--fence-state without vars is a usage error" -- --fence-state -- fs-good.md
167
167
 
168
+ # --fence-state: a FENCE_BOOTSTRAP source line binds the variables scripts/lib/fence-bootstrap.sh derives (and only those, and only on a marked line)
169
+ cat > "$T/fs-boot.md" <<EOF
170
+ ${FENCE}bash
171
+ . "\$(forge_resolve lib/fence-bootstrap.sh)" || exit 1 # FENCE_BOOTSTRAP
172
+ gh pr view "\$PR_NUMBER" -R "\$GH_REPO" --json state
173
+ ${FENCE}
174
+ EOF
175
+ cat > "$T/fs-boot-bad.md" <<EOF
176
+ ${FENCE}bash
177
+ . "\$(forge_resolve lib/fence-bootstrap.sh)" || exit 1
178
+ gh pr view "\$PR_NUMBER" -R "\$GH_REPO" --json state
179
+ ${FENCE}
180
+ ${FENCE}bash
181
+ . "\$(forge_resolve lib/fence-bootstrap.sh)" || exit 1 # FENCE_BOOTSTRAP
182
+ echo "\$MERGE_ISSUE"
183
+ ${FENCE}
184
+ EOF
185
+ check 0 "violations=0" "--fence-state treats a FENCE_BOOTSTRAP source line as binding PR_NUMBER and GH_REPO" -- --fence-state PR_NUMBER GH_REPO -- fs-boot.md
186
+ check 1 "violations=3" "--fence-state ignores an unmarked source line and a var the bootstrap does not derive" -- --fence-state PR_NUMBER GH_REPO MERGE_ISSUE -- fs-boot-bad.md
187
+
168
188
  # --fence-funcs: a function defined only in another fence, then called, is flagged.
169
189
  cat > "$T/ff-bad.md" <<EOF
170
190
  ${FENCE}bash
@@ -51,4 +51,73 @@ printf '%s\n' "$OUT" | grep -q 'gh issue close 5 .*All sub-issues complete' || f
51
51
  printf '%s\n' "$OUT" | grep -q 'set_workflow_state 5 merged' || fail "open parent: merged not set"
52
52
  printf '%s\n' "$OUT" | grep -q 'gh issue comment' && fail "open parent: no separate comment expected"
53
53
 
54
+ # ---- Verified-retry checkbox write + state-derived OPEN_SUBS (concurrent sibling close race) ----
55
+ WRITE_BLOCK=$(awk '/^PARENT_UPDATED=false$/{p=1} p{print} p&&/^\[ "\$PARENT_UPDATED" = true \]/{exit}' "$CLOSE")
56
+ DERIVE_BLOCK=$(awk '/^PARENT_FRESH=/{p=1} p{print} p&&/^OPEN_SUBS=\$\{OPEN_SUBS:-0\}$/{exit}' "$CLOSE")
57
+ [ -n "$WRITE_BLOCK" ] || fail "close.md C3 verified-retry write block not found"
58
+ [ -n "$DERIVE_BLOCK" ] || fail "close.md C3 state-derivation block not found"
59
+
60
+ # run_derive <parent body> <states "N=STATE ..."> [lost_edits]
61
+ # gh stub: parent body lives in a file; the first <lost_edits> edits are clobbered by a stale sibling write.
62
+ run_derive() {
63
+ local body="$1" states="$2" lost="${3:-0}" dir rendered
64
+ dir=$(mktemp -d)
65
+ printf '%s\n' "$body" >"$dir/body"; printf '%s\n' "$body" >"$dir/stale"
66
+ echo 0 >"$dir/edits"
67
+ gh() {
68
+ case "$1 $2" in
69
+ "issue view")
70
+ if [ "$6" = "--json" ] && [ "$7" = "body" ]; then cat "$dir/body"; return 0; fi
71
+ if [ "$6" = "--json" ] && [ "$7" = "state" ]; then
72
+ local kv; for kv in $states; do [ "${kv%%=*}" = "$3" ] && { echo "${kv#*=}"; return 0; }; done
73
+ return 1
74
+ fi ;;
75
+ "issue edit")
76
+ local n; n=$(( $(cat "$dir/edits") + 1 )); echo "$n" >"$dir/edits"
77
+ if [ "$n" -le "$lost" ]; then cp "$dir/stale" "$dir/body"; else printf '%s\n' "$7" >"$dir/body"; fi ;;
78
+ esac
79
+ }
80
+ sleep() { :; }
81
+ rendered=$(printf '%s\n%s\n' "$WRITE_BLOCK" "$DERIVE_BLOCK" | sed -e 's/{GH_FLAG}/-R o\/r/g' -e 's/{GH_REPO}/o\/r/g' -e 's/{NUMBER}/7/g')
82
+ PARENT_REF=5
83
+ eval "$rendered" >/dev/null 2>&1 || true
84
+ printf 'OPEN_SUBS=%s UPDATED=%s EDITS=%s CHECKED7=%s\n' "$OPEN_SUBS" "$PARENT_UPDATED" "$(cat "$dir/edits")" \
85
+ "$(grep -cE '^- \[x\] #7( |$)' "$dir/body" || true)"
86
+ rm -rf "$dir"; unset -f gh sleep
87
+ }
88
+
89
+ TWO=$'- [ ] #7 first\n- [ ] #8 second'
90
+
91
+ # Lost checkbox is re-applied by the retry loop (first write clobbered by a sibling).
92
+ OUT=$(run_derive "$TWO" "7=CLOSED 8=CLOSED" 1)
93
+ printf '%s\n' "$OUT" | grep -q 'CHECKED7=1' || fail "retry: checkbox not restored after a lost write ($OUT)"
94
+ printf '%s\n' "$OUT" | grep -q 'UPDATED=true' || fail "retry: PARENT_UPDATED not set ($OUT)"
95
+ printf '%s\n' "$OUT" | grep -q 'EDITS=2' || fail "retry: expected exactly 2 writes ($OUT)"
96
+
97
+ # Race: every write is lost, boxes stay unchecked, but all subs are CLOSED => 0 open (parent completes).
98
+ OUT=$(run_derive "$TWO" "7=CLOSED 8=CLOSED" 9)
99
+ printf '%s\n' "$OUT" | grep -q 'OPEN_SUBS=0' || fail "race: stale checklist with all subs CLOSED must derive 0 open ($OUT)"
100
+ printf '%s\n' "$OUT" | grep -q 'EDITS=3' || fail "race: retries must be bounded at 3 writes ($OUT)"
101
+
102
+ # A still-OPEN sibling keeps the parent open.
103
+ OUT=$(run_derive "$TWO" "7=CLOSED 8=OPEN")
104
+ printf '%s\n' "$OUT" | grep -q 'OPEN_SUBS=1' || fail "one open sibling must derive 1 open ($OUT)"
105
+
106
+ # State lookup failure falls back to the checklist count (fail closed, never 0 from a failed lookup).
107
+ OUT=$(run_derive "$TWO" "7=CLOSED")
108
+ printf '%s\n' "$OUT" | grep -q 'OPEN_SUBS=1' || fail "lookup failure must fall back to checklist count ($OUT)"
109
+
110
+ # Already-checked box: no write needed.
111
+ OUT=$(run_derive $'- [x] #7 first\n- [ ] #8 second' "7=CLOSED 8=CLOSED")
112
+ printf '%s\n' "$OUT" | grep -q 'EDITS=0' || fail "already checked: must not rewrite the parent ($OUT)"
113
+
114
+ # #7 must not match #70.
115
+ OUT=$(run_derive $'- [ ] #70 other\n- [ ] #7 first' "7=CLOSED 70=OPEN")
116
+ printf '%s\n' "$OUT" | grep -q 'OPEN_SUBS=1' || fail "#70 must stay open and not be confused with #7 ($OUT)"
117
+ printf '%s\n' "$OUT" | grep -q 'CHECKED7=1' || fail "#7 checkbox must be checked without touching #70 ($OUT)"
118
+
119
+ # A non-leading #N mention on an unchecked line is an open item, not a sub-issue ref (no fail-open close).
120
+ OUT=$(run_derive $'- [ ] #7 first\n- [ ] #8 second\n- [ ] Write docs (see #9)' "7=CLOSED 8=CLOSED")
121
+ printf '%s\n' "$OUT" | grep -q 'OPEN_SUBS=1' || fail "non-leading ref line must count as open ($OUT)"
122
+
54
123
  printf 'PASS: close-parent-c3\n'
@@ -1,9 +1,12 @@
1
1
  #!/usr/bin/env bash
2
- # diff-size.sh — the ONE measurement of a build's diff size for the build phase size gate (forge#3450).
2
+ # diff-size.sh — the ONE measurement of a build's diff size and scope (forge#3450, forge#3862).
3
3
  # SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios
4
4
  # SPDX-License-Identifier: AGPL-3.0-or-later
5
5
  #
6
6
  # Usage: diff-size.sh --repo-path P --base B [--threshold N] [--exclude-glob G]...
7
+ # [--test-weight PCT] [--module-depth N] [--plan-files FILE] [--estimate N] [--overrun-pct PCT]
8
+ # The size is never a gate (forge#3862): review_weight scales review depth and the scope diff (--plan-files) flags
9
+ # files outside the architect plan. --threshold/over/cohesive are DEPRECATED (kept one release for old callers).
7
10
  # Measures lines changed (added + deleted) by the index plus any earlier commits against the MERGE BASE of
8
11
  # HEAD and origin/B (git diff --cached --numstat --no-renames $(git merge-base HEAD origin/B)), so commits that
9
12
  # landed on the base after the branch point are never counted as this build's changes. Exit 2 when the merge
@@ -15,7 +18,7 @@
15
18
  # build/ dist/ generated/ fixtures/ vendor/ are anchored to the repo root (so first-party dirs such as
16
19
  # commands/work-on/build/ are counted); __generated__/ and __snapshots__/ match at any depth.
17
20
  # Globs are data, never evaluated.
18
- # Default globs (mirrored in docs/CONFIG.md under build.diff_size.exclude_globs):
21
+ # Default globs (mirrored in docs/CONFIG.md under build.scope.exclude_globs, deprecated alias build.diff_size.exclude_globs):
19
22
  # lockfiles (package-lock.json yarn.lock pnpm-lock.yaml Cargo.lock poetry.lock uv.lock go.sum Gemfile.lock
20
23
  # composer.lock), *.min.*, *.snap, root-anchored dist/ build/ generated/ fixtures/ vendor/, any-depth
21
24
  # __generated__/ __snapshots__/
@@ -27,6 +30,17 @@
27
30
  # by basename); cohesive_lines = the union (a new test file counts once). cohesive=true when diff_lines>0 and
28
31
  # cohesive_lines*100 >= 70*diff_lines and diff_lines <= threshold*3/2 (integer math; threshold 0 -> false).
29
32
  # Test lines are classified, never discounted from diff_lines.
33
+ # review_weight (forge#3862): ws_lines = lines changed with whitespace ignored (git diff -w; a pure re-indent is 0)
34
+ # over the counted (non-excluded) files; review_weight = ceil((ws_lines of non-test files + --test-weight percent
35
+ # (default 50) of ws_lines of test files)); modules = distinct directories (first --module-depth path segments,
36
+ # default 2) holding a file with at least one non-whitespace changed line. Keys ws_lines, review_weight, modules
37
+ # follow cohesive= and precede top=.
38
+ # Scope diff, emitted only with --plan-files and/or --estimate: --plan-files FILE lists the architect's planned files
39
+ # (one per line; a leading "- " or "* ", backticks and "./" are stripped; an entry ending in "/" is a directory prefix;
40
+ # blank and "#" lines are ignored). scope=no_baseline (no entries) | ok | out_of_plan; plan_files=N; out_of_plan_count=K;
41
+ # up to 50 "out_of_plan=<path>" lines (counted files absent from the plan). --estimate N with --overrun-pct P
42
+ # (default 200) sets overrun=true when ws_lines*100 > N*P (N=0 means no estimate: overrun=false). Unreadable
43
+ # --plan-files is exit 2 (fail closed: an unreadable plan is never "in plan").
30
44
  # Exit: 0 measured (even when over); 2 usage or git failure with EMPTY stdout (fail closed: unknown is not zero).
31
45
 
32
46
  set -euo pipefail
@@ -34,7 +48,7 @@ set -euo pipefail
34
48
  _TEL="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/telemetry.sh"; [ -f "$_TEL" ] && . "$_TEL" 2>/dev/null
35
49
  type forge_emit >/dev/null 2>&1 || forge_emit() { :; }
36
50
 
37
- REPO_PATH=""; BASE=""; THRESHOLD="1000"; EXTRA=()
51
+ REPO_PATH=""; BASE=""; THRESHOLD="1000"; EXTRA=(); TEST_WEIGHT="50"; MODULE_DEPTH="2"; PLAN_FILE=""; ESTIMATE=""; OVERRUN_PCT="200"
38
52
  die() { echo "diff-size.sh: $*" >&2; exit 2; }
39
53
  while [ $# -gt 0 ]; do
40
54
  case "$1" in
@@ -42,12 +56,33 @@ while [ $# -gt 0 ]; do
42
56
  --base) [ $# -ge 2 ] || die "missing value for $1"; BASE="$2"; shift 2 ;;
43
57
  --threshold) [ $# -ge 2 ] || die "missing value for $1"; THRESHOLD="$2"; shift 2 ;;
44
58
  --exclude-glob) [ $# -ge 2 ] || die "missing value for $1"; [ -z "$2" ] || EXTRA+=("$2"); shift 2 ;;
59
+ --test-weight) [ $# -ge 2 ] || die "missing value for $1"; TEST_WEIGHT="$2"; shift 2 ;;
60
+ --module-depth) [ $# -ge 2 ] || die "missing value for $1"; MODULE_DEPTH="$2"; shift 2 ;;
61
+ --plan-files) [ $# -ge 2 ] || die "missing value for $1"; PLAN_FILE="$2"; shift 2 ;;
62
+ --estimate) [ $# -ge 2 ] || die "missing value for $1"; ESTIMATE="$2"; shift 2 ;;
63
+ --overrun-pct) [ $# -ge 2 ] || die "missing value for $1"; OVERRUN_PCT="$2"; shift 2 ;;
45
64
  *) die "unknown argument: $1" ;;
46
65
  esac
47
66
  done
48
67
  [ -n "$REPO_PATH" ] && [ -d "$REPO_PATH" ] || die "--repo-path must be an existing directory"
49
68
  [ -n "$BASE" ] || die "--base is required"
50
69
  case "$THRESHOLD" in ""|*[!0-9]*) die "--threshold must be a non-negative integer" ;; esac
70
+ case "$TEST_WEIGHT" in ""|*[!0-9]*) die "--test-weight must be a non-negative integer percent" ;; esac
71
+ case "$MODULE_DEPTH" in ""|0|*[!0-9]*) die "--module-depth must be a positive integer" ;; esac
72
+ case "$OVERRUN_PCT" in ""|*[!0-9]*) die "--overrun-pct must be a non-negative integer percent" ;; esac
73
+ case "$ESTIMATE" in ""|*[!0-9]*) [ -z "$ESTIMATE" ] || die "--estimate must be a non-negative integer" ;; esac
74
+ PLAN=(); PLAN_SET=0
75
+ if [ -n "$PLAN_FILE" ]; then
76
+ [ -f "$PLAN_FILE" ] && [ -r "$PLAN_FILE" ] || die "--plan-files must be a readable file"
77
+ PLAN_SET=1
78
+ while IFS= read -r _l || [ -n "$_l" ]; do
79
+ _l="${_l#"${_l%%[![:space:]]*}"}"; _l="${_l%"${_l##*[![:space:]]}"}"
80
+ case "$_l" in ""|"#"*) continue ;; esac
81
+ case "$_l" in "- "*|"* "*) _l="${_l#??}" ;; esac
82
+ _l="${_l//\`/}"; _l="${_l#./}"
83
+ [ -n "$_l" ] && PLAN+=("$_l")
84
+ done < "$PLAN_FILE"
85
+ fi
51
86
 
52
87
  DEFAULTS=(package-lock.json yarn.lock pnpm-lock.yaml Cargo.lock poetry.lock uv.lock go.sum Gemfile.lock composer.lock
53
88
  '*.min.*' '*.snap' __generated__/ __snapshots__/)
@@ -102,6 +137,32 @@ while IFS= read -r -d $'\001' rec; do
102
137
  fi
103
138
  done <<< "$RAW"
104
139
 
140
+ # Whitespace-insensitive pass (forge#3862): review_weight and module count come from `git diff -w`.
141
+ RAWW="$(git -C "$REPO_PATH" diff --cached -w --numstat --no-renames -z "$MB" -- | tr '\0' '\001')" \
142
+ || die "git diff -w against merge base $MB failed"
143
+ WSL=0; WS_NT=0; WS_T=0; MODS=$'\001'; NMODS=0
144
+ while IFS= read -r -d $'\001' rec; do
145
+ [ -n "$rec" ] || continue
146
+ a="${rec%%$'\t'*}"; rest="${rec#*$'\t'}"; d="${rest%%$'\t'*}"; path="${rest#*$'\t'}"
147
+ case "$a" in ''|*[!0-9]*) a=0 ;; esac
148
+ case "$d" in ''|*[!0-9]*) d=0 ;; esac
149
+ n=$((a + d))
150
+ excluded "$path" && continue
151
+ WSL=$((WSL + n))
152
+ if is_test "$path"; then WS_T=$((WS_T + n)); else WS_NT=$((WS_NT + n)); fi
153
+ if [ "$n" -gt 0 ]; then
154
+ dir="${path%/*}"; [ "$dir" = "$path" ] && dir="."
155
+ mod=""; _i=0; _rest="$dir"
156
+ while [ "$_i" -lt "$MODULE_DEPTH" ] && [ -n "$_rest" ]; do
157
+ _seg="${_rest%%/*}"; mod="${mod:+$mod/}$_seg"
158
+ case "$_rest" in */*) _rest="${_rest#*/}" ;; *) _rest="" ;; esac
159
+ _i=$((_i + 1))
160
+ done
161
+ case "$MODS" in *$'\001'"$mod"$'\001'*) ;; *) MODS="${MODS}${mod}"$'\001'; NMODS=$((NMODS + 1)) ;; esac
162
+ fi
163
+ done <<< "$RAWW"
164
+ REVIEW_WEIGHT=$(( (WS_NT * 100 + WS_T * TEST_WEIGHT + 99) / 100 ))
165
+
105
166
  OVER=false
106
167
  if [ "$THRESHOLD" -gt 0 ] && [ "$TOTAL" -gt "$THRESHOLD" ]; then OVER=true; fi
107
168
  echo "diff_lines=$TOTAL"
@@ -115,8 +176,36 @@ echo "new_lines=$NEWL"
115
176
  echo "test_lines=$TESTL"
116
177
  echo "cohesive_lines=$COHL"
117
178
  echo "cohesive=$COHESIVE"
179
+ echo "ws_lines=$WSL"
180
+ echo "review_weight=$REVIEW_WEIGHT"
181
+ echo "modules=$NMODS"
182
+ OOP_LIST=""; OOP=0; SCOPE=ok
183
+ if [ "$PLAN_SET" = 1 ] || [ -n "$ESTIMATE" ]; then
184
+ if [ "${#PLAN[@]}" -eq 0 ]; then SCOPE=no_baseline; else
185
+ while IFS= read -r -d $'\001' rec; do
186
+ [ -n "$rec" ] || continue
187
+ path="${rec#*$'\t'}"; path="${path#*$'\t'}"
188
+ excluded "$path" && continue
189
+ inplan=0
190
+ for _p in "${PLAN[@]}"; do
191
+ case "$_p" in */) case "$path" in "$_p"*) inplan=1; break ;; esac ;; *) [ "$path" = "$_p" ] && { inplan=1; break; } ;; esac
192
+ done
193
+ if [ "$inplan" = 0 ]; then OOP=$((OOP + 1)); OOP_LIST="${OOP_LIST}${path}"$'\n'; fi
194
+ done <<< "$RAW"
195
+ [ "$OOP" -gt 0 ] && SCOPE=out_of_plan
196
+ fi
197
+ OVERRUN=false
198
+ if [ -n "$ESTIMATE" ] && [ "$ESTIMATE" -gt 0 ] && [ $((WSL * 100)) -gt $((ESTIMATE * OVERRUN_PCT)) ]; then OVERRUN=true; fi
199
+ echo "scope=$SCOPE"
200
+ echo "plan_files=${#PLAN[@]}"
201
+ echo "out_of_plan_count=$OOP"
202
+ [ -n "$ESTIMATE" ] && { echo "estimate=$ESTIMATE"; echo "overrun=$OVERRUN"; }
203
+ fi
118
204
  _res=OK; if [ "$COHESIVE" = true ]; then _res=COHESIVE; elif [ "$OVER" = true ]; then _res=OVER; fi
119
- forge_emit diff_size lines="$TOTAL" excluded="$EXCL" threshold="$THRESHOLD" result="$_res"
205
+ forge_emit diff_size lines="$TOTAL" excluded="$EXCL" threshold="$THRESHOLD" result="$_res" review_weight="$REVIEW_WEIGHT" modules="$NMODS" ws_lines="$WSL"
120
206
  if [ -n "$LIST" ]; then
121
207
  printf '%s' "$LIST" | sort -t' ' -k1,1nr | head -n 10 | sed 's/^/top=/' || true
122
208
  fi
209
+ if [ -n "$OOP_LIST" ]; then
210
+ printf '%s' "$OOP_LIST" | head -n 50 | sed 's/^/out_of_plan=/' || true
211
+ fi