forgedock 1.7.1 → 1.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/engine/admission.mjs +242 -7
- package/bin/engine/admission.test.mjs +183 -2
- package/bin/engine/phases.mjs +31 -22
- package/bin/engine/resolve.mjs +8 -1
- package/bin/engine/resolve.test.mjs +5 -0
- package/bin/engine.mjs +26 -17
- package/bin/forgedock.mjs +15 -2
- package/bin/hooks/pre-tool-use.mjs +573 -2
- package/bin/journey.mjs +11 -7
- package/bin/labels.json +5 -0
- package/bin/opencode-adapter.mjs +47 -13
- package/bin/orchestrate-preflight.mjs +434 -0
- package/bin/runner.mjs +30 -4
- package/commands/audit-agents.md +75 -26
- package/commands/autopilot.md +4 -4
- package/commands/cleanup.md +10 -5
- package/commands/issue.md +95 -35
- package/commands/orchestrate/phase-1-resolve.md +70 -32
- package/commands/orchestrate/phase-3-dependency.md +141 -13
- package/commands/orchestrate/phase-4-execution.md +660 -70
- package/commands/orchestrate/phase-5-cleanup.md +5 -11
- package/commands/orchestrate/phase-6-report.md +27 -1
- package/commands/orchestrate/safety.md +1 -1
- package/commands/orchestrate.md +39 -1
- package/commands/quality-gate.md +30 -10
- package/commands/review-pr-agents/protocols.md +13 -0
- package/commands/review-pr-agents.md +1 -1
- package/commands/review-pr-staging.md +100 -18
- package/commands/review-pr.md +78 -24
- package/commands/upgrade-deps.md +1 -1
- package/commands/work-on/build/validate.md +25 -4
- package/commands/work-on/close.md +40 -10
- package/commands/work-on/decompose.md +33 -1
- package/commands/work-on/investigate.md +33 -0
- package/commands/work-on/remediate.md +40 -5
- package/commands/work-on/review.md +27 -0
- package/commands/work-on.md +48 -5
- package/docs/CONFIG.md +45 -10
- package/docs/OPENCODE.md +21 -6
- package/docs/spec/review-protocol.md +7 -0
- package/package.json +1 -1
- package/scripts/check-admission-parity.mjs +1 -0
- package/scripts/check-command-side-effects.sh +14 -13
- package/scripts/check-shell-portability.sh +36 -0
- package/scripts/check-spec-markers.sh +21 -11
- package/scripts/checkbox-sections.test.sh +67 -0
- package/scripts/extract-affected-files.sh +85 -5
- package/scripts/extract-affected-files.test.sh +154 -2
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
---
|
|
2
|
+
|
|
2
3
|
install: core
|
|
3
4
|
---
|
|
4
5
|
<!-- SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios -->
|
|
@@ -167,7 +168,7 @@ done
|
|
|
167
168
|
|
|
168
169
|
**Two independent filters apply, in sequence**: first the T0-vs-backlog *time* filter (`CASCADE_SEARCH` above — which issues are even fetched), then the generation *depth* filter below (which of the fetched issues are admitted). They answer different questions — "how far back in time" vs. "how many cascade hops deep" — and either can be widened independently of the other.
|
|
169
170
|
|
|
170
|
-
**Generation > `max_generation` findings are excluded by default** by the loop above — a `review-finding` issue whose *source* chain is deeper than `orchestration.cascade.max_generation` (default: 1, i.e. generation ≥ 2 excluded — the pre-#2234 behavior, unchanged when the section is absent) is normally deferred
|
|
171
|
+
**Generation > `max_generation` findings are excluded by default** by the loop above — a `review-finding` issue whose *source* chain is deeper than `orchestration.cascade.max_generation` (default: 1, i.e. generation ≥ 2 excluded — the pre-#2234 behavior, unchanged when the section is absent) is normally deferred by Step 4C's autonomous check. The only automated exception is bounded P3 aggregation: a generation-2 P3 may become part of one auditable batch under `batch_max_generation`; it is never individually re-dispatched. That cap is an **autonomy guard**, not a human-request guard: it exists to stop an unattended run from cascading forever, not to block an operator who explicitly asked for this exact bucket of work. See `phase-4-execution.md` Step 4C rule 1 and the reworded anti-pattern note for the full rationale.
|
|
171
172
|
|
|
172
173
|
This resolve step is a human-requested entry point (the operator typed `cascade`/`review-findings`/`findings` directly), so it honors explicit overrides for both filters:
|
|
173
174
|
|
|
@@ -344,13 +345,31 @@ done
|
|
|
344
345
|
|
|
345
346
|
**MUST NOT**: This hint MUST NOT be used, anywhere in this pipeline, to auto-close an issue, auto-skip it from dispatch, exclude it from the resolved issue set, or substitute for the investigation agent's own evidence-based verdict. Every `likely-moot: yes` issue is still dispatched into a full `/work-on` pipeline exactly like every other issue in the resolved set — the "CRITICAL: No duplicate detection at orchestrator level" rule below governs this identically.
|
|
346
347
|
|
|
347
|
-
###
|
|
348
|
+
### Review-Finding Batching (deterministic grouping rule)
|
|
348
349
|
|
|
349
350
|
<!-- Added: forge#1333 -->
|
|
350
351
|
<!-- Extended: forge#1818 — default-batchable eligibility, surface-area grouping, lowered same-file threshold -->
|
|
351
352
|
<!-- Extended: forge#2232 — priority-label schema tolerance (bare P<n> vs priority:P<n>) -->
|
|
352
353
|
|
|
353
|
-
Before finalizing the issue set, apply the
|
|
354
|
+
Before finalizing the issue set, apply the shared batching rule to reduce full-pipeline overhead on routine review findings.
|
|
355
|
+
|
|
356
|
+
**Single implementation (MANDATORY):** `bin/engine/admission.mjs` owns the eligibility, same-file, leaf-directory, age, eight-member, open-batch-extension, and singleton-reason rules through `planP3Batches()`. Every admission point supplies its complete current candidate registry plus parsed open batch metadata to that function, then executes only its returned `create`/`extend` actions. Do not independently reproduce thresholds or eligibility predicates in this file. The legacy shell examples below describe GitHub data collection and action execution only; their grouping decisions must be replaced by the evaluator result. Include original resolved issues, prior-cycle singletons, cascade findings, predicate re-resolution matches, and completion-sweep candidates in the registry. <!-- Added: forge#2851 -->
|
|
357
|
+
|
|
358
|
+
After executing the plan, retain every singleton in the registry for the next admission event and record `summarizeP3BatchPlan()` in the per-run batching summary: clusters formed, members absorbed, open batches extended, and ungrouped members with their reasons.
|
|
359
|
+
|
|
360
|
+
```bash
|
|
361
|
+
# BATCH_CANDIDATE_REGISTRY_JSON contains every candidate accumulated so far, with
|
|
362
|
+
# a stable `id` (`{repo}:{number}`), repo, title, problem/body, labels, affectedFile,
|
|
363
|
+
# createdAt, and isBatch. OPEN_BATCHES_JSON contains existing batch issue metadata.
|
|
364
|
+
# Run this after initial resolution and every later admission event; execute returned
|
|
365
|
+
# actions using memberIds, never bare issue numbers, so satellite issues cannot collide.
|
|
366
|
+
BATCH_PLAN=$(node -e '
|
|
367
|
+
import("{REPO_PATH}/bin/engine/admission.mjs").then(({ planP3Batches, summarizeP3BatchPlan }) => {
|
|
368
|
+
const plan = planP3Batches({ candidates: JSON.parse(process.argv[1]), openBatches: JSON.parse(process.argv[2]) });
|
|
369
|
+
console.log(JSON.stringify({ plan, summary: summarizeP3BatchPlan(plan) }));
|
|
370
|
+
});
|
|
371
|
+
' "$BATCH_CANDIDATE_REGISTRY_JSON" "$OPEN_BATCHES_JSON")
|
|
372
|
+
```
|
|
354
373
|
|
|
355
374
|
**Priority label schema**: ForgeDock's own issue creator (`review-pr.md`) writes only the canonical `priority:P<n>` label form. Some repos this pipeline operates against (issues opened externally, imported, or predating ForgeDock adoption) instead carry a bare `P<n>` label with no `priority:` prefix. Every priority-read check in this section — and its mirrors in `phase-4-execution.md` and `cleanup.md` — MUST accept both forms. `priority:P<n>` wins when (unusually) both are present on the same issue. Issue-*creation* call sites (the batch-issue creation below) are unaffected and keep writing canonical `priority:P<n>` only — there is no case where this pipeline needs to *write* the bare form. <!-- Added: forge#2232 -->
|
|
356
375
|
|
|
@@ -360,11 +379,10 @@ Before finalizing the issue set, apply the P3 batching rule to reduce full-pipel
|
|
|
360
379
|
|
|
361
380
|
The `<!-- FORGE:BATCHABLE -->` marker (still appended by `review-pr.md` at finding-creation time) is honored when present but is no longer REQUIRED for eligibility — a `review-finding`+`priority:P3` issue is batchable by default unless explicitly excluded. This closes the gap where cascade-spawned findings that never carried the marker sat un-batched indefinitely. <!-- Added: forge#1818 -->
|
|
362
381
|
|
|
363
|
-
**Safety
|
|
364
|
-
- Issue
|
|
365
|
-
- Issue has a `security`, `billing`, `anti-bot`, or `auth` label
|
|
382
|
+
**Safety classification:** P0/P1 findings stay individual because batching adds urgent-work latency. P2 is an ordering signal, not a batching veto. Billing is never batched, and path-owned danger zones, migrations, high-fan-in entrypoints, compose files, and `.env.example` are excluded by `batchExclusionReason()` in `bin/engine/admission.mjs`. The default path map excludes `infra/migrations/*credit_balance*` and `services/api/app/billing/**` regardless of wording. Security-relevant findings may batch only with members of the same coarse class, capped at **3** members. Every exclusion must log its `urgency`, `domain`, or `high-blast-radius` predicate.
|
|
383
|
+
- Issue has a `needs-human`, `blocked`, or `operator-only` label, or explicitly states `operator-only`, `manual action required`, or `human action required` in its title or `## Problem` section
|
|
366
384
|
|
|
367
|
-
|
|
385
|
+
Use `bin/engine/admission.mjs`'s `classifyBatchSafety()` as the reference classifier in every mirror. It recognizes `inject`, `injection`, `xss`, `csrf`, `ssrf`, `bypass`, `escalat`, `credential`, `secret`, `token`, `password`, `pgpassword`, `htpasswd`, `redact`, `sanitiz`, `scheme`, `traversal`, `deserializ`, `rce`, and `privilege`, as well as security/anti-bot and auth. Auth matches compound identifiers (`MaintenanceAuth`, `AdminAuth`, `authz_check`) but not `authority_source`; exact `**Agent**:` attribution lines are stripped before classification. A `FORGE:CLASS` slug takes precedence when present, otherwise use this coarse class. <!-- Changed: forge#2859 -->
|
|
368
386
|
|
|
369
387
|
**Grouping algorithm (surface area — same file first, leaf directory as broader fallback):** <!-- Changed: forge#1818 — was domain-only -->
|
|
370
388
|
```bash
|
|
@@ -372,8 +390,8 @@ These exclusions apply regardless of priority: P1/P2 findings are already never
|
|
|
372
390
|
# NOTE: `--label` is an exact-match GH filter and cannot OR "priority:P3" with bare "P3" in
|
|
373
391
|
# one query, so the P3 test moves into the jq predicate below (schema-tolerant, forge#2232).
|
|
374
392
|
# Only "review-finding" stays in the --label filter.
|
|
375
|
-
#
|
|
376
|
-
#
|
|
393
|
+
# Billing remains the sole absolute exclusion. Security findings stay in the
|
|
394
|
+
# candidate set so they can be grouped by their same `FORGE:CLASS`/coarse class.
|
|
377
395
|
# Word-boundary anchored so it matches whole terms only, not substrings:
|
|
378
396
|
# `authority_source`/`authoritative`/`author`/`authored` no longer trip `auth`.
|
|
379
397
|
# `authentication|authorization|authn|authz` are listed explicitly so real
|
|
@@ -384,7 +402,7 @@ BATCHABLE_P3=$(gh issue list {GH_FLAG} \
|
|
|
384
402
|
--limit 500 \
|
|
385
403
|
--json number,title,body,labels \
|
|
386
404
|
--jq '.[] | select([.labels[].name] | any(test("^(priority:)?P3$")))
|
|
387
|
-
| select((.title | test("\\b(
|
|
405
|
+
| select((.title | test("\\b(billing|operator-only|manual action required|human action required)\\b"; "i")) | not)
|
|
388
406
|
# Strip the review-finding template's attribution boilerplate
|
|
389
407
|
# (**Confidence**/**Severity**/**Review comment** — see forge#2477
|
|
390
408
|
# note below for why **Source**/**Agent** are deliberately excluded
|
|
@@ -410,8 +428,8 @@ BATCHABLE_P3=$(gh issue list {GH_FLAG} \
|
|
|
410
428
|
# is not auto-batched) for closing a real bypass — the safe direction
|
|
411
429
|
# for a security-relevant exclusion. <!-- forge#2477 -->
|
|
412
430
|
| (.body | gsub("(?m)^\\*\\*(?:Confidence\\*\\*: (?:CONFIRMED|LIKELY|POSSIBLE)|Severity\\*\\*: (?:CRITICAL|HIGH|MEDIUM|LOW|INFO)|Review comment\\*\\*: https?://\\S+)$"; "")) as $stripped_body
|
|
413
|
-
| select($stripped_body | test("## Problem[\\s\\S]{0,500}\\b(
|
|
414
|
-
| select(([.labels[].name] | any(. == "
|
|
431
|
+
| select($stripped_body | test("## Problem[\\s\\S]{0,500}\\b(billing|operator-only|manual action required|human action required)\\b"; "i") | not)
|
|
432
|
+
| select(([.labels[].name] | any(. == "billing" or . == "needs-human" or . == "blocked" or . == "operator-only")) | not)')
|
|
415
433
|
|
|
416
434
|
# Surface area = the exact affected file path listed first under "## Affected Files" (primary grouping key).
|
|
417
435
|
# Leaf directory = dirname of that file (broader fallback grouping key, formerly called "domain").
|
|
@@ -428,10 +446,25 @@ BATCHABLE_P3=$(gh issue list {GH_FLAG} \
|
|
|
428
446
|
# Title "fix auth bypass in login flow" → still excluded (genuine auth finding — true positive preserved)
|
|
429
447
|
```
|
|
430
448
|
|
|
431
|
-
**Batch creation rule (
|
|
449
|
+
**Batch creation rule (ordered grouping keys):** <!-- Changed: forge#1818 — added lower same-file tier -->
|
|
432
450
|
- **Same-file cluster** (primary, low threshold): When **2+** batchable P3 issues share the exact same affected file, create a batch issue for that file cluster. Same-file P3 findings are the dominant low-value token sink (dead imports, stale comments, style nits) and already conflict with each other if built individually — the low threshold reflects that they'd otherwise serialize into slow one-at-a-time chains regardless of count.
|
|
433
|
-
- **
|
|
434
|
-
-
|
|
451
|
+
- **Source-PR cohort**: When **2+** remaining findings cite the same `**Source**: PR #N`, create a batch regardless of their affected paths. The source citation is parsed mechanically; a source PR that closed unmerged remains eligible.
|
|
452
|
+
- **Defect-class cluster**: When **2+** remaining findings have the same `<!-- FORGE:CLASS: <slug> -->` annotation, create a batch. This is opt-in and uses only the emitted machine-readable slug; do not infer a class from prose.
|
|
453
|
+
- **Leaf-directory cluster** (broader fallback): When **3+** remaining batchable P3 issues share the same leaf directory but are not already claimed, OR the oldest batchable P3 in that leaf directory exceeds 72 hours, create a batch issue for that leaf-directory cluster.
|
|
454
|
+
- Form groups in this order: same-file, source-PR, defect-class, leaf-directory. A finding is claimed by at most one batch.
|
|
455
|
+
|
|
456
|
+
**Reference implementation and periodic sweep (MANDATORY):** `bin/engine/admission.mjs` exports `planP3BatchGroups()` and `batchExclusionReason()`, the deterministic reference for eligibility, the four ordered keys, the 3-member leaf threshold, and the eight-member cap. Every pass supplies the complete retained candidate registry plus open batches; execute `extensions` before creating new groups, retain `ungrouped` findings, and log each selected group kind or exclusion predicate. Run it at initial resolution and again after every five completions or whenever the deferred queue reaches the concurrency cap. <!-- Added: forge#2858; extended: forge#2851, forge#2852 -->
|
|
457
|
+
|
|
458
|
+
**Generation-cap exception (sanctioned, bounded, and auditable):** P3 batching may aggregate a
|
|
459
|
+
deferred generation-2 finding without `--allow-gen2`: aggregation replaces several low-priority
|
|
460
|
+
pipelines with one reviewed unit and is therefore a bounded alternative to recursively dispatching
|
|
461
|
+
each finding. It is not an implicit unlimited override. Before forming each cluster, use
|
|
462
|
+
`compute_generation` above for every member, retain only members at or below the finite
|
|
463
|
+
`orchestration.cascade.batch_max_generation` ceiling (default `2`), and leave higher-generation
|
|
464
|
+
members deferred. Record the maximum retained member generation as `{BATCH_MAX_GENERATION}` and
|
|
465
|
+
list every retained generation-2-or-higher member in the batch body. This makes the exception
|
|
466
|
+
visible to operators and ensures a generation-6 finding cannot become dispatchable merely by being
|
|
467
|
+
grouped. `policy: all` does not remove this batching ceiling. <!-- Added: forge#2849 -->
|
|
435
468
|
|
|
436
469
|
**Sanitize the surface-area path before interpolation (MANDATORY):** `{SURFACE_AREA}` is an affected-file path derived from an issue body, and git filenames can legally carry shell metacharacters (`` ` ``, `$()`, quotes). Restrict it to a validated `[A-Za-z0-9._/-]` charset before templating it into `--title` / `--body`, so an untrusted issue body cannot break the `gh` argument boundary. The same guard is applied at the mirror site in `phase-4-execution.md`. <!-- forge#1833, forge#1835 -->
|
|
437
470
|
|
|
@@ -452,8 +485,14 @@ Batch of P3 review findings in **{SURFACE_AREA}** (same file or leaf directory),
|
|
|
452
485
|
|
|
453
486
|
<!-- FORGE:BATCH_MEMBERS -->
|
|
454
487
|
{for each member issue: "- [ ] #{NUM}: {TITLE}"}
|
|
488
|
+
{for security-class members only: " - **Verdict**: [ ] live vector [ ] defence-in-depth"}
|
|
455
489
|
<!-- /FORGE:BATCH_MEMBERS -->
|
|
456
490
|
|
|
491
|
+
**Maximum member generation**: {BATCH_MAX_GENERATION}
|
|
492
|
+
<!-- FORGE:BATCH_MAX_GENERATION: {BATCH_MAX_GENERATION} --> <!-- allowlist:check-spec-markers -->
|
|
493
|
+
|
|
494
|
+
**Generation >= 2 members admitted by bounded batching**: {#{NUM} (generation N), or "none"}
|
|
495
|
+
|
|
457
496
|
## Acceptance Criteria
|
|
458
497
|
|
|
459
498
|
- [ ] All member findings addressed or closed as false-positive
|
|
@@ -462,7 +501,7 @@ Batch of P3 review findings in **{SURFACE_AREA}** (same file or leaf directory),
|
|
|
462
501
|
|
|
463
502
|
## Context
|
|
464
503
|
|
|
465
|
-
**Batch policy**: 2+
|
|
504
|
+
**Batch policy**: 2+ same file, 2+ same source PR cohort, 2+ same explicit defect class, or 3+ same leaf directory (or oldest > 72h).
|
|
466
505
|
**Member issues**: #{N1}, #{N2}, #{N3}, ...
|
|
467
506
|
|
|
468
507
|
<!-- FORGE:BATCHABLE -->
|
|
@@ -473,32 +512,32 @@ BATCH_EOF
|
|
|
473
512
|
ISSUE_SKILL_OUTPUT=$(Skill(skill="issue", args="--title \"fix(batch): P3 review findings — ${SAFE_SURFACE_AREA} (batch #{BATCH_N})\" --body-file \"${BATCH_BODY_FILE}\" --label \"review-finding\" --label \"priority:P3\" --label \"batch\" --exclude \"${MEMBER_LIST}\""))
|
|
474
513
|
```
|
|
475
514
|
|
|
476
|
-
**
|
|
515
|
+
**Consume `/issue`'s explicit result contract** (see `commands/issue.md` Phases 2D and 4C). A dedup STOP is an expected, named outcome; any output without either result marker is a hard create failure:
|
|
477
516
|
|
|
478
517
|
```bash
|
|
479
|
-
BATCH_ISSUE_NUM=$(
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
if [ -
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
#
|
|
486
|
-
#
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
echo "WARNING: /issue did not report a created batch issue number — likely a Phase 2D dedup STOP against a non-member issue (a real duplicate — member exclusion via --exclude \"${MEMBER_LIST}\" is already applied above, so this is not a false positive against the cluster's own members) or a usage error. Do not replace member issues with a batch issue for this cluster; leave the members on the standard individual pipeline instead."
|
|
518
|
+
BATCH_ISSUE_NUM=$(printf '%s\n' "$ISSUE_SKILL_OUTPUT" | sed -n 's/.*ISSUE_CREATE_RESULT:CREATED number=\([0-9][0-9]*\).*/\1/p' | head -1)
|
|
519
|
+
DEDUP_NUMBER=$(printf '%s\n' "$ISSUE_SKILL_OUTPUT" | sed -n 's/.*ISSUE_CREATE_RESULT:DEDUP number=\([0-9][0-9]*\).*/\1/p' | head -1)
|
|
520
|
+
|
|
521
|
+
if [ -n "$DEDUP_NUMBER" ]; then
|
|
522
|
+
echo "Batch dedup STOP: existing non-member issue #${DEDUP_NUMBER}; members remain on the standard individual pipeline."
|
|
523
|
+
elif [ -z "$BATCH_ISSUE_NUM" ]; then
|
|
524
|
+
# Neither verified creation nor explicit dedup was reported. Do not replace
|
|
525
|
+
# member issues. <!-- forge#2842 -->
|
|
526
|
+
echo "ERROR: /issue did not report a verified batch issue number. This is a create failure (including a swallowed GitHub secondary-rate-limit response), not a dedup STOP. Do not replace member issues; stop this batch and retry after resolving the GitHub failure." >&2
|
|
527
|
+
exit 1
|
|
490
528
|
fi
|
|
491
529
|
```
|
|
492
530
|
|
|
493
531
|
**Replace member issues with the batch issue** in the resolved issue set. Member issues are NOT individually dispatched to `/work-on` — the batch issue is the single pipeline unit.
|
|
494
532
|
|
|
495
533
|
**Important limits**:
|
|
496
|
-
-
|
|
534
|
+
- Routine batches have at most **8** members. Security-class batches have at most **3** members and never mix classes.
|
|
497
535
|
- P1 and P2 issues are NEVER batched — they keep the standard one-issue-one-PR path
|
|
498
|
-
-
|
|
536
|
+
- Billing is NEVER batched; security P3 findings follow the same-class, three-member rule above.
|
|
537
|
+
- Human-gated findings are NEVER batched: `needs-human`, `blocked`, `operator-only`, and explicit operator-action requests remain independently tracked
|
|
499
538
|
- Batch issues themselves are never nested inside other batch issues
|
|
500
539
|
|
|
501
|
-
If
|
|
540
|
+
If no grouping key reaches its threshold and no leaf-directory candidate exceeds 72h, skip batch creation entirely — individual P3s run through the standard pipeline.
|
|
502
541
|
|
|
503
542
|
### CRITICAL: No duplicate detection at orchestrator level
|
|
504
543
|
|
|
@@ -518,4 +557,3 @@ If fewer than 2 batchable P3 findings share a file, AND fewer than 5 share a lea
|
|
|
518
557
|
**Why**: Surface-level similarity hides critical differences. #3842 (api_key.id lazy load) and #4039 (user.id lazy load) had identical error messages but targeted completely different ORM objects. Closing #4039 as a "duplicate" would have left a customer-impacting P0 bug unfixed.
|
|
519
558
|
|
|
520
559
|
---
|
|
521
|
-
|
|
@@ -67,20 +67,59 @@ Domain estimation (above) catches broad category overlap but misses cases where
|
|
|
67
67
|
|
|
68
68
|
#### Layer 1: Explicit file-overlap extraction
|
|
69
69
|
|
|
70
|
-
**For issues that already have an INVESTIGATOR comment** (from Wave 0 or a prior session), extract their Affected Files list, scoped to that comment's own `### Affected Files` section. **For issues WITHOUT an investigation comment**, fall back to parsing the issue body, scoped to a deliverables-shaped heading (`## Affected Files`, `## Deliverables`, or `### Files to change`) — never the whole body. Both code paths accumulate into a single `LAYER1_FILES` array (declared once, before the loop) — this is the batch-wide file set that Layer 5's co-change query (below) reuses, per the "file list already extracted in Layer 1" reference in Layer 2 and Layer 5. Extraction runs through `scripts/extract-affected-files.sh` (not an inline `grep -oP` over the whole text) so that stray paths mentioned in `## Context` / `## Prior art` / `## Related` / `## Root Cause` — or anywhere outside a deliverables-shaped heading — are never collected as if they were files the issue changes. A populated-but-wrong file list is strictly worse than an empty one: it clears Layer 4's `<2 paths` conservative-serialization threshold with false confidence, defeating the safety net that threshold exists to provide (forge#2436).
|
|
70
|
+
**For issues that already have a `FORGE:CONTRACT` comment** (built in a prior wave or session, or re-extracted mid-batch), extract its `### Deliverables` table first — that table states intent to change, so it outranks every other source (forge#2848). **For issues that already have an INVESTIGATOR comment** (from Wave 0 or a prior session) and no contract-derived paths, extract their Affected Files list, scoped to that comment's own `### Affected Files` section. **For issues WITHOUT an investigation comment**, fall back to parsing the issue body, scoped to a deliverables-shaped heading (`## Affected Files`, `## Deliverables`, or `### Files to change`) — never the whole body. Both code paths accumulate into a single `LAYER1_FILES` array (declared once, before the loop) — this is the batch-wide file set that Layer 5's co-change query (below) reuses, per the "file list already extracted in Layer 1" reference in Layer 2 and Layer 5. Extraction runs through `scripts/extract-affected-files.sh` (not an inline `grep -oP` over the whole text) so that stray paths mentioned in `## Context` / `## Prior art` / `## Related` / `## Root Cause` — or anywhere outside a deliverables-shaped heading — are never collected as if they were files the issue changes. A populated-but-wrong file list is strictly worse than an empty one: it clears Layer 4's `<2 paths` conservative-serialization threshold with false confidence, defeating the safety net that threshold exists to provide (forge#2436).
|
|
71
71
|
|
|
72
|
-
Each issue's extraction also carries a **provenance** tag — `affected-files-section` (from the INVESTIGATOR comment's own scoped section), `body-fallback` (from the raw issue body's scoped section, pre-investigation), or `none` (no scoped section found; correctly yields zero paths so Layer 4 fires). This is recorded per issue in `FILE_SOURCE[$NUM]` and consumed by Layer 5 (below) and Step 3E's plan presentation.
|
|
72
|
+
Each issue's extraction also carries a **provenance** tag, in descending order of confidence — `contract-deliverables` (from a `FORGE:CONTRACT` comment's own `### Deliverables` table), `affected-files-section` (from the INVESTIGATOR comment's own scoped section), `body-fallback` (from the raw issue body's scoped section, pre-investigation), or `none` (no scoped section found; correctly yields zero paths so Layer 4 fires). This is recorded per issue in `FILE_SOURCE[$NUM]` and consumed by Layer 5 (below) and Step 3E's plan presentation.
|
|
73
|
+
|
|
74
|
+
**Why `contract-deliverables` ranks highest** <!-- Added: forge#2848 -->: a deliverables table states *intent to change* ("I will edit this file"), whereas an Affected Files list — and far more so a raw issue body — routinely names files as *context*: "this interacts with X", "similar to the check in Y". The extractor cannot tell those apart, which is what makes a body-derived file list over-predict overlap. **Temporal caveat — do not over-claim this source**: `FORGE:CONTRACT` is posted at *build* time (`work-on/build.md` Phase B2), which is **after** this phase builds the DAG. On a cold first-pass plan no contract exists, so this provenance is inert by construction and extraction lands on one of the three values below. It pays off only on paths that explicitly re-extract later: mid-batch re-derivation on a dropped edge (`phase-4-execution.md`'s DONE-arm handling, lines 1208-1268) and `IN_PROGRESS` predecessors built in an earlier wave or session. Wake/compaction reconstruction does not retain the in-memory edge metadata and does not re-extract. A contract whose deliverables table yields zero paths **falls through** to the investigator source rather than blackholing to `none` — a contract is an upgrade over that source, never a replacement for it.
|
|
73
75
|
|
|
74
76
|
```bash
|
|
75
77
|
LAYER1_FILES=()
|
|
76
78
|
declare -A EDGE_KIND # "{PRED}:{SUCCESSOR}" → same-file | directory | shared-module (forge#1860)
|
|
77
79
|
declare -A EDGE_FILES # "{PRED}:{SUCCESSOR}" → the specific file(s) that triggered the edge (forge#1860)
|
|
78
|
-
declare -A FILE_SOURCE # {NUM} → affected-files-section | body-fallback | none (forge#2436)
|
|
80
|
+
declare -A FILE_SOURCE # {NUM} → contract-deliverables | affected-files-section | body-fallback | none (forge#2436, forge#2848)
|
|
81
|
+
declare -A ISSUE_FILES # {NUM} → newline-separated declared file set (forge#2844)
|
|
82
|
+
|
|
83
|
+
# Resolve ForgeDock's helper from the runtime installation before falling back to
|
|
84
|
+
# the target repository. The orchestrator runs inside the project being worked on,
|
|
85
|
+
# so a bare `scripts/extract-affected-files.sh` silently fails when that project
|
|
86
|
+
# has not copied ForgeDock's helper scripts into its own repository (observed in
|
|
87
|
+
# OpenCode runs against installed ForgeDock). Keep the precedence aligned with
|
|
88
|
+
# phase-4-execution.md's classify-lane resolver.
|
|
89
|
+
resolve_extract_affected_files() {
|
|
90
|
+
local candidates=()
|
|
91
|
+
[ -n "${FORGE_HOME:-}" ] && candidates+=("$FORGE_HOME/scripts/extract-affected-files.sh")
|
|
92
|
+
[ -n "${REPO_PATH:-}" ] && candidates+=("$REPO_PATH/scripts/extract-affected-files.sh")
|
|
93
|
+
candidates+=("$PWD/scripts/extract-affected-files.sh")
|
|
94
|
+
|
|
95
|
+
local candidate
|
|
96
|
+
for candidate in "${candidates[@]}"; do
|
|
97
|
+
if [ -f "$candidate" ]; then
|
|
98
|
+
printf '%s\n' "$candidate"
|
|
99
|
+
return 0
|
|
100
|
+
fi
|
|
101
|
+
done
|
|
102
|
+
echo "ERROR: extract-affected-files.sh is not installed in any configured runtime path." >&2
|
|
103
|
+
return 1
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
AFFECTED_FILES_SCRIPT=$(resolve_extract_affected_files) || {
|
|
107
|
+
echo "ERROR: cannot build file-overlap edges without extract-affected-files.sh" >&2
|
|
108
|
+
exit 1
|
|
109
|
+
}
|
|
110
|
+
|
|
79
111
|
for NUM in {issue_numbers}; do
|
|
80
112
|
echo "=== #$NUM ==="
|
|
81
|
-
EXTRACT_OUT=$(bash
|
|
113
|
+
EXTRACT_OUT=$(bash "$AFFECTED_FILES_SCRIPT" "$NUM" -R "{GH_REPO}")
|
|
82
114
|
FILE_SOURCE[$NUM]=$(echo "$EXTRACT_OUT" | head -1 | sed 's/^PROVENANCE=//')
|
|
83
115
|
FILES_FOR_NUM=$(echo "$EXTRACT_OUT" | tail -n +2)
|
|
116
|
+
ISSUE_FILES[$NUM]="$FILES_FOR_NUM"
|
|
117
|
+
|
|
118
|
+
if [ "${FILE_SOURCE[$NUM]}" = "error" ]; then
|
|
119
|
+
echo "ERROR: affected-file extraction for #$NUM was inconclusive after a GitHub/API failure." >&2
|
|
120
|
+
echo " Do not treat the issue as file-independent; retry the extraction or rerun /orchestrate." >&2
|
|
121
|
+
exit 1
|
|
122
|
+
fi
|
|
84
123
|
|
|
85
124
|
echo "$FILES_FOR_NUM"
|
|
86
125
|
echo " (source: ${FILE_SOURCE[$NUM]})"
|
|
@@ -97,7 +136,7 @@ done
|
|
|
97
136
|
- If two issues share ANY affected file → one MUST be a predecessor of the other (serialized)
|
|
98
137
|
- The issue with lower issue number goes first (stable ordering), unless an explicit `Depends on #` says otherwise
|
|
99
138
|
- Add a conflict note to the DAG plan: "#{A} and #{B} both modify `{file}` — #{A} is predecessor of #{B}"
|
|
100
|
-
- **Record the edge kind and shared file(s)** <!-- Added: forge#1860 -->: alongside the predecessor relationship, set `EDGE_KIND["${A}:${B}"]="same-file"` and `EDGE_FILES["${A}:${B}"]="{file}"` (space-separated if multiple files overlap). Layer 1 is the only point where the specific overlapping file is known — Step 3D's DAG only records that a predecessor relationship exists, not why. `EDGE_KIND`/`EDGE_FILES` have two consumers in `phase-4-execution.md` Step 4B: (1) the DONE-case "same-file current-state brief" forwarding (forge#1860, unchanged), and (2) `verify_file_overlap_edge()` (forge#1904), which re-checks this guessed file list against the predecessor's *actual* PR diff once it reaches FAILED or
|
|
139
|
+
- **Record the edge kind and shared file(s)** <!-- Added: forge#1860 -->: alongside the predecessor relationship, set `EDGE_KIND["${A}:${B}"]="same-file"` and `EDGE_FILES["${A}:${B}"]="{file}"` (space-separated if multiple files overlap). Layer 1 is the only point where the specific overlapping file is known — Step 3D's DAG only records that a predecessor relationship exists, not why. `EDGE_KIND`/`EDGE_FILES` have two consumers in `phase-4-execution.md` Step 4B: (1) the DONE-case "same-file current-state brief" forwarding (forge#1860, unchanged), and (2) `verify_file_overlap_edge()` (forge#1904, extended to the DONE path by forge#2848), which re-checks this guessed file list against the predecessor's *actual* PR diff once it reaches FAILED, GATED, **or DONE** — since Layer 1's file list is extracted from a pre-build investigation guess or a raw issue-body parse, never from real code, it can be wrong, and a predecessor that concludes without ever touching the guessed file should not keep a dependent gated/skipped on it, nor hand it a same-file brief about a file it never wrote. See `phase-4-execution.md`'s "File-overlap edge re-verification" section for the full mechanism.
|
|
101
140
|
|
|
102
141
|
#### Layer 2: Directory-proximity detection
|
|
103
142
|
|
|
@@ -180,6 +219,16 @@ When file extraction yields **fewer than 2 file paths** for an issue (common for
|
|
|
180
219
|
|
|
181
220
|
**Rationale**: The cost of a false-negative (two agents conflict → one fails with merge error, wasting the full agent run) far exceeds the cost of a false-positive (an issue waits for one predecessor before starting). Always err toward serialization when uncertain.
|
|
182
221
|
|
|
222
|
+
**Cohort-confidence guidance — N issues from one audit citing the same file is WEAK evidence, not strong** <!-- Added: forge#2848 -->
|
|
223
|
+
|
|
224
|
+
The rule above is correct and stays. What this guidance qualifies is *what counts as evidence* feeding it, because there is a cohort shape that turns the rule into a total order:
|
|
225
|
+
|
|
226
|
+
When a **single audit, sweep, or analysis run files N issues at once**, those issues are written by one author against one mental model, so they all cite the same handful of files — as background, as "the thing this interacts with", as the example of the pattern. That shared citation reflects **the common origin of the issues, not a real overlap in the code each one will change**. Treat it as *low*-confidence evidence of conflict and prefer a `contract-deliverables` or `affected-files-section` list for every member of the cohort before serializing on it.
|
|
227
|
+
|
|
228
|
+
**Why this matters more here than anywhere else in Step 3C**: a single-domain cohort compounds through the two rules above. Every issue in it shares a domain tag, and audit-filed issues are typically prose-heavy and pre-investigation, so most yield 0–1 extracted paths. That combination — `<2 paths` **and** shared domain — matches the first rule for *every pair in the cohort*, which degenerates to a total order over all N issues. Observed in batch `20260725T180538-orchestrate`: sixteen issues from one backup/DR audit produced a **13-deep serial chain**, an estimated 5–9 hours during which at most one agent could work that track while 16 concurrency slots sat idle. When the chain head merged, its actual diff touched **neither** of the two files the chain had been serialized on; re-splitting into five file-scoped groups dropped the critical path from ~11 to ~4 with no conflict.
|
|
229
|
+
|
|
230
|
+
**Practical rule**: before applying conservative serialization across a cohort, check whether the shared file citation traces to a common filing origin (same audit, same day, same author, near-identical `## Context` prose). If it does, that citation is not independent corroboration from N issues — it is one claim repeated N times. Serialize on a real `EDGE_FILES` entry backed by a contract or a diff, not on the repetition. Layer 1's same-file **hard conflict** rule is unaffected — this guidance never weakens an edge backed by ground truth, and `phase-4-execution.md`'s DONE-arm re-verification is the automatic backstop that unwinds such a chain mid-batch when the first real diff disproves it.
|
|
231
|
+
|
|
183
232
|
#### Layer 5: Historical co-change coupling <!-- Added: forge#1196 --> <!-- Empty-set guard: forge#1206 --> <!-- Matrix lookup: forge#1738 -->
|
|
184
233
|
|
|
185
234
|
Layers 1-4 infer conflict risk from structure — path overlap, directory nesting, hard-coded high-fan-in lists. They miss the case where two files with no directory or naming relationship have historically changed together in the same commits (e.g. `models/user.py` and `services/billing/charge.py`), and they over-serialize the inverse case where files merely sit near each other but have never actually co-changed. Git commit history answers both questions directly and empirically. This layer reads **commit metadata only** — the list of files touched per commit — never file contents, so it does not violate Hard Rule 2 ("dispatcher, not a builder... never read code").
|
|
@@ -326,7 +375,7 @@ fi
|
|
|
326
375
|
|
|
327
376
|
**Apply the signal:**
|
|
328
377
|
- **High co-change pair spans two different issues in the batch** → add a serialization edge between them (same directed-edge convention as Layers 1-4: lower issue number is predecessor), OR, if the pair also carries competing investigation recommendations, flag it for Phase 2.5 arbitration instead of a blind serialization edge (see cross-reference in Step 2.5B below).
|
|
329
|
-
- **Verified-independent pair** → MAY be used to downgrade an existing Layer 2 "broad directory + different domain" or Layer 4 "conservative fallback" serialization to parallel. This downgrade is **never** applied to Layer 1 (same-file hard conflict, which is ground truth from the current batch, not historical inference) or to Layer 3 high-fan-in-file edges (a file can be structurally high-risk even with a thin history window, e.g. a newly added `main.py`). Ubiquitous-file pairs (n/N > 0.2 for either file) are **ineligible** for verified-independent downgrade even with zero co-occurrences. **Also ineligible**: a pair where either issue's `FILE_SOURCE[$NUM]` (Layer 1, above) is `body-fallback` — that file list came from a pre-investigation scrape of the raw issue body, lower-confidence than a post-investigation `affected-files-section` extraction, so a "verified independent" co-change verdict computed against it is not trustworthy enough to remove an edge. Weak provenance may only ever be used to *add* a conflict edge (Layer 1's own same-file cross-reference, unaffected by this rule), never to *remove* one — mirrors the existing "never overrides Layer 1/Layer 3" carve-out on this same line (forge#2436).
|
|
378
|
+
- **Verified-independent pair** → MAY be used to downgrade an existing Layer 2 "broad directory + different domain" or Layer 4 "conservative fallback" serialization to parallel. This downgrade is **never** applied to Layer 1 (same-file hard conflict, which is ground truth from the current batch, not historical inference) or to Layer 3 high-fan-in-file edges (a file can be structurally high-risk even with a thin history window, e.g. a newly added `main.py`). Ubiquitous-file pairs (n/N > 0.2 for either file) are **ineligible** for verified-independent downgrade even with zero co-occurrences. **Also ineligible**: a pair where either issue's `FILE_SOURCE[$NUM]` (Layer 1, above) is `body-fallback` — that file list came from a pre-investigation scrape of the raw issue body, lower-confidence than a post-investigation `affected-files-section` extraction, so a "verified independent" co-change verdict computed against it is not trustworthy enough to remove an edge. Weak provenance may only ever be used to *add* a conflict edge (Layer 1's own same-file cross-reference, unaffected by this rule), never to *remove* one — mirrors the existing "never overrides Layer 1/Layer 3" carve-out on this same line (forge#2436). `contract-deliverables` is **eligible** — like `affected-files-section` it is a **high**-confidence source, and strictly the stronger of the two (a deliverables table states intent to change rather than mere relevance), so it clears this bar for the same reason (forge#2848). Only `body-fallback` and `none` are excluded.
|
|
330
379
|
- If `ALL_AFFECTED_FILES` is empty, the guard above skips the query and Layer 5 contributes nothing for the entire batch. If the matrix or live query returns no data for a pair → Layer 5 contributes nothing; fall back silently to Layers 1-4's existing verdict for that pair.
|
|
331
380
|
|
|
332
381
|
**Wire-through proof (mandatory check)**: When `LAYER5_SOURCE=matrix`, confirm the matrix lookup path executes on at least one pair in the batch and log the verdict. This proves the path is live, not dead code. If no pairs are in the matrix, log that the live fallback ran instead. <!-- Ref: forge#1731, forge#1230, forge#1244 — Layer 5 has had two dead-code defects; this check prevents recurrence. -->
|
|
@@ -362,7 +411,7 @@ Build a **directed acyclic graph (DAG)** of per-issue dependencies. Each issue g
|
|
|
362
411
|
- **Explicit dependencies**: If issue B says "Depends on #A" or "Blocked by #A", add A to B's predecessors
|
|
363
412
|
- **File-conflict edges**: If two issues share affected files (from Step 3C Layer 1), add a directed edge: lower issue number → higher issue number (unless explicit deps say otherwise). The later issue has the earlier issue in its predecessors.
|
|
364
413
|
- **Domain serialization edges**: DATABASE issues form a linear chain (each has the previous DATABASE issue as its predecessor). Same-small-directory issues (Layer 2) and high-fan-in file issues (Layer 3) get directed edges as per Step 3C rules.
|
|
365
|
-
- **Edge-kind tagging for same-file/directory/shared-module briefing and re-verification** <!-- Added: forge#1860; extended forge#1904 -->: Layer 1/2/3 edges are additionally tagged with `EDGE_KIND` (`same-file` / `directory` / `shared-module`) and `EDGE_FILES` at the point they're added (see Step 3C). Explicit-dependency edges, the DATABASE domain chain, Layer 4 conservative-fallback edges, and Layer 5 co-change edges deliberately do NOT receive one of these three `EDGE_KIND` values. This is what lets Step 4B distinguish "predecessor edge came from Step 3C Layer 1/2/3" from every other edge type, for two purposes: a same-file current-state brief (DONE case, forge#1860) and edge re-verification against the predecessor's actual PR diff once it reaches FAILED or
|
|
414
|
+
- **Edge-kind tagging for same-file/directory/shared-module briefing and re-verification** <!-- Added: forge#1860; extended forge#1904 -->: Layer 1/2/3 edges are additionally tagged with `EDGE_KIND` (`same-file` / `directory` / `shared-module`) and `EDGE_FILES` at the point they're added (see Step 3C). Explicit-dependency edges, the DATABASE domain chain, Layer 4 conservative-fallback edges, and Layer 5 co-change edges deliberately do NOT receive one of these three `EDGE_KIND` values. This is what lets Step 4B distinguish "predecessor edge came from Step 3C Layer 1/2/3" from every other edge type, for two purposes: a same-file current-state brief (DONE case, forge#1860) and edge re-verification against the predecessor's actual PR diff once it reaches FAILED, GATED, or DONE (`verify_file_overlap_edge()`, forge#1904, DONE path added by forge#2848 — see `phase-4-execution.md` Step 4B). On the DONE path the two purposes are ordered: re-verification runs first and, on a `DROP`, suppresses the brief entirely. Non-`EDGE_KIND` edges are eligible for neither — they always `KEEP` in the re-verification check and are never candidates for the same-file brief.
|
|
366
415
|
- **Conservative fallback edges**: Low-confidence issues (Layer 4) get edges to same-domain issues as per Step 3C rules.
|
|
367
416
|
- **Co-change coupling edges** <!-- Added: forge#1196 -->: High co-change file pairs (Layer 5, 3+ shared commits in the bounded window) that span two different issues get a directed edge using the same lower-issue-number-is-predecessor convention as Layer 1. Verified-independent pairs (Layer 5, zero shared commits) may instead REMOVE an edge that Layer 2 or Layer 4 would otherwise have added for that pair — Layer 1 and Layer 3 edges are never removed by a Layer 5 downgrade.
|
|
368
417
|
- **Claims-board downgrade (Layer 2/4 edges only)** <!-- Added: forge#1736 -->: After dispatch begins (Phase 4A), when both issues in a Layer-2 or Layer-4 serialized pair post `FORGE:CLAIM` annotations on the coordination issue and their claimed file sets are **disjoint** (no path appears in both claims), the serialization edge for that pair MAY be relaxed — the blocked issue becomes ready. This downgrade is **never** applied to Layer-1 (same-file) or Layer-3 (high-fan-in) edges. See Step 4B: Claims-board relaxation sweep for the runtime check.
|
|
@@ -969,7 +1018,35 @@ done
|
|
|
969
1018
|
|
|
970
1019
|
**Source-PR Hint column**: when `ISSUE_LIKELY_MOOT[$NUM]` (populated by `phase-1-resolve.md`'s "Source-PR Triage Hint" step — see that file) is present for an issue, dereference it into a `Source-PR Hint` column. This column is **informational only** — it never changes a row's `Status` (Ready/Blocked), never removes a row from the table, and never causes a row to be excluded from dispatch. It exists purely so the operator can see, before confirming the plan, which `staging-review`/`review-finding` issues cite a source PR that closed without merging — a reason to look closer during that issue's own investigation phase, not a reason to skip it (see `phase-1-resolve.md`'s counterexamples: #2339/#2342 vs. #2346/#2261 — the same signal was right in one case and would have been wrong as a verdict in the other). <!-- Added: forge#2351 -->
|
|
971
1020
|
|
|
972
|
-
**File Source column** <!-- Added: forge#2436 -->: dereference `FILE_SOURCE[$NUM]` (populated by Step 3C Layer 1, above) into a `File Source` column, rendered as `affected-files-section` / `body-fallback` / `none` / `—` (when the issue has no predecessors and no DAG edges depend on its file list at all). Same informational-only contract as `Source-PR Hint` — never changes `Status`, never removes a row. It exists so the operator can see, before confirming the plan, which edges rest on a pre-investigation `body-fallback` guess (lower confidence — see Layer 5's downgrade-eligibility carve-out) versus a post-investigation `affected-files-section` extraction.
|
|
1021
|
+
**File Source column** <!-- Added: forge#2436 --> <!-- Enum extended: forge#2848 -->: dereference `FILE_SOURCE[$NUM]` (populated by Step 3C Layer 1, above) into a `File Source` column, rendered as `contract-deliverables` / `affected-files-section` / `body-fallback` / `none` / `—` (when the issue has no predecessors and no DAG edges depend on its file list at all). Same informational-only contract as `Source-PR Hint` — never changes `Status`, never removes a row. It exists so the operator can see, before confirming the plan, which edges rest on a pre-investigation `body-fallback` guess (lower confidence — see Layer 5's downgrade-eligibility carve-out) versus a post-investigation `affected-files-section` extraction or a `contract-deliverables` table (highest confidence — states intent to change).
|
|
1022
|
+
|
|
1023
|
+
**Low-confidence serial-chain warning** <!-- Added: forge#2848 -->: print this **above** the plan table, once, when **both** conditions hold:
|
|
1024
|
+
|
|
1025
|
+
1. The critical path (already computed for the `**Critical path**` line below) is **≥ 4 steps** deep, and
|
|
1026
|
+
2. **Every** edge along that path has `FILE_SOURCE` = `body-fallback` for its predecessor.
|
|
1027
|
+
|
|
1028
|
+
```
|
|
1029
|
+
⚠️ LOW-CONFIDENCE SERIAL CHAIN — {DEPTH} steps, every edge from `body-fallback` provenance
|
|
1030
|
+
|
|
1031
|
+
#{A} → #{B} → #{C} → #{D} → #{E}
|
|
1032
|
+
|
|
1033
|
+
No issue on this chain has been investigated or contracted; every edge rests on file
|
|
1034
|
+
paths scraped from raw issue bodies, which name files as context as often as they name
|
|
1035
|
+
files that will actually change. A chain this deep built entirely on that source is the
|
|
1036
|
+
single most likely place for this plan to over-serialize.
|
|
1037
|
+
|
|
1038
|
+
This is informational — the plan below is unchanged and every issue still dispatches.
|
|
1039
|
+
Consider before approving:
|
|
1040
|
+
• Did one audit/sweep file these issues? If so, see Layer 4's cohort-confidence
|
|
1041
|
+
guidance — a shared file citation across a cohort is one claim repeated N times,
|
|
1042
|
+
not N independent corroborations.
|
|
1043
|
+
• Investigating the chain head first upgrades its provenance and lets the DONE-arm
|
|
1044
|
+
re-verification in phase-4-execution.md unwind the rest automatically.
|
|
1045
|
+
```
|
|
1046
|
+
|
|
1047
|
+
Both conditions are required. Depth alone over-fires on legitimately serialized chains (a DATABASE migration chain is deep *by design*), and `body-fallback` alone is unremarkable — most pre-investigation issues have it. It is the **conjunction** that identifies the specific failure mode from forge#2848: a deep chain where nothing along it has ever been checked against reality. A single non-`body-fallback` edge on the path suppresses the warning, because that edge is corroborated and the chain is no longer uniformly guesswork.
|
|
1048
|
+
|
|
1049
|
+
This warning is **strictly informational**, on the same contract as the `File Source` and `Source-PR Hint` columns: it never changes a row's `Status`, never removes a row, never alters dispatch order, and never blocks the plan. It is output *in addition to* the table, never in place of any part of it.
|
|
973
1050
|
|
|
974
1051
|
```
|
|
975
1052
|
## Orchestration Plan
|
|
@@ -1013,7 +1090,7 @@ done
|
|
|
1013
1090
|
| #{G} | — | INFRA | {score} | ${cost} | affected-files-section | likely-moot (PR #{N} closed unmerged — verify first) | Ready (dispatches 5th by score) |
|
|
1014
1091
|
|
|
1015
1092
|
**[ε]** = no cost prior; eligible for exploration reserve (10% of budget guaranteed for these)
|
|
1016
|
-
**File Source** = `${FILE_SOURCE[$NUM]:-none}` (Step 3C Layer 1)
|
|
1093
|
+
**File Source** = `${FILE_SOURCE[$NUM]:-none}` (Step 3C Layer 1) — one of `contract-deliverables` (highest confidence) / `affected-files-section` / `body-fallback` / `none`. Never affects `Status` or row inclusion — see Layer 1's provenance tracking and Layer 5's downgrade-eligibility carve-out (forge#2436, forge#2848) for how each value is actually consumed, and the low-confidence serial-chain warning above for when a run of `body-fallback` edges is worth an operator's attention.
|
|
1017
1094
|
**Source-PR Hint** = `${ISSUE_LIKELY_MOOT[$NUM]:-unknown}` rendered as `—` when `unknown`/absent, or `likely-moot (PR #{ISSUE_SOURCE_PR[$NUM]} closed unmerged — verify first)` when `yes`. Never affects `Status` or row inclusion — see `phase-1-resolve.md`'s "Source-PR Triage Hint" step for how this is computed and why it stays a hint.
|
|
1018
1095
|
|
|
1019
1096
|
**Score** = value / estimated_cost (value = priority_weight × danger_zone_weight; higher = dispatches first within the ready-set)
|
|
@@ -1043,9 +1120,9 @@ The following issues form a circular dependency chain and **cannot be dispatched
|
|
|
1043
1120
|
Proceed? (yes / adjust / pick specific issues)
|
|
1044
1121
|
```
|
|
1045
1122
|
|
|
1046
|
-
**Wait for user confirmation before spawning agents
|
|
1123
|
+
**Wait for user confirmation before spawning agents unless the caller explicitly passed `--auto` or `--confirm`.** Those flags are machine-readable authorization for headless callers such as `/autopilot`; they do not change the plan or bypass any eligibility, dependency, or safety checks. Once the checkpoint is authorized, agents launch and run autonomously.
|
|
1047
1124
|
|
|
1048
|
-
**After confirmation**: If investigations exist, execute Phase 2B-E first.
|
|
1125
|
+
**After confirmation**: If investigations exist, execute Phase 2B-E first. Interactive callers then re-present the expanded plan (with newly spawned issues added to the dependency graph) for a quick confirmation before launching implementation. An explicit `--auto`/`--confirm` caller proceeds through that second checkpoint without prompting, while still rebuilding the graph.
|
|
1049
1126
|
|
|
1050
1127
|
---
|
|
1051
1128
|
|
|
@@ -1090,7 +1167,7 @@ scripts/worktree-lifecycle.sh cleanup <issue-number>
|
|
|
1090
1167
|
|
|
1091
1168
|
**Concurrency cap** (`forge.yaml → orchestration.max_concurrent`): <!-- Updated: forge#1912 -->
|
|
1092
1169
|
- Default: **12** — the dispatch loop holds at most 12 in-flight workers unless overridden. This is an enforced default, not opt-in: earlier revisions defaulted to uncapped, which let a large ready set (e.g. 40+ issues) dispatch in one burst and saturate the Anthropic API rate limit.
|
|
1093
|
-
- When `max_concurrent: N` is set, the dispatch loop holds at most N in-flight workers instead of the default 12. Newly ready issues queue and start as running workers complete.
|
|
1170
|
+
- When `max_concurrent: N` is set, the dispatch loop holds at most N in-flight workers instead of the default 12. Newly ready issues queue and start as running workers complete. It is a top-level worker cap only: each worker normally consumes roughly **8 total subagent spawns** across `/work-on`, build, quality-gate, and review. Set `N <= session_subagent_budget / 8`; for a 200-spawn session budget, use 25 or fewer.
|
|
1094
1171
|
- Prevents wave-triggered rate-limit storms on large batches (e.g., 40-issue milestone dispatches).
|
|
1095
1172
|
- See `phase-4-execution.md` Step 4A-pre.0.2 for the concrete initialization and headroom-gated dispatch logic that enforces this cap on both the engine-first and Agent-spawn-fallback dispatch paths.
|
|
1096
1173
|
|
|
@@ -1112,6 +1189,7 @@ fi
|
|
|
1112
1189
|
- `FORGE_RATE_LIMIT_FLOOR` defaults to 200 remaining requests. Override in `forge.yaml → orchestration.rate_limit_floor`.
|
|
1113
1190
|
- Already-in-flight workers are **never interrupted** by the backpressure gate — only new dispatches pause.
|
|
1114
1191
|
- The gate is re-checked after each worker completion, not on a timer, so dispatch resumes immediately once the floor is cleared.
|
|
1192
|
+
- This core-quota check cannot detect GitHub secondary content-creation throttles. A `403` response containing `secondary rate limit` is a separate first-class signal: stop new dispatch, retain the queues, and require an operator resume rather than retrying or polling. In-flight workers must stop their GitHub write/create retry loops and report the error (see Phase 4 Step 4A-pre.0.3).
|
|
1115
1193
|
|
|
1116
1194
|
**Configuration reference** (`forge.yaml`):
|
|
1117
1195
|
```yaml
|
|
@@ -1215,6 +1293,33 @@ For the same reason, this reconstruction also MUST call `verify_file_overlap_edg
|
|
|
1215
1293
|
# Reconstruct dispatch state from GitHub after compaction / wake
|
|
1216
1294
|
# Run this block at the top of every resumed Phase 4 loop iteration.
|
|
1217
1295
|
|
|
1296
|
+
# Keep this byte-identical to phase-4-execution.md Step 4A. A wake can start in a
|
|
1297
|
+
# fresh context, so it cannot rely on the live dispatch helper still being defined.
|
|
1298
|
+
read_active_claims() {
|
|
1299
|
+
local COORD_NUM="$1"
|
|
1300
|
+
local CLAIMS HOLDER TERMINAL
|
|
1301
|
+
CLAIMS=$(gh api --paginate --slurp "repos/{GH_REPO}/issues/${COORD_NUM}/comments" 2>/dev/null \
|
|
1302
|
+
| jq -c '
|
|
1303
|
+
flatten as $comments |
|
|
1304
|
+
[$comments[]
|
|
1305
|
+
| select(.body | contains("<!-- FORGE:CLAIM -->"))
|
|
1306
|
+
| . as $claim
|
|
1307
|
+
| ($claim.body | capture("\\*\\*Holder\\*\\*: #(?<holder>[0-9]+)").holder) as $holder
|
|
1308
|
+
| select([$comments[]
|
|
1309
|
+
| select(.body | contains("<!-- FORGE:CLAIM_RELEASED -->"))
|
|
1310
|
+
| select((.body | capture("\\*\\*Holder\\*\\*: #(?<holder>[0-9]+)").holder) == $holder)
|
|
1311
|
+
| select(.created_at > $claim.created_at)] | length == 0)
|
|
1312
|
+
| {holder: $holder,
|
|
1313
|
+
files: ($claim.body | capture("\\*\\*Files\\*\\*: (?<files>[\\s\\S]*?)(?:\\n\\*\\*Interfaces\\*\\*:|$)").files)}]') || return 1
|
|
1314
|
+
|
|
1315
|
+
for HOLDER in $(echo "$CLAIMS" | jq -r '.[].holder'); do
|
|
1316
|
+
TERMINAL=$(gh issue view "$HOLDER" -R {GH_REPO} --json labels --jq \
|
|
1317
|
+
'[.labels[].name | select(. == "workflow:merged" or . == "workflow:invalid" or . == "workflow:awaiting-merge" or . == "needs-human")] | length > 0' 2>/dev/null)
|
|
1318
|
+
[ "$TERMINAL" = "true" ] && CLAIMS=$(echo "$CLAIMS" | jq --arg holder "$HOLDER" '[.[] | select(.holder != $holder)]')
|
|
1319
|
+
done
|
|
1320
|
+
echo "$CLAIMS"
|
|
1321
|
+
}
|
|
1322
|
+
|
|
1218
1323
|
# 0. Lease check (MANDATORY, forge#2627) — run BEFORE any of the reconstruction below.
|
|
1219
1324
|
# A wake/resume is either this same orchestrator continuing its own batch or, less
|
|
1220
1325
|
# commonly, a fresh invocation resuming someone else's interrupted batch. Either way,
|
|
@@ -1282,6 +1387,7 @@ fi
|
|
|
1282
1387
|
# (same classify_predecessor_state() function defined in phase-4-execution.md Step 4B —
|
|
1283
1388
|
# re-declare it here if this block runs in a fresh context that hasn't sourced Step 4B yet).
|
|
1284
1389
|
declare -A ISSUE_CLASS
|
|
1390
|
+
declare -A ISSUE_FILES
|
|
1285
1391
|
DONE_ISSUES=()
|
|
1286
1392
|
GATED_ISSUES=()
|
|
1287
1393
|
FAILED_ISSUES=()
|
|
@@ -1298,6 +1404,22 @@ for NUM in {all_issue_numbers_in_batch}; do
|
|
|
1298
1404
|
esac
|
|
1299
1405
|
done
|
|
1300
1406
|
|
|
1407
|
+
# 1.5. Rebuild the durable file-claim map. The claims board, not the orchestrator's
|
|
1408
|
+
# in-context memory, is authoritative before every later dispatch. Rebuild the target
|
|
1409
|
+
# declarations too because ISSUE_FILES is lost across compaction.
|
|
1410
|
+
if [ -n "${FORGE_COORD_ISSUE:-}" ] && [ -n "${COORD_ISSUE_NUMBER:-}" ]; then
|
|
1411
|
+
ACTIVE_CLAIMS=$(read_active_claims "$COORD_ISSUE_NUMBER")
|
|
1412
|
+
declare -A ACTIVE_CLAIM_FILES
|
|
1413
|
+
for HOLDER in $(echo "$ACTIVE_CLAIMS" | jq -r '.[].holder'); do
|
|
1414
|
+
ACTIVE_CLAIM_FILES["$HOLDER"]=$(echo "$ACTIVE_CLAIMS" | jq -r --arg holder "$HOLDER" '.[] | select(.holder == $holder) | .files')
|
|
1415
|
+
done
|
|
1416
|
+
|
|
1417
|
+
for NUM in {all_issue_numbers_in_batch}; do
|
|
1418
|
+
EXTRACT_OUT=$(bash "$AFFECTED_FILES_SCRIPT" "$NUM" -R "{GH_REPO}")
|
|
1419
|
+
ISSUE_FILES[$NUM]=$(echo "$EXTRACT_OUT" | tail -n +2)
|
|
1420
|
+
done
|
|
1421
|
+
fi
|
|
1422
|
+
|
|
1301
1423
|
# 2. Re-derive the ready set: any non-terminal issue whose predecessors are ALL classified DONE
|
|
1302
1424
|
# (or GATED-but-edge-dropped — see the re-verification gate below).
|
|
1303
1425
|
# A GATED predecessor blocks dispatch but does NOT fail the dependent — see step 2.5 below.
|
|
@@ -1318,6 +1440,12 @@ for NUM in "${ACTIVE_ISSUES[@]}"; do
|
|
|
1318
1440
|
GATING_PRED=""
|
|
1319
1441
|
for PRED in {predecessors_of_NUM}; do
|
|
1320
1442
|
case "${ISSUE_CLASS[$PRED]:-IN_PROGRESS}" in
|
|
1443
|
+
# DONE needs no re-verification call HERE because it already satisfies the edge, and
|
|
1444
|
+
# this reconstruction builds no SAME_FILE_BRIEF. Unlike the live DONE arm, this path
|
|
1445
|
+
# does not retain EDGE_KIND/EDGE_FILES or re-run Layer 1 extraction; consequently it
|
|
1446
|
+
# does not re-derive stale DONE-path edges after wake. That live-only re-derivation is
|
|
1447
|
+
# in phase-4-execution.md lines 1208-1268. Adding the helper call here would therefore
|
|
1448
|
+
# add API cost without affecting wake-time dispatch readiness.
|
|
1321
1449
|
DONE) ;;
|
|
1322
1450
|
GATED|FAILED)
|
|
1323
1451
|
EDGE_VERDICT=$(verify_file_overlap_edge "$PRED" "$NUM")
|
|
@@ -1394,6 +1522,6 @@ done
|
|
|
1394
1522
|
# predecessor's edge dropped (forge#1904); that case is legitimately ready, not a re-add.
|
|
1395
1523
|
```
|
|
1396
1524
|
|
|
1397
|
-
**Why this keeps context small**: Each `Agent()` call returns an agent ID stored only in `AGENT_ISSUE_MAP`,
|
|
1525
|
+
**Why this keeps context small**: Each `Agent()` call returns an agent ID stored only in `AGENT_ISSUE_MAP`, and engine/OpenCode dispatches use equivalent ephemeral `ENGINE_DISPATCH_MAP` and `OPENCODE_DISPATCH_MAP` caches. After compaction, those maps are gone — but the DAG state, including `blocked-on-human-merge` tracking (a durable `FORGE:BLOCKED_ON_HUMAN_MERGE` comment plus label, not an in-context variable) and OpenCode child correlation (`FORGE:DISPATCH`), is fully on GitHub. The reconstruction above re-derives the ready set, the gated set, and the blocked-on-human-merge set from labels and comments alone, so the orchestrator context never needs to hold cumulative dispatch history.
|
|
1398
1526
|
|
|
1399
1527
|
---
|