forgedock 1.4.0 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/engine/runlog.mjs +11 -1
- package/bin/engine-cli.mjs +6 -0
- package/bin/engine.mjs +194 -59
- package/bin/forgedock.mjs +65 -4
- package/bin/hooks/pre-tool-use.mjs +29 -7
- package/bin/journey.mjs +578 -42
- package/bin/runner.mjs +159 -2
- package/bin/watch-utils.mjs +4 -1
- package/commands/cleanup.md +27 -2
- package/commands/issue.md +14 -4
- package/commands/orchestrate/config.md +3 -1
- package/commands/orchestrate/phase-1-resolve.md +125 -14
- package/commands/orchestrate/phase-3-dependency.md +254 -40
- package/commands/orchestrate/phase-4-execution.md +402 -39
- package/commands/orchestrate/phase-5-cleanup.md +2 -0
- package/commands/review-pr-staging.md +51 -8
- package/commands/review-pr.md +97 -32
- package/commands/work-on/remediate.md +53 -4
- package/package.json +1 -1
- package/packages/protocol/src/validate.js +24 -1
- package/scripts/check-phase-registry-drift.mjs +23 -1
- package/scripts/check-spec-markers.sh +2 -0
- package/scripts/classify-lane.sh +29 -3
- package/scripts/derive-finding-milestone.sh +184 -0
- package/scripts/doctor-pipeline-state.sh +1 -0
- package/scripts/extract-affected-files.sh +232 -0
- package/scripts/extract-affected-files.test.sh +331 -0
- package/scripts/forge-run.sh +4 -0
- package/scripts/issue-dedup.sh +69 -8
- package/scripts/issue-dedup.test.sh +142 -0
- package/scripts/severity-to-priority.sh +81 -0
|
@@ -10,6 +10,94 @@ install: core
|
|
|
10
10
|
|
|
11
11
|
**Entry requires `phase-3-dependency.md`'s Step 3D.6 completion gate to have passed.** <!-- Added: forge#1913 --> Every step below (budget init, dispatch, dependency resolution) reads `ISSUES[]`, `PREDECESSORS[]`, `ISSUE_DOMAIN[]`, `ISSUE_SCORE[]`, `ISSUE_COST_ESTIMATE[]`, and `ISSUE_HAS_PRIOR[]` as authoritative Phase 3 output — none of these are re-derived here. If Phase 4 is being entered without having actually run Phase 3's Steps 3A–3E.5 in this session (e.g. a fresh session resuming mid-batch), reconstruct from GitHub via the "Orchestrator state reconstruction on wake / after compaction" procedure in `phase-3-dependency.md` rather than assuming these variables are already populated.
|
|
12
12
|
|
|
13
|
+
### Step 4A-pre.-1: Lease gate (MANDATORY, before any dispatch) <!-- Added: forge#2627 -->
|
|
14
|
+
|
|
15
|
+
**WHY THIS EXISTS**: Nothing today checks whether another live `/orchestrate` instance already holds this batch before dispatching (see issue #2627). Two concurrent loops (a stale survivor plus a restart, or two independent invocations) then both re-derive a ready set from GitHub and both dispatch — duplicate/backlog dispatch, and a restarted instance's own bookkeeping can diverge from what the first loop already did. This step closes that gap at the single entry point every dispatch group (initial ready set + every subsequent newly-unblocked batch) passes through, by extending the Step 3D.1 coordination issue into a single-instance lease (see `phase-3-dependency.md` Step 3D.2 for `check_orchestrator_lease()` and the `FORGE:LEASE`/`FORGE:LEASE_RELEASED` comment format).
|
|
16
|
+
|
|
17
|
+
**Run this once, before the first dispatch of any group in this session** — not per-chunk, per-issue, or per-newly-ready-batch.
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
# --- Lease gate (uses check_orchestrator_lease() from phase-3-dependency.md Step 3D.2;
|
|
21
|
+
# re-declare it here if this context hasn't sourced that file) ---
|
|
22
|
+
if [ -n "${FORGE_COORD_ISSUE:-}" ] && [ -n "${COORD_ISSUE_NUMBER:-}" ] && [ -n "${BATCH_ID:-}" ]; then
|
|
23
|
+
LEASE_STATE=$(check_orchestrator_lease "$COORD_ISSUE_NUMBER" "$BATCH_ID")
|
|
24
|
+
case "$LEASE_STATE" in
|
|
25
|
+
held:*)
|
|
26
|
+
HELD_BY="${LEASE_STATE#held:}"
|
|
27
|
+
echo "REFUSING TO DISPATCH: coordination issue #${COORD_ISSUE_NUMBER} shows an unexpired lease held by batch ${HELD_BY}, not this session's batch ${BATCH_ID}."
|
|
28
|
+
echo "Another live /orchestrate instance appears to be dispatching this batch already. Stop this invocation, or wait for the other lease to go stale."
|
|
29
|
+
exit 1
|
|
30
|
+
;;
|
|
31
|
+
free|self)
|
|
32
|
+
HOSTNAME_ID=$(hostname 2>/dev/null || echo "unknown-host")
|
|
33
|
+
# GOVERNOR-exempt: intentional coordination side-effect (best-effort lease/board/finding post), DRY_RUN-safe — reviewed & accepted for the check-command-side-effects gate. Flagged only by the staging->main full-diff; passes on every feature PR. forge#2627
|
|
34
|
+
gh issue comment "$COORD_ISSUE_NUMBER" -R {GH_REPO} --body "<!-- FORGE:LEASE -->
|
|
35
|
+
**Holder Batch ID**: ${BATCH_ID}
|
|
36
|
+
**Holder**: ${HOSTNAME_ID} (pid ${$})
|
|
37
|
+
**Acquired/refreshed**: $(date -u +%Y-%m-%dT%H:%M:%SZ)
|
|
38
|
+
**TTL**: ${LEASE_TTL_SECONDS:-900}s (refreshed once per dispatch chunk in Step 4A)" 2>/dev/null || \
|
|
39
|
+
echo "WARNING: failed to post FORGE:LEASE — continuing without a lease record (best-effort primitive)"
|
|
40
|
+
;;
|
|
41
|
+
*)
|
|
42
|
+
# Defensive default (MANDATORY — do not remove): see the matching comment at
|
|
43
|
+
# phase-3-dependency.md Step 3D.2's acquisition case block. An unexpected LEASE_STATE
|
|
44
|
+
# here must warn loudly, not silently fall through and let dispatch proceed as if the
|
|
45
|
+
# lease gate had passed.
|
|
46
|
+
echo "WARNING: check_orchestrator_lease() returned unexpected value '${LEASE_STATE}' — lease gate could not be evaluated. Proceeding without a confirmed lease; investigate rather than ignore." >&2
|
|
47
|
+
;;
|
|
48
|
+
esac
|
|
49
|
+
else
|
|
50
|
+
echo "INFO: no coordination issue / BATCH_ID available — lease enforcement disabled for this batch."
|
|
51
|
+
fi
|
|
52
|
+
# --- End lease gate ---
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
**Lease release**: `LEASE_RELEASED_THIS_SESSION` (declared just below, alongside `release_orchestrator_lease()`) guards against double-posting `FORGE:LEASE_RELEASED`. The release call is invoked from both the **Step 4A-pre.-0.5 "Stopping the orchestrator"** procedure (interrupted path) and the **Termination condition** at the end of Step 4B (normal clean/paused drain) — both exit paths route through the same idempotent function so the lease is never left dangling either way.
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
LEASE_RELEASED_THIS_SESSION=false
|
|
59
|
+
|
|
60
|
+
release_orchestrator_lease() {
|
|
61
|
+
# Idempotent — safe to call from both the interrupted-stop procedure and normal
|
|
62
|
+
# end-of-batch completion without double-posting FORGE:LEASE_RELEASED.
|
|
63
|
+
if [ "$LEASE_RELEASED_THIS_SESSION" = "true" ]; then
|
|
64
|
+
return
|
|
65
|
+
fi
|
|
66
|
+
if [ -n "${FORGE_COORD_ISSUE:-}" ] && [ -n "${COORD_ISSUE_NUMBER:-}" ]; then
|
|
67
|
+
# GOVERNOR-exempt: intentional coordination side-effect (best-effort lease/board/finding post), DRY_RUN-safe — reviewed & accepted for the check-command-side-effects gate. Flagged only by the staging->main full-diff; passes on every feature PR. forge#2627
|
|
68
|
+
gh issue comment "$COORD_ISSUE_NUMBER" -R {GH_REPO} --body "<!-- FORGE:LEASE_RELEASED -->
|
|
69
|
+
**Holder Batch ID**: ${BATCH_ID:-unknown}
|
|
70
|
+
**Released**: $(date -u +%Y-%m-%dT%H:%M:%SZ)" 2>/dev/null || true
|
|
71
|
+
fi
|
|
72
|
+
LEASE_RELEASED_THIS_SESSION=true
|
|
73
|
+
}
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### Step 4A-pre.-0.5: Backgrounded engine-child tracking + interruption handling ("Stopping the orchestrator") (MANDATORY) <!-- Added: forge#2627 -->
|
|
77
|
+
|
|
78
|
+
**WHY THIS EXISTS**: Step 4A's engine-first dispatch (fixed by forge#2466) already dispatches each `forgedock run-issue` invocation as its own `Bash(run_in_background=true, ...)` call and records the returned task id in `ENGINE_DISPATCH_MAP[{NUM}]` — this is a harness-managed background task, not a raw shell `&` job. **Because it is harness-managed, it cannot be reaped by a shell-level `trap`/`kill $PID`**: a bash `trap` only runs inside that one Bash tool invocation's own process and has no way to call back into the harness's own task-management surface (`TaskStop`). Reaping these tasks on interrupt is therefore something the orchestrating agent itself must do — as an explicit step in its own routing loop — not something a background shell script can do on its own behalf.
|
|
79
|
+
|
|
80
|
+
**Contract — the orchestrating agent (not a shell trap) is responsible for reaping**: Whenever the interactive `/orchestrate` session is being stopped (the operator sends an interrupt, or the harness delivers a stop signal to the top-level orchestrator agent) **before** all dispatched issues have reached a terminal `workflow:*` label, the orchestrator MUST, as part of handling that stop — not silently exit —:
|
|
81
|
+
|
|
82
|
+
1. Enumerate every task id still tracked in `ENGINE_DISPATCH_MAP` whose issue has not yet reached a terminal `workflow:*` label (per Step 4B's `classify_predecessor_state()`).
|
|
83
|
+
2. Call `TaskStop(task_id)` (the harness tool) for each one, in the same turn, before ending the session.
|
|
84
|
+
3. Call `release_orchestrator_lease()` (Step 4A-pre.-1) so a future resume or a different operator is not blocked by a stale lease from this now-stopped session.
|
|
85
|
+
4. Report which issues had in-flight dispatch stopped mid-pipeline (their `workflow:*` label will reflect whatever phase they reached — a future `/work-on {NUMBER}` or orchestrator resume picks them back up from GitHub state, per the Universal Phase Dispatcher in `commands/work-on.md`).
|
|
86
|
+
|
|
87
|
+
**This is a documentation/behavioral contract, not a bash block**: unlike the lease gate above, there is no shell construct that reliably intercepts "the orchestrator's own session is being stopped" — that is a harness-level event the orchestrating agent must handle in its own turn when it observes an interrupt, using its own tools (`TaskStop`), exactly the way `commands/orchestrate/phase-5-cleanup.md` already documents cleanup as an agent-driven procedure rather than a background script.
|
|
88
|
+
|
|
89
|
+
**Known limitation — Windows Git Bash (documented, not silently overclaimed)**: Even a harness-issued `TaskStop` against the tracked background task is not guaranteed to terminate the underlying `forgedock run-issue` Win32 process tree on Windows Git Bash — MSYS-launched child processes are not always attached to the parent's job object in a way that a stop signal reliably cascades through. `TaskStop` is the correct in-spec, harness-native mitigation and closes the common case (an operator deliberately stopping a live, still-running orchestrator session). It does **not** guarantee full termination in every case — per the issue's own report, an operator may still need to fall back to a manual, command-line-matched kill as a belt-and-suspenders measure:
|
|
90
|
+
|
|
91
|
+
```powershell
|
|
92
|
+
# Windows fallback — only if a forgedock run-issue process is confirmed still running
|
|
93
|
+
# after TaskStop (e.g. `docker ps` / dispatch-state still show activity post-stop):
|
|
94
|
+
Get-CimInstance Win32_Process | Where-Object { $_.CommandLine -match 'forgedock run-issue' } | ForEach-Object {
|
|
95
|
+
Stop-Process -Id $_.ProcessId -Force
|
|
96
|
+
}
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Do not claim `TaskStop` eliminates every zombie-process scenario on Windows; it eliminates the scenario this issue's fix is scoped to (an operator or harness cleanly stopping a live orchestrator session that is still tracking its own dispatched tasks) — the PowerShell fallback above remains documented for the narrower case where the underlying process tree survives regardless.
|
|
100
|
+
|
|
13
101
|
### Step 4A-pre.0: Budget initialization (MANDATORY when --budget is set) <!-- Added: forge#1743 -->
|
|
14
102
|
|
|
15
103
|
Initialize budget tracking state before the first dispatch. Read `--budget N` from the orchestrator's argument list (passed from the top-level `/orchestrate` invocation). When `--budget` is not set, `BUDGET_LIMIT` is `Infinity` (uncapped — current default behavior preserved).
|
|
@@ -276,6 +364,20 @@ Before building agent prompts, run `classify-lane.sh` for every issue in the cur
|
|
|
276
364
|
declare -A ISSUE_LANE
|
|
277
365
|
declare -A ISSUE_PR_BASE
|
|
278
366
|
|
|
367
|
+
# Batch-start T0 (forge#2628) — reuse the value phase-1-resolve.md captured at the very
|
|
368
|
+
# start of Phase 1 and persisted per "Predicate Persistence." Step 4C's run-spawned cascade
|
|
369
|
+
# time filter (Method 2 below) anchors to this, NOT to a rolling "N hours ago" window — a
|
|
370
|
+
# rolling window can still admit pre-existing backlog issues that happen to have been
|
|
371
|
+
# created recently for unrelated reasons. Degraded fallback only: if BATCH_T0 is not in
|
|
372
|
+
# context (e.g. a session resumed mid-run after compaction and Phase 1's capture was lost),
|
|
373
|
+
# capture one here — this is later than true batch start and will admit a narrower window
|
|
374
|
+
# than the real run, never wider, so it fails safe.
|
|
375
|
+
if [ -z "${BATCH_T0:-}" ]; then
|
|
376
|
+
echo "WARNING: BATCH_T0 not found in context — Phase 1 should have captured it. Falling back to capturing it now (narrower window than the true batch start, but never wider)."
|
|
377
|
+
BATCH_T0=$(date -u +%Y-%m-%dT%H:%M:%SZ)
|
|
378
|
+
fi
|
|
379
|
+
echo "Step 4A.pre: using BATCH_T0=${BATCH_T0} for Step 4C's run-spawned cascade time filter"
|
|
380
|
+
|
|
279
381
|
# Batch-level accumulators for review-finding cascade control (Step 4C) and
|
|
280
382
|
# Completion Sweep (Step 4F). Declared here so they persist across ALL agent
|
|
281
383
|
# completions — Step 4C runs per-agent and must NOT re-initialize these.
|
|
@@ -283,6 +385,12 @@ DEFERRED_FINDINGS=()
|
|
|
283
385
|
QUEUED_FINDINGS=()
|
|
284
386
|
declare -A DEFERRED_REASONS
|
|
285
387
|
declare -A AGENT_ISSUE_MAP
|
|
388
|
+
# Engine-first dispatch equivalent of AGENT_ISSUE_MAP (fixed forge#2466): keyed by issue
|
|
389
|
+
# number, holds the task id returned by each backgrounded `Bash(run_in_background=true,
|
|
390
|
+
# command="forgedock run-issue ...")` call from Step 4A's engine-first path. Step 4B reads
|
|
391
|
+
# this map to identify which issue a background-Bash completion notification belongs to,
|
|
392
|
+
# the same role AGENT_ISSUE_MAP plays for Agent-tool `agent_completed` notifications.
|
|
393
|
+
declare -A ENGINE_DISPATCH_MAP
|
|
286
394
|
|
|
287
395
|
# Same-file current-state brief forwarding (forge#1860). Populated by the core streaming
|
|
288
396
|
# dispatch loop below (Step 4B) whenever a Layer 1/2/3 structural predecessor edge (see
|
|
@@ -413,6 +521,8 @@ Use `${ISSUE_LANE[$NUM]}` and `${ISSUE_PR_BASE[$NUM]}` to populate `{LANE}` and
|
|
|
413
521
|
|
|
414
522
|
**Engine-first dispatch (default)**: When `forgedock` is in PATH, dispatch each ready issue via the durable engine rather than spawning prose Agent sub-agents. The engine's phase table enforces gate semantics in code — its fail-closed review gate and deterministic phase ordering are not subject to LLM interpretation.
|
|
415
523
|
|
|
524
|
+
**CRITICAL — never background via shell `&`/`wait`** (fixed forge#2466): A single `forgedock run-issue` invocation drives an issue through investigate → build → review → close and routinely runs 30+ minutes. Backgrounding the process at the *shell* level (`cmd &` … `wait`) does not escape the Bash tool's own per-invocation ceiling — `wait` is itself the foreground command the tool watches, and it blocks for the combined duration of every process in the chunk. This made engine-first dispatch effectively dead code: any chunk running longer than the ceiling was always killed, so every run silently fell through to the Agent-spawn fallback below. The fix: dispatch each `forgedock run-issue` invocation as its **own `Bash` tool call with `run_in_background=true`** — the harness's native "start it, don't wait, notify me on completion" primitive — never with shell-level `&`/`wait`. This is exactly the same async model the Agent-spawn-fallback path already uses (and Step 4B's notification-driven completion loop already expects), so one monitoring loop now covers both dispatch styles.
|
|
525
|
+
|
|
416
526
|
```bash
|
|
417
527
|
# Engine-first dispatch: check CLI availability, then dispatch ready issues in score order
|
|
418
528
|
# Uses SORTED_READY_SET[] from Step 3E.5 (descending value/cost) and budget gate from Step 4A-pre.0
|
|
@@ -428,30 +538,61 @@ if [ "$FORGEDOCK_AVAILABLE" = "true" ]; then
|
|
|
428
538
|
fi # else: added to DEFERRED_BUDGET_ISSUES[] by should_dispatch()
|
|
429
539
|
done
|
|
430
540
|
|
|
431
|
-
# Concurrency gate (forge#1912
|
|
432
|
-
#
|
|
433
|
-
#
|
|
434
|
-
#
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
541
|
+
# Concurrency gate (forge#1912, mechanism fixed forge#2466): compute this dispatch batch the
|
|
542
|
+
# SAME way the Agent-spawn-fallback path below does — via dispatch_headroom()/
|
|
543
|
+
# DEFERRED_CONCURRENCY_ISSUES (Step 4A-pre.0.2) — NOT a shell `&`/`wait` chunk loop.
|
|
544
|
+
#
|
|
545
|
+
# MUST merge previously-deferred issues back in before recomputing, exactly like the
|
|
546
|
+
# Agent-spawn path's own batch computation does (CANDIDATES=("${DEFERRED_CONCURRENCY_ISSUES[@]}"
|
|
547
|
+
# ...); DEFERRED_CONCURRENCY_ISSUES=()). Without this merge-and-reset, an issue that lands in
|
|
548
|
+
# DEFERRED_CONCURRENCY_ISSUES on THIS path has no way back into a future DISPATCH_QUEUE — Step
|
|
549
|
+
# 4B item 5 only re-runs Step 4A for issues that just became DAG-unblocked, not for issues that
|
|
550
|
+
# were already ready but held back purely by the concurrency cap. DEFERRED_CONCURRENCY_ISSUES
|
|
551
|
+
# would become a write-only sink on the engine-first path, silently starving deferred work —
|
|
552
|
+
# the same class of bug this PR exists to fix, just relocated.
|
|
553
|
+
HEADROOM=$(dispatch_headroom)
|
|
554
|
+
CANDIDATES=("${DEFERRED_CONCURRENCY_ISSUES[@]}" "${DISPATCH_QUEUE[@]}")
|
|
555
|
+
DEFERRED_CONCURRENCY_ISSUES=()
|
|
556
|
+
DISPATCH_NOW=()
|
|
557
|
+
for NUM in "${CANDIDATES[@]}"; do
|
|
558
|
+
if [ "${#DISPATCH_NOW[@]}" -lt "$HEADROOM" ]; then
|
|
559
|
+
DISPATCH_NOW+=("$NUM")
|
|
560
|
+
else
|
|
561
|
+
DEFERRED_CONCURRENCY_ISSUES+=("$NUM")
|
|
562
|
+
fi
|
|
563
|
+
done
|
|
564
|
+
|
|
565
|
+
if [ "${#DEFERRED_CONCURRENCY_ISSUES[@]}" -gt 0 ]; then
|
|
566
|
+
echo "CONCURRENCY DEFER: ${#DEFERRED_CONCURRENCY_ISSUES[@]} ready issue(s) held back — ${ACTIVE_DISPATCH_COUNT}/${MAX_CONCURRENT} already in flight. Will dispatch as slots free up: ${DEFERRED_CONCURRENCY_ISSUES[*]}"
|
|
567
|
+
fi
|
|
568
|
+
echo "Dispatching ${#DISPATCH_NOW[@]} issue(s) this message via forgedock run-issue (headroom was ${HEADROOM})"
|
|
569
|
+
|
|
570
|
+
for NUM in "${DISPATCH_NOW[@]}"; do
|
|
571
|
+
LANE="${ISSUE_LANE[$NUM]}"
|
|
572
|
+
PR_BASE="${ISSUE_PR_BASE[$NUM]}"
|
|
573
|
+
COST="${ISSUE_COST_ESTIMATE[$NUM]:-0.35}"
|
|
574
|
+
|
|
575
|
+
# Advance PROJECTED_SPEND before dispatching so subsequent iterations see the updated total
|
|
576
|
+
PROJECTED_SPEND=$(echo "scale=4; $PROJECTED_SPEND + $COST" | bc 2>/dev/null || echo "$PROJECTED_SPEND")
|
|
577
|
+
|
|
578
|
+
echo "Dispatching #$NUM via forgedock run-issue --lane $PR_BASE (score=${ISSUE_SCORE[$NUM]:-?} est_cost=\$${COST} projected_total=\$${PROJECTED_SPEND})"
|
|
453
579
|
done
|
|
454
|
-
|
|
580
|
+
|
|
581
|
+
# Lease heartbeat refresh (forge#2627) — once per dispatch chunk, so a long-running batch's
|
|
582
|
+
# lease never goes stale purely from elapsed wall-clock time while dispatch is still active.
|
|
583
|
+
if [ "${#DISPATCH_NOW[@]}" -gt 0 ] && [ -n "${FORGE_COORD_ISSUE:-}" ] && [ -n "${COORD_ISSUE_NUMBER:-}" ] && [ -n "${BATCH_ID:-}" ]; then
|
|
584
|
+
HOSTNAME_ID=$(hostname 2>/dev/null || echo "unknown-host")
|
|
585
|
+
# GOVERNOR-exempt: intentional coordination side-effect (best-effort lease/board/finding post), DRY_RUN-safe — reviewed & accepted for the check-command-side-effects gate. Flagged only by the staging->main full-diff; passes on every feature PR. forge#2627
|
|
586
|
+
gh issue comment "$COORD_ISSUE_NUMBER" -R {GH_REPO} --body "<!-- FORGE:LEASE -->
|
|
587
|
+
**Holder Batch ID**: ${BATCH_ID}
|
|
588
|
+
**Holder**: ${HOSTNAME_ID} (pid ${$})
|
|
589
|
+
**Acquired/refreshed**: $(date -u +%Y-%m-%dT%H:%M:%SZ)
|
|
590
|
+
**TTL**: ${LEASE_TTL_SECONDS:-900}s (refreshed once per dispatch chunk in Step 4A)" 2>/dev/null || true
|
|
591
|
+
fi
|
|
592
|
+
|
|
593
|
+
if [ "${#DISPATCH_NOW[@]}" -eq 0 ]; then
|
|
594
|
+
echo "Engine dispatch: no headroom this cycle — waiting for the next completion notification (Step 4B) before dispatching more."
|
|
595
|
+
fi
|
|
455
596
|
else
|
|
456
597
|
echo "INFO: Using agent dispatch mode (forgedock CLI not in PATH — run \`npm install -g forgedock\` for engine-mode dispatch)"
|
|
457
598
|
# Fall through to Agent-spawn template below. The SubagentStop hook (bin/hooks/interactive-engine.mjs)
|
|
@@ -459,6 +600,23 @@ else
|
|
|
459
600
|
fi
|
|
460
601
|
```
|
|
461
602
|
|
|
603
|
+
**Dispatch each issue in `DISPATCH_NOW` via its own backgrounded `Bash` call (MANDATORY when `FORGEDOCK_AVAILABLE=true`) — never shell `&`/`wait`.** Issue one `Bash(...)` call per issue in `DISPATCH_NOW`, all in the same message, so they run concurrently within the headroom already computed above:
|
|
604
|
+
|
|
605
|
+
```
|
|
606
|
+
Bash(command="forgedock run-issue {NUM} --lane {PR_BASE}", run_in_background=true, description="Engine-drive issue #{NUM}")
|
|
607
|
+
```
|
|
608
|
+
|
|
609
|
+
Capture the task id each call returns into `ENGINE_DISPATCH_MAP[{NUM}]` (declared alongside `AGENT_ISSUE_MAP` below — Step 4B's completion handler uses this map to identify which issue a backgrounded engine-mode `Bash` completion notification belongs to, the same role `AGENT_ISSUE_MAP` plays for `agent_completed` notifications):
|
|
610
|
+
|
|
611
|
+
```
|
|
612
|
+
# After the batch of Bash(run_in_background=true, ...) calls, capture each returned task id:
|
|
613
|
+
ENGINE_DISPATCH_MAP[{NUM}] = <task_id returned by Bash(run_in_background=true, ...)>
|
|
614
|
+
```
|
|
615
|
+
|
|
616
|
+
`ENGINE_DISPATCH_MAP` starts empty and accumulates one entry per dispatched issue, exactly like `AGENT_ISSUE_MAP` below. After the batch, increment `ACTIVE_DISPATCH_COUNT` by `${#DISPATCH_NOW[@]}` — identical accounting to the Agent-spawn path's own post-batch increment.
|
|
617
|
+
|
|
618
|
+
If `DISPATCH_NOW` is empty (headroom is 0), do not dispatch any issues this cycle — wait for the next completion notification, exactly like the Agent-spawn path.
|
|
619
|
+
|
|
462
620
|
**Agent-spawn path (fallback when forgedock CLI unavailable)**: When `FORGEDOCK_AVAILABLE=false`, spawn Agent sub-agents per issue using the template below. This preserves engine state via the SubagentStop hook even without the CLI.
|
|
463
621
|
|
|
464
622
|
**REMINDER: You MUST use the template below verbatim when on the Agent-spawn fallback path. Only fill in `{VARIABLES}`. Do NOT rewrite the agent prompt. Do NOT write custom implementation instructions. The agent MUST invoke `/work-on` via the Skill tool — this is the HARD RULE from the top of this file.**
|
|
@@ -689,9 +847,9 @@ If `DISPATCH_NOW` is empty (headroom is 0), do not spawn any agents this cycle
|
|
|
689
847
|
|
|
690
848
|
### Step 4B: Monitor completions and dispatch newly ready issues
|
|
691
849
|
|
|
692
|
-
You will be automatically notified when each background agent completes. **Do NOT use `sleep` loops to poll for completion.** Instead, wait for the automatic notification. When
|
|
850
|
+
You will be automatically notified when each background agent completes — or, for an engine-first dispatch (Step 4A, `FORGEDOCK_AVAILABLE=true`), when a backgrounded `Bash(run_in_background=true, command="forgedock run-issue ...")` call completes. Both are the same underlying "background task finished" notification the harness delivers; treat them identically for the purposes of this step. **Do NOT use `sleep` loops to poll for completion.** Instead, wait for the automatic notification. When one arrives, look up which issue it belongs to — `AGENT_ISSUE_MAP` for an `agent_completed` notification, `ENGINE_DISPATCH_MAP` (fixed forge#2466) for a background-Bash notification from the engine-first path — and immediately process it exactly the same way regardless of which map resolved it; every check below (`classify_predecessor_state()`, dependent dispatch, stall/staging checks) keys off GitHub labels/state, not which dispatch mechanism produced them. **Fallback**: if a runtime ever fails to deliver a background-Bash completion notification, poll `gh issue view --json labels` for the in-flight engine-dispatched issue's workflow label instead of blocking — the same degrade-gracefully behavior already used when `BACKGROUND_DISPATCH_ENABLED=false` for the Agent-spawn path (see `phase-3-dependency.md` → Background Dispatch Mode).
|
|
693
851
|
|
|
694
|
-
**Concurrency slot release (MANDATORY — first action on every completion)** <!-- Added: forge#1912 -->: The instant
|
|
852
|
+
**Concurrency slot release (MANDATORY — first action on every completion)** <!-- Added: forge#1912 -->: The instant a completion notification arrives — `agent_completed`, or a background-Bash completion for an engine-dispatched issue — decrement `ACTIVE_DISPATCH_COUNT` by 1 — a worker slot has just freed, regardless of what terminal state the issue ended up in. Do this before any stall-recovery/resume logic below. If that agent is then resumed (item 2 below, or Step 4B.5's stall recovery) because it stalled mid-pipeline rather than truly finishing, re-increment `ACTIVE_DISPATCH_COUNT` when the `Agent(resume=...)` call is issued — a resume re-occupies a worker slot exactly like a fresh dispatch does. (Step 4B.5's stall-resume mechanism is `Agent(resume=...)`-specific and is unchanged by this fix — an engine-dispatched issue that stalls is not currently auto-resumed by Step 4B.5; it surfaces the same way any other stalled issue does, via the standard stall-detection alert, and `forgedock resume-stalled` remains available as a separate manual/scripted recovery path outside this file's scope.)
|
|
695
853
|
|
|
696
854
|
**Ordering requirement (MANDATORY — prevents transient over-cap)**: For every agent that completed in this notification batch, finish its terminal-state check and, if applicable, its resume re-increment (items 1-2 below) BEFORE computing `dispatch_headroom` for item 5's newly-ready-issue dispatch. Computing headroom before a to-be-resumed agent's re-increment would let a genuinely-still-running agent's freed slot be double-booked — once by a fresh dispatch, once by the resume itself — transiently exceeding `MAX_CONCURRENT`. Process every completed agent's items 1-2 to a decision (terminal vs. resumed) first; only then compute headroom once for the batch's item 5 dispatch.
|
|
697
855
|
|
|
@@ -991,7 +1149,7 @@ done
|
|
|
991
1149
|
|
|
992
1150
|
Report every dependent whose skip was avoided this way — e.g. "#{DEP} — predecessor #{PRED} failed but never touched the shared file(s); edge dropped, #{DEP} not skipped." For all remaining dependents (real `EDGE_KIND` overlap confirmed, or a non-`EDGE_KIND` edge type), mark them "skipped — dependency #{X} failed" and report them. Do NOT dispatch them.
|
|
993
1151
|
|
|
994
|
-
6.4. **Auto-dispatch remediation against a `needs-human`-gated issue's own PR — runs unconditionally per completion, with or without dependents** <!-- Added: forge#1813, fixed: forge#2243 --> — item 6.5 below tracks the *dependents* of a `GATED` predecessor; this item handles the gated issue's own PR, which item 6.5/6.6 never re-drive on their own. **This check is bound directly to the issue that just completed — `PRED="$NUM"` (the same issue number carried from items 1-4 of this Step 4B sequence) — NOT to any `$PRED` produced by walking a dependent's predecessor list (item 5's readiness loop) or by item 6's FAILED-specific dependent-walk.** Run it for every completed agent, every cycle, regardless of whether that issue has any dependents in the DAG at all — a leaf issue (no dependents) is just as eligible as an issue that blocks others. Trigger condition: the completed issue classifies `GATED` **specifically via `needs-human`** — NOT `workflow:awaiting-merge`. That second state already means "remediated and re-reviewed to a clean verdict
|
|
1152
|
+
6.4. **Auto-dispatch remediation against a `needs-human`-gated issue's own PR — runs unconditionally per completion, with or without dependents** <!-- Added: forge#1813, fixed: forge#2243 --> — item 6.5 below tracks the *dependents* of a `GATED` predecessor; this item handles the gated issue's own PR, which item 6.5/6.6 never re-drive on their own. **This check is bound directly to the issue that just completed — `PRED="$NUM"` (the same issue number carried from items 1-4 of this Step 4B sequence) — NOT to any `$PRED` produced by walking a dependent's predecessor list (item 5's readiness loop) or by item 6's FAILED-specific dependent-walk.** Run it for every completed agent, every cycle, regardless of whether that issue has any dependents in the DAG at all — a leaf issue (no dependents) is just as eligible as an issue that blocks others. Trigger condition: the completed issue classifies `GATED` **specifically via `needs-human`** — NOT `workflow:awaiting-merge`. That second state already means "remediated and re-reviewed to a clean verdict" (see forge#1810's guard) — dispatching remediation again would be redundant, not just wasteful, since there is nothing left to fix. A PR that reaches `workflow:awaiting-merge` now does so only when it targets `main` / the deploy gate (where `staging → main` is the genuine human gate and a human's merge click is required); a remediated-clean PR targeting a non-`main` base (`staging`, `milestone/*`) auto-lands to `workflow:merged` via remediate.md Phase M7's base-scoped auto-land bar (forge#2570) and never parks here. Either way this item's skip of `workflow:awaiting-merge` is unchanged.
|
|
995
1153
|
|
|
996
1154
|
```bash
|
|
997
1155
|
# Bind PRED to the issue that just completed — independent of item 5's dependent-walk loop
|
|
@@ -1176,7 +1334,7 @@ Gating predecessor #${NUM} reached \`workflow:merged\` — dispatching now. (Was
|
|
|
1176
1334
|
✓ #{NUMBER} — {title} → PR #{PR} merged to {target}
|
|
1177
1335
|
✗ #{NUMBER} — {title} → {reason for failure}
|
|
1178
1336
|
⚠ #{NUMBER} — {title} → PIPELINE BYPASS (no /work-on — PR invalid)
|
|
1179
|
-
⏸ #{NUMBER} — {title} → PR #{PR} awaiting-merge (remediated + re-approved — human merge only, no diagnosis needed)
|
|
1337
|
+
⏸ #{NUMBER} — {title} → PR #{PR} awaiting-merge (remediated + re-approved, `main`/deploy-gate base — human merge only, no diagnosis needed; non-`main` bases now auto-land to `workflow:merged` via remediate.md M7, forge#2570)
|
|
1180
1338
|
🔗 #{NUMBER} — {title} → blocked-on-human-merge (gated by #{PRED}, will auto-dispatch on #{PRED} merge)
|
|
1181
1339
|
⏳ Progress: {completed}/{total} complete, {active} active, {blocked} blocked
|
|
1182
1340
|
→ Dispatched #{NEWLY_READY} (predecessor #{PRED} completed)
|
|
@@ -1202,7 +1360,9 @@ Gating predecessor #${NUM} reached \`workflow:merged\` — dispatching now. (Was
|
|
|
1202
1360
|
|
|
1203
1361
|
9. **Run staging integrity check** (from Step 4A-pre) if the completed agent merged a PR targeting staging.
|
|
1204
1362
|
|
|
1205
|
-
**Termination condition**: All issues in the DAG have reached `DONE` or `FAILED` (merged, invalid, or skipped due to dependency failure) — OR are `blocked-on-human-merge` (item 6.5) with no further dispatchable work remaining in the batch. These two outcomes are reported differently: a batch where every issue is `DONE`/`FAILED` is a **clean drain**; a batch where one or more issues remain `blocked-on-human-merge` is a **paused drain** — the active dispatch loop stops (there is nothing left to do until a human merges a gating PR) but this MUST be reported as paused, not as fully complete (see `phase-6-report.md`'s `🔗 Blocked-on-Merge` section). `needs-human` predecessors with no open PR are neither — they remain GATED indefinitely until either a PR appears (dependent moves to blocked-on-human-merge) or the predecessor itself resolves; do not treat isolated `needs-human` issues with no dependents as blocking termination. When either drain condition is met, check whether deferred review-spawned findings exist (accumulated in `DEFERRED_FINDINGS` during Step 4C). If deferred findings exist → proceed to Step 4F (Completion Sweep). If no deferred findings → proceed to Phase 5.
|
|
1363
|
+
**Termination condition**: All issues in the DAG have reached `DONE` or `FAILED` (merged, invalid, or skipped due to dependency failure) — OR are `blocked-on-human-merge` (item 6.5) with no further dispatchable work remaining in the batch. These two outcomes are reported differently: a batch where every issue is `DONE`/`FAILED` is a **clean drain**; a batch where one or more issues remain `blocked-on-human-merge` is a **paused drain** — the active dispatch loop stops (there is nothing left to do until a human merges a gating PR) but this MUST be reported as paused, not as fully complete (see `phase-6-report.md`'s `🔗 Blocked-on-Merge` section). `needs-human` predecessors with no open PR are neither — they remain GATED indefinitely until either a PR appears (dependent moves to blocked-on-human-merge) or the predecessor itself resolves; do not treat isolated `needs-human` issues with no dependents as blocking termination. When either drain condition is met, check whether deferred review-spawned findings exist (accumulated in `DEFERRED_FINDINGS` during Step 4C). If deferred findings exist → proceed to Step 4F (Completion Sweep). If no deferred findings → **call `release_orchestrator_lease()` (Step 4A-pre.-1)** — this is a clean/paused drain of this session's own dispatch loop, so the lease should not be left held for a now-idle session — then proceed to Phase 5. <!-- Added: forge#2627 -->
|
|
1364
|
+
|
|
1365
|
+
**Reminder — this is the normal-exit lease release, distinct from the interrupted-stop procedure**: whether the drain is clean or paused, this session is no longer actively dispatching, so its lease must be released here (or, for a paused drain, at minimum have its heartbeat refresh stop — see Step 4A-pre.-1). This complements, but does not replace, the **Stopping the orchestrator** procedure (Step 4A-pre.-0.5) which handles the abnormal case of a mid-dispatch interrupt.
|
|
1206
1366
|
|
|
1207
1367
|
**Relationship to `DEFERRED_CONCURRENCY_ISSUES` (forge#1912)**: A non-empty `DEFERRED_CONCURRENCY_ISSUES[]` never satisfies either termination condition above — it means dispatchable work exists but is temporarily held back by the concurrency cap, not that the DAG has stalled. Unlike `blocked-on-human-merge`, this is never reported as a "paused drain" requiring human action — it self-resolves automatically the moment any in-flight agent completes and frees a slot (Step 4A's `dispatch_headroom` recomputes every cycle). Only treat the DAG as drained once `DEFERRED_CONCURRENCY_ISSUES` is also empty.
|
|
1208
1368
|
|
|
@@ -1357,6 +1517,15 @@ done
|
|
|
1357
1517
|
|
|
1358
1518
|
After each agent reaches a terminal state, check if its `/work-on` run spawned review-finding issues during the review phase. These are new work items that should be added to the dependency DAG and dispatched when ready.
|
|
1359
1519
|
|
|
1520
|
+
**This step is the mid-run half of the run-spawned cascade mode** (`phase-1-resolve.md`
|
|
1521
|
+
"Cascade / Review-Finding Resolution" — mode (a)). It scopes to `BATCH_T0` for the same reason
|
|
1522
|
+
that resolve step does: this loop must only ever fold in `review-finding` issues *this batch's own
|
|
1523
|
+
agents* produced, never the pre-existing open backlog. There is no whole-backlog-sweep mode here —
|
|
1524
|
+
that mode only exists as the explicit, one-shot `--include-backlog` opt-in at Phase 1 resolve time
|
|
1525
|
+
(a human directly asking for it); Step 4C runs autonomously on every completion cycle and must
|
|
1526
|
+
never silently widen to the full backlog regardless of any flag, since nothing here is a
|
|
1527
|
+
human-in-the-loop request. <!-- Added: forge#2628 -->
|
|
1528
|
+
|
|
1360
1529
|
```bash
|
|
1361
1530
|
# Method 1: Read TRAJECTORY comments from completed issues for "Finding issues" row
|
|
1362
1531
|
for NUM in {completed_issue_number}; do
|
|
@@ -1365,10 +1534,14 @@ for NUM in {completed_issue_number}; do
|
|
|
1365
1534
|
| grep -oP 'Finding issues\s*\|\s*#?\K\d+[^|]*' | grep -oP '\d+' | sort -u
|
|
1366
1535
|
done
|
|
1367
1536
|
|
|
1368
|
-
# Method 2 (fallback): Check for
|
|
1369
|
-
|
|
1370
|
-
|
|
1371
|
-
|
|
1537
|
+
# Method 2 (fallback): Check for review-finding issues created since this batch started
|
|
1538
|
+
# (BATCH_T0 — see Step 4A.pre above; reused from phase-1-resolve.md's Phase 1 capture, never
|
|
1539
|
+
# a rolling "N hours ago" window). A rolling window can still admit pre-existing backlog
|
|
1540
|
+
# issues that happen to have been created recently for reasons unrelated to this batch;
|
|
1541
|
+
# anchoring to BATCH_T0 cannot, by construction — anything created before this batch began
|
|
1542
|
+
# is excluded regardless of how recent it looks.
|
|
1543
|
+
gh issue list -R {GH_REPO} --state open --search "label:review-finding created:>=${BATCH_T0}" --limit 20 \
|
|
1544
|
+
--json number,title,body,createdAt
|
|
1372
1545
|
```
|
|
1373
1546
|
|
|
1374
1547
|
**If review-finding issues were spawned:**
|
|
@@ -1419,8 +1592,122 @@ ALL_BATCH_FILES=$(echo "$ALL_BATCH_FILES" | sort -u | grep -v '^$')
|
|
|
1419
1592
|
# NOTE: DEFERRED_FINDINGS, QUEUED_FINDINGS, and DEFERRED_REASONS are declared at
|
|
1420
1593
|
# batch scope in Step 4A.pre — do NOT re-initialize them here (Step 4C runs per-agent).
|
|
1421
1594
|
for FINDING_NUM in {spawned_finding_numbers}; do
|
|
1422
|
-
FINDING_DATA=$(gh issue view $FINDING_NUM -R {GH_REPO} --json labels,title,body \
|
|
1423
|
-
--jq '{labels: [.labels[].name], title: .title, body: .body}')
|
|
1595
|
+
FINDING_DATA=$(gh issue view $FINDING_NUM -R {GH_REPO} --json labels,title,body,updatedAt \
|
|
1596
|
+
--jq '{labels: [.labels[].name], title: .title, body: .body, updatedAt: .updatedAt}')
|
|
1597
|
+
|
|
1598
|
+
# Code-branch repair guard (MANDATORY — before priority/defer heuristics below).
|
|
1599
|
+
# A review-finding with no **Code branch** annotation, whose parent PR's base is
|
|
1600
|
+
# not the staging fast lane, would otherwise silently fall through to
|
|
1601
|
+
# classify-lane.sh's implicit staging default — the one branch where the
|
|
1602
|
+
# finding's subject code is guaranteed absent (forge#2443: 5 findings from
|
|
1603
|
+
# milestone-lane PRs were misrouted this way in one batch; absent manual
|
|
1604
|
+
# correction, /work-on's investigation phase would have closed all 5 as
|
|
1605
|
+
# invalid). This is a synchronous per-finding check, distinct from the
|
|
1606
|
+
# periodic Step 4C.5 lane-consistency sweep below (that one audits an
|
|
1607
|
+
# entire milestone's PRs for base-branch drift after the fact; this one
|
|
1608
|
+
# repairs a single finding's missing provenance at discovery time).
|
|
1609
|
+
FINDING_BODY_RAW=$(echo "$FINDING_DATA" | jq -r '.body')
|
|
1610
|
+
# Snapshot freshness marker alongside the body read above — compared immediately
|
|
1611
|
+
# before the body-mutating write below to detect a concurrent edit (forge#2512).
|
|
1612
|
+
FINDING_UPDATED_AT_SNAPSHOT=$(echo "$FINDING_DATA" | jq -r '.updatedAt')
|
|
1613
|
+
if ! echo "$FINDING_BODY_RAW" | grep -q '\*\*Code branch\*\*:'; then
|
|
1614
|
+
# Portable (non-PCRE) extraction — PCRE grep lookbehinds are not supported by
|
|
1615
|
+
# Git Bash's grep build on Windows (same convention as
|
|
1616
|
+
# scripts/derive-finding-milestone.sh and scripts/code-index.sh), so use a
|
|
1617
|
+
# bash regex + BASH_REMATCH instead of a lookbehind.
|
|
1618
|
+
REPAIR_SOURCE_PR=""
|
|
1619
|
+
FINDING_BODY_LOWER=$(echo "$FINDING_BODY_RAW" | tr '[:upper:]' '[:lower:]')
|
|
1620
|
+
if [[ "$FINDING_BODY_LOWER" =~ \*\*source\*\*:[[:space:]]*pr[[:space:]]*#([0-9]+) ]]; then
|
|
1621
|
+
REPAIR_SOURCE_PR="${BASH_REMATCH[1]}"
|
|
1622
|
+
fi
|
|
1623
|
+
if [ -n "$REPAIR_SOURCE_PR" ]; then
|
|
1624
|
+
REPAIR_PARENT_BASE=$(gh pr view "$REPAIR_SOURCE_PR" -R {GH_REPO} --json baseRefName --jq '.baseRefName' 2>/dev/null || echo "")
|
|
1625
|
+
if [ -n "$REPAIR_PARENT_BASE" ] && [ "$REPAIR_PARENT_BASE" != "$STAGING_BRANCH" ]; then
|
|
1626
|
+
# GOVERNOR: cap the number of body-mutating repairs Step 4C will attempt
|
|
1627
|
+
# in a single run — bounds the blast radius of this new autonomous
|
|
1628
|
+
# `gh issue edit --body` mutation regardless of how many findings in the
|
|
1629
|
+
# batch happen to be missing Code branch. Declared with `:=` so it is
|
|
1630
|
+
# safe to reference whether or not a prior iteration already set it
|
|
1631
|
+
# (bash arithmetic increment below persists it across loop iterations).
|
|
1632
|
+
: "${REPAIR_GOVERNOR_COUNT:=0}"
|
|
1633
|
+
REPAIR_GOVERNOR_MAX=25
|
|
1634
|
+
if [ "$REPAIR_GOVERNOR_COUNT" -ge "$REPAIR_GOVERNOR_MAX" ]; then
|
|
1635
|
+
echo "REPAIR: #${FINDING_NUM} skipped — GOVERNOR cap reached (${REPAIR_GOVERNOR_MAX} repairs already attempted this run); flagging instead of repairing"
|
|
1636
|
+
gh issue edit "$FINDING_NUM" -R {GH_REPO} --add-label needs-human 2>/dev/null || true # governor-cap path
|
|
1637
|
+
else
|
|
1638
|
+
REPAIR_GOVERNOR_COUNT=$((REPAIR_GOVERNOR_COUNT + 1))
|
|
1639
|
+
echo "REPAIR: #${FINDING_NUM} has no **Code branch** and parent PR #${REPAIR_SOURCE_PR} bases on '${REPAIR_PARENT_BASE}' (non-staging) — attempting repair (${REPAIR_GOVERNOR_COUNT}/${REPAIR_GOVERNOR_MAX} this run)"
|
|
1640
|
+
# Read-then-append: never blind-overwrite the existing body, only extend it.
|
|
1641
|
+
REPAIRED_BODY=$(printf '%s\n\n## Source Branch Context (repaired by orchestrate Step 4C — forge#2443)\n\n**Code branch**: `%s`\n**Worktree base**: `origin/%s`\n' \
|
|
1642
|
+
"$FINDING_BODY_RAW" "$REPAIR_PARENT_BASE" "$REPAIR_PARENT_BASE")
|
|
1643
|
+
# Concurrency guard (forge#2512 — TOCTOU fix): REPAIRED_BODY above was built purely
|
|
1644
|
+
# by string-appending to the FINDING_BODY_RAW snapshot captured at the top of this
|
|
1645
|
+
# loop iteration. The write below is a full-body overwrite — if the issue's body
|
|
1646
|
+
# changed since that read (a human edit, a second orchestrator run, or the finding's
|
|
1647
|
+
# own creation process still settling), a blind write here would silently clobber
|
|
1648
|
+
# that concurrent edit with no conflict signal. Re-fetch just `updatedAt` (cheap —
|
|
1649
|
+
# no body/labels payload) immediately before the write and compare against the
|
|
1650
|
+
# snapshot captured alongside FINDING_BODY_RAW. Bounded by the same
|
|
1651
|
+
# REPAIR_GOVERNOR_MAX cap as the write itself — at most one extra `gh issue view`
|
|
1652
|
+
# call per repair attempt.
|
|
1653
|
+
FINDING_UPDATED_AT_CURRENT=$(gh issue view "$FINDING_NUM" -R {GH_REPO} --json updatedAt --jq '.updatedAt' 2>/dev/null || echo "")
|
|
1654
|
+
if [ -z "$FINDING_UPDATED_AT_CURRENT" ]; then
|
|
1655
|
+
# forge#2565: the re-fetch itself failed (network error, API rate limit, or any
|
|
1656
|
+
# other non-zero `gh issue view` exit) — the `|| echo ""` fallback above produces
|
|
1657
|
+
# an empty string that is indistinguishable, to a plain `[ -n ... ] && [ ... != ... ]`
|
|
1658
|
+
# check, from "confirmed no concurrent edit." Treat an unverifiable freshness check
|
|
1659
|
+
# as its own outcome and fail closed, mirroring the concurrent-edit branch below
|
|
1660
|
+
# rather than silently falling through to the unprotected write.
|
|
1661
|
+
echo "REPAIR: #${FINDING_NUM} skipped — freshness re-fetch failed (network error or API rate limit); cannot verify no concurrent edit occurred, failing closed instead of risking a silent clobber"
|
|
1662
|
+
# forge#2584: stage the body in a mktemp file and deliver it via --body-file
|
|
1663
|
+
# instead of interpolating finding-derived variables (${FINDING_NUM},
|
|
1664
|
+
# ${REPAIR_SOURCE_PR}, ${REPAIR_PARENT_BASE}, ${FINDING_UPDATED_AT_SNAPSHOT})
|
|
1665
|
+
# directly into a double-quoted --body argument at the gh call site — mirrors
|
|
1666
|
+
# REPAIRED_BODY's existing printf-into-variable approach and the repo-wide
|
|
1667
|
+
# mktemp + --body-file convention (forge#2198). mktemp avoids /tmp path
|
|
1668
|
+
# collisions with other concurrently-running orchestrated agents.
|
|
1669
|
+
GATE_BODY_TMPFILE="$(mktemp)"
|
|
1670
|
+
printf '%s' "<!-- FORGE:GATE_FAILURE -->
|
|
1671
|
+
## Code Branch Repair Skipped — Freshness Re-fetch Failed
|
|
1672
|
+
|
|
1673
|
+
Finding #${FINDING_NUM} has no **Code branch** annotation and its parent PR #${REPAIR_SOURCE_PR} bases on \`${REPAIR_PARENT_BASE}\` — not the staging fast lane. Automatic repair was skipped because the freshness re-fetch (\`gh issue view --json updatedAt\`) failed, so the concurrency guard could not confirm the issue body is still at the snapshot captured earlier in this iteration (\`${FINDING_UPDATED_AT_SNAPSHOT}\`). Proceeding with the write here would risk silently clobbering a concurrent edit with no way to verify. Human review required to confirm the current body state and, if still needed, re-apply the Code branch stamp manually. <!-- forge#2565 -->" > "$GATE_BODY_TMPFILE"
|
|
1674
|
+
gh issue comment "$FINDING_NUM" -R {GH_REPO} --body-file "$GATE_BODY_TMPFILE" 2>/dev/null || true
|
|
1675
|
+
rm -f "$GATE_BODY_TMPFILE"
|
|
1676
|
+
gh issue edit "$FINDING_NUM" -R {GH_REPO} --add-label needs-human 2>/dev/null || true # freshness-recheck-failure path
|
|
1677
|
+
elif [ "$FINDING_UPDATED_AT_CURRENT" != "$FINDING_UPDATED_AT_SNAPSHOT" ]; then
|
|
1678
|
+
echo "REPAIR: #${FINDING_NUM} skipped — concurrent edit detected (updatedAt changed from ${FINDING_UPDATED_AT_SNAPSHOT} to ${FINDING_UPDATED_AT_CURRENT} since this iteration's initial read); flagging instead of repairing"
|
|
1679
|
+
# forge#2584: mktemp + --body-file, same rationale as the freshness-recheck-failure path above.
|
|
1680
|
+
CONCURRENT_EDIT_BODY_TMPFILE="$(mktemp)"
|
|
1681
|
+
printf '%s' "<!-- FORGE:GATE_FAILURE -->
|
|
1682
|
+
## Code Branch Repair Skipped — Concurrent Edit Detected
|
|
1683
|
+
|
|
1684
|
+
Finding #${FINDING_NUM} has no **Code branch** annotation and its parent PR #${REPAIR_SOURCE_PR} bases on \`${REPAIR_PARENT_BASE}\` — not the staging fast lane. Automatic repair was skipped because the issue body was edited concurrently (\`updatedAt\` changed from \`${FINDING_UPDATED_AT_SNAPSHOT}\` to \`${FINDING_UPDATED_AT_CURRENT}\` between this run's initial read and the repair write). Overwriting the body now would have silently discarded that concurrent edit. Human review required to reconcile and, if still needed, re-apply the Code branch stamp manually. <!-- forge#2512 -->" > "$CONCURRENT_EDIT_BODY_TMPFILE"
|
|
1685
|
+
gh issue comment "$FINDING_NUM" -R {GH_REPO} --body-file "$CONCURRENT_EDIT_BODY_TMPFILE" 2>/dev/null || true
|
|
1686
|
+
rm -f "$CONCURRENT_EDIT_BODY_TMPFILE"
|
|
1687
|
+
gh issue edit "$FINDING_NUM" -R {GH_REPO} --add-label needs-human 2>/dev/null || true # concurrent-edit path
|
|
1688
|
+
else
|
|
1689
|
+
if gh issue edit "$FINDING_NUM" -R {GH_REPO} --body "$REPAIRED_BODY" 2>/dev/null; then
|
|
1690
|
+
echo "REPAIR: #${FINDING_NUM} Code branch repaired to '${REPAIR_PARENT_BASE}'"
|
|
1691
|
+
# Re-fetch so the rest of this loop iteration (priority/defer heuristics
|
|
1692
|
+
# below) sees the repaired body, not the stale pre-repair one.
|
|
1693
|
+
FINDING_DATA=$(gh issue view $FINDING_NUM -R {GH_REPO} --json labels,title,body,updatedAt \
|
|
1694
|
+
--jq '{labels: [.labels[].name], title: .title, body: .body, updatedAt: .updatedAt}')
|
|
1695
|
+
else
|
|
1696
|
+
# forge#2584: mktemp + --body-file, same rationale as the two failure paths above.
|
|
1697
|
+
REPAIR_FAILED_BODY_TMPFILE="$(mktemp)"
|
|
1698
|
+
printf '%s' "<!-- FORGE:GATE_FAILURE -->
|
|
1699
|
+
## Code Branch Repair Failed
|
|
1700
|
+
|
|
1701
|
+
Finding #${FINDING_NUM} has no **Code branch** annotation and its parent PR #${REPAIR_SOURCE_PR} bases on \`${REPAIR_PARENT_BASE}\` — not the staging fast lane. Automatic repair (\`gh issue edit --body\`) failed. Without this annotation, \`/work-on\`'s investigation phase may look for the code on the wrong branch (staging, where it is absent) and misclassify this confirmed finding as invalid. Human review required. <!-- forge#2443 -->" > "$REPAIR_FAILED_BODY_TMPFILE"
|
|
1702
|
+
gh issue comment "$FINDING_NUM" -R {GH_REPO} --body-file "$REPAIR_FAILED_BODY_TMPFILE" 2>/dev/null || true
|
|
1703
|
+
rm -f "$REPAIR_FAILED_BODY_TMPFILE"
|
|
1704
|
+
gh issue edit "$FINDING_NUM" -R {GH_REPO} --add-label needs-human 2>/dev/null || true # repair-failure path
|
|
1705
|
+
fi
|
|
1706
|
+
fi
|
|
1707
|
+
fi
|
|
1708
|
+
fi
|
|
1709
|
+
fi
|
|
1710
|
+
fi
|
|
1424
1711
|
|
|
1425
1712
|
# Priority extraction accepts both label schemas: canonical `priority:P<n>` (the only
|
|
1426
1713
|
# form review-pr.md ever creates) and bare `P<n>` (carried by some consumer repos'
|
|
@@ -1526,9 +1813,34 @@ for FINDING_NUM in "${QUEUED_FINDINGS[@]}"; do
|
|
|
1526
1813
|
|
|
1527
1814
|
# Safety exclusions — never batch, at any priority. Same jq test() engine and
|
|
1528
1815
|
# patterns as phase-1-resolve.md's batching rule (single shared mechanism).
|
|
1816
|
+
# Word-boundary anchored (not bare substrings) and attribution-boilerplate
|
|
1817
|
+
# (**Confidence**/**Severity**/**Review comment** — see forge#2477 note below
|
|
1818
|
+
# for why **Source**/**Agent** are deliberately excluded from this list)
|
|
1819
|
+
# stripped from the body before scanning, so a finding naming its own
|
|
1820
|
+
# reviewing agent ("Security") or an identifier like `authority_source` no
|
|
1821
|
+
# longer false-positives.
|
|
1822
|
+
# `authentication|authorization|authn|authz` preserve real auth-domain coverage
|
|
1823
|
+
# as separate whole-word alternatives now that bare `auth` is boundary-anchored.
|
|
1824
|
+
# <!-- forge#2423 -->
|
|
1825
|
+
# Each stripped alternative is anchored to the field's real generator-output
|
|
1826
|
+
# shape (enum for Confidence/Severity, URL for Review comment) rather than a
|
|
1827
|
+
# bare label-prefix + `.*$` — matching on label shape alone lets
|
|
1828
|
+
# attacker-controlled body text on one of these lines get stripped along
|
|
1829
|
+
# with the label, smuggling banned keywords past the scan below. Source/Agent
|
|
1830
|
+
# are deliberately NOT stripped: both hold genuinely free-text generator
|
|
1831
|
+
# output (a PR title; an agent's self-description) with no fixed vocabulary,
|
|
1832
|
+
# so no shape bound can distinguish legitimate attribution from
|
|
1833
|
+
# attacker-authored payload placed in the same position — a length-bounded
|
|
1834
|
+
# free-text alternative for either field re-opens the exact smuggling gap
|
|
1835
|
+
# this fix closes (an attacker need only prefix their payload with a fake
|
|
1836
|
+
# "PR #N — " or agent name to satisfy the bound). Leaving them unstripped
|
|
1837
|
+
# trades a narrow, already-known false-positive (forge#2423's Agent-line
|
|
1838
|
+
# case; a P3 finding whose Source/Agent text happens to mention a domain
|
|
1839
|
+
# keyword is not auto-batched) for closing a real bypass — the safe
|
|
1840
|
+
# direction for a security-relevant exclusion. <!-- forge#2477 -->
|
|
1529
1841
|
echo "$FINDING_DATA" | jq -e '
|
|
1530
|
-
(.title | test("security|billing|anti-bot|auth"; "i"))
|
|
1531
|
-
or (.body
|
|
1842
|
+
(.title | test("\\b(security|billing|anti-bot|auth|authentication|authorization|authn|authz)\\b"; "i"))
|
|
1843
|
+
or ((.body | gsub("(?m)^\\*\\*(?:Confidence\\*\\*: (?:CONFIRMED|LIKELY|POSSIBLE)|Severity\\*\\*: (?:CRITICAL|HIGH|MEDIUM|LOW|INFO)|Review comment\\*\\*: https?://\\S+)$"; "")) | test("## Problem[\\s\\S]{0,500}\\b(security|billing|anti-bot|auth|authentication|authorization|authn|authz)\\b"; "i"))
|
|
1532
1844
|
or ([.labels[]] | any(. == "security" or . == "billing" or . == "anti-bot" or . == "auth"))
|
|
1533
1845
|
' >/dev/null && continue
|
|
1534
1846
|
|
|
@@ -1569,8 +1881,31 @@ for FILE in "${!SURFACE_FILE_MEMBERS[@]}"; do
|
|
|
1569
1881
|
MEMBER_LINES="${MEMBER_LINES}- [ ] #${M}: ${MTITLE}"$'\n'
|
|
1570
1882
|
done
|
|
1571
1883
|
|
|
1884
|
+
# Dedup-check with member exclusion (MANDATORY — forge#2432, identical mechanism to
|
|
1885
|
+
# phase-1-resolve.md's mirror block; this site keeps its existing direct-to-gh issue
|
|
1886
|
+
# creation below rather than migrating to /issue's Skill routing, since this loop runs
|
|
1887
|
+
# per-completion-cycle and a direct scripts/issue-dedup.sh call is cheaper here). A
|
|
1888
|
+
# batch title necessarily restates its own members' subject matter by construction —
|
|
1889
|
+
# excluding the cluster's own member numbers means Phase-2D-style dedup only fires on
|
|
1890
|
+
# a GENUINE non-member duplicate. --exclude narrows the candidate set; it is NOT a
|
|
1891
|
+
# --force equivalent.
|
|
1892
|
+
CHUNK_LIST=$(IFS=,; echo "${CHUNK[*]}")
|
|
1893
|
+
PROPOSED_BATCH_TITLE="fix(batch): P3 review findings — ${SAFE_SURFACE_AREA} (same-run batch)"
|
|
1894
|
+
DEDUP_RESULT=$(scripts/issue-dedup.sh "$PROPOSED_BATCH_TITLE" {GH_FLAG} --exclude "$CHUNK_LIST" 2>&1)
|
|
1895
|
+
DEDUP_EXIT=$?
|
|
1896
|
+
|
|
1897
|
+
if [ "$DEDUP_EXIT" -eq 1 ]; then
|
|
1898
|
+
# Genuine non-member duplicate — some other open issue already covers this exact
|
|
1899
|
+
# surface area. Do NOT create the batch; its members stay individually queued.
|
|
1900
|
+
echo "Batch dedup STOP (non-member match): $DEDUP_RESULT — skipping batch creation for $SAFE_SURFACE_AREA, members stay individually queued"
|
|
1901
|
+
continue
|
|
1902
|
+
elif [ "$DEDUP_EXIT" -eq 2 ]; then
|
|
1903
|
+
echo "Batch dedup usage error: $DEDUP_RESULT — skipping batch creation for $SAFE_SURFACE_AREA, members stay individually queued"
|
|
1904
|
+
continue
|
|
1905
|
+
fi
|
|
1906
|
+
|
|
1572
1907
|
BATCH_ISSUE_NUM=$(gh issue create {GH_FLAG} \
|
|
1573
|
-
--title "
|
|
1908
|
+
--title "$PROPOSED_BATCH_TITLE" \
|
|
1574
1909
|
--label "review-finding,priority:P3,batch" \
|
|
1575
1910
|
--body "$(cat <<BATCH_EOF
|
|
1576
1911
|
## Problem
|
|
@@ -1654,10 +1989,37 @@ if [ "${#TOKEN_BUDGET_DEFERRED_THIS_RUN[@]}" -gt 0 ]; then
|
|
|
1654
1989
|
fi
|
|
1655
1990
|
```
|
|
1656
1991
|
|
|
1992
|
+
**Classify lane for each queued finding (MANDATORY — before dispatch)** <!-- Added: forge#2629 -->
|
|
1993
|
+
|
|
1994
|
+
Cascade-dispatched findings inherit NO lane assumption from the parent batch. Step 4A.pre's classify-lane loop (above) only covers `{ready_issue_numbers}` — the *original* dispatch group — and does not see findings discovered mid-run by this Step 4C pass. Without an explicit gate here, a finding reaches the Step 4A/4B dispatch loop with `${ISSUE_LANE[$NUM]}`/`${ISSUE_PR_BASE[$NUM]}` unset for its number, which is exactly the milestone-owned-finding-misrouted-to-staging risk this issue exists to close. Run this loop over `QUEUED_FINDINGS` (the post-clubbing, post-token-budget-gate list) before any of the numbered steps below, reusing the identical `classify-lane.sh` invocation and failure handling as Step 4A.pre — same script, same batch-scope `ISSUE_LANE`/`ISSUE_PR_BASE` arrays (declared once, line ~362-363), no new mechanism:
|
|
1995
|
+
|
|
1996
|
+
```bash
|
|
1997
|
+
for FINDING_NUM in "${QUEUED_FINDINGS[@]}"; do
|
|
1998
|
+
PR_BASE=$(bash ~/.claude/scripts/classify-lane.sh "$FINDING_NUM" -R {GH_REPO}) || {
|
|
1999
|
+
echo "ERROR: classify-lane.sh failed for #$FINDING_NUM — adding needs-human label and removing from QUEUED_FINDINGS" >&2
|
|
2000
|
+
# GOVERNOR-exempt: intentional coordination side-effect (best-effort lease/board/finding post), DRY_RUN-safe — reviewed & accepted for the check-command-side-effects gate. Flagged only by the staging->main full-diff; passes on every feature PR. forge#2627
|
|
2001
|
+
gh issue edit "$FINDING_NUM" -R {GH_REPO} --add-label "needs-human" 2>/dev/null || true
|
|
2002
|
+
QUEUED_FINDINGS=($(printf '%s\n' "${QUEUED_FINDINGS[@]}" | grep -vxF "$FINDING_NUM" || true))
|
|
2003
|
+
continue
|
|
2004
|
+
}
|
|
2005
|
+
# Derive LANE label from PR_BASE — identical rule to Step 4A.pre.
|
|
2006
|
+
if [ "$PR_BASE" = "staging" ]; then
|
|
2007
|
+
LANE="fast-lane"
|
|
2008
|
+
else
|
|
2009
|
+
LANE="feature-lane"
|
|
2010
|
+
fi
|
|
2011
|
+
ISSUE_LANE[$FINDING_NUM]="$LANE"
|
|
2012
|
+
ISSUE_PR_BASE[$FINDING_NUM]="$PR_BASE"
|
|
2013
|
+
echo "#$FINDING_NUM → lane=$LANE, PR_BASE=$PR_BASE (cascade finding)"
|
|
2014
|
+
done
|
|
2015
|
+
```
|
|
2016
|
+
|
|
2017
|
+
A finding that fails classify-lane is removed from `QUEUED_FINDINGS` (mirrors Step 4A.pre's `continue`, which never admits a failed classification into the ready set) and flagged `needs-human` instead of silently falling through to the dispatch loop with an unset lane.
|
|
2018
|
+
|
|
1657
2019
|
**For queued (non-deferred) findings:**
|
|
1658
2020
|
|
|
1659
2021
|
1. **Add them to the dependency DAG.** They are implementation issues — same as issues spawned by investigations in Phase 2. Compute their predecessor sets using the same conflict detection (Step 3C Layers 1-4) against all remaining blocked/active issues.
|
|
1660
|
-
2. **Respect source branch context.** Review-finding issues have `**Code branch**: \`{branch}\`` in their body — the `/work-on` agent will read this and branch from the right origin.
|
|
2022
|
+
2. **Respect source branch context.** Review-finding issues have `**Code branch**: \`{branch}\`` in their body — the `/work-on` agent will read this and branch from the right origin (the body annotation is repaired earlier in this step if missing — see the Code-branch repair guard above). Separately, and mandatorily, `${ISSUE_LANE[$NUM]}`/`${ISSUE_PR_BASE[$NUM]}` are now populated by the classify-lane loop immediately above this list — every queued finding is lane-classified individually, inheriting no lane assumption from the parent batch, before it reaches the standard Step 4A/4B dispatch loop.
|
|
1661
2023
|
3. **Report to user:**
|
|
1662
2024
|
```
|
|
1663
2025
|
Agent #{COMPLETED} spawned {count} new finding issues: #{A}, #{B}
|
|
@@ -1985,6 +2347,7 @@ declare -A SWEEP_PR_BASE
|
|
|
1985
2347
|
for FINDING_NUM in "${SWEEP_EXECUTE[@]}"; do
|
|
1986
2348
|
SWEEP_BASE=$(bash ~/.claude/scripts/classify-lane.sh "$FINDING_NUM" -R {GH_REPO}) || {
|
|
1987
2349
|
echo "ERROR: classify-lane.sh failed for #$FINDING_NUM — adding needs-human and skipping" >&2
|
|
2350
|
+
# GOVERNOR-exempt: intentional coordination side-effect (best-effort lease/board/finding post), DRY_RUN-safe — reviewed & accepted for the check-command-side-effects gate. Flagged only by the staging->main full-diff; passes on every feature PR. forge#2627
|
|
1988
2351
|
gh issue edit "$FINDING_NUM" -R {GH_REPO} --add-label "needs-human" 2>/dev/null || true
|
|
1989
2352
|
continue
|
|
1990
2353
|
}
|
|
@@ -2121,7 +2484,7 @@ Completion Sweep Results:
|
|
|
2121
2484
|
Token-gated — still deferred: #{H} (sweep allowance also exhausted — re-evaluable next run)
|
|
2122
2485
|
```
|
|
2123
2486
|
|
|
2124
|
-
**After sweep agents complete** (or if no findings were dispatched): output the budget deferred-issues report (if applicable), then proceed to Phase 5.
|
|
2487
|
+
**After sweep agents complete** (or if no findings were dispatched): output the budget deferred-issues report (if applicable), **call `release_orchestrator_lease()` (Step 4A-pre.-1)** — the sweep is this session's last dispatch activity, so the lease must not be left held for a now-idle session (the Termination condition's own release call, above, only fires on the no-deferred-findings branch; this is the equivalent release for the deferred-findings/Completion Sweep branch) <!-- Added: forge#2627 -->, then proceed to Phase 5.
|
|
2125
2488
|
|
|
2126
2489
|
**Anti-patterns — DO NOT DO THIS:**
|
|
2127
2490
|
- Re-sweeping findings spawned during the sweep itself — this creates unbounded recursion. Sweep is a single pass.
|
|
@@ -45,6 +45,8 @@ Include the audit summary in the final report (Phase 6). Key metrics to surface:
|
|
|
45
45
|
|
|
46
46
|
**Run after Step 5B, before Step 5D.** No-ops cleanly if the claims board was never created this run (`COORD_ISSUE_NUMBER` unset/empty — e.g. Step 3D.1's `gh issue create` failed, or this batch never reached DAG construction). Tolerates GitHub API failures without aborting the rest of Phase 5 — the same tolerant-failure convention (`2>/dev/null || echo ...`, `|| true`) used throughout `phase-3-dependency.md`/`phase-4-execution.md`.
|
|
47
47
|
|
|
48
|
+
**Orchestrator lease should already be released by this point** (forge#2627): `phase-4-execution.md`'s Termination condition (end of Step 4B) calls `release_orchestrator_lease()` before handing off to Phase 5 on a normal drain, and the "Stopping the orchestrator" procedure (Step 4A-pre.-0.5) does the same on an interrupted exit. Closing the coordination issue here does not depend on lease state either way — an unreleased lease on an issue about to be closed is harmless (a future batch creates its own new coordination issue per Step 3D.1), but if `ACTIVE_CLAIMS_NOTE` logic is ever extended to also surface lease state, prefer checking for a still-unreleased `FORGE:LEASE` the same way `ACTIVE_CLAIMS` is computed above.
|
|
49
|
+
|
|
48
50
|
```bash
|
|
49
51
|
if [ -n "${COORD_ISSUE_NUMBER:-}" ]; then
|
|
50
52
|
# Detect any still-active (unreleased) claims — a claim posted with no matching
|