forgedock 1.35.0 → 1.37.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +78 -324
- package/bin/bot-token-status.mjs +26 -0
- package/bin/engine/events.mjs +161 -0
- package/bin/engine/events.test.mjs +142 -0
- package/bin/engine/outcome-codes.mjs +39 -1
- package/bin/engine/phases.mjs +65 -5
- package/bin/engine/telemetry-seam.mjs +5 -4
- package/bin/engine/telemetry-seam.test.mjs +4 -0
- package/bin/engine-cli.mjs +372 -46
- package/bin/engine.mjs +3 -1
- package/bin/experiment.mjs +55 -0
- package/bin/forgedock.mjs +108 -5
- package/bin/hooks/reread-nudge.mjs +15 -4
- package/bin/labels.json +50 -0
- package/bin/lean/batch.mjs +52 -6
- package/bin/lean/batch.test.mjs +176 -2
- package/bin/lean/conformance.e2e.test.mjs +1 -1
- package/bin/lean/conformance.mjs +136 -11
- package/bin/lean/conformance.test.mjs +148 -1
- package/bin/lean/decisions.mjs +181 -0
- package/bin/lean/decisions.test.mjs +71 -0
- package/bin/lean/effects.mjs +32 -14
- package/bin/lean/execution.mjs +62 -16
- package/bin/lean/execution.test.mjs +59 -2
- package/bin/lean/gatewatch.mjs +31 -0
- package/bin/lean/gatewatch.test.mjs +59 -0
- package/bin/lean/guardgate.mjs +170 -0
- package/bin/lean/guardgate.test.mjs +153 -0
- package/bin/lean/guards.mjs +243 -0
- package/bin/lean/guards.test.mjs +177 -0
- package/bin/lean/heavy.mjs +54 -8
- package/bin/lean/heavy.test.mjs +74 -1
- package/bin/lean/hostkill.mjs +1 -1
- package/bin/lean/i18n.mjs +65 -0
- package/bin/lean/i18n.test.mjs +47 -0
- package/bin/lean/integration.mjs +276 -0
- package/bin/lean/integration.test.mjs +189 -0
- package/bin/lean/lane.mjs +5 -2
- package/bin/lean/lean.mjs +169 -30
- package/bin/lean/lean.test.mjs +360 -24
- package/bin/lean/lifecycle.mjs +129 -0
- package/bin/lean/lifecycle.test.mjs +118 -0
- package/bin/lean/light.mjs +1 -1
- package/bin/lean/milestonesync.mjs +130 -0
- package/bin/lean/milestonesync.test.mjs +133 -0
- package/bin/lean/orchestrate.mjs +132 -20
- package/bin/lean/parked-sweep.mjs +45 -0
- package/bin/lean/parked-sweep.test.mjs +71 -0
- package/bin/lean/postdeploy-check.mjs +181 -0
- package/bin/lean/postdeploy-check.test.mjs +206 -0
- package/bin/lean/postdeploy-monitor.mjs +97 -0
- package/bin/lean/postdeploy-monitor.test.mjs +147 -0
- package/bin/lean/precheck.mjs +6 -4
- package/bin/lean/precheck.test.mjs +17 -2
- package/bin/lean/prefilter.mjs +106 -21
- package/bin/lean/prefilter.test.mjs +86 -6
- package/bin/lean/probe.mjs +4 -3
- package/bin/lean/probe.test.mjs +1 -1
- package/bin/lean/prompts.mjs +23 -9
- package/bin/lean/resume.test.mjs +1 -1
- package/bin/lean/retry.mjs +28 -0
- package/bin/lean/sandbox-provider.contract.test.mjs +36 -3
- package/bin/lean/sandbox-remote.mjs +11 -4
- package/bin/lean/schedd.mjs +77 -13
- package/bin/lean/schedd.test.mjs +131 -5
- package/bin/lean/sequence.mjs +24 -0
- package/bin/lean/sequence.test.mjs +31 -1
- package/bin/lean/signoff.mjs +66 -0
- package/bin/lean/signoff.test.mjs +102 -0
- package/bin/lean/train.mjs +177 -0
- package/bin/lean/train.test.mjs +113 -0
- package/bin/pool/cli.mjs +2 -0
- package/bin/pool/health.mjs +30 -0
- package/bin/review-context.mjs +298 -0
- package/bin/runner.mjs +26 -3
- package/bin/stats/aggregate.mjs +1 -1
- package/bin/stats/compounding.mjs +157 -0
- package/bin/stats/compounding.test.mjs +74 -0
- package/bin/stats/experiment.mjs +44 -0
- package/bin/stats/github-source.mjs +2 -1
- package/bin/stats/index.mjs +62 -3
- package/bin/stats/release.mjs +75 -0
- package/bin/stats/release.test.mjs +68 -0
- package/commands/bootstrap.md +10 -8
- package/commands/deploy-info.md +3 -0
- package/commands/issue.md +2 -0
- package/commands/milestone.md +43 -54
- package/commands/orchestrate/phase-4-execution.md +2 -0
- package/commands/orchestrate.md +15 -3
- package/commands/pipeline-status.md +20 -0
- package/commands/review-pr.md +1 -0
- package/commands/work-on/build/implement.md +1 -1
- package/commands/work-on/build.md +4 -2
- package/commands/work-on/investigate.md +3 -1
- package/commands/work-on/remediate.md +1 -1
- package/docs/CONFIG.md +120 -2
- package/docs/EVENTS.md +65 -0
- package/docs/OUTCOME-CODES.md +9 -1
- package/docs/STATS.md +51 -0
- package/docs/TELEMETRY.md +3 -1
- package/docs/site/install.md +89 -0
- package/docs/site/start-a-new-project.md +1 -1
- package/docs/spec/label-state-machine.md +6 -0
- package/package.json +1 -1
- package/pool/install/install-agent.sh +4 -2
- package/pool/install/install-router.sh +4 -2
- package/pool/install/install-schedd.sh +28 -0
- package/pool/install/unit-hardening.sh +34 -0
- package/scripts/acceptance-red-green.sh +19 -2
- package/scripts/acceptance-red-green.test.sh +2 -0
- package/scripts/bench-lightlane.mjs +2 -1
- package/scripts/bootstrap-create.mjs +3 -2
- package/scripts/bootstrap-validate.mjs +31 -4
- package/scripts/eval-gate-scorecard.mjs +7 -0
- package/scripts/file-deferred-followups.sh +286 -77
- package/scripts/file-deferred-followups.test.sh +117 -2
- package/scripts/hotspots.sh +23 -2
- package/scripts/hotspots.test.sh +9 -0
- package/scripts/plan-validate-live.mjs +111 -0
- package/scripts/readiness.mjs +141 -0
- package/scripts/review-prepare.sh +17 -1
- package/scripts/review-prepare.test.sh +17 -0
- package/scripts/settle-scope.mjs +74 -0
- package/scripts/systemd/forgedock-bot-token-refresh.service +7 -0
- package/scripts/systemd/forgedock-bot-token-refresh.timer +11 -0
- package/scripts/wait-ci-green.sh +14 -3
- package/scripts/wait-ci-green.test.sh +3 -1
- package/templates/bootstrap/needs-human-issue.md +4 -1
- package/templates/github-actions/migration-journal-guard.test.mjs +26 -0
package/commands/orchestrate.md
CHANGED
|
@@ -102,7 +102,10 @@ remaining maintenance once no feature is pending; otherwise the rest ends `held
|
|
|
102
102
|
`--resume`; a batch started without the daemon still needs `forgedock orchestrate --resume <id>`. Change K of a running batch with
|
|
103
103
|
`forgedock orchestrate --set-parallel <id> K`. Issues filed while a batch runs join it with
|
|
104
104
|
`forgedock orchestrate --add <id> N...` (or `--resume <id> N...`), under its K and DAG; never start a second batch
|
|
105
|
-
for them.
|
|
105
|
+
for them. After a decision (a `FORGE:DECISION` posted and `needs-human` removed), a blocked issue that already ended in the batch
|
|
106
|
+
(`needs-human`, `invalid`, `skipped`, `dependency-*`, `ci-pending`) returns to it with
|
|
107
|
+
`forgedock orchestrate --requeue <id> N... --reason "..."` (refused while it is labelled `needs-human` or closed; skipped dependents
|
|
108
|
+
return to pending with it; a finished batch is reopened); `--add` on a `skipped` issue re-queues it (other ended issues still report `already in batch`). A dependency-skipped issue shows as `waiting on #X` while the batch runs (the blocker is re-checked each pass; merged or closed re-admits it, closed as not planned keeps it skipped) and ends `skipped` only if the blocker never resolved. Its pre-filter still closes already-fixed ones. The runner requeues `deferred` lanes by code (host busy and a full sandbox pool after their
|
|
106
109
|
retry-in, uncapped, a usage limit holds admissions for that account (backend identity) until the reset, across every batch on
|
|
107
110
|
the host; `forgedock limits show|clear <id>|--all` lists or drops holds; transient merge/push/idle failures back off; other
|
|
108
111
|
deferrals stop after `orchestration.max_deferrals`), relaunches an `engine-error` once, and stops admitting after
|
|
@@ -137,7 +140,8 @@ When a lane finishes (agent lanes; the batch runner applies the same rules itsel
|
|
|
137
140
|
ran (no lease, comment or worktree). Report the code and its detail (for example a missing `milestone/<slug>`
|
|
138
141
|
branch), and mark the issue's dependents skipped. Do not retry it with a different `--lane`.
|
|
139
142
|
- `dependency-blocked`: its dependency is open and labelled `needs-human`. Report it as blocked by that issue,
|
|
140
|
-
do not label it `needs-human`, and mark its dependents skipped.
|
|
143
|
+
do not label it `needs-human`, and mark its dependents skipped. Once the blocker is cleared, `forgedock orchestrate --requeue <id> N...`
|
|
144
|
+
returns the issue and its skipped dependents to the batch.
|
|
141
145
|
- `decomposed`: the builder split the issue and the engine filed sub-issues (listed in its output, each with
|
|
142
146
|
`**Parent**` and `**Depends on**` lines). Add them to the DAG, launch them, and let the parent's dependents
|
|
143
147
|
wait on the sub-issues.
|
|
@@ -159,7 +163,8 @@ When a lane finishes (agent lanes; the batch runner applies the same rules itsel
|
|
|
159
163
|
- `needs-human` with a **human code** (`builder.blocked`: product-decision, spend, credentials, promotion, customer): an
|
|
160
164
|
external action agents cannot take, or a product decision too critical for agents. It is never auto-recovered. Read
|
|
161
165
|
the issue's last comment for the reason, mark its dependents skipped, and add it to the escalation list (step 4). Only
|
|
162
|
-
explicit or logical edges skip a dependent
|
|
166
|
+
explicit or logical edges skip a dependent (after you decide the block in scope, clear the label and run
|
|
167
|
+
`forgedock orchestrate --requeue <id> N... --reason "..."`, never a second batch); an issue that merely overlaps the same file keeps running.
|
|
163
168
|
- `needs-human` with a **terminal code** that is neither recoverable nor human (for example a `conformance.*` code, or
|
|
164
169
|
`contract.invalid` without a mid-run issue edit): the runner does not recover it and it is not an escalation. The
|
|
165
170
|
batch report lists it as `unrecovered: #N (<code>: <detail>)`, like a recovery that did not merge. Mark its dependents
|
|
@@ -188,6 +193,13 @@ Escalate only these:
|
|
|
188
193
|
Collect escalations into one question list and ask it once, at the end of the batch. Never ask per issue, and keep every
|
|
189
194
|
other lane running meanwhile.
|
|
190
195
|
|
|
196
|
+
## 3b. Settle scope (#4665)
|
|
197
|
+
|
|
198
|
+
When settle begins, freeze the milestone's issue set (`scripts/settle-scope.mjs` `freezeScope`). Issues filed later are
|
|
199
|
+
classified by `classifyNew`: decomposition children of a frozen issue stay in the milestone and join the measured set;
|
|
200
|
+
deferred follow-ups go to the next milestone or the backlog; review findings and others count as growth only.
|
|
201
|
+
`convergence` measures the frozen set plus those children; a STOP prints `formatStopReport` (growth and sources).
|
|
202
|
+
|
|
191
203
|
## 4. Report
|
|
192
204
|
|
|
193
205
|
When all lanes have finished, print one table: issue, outcome, PR, minutes (issues the pre-filter closed
|
|
@@ -297,8 +297,14 @@ else
|
|
|
297
297
|
echo "$MILESTONES" | while IFS= read -r line; do
|
|
298
298
|
echo " $line"
|
|
299
299
|
done
|
|
300
|
+
forgedock integration status --repo "$GH_REPO" 2>/dev/null || true # last green full-suite run (forge#4667)
|
|
300
301
|
fi
|
|
301
302
|
|
|
303
|
+
# forge#4657: lifecycle (FORGE:LIFECYCLE in the milestone description, bin/lean/lifecycle.mjs).
|
|
304
|
+
gh api "repos/$GH_REPO/milestones?state=open&per_page=100" \
|
|
305
|
+
--jq '.[] | {t: .title, open: .open_issues, st: (first((.description // "") | capture("FORGE:LIFECYCLE[^{]*\\{[^}]*\"state\":\"(?<s>[^\"]+)\"") | .s) // null)} |
|
|
306
|
+
if (.st == "shipped:staging" or .st == "deployed:default") and .open > 0 then " SHIPPED_WITH_OPEN \(.t) state=\(.st) open=\(.open)"
|
|
307
|
+
elif .open == 0 then " EMPTY_BUT_OPEN \(.t) state=\(.st // "building")" else empty end' 2>/dev/null || true
|
|
302
308
|
echo ""
|
|
303
309
|
```
|
|
304
310
|
|
|
@@ -316,6 +322,8 @@ INVESTIGATING=$(gh issue list $GH_FLAG --state open --label "workflow:investigat
|
|
|
316
322
|
BUILDING=$(gh issue list $GH_FLAG --state open --label "workflow:building" --limit 100 --json number --jq '. | length' 2>/dev/null || echo 0)
|
|
317
323
|
IN_REVIEW=$(gh issue list $GH_FLAG --state open --label "workflow:in-review" --limit 100 --json number --jq '. | length' 2>/dev/null || echo 0)
|
|
318
324
|
BLOCKED=$(gh issue list $GH_FLAG --state open --label "needs-human" --limit 100 --json number,labels --jq '[.[] | select(.labels | map(.name) | index("workflow:decomposed") | not)] | length' 2>/dev/null || echo 0) # workflow:decomposed trackers are excluded from work counts
|
|
325
|
+
PARKED=$(gh issue list $GH_FLAG --state open --label "remediation-queue" --limit 100 --json number --jq '. | length' 2>/dev/null || echo 0) # forge#4656: mechanical parks (parked:<code>) wait for /work-on:remediate, not a human
|
|
326
|
+
EXTERNAL=$(gh issue list $GH_FLAG --state open --label "needs-human:external" --limit 100 --json number --jq '. | length' 2>/dev/null || echo 0)
|
|
319
327
|
READY=$(gh issue list $GH_FLAG --state open --label "workflow:ready-to-build" --limit 100 --json number --jq '. | length' 2>/dev/null || echo 0)
|
|
320
328
|
|
|
321
329
|
TOTAL_INFLIGHT=$((INVESTIGATING + BUILDING + IN_REVIEW + READY))
|
|
@@ -325,9 +333,21 @@ echo " ready-to-build : $READY"
|
|
|
325
333
|
echo " building : $BUILDING"
|
|
326
334
|
echo " in-review : $IN_REVIEW"
|
|
327
335
|
echo " needs-human : $BLOCKED $([ "$BLOCKED" -gt 0 ] && echo '← ACTION REQUIRED' || true)"
|
|
336
|
+
echo " needs-human:external : $EXTERNAL"
|
|
337
|
+
echo " parked (remediation-queue) : $PARKED"
|
|
328
338
|
echo ""
|
|
329
339
|
echo " Total in-flight: $TOTAL_INFLIGHT"
|
|
330
340
|
|
|
341
|
+
# forge#4656: GATE_DEADLOCK - a ship gate / hold / settle waiting on an issue the scheduler will not admit. Never poll silently.
|
|
342
|
+
if [ "$PARKED" -gt 0 ]; then
|
|
343
|
+
gh issue list $GH_FLAG --state open --label "remediation-queue" --limit 20 --json number,labels \
|
|
344
|
+
--jq '.[] | " GATE_DEADLOCK candidate: #\(.number) \([.labels[].name | select(startswith("parked:"))] | join(","))"' 2>/dev/null || true
|
|
345
|
+
echo " (a gate waiting on these emits GATE_DEADLOCK issue=#N gate=G age=Nm in the batch report; run /work-on:remediate)"
|
|
346
|
+
fi
|
|
347
|
+
|
|
348
|
+
# forge#4660: Readiness - open external prerequisites per gate, plus the dependency-ordered digest.
|
|
349
|
+
for A in "" --digest; do node "${FORGEDOCK_HOME:-${CLAUDE_PLUGIN_ROOT}}/scripts/readiness.mjs" $GH_FLAG $A 2>/dev/null; done
|
|
350
|
+
|
|
331
351
|
if [ "$BLOCKED" -gt 0 ]; then
|
|
332
352
|
echo ""
|
|
333
353
|
echo " ⚠ $BLOCKED issue(s) need human attention:"
|
package/commands/review-pr.md
CHANGED
|
@@ -509,6 +509,7 @@ For each agent in `PENDING_AGENTS` build its prompt from the persona template (v
|
|
|
509
509
|
1. Extract its template from the persona file
|
|
510
510
|
2. `[PR_NUMBER]`, `[REVIEW_SHA]`, `[REVIEW_SHA_SHORT]`, `[AGENT_DOMAIN]` (canonical marker domain from the pre-dispatch fence's `agent_domain`; it keys the slice path), `[TITLE]`, `[FILE_LIST]` (Domain Diff Slicing)
|
|
511
511
|
3. `[PROJECT_NAME]` / `[PROJECT_CONTEXT]` → `config.project_name` / `config.project_context`; `[DOMAIN_CONTEXT]` → `config.domains.<domain>` (`scraping` for the scraper, which only runs when `config.scraping_configured`)
|
|
512
|
+
3a. When `config.context_generated_at` is non-empty, append `Project context generated: {config.context_generated_at}` after `[PROJECT_CONTEXT]`. When `config.context_advisory` is `true` (more than N merged PRs since, `review.context_max_merged_prs`, default 50), also append "The project context is older than N merged PRs: treat it as advisory and verify architecture claims against the code." When `phase2.findings` holds a `REVIEW_CONTEXT_STALE:` line (stale tech stack, superseded decision, or docs-only phrasing), print one line `⚠ REVIEW_CONTEXT_STALE: <reason>; run npx forgedock init --refresh-context` at the top of the review output and tell agents the same. <!-- Added: forge#4666 -->
|
|
512
513
|
4. `[CHURN_CONTEXT]` → the printed `CHURN_CONTEXT` (one PR-level value for every agent)
|
|
513
514
|
5. `[INDEX_SLICE]` → `index_slices.<domain>` (`auth`, `billing`, `database`, `api`, `frontend`, `infrastructure`; other agents have none). Agents query the index first and grep only when it is empty.
|
|
514
515
|
6. `[DOMAIN_DIFF_SLICE]` → the content of `slices.<domain>.diff_file` (written by the script, capped at 100K chars). Persona templates and `protocols.md` §1 materialize it into `$SLICE_FILE` (`${TMPDIR:-/tmp}/forge-slice-[PR_NUMBER]-[REVIEW_SHA_SHORT]-[AGENT_DOMAIN].diff`, re-declared in every fence that reads it) and grep that file instead of calling `gh pr diff [PR_NUMBER]` (only `--name-only` is allowed).
|
|
@@ -171,7 +171,7 @@ Work in `{WORKTREE_PATH}`. Follow the contract deliverables table exactly — im
|
|
|
171
171
|
- **Scratch placement (venvs, caches)**: create virtualenvs, package caches and other large scratch data under `"$TMPDIR/forge-venv-{NUMBER}"` (the engine exports a per-run, disk-backed `TMPDIR`) or inside the worktree. Never use a literal `/tmp/<name>` path: a size-limited `/tmp` fails concurrent lanes with EDQUOT/ENOSPC. Remove what you created when the run finishes. <!-- Added: forge#3572 -->
|
|
172
172
|
- **Worktree-aware path derivation**: When writing shell code that derives repository paths, ALWAYS use `git rev-parse --show-toplevel` for the repo root in regular checkouts, or `git rev-parse --git-common-dir` (then `dirname`) to get the shared `.git` directory when the context may be a linked worktree. For worktree cleanup code specifically, ALWAYS use `--git-common-dir` — `--show-toplevel` returns the worktree path itself, NOT the main repo root. NEVER use `pwd`, relative paths, or `dirname` chains on `--show-toplevel` output for repo root derivation. Test path logic for both regular checkouts and linked worktrees. (Ref: review-findings #104, #105 — 4 PATH_DERIVATION defects, 15% of all review findings)
|
|
173
173
|
- **State machine completeness verification**: When implementing routing logic, state transition tables, or phase dispatch code (e.g., adding a `Skill()` call in a routing loop, adding a new phase to a state machine, creating a new subcommand file), you MUST run three checks BEFORE staging: (1) **Routing target existence** — for each `Skill("subcommand", ...)` call or file reference in routing logic, verify the target file exists at `commands/{subcommand-path}.md`; (2) **State reachability** — for each declared state or phase, verify at least one prior state or entry condition in the router routes to it; (3) **Subcommand invocation wiring** — for each `commands/work-on/*.md` file that declares its invocation condition (e.g., `Invoked by work-on.md routing loop, when X`), verify that condition is actually handled in the router. A missing target file, an unreachable state, or a declared-but-unwired subcommand will not surface until review. (Ref: review-findings #85, #116, #137 — 4 ROUTING defects, 15% of all review findings)
|
|
174
|
-
- **Migration safety checklist** *(trigger: diff includes `*.sql` files or files under a `migrations/` path)*: When any migration file is in the diff, verify ALL of the following BEFORE staging. Fix each violation inline — do not defer to review or the quality gate: (a) **Rollback file**: a corresponding down/rollback migration file exists (e.g. `0042_down_*.sql`, `rollback_*.sql`) or the migration is explicitly self-reversing (DROP of a previously-added column); (b) **NOT NULL safety**: any `ADD COLUMN ... NOT NULL` either includes a `DEFAULT` clause or is preceded by a backfill step — a NOT NULL column without DEFAULT locks the table and fails on existing rows; (c) **Constraint name consistency**: constraint names in the migration match ORM model declarations — a mismatch causes `alembic stamp` and FK introspection to silently diverge; (d) **CREATE TRIGGER idempotency**: every `CREATE TRIGGER` uses `CREATE OR REPLACE TRIGGER` or is preceded by `DROP TRIGGER IF EXISTS` — a bare `CREATE TRIGGER` fails on re-run in test and CI environments; (e) **Migration prefix uniqueness**: confirm the new file's numeric prefix does not already exist in the `infra/migrations/` tree (`ls infra/migrations/*.sql | xargs -n1 basename | grep -oE '^[0-9]+' | sort | uniq -d` prints duplicates) — a duplicate prefix hard-fails the deploy gate regardless of file content. <!-- Added: forge#373 -->
|
|
174
|
+
- **Migration safety checklist** *(trigger: diff includes `*.sql` files or files under a `migrations/` path)*: When any migration file is in the diff, verify ALL of the following BEFORE staging. Fix each violation inline — do not defer to review or the quality gate: (a) **Rollback file**: a corresponding down/rollback migration file exists (e.g. `0042_down_*.sql`, `rollback_*.sql`) or the migration is explicitly self-reversing (DROP of a previously-added column); (b) **NOT NULL safety**: any `ADD COLUMN ... NOT NULL` either includes a `DEFAULT` clause or is preceded by a backfill step — a NOT NULL column without DEFAULT locks the table and fails on existing rows; (c) **Constraint name consistency**: constraint names in the migration match ORM model declarations — a mismatch causes `alembic stamp` and FK introspection to silently diverge; (d) **CREATE TRIGGER idempotency**: every `CREATE TRIGGER` uses `CREATE OR REPLACE TRIGGER` or is preceded by `DROP TRIGGER IF EXISTS` — a bare `CREATE TRIGGER` fails on re-run in test and CI environments; (e) **Migration prefix uniqueness**: confirm the new file's numeric prefix does not already exist in the `infra/migrations/` tree (`ls infra/migrations/*.sql | xargs -n1 basename | grep -oE '^[0-9]+' | sort | uniq -d` prints duplicates) — a duplicate prefix hard-fails the deploy gate regardless of file content; (f) **Journal `when` order (parallel milestones)**: with a drizzle-style `meta/_journal.json`, every entry your branch adds must have `when` above the max `when` on `origin/{PR_BASE}` — newest-only migrators (drizzle `migrate()`) silently skip an older entry on a persistent database while fresh-DB CI passes; re-stamp through the generator (same SQL, regenerated snapshot) if not. <!-- Added: forge#373 -->
|
|
175
175
|
- **Cross-lane import guard**: Before adding any `import` or `from X import Y` statement for a service-internal module (`app.*`), verify the module exists on the PR's base branch (`{PR_BASE}`) — NOT just on your local disk or a milestone branch. Run `git show origin/{PR_BASE}:{module_path}.py` (replacing dots with slashes) to confirm. If the module only exists on a milestone branch, do NOT import it — find an alternative implementation or make the import conditional with a `try/except ImportError` fallback. A milestone-only import on a fast-lane PR will crash production on every request with `ModuleNotFoundError`. <!-- Added: forge#277 -->
|
|
176
176
|
- **Deliverable-type consistency check**: Before committing, compare the actual output against the CONTRACT's deliverable list. If the CONTRACT explicitly states "no code changes required", "docs only", or "configuration update only" AND the diff introduces new executable files (`.py`, `.js`, `.ts`, `.sh` — not test, config, or documentation files), STOP. Do NOT commit. Re-read the CONTRACT, the investigation recommendation, and the ARCHITECT plan. If all three agree that code is needed, update the CONTRACT comment to reflect the new deliverable type before proceeding. If only the builder decided to add code without contract support, discard the code change and implement the contracted deliverable instead. <!-- Added: forge#279 -->
|
|
177
177
|
- **Pipeline check documentation — generalization rule**: When writing or updating pipeline check documentation (in `commands/*.md`), describe the **bug class**, not a specific incident. Do NOT embed: PR numbers, issue numbers, run IDs, timestamps, function names, dollar amounts, or multi-sentence incident timelines in check prose or `**Evidence**:` blocks. One brief HTML comment `<!-- Added: forge#NNN -->` is acceptable per check for traceability. `**Evidence**:` blocks must describe the vulnerability pattern (what the class of bug looks like, why it's dangerous) — not narrate a single historical occurrence. CHANGELOG entries may reference originating issues; command prompt text must not.
|
|
@@ -376,13 +376,15 @@ Every item in the contract's `### Out of Scope` section must carry exactly one d
|
|
|
376
376
|
- **Known-flaw rule**: a contract that keeps a known-flawed design (a design the investigation or architect plan identified as flawed, a fail-open caller, an unsound threshold) must state the flaw and list it as a `deferred → #N` item with a linked follow-up. It may not hide behind `accepted-risk` or free text.
|
|
377
377
|
- Free-text exclusions without a disposition are invalid. `->` is accepted for `→`.
|
|
378
378
|
|
|
379
|
-
**Fold, then batch** <!-- Added: forge#3843 -->: before filing anything, fold small siblings into scope. A candidate deferred item that is the same mechanical change at a sibling path and fits the remaining scope budget (a small fraction of the plan's own estimate: at most 20% of the current `ESTIMATED_DIFF_LINES`, or 200 lines when no estimate exists) goes into the contract's Deliverables, not Out of Scope; count it as `folded`. Defer only genuinely separate work (different module, design or owner). All remaining deferred items of this build go into ONE follow-up issue with a checklist; `scripts/file-deferred-followups.sh` appends to an existing trusted follow-up for the same parent (exact path match; a closed one is reopened) or creates one, then records a `<!-- FORGE:DEFERRED_ISSUE: #M -->` claim marker on the parent.
|
|
379
|
+
**Fold, then batch** <!-- Added: forge#3843 -->: before filing anything, fold small siblings into scope. A candidate deferred item that is the same mechanical change at a sibling path and fits the remaining scope budget (a small fraction of the plan's own estimate: at most 20% of the current `ESTIMATED_DIFF_LINES`, or 200 lines when no estimate exists) goes into the contract's Deliverables, not Out of Scope; count it as `folded`. Defer only genuinely separate work (different module, design or owner). All remaining deferred items of this build go into ONE follow-up issue per class (engineering | post-deploy | external | ops, each labelled `followup:<class>` and carrying the parent's milestone, #4664) with a checklist; `scripts/file-deferred-followups.sh` appends to an existing trusted follow-up for the same parent (exact path match; a closed one is reopened) or creates one, then records a `<!-- FORGE:DEFERRED_ISSUE: #M -->` claim marker on the parent.
|
|
380
380
|
|
|
381
381
|
**File the `deferred` follow-ups** (skip the filing on resume when the contract already exists):
|
|
382
382
|
|
|
383
383
|
```bash
|
|
384
384
|
# <Script resolution block from work-on/build.md, verbatim>
|
|
385
|
-
# DEFERRED_TSV: one line per remaining deferred item: path
|
|
385
|
+
# DEFERRED_TSV: one line per remaining deferred item: path-or-`-`<TAB>one-line why<TAB>known-flaw (or empty)[<TAB>class<TAB>label]
|
|
386
|
+
# Column 1 must be a path or `-` (never requirement text or an `R<n>:` prefix: that exits 2, nothing filed); a requirement-level item
|
|
387
|
+
# is `-` with its text in the label column. class: engineering|post-deploy|external|ops (default: the why's `post-deploy:`/`external:`/`ops:` prefix, else engineering).
|
|
386
388
|
SCRIPT_REF=$(resolve_script 'file-deferred-followups')
|
|
387
389
|
case "$SCRIPT_REF" in
|
|
388
390
|
prose:*) echo "DEFERRED_SCRIPT=PROSE" ;; # file ONE issue by hand: marker + checklist of every path (see below)
|
|
@@ -553,7 +553,7 @@ bash {REPO_PATH}/scripts/code-index.sh query --domain {DOMAIN_LABEL} --repo-path
|
|
|
553
553
|
Any hit here is a candidate prior fix or reintroduced defect — read the commit body (`git show {hash}`) to confirm before citing it. Feed confirmed hits into the History findings field and let them inform the verdict (e.g. a defect being reintroduced raises severity).
|
|
554
554
|
6. **Determine root cause** — what's actually broken or missing?
|
|
555
555
|
7. **Identify affected files** — full list of files that need changes
|
|
556
|
-
7.5. **Sibling Pattern Sweep** *(conditional — when the bug is a condition, gated function call, or field presence check)*: After identifying the affected files, grep for the same pattern in sibling files within the same directory. The issue spec may name only the file where the error was first observed — but the same commit or PR that introduced the bug often applied it uniformly across related handlers.
|
|
556
|
+
7.5. **Sibling Pattern Sweep** *(conditional — when the bug is a condition, gated function call, or field presence check, or a CONFIRMED wrong/non-existent route, URL or link)*: After identifying the affected files, grep for the same pattern in sibling files within the same directory. The issue spec may name only the file where the error was first observed — but the same commit or PR that introduced the bug often applied it uniformly across related handlers.
|
|
557
557
|
```bash
|
|
558
558
|
# Identify the broken condition or gated function call from the issue
|
|
559
559
|
# Then search sibling files in the same router/service directory
|
|
@@ -563,6 +563,8 @@ bash {REPO_PATH}/scripts/code-index.sh query --domain {DOMAIN_LABEL} --repo-path
|
|
|
563
563
|
**If identical patterns are found in files NOT listed in the issue spec**, output a scope-gap warning:
|
|
564
564
|
> **Scope-Gap Warning**: The issue spec lists `{PRIMARY_FILE}` but the same pattern exists in `{SIBLING_FILE}:{LINE}`. These were likely introduced together. Recommend widening scope to fix all callers in this PR, or creating follow-up issues for the other files before proceeding.
|
|
565
565
|
|
|
566
|
+
**Hard-coded route/URL/link findings**: grep the finding's file and its template siblings for every other constructed URL (`f"{base}/path"`, `href=`, `redirect(`), verify each against the route table, and either fix those found broken or list them in a follow-up. Fixing only the cited link lets its siblings ship broken.
|
|
567
|
+
|
|
566
568
|
Do NOT silently exclude sibling matches. The appropriate output when sibling files have the same bug is to flag them explicitly — even if the issue spec's silence appears intentional. The fix-approach validation step (step 8) will confirm whether to widen scope or create follow-ups. <!-- Added: forge#383 -->
|
|
567
569
|
7.6. **Finding Pattern Sweep** *(conditional — eligible when the issue has label `review-finding`, or its body carries a `FORGE:PATTERN` or `FORGE:PATTERN-CLASS` tag)*: Review findings are instances of a defect class. Fixing only the cited instance lets sibling instances resurface in the next review, so sweep the whole repo for the class before settling the affected-file list. For eligible issues this step replaces step 7.5; step 7.5 runs for non-eligible issues, and as the fallback when 7.6 is skipped or degrades. <!-- Added: forge#3449 -->
|
|
568
570
|
```bash
|
|
@@ -80,7 +80,7 @@ ISSUE_STATE=$(gh issue view {ISSUE_NUMBER} {GH_FLAG} --json labels,state,body,mi
|
|
|
80
80
|
ISSUE_LABELS=$(echo "$ISSUE_STATE" | jq -r '[.labels[].name] | join(",")')
|
|
81
81
|
```
|
|
82
82
|
|
|
83
|
-
- If `PR_ALREADY_MERGED` is not `true` and none of `needs-human`, `workflow:remediating` or `workflow:awaiting-merge` is among `ISSUE_LABELS` → EXIT `REMEDIATE_RESULT: status: BLOCKED`, blocker: "issue #{ISSUE_NUMBER} is none of `needs-human`, `workflow:remediating` or `workflow:awaiting-merge` — remediation mode only targets PRs in one of those states; use the normal `/work-on {ISSUE_NUMBER}` resume path instead." This keeps blast radius scoped to exactly the gap this mode fills — it is not a general-purpose re-review trigger. `workflow:remediating` is the non-human state review sets when it hands an autonomous ci-gate / in-pr-fix / base-sync fix to this phase (forge#3541); a bare `needs-human` (older runs, genuine escalations being re-driven) is still accepted. <!-- Added: forge#3541 -->
|
|
83
|
+
- If `PR_ALREADY_MERGED` is not `true` and none of `needs-human`, `remediation-queue`, `workflow:remediating` or `workflow:awaiting-merge` is among `ISSUE_LABELS` → EXIT `REMEDIATE_RESULT: status: BLOCKED`, blocker: "issue #{ISSUE_NUMBER} is none of `needs-human`, `workflow:remediating` or `workflow:awaiting-merge` — remediation mode only targets PRs in one of those states; use the normal `/work-on {ISSUE_NUMBER}` resume path instead." This keeps blast radius scoped to exactly the gap this mode fills — it is not a general-purpose re-review trigger. `workflow:remediating` is the non-human state review sets when it hands an autonomous ci-gate / in-pr-fix / base-sync fix to this phase (forge#3541); a bare `needs-human` (older runs, genuine escalations being re-driven) is still accepted. <!-- Added: forge#3541 --> `parked:<code>` / `remediation-queue` (forge#4656) is the single remediation queue of mechanical terminal parks and is accepted like `needs-human`; on a FIXABLE run remove `parked:<code>` and `remediation-queue` together with `needs-human`.
|
|
84
84
|
- **Resume-at-M7 (forge#3647, hardened forge#3684)**: `workflow:awaiting-merge` is admitted ONLY as a resume of an interrupted M7 auto-land: the PR is OPEN, `needs-human` and `workflow:remediating` are both absent, the idempotency check below finds NO completed `FORGE:REMEDIATION:COMPLETE` trail (`REMEDIATION_COMPLETE_N` = 0), AND there is evidence the PR was engine-parked for the M7 bar rather than parked for a human: the live base is non-empty, non-`null` and not the deploy gate (literal `main`, the GitHub `default_branch`, or the default branch's committed `branches.default`; never read from the PR worktree, forge#3753), and the LATEST trusted (OWNER/MEMBER/COLLABORATOR or Bot) `review-pr.md` Phase 8 park comment on the issue is the non-default-branch "parked at `workflow:awaiting-merge` for the remediation auto-land bar" comment AND names this PR (`PR #{PR_NUMBER} was previously escalated`, so a park for a different PR never admits the resume; M7 then binds approval to the live head SHA, not a timestamp). When all hold, set `RESUME_AT_M7=true`: skip Phases M1-M6 entirely (no checkout, no fixes, no re-review) and continue at Phase M7, which re-verifies the bar from scratch. Without park evidence, return `BLOCKED` (blocker: "awaiting-merge PR is not engine-parked for the remediation auto-land bar"); the issue stays at `workflow:awaiting-merge`. A COMPLETE trail still wins via the `ALREADY_DONE` guard below. A resume that ends `HELD-AWAITING-MERGE` (CI pending, stale approval head, deploy-gate base) writes NO `FORGE:REMEDIATION:COMPLETE` (Phase M8 posts a `FORGE:REMEDIATION_HELD` record instead), so the next invocation re-enters this same resume path rather than exiting `ALREADY_DONE` (forge#3685). On a BLOCKED exit from this path the issue stays at `workflow:awaiting-merge` (it never gains `workflow:remediating`).
|
|
85
85
|
- **Stranded-state rule**: `workflow:remediating` must never outlive this run. Every exit that returns `BLOCKED` or `UNFIXABLE` (including exits before Phase M1; except a `RESUME_AT_M7` run, which leaves the issue at `workflow:awaiting-merge` untouched, and except a `deferred-live-run:` exit, which leaves every label untouched: a live run owns the issue) removes `workflow:remediating` and adds `needs-human`, because at that point a human really is needed:
|
|
86
86
|
|
package/docs/CONFIG.md
CHANGED
|
@@ -322,6 +322,22 @@ orchestration:
|
|
|
322
322
|
# (stalled, needs a human) instead of IN_PROGRESS. Default: 48.
|
|
323
323
|
in_progress_max_hours: 48
|
|
324
324
|
|
|
325
|
+
# Extra env var names (comma-separated) a batch pins for its lanes in plan.json,
|
|
326
|
+
# beyond the built-in allowlist (PATH, HOME, USER, SHELL, LANG, LC_*, TMPDIR,
|
|
327
|
+
# CLAUDE_CONFIG_DIR, CLAUDE_CODE_USE_*, ANTHROPIC_BASE_URL, ANTHROPIC_MODEL,
|
|
328
|
+
# ANTHROPIC_SMALL_FAST_MODEL, ANTHROPIC_DEFAULT_*_MODEL, FORGEDOCK_*, GH_*, GITHUB_*;
|
|
329
|
+
# secret-looking names and the launching session's other CLAUDE_CODE_*, CLAUDE_PID,
|
|
330
|
+
# ANTHROPIC_* are never pinned unless listed here; the configured execution.token_env
|
|
331
|
+
# name is also kept). Submit prints one line naming the CLAUDE_*/ANTHROPIC_*/GH_*/
|
|
332
|
+
# GITHUB_*/FORGEDOCK_* keys left out (names only).
|
|
333
|
+
# Inspect or edit the pinned env with `forgedock orchestrate --show-env <id>`,
|
|
334
|
+
# `--set-env <id> KEY=VAL`, `--unset-env <id> KEY` (applies to newly launched lanes).
|
|
335
|
+
env_passthrough: ANTHROPIC_BASE_URL
|
|
336
|
+
|
|
337
|
+
# Globs (comma-separated) of PATH directories removed from the pinned PATH. Non-existent
|
|
338
|
+
# dirs and per-session temp dirs (an arg0 or claude* dir under a temp root) are always dropped.
|
|
339
|
+
path_exclude: /opt/wrappers/*
|
|
340
|
+
|
|
325
341
|
cascade:
|
|
326
342
|
# Named preset expanding to the granular keys below. Any granular key set
|
|
327
343
|
# explicitly overrides just that field — the preset supplies defaults, not
|
|
@@ -568,7 +584,7 @@ engine:
|
|
|
568
584
|
| `cli_busy_minutes` | number | No | Extended idle window for a core builder (#4405). When the idle timeout is reached but the session has a live heavy step (a heavy-shim step, test runner, linter or type checker child) or a queued or running heavy-queue entry of its own processes, the kill waits until `cli_idle_minutes` + this many minutes of silence. The run log records `CLI_IDLE_EXTENDED` once per silent stretch, and a later `CLI_IDLE_KILL` carries `heavyQueue` ({queued, running}) and `extended: true`. A silent session with no such child is still killed at `cli_idle_minutes`. Default 45 (the heavy queue's 30-minute maximum wait plus the 15-minute idle default). |
|
|
569
585
|
| `cli_max_minutes` | number | No | Absolute ceiling for a core builder session (code `cli.timeout`), replacing the fixed 60-minute kill. A session that keeps making progress is never idle-killed. Default 180. On either kill, uncommitted work is committed as `wip(forge): checkpoint after <reason>` and the lane branch is pushed (with the push retry), the builder retries once from the checkpoint, and a second kill ends `deferred`; a rerun resumes from the branch. |
|
|
570
586
|
| `turn_budget_s` / `turn_budget_m` / `turn_budget_l` | number | No | Core implement turn budget per lane size (#4392). Size comes from a `size:S|M|L` issue label, else the frozen contract's distinct deliverable paths (S ≤ 2, M ≤ 5, else L). The build call and its context/timeout restarts share the budget; at the limit the session is stopped, checkpointed, and one fresh wrap-up session of max(5, budget/5) turns either finishes or writes `status: budget-exceeded` with a digest. A stop or overrun ends with the non-human `budget-exceeded` reason and code `budget.exceeded` (no `needs-human` label, forge#4505), which `forgedock orchestrate` recovers once (continue from the checkpoint or split); a batch that does not recover it labels it `needs-human` once. Lane results carry `turns: {size, budget, build, total, exceeded}`, and the batch report prints `turns by size`. Defaults 25 / 60 / 100; 0 disables that size. An L lane's budget scales with the contract: +10 turns per distinct deliverable path above 5 (capped at +2x `turn_budget_l`), and a `budget.exceeded` recovery of an L lane gets a further +50% of `turn_budget_l` (#4587). |
|
|
571
|
-
| `lane_limits` | map | No | Cap each core lane's whole process tree (#4115), under `pipeline:` or `engine:`: `memory_max` (for example `3G`) and `cpu_quota` (for example `150%`).
|
|
587
|
+
| `lane_limits` | map | No | Cap each core lane's whole process tree (#4115, #4690), under `pipeline:` or `engine:`: `tasks_max` (default `1024`; `0`/`off` disables), `memory_max` (for example `3G`) and `cpu_quota` (for example `150%`). Where `systemd-run --user --scope` works (probed once) every lane's builder CLI is spawned as `systemd-run --user --scope -q --unit=forgedock-lane-... -p TasksMax=... -p MemoryMax=... -p CPUQuota=... -- claude ...` by default. `memory_max` has no default: lanes get no `MemoryMax` unless it is set (`off` also disables; #4740). A lane that hits `TasksMax` or `MemoryMax` is stopped and ends `needs-human` with `env.lane_limit` (not an engine-error, so the breaker ignores it). Where scopes are unavailable the run logs one line saying lanes are uncapped and the limits are ignored. Write it nested or flat (`lane_limits_tasks_max`, `lane_limits_memory_max`, `lane_limits_cpu_quota`). While a builder runs the engine polls host load and memory every 15 s against the `admission` thresholds: over them it SIGSTOPs the lane's process group, under them SIGCONT, and never pauses longer than 10 minutes. |
|
|
572
588
|
| `admission` | map | No | Host-resource admission for the core work-on (#4105), checked before the lease, worktree or any GitHub call. When a check fails the run ends `deferred` (code `host.busy`, exit 75) with a last line such as `issue #N → deferred (host busy: mem 1.2G < 2G; retry in 60s)`; no lease is taken and nothing is posted. Keys: `enabled` (default `true`; `false` disables admission), `min_free_mem_mb` (default 2048; `MemAvailable` from `/proc/meminfo`, else `os.freemem()`), `min_free_tmp_mb` (default 1024; free space of the run TMPDIR filesystem, `~/.forge/tmp`, and of `/tmp`, via `statfs`), `max_load_per_core` (default 4; 1-minute load average divided by cores). Write it nested (`engine:` then `admission:` with the keys indented) or flat as `admission_enabled`, `admission_min_free_mem_mb`, `admission_min_free_tmp_mb`, `admission_max_load_per_core`; the parser reads one nested level into `engine`. `/orchestrate` requeues a host-busy lane and lowers K by 1 for 5 minutes. |
|
|
573
589
|
| `merge_wait` | string | No | Core work-on merge gate. `required` (default): wait only for the status checks the base branch requires: the union of classic branch protection (`gh api repos/{o}/{r}/branches/{base}/protection/required_status_checks`, contexts and checks) and repository rulesets (`gh api repos/{o}/{r}/rules/branches/{base}`, `required_status_checks` rules); with neither (404 or empty) merge right after review PASS. A merge rejected by base branch policy (PR `MERGEABLE` but `BLOCKED`) is waited on and retried within the CI budget, then left `ci-pending`; it is never base-synced and never `base.conflict`. A failing required check is `needs-human`; a pending one is polled with backoff and never `needs-human`. `all`: wait for every check (`scripts/wait-ci-green.sh`). `none`: merge right after review. Always `--match-head-commit` at the reviewed head. The policy used is recorded in the `FORGE:WORK-ON` comment. |
|
|
574
590
|
| `context_pack` | string | No | Context pack (#4275). Before the plan call the core compiles the issue's GitHub/git neighbourhood into a small, budgeted pack: linked issues and PRs (parent, `Depends on`, body links, same-repo cross-references, siblings), prior attempts, merged PRs on the issue's files that were reverted or whose decisions/findings name a symbol from the issue, structured `FORGE:FACTS` records of earlier lanes on the issue's files (rejected approaches with why, hot spots, decisions, conventions, baselines; #4401), and repo-fact file pointers (found in the lane worktree or, for untracked caches, the main checkout; #4400). `shadow` (default): the pack is compiled, posted on the issue as a collapsed `FORGE:CONTEXT` comment and logged as a `CONTEXT_PACK` event; the builder prompt is unchanged. `on`: it is also appended to the plan prompt as hints. `off`: nothing is gathered, posted or logged. `ab` (#4444): the A/B experiment. Each issue gets a stable arm, `on` or `shadow`, from a hash of `repo#issue`, logged as that mode with `context_ab: true`; the default stays `shadow`. The pack never enters the text contract quotes are checked against. Values read from linked issues' `FORGE:STATE` are kept only when they are outcome codes. Compare with `forgedock stats --context-ab` (per lane type and mode: first-pass yield, turns and cache-read per merge, rework rate, and recovery lanes' implement attempt against the first run on the same issue). Unless `off`, a resumed or recovery lane also gets a bounded resume digest of its last run in the builder prompt (end reason, validate results per command, deliver requirements, files changed; logged as `RESUME_DIGEST`; #4400), whatever the mode. |
|
|
@@ -594,6 +610,63 @@ engine:
|
|
|
594
610
|
|
|
595
611
|
---
|
|
596
612
|
|
|
613
|
+
## `followups` (OPTIONAL)
|
|
614
|
+
|
|
615
|
+
Deferred follow-ups (`scripts/file-deferred-followups.sh`) are filed as one issue per class per parent: `engineering`, `post-deploy`, `external` and `ops`. Each carries the label `followup:<class>` (`external` also `external-prerequisite`, and states the parent it is a prerequisite of) and inherits the parent's milestone. A requirement may set an optional `class` in the contract; otherwise `post-deploy` items are `post-deploy` and the rest `engineering`. A parent milestone that has shipped (closed) is replaced by the next open milestone by due date, else `followups.default_milestone`.
|
|
616
|
+
|
|
617
|
+
```yaml
|
|
618
|
+
followups:
|
|
619
|
+
default_milestone: "Backlog" # optional: milestone title used when the parent's milestone has shipped and no later one is open
|
|
620
|
+
```
|
|
621
|
+
|
|
622
|
+
## `post_deploy` (OPTIONAL)
|
|
623
|
+
|
|
624
|
+
Deferred follow-ups whose items are all post-deploy checks are parked (`awaiting-promotion`, no lane) until the parent's fix is on `branches.default`. Keys: `precondition_cmd` (exit 0 ready, 1 parked) and `min_age` (soak window, e.g. `24h`).
|
|
625
|
+
|
|
626
|
+
A post-deploy item may carry `due:YYYY-MM-DD` (`- [ ] \`R2: …\` — post-deploy: [check:id] due:2026-11-03 why`): it parks `awaiting-due` (with a `due:<date>` label) until that UTC date. A `decision:` item parks with one typed `FORGE:OWNER_ASK` comment and no lane. The daemon timer (and `forgedock probe --due`) re-evaluates parked follow-ups (`awaiting-promotion`, `awaiting-due`, `awaiting-measurement`) and removes the wait labels when the parent's fix is on the default branch, the date has passed or the check is defined. The only production access a check has is the checks file's command and the `acceptance_probe.env` allowlist; a pass ticks the items, a fail files one `followup:engineering` issue per failing check, and an item with no runnable check gets the owner ask. The batch report lists `due date:` and `owner ask:` apart from failures.
|
|
627
|
+
|
|
628
|
+
### Named checks (#4626)
|
|
629
|
+
|
|
630
|
+
A post-deploy requirement carries a check id (`[check:<id>]` in the follow-up item). The command is read from `.forge/post-deploy-checks.json` on the default branch (`git show origin/<default>:...`), never from an issue body or comment:
|
|
631
|
+
|
|
632
|
+
```json
|
|
633
|
+
{ "login-smoke": { "run": "scripts/smoke.sh", "pass": "exit 0", "timeout": 120 } }
|
|
634
|
+
```
|
|
635
|
+
|
|
636
|
+
`pass` is `exit 0` (default) or a `/regex/flags` over the output; `timeout` is seconds. When the gate reports the follow-up ready, the check runs with no model call: pass closes the follow-up with a `FORGE:PROBE` record, fail reopens the parent with the output. An item with no check id, or an id the default branch does not define, gets the `awaiting-measurement` label and never a lane. A contract's check id must be on the default branch or added by the same PR.
|
|
637
|
+
|
|
638
|
+
Closing an experiment follow-up (#4634): add a check that runs `forgedock stats --experiment <id> --require-met` (exit 0 only when every target of the forge.yaml `experiment:` is met; n/a below min n is not met), and put `[check:<check id>]` in the follow-up item:
|
|
639
|
+
|
|
640
|
+
```json
|
|
641
|
+
{ "exp-light-vs-full": { "run": "forgedock stats --experiment light-vs-full --require-met" } }
|
|
642
|
+
```
|
|
643
|
+
|
|
644
|
+
Met targets pass the check and close the follow-up; with `for`/`every` it is re-measured through the window. The probe env is minimal: list the GitHub token variable in the forge.yaml `acceptance_probe.env` allowlist so stats can read telemetry.
|
|
645
|
+
|
|
646
|
+
### Windowed monitoring (#4627)
|
|
647
|
+
|
|
648
|
+
A check may declare `for` (the window, `30m`/`24h`/`7d`/`2w`) and `every` (the cadence, default `6h`; needs `for`); an entry with an invalid duration, or `every` without `for`, is ignored:
|
|
649
|
+
|
|
650
|
+
```json
|
|
651
|
+
{ "login-smoke": { "run": "scripts/smoke.sh", "for": "7d", "every": "6h" } }
|
|
652
|
+
```
|
|
653
|
+
|
|
654
|
+
When every item's check is windowed, the first pass records the release (the `origin/<default>` sha) and the window end in a `FORGE:PROBE` record, labels the follow-up `post-deploy-monitoring` and leaves it open (no lane, no second measurement by the pre-filter). It re-runs, with the command read from the default branch each time, when `every` has elapsed or `origin/<default>` moved (a later release). At the window end it closes. A fail after a pass opens a regression issue (`FORGE:REGRESSION`) naming the fix commit, the last passing release and the breaking release, and closes the follow-up; the parent is not reopened. A fail on the first run behaves as above (the parent reopens). No model call.
|
|
655
|
+
|
|
656
|
+
Triggers: `forgedock probe --due [--dry-run] [--repo owner/repo]` runs the due ones once; the `forgedock schedd` daemon runs them on a timer while it is up (`FORGEDOCK_SCHEDD_DUE_MS`, default 15 minutes).
|
|
657
|
+
|
|
658
|
+
|
|
659
|
+
## `events` (OPTIONAL)
|
|
660
|
+
|
|
661
|
+
Outbound lifecycle events (#4630). The local JSONL sink is always on; the webhook needs both keys. See [EVENTS.md](EVENTS.md) for the schema and signature.
|
|
662
|
+
|
|
663
|
+
```yaml
|
|
664
|
+
events:
|
|
665
|
+
webhook_url: https://hooks.example.com/forgedock # https, or http on localhost; unset = no webhook
|
|
666
|
+
webhook_secret_env: FORGE_EVENTS_SECRET # NAME of the env var holding the HMAC secret (default shown); never put the secret here
|
|
667
|
+
dir: ~/.forge/events # local JSONL directory
|
|
668
|
+
```
|
|
669
|
+
|
|
597
670
|
## `lean` (OPTIONAL)
|
|
598
671
|
|
|
599
672
|
Settings of the core work-on (`bin/lean/`) that change its contract.
|
|
@@ -629,7 +702,8 @@ lean:
|
|
|
629
702
|
| `light.max_lines` | number | No | Upgrade trigger (#4448): a light lane whose non-test diff (added + deleted lines) exceeds this moves to the full lane. Default 40. The other triggers: more than `max_files` non-test changed files, a repro that is not red on the base, two failed test runs, a touched risk path, or a public-API-shape change (#4449). Flat `lean.light_max_lines` wins. |
|
|
630
703
|
| `light.max_files` | number | No | Max predicted non-test files for eligibility and max changed non-test files before an upgrade (#4448). Default 2. Flat `lean.light_max_files` wins. |
|
|
631
704
|
| `sequence.regenerate` | string | No | Command run in the lane worktree (`sh -c`) after the engine renumbers a schema-changing sequence-numbered migration (#4386), to regenerate its snapshot, e.g. `pnpm --filter @x/db db:generate`. The flat form `lean.sequence_regenerate` also works and wins. When a lane-added numbered file (`NNNN_*.sql`, a drizzle `meta/_journal.json` + `meta/NNNN_snapshot.json`, a numbered ADR) shares its number with the base, the engine renumbers it without a builder session: `git mv` to max(base)+1…, the document's own number in its heading, references to the old name repo-wide, the base journal plus the lane's entries, and a data-only migration's snapshot cloned from the base tip with a new `id`/`prevId`. A schema-changing migration without this command, or a migration in another language (`.py`, `.ts`), falls back to the builder renumber, and the trajectory records why. When the generator writes no snapshot for a hand-written migration (no schema diff), the engine reruns it with ` --custom` appended (drizzle-kit's empty migration), keeps its snapshot and restores the lane's SQL byte-identical, so the command must forward trailing args (a pnpm/yarn script, or `npm run x --`). A base-sync whose conflicts are only in such files is resolved the same way, with no re-review. At the merge gate a lane with numbered files holds a host sequence lease per (repo, lane branch, directory) from its last renumber check through the merge, so only one numbered-file PR is in the renumber→CI→merge window at a time. |
|
|
632
|
-
| `base_merge_union_globs` / `base_merge_json_globs` / `base_merge_generated_globs` / `base_merge_generated_command` | string | No | Deterministic resolution of sibling-lane conflicts when the lane merges its base (#4588, `bin/lean/conflictres.mjs`; flat keys only). Comma-separated globs (`*` within a segment, `**` across; no `/` matches the file name anywhere). `union_globs`: hunks where both sides only added lines keep both. `json_globs`: a 3-way key union of a JSON object (a key both sides changed differently is refused). `generated_globs` + `generated_command`: the base side is taken, then the command runs once in the lane worktree (`sh -c`) and the files are re-added. A file that cannot be fully resolved, and every sequence/migration file, goes on as before (sequence resolver, then the builder). Default: none, so nothing changes. |
|
|
705
|
+
| `base_merge_union_globs` / `base_merge_json_globs` / `base_merge_generated_globs` / `base_merge_generated_command` | string | No | Deterministic resolution of sibling-lane conflicts when the lane merges its base (#4588, `bin/lean/conflictres.mjs`; flat keys only). Comma-separated globs (`*` within a segment, `**` across; no `/` matches the file name anywhere). `union_globs`: hunks where both sides only added lines keep both. `json_globs`: a 3-way key union of a JSON object (a key both sides changed differently is refused). `generated_globs` + `generated_command`: the base side is taken, then the command runs once in the lane worktree (`sh -c`) and the files are re-added. A file that cannot be fully resolved, and every sequence/migration file, goes on as before (sequence resolver, then the builder). Default: none, so nothing changes. The same keys apply to milestone branch syncs (`forgedock milestone-sync`, #4663). |
|
|
706
|
+
| `milestones.sync_debounce` | string | No | Debounce window of the milestone sync PR (#4663, `bin/lean/milestonesync.mjs`): on a protected milestone branch `forgedock milestone-sync` keeps ONE `sync/milestone-{slug}` PR and force-pushes its head in place; a base that moved again inside the window reports `debounced` (a `ship` sync ignores the window). `10m`, `1h`, `1d`. Default: `10m`. |
|
|
633
707
|
|
|
634
708
|
## `stats` (OPTIONAL)
|
|
635
709
|
|
|
@@ -709,6 +783,8 @@ verify:
|
|
|
709
783
|
| `e2e_paths` | string | No | Comma-separated globs of e2e specs (same glob rules as `skip_paths`). Default `e2e/**,**/e2e/**,*.e2e.*,*.e2e-spec.*,cypress/**,playwright/**`. |
|
|
710
784
|
| `e2e_blocking` | string | No | `changed` (default): only failures of the lane's own specs block the merge; other e2e failures are reported as an anomaly. `all`: every e2e failure blocks. |
|
|
711
785
|
|
|
786
|
+
**English-copy check (forge#4596).** At hand-off and after each base-sync merge, keys the lane added to a non-`en` JSON locale catalog (`messages/ta.json`, `locales/ta/common.json`) whose value equals the same key in the `en` counterpart go back to the builder once to translate (placeholders kept, status entries inserted in place, never re-sorted); a copy left over is an anomaly. Always on, deterministic, no cost without catalogs. The repo's own `i18n:check` is not run by the engine: list it in `lean.precheck.commands` to run it before the PR.
|
|
787
|
+
|
|
712
788
|
**Flow** (`ci`/`hybrid`, `bin/lean/verify.mjs` `runCiVerify`):
|
|
713
789
|
|
|
714
790
|
1. After the build and before every conformance review, the engine pushes the lane branch (a plain push, never forced) and runs CI on the head. A base-sync that moves the head is verified again.
|
|
@@ -736,6 +812,33 @@ Each run emits an `E2E_VERIFY` run-log/telemetry event (`pr`, `head`, `reason`,
|
|
|
736
812
|
|
|
737
813
|
**Commands that use this section**: core work-on (`forgedock run-issue`, `forgedock orchestrate`; `bin/engine-cli.mjs` `readVerify`)
|
|
738
814
|
|
|
815
|
+
## `ci` (OPTIONAL)
|
|
816
|
+
|
|
817
|
+
Scheduled full-suite integration runs on active milestone branches (forge#4667), report-only. Under the fast-lane/heavy-ship-gate model the whole e2e suite otherwise runs only at the ship PR. Write it as a block-style section; the parser flattens the keys of `milestone_integration:` under `ci`. Off unless `workflow` is set.
|
|
818
|
+
|
|
819
|
+
```yaml
|
|
820
|
+
ci:
|
|
821
|
+
milestone_integration:
|
|
822
|
+
workflow: e2e.yml # workflow_dispatch workflow that runs the full suite
|
|
823
|
+
# project: web-e2e # optional project passed to the workflow
|
|
824
|
+
# project_input: project # its input name (default project)
|
|
825
|
+
interval_hours: 24 # run when this old (default 24, nightly)
|
|
826
|
+
every_merges: 10 # optional: also run after N lane merges into the milestone
|
|
827
|
+
block_admission_after: 2 # optional: hold new lanes after N consecutive red runs, until green
|
|
828
|
+
max_minutes: 360 # optional wait budget per run (default 360)
|
|
829
|
+
```
|
|
830
|
+
|
|
831
|
+
| Key | Type | Required | Description |
|
|
832
|
+
|---|---|---|---|
|
|
833
|
+
| `workflow` | string | Yes | Workflow file dispatched on `milestone/<slug>` (needs `on: workflow_dispatch`). |
|
|
834
|
+
| `project` / `project_input` | string | No | Value and input name passed to the workflow. |
|
|
835
|
+
| `interval_hours` | number | No | Hours between runs of a milestone. Default 24. |
|
|
836
|
+
| `every_merges` | integer | No | Also run once this many lane merges happened since the last run (checked by `forgedock orchestrate`). |
|
|
837
|
+
| `block_admission_after` | integer | No | After this many consecutive red runs, `forgedock orchestrate` admits no new lane of that milestone (ends `held`) until a run is green. |
|
|
838
|
+
| `max_minutes` | integer | No | Wait budget of one run. Default 360. |
|
|
839
|
+
|
|
840
|
+
State lives in a `FORGE:INTEGRATION` block in the milestone description. A red run files or updates one `integration-red` issue on the milestone (failing specs, merge range); the next green run closes it. `/deploy-info` shows open ones as a ship-gate risk and `/pipeline-status` shows each milestone's last green run. Schedule it with cron or a scheduled workflow: `forgedock integration run [--repo owner/repo] [--milestone TITLE] [--force]` (exit 1 when a run is red; `forgedock integration status` prints each last green run); `forgedock orchestrate` also runs it every `every_merges` merges.
|
|
841
|
+
|
|
739
842
|
## `execution` (OPTIONAL)
|
|
740
843
|
|
|
741
844
|
Where `forgedock orchestrate` runs its lanes (forge#4253). Block style only.
|
|
@@ -778,6 +881,7 @@ execution:
|
|
|
778
881
|
| `budget_minutes`, `budget_usd` | number | No | The `execution.budget` per batch: once the batch's sandbox minutes / estimated USD (settled lanes plus the running ones' elapsed time) reach the cap, no new remote lane is admitted. Missing, non-numeric or negative means no cap. |
|
|
779
882
|
| `day_budget_minutes`, `day_budget_usd` | number | No | The same cap per host UTC day, summed over every batch on the host from the host-day store (`execution-budget.json` in `~/.forge/sched/`, or `FORGEDOCK_EXECUTION_BUDGET_FILE`). A corrupt or unreadable store refuses remote lanes while a day cap is set; remove the file to reset it. |
|
|
780
883
|
| `fallback_local` | boolean | No | `true`: a remote-placed lane runs on the host while the provider reports no agent online (`GET {endpoint}/status -> {"online": N}`, the built-in router answers it with `online`, `total` and per-agent `ok`/`paused`), so a batch completes locally with every pool agent offline or asleep (forge#4547). Default `false`: such lanes defer (`sandbox.capacity`). A missing or failing `/status` is unknown and never triggers it. |
|
|
884
|
+
| `unreachable_window_min` | number | No | Remote providers: how long a lane keeps polling (with backoff, up to 30s) after the provider's transport fails (connection refused/reset, `fetch failed`, timeout, 502/503/504) before it ends `deferred` with `sandbox.unreachable` (forge#4672). Inside the window the lane never ends and the sandbox is never torn down; after it the sandbox is kept and adopted on the next attempt or `--resume`; the engine-error breaker does not count it. `adopt` applies the same window. Default: 5 |
|
|
781
885
|
| `on_budget` | string | No | `stop` (default): running lanes finish and the rest end `held` (`execution budget: ...`) for a later `--resume`. `local`: the rest run as detached host lanes. |
|
|
782
886
|
| `host_capabilities` | list | No | Host-only capabilities the dispatching host has and a sandbox does not relay (forge#4542): names such as `sops`, `prod_monitor`, `ops_credentials`. A lane whose issue text names one (built-in signals: `.secrets/*.enc.yaml`, `sops`, `ssh *-monitor`, `obs-psql`, `credentials.yaml`; a declared name written out; `host_capability_patterns`) is placed `local` in `remote` and `hybrid` mode with the reason `host capability: <name>`, since remote providers (computeshare included) relay none of them. A remote lane that still blocks as `credentials` naming only a declared capability ends `env.capability_remote` (recoverable) and is rerun locally once. |
|
|
783
887
|
| `host_capability_patterns` | list | No | `name:regex` entries (case-insensitive) that also mean the lane needs capability `name`, e.g. `vault:vault\s+kv`. An invalid entry is ignored. |
|
|
@@ -1383,6 +1487,19 @@ SHOW_ATTRIBUTION=$(yq '.branding.show_attribution // "true"' forge.yaml 2>/dev/n
|
|
|
1383
1487
|
|
|
1384
1488
|
---
|
|
1385
1489
|
|
|
1490
|
+
## Bot token refresh
|
|
1491
|
+
|
|
1492
|
+
Installation tokens expire after 1 hour. `scripts/refresh-bot-token.sh` mints a new one and records `~/.forge/bot-token.json` (`refreshedAt`, `expiresAt`; override the directory with `FORGE_HOME_DIR`). To run it on a schedule, install the systemd user timer (`OnBootSec=2min`, `OnUnitActiveSec=45min`, `Persistent=true`, so a missed run fires after a restart):
|
|
1493
|
+
|
|
1494
|
+
```bash
|
|
1495
|
+
mkdir -p ~/.config/systemd/user
|
|
1496
|
+
cp scripts/systemd/forgedock-bot-token-refresh.{service,timer} ~/.config/systemd/user/
|
|
1497
|
+
# edit ExecStart in the .service to the path of refresh-bot-token.sh on this host
|
|
1498
|
+
systemctl --user daemon-reload && systemctl --user enable --now forgedock-bot-token-refresh.timer
|
|
1499
|
+
```
|
|
1500
|
+
|
|
1501
|
+
`forgedock doctor` (Check 12) reports the remaining lifetime and the last refresh, and fails when less than 10 min remain or the last refresh is older than 50 min. Without the stamp file it stays an informational warning. A core run that gets HTTP 401 from `gh` runs the refresh script once and retries; if that does not help the run ends `deferred` with `gh.auth` (see `docs/OUTCOME-CODES.md`), and `FORGE_REFRESH_TOKEN_SCRIPT` overrides the script it runs. `scripts/wait-ci-green.sh` retries a 401 a few times, then exits 2 with an auth message.
|
|
1502
|
+
|
|
1386
1503
|
## GitHub App Install
|
|
1387
1504
|
|
|
1388
1505
|
During `npx forgedock`, the installer prompts you to install the ForgeDock GitHub App (`rapiercraft-forgedock`) on your account or org.
|
|
@@ -1438,6 +1555,7 @@ These variables are read from the operator's runner environment, not from `forge
|
|
|
1438
1555
|
| `FORGE_TRAIL_QG_SINCE` | `2026-10-07T03:40:12Z` | ISO-8601 UTC cutoff for the legacy `FORGE:QUALITY_GATE` grace. An issue whose trusted `FORGE:BUILDER:COMPLETE` comment was last updated before this time has `QUALITY_GATE` waived. Set to an empty string to disable the grace. A malformed or future value (including a runner clock earlier than this default) fails closed: the verifier exits 2, an infrastructure block rather than `MISSING`. |
|
|
1439
1556
|
| `FORGEDOCK_ALLOW_API_BACKEND` | unset | Set to `1` (host owner approval) together with forge.yaml `execution.agent_backend: api` to let `--backend auto` use the paid API backend when no `claude` CLI is available. Without both, `auto` never selects `api` (#4257). |
|
|
1440
1557
|
| `FORGE_TELEMETRY` | `on` | Set to `off` to disable gate telemetry. Gate scripts append one structured record per decision to a trusted `FORGE:TELEMETRY v1` comment; emission is best-effort and never changes a gate's output or exit code. See [TELEMETRY.md](TELEMETRY.md). |
|
|
1558
|
+
| `FORGE_EVENTS` | `on` | Set to `off` to disable the lifecycle event stream (local JSONL and webhook). `FORGE_EVENTS_DIR` overrides the JSONL directory; the webhook secret is read from the env var named by `events.webhook_secret_env` (default `FORGE_EVENTS_SECRET`). See [EVENTS.md](EVENTS.md). |
|
|
1441
1559
|
|
|
1442
1560
|
### Install root resolution
|
|
1443
1561
|
|
package/docs/EVENTS.md
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# Lifecycle Events
|
|
2
|
+
|
|
3
|
+
ForgeDock emits a small, versioned stream of lifecycle events for outside consumers (dashboards, chat bots, audit logs). It
|
|
4
|
+
is derived from the records the engine already produces (the run log and `docs/TELEMETRY.md` inputs): deterministic, no model
|
|
5
|
+
call, no consumer required. With no configuration the events go only to a local JSONL file.
|
|
6
|
+
|
|
7
|
+
## Envelope (v1)
|
|
8
|
+
|
|
9
|
+
```json
|
|
10
|
+
{"schema":"forgedock.event","v":1,"id":"1793012345-9f2c1ab04d7e","type":"pr.merged","ts":"2026-10-26T12:00:00Z","repo":"o/r","issue":9,"pr":40,"run":"core_9_1793012000","cost_usd":0.75,"wall_ms":90000,"data":{"milestone":"m1","code":"ok"}}
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
| Key | Meaning |
|
|
14
|
+
|-----|---------|
|
|
15
|
+
| `schema` | Always `forgedock.event`. |
|
|
16
|
+
| `v` | Schema version (integer). Adding an event type or an optional key keeps `v`; renaming or removing one, or changing a meaning, bumps it. |
|
|
17
|
+
| `id` | Unique per event; consumers de-duplicate on it. |
|
|
18
|
+
| `type` | One of the types below. |
|
|
19
|
+
| `ts` | UTC, second precision. |
|
|
20
|
+
| `repo`, `issue`, `pr`, `run` | `owner/name`, issue number, PR number, run id; `null` when not known. |
|
|
21
|
+
| `cost_usd`, `wall_ms` | Present on `pr.merged`: the lane's total model spend (sum of every phase attempt's `cost_usd_delta`, else `cost_usd`) and the engine process wall time. |
|
|
22
|
+
| `data` | The type's keys (below); a key with no value is omitted. Strings are capped at 200 characters. |
|
|
23
|
+
|
|
24
|
+
## Event types
|
|
25
|
+
|
|
26
|
+
| Type | When | `data` |
|
|
27
|
+
|------|------|--------|
|
|
28
|
+
| `lane.picked_up` | A core lane starts an issue. | `lane`, `path` |
|
|
29
|
+
| `contract.frozen` | The lane's contract is frozen. | `hash`, `requirements`, `deferred`, `derived` |
|
|
30
|
+
| `pr.opened` | The lane opens its PR (`pr` is set). | none |
|
|
31
|
+
| `pr.merged` | The run ends `merged` (`pr`, `cost_usd`, `wall_ms` set). | `milestone`, `code` |
|
|
32
|
+
| `probe.passed` / `probe.failed` | A declared acceptance probe is recorded (`forgedock probe`, batch sweep). | `detail`, `fix` |
|
|
33
|
+
| `release.promoted` | A probed fix reached the default branch (`forgedock probe` sweep: a recorded pass or fail, so once per fix, only for issues declaring `forge-probe`). | `fix`, `default_branch` |
|
|
34
|
+
| `regression.detected` | A windowed post-deploy check failed and filed a regression issue (`forgedock probe --due`, daemon timer; `issue` is the monitored follow-up). | `subject`, `metric`, `detail` |
|
|
35
|
+
| `experiment.verdict` | `forgedock stats --experiment <id> --emit-event` found targets `met` or `missed` (`inconclusive` is not emitted; plain `stats` never emits). | `experiment`, `verdict`, `detail` |
|
|
36
|
+
|
|
37
|
+
## Local sink
|
|
38
|
+
|
|
39
|
+
One JSON object per line, appended to `~/.forge/events/<owner>_<repo>.jsonl` (directory override: forge.yaml `events.dir` or
|
|
40
|
+
`FORGE_EVENTS_DIR`). The file is a cache of what was emitted on this machine, not a record of truth: GitHub stays the register.
|
|
41
|
+
|
|
42
|
+
## Webhook (optional)
|
|
43
|
+
|
|
44
|
+
```yaml
|
|
45
|
+
events:
|
|
46
|
+
webhook_url: https://hooks.example.com/forgedock # https, or http on localhost
|
|
47
|
+
webhook_secret_env: FORGE_EVENTS_SECRET # NAME of the env var holding the secret (this is the default name)
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Both the URL and the env var must be set, else nothing is sent. The secret is never read from forge.yaml. Each event is one `POST`
|
|
51
|
+
of the envelope JSON with a 5 second timeout, no redirects and no retry; a failed delivery is dropped and never affects the run.
|
|
52
|
+
|
|
53
|
+
Headers: `X-Forge-Event` (the type), `X-Forge-Timestamp` (unix seconds), `X-Forge-Signature` = `sha256=` + hex
|
|
54
|
+
`HMAC-SHA256(secret, "<timestamp>.<raw body>")`. Verify against the raw body and reject old timestamps:
|
|
55
|
+
|
|
56
|
+
```js
|
|
57
|
+
const expected = createHmac("sha256", secret).update(`${ts}.${rawBody}`).digest("hex");
|
|
58
|
+
const ok = timingSafeEqual(Buffer.from(sig.replace(/^sha256=/, ""), "hex"), Buffer.from(expected, "hex"));
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
## Guarantees
|
|
62
|
+
|
|
63
|
+
- Never fails or slows the run beyond the webhook timeout; `FORGE_EVENTS=off` disables the sink and the webhook.
|
|
64
|
+
- Ids, counts, codes and costs only: no issue text, diffs, file contents or secrets.
|
|
65
|
+
- Contract test: `bin/engine/events.test.mjs` pins the types, keys and the signature algorithm.
|
package/docs/OUTCOME-CODES.md
CHANGED
|
@@ -32,6 +32,7 @@ An absent or unknown code is recorded as `unclassified`, with the raw reason kep
|
|
|
32
32
|
| `remediate.inpr_requested` | Re-gate approved but requested one further in-PR fix round; the run continues, never `needs-human` | remediate | `**Re-gate outcome**: INPR-FIX-REQUESTED` |
|
|
33
33
|
| `ci.red` | CI failed | reserved | `REVIEW_RESULT` carries no CI field yet |
|
|
34
34
|
| `ci.timeout` | CI timed out | reserved | `REVIEW_RESULT` carries no CI field yet |
|
|
35
|
+
| `ci.dco` | Core work-on (forge#4651): a check named DCO failed on the PR head (required on the lane branch or not), or the lane's unpushed commits lack a sign-off and the worktree has no usable git identity to rewrite them. Not recoverable: a new commit cannot fix author/sign-off history | lean | needs-human; nothing merged |
|
|
35
36
|
| `base.conflict` | Base-branch conflict blocked the PR | reserved | `REVIEW_RESULT` carries no conflict field yet |
|
|
36
37
|
| `path.switch_conflict` | The full path was started on an issue that has an open core-path PR (`feat/issue-<n>-lean`); refused to avoid a second implementation. Rerun on the default path (base-sync) or close that PR first | engine | `bin/engine.mjs` admission |
|
|
37
38
|
| `merge.transient` | `gh pr merge` kept failing with a transient error (`Base branch was modified`, GraphQL or 5xx) after the retry budget; the run is `deferred` and a rerun resumes at the merge | lean | `bin/lean/lean.mjs` |
|
|
@@ -41,18 +42,23 @@ An absent or unknown code is recorded as `unclassified`, with the raw reason kep
|
|
|
41
42
|
| `run.interrupted` | A core run got SIGTERM/SIGINT; the lane tore down from its resource ledger (unpushed work checkpointed to the lane branch, worktree removed, lease released) and ended `deferred`, so a batch requeues it | lean | `bin/engine-cli.mjs` |
|
|
42
43
|
| `host.busy` | Core run deferred at admission: free memory, free tmp space or load per core outside `engine.admission` limits; no lease taken, nothing posted | lean | `bin/lean/admission.mjs` |
|
|
43
44
|
| `gh.unavailable` | `run-issue --repo` could not verify the cwd repo because `gh repo view` kept failing after 3 attempts with backoff (typically a timeout under host load); the core or full run is `deferred` (exit 75, retry in 60s) before any lease, nothing posted | engine CLI | `bin/engine-cli.mjs` |
|
|
45
|
+
| `gh.auth` | A `gh` call in a core run answered HTTP 401 (expired or revoked bot token) and one run of `scripts/refresh-bot-token.sh` plus a retry did not fix it; the run is `deferred` (retry in 120s), never `engine-error`. See `docs/CONFIG.md` "Bot token refresh" | lean | `bin/lean/retry.mjs` |
|
|
44
46
|
| `rate_limited` | Session or usage limit outlasted the pause budget | engine | `err.usageLimit` / `err.resetAtEpochMs` |
|
|
45
47
|
| `engine.error` | Engine or tool failure (missing key/SDK, CLI crash, max turns, unresolved args, runner threw every attempt) | engine | `err.code` |
|
|
48
|
+
| `engine.validate_result_lost` | Core work-on (forge#4708): a validate result recorded before a merge-time CI fix round is missing from the fix session's result file and could not be carried forward; an engine bookkeeping error, not an unimplemented requirement | lean | `bin/lean/lean.mjs` `mergeFixRound` |
|
|
46
49
|
| `infra.disk_quota` | Disk full or quota exhausted (EDQUOT/ENOSPC, errno -28/-122); retryable infra failure, never `needs-human` | engine | Runner error text |
|
|
47
50
|
| `infra.cli_spawn_transient` | CLI spawn failed transiently before the child ran (ETIMEDOUT, EAGAIN, ENOMEM, EMFILE); retried with backoff on a separate bounded counter that does not consume a phase attempt | engine | `err.code` `CLI_SPAWN_TRANSIENT` |
|
|
48
51
|
| `cli.timeout` | CLI child ran and was killed at the wall-clock limit; a normal attempt (consumes one of `maxAttempts`), usage recorded when the CLI emitted any | engine | `err.code` `CLI_TIMEOUT` |
|
|
49
52
|
| `cli.spawn_failed` | The CLI could not be started for a reason that repeats on every rerun (`E2BIG` argv too large, `ENOENT`/`EACCES`/`ENOEXEC` binary missing or not executable, `ENAMETOOLONG`). Core work-on ends `needs-human` with this code instead of `engine-error`, so `forgedock orchestrate` does not spend its relaunch on it | lean, engine | `err.code` `CLI_SPAWN_FAILED` (E2BIG) or `err.spawnDeterministic` (other errnos keep their `err.code`); `err.spawnErrno` holds the errno |
|
|
50
53
|
| `env.setup_failed` | forge.yaml `env.setup` (run once per fresh lane worktree, before the builder) exited non-zero, timed out or could not start. An environment error, not a build failure: the builder never ran. Core work-on ends `needs-human` with this code instead of `engine-error`, so `forgedock orchestrate` does not relaunch it; fix the setup command or the environment and rerun | lean | `io.runSetup` exit code / spawn error |
|
|
51
|
-
| `env.
|
|
54
|
+
| `env.lane_limit` | Core work-on (forge#4690): the lane's systemd scope hit its own `TasksMax` (forks refused, `pids.events max`) or `MemoryMax` (`memory.events oom_kill`) and the engine stopped the lane after checkpointing its work. The message names the limit. An environment error, not an engine-error: it ends `needs-human`, is not relaunched and is not counted by the circuit breaker, so other lanes keep being admitted. Raise `engine.lane_limits.tasks_max` / `memory_max` if the work is legitimate | lean | scope `pids.events` / `memory.events` sampled by `bin/runner.mjs` `readScopeLimitEvents` |
|
|
55
|
+
| `env.tool_missing` | Core work-on (forge#4323): a validate command the builder recorded as failed was rerun by the engine and could not run because the host lacks a tool or module (`command not found` / exit 127, Python `No module named x`, Node `Cannot find module|package 'x'` for a bare name the repo does not track). An environment error, not a conformance failure: no builder resume, no reviewer. Also ends this way when a remote-sandbox builder reports a missing tool as a `credentials` block (forge#4685, `missingToolBlock`). Ends `needs-human` naming the tool; recoverable by `forgedock orchestrate` (the recovery lane runs the check through the repo's own toolchain). Skipped on a CI-verified head | lean | `gate()` rerun via `io.runCommand`, `{rc, output}` classified by `conformance.mjs` `missingTool` |
|
|
52
56
|
| `env.repro_unrunnable` | Core work-on (forge#4334): on a bug lane the frozen `repro_test` could not be run on head by `scripts/acceptance-red-green.sh` (`HEAD=UNRUNNABLE`: timeout, exit 126/127, unclassifiable output) or the harness returned no result. An environment error, not a conformance failure: no builder resume, no reviewer. Ends `needs-human`; recoverable by `forgedock orchestrate` (the recovery lane reruns the repro command through the repo's own toolchain and installed dependencies). Other non-proofs (passes on base, fails on head, base unrunnable) stay `conformance.unimplemented` | lean | `gate()` `io.redGreen` words, classified by `conformance.mjs` `reproFailures` |
|
|
53
57
|
| `env.capability_remote` | Core work-on (forge#4542): a lane placed in a remote sandbox ended `builder.blocked (credentials)` and its summary names only host capabilities declared in `execution.host_capabilities` (SOPS/age key, ops credentials, prod monitor SSH), which the sandbox does not relay but the dispatching host has. Recoverable by `forgedock orchestrate`: the recovery lane is placed local, once, under the same lease. A credential need the host lacks too stays `builder.blocked` | lean | `endBlocked` via `execution.mjs` `hostCapabilityBlock` |
|
|
54
58
|
| `sandbox.start_failed` | A `forgedock orchestrate` lane sandbox (forge.yaml `execution`, provider `container` or `remote`) could not start: bad image, missing endpoint or credential, refused launch. The lane never ran. Final `needs-human` row: not relaunched and not counted by the engine-error circuit breaker | orchestrate | `bin/lean/execution.mjs` `providerLanes`: `provider.start` threw without `retryable` |
|
|
55
59
|
| `sandbox.capacity` | The sandbox provider had no capacity for the lane (HTTP 429/503 from the remote transport). `deferred`, requeued after `Retry-After` when sent, else 60s; not counted toward `orchestration.max_deferrals`, like `host.busy` (forge#4528) | orchestrate | `provider.start` threw with `err.retryable: true` |
|
|
60
|
+
| `sandbox.auth` | The remote sandbox provider refused the lane's credential (HTTP 401/403) at start (forge#4685). `deferred`, retried after 300s, never `needs-human`; the pool token resolves from the env, the keychain, then `~/.config/computeshare/forgedock.token` | orchestrate | `provider.start` threw with `err.status` 401/403 |
|
|
61
|
+
| `sandbox.unreachable` | The remote sandbox provider stayed unreachable (connection refused/reset, `fetch failed`, timeout, 502/503/504) for `execution.unreachable_window_min` while a lane polled or adopted its sandbox (forge#4672). `deferred`, requeued after 60s; not counted by the engine-error circuit breaker. The sandbox is NOT torn down: the next attempt or `--resume` adopts it | orchestrate | `bin/lean/execution.mjs` `providerLanes` poll guard |
|
|
56
62
|
| `lease.held` | Another live agent holds the issue lease | engine | Foreign live lease on acquire |
|
|
57
63
|
| `dependency.open` | A dependency is still open: a trusted `**Depends on**: #N` line refused the run before any state write, or the core builder reported `block.kind: "dependency"` naming an issue that is open. Carries `dependencies`; never `needs-human`. `forgedock orchestrate` adds the edge and requeues the issue behind it, without skipping dependents | engine, lean | Trusted `**Depends on**` line, or `result.json` `block.issues`, + the dependency issue's live open state |
|
|
58
64
|
| `dependency.unreadable` | The issue body or a declared dependency could not be read, so the run was refused (fail closed) | engine, lean | Failed or unparsable `gh api` / `gh issue view` read |
|
|
@@ -77,9 +83,11 @@ An absent or unknown code is recorded as `unclassified`, with the raw reason kep
|
|
|
77
83
|
| `conformance.unimplemented` | Core work-on: a contracted requirement is partial, missing or unvalidated, or a validate command failed, after one builder resume (a validate the host cannot run for a missing tool or module is `env.tool_missing`, and a repro test the harness cannot run on head is `env.repro_unrunnable`, instead) | lean | Precheck (result.json, validation) or review requirement row |
|
|
78
84
|
| `conformance.scope_creep` | Core work-on: a changed file is not authorized by a deliverable or typed amendment, after one builder resume | lean | Precheck changed files or review `scope` row `none` |
|
|
79
85
|
| `conformance.untracked_deferral` | Core work-on: a requirement was deferred without a typed amendment or a tracking issue, after one builder resume | lean | Precheck or review requirement row `deferred` on a `deliver` requirement |
|
|
86
|
+
| `conformance.e2e_unverified` | Core work-on: a `[type:e2e]` requirement is implemented but its validate is not a real end-to-end check (the repo's e2e set, Playwright, live stack), after one builder resume | lean | Review requirement row `implemented` with a unit-test `validated_by` (`bin/lean/conformance.mjs` `e2eVerified`) |
|
|
80
87
|
| `conformance.unjustified_deferral` | Core work-on: the reviewer judged a deferred or post-deploy requirement unjustified (it belongs in this change), after one builder resume; nothing is filed or merged | lean | Review requirement row `deferral_ok: false` |
|
|
81
88
|
| `conformance.preserve_violated` | Core work-on: a preserve constraint is violated or unverified, after one builder resume | lean | Review `preserve` row not `intact` |
|
|
82
89
|
| `conformance.regression` | Core work-on: the reviewer named a concrete regression, after one builder resume | lean | Review `regressions` row |
|
|
90
|
+
| `conformance.guard_broken` | Core work-on: the base version of a merged fix's guarding test fails against the lane's tree and no `supersedes` amendment retires it, after one builder resume | lean | `bin/lean/guardgate.mjs` guard rerun |
|
|
83
91
|
| `docs.stale` | Core work-on: the target repo's own `scripts/check-devdocs-freshness.sh` (docs must move with `bin/lean/` / `scripts/` behaviour) still fails after one builder resume; caught before the PR instead of by its required check | lean | the repo's `scripts/check-devdocs-freshness.sh <base>` exit 1 |
|
|
84
92
|
| `structure.undeclared_edge` | Core work-on: the change adds a package.json dependency between workspace packages that the frozen contract does not name, still present after one builder resume | lean | `unnamedEdges` over `scripts/structure-delta.sh` `package_edges_added` |
|
|
85
93
|
| `maintenance.hotspot_growth` | Core work-on: the change grows a file already over `maintenance.hotspot.file_lines` by more than `growth_lines` without a `hotspot-exempt: <reason>` in the contract, still present after one builder resume; blocks in shadow and enforce | lean | `hotspotGrowth` precheck over `scripts/hotspots.sh --base` `growth` |
|
package/docs/STATS.md
CHANGED
|
@@ -113,6 +113,26 @@ remote lane minutes, lanes and share of the pool per agent (`sandbox.by_agent`).
|
|
|
113
113
|
The same per-agent minutes appear in the batch report's `sandbox:` line as `by agent: <name> <min> min, ...`.
|
|
114
114
|
Live capacity is `forgedock pool status`.
|
|
115
115
|
|
|
116
|
+
## Compounding (`--compounding`, forge#4696)
|
|
117
|
+
|
|
118
|
+
`forgedock stats --compounding [--window 30d] [--bucket 100,500] [--since <date>] [--min-n N] [--json]` answers "does the repo
|
|
119
|
+
get safer the longer ForgeDock works on it". No model: FORGE:GUARD comments (#4692, `forgedock guards list`), `reopened` issue
|
|
120
|
+
events (one paginated `gh api repos/<repo>/issues/events`), `git blame` and `git diff` of the current clone, and the repo's
|
|
121
|
+
FORGE:TELEMETRY gate events. Output: a rolling window (default 30d), per release and per lane-age bucket (merge order of the
|
|
122
|
+
repo's lanes: 1-100, 101-500, 501+, `unranked` for guards without a merged run).
|
|
123
|
+
|
|
124
|
+
- **reintroduction**: guarded fixes whose issue was reopened after the lane closed it, or whose merge commit is the blame of the
|
|
125
|
+
old-side lines of a later guarded fix within 90 days (the guard's `merge_sha` is the commit, as for squash and rebase merges).
|
|
126
|
+
Rate = reintroduced / guarded. A guard whose sha is not in the clone is counted under `blame_unresolved`, never as 0.
|
|
127
|
+
- **reopen-after-close**: merged (lane-closed) issues reopened afterwards / merged lanes.
|
|
128
|
+
- **guard activity**: counts of `guard_run`, `guard_broken` (pre-merge saves) and `supersedes_accepted` events (#4693).
|
|
129
|
+
- **decision churn**: `decision_superseded` vs `decision_violation` events (#4695).
|
|
130
|
+
- Event kinds a repo has never recorded print `n/a`, not 0. Groups under `--min-n` lanes are flagged `low n`.
|
|
131
|
+
|
|
132
|
+
Efficiency next to quality: run `forgedock stats --efficiency --compare <date|version>` around each child's deploy for
|
|
133
|
+
$/merge, turns/merge and wall p50. Baseline before #4693 enforces: run `forgedock stats --compounding --json --since <date>` in
|
|
134
|
+
a full clone of this repo and one other and keep the output with the #4697 tracker.
|
|
135
|
+
|
|
116
136
|
## Explorer A/B (`--explore-ab`, forge#4418)
|
|
117
137
|
|
|
118
138
|
Compares lanes run with forge.yaml `lean.explore: on` against `off` (#4391), from the local run logs only (no GitHub
|
|
@@ -168,3 +188,34 @@ Comments are read from the repo-wide `issues/comments?since=` endpoint (a few pa
|
|
|
168
188
|
telemetry comment in the window get their full comment list re-read (6 in parallel, backing off when the API rate limit is
|
|
169
189
|
low). A run with no `--since` and no cache still walks the whole history once. Telemetry comments are edited in place, which
|
|
170
190
|
the comment window catches; if a cached item still looks stale, run with `--refresh`.
|
|
191
|
+
|
|
192
|
+
## Per-release regression rule (`--by version`, forge#4628)
|
|
193
|
+
|
|
194
|
+
```bash
|
|
195
|
+
npx forgedock stats --by version [--post-metrics <pr>] [--min-n N] [--repo <owner/repo>] [--json]
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
Median cost per run, wall p50, first-pass rate and needs-human rate of each release against the previous known one.
|
|
199
|
+
A metric whose own sample is under `--min-n` (default 10) is null, and so is its verdict; verdicts are
|
|
200
|
+
`regressed|improved|flat|null`, never a pass. `--post-metrics <pr>` posts (or updates) a `<!-- FORGE:METRICS v1 -->`
|
|
201
|
+
comment on that release PR with the table and the snapshot as JSON; without it nothing is written to GitHub.
|
|
202
|
+
|
|
203
|
+
## Live A/B experiments (`--experiment <id>`, forge#4629)
|
|
204
|
+
|
|
205
|
+
Declare one experiment in forge.yaml (flat keys under `experiment:`):
|
|
206
|
+
|
|
207
|
+
```yaml
|
|
208
|
+
experiment:
|
|
209
|
+
id: light-vs-full
|
|
210
|
+
arms: light,full # at least two
|
|
211
|
+
ratio: 1,1 # integer weights, default equal
|
|
212
|
+
metrics: usd_median,wall_p50_ms,first_pass_rate,needs_human_rate # default all four
|
|
213
|
+
targets: light.usd_median<=1.2xfull,light.first_pass_rate>=0.8 # arm.metric<=|>=limit, optionally x<other arm>
|
|
214
|
+
min_n: 10
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
The lean lane assigns each issue an arm (sha1 of id and issue number over the ratio, so a re-run keeps its arm) and tags
|
|
218
|
+
`experiment` and `arm` on `phase_attempt` and `run_terminal`. `forgedock stats --experiment <id>` reports the finished
|
|
219
|
+
runs per arm with the metrics of `--by version`; a metric under min n is null and so is any target that needs it, never a
|
|
220
|
+
pass. `targets_met` is true only when every target is met. `--require-met` (with `--experiment`) prints the same report but exits 1 unless `targets_met` is true (0 otherwise; 2 without `--experiment`), so a post-deploy check can gate on it (#4634). The report, `scripts/bench-lightlane.mjs aggregate` and
|
|
221
|
+
`scripts/eval-gate-scorecard.mjs` share one record: `{schema: "forge.bench.v1", source, arms, targets}`.
|