forgedock 1.46.0 → 1.47.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (65) hide show
  1. package/bin/engine/outcome-codes.mjs +2 -1
  2. package/bin/engine-cli.mjs +22 -8
  3. package/bin/lean/conformance.e2e.test.mjs +20 -0
  4. package/bin/lean/conformance.mjs +99 -9
  5. package/bin/lean/conformance.test.mjs +121 -4
  6. package/bin/lean/context.mjs +108 -6
  7. package/bin/lean/context.test.mjs +84 -2
  8. package/bin/lean/coupled.mjs +38 -0
  9. package/bin/lean/coupled.test.mjs +57 -0
  10. package/bin/lean/decision-engine.mjs +5 -1
  11. package/bin/lean/decision-engine.test.mjs +21 -0
  12. package/bin/lean/drift.mjs +3 -2
  13. package/bin/lean/drift.test.mjs +23 -1
  14. package/bin/lean/effects.mjs +3 -2
  15. package/bin/lean/execution.mjs +41 -8
  16. package/bin/lean/execution.test.mjs +72 -2
  17. package/bin/lean/integration.mjs +2 -2
  18. package/bin/lean/integration.test.mjs +22 -1
  19. package/bin/lean/lane.mjs +13 -0
  20. package/bin/lean/lane.test.mjs +28 -1
  21. package/bin/lean/lean.mjs +60 -12
  22. package/bin/lean/lean.test.mjs +80 -0
  23. package/bin/lean/limits.mjs +6 -2
  24. package/bin/lean/limits.test.mjs +86 -0
  25. package/bin/lean/milestonesync.mjs +2 -2
  26. package/bin/lean/milestonesync.test.mjs +22 -0
  27. package/bin/lean/orchestrate.mjs +79 -9
  28. package/bin/lean/probe.mjs +4 -2
  29. package/bin/lean/probe.test.mjs +4 -0
  30. package/bin/lean/prompts.mjs +18 -11
  31. package/bin/lean/rotator.mjs +96 -0
  32. package/bin/lean/rotator.test.mjs +143 -0
  33. package/bin/lean/routecheck.mjs +48 -0
  34. package/bin/lean/routecheck.test.mjs +69 -0
  35. package/bin/lean/sandbox-provider.contract.test.mjs +9 -0
  36. package/bin/lean/sandbox-remote.mjs +4 -2
  37. package/bin/lean/verify.mjs +2 -1
  38. package/bin/review-staging.mjs +20 -3
  39. package/commands/deploy-info.md +4 -4
  40. package/commands/issue.md +6 -1
  41. package/commands/review-pr-agents/frontend.md +1 -0
  42. package/commands/review-pr-agents/infra.md +2 -0
  43. package/commands/review-pr-staging.md +2 -0
  44. package/commands/work-on/build/context.md +7 -1
  45. package/commands/work-on/investigate.md +15 -3
  46. package/docs/CONFIG.md +19 -2
  47. package/docs/OUTCOME-CODES.md +2 -1
  48. package/package.json +1 -1
  49. package/scripts/file-deferred-followups.sh +125 -28
  50. package/scripts/file-deferred-followups.test.sh +49 -0
  51. package/scripts/issue-create.sh +7 -0
  52. package/scripts/issue-create.test.sh +13 -0
  53. package/scripts/issue-dedup.sh +5 -4
  54. package/scripts/issue-dedup.test.sh +15 -0
  55. package/scripts/owner-route-parity.test.sh +32 -0
  56. package/scripts/prior-attempts.mjs +72 -0
  57. package/scripts/prior-attempts.test.mjs +52 -0
  58. package/scripts/review-finalize.sh +35 -4
  59. package/scripts/review-finalize.test.sh +10 -0
  60. package/scripts/review-prepare.sh +16 -4
  61. package/scripts/review-prepare.test.sh +8 -0
  62. package/scripts/review-roster.sh +3 -1
  63. package/scripts/review-roster.test.sh +30 -2
  64. package/scripts/review-staging-pre.sh +8 -6
  65. package/scripts/review-staging-pre.test.sh +11 -1
package/docs/CONFIG.md CHANGED
@@ -730,11 +730,21 @@ issue_watch:
730
730
 
731
731
  ## `followups` (OPTIONAL)
732
732
 
733
- Deferred follow-ups (`scripts/file-deferred-followups.sh`) are filed as one issue per class per parent: `engineering`, `post-deploy`, `external` and `ops`. Each carries the label `followup:<class>` (`external` also `external-prerequisite`, and states the parent it is a prerequisite of) and inherits the parent's milestone. A requirement may set an optional `class` in the contract; otherwise `post-deploy` items are `post-deploy` and the rest `engineering`. A parent milestone that has shipped (closed) is replaced by the next open milestone by due date, else `followups.default_milestone`.
733
+ Deferred follow-ups (`scripts/file-deferred-followups.sh`) are routed by kind (#5078):
734
+
735
+ | Kind | Where it goes |
736
+ |---|---|
737
+ | Post-promotion, production-verify, rollout-promote (`post-deploy`, `owner:orchestrator`, `owner:promote:<flag>`, or an owner why such as "after release", "rollout", "purge") | A checklist line in the repo's ONE open post-deploy rollup (`FORGE:POSTDEPLOY_ROLLUP`, else the lowest open issue labelled `followups.rollup_issue_label`). Never a new issue when one is open. |
738
+ | Read-only production data or capture (bucket `host`, or an owner why naming obs-psql, a `*-monitor` or a read-only prod query) | Class `host`: added to the parent's engineering follow-up of the same run, else one per-parent follow-up. Labels `followup:host` and `host-task`, never `needs-human`. |
739
+ | Truly external (credentials, spend, an account, a third party, a customer) | ONE issue per batch (`FORGE:EXTERNAL_BATCH: <batch id>`) holding every item under a `### #<parent>` heading, labelled `followup:external`, `external-prerequisite` and `needs-human`. |
740
+ | Engineering | The parent's open follow-up when one exists, else one issue per parent with all its items together. `ops` and `verify-now` also keep one issue per parent. |
741
+
742
+ Each issue carries the label `followup:<class>` and inherits the parent's milestone. A requirement may set an optional `class` in the contract. Otherwise `post-deploy` items are `post-deploy` and the rest `engineering`. A parent milestone that has shipped (closed) is replaced by the next open milestone by due date, else `followups.default_milestone`. The batch report lists `follow-ups appended: #N (<class>: k item(s) from #a,#b)` per issue, not "issues filed".
734
743
 
735
744
  ```yaml
736
745
  followups:
737
- default_milestone: "Backlog" # optional: milestone title used when the parent's milestone has shipped and no later one is open
746
+ default_milestone: "Backlog" # optional: milestone title used when the parent's milestone has shipped and no later one is open
747
+ rollup_issue_label: post-deploy # optional: a repo's own open issue with this label is the post-deploy rollup when no marked one is open
738
748
  ```
739
749
 
740
750
  ## `rollouts` (OPTIONAL)
@@ -888,6 +898,7 @@ verify:
888
898
  guard_command: # optional; per-extension command for the guard gate's base test files
889
899
  ".py": "scripts/pytest {path}"
890
900
  e2e: auto # optional e2e merge gate: auto (default) | always | never
901
+ precheck_block: false # optional (#5079); true: a declared pre-PR check still failing after the fix round (or a round with no commit) ends the lane needs-human ci.red (recoverable) instead of opening the PR with an anomaly
891
902
  e2e_label: full-ci # PR label that makes the repo's CI run e2e (label path)
892
903
  e2e_checks: "e2e" # check names of the e2e jobs the label path waits on
893
904
  # e2e_workflow: e2e.yml # or: a workflow_dispatch workflow that runs e2e (wins over the label)
@@ -1018,6 +1029,7 @@ execution:
1018
1029
  | `budget_minutes`, `budget_usd` | number | No | The `execution.budget` per batch: once the batch's sandbox minutes / estimated USD (settled lanes plus the running ones' elapsed time) reach the cap, no new remote lane is admitted. Missing, non-numeric or negative means no cap. |
1019
1030
  | `day_budget_minutes`, `day_budget_usd` | number | No | The same cap per host UTC day, summed over every batch on the host from the host-day store (`execution-budget.json` in `~/.forge/sched/`, or `FORGEDOCK_EXECUTION_BUDGET_FILE`). A corrupt or unreadable store refuses remote lanes while a day cap is set; remove the file to reset it. |
1020
1031
  | `fallback_local` | boolean | No | `true`: a remote-placed lane runs on the host while the provider reports no agent online (`GET {endpoint}/status -> {"online": N}`, the built-in router answers it with `online`, `total` and per-agent `ok`/`paused`), so a batch completes locally with every pool agent offline or asleep (forge#4547). Default `false`: such lanes defer (`sandbox.capacity`). A missing or failing `/status` is unknown and never triggers it. |
1032
+ | `account_rotator` | string | No | `tokenbaton`: the `claude` on PATH is the tokenbaton shim, which runs each call on the logged-in account with the most room (`CLAUDE_CONFIG_DIR=~/.claude-<n>`) (forge#5094). forgedock reads `~/.tokenbaton/current` (the account number) and `~/.tokenbaton/live.<n>` (`<5h used %> <5h reset s> <weekly used %> <weekly reset s> <updated s>`), re-read on every check and never written. An account has headroom when its 5-hour use is under 95% and its weekly use under 100% (a window whose reset has passed counts as 0). While any account has headroom no host-wide usage hold is recorded and admissions continue; they stop only when every account is exhausted and resume when the first one resets. A lane whose call hits a usage limit retries that same call (same session) on the next account at once, at most once per account, before it pauses or defers. `forgedock limits show` lists each account's 5-hour use. Any other value, or unreadable files: off (per-account holds as before). |
1021
1033
  | `warm_sandboxes` | integer | No | Warm sandboxes per repo the pool agent keeps ready (checkout at the base ref, dependency cache primed). When `N > 0` the launch spec carries `warm: N` and the lane first calls `POST {endpoint}/sandboxes/claim`; a ready warm sandbox takes the lane (refreshed to the lane's base ref with an incremental fetch before work starts), and when none is ready, the claim fails or the endpoint lacks the route the lane starts cold. Default `0` (off): no claim call and the launch spec is unchanged (forge#4912). |
1022
1034
  | `unreachable_window_min` | number | No | Remote providers: how long a lane keeps polling (with backoff, up to 30s) after the provider's transport fails (connection refused/reset, `fetch failed`, timeout, 502/503/504) before it ends `deferred` with `sandbox.unreachable` (forge#4672). Inside the window the lane never ends and the sandbox is never torn down; after it the sandbox is kept and adopted on the next attempt or `--resume`; the engine-error breaker does not count it. `adopt` applies the same window. Default: 5 |
1023
1035
  | `on_budget` | string | No | `stop` (default): running lanes finish and the rest end `held` (`execution budget: ...`) for a later `--resume`. `local`: the rest run as detached host lanes. |
@@ -1343,6 +1355,9 @@ verification:
1343
1355
  test_runners: # path prefix -> runner command prefix (block-style map)
1344
1356
  services/worker: "poetry -C services/worker run"
1345
1357
  web: "pnpm --filter web exec"
1358
+
1359
+ topology:
1360
+ route_reachability: "bash scripts/check-ingress.sh" # optional; blocking when declared
1346
1361
  ```
1347
1362
 
1348
1363
  | Field | Type | Required | Description |
@@ -1360,7 +1375,9 @@ verification:
1360
1375
  | `consumer_sweep.mode` | string | No | `advisory` (default): the validate consumer sweep (`scripts/consumer-sweep.sh`, V4.6) records unaddressed referencing sites and never fails. `blocking`: unaddressed sites fail validate with `consumer_unaddressed`, build runs one repair round, and sites still unaddressed are recorded as `CONSUMERS: unresolved` and passed to review as known risk (never `needs-human`). Flip to `blocking` only after `bash scripts/consumer-sweep.sh --eval-history 20` shows a mean of at most 2 flagged sites per merge and none above 10 on your repo |
1361
1376
  | `consumer_sweep.budget_seconds` | integer | No | Wall-clock budget for the sweep (default `60`). When it runs out the sweep stops with `BUDGET_EXCEEDED`, reports the partial result, and never fails, in either mode |
1362
1377
  | `test_runners` | map (path prefix → runner) | No | Default unset (no runner enforced). For repos whose validate commands need a managed runner (uv, poetry, pnpm workspace). The lean engine checks it at contract freeze (`testRunnerProblems`, `bin/lean/preflight.mjs`): a validate command that touches a configured path prefix without containing that prefix's runner string is reported as a contract problem, instead of failing later as `contract.invalid_command` / `env.tool_missing`. Paths in the command are relative to the runner's cwd. Write it as a block-style map: `parseForgeYaml` does not read inline `{ ... }` maps. |
1378
+ | `coupled_files` | map (glob → glob list) | No | Default unset (#5079). "If file A changes, file B must change": a block map `{when glob: required glob[, glob]}`, e.g. `"infra/guard/bootstrap.sh": "infra/guard/MANIFEST.sha256"`. A key prefixed `added:` fires only for files the lane adds (`"added:scripts/tests/test_*.py": ".github/workflows/ci.yml"`). A lane that changes a `when` file without a matching required file fails precheck with `coupled_file_missing` (`conformance.unimplemented`, builder resumed once). Quote keys that contain `*`, and list alternatives as separate keys: brace globs (`*.{ts,tsx}`) are not expanded. |
1363
1379
  | `validate.recipes` | map (regex → replacement) | No | Default unset. A top-level `validate:` block with a block-style `recipes:` map, e.g. `"poetry run pytest": "uv run --with-requirements r.txt pytest"`. Each key is a regex source; the first recipe that matches a frozen validate or `repro_test` command rewrites it at contract freeze (`applyValidateRecipes`), recorded as a decision. Use it for commands whose environment is not installed in lane sandboxes. |
1380
+ | `topology.route_reachability` | string (command) | No | Default unset (no check, no behaviour change). A repo-supplied command that takes the route paths a diff adds (Next.js `app/**/route.*`, `pages/api/**`; URL form, route groups removed) as arguments and exits non-zero when the production ingress does not deliver them to the service that implements them. Runs in the lean lane after validate and in `review-pr-staging`; a non-zero exit blocks (`conformance.unimplemented`, kind `route_unreachable`). Read by `readEnvConfig` (`bin/engine-cli.mjs`), run by `bin/lean/routecheck.mjs`. |
1364
1381
 
1365
1382
  **Commands that use this section**: `quality-gate`, `validate` (V2 runs `verification.commands`; V4.8 runs the local CI gate; health checks and `consumer_sweep`), `work-on`
1366
1383
 
@@ -56,8 +56,9 @@ An absent or unknown code is recorded as `unclassified`, with the raw reason kep
56
56
  | `env.tool_missing` | Core work-on (forge#4323): a validate command the builder recorded as failed was rerun by the engine and could not run because the host lacks a tool or module (`command not found` / exit 127, Python `No module named x`, Node `Cannot find module|package 'x'` for a bare name the repo does not track). An environment error, not a conformance failure: no builder resume, no reviewer. Also ends this way when a remote-sandbox builder reports a missing tool as a `credentials` block (forge#4685, `missingToolBlock`). Ends `needs-human` naming the tool; recoverable by `forgedock orchestrate` (the recovery lane runs the check through the repo's own toolchain). Skipped on a CI-verified head | lean | `gate()` rerun via `io.runCommand`, `{rc, output}` classified by `conformance.mjs` `missingTool` |
57
57
  | `env.repro_unrunnable` | Core work-on (forge#4334): on a bug lane the frozen `repro_test` could not be run on head by `scripts/acceptance-red-green.sh` (`HEAD=UNRUNNABLE`: timeout, exit 126/127, unclassifiable output) or the harness returned no result. An environment error, not a conformance failure: no builder resume, no reviewer. Ends `needs-human`; recoverable by `forgedock orchestrate` (the recovery lane reruns the repro command through the repo's own toolchain and installed dependencies). Other non-proofs (passes on base, fails on head, base unrunnable) stay `conformance.unimplemented` | lean | `gate()` `io.redGreen` words, classified by `conformance.mjs` `reproFailures` |
58
58
  | `env.capability_remote` | Core work-on (forge#4542): a lane placed in a remote sandbox ended `builder.blocked (credentials)` and its summary names only host capabilities declared in `execution.host_capabilities` (SOPS/age key, ops credentials, prod monitor SSH), which the sandbox does not relay but the dispatching host has. Recoverable by `forgedock orchestrate`: the recovery lane is placed local, once, under the same lease. A credential need the host lacks too stays `builder.blocked` | lean | `endBlocked` via `execution.mjs` `hostCapabilityBlock` |
59
- | `sandbox.start_failed` | A `forgedock orchestrate` lane sandbox (forge.yaml `execution`, provider `container` or `remote`) could not start: bad image, missing endpoint or credential, refused launch. The lane never ran. Final `needs-human` row: not relaunched and not counted by the engine-error circuit breaker | orchestrate | `bin/lean/execution.mjs` `providerLanes`: `provider.start` threw without `retryable` |
59
+ | `sandbox.start_failed` | A `forgedock orchestrate` lane sandbox (forge.yaml `execution`, provider `container` or `remote`) could not start: bad image, missing endpoint, a refused launch (HTTP 4xx other than 401/403/429) or a non-HTTP start failure. The lane never ran. Final `needs-human` row: not relaunched and not counted by the engine-error circuit breaker | orchestrate | `bin/lean/execution.mjs` `providerLanes`: `provider.start` threw without `retryable`, `transient` or a 5xx/401/403 `status` |
60
60
  | `sandbox.capacity` | The sandbox provider had no capacity for the lane (HTTP 429/503 from the remote transport). `deferred`, requeued after `Retry-After` when sent, else 60s; not counted toward `orchestration.max_deferrals`, like `host.busy` (forge#4528) | orchestrate | `provider.start` threw with `err.retryable: true` |
61
+ | `sandbox.transient` | The remote sandbox provider answered HTTP 500/502/504 on launch: a router or agent infrastructure failure, not a code or human problem (forge#5095). `deferred`, requeued after `Retry-After` when sent, else 60s; counted toward `orchestration.max_deferrals` (so a broken pool ends `deferred`, never `needs-human`); not counted by the engine-error circuit breaker | orchestrate | `bin/lean/execution.mjs` `providerLanes` `startFailed`: `provider.start` threw with `err.status` 5xx (not retryable) or `err.transient` |
61
62
  | `sandbox.auth` | The remote sandbox provider refused the lane's credential (HTTP 401/403) at start (forge#4685). `deferred`, retried after 300s, never `needs-human`; the pool token resolves from the env, the keychain, then `~/.config/computeshare/forgedock.token` | orchestrate | `provider.start` threw with `err.status` 401/403 |
62
63
  | `sandbox.unreachable` | The remote sandbox provider stayed unreachable (connection refused/reset, `fetch failed`, timeout, 502/503/504) for `execution.unreachable_window_min` while a lane polled or adopted its sandbox (forge#4672). `deferred`, requeued after 60s; not counted by the engine-error circuit breaker. The sandbox is NOT torn down: the next attempt or `--resume` adopts it | orchestrate | `bin/lean/execution.mjs` `providerLanes` poll guard |
63
64
  | `lease.held` | Another live agent holds the issue lease | engine | Foreign live lease on acquire |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "forgedock",
3
- "version": "1.46.0",
3
+ "version": "1.47.0",
4
4
  "description": "Autonomous development pipeline for coding agents. GitHub as institutional memory, with native Claude Code and OpenCode entrypoints.",
5
5
  "bin": {
6
6
  "forgedock": "bin/forgedock.mjs"
@@ -7,7 +7,7 @@
7
7
  # that stay deferred: trusted lookup (claim marker <!-- FORGE:DEFERRED_ISSUE: #M --> on the parent, then a state-all search) of a follow-up carrying
8
8
  # <!-- FORGE:DEFERRED_FROM: #N -->, append the missing items (exact path match; a closed one is reopened only if completed, not resolved by a person, and given new items), or create ONE issue
9
9
  # with a checklist, claim it on the parent and re-check once for a concurrent lower-numbered duplicate.
10
- # Items are grouped by class (engineering | post-deploy | external | ops, #4664): one follow-up per class per parent, each
10
+ # Items are grouped by class (engineering | post-deploy | external | ops | host, #4664/#5078): one follow-up per class per parent, each
11
11
  # labelled `followup:<class>`, carrying `<!-- FORGE:DEFERRED_CLASS: <class> -->` (absent = engineering, for older issues) and the
12
12
  # parent's milestone (a shipped/closed one: the next open milestone by due date, else FORGE_FOLLOWUPS_DEFAULT_MILESTONE).
13
13
  #
@@ -17,7 +17,12 @@
17
17
  # error (exit 2, DEFERRED_TSV=INVALID, nothing filed). The label (default: the path) is the item's identity in the checklist
18
18
  # and in the stdout rows; repeated `R<n>: R<m>:` prefixes in it collapse to the first. Class: column 4, else the why's
19
19
  # `post-deploy:|external:|ops:` prefix, else engineering.
20
- # #4724 buckets: a why carrying `[bucket:<kind>]` is routed by it: `owner` is always class external (never the rollup),
20
+ # #4724 buckets: a why carrying `[bucket:<kind>]` is routed by it (#5078): `owner` is class external only for credentials, spend,
21
+ # an account, a third party or a customer; `owner: orchestrator`, `owner: promote...` and post-promotion text join the rollup and
22
+ # read-only production data or capture is class host. `host` (labels `followup:host` + `host-task`, never `needs-human`) joins
23
+ # the parent's engineering follow-up when the run has one. External items go to ONE aggregate per batch, found by
24
+ # `<!-- FORGE:EXTERNAL_BATCH: <FORGE_BATCH_ID or adhoc> -->` (an older per-parent external follow-up is still appended to).
25
+ # The rollup may also be the lowest open trusted issue labelled FORGE_ROLLUP_LABEL (forge.yaml followups.rollup_issue_label).
21
26
  # `new-issue` engineering, `verify-now` (the parent is already on the default branch) its own per-parent follow-up class
22
27
  # `verify-now` that the gate releases at once, and `post-deploy`/`date` go to the ONE open rollup of the release lane.
23
28
  # The rollup (class post-deploy) is found by `<!-- FORGE:POSTDEPLOY_ROLLUP: <lane> -->`, not by parent: every lane appends
@@ -57,7 +62,20 @@ if [ "$ITEMS" = "-" ]; then cat > "$TMP/items.raw"; else cat "$ITEMS" > "$TMP/it
57
62
  # Internal file uses a non-whitespace separator (US, \x1f): a tab IFS collapses an empty `why` and drops the known-flaw flag.
58
63
  # Fields: identity, why, flag, path (`-` when none), class.
59
64
  SEP=$(printf '\037')
60
- CLASSES="engineering post-deploy external ops verify-now"
65
+ CLASSES="engineering post-deploy external ops verify-now host"
66
+ # #5078: an owner item is external only when it needs credentials, spend, an account, a third party or a customer. A
67
+ # post-promotion action (owner:orchestrator, owner:promote:*, after promotion/release, rollout, purge) joins the rollup and
68
+ # read-only production data or capture (obs-psql, *-monitor, a read-only prod query) is a host task. Mirrors
69
+ # bin/lean/conformance.mjs ownerRoute; word boundaries spelled (^|[^[:alnum:]_]) / ([^[:alnum:]_]|$), no \b, for BSD grep (bash 3.2 hosts).
70
+ POSTPROM_ERE='(after|post)[- ]?(promotion|release|deploy)|in production after|production[- ]verif|promot(e|ion)|rollout|purge|deploy[- ]time'
71
+ ROPROD_ERE='ssh[[:space:]]+[[:alnum:]_.-]*-monitor([^[:alnum:]_]|$)|(^|[^[:alnum:]_])obs-psql([^[:alnum:]_]|$)|(^|[^[:alnum:]_])prod(uction)?[ _-]monitor([^[:alnum:]_]|$)|(^|[^[:alnum:]_])read[- ]only[^[:alnum:]_.]([^.]{0,38}[^[:alnum:]_.])?prod(uction)?([^[:alnum:]_]|$)|(^|[^[:alnum:]_])prod(uction)?[^[:alnum:]_.]([^.]{0,28}[^[:alnum:]_.])?(query|queries|capture|telemetry|logs?)([^[:alnum:]_]|$)'
72
+ EXTNEED_ERE='credential|secret|api[- ]?key|token|password|passphrase|ssh[- ]?keys?([^[:alnum:]_]|$)|private[- ]key|(^|[^[:alnum:]_])login([^[:alnum:]_]|$)|(^|[^[:alnum:]_])sudo([^[:alnum:]_]|$)|spend|budget|billing|purchase|payment|account|third[- ]party|vendor|customer|contract sign|legal|(^|[^[:alnum:]_])(delete|drop|truncate|wipe|destroy|modify|migrate|write)([^[:alnum:]_]|$)'
73
+ owner_route() { # why -> post-deploy | host | external
74
+ case "$1" in "owner: orchestrator"*|"owner: promote"*) echo "post-deploy"; return 0 ;; esac
75
+ if printf '%s' "$1" | grep -qiE "$POSTPROM_ERE" && ! printf '%s' "$1" | grep -qiE "$EXTNEED_ERE"; then echo "post-deploy"
76
+ elif printf '%s' "$1" | grep -qiE "$ROPROD_ERE" && ! printf '%s' "$1" | grep -qiE "$EXTNEED_ERE"; then echo "host"
77
+ else echo "external"; fi
78
+ }
61
79
  : > "$TMP/items"
62
80
  BAD=0; ROW=0
63
81
  bad_row() { BAD=$((BAD+1)); echo "file-deferred-followups: row $ROW: $1" >&2; }
@@ -86,12 +104,13 @@ while IFS= read -r line || [ -n "$line" ]; do
86
104
  if [ "$id" = "-" ]; then bad_row "no path and no label"; continue; fi
87
105
  w=$(printf '%s' "$w" | tr '\r\n' ' ')
88
106
  if [ -z "$c" ]; then
89
- case "$w" in post-deploy:*) c="post-deploy" ;; external:*) c="external" ;; ops:*) c="ops" ;; *) c="engineering" ;; esac
107
+ case "$w" in post-deploy:*) c="post-deploy" ;; external:*) c="external" ;; ops:*) c="ops" ;; host:*) c="host" ;; *) c="engineering" ;; esac
90
108
  fi
91
- case " $CLASSES " in *" $c "*) ;; *) bad_row "unknown class \`$c\` (engineering|post-deploy|external|ops)"; continue ;; esac
109
+ case " $CLASSES " in *" $c "*) ;; *) bad_row "unknown class \`$c\` (engineering|post-deploy|external|ops|host)"; continue ;; esac
92
110
  # #4724: the bucket token decides the class: owner actions never join the post-deploy rollup, verify-now needs no release.
93
111
  case "$w" in
94
- *"[bucket:owner]"*) [ "$c" = "ops" ] || c="external" ;;
112
+ *"[bucket:owner]"*) [ "$c" = "ops" ] || c=$(owner_route "$w") ;;
113
+ *"[bucket:host]"*) c="host" ;;
95
114
  *"[bucket:new-issue]"*) [ "$c" = "post-deploy" ] && c="engineering" ;;
96
115
  *"[bucket:verify-now]"*) [ "$c" = "post-deploy" ] && c="verify-now" ;;
97
116
  esac
@@ -100,6 +119,12 @@ while IFS= read -r line || [ -n "$line" ]; do
100
119
  printf '%s%s%s%s%s%s%s%s%s\n' "$id" "$SEP" "$w" "$SEP" "$k" "$SEP" "${p:--}" "$SEP" "$c" >> "$TMP/items"
101
120
  done < "$TMP/items.raw"
102
121
  if [ "$BAD" -gt 0 ]; then echo "DEFERRED_TSV=INVALID"; exit 2; fi
122
+ # #5078: host items join the parent's engineering follow-up of this run (dependent items together; it gets `host-task`).
123
+ HOST_JOIN=""
124
+ if awk -F"$SEP" '$5 == "engineering" { e = 1 } $5 == "host" { h = 1 } END { exit !(e && h) }' "$TMP/items"; then
125
+ HOST_JOIN=1
126
+ awk -F"$SEP" -v OFS="$SEP" '$5 == "host" { $5 = "engineering" } { print }' "$TMP/items" > "$TMP/items.j" && mv "$TMP/items.j" "$TMP/items"
127
+ fi
103
128
  TOTAL=$(wc -l < "$TMP/items" | tr -d ' ')
104
129
 
105
130
  MARKER="<!-- FORGE:DEFERRED_FROM: #${PARENT} -->"
@@ -181,27 +206,50 @@ find_canonical() {
181
206
  return 0
182
207
  }
183
208
  # --- #4724: the post-deploy rollup: ONE open issue per release lane, found by marker (not by parent) ------------------
209
+ # #5078: the same shape holds every external item of one batch (`FORGE:EXTERNAL_BATCH: <batch>`), and the rollup may also
210
+ # be a repo's own open issue carrying forge.yaml `followups.rollup_issue_label` (FORGE_ROLLUP_LABEL) when no marked one is open.
184
211
  RELEASE_LANE="${FORGE_RELEASE_LANE:-default}"
185
212
  ROLLUP_MARKER="<!-- FORGE:POSTDEPLOY_ROLLUP: ${RELEASE_LANE} -->"
186
- verify_rollup() { # number -> `number<TAB>open` when a trusted, open, non-PR issue carries the rollup marker
187
- local out rc
213
+ BATCH="${FORGE_BATCH_ID:-adhoc}"
214
+ EXTERNAL_MARKER="<!-- FORGE:EXTERNAL_BATCH: ${BATCH} -->"
215
+ ROLLUP_LABEL="${FORGE_ROLLUP_LABEL:-}"
216
+ agg_marker() { if [ "$1" = "post-deploy" ]; then printf '%s' "$ROLLUP_MARKER"; else printf '%s' "$EXTERNAL_MARKER"; fi; }
217
+ verify_rollup() { # number [marker] -> `number<TAB>open` when a trusted, open, non-PR issue carries the marker
218
+ local out rc m="${2:-$ROLLUP_MARKER}"
188
219
  out=$(gh api "repos/${REPO}/issues/$1" 2>"$TMP/verr"); rc=$?
189
220
  if [ "$rc" -ne 0 ]; then grep -qE '404|Not Found|410' "$TMP/verr" 2>/dev/null && return 0; return 1; fi
190
- printf '%s' "$out" | jq -r --arg assoc "$TRUSTED_ASSOC" --arg logins "$TRUSTED_LOGINS" --arg m "$ROLLUP_MARKER" "$JQ_PRE"'
221
+ printf '%s' "$out" | jq -r --arg assoc "$TRUSTED_ASSOC" --arg logins "$TRUSTED_LOGINS" --arg m "$m" "$JQ_PRE"'
191
222
  select(.pull_request == null) | select(trusted) | select(.state == "open") | select((.body // "") | contains($m))
192
223
  | "\(.number)\t\(.state)"' 2>/dev/null
193
224
  }
194
- find_rollup() { # sets CANON/CSTATE to the lowest open rollup of the lane; non-zero only when a lookup failed
195
- local cands n r
225
+ verify_labelled() { # number -> `number<TAB>open` when a trusted, open, non-PR issue carries label $ROLLUP_LABEL
226
+ local out rc
227
+ out=$(gh api "repos/${REPO}/issues/$1" 2>"$TMP/verr"); rc=$?
228
+ if [ "$rc" -ne 0 ]; then grep -qE '404|Not Found|410' "$TMP/verr" 2>/dev/null && return 0; return 1; fi
229
+ printf '%s' "$out" | jq -r --arg assoc "$TRUSTED_ASSOC" --arg logins "$TRUSTED_LOGINS" --arg l "$ROLLUP_LABEL" "$JQ_PRE"'
230
+ select(.pull_request == null) | select(trusted) | select(.state == "open") | select(any(.labels[]?; (.name // .) == $l))
231
+ | "\(.number)\t\(.state)"' 2>/dev/null
232
+ }
233
+ find_aggregate() { # kind (post-deploy|external) ; sets CANON/CSTATE to the lowest open one; non-zero only when a lookup failed
234
+ local kind="$1" m q cands n r
196
235
  CANON=""; CSTATE=""
197
- cands=$(gh api -X GET search/issues -f per_page=50 -f q="\"FORGE:POSTDEPLOY_ROLLUP: ${RELEASE_LANE}\" in:body repo:${REPO} is:issue is:open" 2>/dev/null \
236
+ m=$(agg_marker "$kind"); q="${m#<!-- }"; q="${q% -->}"
237
+ cands=$(gh api -X GET search/issues -f per_page=50 -f q="\"${q}\" in:body repo:${REPO} is:issue is:open" 2>/dev/null \
238
+ | jq -r '[.items[].number] | sort | .[]' 2>/dev/null) || return 1
239
+ for n in $cands; do
240
+ r=$(verify_rollup "$n" "$m") || return 1
241
+ if [ -n "$r" ]; then CANON="${r%%$'\t'*}"; CSTATE="${r#*$'\t'}"; return 0; fi
242
+ done
243
+ [ "$kind" = "post-deploy" ] && [ -n "$ROLLUP_LABEL" ] || return 0
244
+ cands=$(gh api -X GET search/issues -f per_page=50 -f q="label:\"${ROLLUP_LABEL}\" repo:${REPO} is:issue is:open" 2>/dev/null \
198
245
  | jq -r '[.items[].number] | sort | .[]' 2>/dev/null) || return 1
199
246
  for n in $cands; do
200
- r=$(verify_rollup "$n") || return 1
247
+ r=$(verify_labelled "$n") || return 1
201
248
  if [ -n "$r" ]; then CANON="${r%%$'\t'*}"; CSTATE="${r#*$'\t'}"; return 0; fi
202
249
  done
203
250
  return 0
204
251
  }
252
+ find_rollup() { find_aggregate post-deploy; }
205
253
  # The root parent: a parent that is itself a follow-up (or the rollup) files its items under the parent it came from.
206
254
  ROOT="$PARENT"
207
255
  resolve_root() {
@@ -238,8 +286,9 @@ rollup_append() { # number ; non-zero = failure
238
286
  else gh issue edit "$num" -R "$REPO" --body-file "$TMP/newbody" >/dev/null 2>&1 || return 1; fi
239
287
  return 0
240
288
  }
241
- file_rollup() { # items in $ITEMS_F; sets FILED_N, FILED_NEW
289
+ file_rollup() { # items in $ITEMS_F, class $CLS (post-deploy: the release-lane rollup; external: the batch aggregate, #5078); sets FILED_N, FILED_NEW
242
290
  local n_items; n_items=$(wc -l < "$ITEMS_F" | tr -d ' ')
291
+ local m title what lbls
243
292
  FILED_N=""; FILED_NEW=0
244
293
  resolve_root
245
294
  CANON=$(cut -f1 "$TMP/canon.$CLS")
@@ -247,31 +296,53 @@ file_rollup() { # items in $ITEMS_F; sets FILED_N, FILED_NEW
247
296
  rollup_append "$CANON" || { echo "DEFERRED_FILE=FAILED"; exit 4; }
248
297
  FILED_N="$CANON"; return 0
249
298
  fi
299
+ m=$(agg_marker "$CLS")
300
+ if [ "$CLS" = "post-deploy" ]; then
301
+ title="post-deploy: release checklist (${RELEASE_LANE})"; what="post-deploy rollup ${RELEASE_LANE}"; lbls="followup:post-deploy"
302
+ else
303
+ title="external: prerequisites of batch ${BATCH}"; what="external aggregate of batch ${BATCH}"; lbls="followup:external external-prerequisite needs-human"
304
+ fi
250
305
  : > "$TMP/body"; rollup_missing "$ITEMS_F" "$TMP/body"
251
306
  {
252
- printf '## Problem\n\nPost-deploy checklist of release lane `%s`%s. Every merged lane appends its post-deploy items under a `### #<parent>` heading (deduplicated), so one read-only verification works through this one issue per release.\n\n' "$RELEASE_LANE" "${FORGE_RELEASE_SHA:+ (deploy target at $FORGE_RELEASE_SHA)}"
307
+ if [ "$CLS" = "post-deploy" ]; then
308
+ printf '## Problem\n\nPost-deploy checklist of release lane `%s`%s. Every merged lane appends its post-deploy items under a `### #<parent>` heading (deduplicated), so one read-only verification works through this one issue per release.\n\n' "$RELEASE_LANE" "${FORGE_RELEASE_SHA:+ (deploy target at $FORGE_RELEASE_SHA)}"
309
+ else
310
+ printf '## Problem\n\nExternal prerequisites of batch `%s`: every item below waits on something outside the repo (credentials, spend, an account, a third party or a customer). Every lane of the batch appends its items under a `### #<parent>` heading (deduplicated), so the batch has ONE external issue.' "$BATCH"
311
+ [ -n "${FORGE_EXTERNAL_PREREQ:-}" ] && printf ' Linked prerequisite: #%s.' "${FORGE_EXTERNAL_PREREQ#\#}"
312
+ printf '\n\n'
313
+ fi
253
314
  printf '## Acceptance Criteria\n\n### #%s\n<!-- FORGE:DEFERRED_FROM: #%s -->\n\n' "$ROOT" "$ROOT"
254
315
  cat "$TMP/add"
255
- printf '\n%s\n<!-- FORGE:DEFERRED_CLASS: post-deploy -->\n' "$ROLLUP_MARKER"
316
+ printf '\n%s\n<!-- FORGE:DEFERRED_CLASS: %s -->\n' "$m" "$CLS"
256
317
  } > "$TMP/newissue"
257
318
  local NUM
258
319
  if [ -n "${DRY_RUN:-}" ]; then
259
- echo "DRY_RUN: gh issue create (post-deploy rollup ${RELEASE_LANE}: ${n_items} item(s) from #${PARENT})" >&2
320
+ echo "DRY_RUN: gh issue create (${what}: ${n_items} item(s) from #${PARENT})" >&2
260
321
  NUM=0
261
322
  else
262
323
  resolve_milestone
263
- local args=() URL
264
- gh label create "followup:post-deploy" -R "$REPO" --force --color BFD4F2 --description "Deferred post-deploy checks. Managed by ForgeDock." >/dev/null 2>&1 || true
265
- args=(--label "followup:post-deploy")
324
+ local args=() URL l
325
+ for l in $lbls; do
326
+ case "$l" in
327
+ needs-human) gh label create "$l" -R "$REPO" --color D93F0B --description "Needs a human decision or action." >/dev/null 2>&1 || true ;; # never --force: the repo's own label wins
328
+ external-prerequisite) gh label create "$l" -R "$REPO" --force --color E99695 --description "External prerequisite. Managed by ForgeDock." >/dev/null 2>&1 || true ;;
329
+ *) gh label create "$l" -R "$REPO" --force --color BFD4F2 --description "Deferred follow-up (${CLS}). Managed by ForgeDock." >/dev/null 2>&1 || true ;;
330
+ esac
331
+ args+=(--label "$l")
332
+ done
266
333
  [ -n "$MS_TITLE" ] && args+=(--milestone "$MS_TITLE")
267
- URL=$(gh issue create -R "$REPO" --title "post-deploy: release checklist (${RELEASE_LANE})" --body-file "$TMP/newissue" "${args[@]}" 2>/dev/null) || URL=""
334
+ URL=$(gh issue create -R "$REPO" --title "$title" --body-file "$TMP/newissue" "${args[@]}" 2>/dev/null) || URL=""
268
335
  NUM=$(printf '%s' "$URL" | grep -oE '[0-9]+$' | tail -n 1)
269
336
  [ -n "$NUM" ] || { echo "DEFERRED_FILE=FAILED"; exit 4; }
270
- # Post-create re-check: a concurrent lane may have filed first. The lowest open rollup is canonical; the newer one is closed.
271
- if find_rollup; then
337
+ if [ "$CLS" = "external" ]; then # the parent's claim marker, as for a per-parent follow-up (best effort)
338
+ gh issue comment "$PARENT" -R "$REPO" --body "<!-- FORGE:DEFERRED_ISSUE: #${NUM} class=external -->" >/dev/null 2>&1 \
339
+ || echo "file-deferred-followups: warning: could not post claim marker for #${NUM} on #${PARENT}" >&2
340
+ fi
341
+ # Post-create re-check: a concurrent lane may have filed first. The lowest open one is canonical; the newer one is closed.
342
+ if find_aggregate "$CLS"; then
272
343
  if [ -n "$CANON" ] && [ "$CANON" -lt "$NUM" ]; then
273
344
  if rollup_append "$CANON"; then
274
- gh issue close "$NUM" -R "$REPO" --reason "not planned" --comment "Duplicate of #${CANON}: the post-deploy items of lane ${RELEASE_LANE} are tracked there." >/dev/null 2>&1 \
345
+ gh issue close "$NUM" -R "$REPO" --reason "not planned" --comment "Duplicate of #${CANON}: the ${what} items are tracked there." >/dev/null 2>&1 \
275
346
  || echo "file-deferred-followups: warning: could not close duplicate #${NUM}" >&2
276
347
  NUM="$CANON"
277
348
  else
@@ -286,19 +357,34 @@ file_rollup() { # items in $ITEMS_F; sets FILED_N, FILED_NEW
286
357
  }
287
358
 
288
359
  CLS_LIST=$(cut -d "$SEP" -f5 "$TMP/items" | sort -u)
289
- CLS=""
360
+ CLS=""; CKIND=""
361
+ # #5078: post-deploy -> the release-lane rollup; external -> the batch aggregate, else (older runs) the parent's own external
362
+ # follow-up; every other class -> the parent's follow-up of that class. Sets CANON/CSTATE/CKIND (agg|legacy|class).
363
+ lookup_class() {
364
+ CKIND="agg"
365
+ case "$CLS" in
366
+ post-deploy) find_aggregate post-deploy ;;
367
+ external)
368
+ find_aggregate external || return 1
369
+ [ -n "$CANON" ] && return 0
370
+ find_canonical || return 1
371
+ [ -n "$CANON" ] && CKIND="legacy"
372
+ return 0 ;;
373
+ *) CKIND="class"; find_canonical ;;
374
+ esac
375
+ }
290
376
  # Phase 1: look up the canonical follow-up of every class before filing anything (a failed lookup files nothing).
291
377
  for CLS in $CLS_LIST; do
292
378
  LOOKUP_OK=""
293
379
  for attempt in 1 2; do
294
- if { [ "$CLS" = "post-deploy" ] && find_rollup; } || { [ "$CLS" != "post-deploy" ] && find_canonical; }; then LOOKUP_OK=1; break; fi
380
+ if lookup_class; then LOOKUP_OK=1; break; fi
295
381
  [ "$attempt" = 1 ] && sleep "${FORGE_DEFERRED_RETRY_SLEEP:-2}"
296
382
  done
297
383
  if [ -z "$LOOKUP_OK" ]; then
298
384
  echo "DEFERRED_LOOKUP=FAILED"
299
385
  exit 3
300
386
  fi
301
- printf '%s\t%s\n' "$CANON" "$CSTATE" > "$TMP/canon.$CLS"
387
+ printf '%s\t%s\t%s\n' "$CANON" "$CSTATE" "$CKIND" > "$TMP/canon.$CLS"
302
388
  awk -F"$SEP" -v c="$CLS" '$5 == c' "$TMP/items" > "$TMP/items.$CLS"
303
389
  done
304
390
 
@@ -366,6 +452,10 @@ file_class() {
366
452
  CANON=$(cut -f1 "$TMP/canon.$CLS"); CSTATE=$(cut -f2 "$TMP/canon.$CLS")
367
453
  if [ -n "$CANON" ]; then
368
454
  append_items "$CANON" "$CSTATE" || { echo "DEFERRED_FILE=FAILED"; exit 4; }
455
+ if [ "$CLS" = "engineering" ] && [ -n "$HOST_JOIN" ]; then
456
+ if [ -n "${DRY_RUN:-}" ]; then echo "DRY_RUN: gh issue edit $CANON (add label host-task)" >&2
457
+ else gh issue edit "$CANON" -R "$REPO" --add-label host-task >/dev/null 2>&1 || echo "file-deferred-followups: warning: could not label #${CANON} host-task" >&2; fi
458
+ fi
369
459
  FILED_N="$CANON"; return 0
370
460
  fi
371
461
  local claim="<!-- FORGE:DEFERRED_ISSUE: #%s -->" title="follow-up: ${n_items} deferred item(s) (deferred from #${PARENT})"
@@ -400,6 +490,10 @@ file_class() {
400
490
  gh label create external-prerequisite -R "$REPO" --force --color E99695 --description "External prerequisite. Managed by ForgeDock." >/dev/null 2>&1 || true
401
491
  args+=(--label external-prerequisite)
402
492
  fi
493
+ if [ "$CLS" = "host" ] || { [ "$CLS" = "engineering" ] && [ -n "$HOST_JOIN" ]; }; then # #5078: never needs-human
494
+ gh label create host-task -R "$REPO" --color 0E8A16 --description "Work the host can run (read-only production access)." >/dev/null 2>&1 || true
495
+ args+=(--label host-task)
496
+ fi
403
497
  [ -n "$MS_TITLE" ] && args+=(--milestone "$MS_TITLE")
404
498
  URL=$(gh issue create -R "$REPO" --title "$title" --body-file "$TMP/newissue" "${args[@]}" 2>/dev/null) || URL=""
405
499
  NUM=$(printf '%s' "$URL" | grep -oE '[0-9]+$' | tail -n 1)
@@ -434,7 +528,10 @@ file_class() {
434
528
  INTO=""; FILED=0
435
529
  for CLS in $CLS_LIST; do
436
530
  ITEMS_F="$TMP/items.$CLS"
437
- if [ "$CLS" = "post-deploy" ]; then file_rollup; else file_class; fi
531
+ case "$CLS:$(cut -f3 "$TMP/canon.$CLS")" in
532
+ post-deploy:*|external:agg) file_rollup ;;
533
+ *) file_class ;;
534
+ esac
438
535
  case " $INTO " in *" $FILED_N "*) ;; *) INTO="${INTO:+$INTO }$FILED_N" ;; esac
439
536
  FILED=$((FILED + FILED_NEW))
440
537
  while IFS="$SEP" read -r p w k fp cl; do printf '%s\t%s\n' "$p" "$FILED_N"; done < "$ITEMS_F"
@@ -338,5 +338,54 @@ printf -- '-\tdefer: later\t\t\tR2: wire `pd.py` in\n' > "$T/fu.tsv"
338
338
  OUT=$(bash "$S" --parent 20 --repo o/r --items "$T/fu.tsv"); RC=$?
339
339
  check "follow-up parent: new follow-up issue, parent body unchanged" "[ $RC -eq 0 ] && [ \$(grep -c '^issue create' '$GH_LOG') -eq 1 ] && ! grep -q '^issue edit 20' '$GH_LOG' && [ ! -e '$ISSUE_BODY.edited' ] && grep -q '^\*\*Parent\*\*: #20\$' '$T/created.901.body' && grep -q 'FORGE:DEFERRED_FROM: #20' '$T/created.901.body' && printf '%s\n' \"\$OUT\" | grep -q '^R2: wire pd.py in.901\$'"
340
340
 
341
+ # 24. #5078: routing by kind
342
+ # post-promotion owner deferral appends to the open rollup, creates no issue
343
+ reset
344
+ printf '{"number":902,"state":"open","author_association":"OWNER","user":{"type":"User"},"body":"<!-- FORGE:POSTDEPLOY_ROLLUP: default -->"}\n' > "$T/issue.902.json"
345
+ echo '{"items":[{"number":902}]}' > "$SEARCH_JSON"; printf '<!-- FORGE:POSTDEPLOY_ROLLUP: default -->\n' > "$ISSUE_BODY"
346
+ printf -- '-\towner: orchestrator. [bucket:owner] post-deploy action with no registered check\t\t\tR1: verify the banner in prod\n-\towner: promote:new-nav. [bucket:owner] flip the flag\t\t\tR2: promote the new nav\n-\towner: ops. [bucket:owner] purge the CDN after release\t\t\tR3: purge the cdn\n' > "$T/pp.tsv"
347
+ OUT=$(bash "$S" --parent 10 --repo o/r --items "$T/pp.tsv"); RC=$?
348
+ check "routing: post-promotion owner items append to the rollup, no issue create" "[ $RC -eq 0 ] && ! grep -q '^issue create' '$GH_LOG' && grep -q '^issue edit 902' '$GH_LOG' && [ \$(grep -c '^- \[ \] ' '$ISSUE_BODY.edited') -eq 3 ] && printf '%s' \"\$OUT\" | grep -q 'classes=post-deploy '"
349
+ # a configured-label rollup (forge.yaml followups.rollup_issue_label) is used when no marked rollup is open
350
+ reset
351
+ printf '{"number":950,"state":"open","author_association":"OWNER","user":{"type":"User"},"labels":[{"name":"post-deploy"}],"body":"release checklist"}\n' > "$T/issue.950.json"
352
+ echo '{"items":[{"number":950}]}' > "$SEARCH_JSON"; printf 'release checklist\n' > "$ISSUE_BODY"
353
+ printf -- '-\tpost-deploy: [bucket:post-deploy] [check:smoke] check it\t\t\tR4: smoke the login\n' > "$T/lab.tsv"
354
+ OUT=$(FORGE_ROLLUP_LABEL=post-deploy bash "$S" --parent 10 --repo o/r --items "$T/lab.tsv"); RC=$?
355
+ check "routing: labelled rollup #950 appended, no issue create" "[ $RC -eq 0 ] && ! grep -q '^issue create' '$GH_LOG' && grep -q '^issue edit 950' '$GH_LOG' && grep -q 'smoke the login' '$ISSUE_BODY.edited' && printf '%s\n' \"\$OUT\" | grep -q '^R4: smoke the login.950\$'"
356
+ reset
357
+ OUT=$(bash "$S" --parent 10 --repo o/r --items "$T/lab.tsv"); RC=$?
358
+ check "routing: without the label setting an unmarked labelled issue is not the rollup" "[ $RC -eq 0 ] && [ \$(grep -c '^issue create' '$GH_LOG') -eq 1 ]"
359
+ # read-only production data -> host follow-up, host-task, never needs-human / external-prerequisite
360
+ reset
361
+ printf -- '-\towner: obs-psql. [bucket:owner] capture the read-only production query counts\t\t\tR5: capture prod counts\n' > "$T/host.tsv"
362
+ OUT=$(bash "$S" --parent 10 --repo o/r --items "$T/host.tsv"); RC=$?
363
+ check "routing: read-only prod owner item -> one host follow-up labelled host-task" "[ $RC -eq 0 ] && [ \$(grep -c '^issue create' '$GH_LOG') -eq 1 ] && grep '^issue create' '$GH_LOG' | grep -q -- '--label followup:host' && grep '^issue create' '$GH_LOG' | grep -q -- '--label host-task' && grep -q 'FORGE:DEFERRED_CLASS: host' '$T/created.901.body'"
364
+ check "routing: host follow-up carries no needs-human or external-prerequisite" "! grep '^issue create' '$GH_LOG' | grep -qE 'needs-human|external-prerequisite'"
365
+ reset
366
+ printf -- '-\t[bucket:host] host: run the obs-psql report\t\t\tR6: report rows\n-\tdefer: [bucket:new-issue] wire it\t\t\tR7: wire the report\n' > "$T/join.tsv"
367
+ OUT=$(bash "$S" --parent 10 --repo o/r --items "$T/join.tsv"); RC=$?
368
+ check "routing: host items join the engineering follow-up of the same run" "[ $RC -eq 0 ] && [ \$(grep -c '^issue create' '$GH_LOG') -eq 1 ] && grep -q 'report rows' '$T/created.901.body' && grep -q 'wire the report' '$T/created.901.body' && grep '^issue create' '$GH_LOG' | grep -q -- '--label followup:engineering' && grep '^issue create' '$GH_LOG' | grep -q -- '--label host-task' && ! grep '^issue create' '$GH_LOG' | grep -q needs-human"
369
+ # N external deferrals in one batch -> ONE aggregated issue carrying needs-human
370
+ reset
371
+ export FORGE_BATCH_ID=b42
372
+ printf -- '-\towner: stripe. [bucket:owner] needs live Stripe credentials\t\t\tR8: charge a real card\n' > "$T/e1.tsv"
373
+ OUT=$(bash "$S" --parent 10 --repo o/r --items "$T/e1.tsv"); RC=$?
374
+ check "external batch: first parent creates the aggregate with needs-human" "[ $RC -eq 0 ] && [ \$(grep -c '^issue create' '$GH_LOG') -eq 1 ] && grep -q 'FORGE:EXTERNAL_BATCH: b42' '$T/created.901.body' && grep -q '^### #10\$' '$T/created.901.body' && grep '^issue create' '$GH_LOG' | grep -q -- '--label needs-human' && grep '^issue create' '$GH_LOG' | grep -q -- '--label external-prerequisite'"
375
+ printf '{"number":901,"state":"open","author_association":"OWNER","user":{"type":"User"},"body":"<!-- FORGE:EXTERNAL_BATCH: b42 -->"}\n' > "$T/issue.901.json"
376
+ echo '{"items":[{"number":901}]}' > "$SEARCH_JSON"; cp "$T/created.901.body" "$ISSUE_BODY"
377
+ printf -- '-\texternal: [bucket:owner] buy the domain (spend)\t\t\tR1: register the domain\n' > "$T/e2.tsv"
378
+ OUT2=$(bash "$S" --parent 11 --repo o/r --items "$T/e2.tsv"); cp "$ISSUE_BODY.edited" "$ISSUE_BODY"
379
+ printf -- '-\texternal: customer sign-off\t\t\tR2: customer signs the DPA\n-\texternal: dup\t\t\tR9: Charge a real card!\n' > "$T/e3.tsv"
380
+ OUT3=$(bash "$S" --parent 12 --repo o/r --items "$T/e3.tsv"); RC=$?
381
+ check "external batch: three parents, exactly one issue create, later lanes append to it" "[ $RC -eq 0 ] && [ \$(grep -c '^issue create' '$GH_LOG') -eq 1 ] && [ \$(grep -c '^issue edit 901' '$GH_LOG') -eq 2 ] && printf '%s\n' \"\$OUT3\" | grep -q '^R2: customer signs the DPA.901\$'"
382
+ check "external batch: per-parent headings, duplicate text stored once" "grep -q '^### #11\$' '$ISSUE_BODY.edited' && grep -q '^### #12\$' '$ISSUE_BODY.edited' && [ \$(grep -ci 'charge a real card' '$ISSUE_BODY.edited') -eq 1 ]"
383
+ unset FORGE_BATCH_ID
384
+ # engineering deferrals from one parent land in one issue
385
+ reset
386
+ printf -- '-\tdefer: parser first\t\t\tR1: add the parser\n-\tdefer: needs the parser\t\t\tR2: use the parser\n' > "$T/eng.tsv"
387
+ OUT=$(bash "$S" --parent 10 --repo o/r --items "$T/eng.tsv"); RC=$?
388
+ check "engineering: dependent items of one parent in one issue" "[ $RC -eq 0 ] && [ \$(grep -c '^issue create' '$GH_LOG') -eq 1 ] && grep -q 'add the parser' '$T/created.901.body' && grep -q 'use the parser' '$T/created.901.body'"
389
+
341
390
  echo "passed=$PASS failed=$FAIL"
342
391
  [ "$FAIL" -eq 0 ]
@@ -144,6 +144,13 @@ if [ -n "$APPEND" ]; then
144
144
  BODY="${BODY}${APPEND}"
145
145
  echo "WARNING: body was missing mandatory sections; placeholders added before creation" >&2
146
146
  fi
147
+ # #5085: a bug/fix issue should carry a ## Prior Attempts table (prior-attempts.mjs); warn only, never block
148
+ IS_BUG=false
149
+ for l in ${LABELS[@]+"${LABELS[@]}"}; do [ "$l" = bug ] && IS_BUG=true; done
150
+ printf '%s' "$TITLE" | grep -qE '^(fix|bug)(\(|:|!)' && IS_BUG=true
151
+ if [ "$IS_BUG" = true ] && ! printf '%s\n' "$BODY" | grep -q '^## Prior Attempts'; then
152
+ echo "WARNING: bug/fix issue has no ## Prior Attempts section (run prior-attempts.mjs)" >&2
153
+ fi
147
154
 
148
155
  if [ "$DRY_RUN" = true ]; then
149
156
  LABEL_TEXT=""; for l in ${LABELS[@]+"${LABELS[@]}"}; do LABEL_TEXT="${LABEL_TEXT:+$LABEL_TEXT, }$l"; done
@@ -158,6 +158,19 @@ log_lacks "no milestone after failed read-back" "--milestone"
158
158
  MOCK_READBACK=missing-sections run "post-create repair is not fatal" 0 --repo o/r --title "fix: a" --body-file "$GOOD"
159
159
  log_has "missing section appended after create" "|issue|edit|901|-R|o/r|--body|"
160
160
 
161
+ # --- #5085: bug/fix issues without ## Prior Attempts warn, never block ---------------
162
+ WARN="no ## Prior Attempts section"
163
+ run "fix title without prior attempts still creates" 0 --repo o/r --title "fix(api): a" --body-file "$GOOD"
164
+ grep -qF "$WARN" "$TMP/err" && ok || bad "fix title warns about missing ## Prior Attempts"
165
+ out_has "warning keeps the CREATED line" "ISSUE_CREATE_RESULT:CREATED"
166
+ run "bug label warns" 0 --repo o/r --title "chore: a" --body-file "$GOOD" --label bug
167
+ grep -qF "$WARN" "$TMP/err" && ok || bad "bug label warns about missing ## Prior Attempts"
168
+ PA="$TMP/pa.md"; { printf '## Prior Attempts\n\nNone found (terms: a).\n\n'; cat "$GOOD"; } > "$PA"
169
+ run "fix with prior attempts" 0 --repo o/r --title "fix: a" --body-file "$PA"
170
+ grep -qF "$WARN" "$TMP/err" && bad "warned although ## Prior Attempts is present" || ok
171
+ run "feat never warns" 0 --repo o/r --title "feat: a" --body-file "$GOOD"
172
+ grep -qF "$WARN" "$TMP/err" && bad "feat issue warned about ## Prior Attempts" || ok
173
+
161
174
  # --- dedup helper comes from the script's own directory, never the cwd ---------------
162
175
  printf '#!/usr/bin/env bash\necho "DUPLICATE: #666 planted"; exit 1\n' > "$TMP/work/scripts/issue-dedup.sh"
163
176
  run "planted cwd dedup helper ignored" 0 --repo o/r --title "fix: a" --body-file "$GOOD"
@@ -43,8 +43,8 @@
43
43
  # 3. Remove any --exclude'd issue numbers from the candidate set.
44
44
  # 4. For each remaining candidate open issue, count shared tokens (length ≥ 4)
45
45
  # between the proposed title and the candidate title.
46
- # 5. MATCH threshold: ≥ 3 shared tokens OR shared token count ≥ 50% of
47
- # the shorter title's token set — whichever fires first.
46
+ # 5. MATCH threshold: ≥ 3 shared tokens OR (≥ 2 shared tokens AND ≥ 50% of
47
+ # the shorter title's token set). One shared word never matches (#5080).
48
48
  # 6. On match: print "DUPLICATE: #<N> <title>" to stdout, exit 1.
49
49
  # 7. On no match: exit 0 (silent).
50
50
  #
@@ -243,7 +243,8 @@ fi
243
243
  # --------------------------------------------------------------------------- #
244
244
  # Token overlap matching
245
245
  # For each candidate, count shared tokens between proposed and candidate titles.
246
- # Match if: shared_count >= 3 OR shared_count >= 50% of the shorter token set.
246
+ # Match if: shared_count >= 3 OR (shared_count >= 2 AND >= 50% of the shorter token set).
247
+ # A single shared word never matches (#5080: one generic noun is not a duplicate).
247
248
  # --------------------------------------------------------------------------- #
248
249
  MATCH_NUMBER=""
249
250
  MATCH_TITLE=""
@@ -279,7 +280,7 @@ while IFS=$'\t' read -r CAND_NUMBER CAND_TITLE; do
279
280
  # 50% threshold: shared >= ceil(shorter / 2)
280
281
  HALF_THRESHOLD=$(( (SHORTER + 1) / 2 ))
281
282
 
282
- if [[ "$SHARED_COUNT" -ge 3 ]] || [[ "$SHARED_COUNT" -ge "$HALF_THRESHOLD" && "$SHORTER" -ge 2 ]]; then
283
+ if [[ "$SHARED_COUNT" -ge 3 ]] || [[ "$SHARED_COUNT" -ge 2 && "$SHARED_COUNT" -ge "$HALF_THRESHOLD" && "$SHORTER" -ge 2 ]]; then
283
284
  MATCH_NUMBER="$CAND_NUMBER"
284
285
  MATCH_TITLE="$CAND_TITLE"
285
286
  break
@@ -137,6 +137,21 @@ MOCK_GH_CANDIDATES="$MEMBERS_ONLY" assert_exit \
137
137
  "--exclude alongside pre-joined -R token -> exit 0" \
138
138
  0 "$PROPOSED_TITLE" "-R test/repo" --exclude "2422,2424"
139
139
 
140
+ # --------------------------------------------------------------------------- #
141
+ # Scenario F (#5080): one shared generic noun is not a duplicate; two shared tokens still are.
142
+ # --------------------------------------------------------------------------- #
143
+ ONE_NOUN="$TMP_FIXTURES/one_noun.tsv"
144
+ printf '30575\tintegrate platform policy with browser actions and workflows\n' > "$ONE_NOUN"
145
+ MOCK_GH_CANDIDATES="$ONE_NOUN" assert_exit \
146
+ "one shared noun (workflows) does not match an unrelated feature -> exit 0" \
147
+ 0 "bug: workflows broken" -R test/repo
148
+
149
+ TWO_SHARED="$TMP_FIXTURES/two_shared.tsv"
150
+ printf '31000\tworkflows page returns 401\n' > "$TWO_SHARED"
151
+ MOCK_GH_CANDIDATES="$TWO_SHARED" assert_exit \
152
+ "two shared tokens on a short title still match -> exit 1" \
153
+ 1 "bug: workflows page returns 401 unauthorized" -R test/repo
154
+
140
155
  echo ""
141
156
  echo "Results: $PASS passed, $FAIL failed"
142
157
  [[ "$FAIL" -eq 0 ]]
@@ -0,0 +1,32 @@
1
+ #!/usr/bin/env bash
2
+ # SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios
3
+ # SPDX-License-Identifier: AGPL-3.0-or-later
4
+ # Release review #5103 (QA-3, BUG-2): scripts/file-deferred-followups.sh owner_route must agree with
5
+ # bin/lean/conformance.mjs ownerRoute on every case, including the over-broad matches the review found.
6
+ set -u
7
+ ROOT=$(cd "$(dirname "$0")/.." && pwd)
8
+ eval "$(sed -n '/^POSTPROM_ERE=/,/^}/p' "$ROOT/scripts/file-deferred-followups.sh")"
9
+ PASS=0; FAIL=0
10
+ while IFS='|' read -r want why; do
11
+ [ -z "$why" ] && continue
12
+ sh=$(owner_route "$why")
13
+ js=$(node -e 'import(process.argv[1]).then(m=>console.log(m.ownerRoute({bucket:"owner:x", why: process.argv[2]})))' "$ROOT/bin/lean/conformance.mjs" "$why")
14
+ if [ "$sh" = "$want" ] && [ "$js" = "$want" ]; then PASS=$((PASS+1)); else FAIL=$((FAIL+1)); echo "FAIL: '$why' want=$want sh=$sh js=$js"; fi
15
+ done <<'CASES'
16
+ host|read-only production telemetry from ovh
17
+ host|ssh ovh-monitor to capture the queue depth
18
+ host|obs-psql count of the jobs table
19
+ host|run a production query against the replica
20
+ host|capture production logs for the failing run
21
+ host|prod query of the job table
22
+ external|Should we delete production data older than 90 days?
23
+ external|add uptime-monitor dashboard
24
+ external|needs the production database password to query prod data
25
+ external|needs the prod ssh key to read production logs
26
+ external|product manager decides the pricing page
27
+ external|production dataset retention policy
28
+ external|provide the Stripe API key, needed before rollout
29
+ post-deploy|verify the dashboard after release
30
+ CASES
31
+ echo "passed=$PASS failed=$FAIL"
32
+ [ "$FAIL" -eq 0 ]