forgedock 1.9.0 → 1.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/commands/orchestrate/phase-4-execution.md +12 -2
- package/commands/quality-gate.md +29 -0
- package/commands/review-pr.md +67 -10
- package/commands/work-on/build/validate.md +15 -9
- package/commands/work-on/remediate.md +1 -1
- package/commands/work-on/review.md +18 -2
- package/docs/RELEASE-NOTES.md +20 -0
- package/package.json +1 -1
- package/scripts/check-spec-bash.sh +120 -0
- package/scripts/check-spec-bash.test.sh +90 -0
- package/scripts/check-spec-markers.sh +3 -0
- package/scripts/classify-finding.sh +31 -4
- package/scripts/classify-finding.test.sh +24 -4
|
@@ -2837,10 +2837,20 @@ When `CASCADE_MAX_AMPLIFICATION` is not `off`, and the current ratio is greater
|
|
|
2837
2837
|
**Deterministic breaker check (MANDATORY on every dispatch path — engine CLI, Agent-spawn fallback, or a hand-driven orchestrator):** the in-spec bookkeeping above only runs when the Step 4B/4C bash runs. A batch driven through the Agent-spawn fallback never evaluated it and reached about 2 findings per merge (2026-10-08 audit). Before dispatching ANY `review-finding` issue at `priority:P3` or below, and after every completed merge, run `scripts/amplification-breaker.sh` against GitHub state. Its exit code is authoritative, and it overrides an in-memory `AMPLIFICATION_BREAKER_TRIPPED=false`:
|
|
2838
2838
|
|
|
2839
2839
|
```bash
|
|
2840
|
+
# Same resolver as review-pr §6B.5: plugin root, FORGE_ROOT, FORGEDOCK_HOME, newest pinned plugin cache
|
|
2841
|
+
# (CLAUDE_CONFIG_DIR, then ~/.claude), then the repo's own scripts/.
|
|
2840
2842
|
AMP_SCRIPT=""
|
|
2841
|
-
|
|
2842
|
-
|
|
2843
|
+
_cands="$(printf '%s\n' '${CLAUDE_PLUGIN_ROOT}' "${FORGE_ROOT:-}" "${FORGEDOCK_HOME:-}")"
|
|
2844
|
+
for _cfg in "${CLAUDE_CONFIG_DIR:-}" "$HOME/.claude"; do
|
|
2845
|
+
[ -n "$_cfg" ] || continue
|
|
2846
|
+
_cands="$_cands
|
|
2847
|
+
$(find -L "$_cfg/plugins/cache/forgedock/forgedock" -mindepth 1 -maxdepth 1 -type d 2>/dev/null | awk -F/ '$NF ~ /^[0-9]+\.[0-9]+\.[0-9]+$/{split($NF,a,".");printf "%d %d %d %s\n",a[1],a[2],a[3],$0}' | sort -k1,1nr -k2,2nr -k3,3nr | cut -d' ' -f4- || true)"
|
|
2843
2848
|
done
|
|
2849
|
+
_cands="$_cands
|
|
2850
|
+
$PWD"
|
|
2851
|
+
while IFS= read -r _c; do
|
|
2852
|
+
case "$_c" in /*) [ -z "$AMP_SCRIPT" ] && [ -f "$_c/scripts/amplification-breaker.sh" ] && AMP_SCRIPT="$_c/scripts/amplification-breaker.sh" ;; esac
|
|
2853
|
+
done <<< "$_cands"
|
|
2844
2854
|
if [ "$AMPLIFICATION_BREAKER" != "off" ]; then
|
|
2845
2855
|
if [ -n "$AMP_SCRIPT" ]; then
|
|
2846
2856
|
AMP_LINE=$(bash "$AMP_SCRIPT" --since "$BATCH_T0" -R {GH_REPO}); AMP_RC=$?
|
package/commands/quality-gate.md
CHANGED
|
@@ -277,6 +277,7 @@ the gate rather than leaving the workflow blocked.
|
|
|
277
277
|
- **2R (Registry checks)**: ALWAYS run — executes all promoted checks from `scripts/check-registry/manifest.json` <!-- Added: forge#1331 -->
|
|
278
278
|
- **2R.5 (Adaptive gate.d checks)**: ALWAYS run when `adaptive_scripts.enabled` — executes `.forgedock/scripts/gate.d/*.sh`; each script is repo-specific and was promoted from a recurring finding pattern <!-- Added: forge#1739 -->
|
|
279
279
|
- **2G.8 (Wire-through proof)**: Run if `WIRE_THROUGH` in DOMAINS — demands each newly added conditional path be demonstrably reachable <!-- Added: forge#1731 -->
|
|
280
|
+
- **2G.10 (Spec bash syntax)**: Run if `FORGE_GRAPH` in DOMAINS and any `commands/**/*.md` changed — `bash -n` on changed ```bash blocks in specs (HIGH); shellcheck advisory
|
|
280
281
|
- **2G.9 (Danger-zone recurrence)**: Run if `FORGE_GRAPH` in DOMAINS — cross-references injected danger-zone rule cards from FORGE:CONTEXT against the diff; violations tagged `known-pattern-recurrence` (HIGH) <!-- Added: forge#1744 -->
|
|
281
282
|
- **2U (Coverage-reduction check)**: ALWAYS run (`COVERAGE_REDUCTION` in DOMAINS) — flags diffs that delete test files, remove test cases, add skip markers, or remove/disable workflow test steps <!-- Added: forge#3257 -->
|
|
282
283
|
- **2S (Test failure classification)**: Run if any Step 2 check invoked a test suite and it failed — classifies the failure as PRE_BROKEN, FLAKY, or REAL using `scripts/flaky-quarantine.sh` <!-- Added: forge#1336 -->
|
|
@@ -718,6 +719,34 @@ fi
|
|
|
718
719
|
|
|
719
720
|
Do NOT suppress the finding by making the check less strict or adding fake no-op tests. The intent is a real proof that the path is reachable. <!-- Added: forge#1731 -->
|
|
720
721
|
|
|
722
|
+
### 2G.10: Spec bash syntax check (FORGE_GRAPH domain)
|
|
723
|
+
|
|
724
|
+
**Triggered when**: changed files include `commands/**/*.md`. Skips silently in any project without `scripts/check-spec-bash.sh`.
|
|
725
|
+
|
|
726
|
+
**Why this matters**: in ForgeDock the pipeline code is bash inside Markdown specs. The executable-file checks above exclude `.md` files, and acceptance checks are mostly greps, so spec bash used to merge without ever being parsed. Reviewers were then the first to run it, which drove most review findings (2026-10-08 cascade audit). This check runs `bash -n` on every ```bash block that contains a changed line, after neutralising `{PLACEHOLDER}` notation. Blocks with pseudo tool calls (`Skill(`/`Agent(`/`Task(`) and blocks preceded by `<!-- allowlist:check-spec-bash -->` are skipped. Pre-existing blocks are never checked, so the baseline cannot block a change. `shellcheck -S error` results are advisory only.
|
|
727
|
+
|
|
728
|
+
```bash
|
|
729
|
+
SPEC_BASH_CHECKER="$FORGEDOCK_SCRIPTS/check-spec-bash.sh"
|
|
730
|
+
[ -f "$SPEC_BASH_CHECKER" ] || SPEC_BASH_CHECKER="{WORKTREE_PATH}/scripts/check-spec-bash.sh"
|
|
731
|
+
SPEC_MD_FILES=$(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '^commands/.*\.md$' || true)
|
|
732
|
+
if [ -f "$SPEC_BASH_CHECKER" ] && [ -n "$SPEC_MD_FILES" ]; then
|
|
733
|
+
PR_BASE="${PR_BASE:-staging}"
|
|
734
|
+
SPEC_BASH_BASE=$(git -C "{WORKTREE_PATH}" merge-base "origin/${PR_BASE}" HEAD 2>/dev/null || true)
|
|
735
|
+
if [ -z "$SPEC_BASH_BASE" ]; then
|
|
736
|
+
echo "FORGE_GRAPH | HIGH | spec-bash | cannot resolve merge-base with origin/${PR_BASE} — spec bash not checked (fail closed)"
|
|
737
|
+
else
|
|
738
|
+
# shellcheck disable=SC2086 # SPEC_MD_FILES is a newline list of repo-relative paths without spaces
|
|
739
|
+
SPEC_BASH_REPORT=$(cd "{WORKTREE_PATH}" && bash "$SPEC_BASH_CHECKER" --base "$SPEC_BASH_BASE" --shellcheck $SPEC_MD_FILES 2>&1)
|
|
740
|
+
echo "$SPEC_BASH_REPORT"
|
|
741
|
+
echo "$SPEC_BASH_REPORT" | grep -E '^FAIL ' | while IFS= read -r hit; do
|
|
742
|
+
echo "FORGE_GRAPH | HIGH | spec-bash | ${hit#FAIL }"
|
|
743
|
+
done
|
|
744
|
+
fi
|
|
745
|
+
fi
|
|
746
|
+
```
|
|
747
|
+
|
|
748
|
+
Report each `FAIL` line as **HIGH**: the changed block does not parse, so any agent that runs it hits a syntax error. Fix the bash. If the block is an intentional fragment (for example an `if` opened in one block and closed in the next), put `<!-- allowlist:check-spec-bash -->` on the line before the fence. `WARN` (shellcheck) lines are advisory and go in the report without blocking.
|
|
749
|
+
|
|
721
750
|
### 2G.9: Danger-zone recurrence check (FORGE_GRAPH domain) <!-- Added: forge#1744 -->
|
|
722
751
|
|
|
723
752
|
**Triggered when**: `FORGE_GRAPH` in DOMAINS (changed files include `commands/*.md` specs or `scripts/*` files). Skips silently when the FORGE:CONTEXT comment on the issue contains no `### Danger-Zone Rule Cards` section (i.e., Phase C0.5 injected zero cards — nothing to cross-reference).
|
package/commands/review-pr.md
CHANGED
|
@@ -1692,25 +1692,51 @@ If still 0: review is clean — skip to Phase 7.
|
|
|
1692
1692
|
Filing a standalone `review-finding` issue for every LOW/POSSIBLE reviewer note makes the cascade amplify (each merged fix PR spawns its own P3 issues, which spawn more). Classify each deduped finding before 6C with the deterministic classifier — do NOT hand-classify when the script resolves:
|
|
1693
1693
|
|
|
1694
1694
|
- **ISSUE always**: `**Severity**: HIGH` or `CRITICAL`, or a missing/unparseable severity.
|
|
1695
|
-
- **Safety exemption is content-based**: a LOW or POSSIBLE finding is still filed when its file path / title / body matches the security/billing keyword set (`security|auth|authz|authn|billing|payment|stripe|charge|invoice|injection|xss|csrf|ssrf|idor|secret(s)|credential(s)|permission(s)|sql|token|password|redact`, whole words, `_`/`-`/`/` separate words — so `auth_service` matches but `author`/`tokenizer` do not), or when it came from a dedicated, signal-selected domain agent (Auth, Billing, Concurrency, Database). **Origin from the always-on General Security & Quality agent alone does NOT exempt a finding**: that agent runs on every PR, so origin-based exemption filed nearly every LOW note it raised (the 2026-10-08 cascade audit: 50 of 60 would-be notes in one batch were filed only for that reason).
|
|
1695
|
+
- **Safety exemption is content-based**: a LOW or POSSIBLE finding is still filed when its file path / title / body matches the security/billing keyword set (`security|auth|authz|authn|billing|payment|stripe|charge|invoice|injection|xss|csrf|ssrf|idor|secret(s)|credential(s)|permission(s)|sql|token|password|redact`, whole words, `_`/`-`/`/` separate words — so `auth_service` matches but `author`/`tokenizer` do not), or when it came from a dedicated, signal-selected domain agent (Auth, Billing, Concurrency, Database) AND is MEDIUM+ or CONFIRMED — a LOW/POSSIBLE finding stays a NOTE whichever reviewer raised it. **Origin from the always-on General Security & Quality agent alone does NOT exempt a finding**: that agent runs on every PR, so origin-based exemption filed nearly every LOW note it raised (the 2026-10-08 cascade audit: 50 of 60 would-be notes in one batch were filed only for that reason).
|
|
1696
1696
|
- **Review-finding lineage** (the PR's linked issue `MERGE_ISSUE` carries the `review-finding` label, any priority — this PR is itself a fix for a finding): only MEDIUM findings that are CONFIRMED, or LIKELY and safety-exempt, become issues; LOW and POSSIBLE are always NOTEs. A fix for a finding must not mint a new generation of findings.
|
|
1697
1697
|
- **Otherwise**: `**Severity**: LOW`, or `**Confidence**: POSSIBLE` below HIGH, is a NOTE unless safety-exempt; everything else (MEDIUM CONFIRMED/LIKELY) is an ISSUE.
|
|
1698
1698
|
|
|
1699
1699
|
```bash
|
|
1700
|
-
# Resolve the classifier: running plugin
|
|
1700
|
+
# Resolve the classifier: running plugin, FORGE_ROOT, FORGEDOCK_HOME, newest pinned forgedock plugin cache
|
|
1701
|
+
# under the active config dir (CLAUDE_CONFIG_DIR) then ~/.claude, then the repo's own scripts/ (ForgeDock itself).
|
|
1702
|
+
# The plugin-root placeholder is not always substituted in forked review runs, so the cache scan is what makes
|
|
1703
|
+
# consumer repos resolve the script instead of falling back to classifier=manual.
|
|
1701
1704
|
CLASSIFY_SCRIPT=""
|
|
1702
|
-
|
|
1703
|
-
|
|
1705
|
+
_cands="$(printf '%s\n' '${CLAUDE_PLUGIN_ROOT}' "${FORGE_ROOT:-}" "${FORGEDOCK_HOME:-}")"
|
|
1706
|
+
for _cfg in "${CLAUDE_CONFIG_DIR:-}" "$HOME/.claude"; do
|
|
1707
|
+
[ -n "$_cfg" ] || continue
|
|
1708
|
+
_cands="$_cands
|
|
1709
|
+
$(find -L "$_cfg/plugins/cache/forgedock/forgedock" -mindepth 1 -maxdepth 1 -type d 2>/dev/null | awk -F/ '$NF ~ /^[0-9]+\.[0-9]+\.[0-9]+$/{split($NF,a,".");printf "%d %d %d %s\n",a[1],a[2],a[3],$0}' | sort -k1,1nr -k2,2nr -k3,3nr | cut -d' ' -f4- || true)"
|
|
1704
1710
|
done
|
|
1711
|
+
_cands="$_cands
|
|
1712
|
+
$PWD"
|
|
1713
|
+
while IFS= read -r _c; do
|
|
1714
|
+
case "$_c" in /*) [ -z "$CLASSIFY_SCRIPT" ] && [ -f "$_c/scripts/classify-finding.sh" ] && CLASSIFY_SCRIPT="$_c/scripts/classify-finding.sh" ;; esac
|
|
1715
|
+
done <<< "$_cands"
|
|
1705
1716
|
FINDING_LINEAGE="none"
|
|
1706
1717
|
if [ -n "${MERGE_ISSUE:-}" ] && gh issue view "$MERGE_ISSUE" -R {GH_REPO} --json labels --jq '.labels[].name' 2>/dev/null | grep -qx 'review-finding'; then
|
|
1707
1718
|
FINDING_LINEAGE="review-finding"
|
|
1708
1719
|
fi
|
|
1709
|
-
#
|
|
1720
|
+
# In-PR fix gate (§6B.6): only for an auto-merge review with a linked issue, and only for the first
|
|
1721
|
+
# fix round on this PR. Any trusted FORGE:INPR_FIX* marker already on the PR means the round is used.
|
|
1722
|
+
INPR_DIFF_FILE=""
|
|
1723
|
+
if [ "${AUTO_MERGE:-false}" = "true" ] && [ -n "${MERGE_ISSUE:-}" ]; then
|
|
1724
|
+
INPR_MARKERS=$(gh api --paginate "repos/{GH_REPO}/issues/{PR_NUMBER}/comments" 2>/dev/null \
|
|
1725
|
+
| jq -s '[.[][] | select(.author_association == "OWNER" or .author_association == "MEMBER" or .author_association == "COLLABORATOR") | select(.body | test("^<!-- FORGE:INPR_FIX"))] | length' 2>/dev/null || echo "")
|
|
1726
|
+
if [ "$INPR_MARKERS" = "0" ]; then
|
|
1727
|
+
mkdir -p "${FORGE_SCRATCHPAD:-$PWD/.forge-scratch}"
|
|
1728
|
+
INPR_DIFF_FILE=$(mktemp "${FORGE_SCRATCHPAD:-$PWD/.forge-scratch}/{PR_NUMBER}_inpr-diff.XXXXXX")
|
|
1729
|
+
gh pr diff {PR_NUMBER} -R {GH_REPO} --name-only > "$INPR_DIFF_FILE" 2>/dev/null || { rm -f "$INPR_DIFF_FILE"; INPR_DIFF_FILE=""; }
|
|
1730
|
+
fi
|
|
1731
|
+
# An unreadable marker list or diff leaves INPR_DIFF_FILE empty: findings are filed as before (never gated blind).
|
|
1732
|
+
fi
|
|
1733
|
+
# Per deduped finding (FINDING_TEXT_FILE = mktemp file holding title + body + affected paths;
|
|
1734
|
+
# FINDING_FILE = the finding's file path, file:line accepted):
|
|
1710
1735
|
# DISPOSITION=$(bash "$CLASSIFY_SCRIPT" --severity "$FINDING_SEVERITY" --confidence "$FINDING_CONFIDENCE" \
|
|
1711
|
-
# --agent "$FINDING_AGENT" --lineage "$FINDING_LINEAGE" --text-file "$FINDING_TEXT_FILE"
|
|
1712
|
-
#
|
|
1713
|
-
#
|
|
1736
|
+
# --agent "$FINDING_AGENT" --lineage "$FINDING_LINEAGE" --text-file "$FINDING_TEXT_FILE" \
|
|
1737
|
+
# ${INPR_DIFF_FILE:+--inpr-diff "$INPR_DIFF_FILE" --file "$FINDING_FILE"})
|
|
1738
|
+
# case "$DISPOSITION" in ISSUE*) ;; NOTE*) ;; INPR_FIX*) ;; esac # first word is the class, the rest is the reason
|
|
1739
|
+
# If CLASSIFY_SCRIPT is empty, apply the four rules above (and §6B.6) by hand and record classifier=manual below.
|
|
1714
1740
|
```
|
|
1715
1741
|
|
|
1716
1742
|
Each NOTE gets exactly one disposition, in this preference order:
|
|
@@ -1720,17 +1746,38 @@ Each NOTE gets exactly one disposition, in this preference order:
|
|
|
1720
1746
|
|
|
1721
1747
|
**Metrics note (forge#3106)**: because NOTEs are not filed as issues, `review-finding` issue volume and any metric derived from it (findings per PR, `/pipeline-health` finding rates, amplification ratio) drop relative to pre-#3060 history. Compare against the `notes_*` counts in the review summary, not issue counts alone, when judging review depth across the change.
|
|
1722
1748
|
|
|
1723
|
-
Every NOTE disposition must be recorded (never silently dropped): list each NOTE (id, file:line, one-line reason) with its disposition in the review summary comment, including dropped ones, so a reviewer can audit the decision. NOTES are never passed to `Skill(issue)`. Record counts in the review summary (`notes_fixed`, `notes_listed`, `notes_dropped`, `findings_filed`). If no finding is an ISSUE after this step, skip 6C and continue to Phase 7.
|
|
1749
|
+
Every NOTE disposition must be recorded (never silently dropped): list each NOTE (id, file:line, one-line reason) with its disposition in the review summary comment, including dropped ones, so a reviewer can audit the decision. NOTES are never passed to `Skill(issue)`. Record counts in the review summary (`notes_fixed`, `notes_listed`, `notes_dropped`, `findings_filed`, `inpr_fix`). If no finding is an ISSUE after this step, skip 6C and continue to Phase 7. Findings classified `INPR_FIX` (§6B.6) are neither notes nor filed this round.
|
|
1724
1750
|
|
|
1725
1751
|
**Disposition marker (MANDATORY whenever Phase 6A extracted at least one finding):** post one PR comment whose first line is the machine-readable record below, followed by the NOTE list. Phase 8 refuses to auto-merge a PR that has `FINDING:` markers but no `FORGE:NOTE_DISPOSITION` comment — the disposition step is enforced, not advisory.
|
|
1726
1752
|
|
|
1727
1753
|
```bash
|
|
1728
1754
|
CLASSIFIER_MODE=$([ -n "$CLASSIFY_SCRIPT" ] && echo script || echo manual)
|
|
1729
1755
|
# NOTE_LIST_FILE: mktemp file named for the PR number, holding one line per NOTE (id, file:line, disposition, reason).
|
|
1730
|
-
gh pr comment {PR_NUMBER} -R {GH_REPO} --body "<!-- FORGE:NOTE_DISPOSITION: notes_fixed=${NOTES_FIXED:-0} notes_listed=${NOTES_LISTED:-0} notes_dropped=${NOTES_DROPPED:-0} findings_filed=${FINDINGS_FILED:-0} lineage=${FINDING_LINEAGE} classifier=${CLASSIFIER_MODE} -->
|
|
1756
|
+
gh pr comment {PR_NUMBER} -R {GH_REPO} --body "<!-- FORGE:NOTE_DISPOSITION: notes_fixed=${NOTES_FIXED:-0} notes_listed=${NOTES_LISTED:-0} notes_dropped=${NOTES_DROPPED:-0} findings_filed=${FINDINGS_FILED:-0} inpr_fix=${INPR_FIX_COUNT:-0} lineage=${FINDING_LINEAGE} classifier=${CLASSIFIER_MODE} -->
|
|
1731
1757
|
$(cat "$NOTE_LIST_FILE" 2>/dev/null)" # allowlist:check-command-side-effects
|
|
1732
1758
|
```
|
|
1733
1759
|
|
|
1760
|
+
### 6B.6: In-PR fix gate for CONFIRMED MEDIUM findings in the PR's own files (#3387, narrowed)
|
|
1761
|
+
|
|
1762
|
+
A CONFIRMED MEDIUM defect in a file this PR changed is cheaper to fix before merge than through a separate `/work-on` cycle, and merging it knowingly ships the defect. A finding classified `INPR_FIX` is therefore fixed on the PR instead of filed. The gate is deliberately narrow:
|
|
1763
|
+
|
|
1764
|
+
- **Only `MEDIUM` + `CONFIRMED`.** LIKELY and POSSIBLE are not gated. HIGH/CRITICAL already withhold APPROVED through §7B.
|
|
1765
|
+
- **Only files in the PR diff** (`gh pr diff --name-only`). Pre-existing defects elsewhere are filed as before.
|
|
1766
|
+
- **One fix round per PR.** Once any `FORGE:INPR_FIX*` marker is on the PR, later reviews classify normally: anything still present is filed as an issue and the PR merges.
|
|
1767
|
+
- **Never stops at `needs-human` because of this gate.** If the fix round fails, `/work-on` waives the gate for the current head and the findings are filed (work-on/review.md R4).
|
|
1768
|
+
- **Auto-merge reviews with `--issue` only.** A standalone review without `--auto-merge` files findings as before.
|
|
1769
|
+
|
|
1770
|
+
When at least one finding is `INPR_FIX`, do NOT file those findings in 6C. Post the request on the PR as one comment, after the §6B.5 disposition marker. Phase 8 then refuses to merge this head and returns `REVIEW_RESULT: status: BLOCKED`, blocker `in-pr fix required`:
|
|
1771
|
+
|
|
1772
|
+
```bash
|
|
1773
|
+
INPR_HEAD=$(gh pr view {PR_NUMBER} -R {GH_REPO} --json headRefOid --jq .headRefOid)
|
|
1774
|
+
# INPR_FIX_IDS: comma-separated finding ids classified INPR_FIX. INPR_FIX_LIST_FILE: mktemp file, one line per finding:
|
|
1775
|
+
# id, file:line, one-sentence problem, suggested fix — this is the remediation work order.
|
|
1776
|
+
gh pr comment {PR_NUMBER} -R {GH_REPO} --body "<!-- FORGE:INPR_FIX: round=1 head=${INPR_HEAD} findings=${INPR_FIX_IDS} -->
|
|
1777
|
+
In-PR fix requested before merge (CONFIRMED MEDIUM findings in files this PR changed). One fix round; anything still present on the next review is filed as an issue.
|
|
1778
|
+
$(cat "$INPR_FIX_LIST_FILE" 2>/dev/null)" # allowlist:check-command-side-effects
|
|
1779
|
+
```
|
|
1780
|
+
|
|
1734
1781
|
### 6C: Create Issues
|
|
1735
1782
|
|
|
1736
1783
|
```bash
|
|
@@ -2248,6 +2295,16 @@ if [ "${FINDING_COUNT:-0}" -gt 0 ] && [ "${DISPOSITION_COUNT:-0}" -eq 0 ]; then
|
|
|
2248
2295
|
# STOP — return REVIEW_RESULT: status: BLOCKED, blocker: "note disposition missing".
|
|
2249
2296
|
exit 1
|
|
2250
2297
|
fi
|
|
2298
|
+
# In-PR fix gate (§6B.6): an unanswered INPR_FIX request at the current head blocks the merge until the fix
|
|
2299
|
+
# round runs (a new head) or /work-on waives it for this head. Only trusted authors' markers count.
|
|
2300
|
+
PR_HEAD_NOW=$(gh pr view {PR_NUMBER} {MERGE_GH_FLAG} --json headRefOid --jq .headRefOid 2>/dev/null || echo "")
|
|
2301
|
+
INPR_REQ=$(printf '%s' "$DISPO_JSON" | jq -s -r '[.[][] | select(.author_association == "OWNER" or .author_association == "MEMBER" or .author_association == "COLLABORATOR") | select(.body | test("^<!-- FORGE:INPR_FIX: ")) | .body | capture("head=(?<h>[0-9a-f]{7,40})").h] | last // ""')
|
|
2302
|
+
INPR_WAIVED=$(printf '%s' "$DISPO_JSON" | jq -s -r --arg h "$PR_HEAD_NOW" '[.[][] | select(.author_association == "OWNER" or .author_association == "MEMBER" or .author_association == "COLLABORATOR") | select(.body | test("^<!-- FORGE:INPR_FIX_WAIVED: head=" + $h))] | length')
|
|
2303
|
+
if [ -n "$INPR_REQ" ] && [ "$INPR_REQ" = "$PR_HEAD_NOW" ] && [ "${INPR_WAIVED:-0}" -eq 0 ]; then
|
|
2304
|
+
echo "IN-PR FIX: CONFIRMED MEDIUM findings in this PR's files were requested at head ${PR_HEAD_NOW} — fix them before merging (§6B.6)" >&2
|
|
2305
|
+
# STOP — return REVIEW_RESULT: status: BLOCKED, blocker: "in-pr fix required". Not needs-human: /work-on R4 runs one fix round.
|
|
2306
|
+
exit 1
|
|
2307
|
+
fi
|
|
2251
2308
|
```
|
|
2252
2309
|
|
|
2253
2310
|
**Phase-trail preflight (MANDATORY before any merge attempt)** <!-- Added: forge#3061 -->: a reviewer verdict alone must not merge work whose earlier pipeline phases were skipped. Run the deterministic verifier against the linked issue; do NOT run the merge block below unless it exits 0.
|
|
@@ -150,7 +150,7 @@ gh issue edit {NUMBER} {GH_FLAG} --add-label "needs-human" # allowlist:check-com
|
|
|
150
150
|
|
|
151
151
|
## Phase V2: Format and Verify
|
|
152
152
|
|
|
153
|
-
Run after quality gate passes. All tool commands are read from `forge.yaml → verification.commands
|
|
153
|
+
Run after quality gate passes. All tool commands are read from `forge.yaml → verification.commands`. `forge.yaml` is usually gitignored, so it is absent from the worktree: every block below resolves `FORGE_CFG` to the worktree copy if present, else the main checkout's `forge.yaml` (via `git rev-parse --git-common-dir`). Before this, every verification command, learned test command and the SOPS chain check were silently skipped in worktree builds; each step logs `SKIPPED — not configured` when the corresponding key is absent rather than silently passing.
|
|
154
154
|
|
|
155
155
|
**Track skipped checks** — initialize before any check runs:
|
|
156
156
|
```bash
|
|
@@ -178,8 +178,9 @@ run_verification_command() {
|
|
|
178
178
|
**Python**:
|
|
179
179
|
```bash
|
|
180
180
|
cd {WORKTREE_PATH}
|
|
181
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
181
182
|
|
|
182
|
-
PYTHON_FORMAT=$(yq '.verification.commands.python.format // ""'
|
|
183
|
+
PYTHON_FORMAT=$(yq '.verification.commands.python.format // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
183
184
|
if [ -n "$PYTHON_FORMAT" ]; then
|
|
184
185
|
run_verification_command "python.format" "$PYTHON_FORMAT"
|
|
185
186
|
else
|
|
@@ -195,10 +196,11 @@ Failures in `py_compile` are BLOCKING — fix before continuing.
|
|
|
195
196
|
**TypeScript**:
|
|
196
197
|
```bash
|
|
197
198
|
cd {WORKTREE_PATH}
|
|
199
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
198
200
|
|
|
199
|
-
TS_FORMAT=$(yq '.verification.commands.typescript.format // ""'
|
|
200
|
-
TS_TYPECHECK=$(yq '.verification.commands.typescript.typecheck // ""'
|
|
201
|
-
TS_BUILD=$(yq '.verification.commands.typescript.build // ""'
|
|
201
|
+
TS_FORMAT=$(yq '.verification.commands.typescript.format // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
202
|
+
TS_TYPECHECK=$(yq '.verification.commands.typescript.typecheck // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
203
|
+
TS_BUILD=$(yq '.verification.commands.typescript.build // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
202
204
|
|
|
203
205
|
if [ -n "$TS_FORMAT" ]; then
|
|
204
206
|
run_verification_command "typescript.format" "$TS_FORMAT"
|
|
@@ -234,8 +236,9 @@ Before running any test command below, check it against `verification.known_slow
|
|
|
234
236
|
|
|
235
237
|
```bash
|
|
236
238
|
cd {WORKTREE_PATH}
|
|
239
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
237
240
|
# Read directly from forge.yaml (static, operator-declared config).
|
|
238
|
-
KNOWN_SLOW_TESTS=$(yq -o=json -I=0 '.verification.known_slow_tests // []'
|
|
241
|
+
KNOWN_SLOW_TESTS=$(yq -o=json -I=0 '.verification.known_slow_tests // []' "$FORGE_CFG" 2>/dev/null || echo '[]')
|
|
239
242
|
|
|
240
243
|
# apply_known_slow_filter <cmd> — echoes the command to actually run, or "" to
|
|
241
244
|
# skip it entirely. Matching is substring match of `pattern` against the full
|
|
@@ -274,7 +277,8 @@ After all `verification.commands` steps complete, run any commands from `forge.y
|
|
|
274
277
|
|
|
275
278
|
```bash
|
|
276
279
|
cd {WORKTREE_PATH}
|
|
277
|
-
|
|
280
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
281
|
+
LEARNED_TEST_COMMANDS=$(yq -o=json -I=0 '.learned.test_commands // []' "$FORGE_CFG" 2>/dev/null || echo '[]')
|
|
278
282
|
LEARNED_FAILED=0
|
|
279
283
|
if [ -n "$LEARNED_TEST_COMMANDS" ] && [ "$LEARNED_TEST_COMMANDS" != "[]" ] && [ "$LEARNED_TEST_COMMANDS" != "null" ]; then
|
|
280
284
|
echo "Running learned test commands..."
|
|
@@ -368,7 +372,8 @@ After static proxy checks, run a lightweight live browser check using Playwright
|
|
|
368
372
|
|
|
369
373
|
```bash
|
|
370
374
|
cd {WORKTREE_PATH}
|
|
371
|
-
|
|
375
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
376
|
+
APP_URL=$(yq '.services.app_url // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
372
377
|
if [ -z "$APP_URL" ]; then
|
|
373
378
|
echo "SKIPPED — services.app_url not configured in forge.yaml (browser signal check requires a running app URL)"
|
|
374
379
|
else
|
|
@@ -442,7 +447,8 @@ ADDED_LINES=$(git diff HEAD -- {CHANGED_FILES} | grep -E '^\+' | grep -v '^+++')
|
|
|
442
447
|
|
|
443
448
|
```bash
|
|
444
449
|
cd {WORKTREE_PATH}
|
|
445
|
-
|
|
450
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
451
|
+
SECRETS_BACKEND=$(yq '.deploy.secrets_backend // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
446
452
|
if [ "$SECRETS_BACKEND" != "sops" ]; then
|
|
447
453
|
echo 'SKIP: SOPS chain check — deploy.secrets_backend is not "sops". Configure deploy.secrets_backend in forge.yaml to enable.'
|
|
448
454
|
fi
|
|
@@ -112,7 +112,7 @@ BLOCK_COMMENTS=$(gh api repos/{GH_REPO}/issues/{ISSUE_NUMBER}/comments \
|
|
|
112
112
|
```
|
|
113
113
|
|
|
114
114
|
**Classify into FIXABLE vs. UNFIXABLE**:
|
|
115
|
-
- **FIXABLE** — open `review-finding` issues (CONFIRMED/LIKELY code defects), a `VERDICT=CHANGES REQUESTED` block with concrete findings attached, a mergeability guard failure (`CONFLICTING`/`DIRTY`/`BLOCKED` — resolvable by rebasing onto `{PR_BASE}`), a quality-gate/build failure, or a CI-gate refusal (`ci gate not green` — the failing/cancelled/timed-out checks are listed on the linked issue; read each failing job's log with `gh run view --log-failed`, fix the cause on the PR branch, or re-run a check that failed for an infrastructure reason).
|
|
115
|
+
- **FIXABLE** — open `review-finding` issues (CONFIRMED/LIKELY code defects), a `VERDICT=CHANGES REQUESTED` block with concrete findings attached, a mergeability guard failure (`CONFLICTING`/`DIRTY`/`BLOCKED` — resolvable by rebasing onto `{PR_BASE}`), a quality-gate/build failure, or a CI-gate refusal (`ci gate not green` — the failing/cancelled/timed-out checks are listed on the linked issue; read each failing job's log with `gh run view --log-failed`, fix the cause on the PR branch, or re-run a check that failed for an infrastructure reason), or an in-PR fix request (`in-pr fix required` — review-pr §6B.6 posted a `<!-- FORGE:INPR_FIX: round=1 head=<sha> findings=<ids> -->` comment on the PR listing CONFIRMED MEDIUM findings in files this PR changed. That comment is the work order: fix exactly the listed findings, at the listed `file:line`, and nothing else. Do not file them as issues; the re-review in Phase M6 files any that remain).
|
|
116
116
|
- **UNFIXABLE (policy escalation)** — `HAS_PURPOSE_REGRESSION=true` (the PR's behavior diverges from the issue's intent — a judgment call, not a code defect), `CALIBRATION_NEEDS_HUMAN=true` (statistical trust threshold), or `TRUST_NEEDS_HUMAN=true` (provenance `NOVEL_NEEDS_HUMAN` tier, insufficient prior data — a policy gate, not a bug). None of these are mechanically "fixable" by re-editing code.
|
|
117
117
|
|
|
118
118
|
**If the block reason classifies as UNFIXABLE** (and no FIXABLE item accompanies it): do NOT attempt any fix. Skip directly to Phase M8 with verdict `UNFIXABLE`, re-affirm `needs-human` (it should already be present), and return `REMEDIATE_RESULT: status: UNFIXABLE`. This satisfies AC5 — "genuinely-blocked PRs still terminate at `needs-human`."
|
|
@@ -534,12 +534,26 @@ gh issue view {NUMBER} {GH_FLAG} --json state --jq '.state'
|
|
|
534
534
|
|
|
535
535
|
- `REVIEW_RESULT: status: BLOCKED` from /review-pr whose blocker contains "ci gate" (Phase 8 refused to merge because checks failed, were cancelled, or stayed pending past the gate retries): do NOT attempt the manual merge below — that would bypass the CI gate. Fixing red CI is pipeline work, not a human decision: invoke remediation **once** — `Skill(skill="{FORGE_SKILL_PREFIX}work-on:remediate", args="{PR_NUMBER} --issue {NUMBER} --base {PR_BASE}")` (forked; it classifies a CI-gate refusal as FIXABLE, clears `needs-human`, reads the failing job logs, fixes them on the PR branch, re-runs the quality gate and a full review, and auto-lands through the same CI gate). Bound: count `<!-- FORGE:CI_REMEDIATION: pr={PR_NUMBER} -->` comments on the issue first; if ≥ 1, do not remediate again. Post that marker before invoking. `REMEDIATE_RESULT` re-gate outcome `AUTO-LANDED` → treat as merged and return `REVIEW_RESULT: status: COMPLETE`; `REMEDIATE_RESULT: status: REREVIEW_REQUIRED` (forge#3240: the fix is pushed but remediation had no sub-agent dispatch tool, so no review ran) → do NOT add `needs-human` and do not re-invoke remediation; return `REVIEW_RESULT: status: BLOCKED` with blocker "re-review required: no dispatch tool" so the caller re-dispatches review from a session that has dispatch; any other outcome → leave `needs-human` (remediation sets it) and return `REVIEW_RESULT: status: BLOCKED` with blocker "ci gate not green after remediation". <!-- forge#3191 -->
|
|
536
536
|
|
|
537
|
-
|
|
537
|
+
- `REVIEW_RESULT: status: BLOCKED` from /review-pr whose blocker contains "in-pr fix required" (review-pr §6B.6: CONFIRMED MEDIUM findings in files this PR changed, requested on the PR in a `FORGE:INPR_FIX` comment at the current head): do NOT merge and do NOT leave the issue at `needs-human` because of this gate. Run **one** fix round, bounded by `<!-- FORGE:INPR_REMEDIATION: pr={PR_NUMBER} -->` on the issue (`{BOUND}` = `INPR_REMEDIATION` below).
|
|
538
|
+
- **Bound unused**: post the marker. Add `needs-human` (remediation only targets `needs-human`-gated PRs, and its Phase M1 clears it as FIXABLE). Then invoke `Skill(skill="{FORGE_SKILL_PREFIX}work-on:remediate", args="{PR_NUMBER} --issue {NUMBER} --base {PR_BASE}")`. Remediation reads the `FORGE:INPR_FIX` work order, fixes exactly those findings, re-reviews the new head and auto-lands. On the re-review, any finding still present is filed as an issue (the round is used).
|
|
539
|
+
- `REMEDIATE_RESULT` re-gate outcome `AUTO-LANDED` → return `REVIEW_RESULT: status: COMPLETE`. `REREVIEW_REQUIRED` → same handling as the ci-gate case above.
|
|
540
|
+
- **Any other outcome, or bound already used**: waive the gate for the current head. Remove `needs-human`, post `<!-- FORGE:INPR_FIX_WAIVED: head=<current head SHA> -->` on the **PR**, then re-invoke Phase R3 once under the `STALE_REREVIEW` bound. That review files the remaining findings as issues and merges as it would have before this gate. Only if `STALE_REREVIEW` is also exhausted, add `needs-human`.
|
|
541
|
+
```bash
|
|
542
|
+
INPR_WAIVE_HEAD=$(gh pr view {PR_NUMBER} {GH_FLAG} --json headRefOid --jq .headRefOid)
|
|
543
|
+
gh issue edit {NUMBER} {GH_FLAG} --remove-label "needs-human" 2>/dev/null || true # allowlist:check-command-side-effects
|
|
544
|
+
gh pr comment {PR_NUMBER} {GH_FLAG} --body "<!-- FORGE:INPR_FIX_WAIVED: head=${INPR_WAIVE_HEAD} -->
|
|
545
|
+
In-PR fix round did not land; the remaining CONFIRMED MEDIUM findings are filed as issues on the next review instead of blocking the merge." # allowlist:check-command-side-effects
|
|
546
|
+
```
|
|
547
|
+
|
|
548
|
+
Persisted loop bounds for the cases above — count first, post the marker, then act (`{BOUND}` is `STALE_REREVIEW`, `CI_REMEDIATION` or `INPR_REMEDIATION`):
|
|
538
549
|
```bash
|
|
539
550
|
[ "${DRY_RUN:-false}" = "true" ] && { echo "[DRY_RUN] bound check only"; }
|
|
540
551
|
if [ "{BOUND}" = "STALE_REREVIEW" ]; then
|
|
541
552
|
BOUND_COUNT=$(gh api "repos/{GH_REPO}/issues/{NUMBER}/comments" --paginate \
|
|
542
553
|
--jq '[.[] | select((.body | contains("FORGE:STALE_REREVIEW:")) and (.body | contains("pr={PR_NUMBER}")))] | length' 2>/dev/null | awk '{s+=$1} END {print s+0}')
|
|
554
|
+
elif [ "{BOUND}" = "INPR_REMEDIATION" ]; then
|
|
555
|
+
BOUND_COUNT=$(gh api "repos/{GH_REPO}/issues/{NUMBER}/comments" --paginate \
|
|
556
|
+
--jq '[.[] | select((.body | contains("FORGE:INPR_REMEDIATION:")) and (.body | contains("pr={PR_NUMBER}")))] | length' 2>/dev/null | awk '{s+=$1} END {print s+0}')
|
|
543
557
|
else
|
|
544
558
|
BOUND_COUNT=$(gh api "repos/{GH_REPO}/issues/{NUMBER}/comments" --paginate \
|
|
545
559
|
--jq '[.[] | select((.body | contains("FORGE:CI_REMEDIATION:")) and (.body | contains("pr={PR_NUMBER}")))] | length' 2>/dev/null | awk '{s+=$1} END {print s+0}')
|
|
@@ -551,11 +565,13 @@ gh issue view {NUMBER} {GH_FLAG} --json state --jq '.state'
|
|
|
551
565
|
Stale review on PR #{PR_NUMBER}: quality-gating and re-reviewing the new head once." 2>/dev/null || true # when {BOUND}=STALE_REREVIEW
|
|
552
566
|
gh issue comment {NUMBER} {GH_FLAG} --body "<!-- FORGE:CI_REMEDIATION: pr={PR_NUMBER} -->
|
|
553
567
|
CI gate refused PR #{PR_NUMBER}: dispatching remediation once to fix the failing checks." 2>/dev/null || true # when {BOUND}=CI_REMEDIATION
|
|
568
|
+
gh issue comment {NUMBER} {GH_FLAG} --body "<!-- FORGE:INPR_REMEDIATION: pr={PR_NUMBER} -->
|
|
569
|
+
In-PR fix requested on PR #{PR_NUMBER}: dispatching remediation once to fix the CONFIRMED MEDIUM findings before merge." 2>/dev/null || true # when {BOUND}=INPR_REMEDIATION
|
|
554
570
|
fi
|
|
555
571
|
```
|
|
556
572
|
Post only the comment matching `{BOUND}`. `BOUND_EXHAUSTED` → take the "already ≥ 1" branch of that case.
|
|
557
573
|
|
|
558
|
-
- PR NOT MERGED (and not a phase-trail, auto-merge-gate, stale-review
|
|
574
|
+
- PR NOT MERGED (and not a phase-trail, auto-merge-gate, stale-review, ci-gate or in-pr-fix BLOCKED above) → attempt manual merge, **only if the PR head is still the commit named in the latest `<!-- FORGE:REVIEW -->` APPROVED verdict** (otherwise re-invoke Phase R3 instead, as for "stale review") **and only after the same CI gate**:
|
|
559
575
|
```bash
|
|
560
576
|
# CI gate (MANDATORY before any autonomous merge): merge only when every check on the PR is
|
|
561
577
|
# green. Field test: PRs merged to staging with checks pending or red (#3165), because branch
|
package/docs/RELEASE-NOTES.md
CHANGED
|
@@ -2,6 +2,26 @@
|
|
|
2
2
|
|
|
3
3
|
Operator-facing notes for behavior changes that need action or awareness. Newest first.
|
|
4
4
|
|
|
5
|
+
## In-PR fix round for CONFIRMED MEDIUM findings (#3387, narrowed)
|
|
6
|
+
|
|
7
|
+
Under `--auto-merge` with `--issue` (the `/work-on` pipeline), a review finding that is **MEDIUM + CONFIRMED** in a file the PR itself changed is no longer merged and filed as a follow-up issue. `/review-pr` §6B.6 posts a `FORGE:INPR_FIX` work order on the PR, and Phase 8 holds the merge for that head (`blocker: in-pr fix required`). `/work-on` review then runs **one** remediation round (bound: `FORGE:INPR_REMEDIATION` on the issue) that fixes exactly those findings and re-reviews.
|
|
8
|
+
|
|
9
|
+
- **Narrow by design.** LIKELY/POSSIBLE, findings outside the PR's changed files, and standalone reviews are unchanged. HIGH/CRITICAL already block through §7B.
|
|
10
|
+
- **One round only.** Anything still present on the re-review is filed as an issue, as before.
|
|
11
|
+
- **Never a `needs-human` stop because of this gate.** If the round does not land, `/work-on` posts `FORGE:INPR_FIX_WAIVED` for the current head and re-reviews once, so the findings are filed and the PR merges as it did before.
|
|
12
|
+
- **A missing finding path never gates.** If a finding has no file path, or the PR diff cannot be read, it is filed as before.
|
|
13
|
+
|
|
14
|
+
Action: none. Expect a few PRs to take one extra fix round before merging, and fewer MEDIUM `review-finding` issues.
|
|
15
|
+
|
|
16
|
+
## Cascade follow-ups: script resolution, domain-agent notes, verification in worktrees, spec bash check
|
|
17
|
+
|
|
18
|
+
- **Script resolution works in consumer repos.** `/review-pr` §6B.5 and the orchestrator breaker now find `classify-finding.sh` and `amplification-breaker.sh` through the newest pinned ForgeDock plugin cache under `CLAUDE_CONFIG_DIR`, then `~/.claude`. They also check `FORGEDOCK_HOME` and the repo's own `scripts/`, as before. Previously the plugin-root placeholder was often left unsubstituted in forked review runs, so 4 of 9 AlterLab reviews fell back to `classifier=manual`.
|
|
19
|
+
- **Domain reviewers no longer file speculative LOW findings.** A finding from the Auth, Billing, Concurrency or Database reviewer is still filed as an issue only when it is MEDIUM+ or CONFIRMED. A LOW/POSSIBLE finding becomes a note whichever reviewer raised it. Security/billing keyword content still files as before.
|
|
20
|
+
- **Validate reads your real `forge.yaml` in worktrees.** `forge.yaml` is usually gitignored, so it never existed in the per-issue worktree, and every `verification.commands` entry, `learned.test_commands` entry and the SOPS chain check were silently skipped. Validate now falls back to the main checkout's `forge.yaml`.
|
|
21
|
+
- **The quality gate parses spec bash.** New step 2G.10 runs `scripts/check-spec-bash.sh`, which applies `bash -n` (plus advisory `shellcheck -S error`) to every changed fenced bash block in `commands/**/*.md`. Pre-existing blocks are never checked. Mark an intentional fragment with `<!-- allowlist:check-spec-bash -->` on the line before its fence.
|
|
22
|
+
|
|
23
|
+
Action: expect verification commands that you configured in `forge.yaml` to start running in `/work-on` builds. If a configured command (for example a SOPS chain with `deploy.secrets_backend: sops`) was failing unnoticed, builds will now report it.
|
|
24
|
+
|
|
5
25
|
## Review-finding cascade is now bounded by code, not prose
|
|
6
26
|
|
|
7
27
|
An audit of one `/orchestrate` batch found 93 `review-finding` issues filed for 46 merges, a ratio of 2.02. The §6B.5 note-disposition step was supposed to stop LOW/POSSIBLE findings from becoming issues. It was bypassed in two ways: every finding from the always-on General Security & Quality reviewer was exempt by origin, and reviewers often skipped the step entirely.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "forgedock",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.10.0",
|
|
4
4
|
"description": "Autonomous development pipeline for coding agents. GitHub as institutional memory, with native Claude Code and OpenCode entrypoints.",
|
|
5
5
|
"bin": {
|
|
6
6
|
"forgedock": "bin/forgedock.mjs"
|
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios
|
|
3
|
+
# SPDX-License-Identifier: AGPL-3.0-or-later
|
|
4
|
+
#
|
|
5
|
+
# check-spec-bash.sh — Syntax-check the ```bash fenced blocks inside Markdown spec files.
|
|
6
|
+
#
|
|
7
|
+
# ForgeDock's pipeline is bash embedded in Markdown, and the quality gate's executable
|
|
8
|
+
# checks skip .md files. Spec bash therefore merged without ever being parsed, and
|
|
9
|
+
# reviewers became the first thing to run it (2026-10-08 audit). This script runs
|
|
10
|
+
# `bash -n` on each fenced block after neutralising spec notation, and optionally
|
|
11
|
+
# `shellcheck -S error`.
|
|
12
|
+
#
|
|
13
|
+
# Usage:
|
|
14
|
+
# check-spec-bash.sh [--base <git-ref>] [--shellcheck] <file.md>...
|
|
15
|
+
#
|
|
16
|
+
# --base <ref> Only check blocks that contain a line changed relative to <ref>
|
|
17
|
+
# (`git diff -U0 <ref> -- <file>`), so pre-existing blocks never fail
|
|
18
|
+
# a new change. Without --base every block is checked.
|
|
19
|
+
# --shellcheck Also run `shellcheck -S error -s bash` when installed. Advisory: its hits are
|
|
20
|
+
# reported as WARN and never fail the run (spec blocks are often fragments).
|
|
21
|
+
#
|
|
22
|
+
# Spec notation that is not bash is neutralised before parsing:
|
|
23
|
+
# {PLACEHOLDER} / {FORGE_SKILL_PREFIX}x → a plain word
|
|
24
|
+
# blocks containing pseudo tool calls (Skill( / Agent( / Task( / Read( ) → SKIPPED
|
|
25
|
+
# a block whose preceding non-blank line is exactly `<!-- allowlist:check-spec-bash -->` → SKIPPED
|
|
26
|
+
# (for intentional fragments, e.g. an `if` opened in one block and closed in the next)
|
|
27
|
+
#
|
|
28
|
+
# Output: one line per checked block, `OK|FAIL|WARN|SKIP <file>:<start-line> <reason>`,
|
|
29
|
+
# then `SPEC-BASH: checked=N failed=F skipped=S shellcheck_warnings=W`. Only bash -n fails a run.
|
|
30
|
+
# Exit codes: 0 no failures, 1 at least one block failed, 2 usage error.
|
|
31
|
+
#
|
|
32
|
+
# Portable: bash 3.2, BSD/GNU awk and sed. Requires git only with --base.
|
|
33
|
+
|
|
34
|
+
set -u
|
|
35
|
+
|
|
36
|
+
BASE=""; SHELLCHECK=0; FILES=""
|
|
37
|
+
usage() { echo "ERROR: Usage: check-spec-bash.sh [--base <git-ref>] [--shellcheck] <file.md>..." >&2; exit 2; }
|
|
38
|
+
|
|
39
|
+
while [ "$#" -gt 0 ]; do
|
|
40
|
+
case "$1" in
|
|
41
|
+
--base) [ "$#" -ge 2 ] || usage; BASE="$2"; shift 2 ;;
|
|
42
|
+
--shellcheck) SHELLCHECK=1; shift ;;
|
|
43
|
+
--*) usage ;;
|
|
44
|
+
*) FILES="$FILES
|
|
45
|
+
$1"; shift ;;
|
|
46
|
+
esac
|
|
47
|
+
done
|
|
48
|
+
[ -n "$FILES" ] || usage
|
|
49
|
+
|
|
50
|
+
TMPD=$(mktemp -d "${TMPDIR:-/tmp}/check-spec-bash.XXXXXX") || exit 2
|
|
51
|
+
trap 'rm -rf "$TMPD"' EXIT
|
|
52
|
+
|
|
53
|
+
checked=0; failed=0; skipped=0; warned=0
|
|
54
|
+
|
|
55
|
+
changed_lines() { # changed_lines <file> → one new-side line number per line
|
|
56
|
+
git diff -U0 "$BASE" -- "$1" 2>/dev/null | awk '
|
|
57
|
+
/^@@/ { split($3, a, ","); s = substr(a[1], 2) + 0; n = (a[2] == "") ? 1 : a[2] + 0
|
|
58
|
+
for (i = 0; i < n; i++) print s + i }'
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
while IFS= read -r f; do
|
|
62
|
+
[ -n "$f" ] || continue
|
|
63
|
+
case "$f" in *.md) ;; *) continue ;; esac
|
|
64
|
+
[ -f "$f" ] || { echo "SKIP $f missing"; continue; }
|
|
65
|
+
|
|
66
|
+
CH="$TMPD/changed"; : > "$CH"
|
|
67
|
+
if [ -n "$BASE" ]; then
|
|
68
|
+
changed_lines "$f" > "$CH"
|
|
69
|
+
[ -s "$CH" ] || continue
|
|
70
|
+
fi
|
|
71
|
+
|
|
72
|
+
# Split the file into one file per ```bash block, named by its opening line number.
|
|
73
|
+
rm -f "$TMPD"/blk.*
|
|
74
|
+
awk -v d="$TMPD" '
|
|
75
|
+
!inb && /^[ \t]*```(bash|sh)[ \t]*$/ { inb = 1; start = NR; out = d "/blk." NR; printf "" > out
|
|
76
|
+
if (prev ~ /^[ \t]*<!--[ \t]*allowlist:check-spec-bash[ \t]*-->[ \t]*$/) print "#ALLOWLISTED" > out
|
|
77
|
+
next }
|
|
78
|
+
inb && /^[ \t]*```[ \t]*$/ { inb = 0; close(out); print start "\t" NR > (d "/ranges"); next }
|
|
79
|
+
inb { print > out; next }
|
|
80
|
+
{ if ($0 !~ /^[ \t]*$/) prev = $0 }
|
|
81
|
+
' "$f"
|
|
82
|
+
[ -f "$TMPD/ranges" ] || continue
|
|
83
|
+
|
|
84
|
+
while IFS="$(printf '\t')" read -r start end; do
|
|
85
|
+
blk="$TMPD/blk.$start"
|
|
86
|
+
if [ -n "$BASE" ]; then
|
|
87
|
+
hit=$(awk -v s="$start" -v e="$end" '$1 >= s && $1 <= e { print "y"; exit }' "$CH")
|
|
88
|
+
[ "$hit" = "y" ] || continue
|
|
89
|
+
fi
|
|
90
|
+
if head -1 "$blk" | grep -q '^#ALLOWLISTED$'; then
|
|
91
|
+
skipped=$((skipped + 1)); echo "SKIP $f:$start allowlisted fragment"; continue
|
|
92
|
+
fi
|
|
93
|
+
if grep -Eq '(^|[^A-Za-z_])(Skill|Agent|Task|Read)\(' "$blk"; then
|
|
94
|
+
skipped=$((skipped + 1)); echo "SKIP $f:$start pseudo tool call"; continue
|
|
95
|
+
fi
|
|
96
|
+
# Neutralise {PLACEHOLDER} notation (and {FORGE_SKILL_PREFIX}name) into a variable expansion,
|
|
97
|
+
# so tests like [ -z "{X}" ] are not misread as constant strings.
|
|
98
|
+
sed -E 's/\{[A-Z][A-Z0-9_]*\}/${PH}/g' "$blk" > "$blk.sh"
|
|
99
|
+
checked=$((checked + 1))
|
|
100
|
+
if ! err=$(bash -n "$blk.sh" 2>&1); then
|
|
101
|
+
failed=$((failed + 1))
|
|
102
|
+
echo "FAIL $f:$start bash -n: $(printf '%s' "$err" | head -1 | sed "s|$blk.sh|block|")"
|
|
103
|
+
continue
|
|
104
|
+
fi
|
|
105
|
+
if [ "$SHELLCHECK" = 1 ] && command -v shellcheck >/dev/null 2>&1; then
|
|
106
|
+
if ! err=$(shellcheck -S error -s bash "$blk.sh" 2>&1); then
|
|
107
|
+
warned=$((warned + 1))
|
|
108
|
+
echo "WARN $f:$start shellcheck: $(printf '%s' "$err" | grep -m1 -E 'SC[0-9]+' || printf '%s' "$err" | head -1)"
|
|
109
|
+
continue
|
|
110
|
+
fi
|
|
111
|
+
fi
|
|
112
|
+
echo "OK $f:$start"
|
|
113
|
+
done < "$TMPD/ranges"
|
|
114
|
+
rm -f "$TMPD/ranges"
|
|
115
|
+
done <<EOF
|
|
116
|
+
$FILES
|
|
117
|
+
EOF
|
|
118
|
+
|
|
119
|
+
echo "SPEC-BASH: checked=$checked failed=$failed skipped=$skipped shellcheck_warnings=$warned"
|
|
120
|
+
[ "$failed" -eq 0 ]
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios
|
|
3
|
+
# SPDX-License-Identifier: AGPL-3.0-or-later
|
|
4
|
+
#
|
|
5
|
+
# Fixtures for scripts/check-spec-bash.sh. Run: bash scripts/check-spec-bash.test.sh (bash 3.2 compatible)
|
|
6
|
+
|
|
7
|
+
set -u
|
|
8
|
+
DIR=$(cd "$(dirname "$0")" && pwd)
|
|
9
|
+
S="$DIR/check-spec-bash.sh"
|
|
10
|
+
T=$(mktemp -d "${TMPDIR:-/tmp}/check-spec-bash-test.XXXXXX")
|
|
11
|
+
trap 'rm -rf "$T"' EXIT
|
|
12
|
+
pass=0; fail=0
|
|
13
|
+
FENCE='```'
|
|
14
|
+
|
|
15
|
+
check() { # check <want_rc> <want_substring> <desc> -- args...
|
|
16
|
+
want_rc="$1"; want="$2"; desc="$3"; shift 4
|
|
17
|
+
out=$(cd "$T" && bash "$S" "$@" 2>&1); rc=$?
|
|
18
|
+
case "$out" in *"$want"*) ok=1 ;; *) ok=0 ;; esac
|
|
19
|
+
if [ "$rc" = "$want_rc" ] && [ "$ok" = 1 ]; then pass=$((pass+1)); else fail=$((fail+1)); echo "FAIL: $desc (rc=$rc)"; printf '%s\n' "$out" | sed 's/^/ /'; fi
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
cat > "$T/good.md" <<EOF
|
|
23
|
+
# Good
|
|
24
|
+
${FENCE}bash
|
|
25
|
+
if [ -z "{GH_REPO}" ]; then echo none; fi
|
|
26
|
+
gh issue view {NUMBER} -R {GH_REPO} --json labels
|
|
27
|
+
${FENCE}
|
|
28
|
+
Prose with it's apostrophe and an unbalanced ( paren.
|
|
29
|
+
EOF
|
|
30
|
+
|
|
31
|
+
cat > "$T/bad.md" <<EOF
|
|
32
|
+
# Bad
|
|
33
|
+
${FENCE}bash
|
|
34
|
+
if [ -n "\$X" ]; then
|
|
35
|
+
echo "unterminated
|
|
36
|
+
fi
|
|
37
|
+
${FENCE}
|
|
38
|
+
EOF
|
|
39
|
+
|
|
40
|
+
cat > "$T/pseudo.md" <<EOF
|
|
41
|
+
${FENCE}bash
|
|
42
|
+
OUT=\$(Skill(skill="x", args="y"))
|
|
43
|
+
${FENCE}
|
|
44
|
+
EOF
|
|
45
|
+
|
|
46
|
+
cat > "$T/allow.md" <<EOF
|
|
47
|
+
<!-- allowlist:check-spec-bash -->
|
|
48
|
+
${FENCE}bash
|
|
49
|
+
if true; then
|
|
50
|
+
${FENCE}
|
|
51
|
+
EOF
|
|
52
|
+
|
|
53
|
+
cat > "$T/prose-mention.md" <<EOF
|
|
54
|
+
Use \`<!-- allowlist:check-spec-bash -->\` before an intentional fragment.
|
|
55
|
+
${FENCE}bash
|
|
56
|
+
if true; then
|
|
57
|
+
${FENCE}
|
|
58
|
+
EOF
|
|
59
|
+
|
|
60
|
+
cat > "$T/notbash.md" <<EOF
|
|
61
|
+
${FENCE}yaml
|
|
62
|
+
key: [unclosed
|
|
63
|
+
${FENCE}
|
|
64
|
+
EOF
|
|
65
|
+
|
|
66
|
+
check 0 "OK good.md:2" "valid block with placeholders passes" -- good.md
|
|
67
|
+
check 1 "FAIL bad.md:2 bash -n" "unterminated quote fails" -- bad.md
|
|
68
|
+
check 0 "SKIP pseudo.md:1 pseudo tool" "pseudo tool call skipped" -- pseudo.md
|
|
69
|
+
check 0 "SKIP allow.md:2 allowlisted" "allowlisted fragment skipped" -- allow.md
|
|
70
|
+
check 0 "checked=0 failed=0" "non-bash fence ignored" -- notbash.md
|
|
71
|
+
check 1 "FAIL prose-mention.md:2" "prose mention of the marker does not allowlist" -- prose-mention.md
|
|
72
|
+
check 1 "checked=2 failed=1 skipped=1" "multi-file summary" -- good.md bad.md pseudo.md
|
|
73
|
+
check 2 "" "no files is a usage error" --
|
|
74
|
+
check 2 "" "unknown flag" -- --bogus good.md
|
|
75
|
+
|
|
76
|
+
# --base: only blocks touching changed lines are checked, so pre-existing breakage never fails a change.
|
|
77
|
+
( cd "$T" && git init -q . && git -c user.email=t@t -c user.name=t add bad.md good.md && git -c user.email=t@t -c user.name=t commit -q -m base )
|
|
78
|
+
check 0 "checked=0 failed=0" "--base with no change checks nothing" -- --base HEAD bad.md good.md
|
|
79
|
+
printf '\nnew prose line\n' >> "$T/bad.md"
|
|
80
|
+
check 0 "checked=0 failed=0" "--base: prose change outside a broken block passes" -- --base HEAD bad.md
|
|
81
|
+
cat >> "$T/good.md" <<EOF
|
|
82
|
+
${FENCE}bash
|
|
83
|
+
for x in a b; do
|
|
84
|
+
echo "\$x"
|
|
85
|
+
${FENCE}
|
|
86
|
+
EOF
|
|
87
|
+
check 1 "FAIL good.md:" "--base: newly added broken block fails" -- --base HEAD good.md
|
|
88
|
+
|
|
89
|
+
echo "check-spec-bash.test.sh: passed=$pass failed=$fail"
|
|
90
|
+
[ "$fail" -eq 0 ]
|
|
@@ -21,15 +21,20 @@
|
|
|
21
21
|
# review finding. Default `none`.
|
|
22
22
|
# --text/--text-file Finding title + body + affected file paths, used for the
|
|
23
23
|
# content-based safety exemption.
|
|
24
|
+
# --inpr-diff <F> --file <P> In-PR fix gate (#3387, narrowed): F is a file listing the
|
|
25
|
+
# PR's changed paths, one per line; P is the finding's file path. When both
|
|
26
|
+
# are given, a MEDIUM CONFIRMED finding whose path is in the PR diff is
|
|
27
|
+
# classified INPR_FIX — fix it on the PR before merge instead of filing it.
|
|
28
|
+
# Callers pass these only for the first fix round of an auto-merge review.
|
|
24
29
|
#
|
|
25
|
-
# Output (stdout, one line): `ISSUE <reason>` or `
|
|
30
|
+
# Output (stdout, one line): `ISSUE <reason>`, `NOTE <reason>` or `INPR_FIX <reason>`.
|
|
26
31
|
# Exit codes: 0 classified, 2 usage error.
|
|
27
32
|
#
|
|
28
33
|
# Rules (forge#3060, tightened after the 2026-10-08 cascade audit):
|
|
29
34
|
# 1. HIGH/CRITICAL, or unparseable severity → ISSUE.
|
|
30
35
|
# 2. Safety exemption is CONTENT-based: the text matches the security/billing
|
|
31
36
|
# keyword set, or the finding came from a dedicated, signal-selected domain
|
|
32
|
-
# agent (Auth, Billing, Concurrency, Database). Origin from the always-on
|
|
37
|
+
# agent (Auth, Billing, Concurrency, Database) and is MEDIUM+ or CONFIRMED. Origin from the always-on
|
|
33
38
|
# "General Security & Quality" agent alone NO LONGER exempts a finding —
|
|
34
39
|
# that agent runs on every PR, so origin-based exemption filed nearly every
|
|
35
40
|
# LOW note it raised (50 of 60 would-be notes in the audited batch).
|
|
@@ -50,9 +55,11 @@ CONFIDENCE=""
|
|
|
50
55
|
AGENT=""
|
|
51
56
|
LINEAGE="none"
|
|
52
57
|
TEXT=""
|
|
58
|
+
INPR_DIFF=""
|
|
59
|
+
FINDING_FILE=""
|
|
53
60
|
|
|
54
61
|
usage() {
|
|
55
|
-
echo "ERROR: Usage: classify-finding.sh --severity <S> [--confidence <C>] [--agent <A>] [--lineage none|review-finding] [--text <T> | --text-file <F>]" >&2
|
|
62
|
+
echo "ERROR: Usage: classify-finding.sh --severity <S> [--confidence <C>] [--agent <A>] [--lineage none|review-finding] [--text <T> | --text-file <F>] [--inpr-diff <F> --file <P>]" >&2
|
|
56
63
|
exit 2
|
|
57
64
|
}
|
|
58
65
|
|
|
@@ -63,6 +70,8 @@ while [ "$#" -gt 0 ]; do
|
|
|
63
70
|
--agent) [ "$#" -ge 2 ] || usage; AGENT="$2"; shift 2 ;;
|
|
64
71
|
--lineage) [ "$#" -ge 2 ] || usage; LINEAGE="$2"; shift 2 ;;
|
|
65
72
|
--text) [ "$#" -ge 2 ] || usage; TEXT="$2"; shift 2 ;;
|
|
73
|
+
--inpr-diff) [ "$#" -ge 2 ] || usage; INPR_DIFF="$2"; shift 2 ;;
|
|
74
|
+
--file) [ "$#" -ge 2 ] || usage; FINDING_FILE="$2"; shift 2 ;;
|
|
66
75
|
--text-file)
|
|
67
76
|
[ "$#" -ge 2 ] || usage
|
|
68
77
|
[ -r "$2" ] || { echo "ERROR: --text-file not readable: $2" >&2; exit 2; }
|
|
@@ -76,6 +85,10 @@ case "$LINEAGE" in
|
|
|
76
85
|
*) echo "ERROR: --lineage must be 'none' or 'review-finding' (got '$LINEAGE')" >&2; exit 2 ;;
|
|
77
86
|
esac
|
|
78
87
|
|
|
88
|
+
if [ -n "$INPR_DIFF" ] && [ ! -r "$INPR_DIFF" ]; then
|
|
89
|
+
echo "ERROR: --inpr-diff not readable: $INPR_DIFF" >&2; exit 2
|
|
90
|
+
fi
|
|
91
|
+
|
|
79
92
|
upper() { printf '%s' "$1" | tr '[:lower:]' '[:upper:]' | tr -d '[:space:]'; }
|
|
80
93
|
SEV=$(upper "$SEVERITY")
|
|
81
94
|
CONF=$(upper "$CONFIDENCE")
|
|
@@ -86,6 +99,17 @@ case "$SEV" in
|
|
|
86
99
|
*) echo "ISSUE unparseable-severity"; exit 0 ;;
|
|
87
100
|
esac
|
|
88
101
|
|
|
102
|
+
# In-PR fix gate (#3387, narrowed). MEDIUM CONFIRMED is an ISSUE under every rule below
|
|
103
|
+
# (lineage included), so checking here changes only where it is handled, never whether.
|
|
104
|
+
if [ -n "$INPR_DIFF" ] && [ -n "$FINDING_FILE" ] && [ "$SEV" = "MEDIUM" ] && [ "$CONF" = "CONFIRMED" ]; then
|
|
105
|
+
_path="${FINDING_FILE#./}"; _path="${_path%%:*}"
|
|
106
|
+
if grep -Fxq -- "$_path" "$INPR_DIFF"; then
|
|
107
|
+
echo "INPR_FIX medium-confirmed-in-diff"
|
|
108
|
+
exit 0
|
|
109
|
+
fi
|
|
110
|
+
fi
|
|
111
|
+
|
|
112
|
+
|
|
89
113
|
# Safety exemption. Keywords match whole words, where `_`, `-`, `/` and other
|
|
90
114
|
# punctuation separate words (so `auth_service` matches, `author` does not).
|
|
91
115
|
SAFETY=""
|
|
@@ -93,9 +117,12 @@ KW='security|auth|authz|authn|billing|payment|stripe|charge|invoice|injection|xs
|
|
|
93
117
|
if printf '%s' "$TEXT" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9\n' ' ' | grep -Eqw "($KW)"; then
|
|
94
118
|
SAFETY="keyword"
|
|
95
119
|
fi
|
|
120
|
+
# A dedicated domain agent rescues a finding only when it is MEDIUM+ or CONFIRMED:
|
|
121
|
+
# a LOW/POSSIBLE finding stays a NOTE whichever reviewer raised it.
|
|
96
122
|
AGENT_LC=$(printf '%s' "$AGENT" | tr '[:upper:]' '[:lower:]')
|
|
97
123
|
case "$AGENT_LC" in
|
|
98
|
-
auth*|billing*|concurrency*|database*)
|
|
124
|
+
auth*|billing*|concurrency*|database*)
|
|
125
|
+
if [ "$SEV" = "MEDIUM" ] || [ "$CONF" = "CONFIRMED" ]; then SAFETY="${SAFETY:-domain-agent}"; fi ;;
|
|
99
126
|
esac
|
|
100
127
|
|
|
101
128
|
if [ "$LINEAGE" = "review-finding" ]; then
|
|
@@ -44,10 +44,13 @@ expect ISSUE "LOW with Secrets (case)" -- --severity LOW --confidence LIK
|
|
|
44
44
|
expect ISSUE "LOW with permission-check" -- --severity LOW --confidence LIKELY --text "permission-check skipped"
|
|
45
45
|
expect NOTE "'author' is not 'auth'" -- --severity LOW --confidence LIKELY --text "author field unused"
|
|
46
46
|
expect NOTE "'tokenizer' is not 'token'" -- --severity LOW --confidence LIKELY --text "tokenizer splits oddly"
|
|
47
|
-
expect ISSUE "LOW from Auth domain agent"
|
|
48
|
-
expect ISSUE "
|
|
49
|
-
expect
|
|
50
|
-
expect
|
|
47
|
+
expect ISSUE "LOW CONFIRMED from Auth domain agent" -- --severity LOW --confidence CONFIRMED --agent "Auth" --text "nit"
|
|
48
|
+
expect ISSUE "MEDIUM POSSIBLE from Billing agent" -- --severity MEDIUM --confidence POSSIBLE --agent "billing" --text "nit"
|
|
49
|
+
expect NOTE "LOW LIKELY from Auth domain agent is a NOTE" -- --severity LOW --confidence LIKELY --agent "Auth" --text "nit"
|
|
50
|
+
expect NOTE "LOW POSSIBLE from Database agent is a NOTE" -- --severity LOW --confidence POSSIBLE --agent "Database" --text "nit"
|
|
51
|
+
expect NOTE "LOW POSSIBLE Concurrency reviewer is a NOTE (AlterLab #34671)" -- --severity LOW --confidence POSSIBLE --agent "Concurrency reviewer" --text "lock edge cases"
|
|
52
|
+
expect ISSUE "LOW CONFIRMED Concurrency reviewer files" -- --severity LOW --confidence CONFIRMED --agent "Concurrency reviewer" --text "lock edge cases"
|
|
53
|
+
expect ISSUE "keyword still rescues LOW POSSIBLE from domain agent" -- --severity LOW --confidence POSSIBLE --agent "Database" --text "sql injection"
|
|
51
54
|
|
|
52
55
|
# --- Rule 3: review-finding lineage (fix for a finding) ---
|
|
53
56
|
expect NOTE "lineage LOW CONFIRMED note" -- --severity LOW --confidence CONFIRMED --lineage review-finding --text "nit"
|
|
@@ -63,6 +66,23 @@ printf 'Title: unchecked xss sink\n' > "$TMPF"
|
|
|
63
66
|
expect ISSUE "text-file keyword" -- --severity LOW --confidence LIKELY --text-file "$TMPF"
|
|
64
67
|
rm -f "$TMPF"
|
|
65
68
|
|
|
69
|
+
# --- In-PR fix gate (#3387, narrowed): MEDIUM CONFIRMED in the PR diff ---
|
|
70
|
+
DIFF=$(mktemp "${TMPDIR:-/tmp}/classify-finding-diff.XXXXXX")
|
|
71
|
+
printf 'src/api/jobs.py\ncommands/review-pr.md\n' > "$DIFF"
|
|
72
|
+
expect INPR_FIX "MEDIUM CONFIRMED in diff" -- --severity MEDIUM --confidence CONFIRMED --inpr-diff "$DIFF" --file src/api/jobs.py
|
|
73
|
+
expect INPR_FIX "file:line form matches" -- --severity MEDIUM --confidence CONFIRMED --inpr-diff "$DIFF" --file "src/api/jobs.py:142"
|
|
74
|
+
expect INPR_FIX "./ prefix matches" -- --severity MEDIUM --confidence CONFIRMED --inpr-diff "$DIFF" --file ./commands/review-pr.md
|
|
75
|
+
expect INPR_FIX "lineage PR still fixes in-PR" -- --severity MEDIUM --confidence CONFIRMED --lineage review-finding --inpr-diff "$DIFF" --file src/api/jobs.py
|
|
76
|
+
expect ISSUE "MEDIUM CONFIRMED outside diff files" -- --severity MEDIUM --confidence CONFIRMED --inpr-diff "$DIFF" --file src/api/other.py
|
|
77
|
+
expect ISSUE "MEDIUM LIKELY in diff is not gated" -- --severity MEDIUM --confidence LIKELY --inpr-diff "$DIFF" --file src/api/jobs.py
|
|
78
|
+
expect ISSUE "HIGH in diff stays ISSUE (7B blocks it)" -- --severity HIGH --confidence CONFIRMED --inpr-diff "$DIFF" --file src/api/jobs.py
|
|
79
|
+
expect NOTE "LOW in diff stays NOTE" -- --severity LOW --confidence CONFIRMED --inpr-diff "$DIFF" --file src/api/jobs.py
|
|
80
|
+
expect ISSUE "no --inpr-diff: unchanged behaviour" -- --severity MEDIUM --confidence CONFIRMED --file src/api/jobs.py
|
|
81
|
+
expect ISSUE "no --file: unchanged behaviour" -- --severity MEDIUM --confidence CONFIRMED --inpr-diff "$DIFF"
|
|
82
|
+
expect ISSUE "prefix of a diff path does not match" -- --severity MEDIUM --confidence CONFIRMED --inpr-diff "$DIFF" --file src/api
|
|
83
|
+
rm -f "$DIFF"
|
|
84
|
+
expect_exit 2 "unreadable --inpr-diff" -- --severity MEDIUM --inpr-diff /nonexistent/d --file x
|
|
85
|
+
|
|
66
86
|
# --- usage errors ---
|
|
67
87
|
expect_exit 2 "unknown flag" -- --bogus x
|
|
68
88
|
expect_exit 2 "bad lineage" -- --severity LOW --lineage P3
|