forgedock 1.9.0 → 1.9.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/commands/orchestrate/phase-4-execution.md +12 -2
- package/commands/quality-gate.md +29 -0
- package/commands/review-pr.md +15 -4
- package/commands/work-on/build/validate.md +15 -9
- package/docs/RELEASE-NOTES.md +9 -0
- package/package.json +1 -1
- package/scripts/check-spec-bash.sh +120 -0
- package/scripts/check-spec-bash.test.sh +90 -0
- package/scripts/classify-finding.sh +5 -2
- package/scripts/classify-finding.test.sh +7 -4
|
@@ -2837,10 +2837,20 @@ When `CASCADE_MAX_AMPLIFICATION` is not `off`, and the current ratio is greater
|
|
|
2837
2837
|
**Deterministic breaker check (MANDATORY on every dispatch path — engine CLI, Agent-spawn fallback, or a hand-driven orchestrator):** the in-spec bookkeeping above only runs when the Step 4B/4C bash runs. A batch driven through the Agent-spawn fallback never evaluated it and reached about 2 findings per merge (2026-10-08 audit). Before dispatching ANY `review-finding` issue at `priority:P3` or below, and after every completed merge, run `scripts/amplification-breaker.sh` against GitHub state. Its exit code is authoritative, and it overrides an in-memory `AMPLIFICATION_BREAKER_TRIPPED=false`:
|
|
2838
2838
|
|
|
2839
2839
|
```bash
|
|
2840
|
+
# Same resolver as review-pr §6B.5: plugin root, FORGE_ROOT, FORGEDOCK_HOME, newest pinned plugin cache
|
|
2841
|
+
# (CLAUDE_CONFIG_DIR, then ~/.claude), then the repo's own scripts/.
|
|
2840
2842
|
AMP_SCRIPT=""
|
|
2841
|
-
|
|
2842
|
-
|
|
2843
|
+
_cands="$(printf '%s\n' '${CLAUDE_PLUGIN_ROOT}' "${FORGE_ROOT:-}" "${FORGEDOCK_HOME:-}")"
|
|
2844
|
+
for _cfg in "${CLAUDE_CONFIG_DIR:-}" "$HOME/.claude"; do
|
|
2845
|
+
[ -n "$_cfg" ] || continue
|
|
2846
|
+
_cands="$_cands
|
|
2847
|
+
$(find -L "$_cfg/plugins/cache/forgedock/forgedock" -mindepth 1 -maxdepth 1 -type d 2>/dev/null | awk -F/ '$NF ~ /^[0-9]+\.[0-9]+\.[0-9]+$/{split($NF,a,".");printf "%d %d %d %s\n",a[1],a[2],a[3],$0}' | sort -k1,1nr -k2,2nr -k3,3nr | cut -d' ' -f4- || true)"
|
|
2843
2848
|
done
|
|
2849
|
+
_cands="$_cands
|
|
2850
|
+
$PWD"
|
|
2851
|
+
while IFS= read -r _c; do
|
|
2852
|
+
case "$_c" in /*) [ -z "$AMP_SCRIPT" ] && [ -f "$_c/scripts/amplification-breaker.sh" ] && AMP_SCRIPT="$_c/scripts/amplification-breaker.sh" ;; esac
|
|
2853
|
+
done <<< "$_cands"
|
|
2844
2854
|
if [ "$AMPLIFICATION_BREAKER" != "off" ]; then
|
|
2845
2855
|
if [ -n "$AMP_SCRIPT" ]; then
|
|
2846
2856
|
AMP_LINE=$(bash "$AMP_SCRIPT" --since "$BATCH_T0" -R {GH_REPO}); AMP_RC=$?
|
package/commands/quality-gate.md
CHANGED
|
@@ -277,6 +277,7 @@ the gate rather than leaving the workflow blocked.
|
|
|
277
277
|
- **2R (Registry checks)**: ALWAYS run — executes all promoted checks from `scripts/check-registry/manifest.json` <!-- Added: forge#1331 -->
|
|
278
278
|
- **2R.5 (Adaptive gate.d checks)**: ALWAYS run when `adaptive_scripts.enabled` — executes `.forgedock/scripts/gate.d/*.sh`; each script is repo-specific and was promoted from a recurring finding pattern <!-- Added: forge#1739 -->
|
|
279
279
|
- **2G.8 (Wire-through proof)**: Run if `WIRE_THROUGH` in DOMAINS — demands each newly added conditional path be demonstrably reachable <!-- Added: forge#1731 -->
|
|
280
|
+
- **2G.10 (Spec bash syntax)**: Run if `FORGE_GRAPH` in DOMAINS and any `commands/**/*.md` changed — `bash -n` on changed ```bash blocks in specs (HIGH); shellcheck advisory
|
|
280
281
|
- **2G.9 (Danger-zone recurrence)**: Run if `FORGE_GRAPH` in DOMAINS — cross-references injected danger-zone rule cards from FORGE:CONTEXT against the diff; violations tagged `known-pattern-recurrence` (HIGH) <!-- Added: forge#1744 -->
|
|
281
282
|
- **2U (Coverage-reduction check)**: ALWAYS run (`COVERAGE_REDUCTION` in DOMAINS) — flags diffs that delete test files, remove test cases, add skip markers, or remove/disable workflow test steps <!-- Added: forge#3257 -->
|
|
282
283
|
- **2S (Test failure classification)**: Run if any Step 2 check invoked a test suite and it failed — classifies the failure as PRE_BROKEN, FLAKY, or REAL using `scripts/flaky-quarantine.sh` <!-- Added: forge#1336 -->
|
|
@@ -718,6 +719,34 @@ fi
|
|
|
718
719
|
|
|
719
720
|
Do NOT suppress the finding by making the check less strict or adding fake no-op tests. The intent is a real proof that the path is reachable. <!-- Added: forge#1731 -->
|
|
720
721
|
|
|
722
|
+
### 2G.10: Spec bash syntax check (FORGE_GRAPH domain)
|
|
723
|
+
|
|
724
|
+
**Triggered when**: changed files include `commands/**/*.md`. Skips silently in any project without `scripts/check-spec-bash.sh`.
|
|
725
|
+
|
|
726
|
+
**Why this matters**: in ForgeDock the pipeline code is bash inside Markdown specs. The executable-file checks above exclude `.md` files, and acceptance checks are mostly greps, so spec bash used to merge without ever being parsed. Reviewers were then the first to run it, which drove most review findings (2026-10-08 cascade audit). This check runs `bash -n` on every ```bash block that contains a changed line, after neutralising `{PLACEHOLDER}` notation. Blocks with pseudo tool calls (`Skill(`/`Agent(`/`Task(`) and blocks preceded by `<!-- allowlist:check-spec-bash -->` are skipped. Pre-existing blocks are never checked, so the baseline cannot block a change. `shellcheck -S error` results are advisory only.
|
|
727
|
+
|
|
728
|
+
```bash
|
|
729
|
+
SPEC_BASH_CHECKER="$FORGEDOCK_SCRIPTS/check-spec-bash.sh"
|
|
730
|
+
[ -f "$SPEC_BASH_CHECKER" ] || SPEC_BASH_CHECKER="{WORKTREE_PATH}/scripts/check-spec-bash.sh"
|
|
731
|
+
SPEC_MD_FILES=$(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '^commands/.*\.md$' || true)
|
|
732
|
+
if [ -f "$SPEC_BASH_CHECKER" ] && [ -n "$SPEC_MD_FILES" ]; then
|
|
733
|
+
PR_BASE="${PR_BASE:-staging}"
|
|
734
|
+
SPEC_BASH_BASE=$(git -C "{WORKTREE_PATH}" merge-base "origin/${PR_BASE}" HEAD 2>/dev/null || true)
|
|
735
|
+
if [ -z "$SPEC_BASH_BASE" ]; then
|
|
736
|
+
echo "FORGE_GRAPH | HIGH | spec-bash | cannot resolve merge-base with origin/${PR_BASE} — spec bash not checked (fail closed)"
|
|
737
|
+
else
|
|
738
|
+
# shellcheck disable=SC2086 # SPEC_MD_FILES is a newline list of repo-relative paths without spaces
|
|
739
|
+
SPEC_BASH_REPORT=$(cd "{WORKTREE_PATH}" && bash "$SPEC_BASH_CHECKER" --base "$SPEC_BASH_BASE" --shellcheck $SPEC_MD_FILES 2>&1)
|
|
740
|
+
echo "$SPEC_BASH_REPORT"
|
|
741
|
+
echo "$SPEC_BASH_REPORT" | grep -E '^FAIL ' | while IFS= read -r hit; do
|
|
742
|
+
echo "FORGE_GRAPH | HIGH | spec-bash | ${hit#FAIL }"
|
|
743
|
+
done
|
|
744
|
+
fi
|
|
745
|
+
fi
|
|
746
|
+
```
|
|
747
|
+
|
|
748
|
+
Report each `FAIL` line as **HIGH**: the changed block does not parse, so any agent that runs it hits a syntax error. Fix the bash. If the block is an intentional fragment (for example an `if` opened in one block and closed in the next), put `<!-- allowlist:check-spec-bash -->` on the line before the fence. `WARN` (shellcheck) lines are advisory and go in the report without blocking.
|
|
749
|
+
|
|
721
750
|
### 2G.9: Danger-zone recurrence check (FORGE_GRAPH domain) <!-- Added: forge#1744 -->
|
|
722
751
|
|
|
723
752
|
**Triggered when**: `FORGE_GRAPH` in DOMAINS (changed files include `commands/*.md` specs or `scripts/*` files). Skips silently when the FORGE:CONTEXT comment on the issue contains no `### Danger-Zone Rule Cards` section (i.e., Phase C0.5 injected zero cards — nothing to cross-reference).
|
package/commands/review-pr.md
CHANGED
|
@@ -1692,16 +1692,27 @@ If still 0: review is clean — skip to Phase 7.
|
|
|
1692
1692
|
Filing a standalone `review-finding` issue for every LOW/POSSIBLE reviewer note makes the cascade amplify (each merged fix PR spawns its own P3 issues, which spawn more). Classify each deduped finding before 6C with the deterministic classifier — do NOT hand-classify when the script resolves:
|
|
1693
1693
|
|
|
1694
1694
|
- **ISSUE always**: `**Severity**: HIGH` or `CRITICAL`, or a missing/unparseable severity.
|
|
1695
|
-
- **Safety exemption is content-based**: a LOW or POSSIBLE finding is still filed when its file path / title / body matches the security/billing keyword set (`security|auth|authz|authn|billing|payment|stripe|charge|invoice|injection|xss|csrf|ssrf|idor|secret(s)|credential(s)|permission(s)|sql|token|password|redact`, whole words, `_`/`-`/`/` separate words — so `auth_service` matches but `author`/`tokenizer` do not), or when it came from a dedicated, signal-selected domain agent (Auth, Billing, Concurrency, Database). **Origin from the always-on General Security & Quality agent alone does NOT exempt a finding**: that agent runs on every PR, so origin-based exemption filed nearly every LOW note it raised (the 2026-10-08 cascade audit: 50 of 60 would-be notes in one batch were filed only for that reason).
|
|
1695
|
+
- **Safety exemption is content-based**: a LOW or POSSIBLE finding is still filed when its file path / title / body matches the security/billing keyword set (`security|auth|authz|authn|billing|payment|stripe|charge|invoice|injection|xss|csrf|ssrf|idor|secret(s)|credential(s)|permission(s)|sql|token|password|redact`, whole words, `_`/`-`/`/` separate words — so `auth_service` matches but `author`/`tokenizer` do not), or when it came from a dedicated, signal-selected domain agent (Auth, Billing, Concurrency, Database) AND is MEDIUM+ or CONFIRMED — a LOW/POSSIBLE finding stays a NOTE whichever reviewer raised it. **Origin from the always-on General Security & Quality agent alone does NOT exempt a finding**: that agent runs on every PR, so origin-based exemption filed nearly every LOW note it raised (the 2026-10-08 cascade audit: 50 of 60 would-be notes in one batch were filed only for that reason).
|
|
1696
1696
|
- **Review-finding lineage** (the PR's linked issue `MERGE_ISSUE` carries the `review-finding` label, any priority — this PR is itself a fix for a finding): only MEDIUM findings that are CONFIRMED, or LIKELY and safety-exempt, become issues; LOW and POSSIBLE are always NOTEs. A fix for a finding must not mint a new generation of findings.
|
|
1697
1697
|
- **Otherwise**: `**Severity**: LOW`, or `**Confidence**: POSSIBLE` below HIGH, is a NOTE unless safety-exempt; everything else (MEDIUM CONFIRMED/LIKELY) is an ISSUE.
|
|
1698
1698
|
|
|
1699
1699
|
```bash
|
|
1700
|
-
# Resolve the classifier: running plugin
|
|
1700
|
+
# Resolve the classifier: running plugin, FORGE_ROOT, FORGEDOCK_HOME, newest pinned forgedock plugin cache
|
|
1701
|
+
# under the active config dir (CLAUDE_CONFIG_DIR) then ~/.claude, then the repo's own scripts/ (ForgeDock itself).
|
|
1702
|
+
# The plugin-root placeholder is not always substituted in forked review runs, so the cache scan is what makes
|
|
1703
|
+
# consumer repos resolve the script instead of falling back to classifier=manual.
|
|
1701
1704
|
CLASSIFY_SCRIPT=""
|
|
1702
|
-
|
|
1703
|
-
|
|
1705
|
+
_cands="$(printf '%s\n' '${CLAUDE_PLUGIN_ROOT}' "${FORGE_ROOT:-}" "${FORGEDOCK_HOME:-}")"
|
|
1706
|
+
for _cfg in "${CLAUDE_CONFIG_DIR:-}" "$HOME/.claude"; do
|
|
1707
|
+
[ -n "$_cfg" ] || continue
|
|
1708
|
+
_cands="$_cands
|
|
1709
|
+
$(find -L "$_cfg/plugins/cache/forgedock/forgedock" -mindepth 1 -maxdepth 1 -type d 2>/dev/null | awk -F/ '$NF ~ /^[0-9]+\.[0-9]+\.[0-9]+$/{split($NF,a,".");printf "%d %d %d %s\n",a[1],a[2],a[3],$0}' | sort -k1,1nr -k2,2nr -k3,3nr | cut -d' ' -f4- || true)"
|
|
1704
1710
|
done
|
|
1711
|
+
_cands="$_cands
|
|
1712
|
+
$PWD"
|
|
1713
|
+
while IFS= read -r _c; do
|
|
1714
|
+
case "$_c" in /*) [ -z "$CLASSIFY_SCRIPT" ] && [ -f "$_c/scripts/classify-finding.sh" ] && CLASSIFY_SCRIPT="$_c/scripts/classify-finding.sh" ;; esac
|
|
1715
|
+
done <<< "$_cands"
|
|
1705
1716
|
FINDING_LINEAGE="none"
|
|
1706
1717
|
if [ -n "${MERGE_ISSUE:-}" ] && gh issue view "$MERGE_ISSUE" -R {GH_REPO} --json labels --jq '.labels[].name' 2>/dev/null | grep -qx 'review-finding'; then
|
|
1707
1718
|
FINDING_LINEAGE="review-finding"
|
|
@@ -150,7 +150,7 @@ gh issue edit {NUMBER} {GH_FLAG} --add-label "needs-human" # allowlist:check-com
|
|
|
150
150
|
|
|
151
151
|
## Phase V2: Format and Verify
|
|
152
152
|
|
|
153
|
-
Run after quality gate passes. All tool commands are read from `forge.yaml → verification.commands
|
|
153
|
+
Run after quality gate passes. All tool commands are read from `forge.yaml → verification.commands`. `forge.yaml` is usually gitignored, so it is absent from the worktree: every block below resolves `FORGE_CFG` to the worktree copy if present, else the main checkout's `forge.yaml` (via `git rev-parse --git-common-dir`). Before this, every verification command, learned test command and the SOPS chain check were silently skipped in worktree builds; each step logs `SKIPPED — not configured` when the corresponding key is absent rather than silently passing.
|
|
154
154
|
|
|
155
155
|
**Track skipped checks** — initialize before any check runs:
|
|
156
156
|
```bash
|
|
@@ -178,8 +178,9 @@ run_verification_command() {
|
|
|
178
178
|
**Python**:
|
|
179
179
|
```bash
|
|
180
180
|
cd {WORKTREE_PATH}
|
|
181
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
181
182
|
|
|
182
|
-
PYTHON_FORMAT=$(yq '.verification.commands.python.format // ""'
|
|
183
|
+
PYTHON_FORMAT=$(yq '.verification.commands.python.format // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
183
184
|
if [ -n "$PYTHON_FORMAT" ]; then
|
|
184
185
|
run_verification_command "python.format" "$PYTHON_FORMAT"
|
|
185
186
|
else
|
|
@@ -195,10 +196,11 @@ Failures in `py_compile` are BLOCKING — fix before continuing.
|
|
|
195
196
|
**TypeScript**:
|
|
196
197
|
```bash
|
|
197
198
|
cd {WORKTREE_PATH}
|
|
199
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
198
200
|
|
|
199
|
-
TS_FORMAT=$(yq '.verification.commands.typescript.format // ""'
|
|
200
|
-
TS_TYPECHECK=$(yq '.verification.commands.typescript.typecheck // ""'
|
|
201
|
-
TS_BUILD=$(yq '.verification.commands.typescript.build // ""'
|
|
201
|
+
TS_FORMAT=$(yq '.verification.commands.typescript.format // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
202
|
+
TS_TYPECHECK=$(yq '.verification.commands.typescript.typecheck // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
203
|
+
TS_BUILD=$(yq '.verification.commands.typescript.build // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
202
204
|
|
|
203
205
|
if [ -n "$TS_FORMAT" ]; then
|
|
204
206
|
run_verification_command "typescript.format" "$TS_FORMAT"
|
|
@@ -234,8 +236,9 @@ Before running any test command below, check it against `verification.known_slow
|
|
|
234
236
|
|
|
235
237
|
```bash
|
|
236
238
|
cd {WORKTREE_PATH}
|
|
239
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
237
240
|
# Read directly from forge.yaml (static, operator-declared config).
|
|
238
|
-
KNOWN_SLOW_TESTS=$(yq -o=json -I=0 '.verification.known_slow_tests // []'
|
|
241
|
+
KNOWN_SLOW_TESTS=$(yq -o=json -I=0 '.verification.known_slow_tests // []' "$FORGE_CFG" 2>/dev/null || echo '[]')
|
|
239
242
|
|
|
240
243
|
# apply_known_slow_filter <cmd> — echoes the command to actually run, or "" to
|
|
241
244
|
# skip it entirely. Matching is substring match of `pattern` against the full
|
|
@@ -274,7 +277,8 @@ After all `verification.commands` steps complete, run any commands from `forge.y
|
|
|
274
277
|
|
|
275
278
|
```bash
|
|
276
279
|
cd {WORKTREE_PATH}
|
|
277
|
-
|
|
280
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
281
|
+
LEARNED_TEST_COMMANDS=$(yq -o=json -I=0 '.learned.test_commands // []' "$FORGE_CFG" 2>/dev/null || echo '[]')
|
|
278
282
|
LEARNED_FAILED=0
|
|
279
283
|
if [ -n "$LEARNED_TEST_COMMANDS" ] && [ "$LEARNED_TEST_COMMANDS" != "[]" ] && [ "$LEARNED_TEST_COMMANDS" != "null" ]; then
|
|
280
284
|
echo "Running learned test commands..."
|
|
@@ -368,7 +372,8 @@ After static proxy checks, run a lightweight live browser check using Playwright
|
|
|
368
372
|
|
|
369
373
|
```bash
|
|
370
374
|
cd {WORKTREE_PATH}
|
|
371
|
-
|
|
375
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
376
|
+
APP_URL=$(yq '.services.app_url // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
372
377
|
if [ -z "$APP_URL" ]; then
|
|
373
378
|
echo "SKIPPED — services.app_url not configured in forge.yaml (browser signal check requires a running app URL)"
|
|
374
379
|
else
|
|
@@ -442,7 +447,8 @@ ADDED_LINES=$(git diff HEAD -- {CHANGED_FILES} | grep -E '^\+' | grep -v '^+++')
|
|
|
442
447
|
|
|
443
448
|
```bash
|
|
444
449
|
cd {WORKTREE_PATH}
|
|
445
|
-
|
|
450
|
+
FORGE_CFG=forge.yaml; [ -f "$FORGE_CFG" ] || { _gcd=$(git rev-parse --git-common-dir 2>/dev/null) && _gcd=$(cd "$_gcd" 2>/dev/null && pwd) && [ -f "${_gcd%/.git}/forge.yaml" ] && FORGE_CFG="${_gcd%/.git}/forge.yaml"; } # gitignored forge.yaml exists only in the main checkout, not in worktrees
|
|
451
|
+
SECRETS_BACKEND=$(yq '.deploy.secrets_backend // ""' "$FORGE_CFG" 2>/dev/null || echo '')
|
|
446
452
|
if [ "$SECRETS_BACKEND" != "sops" ]; then
|
|
447
453
|
echo 'SKIP: SOPS chain check — deploy.secrets_backend is not "sops". Configure deploy.secrets_backend in forge.yaml to enable.'
|
|
448
454
|
fi
|
package/docs/RELEASE-NOTES.md
CHANGED
|
@@ -2,6 +2,15 @@
|
|
|
2
2
|
|
|
3
3
|
Operator-facing notes for behavior changes that need action or awareness. Newest first.
|
|
4
4
|
|
|
5
|
+
## Cascade follow-ups: script resolution, domain-agent notes, verification in worktrees, spec bash check
|
|
6
|
+
|
|
7
|
+
- **Script resolution works in consumer repos.** `/review-pr` §6B.5 and the orchestrator breaker now find `classify-finding.sh` and `amplification-breaker.sh` through the newest pinned ForgeDock plugin cache under `CLAUDE_CONFIG_DIR`, then `~/.claude`. They also check `FORGEDOCK_HOME` and the repo's own `scripts/`, as before. Previously the plugin-root placeholder was often left unsubstituted in forked review runs, so 4 of 9 AlterLab reviews fell back to `classifier=manual`.
|
|
8
|
+
- **Domain reviewers no longer file speculative LOW findings.** A finding from the Auth, Billing, Concurrency or Database reviewer is still filed as an issue only when it is MEDIUM+ or CONFIRMED. A LOW/POSSIBLE finding becomes a note whichever reviewer raised it. Security/billing keyword content still files as before.
|
|
9
|
+
- **Validate reads your real `forge.yaml` in worktrees.** `forge.yaml` is usually gitignored, so it never existed in the per-issue worktree, and every `verification.commands` entry, `learned.test_commands` entry and the SOPS chain check were silently skipped. Validate now falls back to the main checkout's `forge.yaml`.
|
|
10
|
+
- **The quality gate parses spec bash.** New step 2G.10 runs `scripts/check-spec-bash.sh`, which applies `bash -n` (plus advisory `shellcheck -S error`) to every changed fenced bash block in `commands/**/*.md`. Pre-existing blocks are never checked. Mark an intentional fragment with `<!-- allowlist:check-spec-bash -->` on the line before its fence.
|
|
11
|
+
|
|
12
|
+
Action: expect verification commands that you configured in `forge.yaml` to start running in `/work-on` builds. If a configured command (for example a SOPS chain with `deploy.secrets_backend: sops`) was failing unnoticed, builds will now report it.
|
|
13
|
+
|
|
5
14
|
## Review-finding cascade is now bounded by code, not prose
|
|
6
15
|
|
|
7
16
|
An audit of one `/orchestrate` batch found 93 `review-finding` issues filed for 46 merges, a ratio of 2.02. The §6B.5 note-disposition step was supposed to stop LOW/POSSIBLE findings from becoming issues. It was bypassed in two ways: every finding from the always-on General Security & Quality reviewer was exempt by origin, and reviewers often skipped the step entirely.
|
package/package.json
CHANGED
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios
|
|
3
|
+
# SPDX-License-Identifier: AGPL-3.0-or-later
|
|
4
|
+
#
|
|
5
|
+
# check-spec-bash.sh — Syntax-check the ```bash fenced blocks inside Markdown spec files.
|
|
6
|
+
#
|
|
7
|
+
# ForgeDock's pipeline is bash embedded in Markdown, and the quality gate's executable
|
|
8
|
+
# checks skip .md files. Spec bash therefore merged without ever being parsed, and
|
|
9
|
+
# reviewers became the first thing to run it (2026-10-08 audit). This script runs
|
|
10
|
+
# `bash -n` on each fenced block after neutralising spec notation, and optionally
|
|
11
|
+
# `shellcheck -S error`.
|
|
12
|
+
#
|
|
13
|
+
# Usage:
|
|
14
|
+
# check-spec-bash.sh [--base <git-ref>] [--shellcheck] <file.md>...
|
|
15
|
+
#
|
|
16
|
+
# --base <ref> Only check blocks that contain a line changed relative to <ref>
|
|
17
|
+
# (`git diff -U0 <ref> -- <file>`), so pre-existing blocks never fail
|
|
18
|
+
# a new change. Without --base every block is checked.
|
|
19
|
+
# --shellcheck Also run `shellcheck -S error -s bash` when installed. Advisory: its hits are
|
|
20
|
+
# reported as WARN and never fail the run (spec blocks are often fragments).
|
|
21
|
+
#
|
|
22
|
+
# Spec notation that is not bash is neutralised before parsing:
|
|
23
|
+
# {PLACEHOLDER} / {FORGE_SKILL_PREFIX}x → a plain word
|
|
24
|
+
# blocks containing pseudo tool calls (Skill( / Agent( / Task( / Read( ) → SKIPPED
|
|
25
|
+
# a block whose preceding non-blank line is exactly `<!-- allowlist:check-spec-bash -->` → SKIPPED
|
|
26
|
+
# (for intentional fragments, e.g. an `if` opened in one block and closed in the next)
|
|
27
|
+
#
|
|
28
|
+
# Output: one line per checked block, `OK|FAIL|WARN|SKIP <file>:<start-line> <reason>`,
|
|
29
|
+
# then `SPEC-BASH: checked=N failed=F skipped=S shellcheck_warnings=W`. Only bash -n fails a run.
|
|
30
|
+
# Exit codes: 0 no failures, 1 at least one block failed, 2 usage error.
|
|
31
|
+
#
|
|
32
|
+
# Portable: bash 3.2, BSD/GNU awk and sed. Requires git only with --base.
|
|
33
|
+
|
|
34
|
+
set -u
|
|
35
|
+
|
|
36
|
+
BASE=""; SHELLCHECK=0; FILES=""
|
|
37
|
+
usage() { echo "ERROR: Usage: check-spec-bash.sh [--base <git-ref>] [--shellcheck] <file.md>..." >&2; exit 2; }
|
|
38
|
+
|
|
39
|
+
while [ "$#" -gt 0 ]; do
|
|
40
|
+
case "$1" in
|
|
41
|
+
--base) [ "$#" -ge 2 ] || usage; BASE="$2"; shift 2 ;;
|
|
42
|
+
--shellcheck) SHELLCHECK=1; shift ;;
|
|
43
|
+
--*) usage ;;
|
|
44
|
+
*) FILES="$FILES
|
|
45
|
+
$1"; shift ;;
|
|
46
|
+
esac
|
|
47
|
+
done
|
|
48
|
+
[ -n "$FILES" ] || usage
|
|
49
|
+
|
|
50
|
+
TMPD=$(mktemp -d "${TMPDIR:-/tmp}/check-spec-bash.XXXXXX") || exit 2
|
|
51
|
+
trap 'rm -rf "$TMPD"' EXIT
|
|
52
|
+
|
|
53
|
+
checked=0; failed=0; skipped=0; warned=0
|
|
54
|
+
|
|
55
|
+
changed_lines() { # changed_lines <file> → one new-side line number per line
|
|
56
|
+
git diff -U0 "$BASE" -- "$1" 2>/dev/null | awk '
|
|
57
|
+
/^@@/ { split($3, a, ","); s = substr(a[1], 2) + 0; n = (a[2] == "") ? 1 : a[2] + 0
|
|
58
|
+
for (i = 0; i < n; i++) print s + i }'
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
while IFS= read -r f; do
|
|
62
|
+
[ -n "$f" ] || continue
|
|
63
|
+
case "$f" in *.md) ;; *) continue ;; esac
|
|
64
|
+
[ -f "$f" ] || { echo "SKIP $f missing"; continue; }
|
|
65
|
+
|
|
66
|
+
CH="$TMPD/changed"; : > "$CH"
|
|
67
|
+
if [ -n "$BASE" ]; then
|
|
68
|
+
changed_lines "$f" > "$CH"
|
|
69
|
+
[ -s "$CH" ] || continue
|
|
70
|
+
fi
|
|
71
|
+
|
|
72
|
+
# Split the file into one file per ```bash block, named by its opening line number.
|
|
73
|
+
rm -f "$TMPD"/blk.*
|
|
74
|
+
awk -v d="$TMPD" '
|
|
75
|
+
!inb && /^[ \t]*```(bash|sh)[ \t]*$/ { inb = 1; start = NR; out = d "/blk." NR; printf "" > out
|
|
76
|
+
if (prev ~ /^[ \t]*<!--[ \t]*allowlist:check-spec-bash[ \t]*-->[ \t]*$/) print "#ALLOWLISTED" > out
|
|
77
|
+
next }
|
|
78
|
+
inb && /^[ \t]*```[ \t]*$/ { inb = 0; close(out); print start "\t" NR > (d "/ranges"); next }
|
|
79
|
+
inb { print > out; next }
|
|
80
|
+
{ if ($0 !~ /^[ \t]*$/) prev = $0 }
|
|
81
|
+
' "$f"
|
|
82
|
+
[ -f "$TMPD/ranges" ] || continue
|
|
83
|
+
|
|
84
|
+
while IFS="$(printf '\t')" read -r start end; do
|
|
85
|
+
blk="$TMPD/blk.$start"
|
|
86
|
+
if [ -n "$BASE" ]; then
|
|
87
|
+
hit=$(awk -v s="$start" -v e="$end" '$1 >= s && $1 <= e { print "y"; exit }' "$CH")
|
|
88
|
+
[ "$hit" = "y" ] || continue
|
|
89
|
+
fi
|
|
90
|
+
if head -1 "$blk" | grep -q '^#ALLOWLISTED$'; then
|
|
91
|
+
skipped=$((skipped + 1)); echo "SKIP $f:$start allowlisted fragment"; continue
|
|
92
|
+
fi
|
|
93
|
+
if grep -Eq '(^|[^A-Za-z_])(Skill|Agent|Task|Read)\(' "$blk"; then
|
|
94
|
+
skipped=$((skipped + 1)); echo "SKIP $f:$start pseudo tool call"; continue
|
|
95
|
+
fi
|
|
96
|
+
# Neutralise {PLACEHOLDER} notation (and {FORGE_SKILL_PREFIX}name) into a variable expansion,
|
|
97
|
+
# so tests like [ -z "{X}" ] are not misread as constant strings.
|
|
98
|
+
sed -E 's/\{[A-Z][A-Z0-9_]*\}/${PH}/g' "$blk" > "$blk.sh"
|
|
99
|
+
checked=$((checked + 1))
|
|
100
|
+
if ! err=$(bash -n "$blk.sh" 2>&1); then
|
|
101
|
+
failed=$((failed + 1))
|
|
102
|
+
echo "FAIL $f:$start bash -n: $(printf '%s' "$err" | head -1 | sed "s|$blk.sh|block|")"
|
|
103
|
+
continue
|
|
104
|
+
fi
|
|
105
|
+
if [ "$SHELLCHECK" = 1 ] && command -v shellcheck >/dev/null 2>&1; then
|
|
106
|
+
if ! err=$(shellcheck -S error -s bash "$blk.sh" 2>&1); then
|
|
107
|
+
warned=$((warned + 1))
|
|
108
|
+
echo "WARN $f:$start shellcheck: $(printf '%s' "$err" | grep -m1 -E 'SC[0-9]+' || printf '%s' "$err" | head -1)"
|
|
109
|
+
continue
|
|
110
|
+
fi
|
|
111
|
+
fi
|
|
112
|
+
echo "OK $f:$start"
|
|
113
|
+
done < "$TMPD/ranges"
|
|
114
|
+
rm -f "$TMPD/ranges"
|
|
115
|
+
done <<EOF
|
|
116
|
+
$FILES
|
|
117
|
+
EOF
|
|
118
|
+
|
|
119
|
+
echo "SPEC-BASH: checked=$checked failed=$failed skipped=$skipped shellcheck_warnings=$warned"
|
|
120
|
+
[ "$failed" -eq 0 ]
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios
|
|
3
|
+
# SPDX-License-Identifier: AGPL-3.0-or-later
|
|
4
|
+
#
|
|
5
|
+
# Fixtures for scripts/check-spec-bash.sh. Run: bash scripts/check-spec-bash.test.sh (bash 3.2 compatible)
|
|
6
|
+
|
|
7
|
+
set -u
|
|
8
|
+
DIR=$(cd "$(dirname "$0")" && pwd)
|
|
9
|
+
S="$DIR/check-spec-bash.sh"
|
|
10
|
+
T=$(mktemp -d "${TMPDIR:-/tmp}/check-spec-bash-test.XXXXXX")
|
|
11
|
+
trap 'rm -rf "$T"' EXIT
|
|
12
|
+
pass=0; fail=0
|
|
13
|
+
FENCE='```'
|
|
14
|
+
|
|
15
|
+
check() { # check <want_rc> <want_substring> <desc> -- args...
|
|
16
|
+
want_rc="$1"; want="$2"; desc="$3"; shift 4
|
|
17
|
+
out=$(cd "$T" && bash "$S" "$@" 2>&1); rc=$?
|
|
18
|
+
case "$out" in *"$want"*) ok=1 ;; *) ok=0 ;; esac
|
|
19
|
+
if [ "$rc" = "$want_rc" ] && [ "$ok" = 1 ]; then pass=$((pass+1)); else fail=$((fail+1)); echo "FAIL: $desc (rc=$rc)"; printf '%s\n' "$out" | sed 's/^/ /'; fi
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
cat > "$T/good.md" <<EOF
|
|
23
|
+
# Good
|
|
24
|
+
${FENCE}bash
|
|
25
|
+
if [ -z "{GH_REPO}" ]; then echo none; fi
|
|
26
|
+
gh issue view {NUMBER} -R {GH_REPO} --json labels
|
|
27
|
+
${FENCE}
|
|
28
|
+
Prose with it's apostrophe and an unbalanced ( paren.
|
|
29
|
+
EOF
|
|
30
|
+
|
|
31
|
+
cat > "$T/bad.md" <<EOF
|
|
32
|
+
# Bad
|
|
33
|
+
${FENCE}bash
|
|
34
|
+
if [ -n "\$X" ]; then
|
|
35
|
+
echo "unterminated
|
|
36
|
+
fi
|
|
37
|
+
${FENCE}
|
|
38
|
+
EOF
|
|
39
|
+
|
|
40
|
+
cat > "$T/pseudo.md" <<EOF
|
|
41
|
+
${FENCE}bash
|
|
42
|
+
OUT=\$(Skill(skill="x", args="y"))
|
|
43
|
+
${FENCE}
|
|
44
|
+
EOF
|
|
45
|
+
|
|
46
|
+
cat > "$T/allow.md" <<EOF
|
|
47
|
+
<!-- allowlist:check-spec-bash -->
|
|
48
|
+
${FENCE}bash
|
|
49
|
+
if true; then
|
|
50
|
+
${FENCE}
|
|
51
|
+
EOF
|
|
52
|
+
|
|
53
|
+
cat > "$T/prose-mention.md" <<EOF
|
|
54
|
+
Use \`<!-- allowlist:check-spec-bash -->\` before an intentional fragment.
|
|
55
|
+
${FENCE}bash
|
|
56
|
+
if true; then
|
|
57
|
+
${FENCE}
|
|
58
|
+
EOF
|
|
59
|
+
|
|
60
|
+
cat > "$T/notbash.md" <<EOF
|
|
61
|
+
${FENCE}yaml
|
|
62
|
+
key: [unclosed
|
|
63
|
+
${FENCE}
|
|
64
|
+
EOF
|
|
65
|
+
|
|
66
|
+
check 0 "OK good.md:2" "valid block with placeholders passes" -- good.md
|
|
67
|
+
check 1 "FAIL bad.md:2 bash -n" "unterminated quote fails" -- bad.md
|
|
68
|
+
check 0 "SKIP pseudo.md:1 pseudo tool" "pseudo tool call skipped" -- pseudo.md
|
|
69
|
+
check 0 "SKIP allow.md:2 allowlisted" "allowlisted fragment skipped" -- allow.md
|
|
70
|
+
check 0 "checked=0 failed=0" "non-bash fence ignored" -- notbash.md
|
|
71
|
+
check 1 "FAIL prose-mention.md:2" "prose mention of the marker does not allowlist" -- prose-mention.md
|
|
72
|
+
check 1 "checked=2 failed=1 skipped=1" "multi-file summary" -- good.md bad.md pseudo.md
|
|
73
|
+
check 2 "" "no files is a usage error" --
|
|
74
|
+
check 2 "" "unknown flag" -- --bogus good.md
|
|
75
|
+
|
|
76
|
+
# --base: only blocks touching changed lines are checked, so pre-existing breakage never fails a change.
|
|
77
|
+
( cd "$T" && git init -q . && git -c user.email=t@t -c user.name=t add bad.md good.md && git -c user.email=t@t -c user.name=t commit -q -m base )
|
|
78
|
+
check 0 "checked=0 failed=0" "--base with no change checks nothing" -- --base HEAD bad.md good.md
|
|
79
|
+
printf '\nnew prose line\n' >> "$T/bad.md"
|
|
80
|
+
check 0 "checked=0 failed=0" "--base: prose change outside a broken block passes" -- --base HEAD bad.md
|
|
81
|
+
cat >> "$T/good.md" <<EOF
|
|
82
|
+
${FENCE}bash
|
|
83
|
+
for x in a b; do
|
|
84
|
+
echo "\$x"
|
|
85
|
+
${FENCE}
|
|
86
|
+
EOF
|
|
87
|
+
check 1 "FAIL good.md:" "--base: newly added broken block fails" -- --base HEAD good.md
|
|
88
|
+
|
|
89
|
+
echo "check-spec-bash.test.sh: passed=$pass failed=$fail"
|
|
90
|
+
[ "$fail" -eq 0 ]
|
|
@@ -29,7 +29,7 @@
|
|
|
29
29
|
# 1. HIGH/CRITICAL, or unparseable severity → ISSUE.
|
|
30
30
|
# 2. Safety exemption is CONTENT-based: the text matches the security/billing
|
|
31
31
|
# keyword set, or the finding came from a dedicated, signal-selected domain
|
|
32
|
-
# agent (Auth, Billing, Concurrency, Database). Origin from the always-on
|
|
32
|
+
# agent (Auth, Billing, Concurrency, Database) and is MEDIUM+ or CONFIRMED. Origin from the always-on
|
|
33
33
|
# "General Security & Quality" agent alone NO LONGER exempts a finding —
|
|
34
34
|
# that agent runs on every PR, so origin-based exemption filed nearly every
|
|
35
35
|
# LOW note it raised (50 of 60 would-be notes in the audited batch).
|
|
@@ -93,9 +93,12 @@ KW='security|auth|authz|authn|billing|payment|stripe|charge|invoice|injection|xs
|
|
|
93
93
|
if printf '%s' "$TEXT" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9\n' ' ' | grep -Eqw "($KW)"; then
|
|
94
94
|
SAFETY="keyword"
|
|
95
95
|
fi
|
|
96
|
+
# A dedicated domain agent rescues a finding only when it is MEDIUM+ or CONFIRMED:
|
|
97
|
+
# a LOW/POSSIBLE finding stays a NOTE whichever reviewer raised it.
|
|
96
98
|
AGENT_LC=$(printf '%s' "$AGENT" | tr '[:upper:]' '[:lower:]')
|
|
97
99
|
case "$AGENT_LC" in
|
|
98
|
-
auth*|billing*|concurrency*|database*)
|
|
100
|
+
auth*|billing*|concurrency*|database*)
|
|
101
|
+
if [ "$SEV" = "MEDIUM" ] || [ "$CONF" = "CONFIRMED" ]; then SAFETY="${SAFETY:-domain-agent}"; fi ;;
|
|
99
102
|
esac
|
|
100
103
|
|
|
101
104
|
if [ "$LINEAGE" = "review-finding" ]; then
|
|
@@ -44,10 +44,13 @@ expect ISSUE "LOW with Secrets (case)" -- --severity LOW --confidence LIK
|
|
|
44
44
|
expect ISSUE "LOW with permission-check" -- --severity LOW --confidence LIKELY --text "permission-check skipped"
|
|
45
45
|
expect NOTE "'author' is not 'auth'" -- --severity LOW --confidence LIKELY --text "author field unused"
|
|
46
46
|
expect NOTE "'tokenizer' is not 'token'" -- --severity LOW --confidence LIKELY --text "tokenizer splits oddly"
|
|
47
|
-
expect ISSUE "LOW from Auth domain agent"
|
|
48
|
-
expect ISSUE "
|
|
49
|
-
expect
|
|
50
|
-
expect
|
|
47
|
+
expect ISSUE "LOW CONFIRMED from Auth domain agent" -- --severity LOW --confidence CONFIRMED --agent "Auth" --text "nit"
|
|
48
|
+
expect ISSUE "MEDIUM POSSIBLE from Billing agent" -- --severity MEDIUM --confidence POSSIBLE --agent "billing" --text "nit"
|
|
49
|
+
expect NOTE "LOW LIKELY from Auth domain agent is a NOTE" -- --severity LOW --confidence LIKELY --agent "Auth" --text "nit"
|
|
50
|
+
expect NOTE "LOW POSSIBLE from Database agent is a NOTE" -- --severity LOW --confidence POSSIBLE --agent "Database" --text "nit"
|
|
51
|
+
expect NOTE "LOW POSSIBLE Concurrency reviewer is a NOTE (AlterLab #34671)" -- --severity LOW --confidence POSSIBLE --agent "Concurrency reviewer" --text "lock edge cases"
|
|
52
|
+
expect ISSUE "LOW CONFIRMED Concurrency reviewer files" -- --severity LOW --confidence CONFIRMED --agent "Concurrency reviewer" --text "lock edge cases"
|
|
53
|
+
expect ISSUE "keyword still rescues LOW POSSIBLE from domain agent" -- --severity LOW --confidence POSSIBLE --agent "Database" --text "sql injection"
|
|
51
54
|
|
|
52
55
|
# --- Rule 3: review-finding lineage (fix for a finding) ---
|
|
53
56
|
expect NOTE "lineage LOW CONFIRMED note" -- --severity LOW --confidence CONFIRMED --lineage review-finding --text "nit"
|