forgedock 1.0.22 → 1.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +255 -211
- package/bin/batch-runner.mjs +339 -0
- package/bin/cinema.mjs +320 -0
- package/bin/demo.mjs +92 -7
- package/bin/engine/phases.mjs +168 -0
- package/bin/engine/projector.mjs +32 -0
- package/bin/engine/reconcile.mjs +11 -0
- package/bin/engine/runlog.mjs +79 -0
- package/bin/engine/state.mjs +32 -0
- package/bin/engine-cli.mjs +222 -0
- package/bin/engine.mjs +138 -0
- package/bin/forgedock.mjs +896 -1540
- package/bin/hooks/interactive-engine.mjs +509 -0
- package/bin/hooks/pre-tool-use.mjs +498 -0
- package/bin/hooks/subagent-stop-enforce.mjs +197 -0
- package/bin/journey.mjs +841 -0
- package/bin/registry.mjs +139 -0
- package/bin/report.mjs +447 -0
- package/bin/runner.mjs +136 -20
- package/bin/settings-hook.mjs +346 -0
- package/bin/tui.mjs +6 -3
- package/commands/adopt.md +4 -1
- package/commands/analytics.md +4 -1
- package/commands/audit-agents.md +110 -27
- package/commands/audit.md +20 -1
- package/commands/autopilot.md +84 -12
- package/commands/changelog.md +2 -1
- package/commands/ci-audit.md +2 -1
- package/commands/cleanup.md +161 -1
- package/commands/compat-audit.md +2 -1
- package/commands/deploy-info.md +63 -4
- package/commands/design-bench.md +6 -1
- package/commands/design-render-critique-loop.md +2 -1
- package/commands/design.md +1 -0
- package/commands/diagnose.md +2 -1
- package/commands/explain.md +2 -1
- package/commands/forge-stats.md +122 -2
- package/commands/forgedock-init.md +1 -1
- package/commands/geo-audit.md +3 -0
- package/commands/incident-response.md +2 -1
- package/commands/issue.md +24 -3
- package/commands/milestone.md +77 -12
- package/commands/optimize.md +2 -1
- package/commands/orchestrate/config.md +67 -0
- package/commands/orchestrate/phase-1-resolve.md +159 -0
- package/commands/orchestrate/phase-2-triage.md +126 -0
- package/commands/orchestrate/phase-2.5-synthesis.md +156 -0
- package/commands/orchestrate/phase-3-dependency.md +591 -0
- package/commands/orchestrate/phase-4-execution.md +962 -0
- package/commands/orchestrate/phase-5-cleanup.md +47 -0
- package/commands/orchestrate/phase-6-report.md +198 -0
- package/commands/orchestrate/safety.md +62 -0
- package/commands/orchestrate.md +25 -2027
- package/commands/pipeline-health/config.md +34 -0
- package/commands/pipeline-health/phase-1-context.md +180 -0
- package/commands/pipeline-health/phase-2-metrics.md +1729 -0
- package/commands/pipeline-health/phase-3-analyze.md +204 -0
- package/commands/pipeline-health/phase-4-proposals.md +117 -0
- package/commands/pipeline-health/phase-5-report.md +284 -0
- package/commands/pipeline-health/phase-6-summary.md +52 -0
- package/commands/pipeline-health.md +25 -2401
- package/commands/pipeline-resume.md +24 -1
- package/commands/pipeline-status.md +65 -0
- package/commands/qa-sweep.md +4 -1
- package/commands/quality-gate.md +326 -42
- package/commands/replay.md +2 -1
- package/commands/review-pr-agents/api.md +135 -0
- package/commands/review-pr-agents/auth.md +137 -0
- package/commands/review-pr-agents/billing.md +101 -0
- package/commands/review-pr-agents/concurrency.md +145 -0
- package/commands/review-pr-agents/database.md +103 -0
- package/commands/review-pr-agents/frontend.md +89 -0
- package/commands/review-pr-agents/infra.md +385 -0
- package/commands/review-pr-agents/protocols.md +185 -0
- package/commands/review-pr-agents/scraper.md +185 -0
- package/commands/review-pr-agents/security.md +406 -0
- package/commands/review-pr-agents.md +45 -1722
- package/commands/review-pr-staging.md +196 -40
- package/commands/review-pr.md +612 -56
- package/commands/rollback.md +2 -1
- package/commands/scope.md +2 -1
- package/commands/security-audit.md +2 -1
- package/commands/signal-planner.md +623 -0
- package/commands/sync-ecosystem.md +1 -0
- package/commands/test-gate.md +17 -1
- package/commands/validate.md +2 -1
- package/commands/work-on/build/architect.md +28 -0
- package/commands/work-on/build/context.md +2 -0
- package/commands/work-on/build/implement.md +19 -5
- package/commands/work-on/build/validate.md +62 -6
- package/commands/work-on/build.md +187 -30
- package/commands/work-on/close.md +132 -5
- package/commands/work-on/decompose.md +19 -1
- package/commands/work-on/investigate.md +117 -2
- package/commands/work-on/review.md +48 -4
- package/commands/work-on-monolithic.md +8 -2
- package/commands/work-on.md +247 -7
- package/docs/BRAND.md +16 -5
- package/docs/CI.md +2 -2
- package/docs/CODEX.md +36 -1
- package/docs/CONFIG.md +44 -42
- package/docs/FORGE-PROTOCOL.md +15 -1064
- package/docs/anthropic-devrel-outreach.md +2 -2
- package/docs/articles/command-decomposition-token-savings.md +214 -0
- package/docs/articles/model-release-playbook.md +131 -0
- package/docs/articles/per-repo-adaptive-scripts-token-savings.md +27 -25
- package/docs/claude-breakpoints.json +75 -5
- package/docs/design/abc-benchmark.md +1 -1
- package/docs/design/design-spec-schema.md +1 -1
- package/docs/design/divergent-generation.md +1 -1
- package/docs/design/eval-gate-runbook.md +178 -0
- package/docs/design/show-hn-draft.md +26 -1
- package/docs/design/token-cost-baseline-1298.md +59 -0
- package/docs/eval/README.md +117 -0
- package/docs/eval/corpus/seed-v1.json +6 -0
- package/docs/eval/published/claude-sonnet-4-6-2026-07-04.json +33 -0
- package/docs/site/annotations-explained.md +24 -0
- package/docs/site/command-learning-path.md +51 -14
- package/docs/site/command-reference.md +339 -113
- package/docs/site/for-companies.md +95 -0
- package/docs/site/forge-annotation-protocol.md +12 -256
- package/docs/site/getting-started.md +58 -12
- package/docs/site/github-is-the-memory.md +113 -0
- package/docs/site/how-it-works.md +3 -13
- package/docs/site/index.md +10 -2
- package/docs/spec/eval-run-result.md +147 -0
- package/docs/spec/forge-protocol-v1.md +441 -18
- package/docs/spec/label-state-machine.md +55 -0
- package/docs/spec/review-protocol.md +158 -0
- package/docs/spec-graph-schema.md +28 -0
- package/docs/superpowers/plans/2026-07-03-cinematic-onboarding-tui.md +2339 -0
- package/docs/superpowers/plans/2026-07-03-durable-execution-engine.md +1223 -0
- package/docs/superpowers/specs/2026-07-03-cinematic-onboarding-tui-design.md +157 -0
- package/docs/superpowers/specs/2026-07-03-durable-execution-engine-design.md +194 -0
- package/docs/superpowers/specs/2026-07-04-economic-self-governance-design.md +181 -0
- package/docs/superpowers/specs/2026-07-04-provenance-trail-design.md +189 -0
- package/examples/forgedock-demo/README.md +41 -22
- package/examples/forgedock-demo/issues/06-bug-tags-type-validation.md +43 -0
- package/examples/forgedock-demo/issues/07-bug-negative-offset-pagination.md +42 -0
- package/examples/forgedock-demo/issues/08-bug-patch-mass-assignment.md +49 -0
- package/examples/forgedock-demo/issues/09-bug-count-injection.md +47 -0
- package/examples/forgedock-demo/issues/10-feature-sort-param.md +39 -0
- package/examples/forgedock-demo/issues/11-feature-patch-tags.md +37 -0
- package/examples/forgedock-demo/issues/12-feature-bulk-create.md +42 -0
- package/examples/forgedock-demo/issues/13-feature-archive-notes.md +46 -0
- package/examples/forgedock-demo/issues/14-refactor-extract-title-validation.md +41 -0
- package/examples/forgedock-demo/issues/15-refactor-protected-route-flag.md +43 -0
- package/examples/forgedock-demo/issues/16-refactor-extract-store.md +45 -0
- package/examples/forgedock-demo/issues/17-perf-tags-dedup-set.md +35 -0
- package/examples/forgedock-demo/issues/18-perf-remove-redundant-sort.md +37 -0
- package/examples/forgedock-demo/issues/19-docs-api-reference-full.md +53 -0
- package/examples/forgedock-demo/issues/20-docs-intentional-flaw-convention.md +46 -0
- package/examples/forgedock-demo/scripts/smoke.js +18 -1
- package/examples/forgedock-demo/src/db.js +36 -11
- package/examples/forgedock-demo/src/routes/notes.js +106 -9
- package/examples/forgedock-demo/src/server.js +4 -1
- package/package.json +12 -4
- package/packages/protocol/LICENSE +21 -0
- package/packages/protocol/src/cli.js +135 -0
- package/packages/protocol/src/emit.js +146 -0
- package/packages/protocol/src/index.js +43 -0
- package/packages/protocol/src/parse.js +229 -0
- package/packages/protocol/src/types.js +171 -0
- package/packages/protocol/src/validate.js +122 -0
- package/scripts/bench-topology-cost.mjs +372 -0
- package/scripts/check-command-docs-drift.sh +198 -0
- package/scripts/check-protocol-restatements.sh +176 -0
- package/scripts/check-registry/README.md +72 -0
- package/scripts/check-registry/manifest.json +6 -0
- package/scripts/code-index.sh +454 -0
- package/scripts/conformance-check.mjs +168 -0
- package/scripts/doctor-pipeline-state.sh +484 -0
- package/scripts/eval-gate-baseline.json +17 -0
- package/scripts/eval-gate-scorecard.mjs +242 -0
- package/scripts/eval-scorecard-aggregator.mjs +252 -0
- package/scripts/flaky-quarantine.sh +241 -0
- package/scripts/forge-annotation.sh +567 -0
- package/scripts/issue-dedup.sh +236 -0
- package/scripts/validate-annotation-node.mjs +233 -0
- package/scripts/verify-env-vars.sh +18 -2
- package/scripts/verify-host-headers.sh +4 -4
- package/scripts/verify-route-registration.sh +2 -2
- package/scripts/worktree-lifecycle.sh +202 -0
- package/templates/devdocs/agent/using-forgedock.md +14 -0
|
@@ -1,140 +1,49 @@
|
|
|
1
1
|
<!-- SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios -->
|
|
2
2
|
<!-- SPDX-License-Identifier: AGPL-3.0-or-later -->
|
|
3
3
|
|
|
4
|
-
# Agent Catalog for `/review-pr`
|
|
4
|
+
# Agent Catalog Router for `/review-pr`
|
|
5
5
|
|
|
6
|
-
This file is the
|
|
6
|
+
This file is the routing index referenced by the `/review-pr` orchestrator during Phase 3C (agent dispatch).
|
|
7
|
+
Per-persona prompt templates have been split into individual files under `commands/review-pr-agents/`
|
|
8
|
+
to avoid loading the full catalog on every invocation. Protocols live in `docs/spec/review-protocol.md`
|
|
9
|
+
(canonical source) and are reproduced in `commands/review-pr-agents/protocols.md`.
|
|
7
10
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
3. All **9 agent templates** — copy the relevant template verbatim when constructing agent prompts
|
|
11
|
+
Do not modify this file without also updating `review-pr.md`. To update either protocol, edit
|
|
12
|
+
`docs/spec/review-protocol.md` and sync the copy to `commands/review-pr-agents/protocols.md`.
|
|
11
13
|
|
|
12
|
-
|
|
14
|
+
**NEVER use the Agent tool.** The orchestrator dispatches review agents via `Task` only. `Agent` spawns
|
|
15
|
+
opaque subprocesses that bypass the `allowed-tools` constraint, cannot post structured findings to the PR,
|
|
16
|
+
and are not tracked by the review pipeline. Every agent template is intended to be passed as the `prompt`
|
|
17
|
+
argument to a `Task(...)` call.
|
|
13
18
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
## Evidence-Based Review Protocol (ALL Agents Follow)
|
|
17
|
-
|
|
18
|
-
Every agent MUST follow this protocol:
|
|
19
|
-
|
|
20
|
-
### 1. Start From the PR Diff
|
|
21
|
-
```bash
|
|
22
|
-
# Verify review is still current before reading diff
|
|
23
|
-
CURRENT_SHA=$(gh pr view [PR_NUMBER] --json headRefOid --jq '.headRefOid')
|
|
24
|
-
if [ "$CURRENT_SHA" != "[REVIEW_SHA]" ]; then
|
|
25
|
-
echo "WARNING: PR HEAD changed during review. Review may be stale."
|
|
26
|
-
echo "Review pinned to: [REVIEW_SHA_SHORT]"
|
|
27
|
-
echo "Current HEAD: $(echo $CURRENT_SHA | cut -c1-7)"
|
|
28
|
-
fi
|
|
29
|
-
|
|
30
|
-
gh pr diff [PR_NUMBER] --name-only
|
|
31
|
-
gh pr diff [PR_NUMBER]
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
**Hot-spot prior**:
|
|
35
|
-
[CHURN_CONTEXT]
|
|
36
|
-
|
|
37
|
-
If a file you are reviewing is listed above as a hot-spot, apply deeper scrutiny to it — high historical churn correlates with defect density. Prefer tracing that file's full code paths (Evidence-Based Review Protocol §2) over a quick pattern scan, and weight ambiguous findings in hot-spot files toward LIKELY rather than POSSIBLE.
|
|
38
|
-
|
|
39
|
-
### 2. Dynamic Exploration
|
|
40
|
-
- From each changed file, follow imports and function calls
|
|
41
|
-
- Trace data flows across service boundaries (API → Redis → Worker)
|
|
42
|
-
- Search for related code: `grep -rn "function_name" services/`
|
|
43
|
-
|
|
44
|
-
### 3. Validation Before Reporting
|
|
45
|
-
|
|
46
|
-
| Confidence | Criteria | Action |
|
|
47
|
-
|------------|----------|--------|
|
|
48
|
-
| **CONFIRMED** | Traced the full code path, found specific lines proving the bug | Report as blocking — P1 issue |
|
|
49
|
-
| **LIKELY** | Code pattern suggests issue but mitigations might exist elsewhere | Report with caveat — P2 issue |
|
|
50
|
-
| **POSSIBLE** | Suspicious pattern but couldn't trace the full flow | Report as informational — P3 advisory (non-blocking) |
|
|
51
|
-
| **UNFOUNDED** | Looked for the issue but found mitigations/correct handling | Do NOT report |
|
|
52
|
-
|
|
53
|
-
### 3.5 REPRODUCTION GATE — Required Before CONFIRMED Classification
|
|
54
|
-
|
|
55
|
-
**MANDATORY**: Before classifying any finding as CONFIRMED, you MUST document one of the following forms of reproduction evidence in your report. A pattern match alone is not sufficient.
|
|
56
|
-
|
|
57
|
-
**Acceptable reproduction evidence (one of)**:
|
|
58
|
-
- **(a) Full code path trace**: List the execution chain from PR-changed code to the failure point. Minimum: 3 steps with specific file + line for each. Example: `services/api/app/routers/billing.py:142 → credits.py:check_balance():87 → returns None → caller at billing.py:148 raises AttributeError`. The chain must terminate at the actual failure — not at "and then it could fail."
|
|
59
|
-
- **(b) Specific input demonstration**: Provide concrete input values that trigger the failure. Example: `POST /api/v1/scrape with {"url": "http://internal:6432/"}` → `requests.get()` hits internal DB port → SSRF confirmed. The values must be specific (not "if an attacker provides a malicious URL") and must map to actual code in the PR diff.
|
|
60
|
-
|
|
61
|
-
**Downgrade rule**: If you cannot produce either (a) or (b) after a reasonable trace attempt, you MUST classify the finding as **POSSIBLE** — not CONFIRMED or LIKELY. Do NOT use CONFIRMED when the finding is based on:
|
|
62
|
-
- A suspicious pattern without tracing whether the condition is reachable via changed code
|
|
63
|
-
- A theoretical exploit path not grounded in specific lines from the diff
|
|
64
|
-
- A heuristic ("this type of code often has X bug") without verification
|
|
65
|
-
|
|
66
|
-
**POSSIBLE findings are informational advisories** — they are logged and tracked but do NOT block merge and do NOT trigger mandatory fix PRs. When in doubt, POSSIBLE is the correct classification. <!-- Added: forge#371 -->
|
|
67
|
-
|
|
68
|
-
### 4. SEVERITY CLASSIFICATION — TRACE THE IMPACT
|
|
69
|
-
|
|
70
|
-
**CRITICAL RULE: Never dismiss a finding as "minor", "cosmetic", or "harmless" without tracing its downstream impact.** If you spot something unusual (redundant code, odd patterns, duplicated values), ask: "Does this cause a runtime error, data corruption, or wrong behavior in any code path that touches it?" Trace forward through every consumer of the construct.
|
|
71
|
-
|
|
72
|
-
**Severity decision tree:**
|
|
73
|
-
1. Will this error at runtime? → **HIGH or CRITICAL** (not "minor redundancy")
|
|
74
|
-
2. Will this produce wrong data silently? → **HIGH**
|
|
75
|
-
3. Will this cause degraded performance? → **MEDIUM**
|
|
76
|
-
4. Is it genuinely cosmetic with no runtime impact after tracing all consumers? → **LOW**
|
|
77
|
-
|
|
78
|
-
If you're unsure whether something is cosmetic or a runtime error, **assume it's a runtime error** and flag it for investigation. A false positive costs a minute of review time. A missed runtime error costs production downtime.
|
|
79
|
-
|
|
80
|
-
### 5. INTERACTION ANALYSIS — "Pre-existing" Is Not "Safe"
|
|
81
|
-
|
|
82
|
-
**CRITICAL RULE: Never dismiss a finding as "pre-existing, not introduced by this PR" without checking whether NEW code in the PR interacts with the pre-existing construct to create a bug.**
|
|
83
|
-
|
|
84
|
-
A redundant import, an unused variable, or a duplicated constant may be harmless in isolation. But new code added in the same scope can turn it into a crash. Example: a local `import os` inside a function is harmless until new code above it calls `os.getenv()` — Python treats `os` as local for the entire function scope, causing `UnboundLocalError` before the import line is reached.
|
|
85
|
-
|
|
86
|
-
**Before dismissing anything as "pre-existing":**
|
|
87
|
-
1. List every NEW line in the PR that references the pre-existing construct
|
|
88
|
-
2. For each reference, ask: "Does the pre-existing construct cause this new line to fail at runtime?"
|
|
89
|
-
3. If yes → CONFIRMED finding, not a dismissal
|
|
90
|
-
|
|
91
|
-
### 6. FALSE POSITIVE PREVENTION
|
|
92
|
-
|
|
93
|
-
**Before claiming variable scope issues:** Read the FULL function, count indentation levels, check if/else structure.
|
|
94
|
-
|
|
95
|
-
**Before claiming type/unit mismatches:** Trace the variable to its source. Check if naming is misleading (e.g., `balanceCents` might hold microcents).
|
|
96
|
-
|
|
97
|
-
**Before claiming missing functions/imports:** `grep -rn "functionName" .` — check re-exports, aliases.
|
|
98
|
-
|
|
99
|
-
**Before claiming unreachable code:** Check all callers, dynamic dispatch, test code.
|
|
100
|
-
|
|
101
|
-
**Before dismissing redundant imports as harmless:** In Python, a local `import X` inside a function makes `X` a local variable for the ENTIRE function scope. Any use of `X` before that import line will raise `UnboundLocalError`. Check whether any code (existing or new) references `X` before the local import. This is a CONFIRMED CRITICAL if found — it crashes at runtime.
|
|
102
|
-
|
|
103
|
-
### 6. Report Format
|
|
104
|
-
|
|
105
|
-
Every finding must include:
|
|
106
|
-
- **File:Line** — Exact location
|
|
107
|
-
- **Code snippet** — The problematic code
|
|
108
|
-
- **Evidence** — Why this is a bug (show the code path)
|
|
109
|
-
- **Confidence** — CONFIRMED/LIKELY/POSSIBLE
|
|
110
|
-
- **What you checked** — List files you read to verify
|
|
19
|
+
<!-- FORGE:PROTOCOL_SOURCE — canonical definition lives in docs/spec/review-protocol.md -->
|
|
111
20
|
|
|
112
21
|
---
|
|
113
22
|
|
|
114
|
-
##
|
|
23
|
+
## How to Load Agents (Phase 3C)
|
|
115
24
|
|
|
116
|
-
|
|
25
|
+
For each agent selected in Phase 3B, read **only** the files relevant to that agent:
|
|
117
26
|
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
`<!-- REVIEW-FINDINGS-START -->`
|
|
123
|
-
`<!-- FINDING:PREFIX-N|CONFIDENCE|SEVERITY|file.py:line|One-line summary -->`
|
|
124
|
-
`<!-- REVIEW-FINDINGS-END -->`
|
|
27
|
+
1. **Always read first** (shared protocols, all agents require this):
|
|
28
|
+
```
|
|
29
|
+
Read: $FORGE_HOME/commands/review-pr-agents/protocols.md
|
|
30
|
+
```
|
|
125
31
|
|
|
126
|
-
|
|
32
|
+
2. **Then read the persona file for each selected agent**:
|
|
127
33
|
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
34
|
+
| Agent | Trigger | File |
|
|
35
|
+
|-------|---------|------|
|
|
36
|
+
| General Security & Quality Scan | ALWAYS RUNS | `review-pr-agents/security.md` |
|
|
37
|
+
| Auth Conventions Auditor | AUTH domain | `review-pr-agents/auth.md` |
|
|
38
|
+
| Billing Integrity Auditor | BILLING domain | `review-pr-agents/billing.md` |
|
|
39
|
+
| Concurrency & Race Condition Auditor | CONCURRENCY or BILLING domain | `review-pr-agents/concurrency.md` |
|
|
40
|
+
| Domain Logic Auditor | SCRAPING domain (requires `review.domains.scraping` in forge.yaml) | `review-pr-agents/scraper.md` |
|
|
41
|
+
| Frontend Quality Auditor | WEB service touched | `review-pr-agents/frontend.md` |
|
|
42
|
+
| API Design & Consistency Auditor | New/modified routers or SDK/OpenAPI files | `review-pr-agents/api.md` |
|
|
43
|
+
| Database & Migration Auditor | DATABASE domain | `review-pr-agents/database.md` |
|
|
44
|
+
| Infrastructure & Deploy Safety Auditor | INFRA service touched | `review-pr-agents/infra.md` |
|
|
136
45
|
|
|
137
|
-
|
|
46
|
+
## Domain Prefixes (for Structured Findings)
|
|
138
47
|
|
|
139
48
|
| Agent | Prefix |
|
|
140
49
|
|-------|--------|
|
|
@@ -142,1614 +51,28 @@ Append this block at the very end of your comment (after the `---` footer line,
|
|
|
142
51
|
| Auth Conventions | `AUTH` |
|
|
143
52
|
| Billing Integrity | `BILL` |
|
|
144
53
|
| Concurrency | `CONC` |
|
|
145
|
-
|
|
|
54
|
+
| Domain Logic | `SCRP` |
|
|
146
55
|
| Frontend Quality | `FE` |
|
|
147
56
|
| API Design | `API` |
|
|
148
57
|
| Database & Migration | `DB` |
|
|
149
58
|
| Infrastructure | `INFRA` |
|
|
150
|
-
| Config Schema | `CFG` |
|
|
151
|
-
|
|
152
|
-
### Example
|
|
153
|
-
|
|
154
|
-
`<!-- REVIEW-FINDINGS-START -->`
|
|
155
|
-
`<!-- FINDING:SEC-1|CONFIRMED|HIGH|services/api/app/routers/scrape.py:45|SQL injection via unsanitized user input in query parameter -->`
|
|
156
|
-
`<!-- FINDING:SEC-2|LIKELY|MEDIUM|services/worker/worker/queues.py:312|Potential SSRF through user-controlled proxy URL -->`
|
|
157
|
-
`<!-- REVIEW-FINDINGS-END -->`
|
|
158
|
-
|
|
159
|
-
---
|
|
160
|
-
|
|
161
|
-
## Agent Catalog
|
|
162
|
-
|
|
163
|
-
### Agent: General Security & Quality Scan (ALWAYS RUNS)
|
|
164
|
-
|
|
165
|
-
**Type**: `security-exploit-auditor` | **Model**: `sonnet`
|
|
166
|
-
|
|
167
|
-
**Prompt template:**
|
|
168
|
-
```
|
|
169
|
-
You are performing a security and code quality scan on PR #[PR_NUMBER] for [PROJECT_NAME].
|
|
170
|
-
|
|
171
|
-
## Context
|
|
172
|
-
- PR title: [TITLE]
|
|
173
|
-
- Services touched: [SERVICES]
|
|
174
|
-
- Files changed: [FILE_LIST]
|
|
175
|
-
[PROJECT_CONTEXT]
|
|
176
|
-
|
|
177
|
-
## Your Mission
|
|
178
|
-
Scan ALL changed code for security vulnerabilities and code quality issues. This is the baseline review that runs on every PR.
|
|
179
|
-
|
|
180
|
-
## Step 1: Read the Diff
|
|
181
|
-
```bash
|
|
182
|
-
gh pr diff [PR_NUMBER] --name-only
|
|
183
|
-
gh pr diff [PR_NUMBER]
|
|
184
|
-
```
|
|
185
|
-
|
|
186
|
-
## Step 2: Security Scan
|
|
187
|
-
For each changed file, check:
|
|
188
|
-
1. **Injection**: SQL, command, template injection via user input. **Specifically**: f-string interpolation in SQL queries is CONFIRMED HIGH — this is the #1 security finding across all reviews. Check every SQL string for `f"..."` or `.format()`.
|
|
189
|
-
2. **SSRF**: User-controlled URLs hitting internal services. Check webhook handlers, URL preview features, any endpoint that fetches a user-provided URL.
|
|
190
|
-
3. **Path traversal**: User input in file paths. Check proxy allowlists — overly broad patterns (e.g., allowing `/v1/` prefix instead of exact paths) enable traversal.
|
|
191
|
-
4. **Unsafe deserialization**: pickle, yaml.load without safe_load
|
|
192
|
-
5. **Hardcoded secrets**: API keys, passwords, tokens in code
|
|
193
|
-
6. **XSS**: User input rendered without sanitization (frontend)
|
|
194
|
-
7. **Token privilege confusion**: Is the correct token used for the operation? Recording/read tokens should NOT be used for write/admin operations. Check every `verify_token` call — should it be `verify_admin_token`?
|
|
195
|
-
8. **Unbounded resource consumption**: Endpoints that accept user input controlling query scope (date ranges, pagination) without limits. An attacker can trigger expensive queries or LLM calls without cost controls.
|
|
196
|
-
9. **Information disclosure via API responses** (conditional — when FastAPI router, model, or main.py files are in the diff): Does the API reveal more than necessary in error responses, health endpoints, and response headers?
|
|
197
|
-
|
|
198
|
-
First, identify which trigger conditions apply:
|
|
199
|
-
```bash
|
|
200
|
-
MODEL_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "\.py$" | xargs grep -lE "class [A-Za-z]+\(BaseModel\)|from pydantic import|@router\.(get|post|put|patch|delete)\(" 2>/dev/null)
|
|
201
|
-
HEALTH_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "(health|router).*\.py$")
|
|
202
|
-
MIDDLEWARE_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "(main|middleware|app)\.py$")
|
|
203
|
-
```
|
|
204
|
-
|
|
205
|
-
**Sub-check 9a: Pydantic validation error handler** (trigger: `$MODEL_FILES` non-empty)
|
|
206
|
-
|
|
207
|
-
FastAPI's default validation error response exposes internal field names, types, and constraints. Attackers can enumerate the complete request schema by sending malformed payloads.
|
|
208
|
-
```bash
|
|
209
|
-
while IFS= read -r f; do
|
|
210
|
-
[ -z "$f" ] && continue
|
|
211
|
-
grep -rn "RequestValidationError\|validation_exception_handler\|@app.exception_handler" services/api/app/main.py services/api/app/ 2>/dev/null | head -5
|
|
212
|
-
done <<< "$MODEL_FILES"
|
|
213
|
-
# If no custom handler found AND the PR adds new Pydantic models or FastAPI endpoints:
|
|
214
|
-
# LIKELY MEDIUM — FastAPI default exposes internal field names/types/constraints to any caller
|
|
215
|
-
# Safe pattern: catch RequestValidationError and return only {"detail": "Invalid request"} or a sanitized message
|
|
216
|
-
```
|
|
217
|
-
**Confidence**: `LIKELY` — a custom handler may exist in a file not in the diff.
|
|
218
|
-
**Severity**: MEDIUM — schema enumeration accelerates fuzzing and targeted injection attempts; not a direct exploit.
|
|
219
|
-
**Evidence**: FastAPI's default validation error response includes the full Pydantic schema field tree, exposing internal field names and model structure to clients. A new endpoint added without a custom exception handler inherits this default, leaking schema details to error-triggering callers.
|
|
220
|
-
|
|
221
|
-
**Sub-check 9b: Public health endpoint data exposure** (trigger: `$HEALTH_FILES` non-empty)
|
|
222
|
-
|
|
223
|
-
Unauthenticated health endpoints are expected to be public — the Auth agent's auth-dependency check does not apply. The relevant question is whether they reveal more than `{"status": "ok"}`.
|
|
224
|
-
```bash
|
|
225
|
-
while IFS= read -r f; do
|
|
226
|
-
[ -z "$f" ] && continue
|
|
227
|
-
# Find unauthenticated endpoints (no Depends() in signature) and check return content
|
|
228
|
-
grep -A25 "@router\.\(get\|head\)\|async def health\|async def ping\|async def live\|async def ready\|async def status" "$f" | \
|
|
229
|
-
grep -iE "version|host|port|redis|postgres|db_url|tesseract|pillow|PIL\.__version__|software|engine|server" && \
|
|
230
|
-
echo "SEC: health endpoint in $f returns operational details. Reveals software versions (enables CVE targeting) or connectivity state (enables DoS confirmation). Safe pattern: return only {\"status\": \"ok\"} or {\"status\": \"degraded\"} — no version strings, no host/port, no connectivity details."
|
|
231
|
-
done <<< "$HEALTH_FILES"
|
|
232
|
-
```
|
|
233
|
-
**Confidence**: `LIKELY` — version strings may be intentionally scoped to internal-only endpoints; verify auth status of the endpoint.
|
|
234
|
-
**Severity**: MEDIUM for software version strings (direct CVE targeting input); LOW for DB/Redis connectivity state (useful for DoS confirmation, not exploit).
|
|
235
|
-
**Evidence**: Health endpoints are often added alongside new services or integrations and return detailed status data by default. Without an explicit check, reviewers tend to verify that the endpoint works — not that it reveals too much. Software version strings in health responses enable CVE targeting; connectivity state enables DoS confirmation.
|
|
236
|
-
|
|
237
|
-
**Sub-check 9c: Response header disclosure** (trigger: `$MIDDLEWARE_FILES` non-empty)
|
|
238
|
-
|
|
239
|
-
Git SHAs and deployment topology exposed via response headers enable targeted CVE research and attack planning.
|
|
240
|
-
```bash
|
|
241
|
-
while IFS= read -r f; do
|
|
242
|
-
[ -z "$f" ] && continue
|
|
243
|
-
grep -nE "X-App-Version|X-Deployment-Color|X-.*-SHA|X-.*-Build|X-.*-Commit|Server:|X-Powered-By" "$f" 2>/dev/null && \
|
|
244
|
-
echo "SEC: response header in $f leaks infrastructure details. Git SHA enables targeted CVE diffing against public commits. Deployment color reveals blue/green topology. Safe pattern: omit these headers or gate them behind an internal-only middleware."
|
|
245
|
-
done <<< "$MIDDLEWARE_FILES"
|
|
246
|
-
```
|
|
247
|
-
**Confidence**: `LIKELY` — header may be intentional for internal observability tooling behind an auth gate; verify it applies to all responses.
|
|
248
|
-
**Severity**: LOW — information disclosure accelerates other attacks but is not directly exploitable without an additional vulnerability.
|
|
249
|
-
**Evidence**: Response headers are often added in middleware for observability tooling (blue/green routing, version tracking) and inadvertently included in all public responses. This check targets middleware/main.py changes specifically, where headers are most commonly added globally.
|
|
250
|
-
|
|
251
|
-
*Note: Step 2.5 Check C also covers response headers in an infra-posture context (trigger: Python files with existing `response.headers[` or `add_header(` patterns). Sub-check 9c complements it with a broader trigger scoped to middleware/main.py diffs.*
|
|
252
|
-
|
|
253
|
-
## Step 2.5: Infrastructure Security Posture (conditional — when infra files are in the diff)
|
|
254
|
-
|
|
255
|
-
These checks ONLY run when the corresponding file type appears in the PR diff. They cover structural security properties that don't change per-PR but must be verified whenever the surrounding config is modified.
|
|
256
|
-
|
|
257
|
-
**Run each block only if the file type is present:**
|
|
258
|
-
|
|
259
|
-
```bash
|
|
260
|
-
INFRA_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "docker-compose.*\.yml$")
|
|
261
|
-
DOCKERFILE_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -iE "Dockerfile")
|
|
262
|
-
WORKFLOW_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "^\.github/workflows/.*\.yml$")
|
|
263
|
-
HEADER_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "\.py$" | xargs grep -lE "response\.headers\[|add_header\(" 2>/dev/null)
|
|
264
|
-
TS_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "\.(ts|tsx)$")
|
|
265
|
-
CONFIG_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "(traefik/|infra/nginx/|infra/|\.github/workflows/).*(\.ya?ml|\.toml|\.json|\.conf|\.ini)$")
|
|
266
|
-
```
|
|
267
|
-
|
|
268
|
-
**A. Port bind address audit** (when `docker-compose*.yml` is in diff — trigger: `$INFRA_FILES` non-empty)
|
|
269
|
-
```bash
|
|
270
|
-
while IFS= read -r f; do
|
|
271
|
-
[ -z "$f" ] && continue
|
|
272
|
-
# Flag non-web ports bound to 0.0.0.0 (no explicit host binding = binds to 0.0.0.0 by default)
|
|
273
|
-
grep -nE "^\s+- \"[0-9]+:[0-9]+\"" "$f" | grep -v "127\.0\.0\.1\|::1" | grep -vE "\"(80|443):" && \
|
|
274
|
-
echo "SEC: port published on 0.0.0.0 in $f — verify UFW/firewall is the ONLY protection. Docker daemon restart can reset iptables and expose the port directly."
|
|
275
|
-
done <<< "$INFRA_FILES"
|
|
276
|
-
```
|
|
277
|
-
**Confidence**: `LIKELY` — a firewall rule may exist that provides protection; flag for verification rather than hard-block.
|
|
278
|
-
**Severity**: HIGH for database/internal service ports (5432, 6432, 6379); MEDIUM for other non-web ports.
|
|
279
|
-
**Evidence**: Internal service ports (DB, cache, proxy) published without an explicit host binding default to `0.0.0.0`, making them reachable from any interface. Docker daemon restarts can reset iptables and bypass firewall rules that were the only protection.
|
|
280
|
-
|
|
281
|
-
**B. Missing USER directive** (when any `Dockerfile` is in diff — trigger: `$DOCKERFILE_FILES` non-empty)
|
|
282
|
-
```bash
|
|
283
|
-
while IFS= read -r f; do
|
|
284
|
-
[ -z "$f" ] && continue
|
|
285
|
-
grep -q "^USER " "$f" || \
|
|
286
|
-
echo "SEC: CONFIRMED — no USER directive in $f. Container runs as UID 0. For services that process user-supplied URLs or untrusted input (worker, Playwright), running as root significantly lowers the bar for container escape."
|
|
287
|
-
done <<< "$DOCKERFILE_FILES"
|
|
288
|
-
```
|
|
289
|
-
**Confidence**: `CONFIRMED` — absence of USER directive is an objective, verifiable fact.
|
|
290
|
-
**Severity**: HIGH for services processing untrusted input; MEDIUM for internal-only services.
|
|
291
|
-
**Evidence**: A Dockerfile without a USER directive runs all processes as UID 0. For services that process user-supplied content (browser automation, file parsing, untrusted URLs), root execution significantly lowers the bar for container escape.
|
|
292
|
-
|
|
293
|
-
**C. Response header information disclosure** (when Python middleware/main.py adds response headers — trigger: `$HEADER_FILES` non-empty)
|
|
294
|
-
```bash
|
|
295
|
-
while IFS= read -r f; do
|
|
296
|
-
[ -z "$f" ] && continue
|
|
297
|
-
grep -nE "X-App-Version|X-Deployment-Color|Server:|X-Powered-By|X-.*-SHA|X-.*-Build" "$f" 2>/dev/null && \
|
|
298
|
-
echo "SEC: response header in $f leaks infrastructure details. Git SHA enables targeted CVE diffing against public commits. Deployment color reveals blue/green topology to attackers."
|
|
299
|
-
done <<< "$HEADER_FILES"
|
|
300
|
-
```
|
|
301
|
-
**Confidence**: `CONFIRMED` — header name reveals infra details by definition.
|
|
302
|
-
**Severity**: MEDIUM — information disclosure accelerates other attacks but is not directly exploitable.
|
|
303
|
-
**Evidence**: Git SHAs in response headers enable targeted CVE diffing against public commits. Deployment color headers reveal blue/green topology. Software version strings in health endpoints enable CVE targeting. These headers are often added for internal observability and inadvertently included in all responses.
|
|
304
|
-
|
|
305
|
-
**D. CI security scan gating** (when `.github/workflows/*.yml` is in diff — trigger: `$WORKFLOW_FILES` non-empty)
|
|
306
|
-
```bash
|
|
307
|
-
while IFS= read -r f; do
|
|
308
|
-
[ -z "$f" ] && continue
|
|
309
|
-
# Find security scanner steps with continue-on-error: true
|
|
310
|
-
grep -B5 "continue-on-error: true" "$f" | grep -iE "trivy|snyk|security|cve|scan|grype|dockle" && \
|
|
311
|
-
echo "SEC: security scanner in $f has continue-on-error: true — CVEs will be logged but will NEVER block a deploy. Remove continue-on-error or add a severity threshold."
|
|
312
|
-
done <<< "$WORKFLOW_FILES"
|
|
313
|
-
```
|
|
314
|
-
**Confidence**: `CONFIRMED` — `continue-on-error: true` on a security scanner step is objectively a non-blocking scan.
|
|
315
|
-
**Severity**: HIGH — security scanner provides zero protection when it cannot block.
|
|
316
|
-
**Evidence**: A security scanner with `continue-on-error: true` reports green CI regardless of findings — CVEs are logged but never surface as failures. This makes the scan purely decorative: PRs introducing known vulnerabilities pass all checks.
|
|
317
|
-
|
|
318
|
-
**E. In-memory rate limiter** (when TypeScript files are in diff — trigger: `$TS_FILES` non-empty)
|
|
319
|
-
```bash
|
|
320
|
-
while IFS= read -r f; do
|
|
321
|
-
[ -z "$f" ] && continue
|
|
322
|
-
grep -nE "new Map\(\)|=\s*\{\}|= new Map" "$f" 2>/dev/null | grep -iE "rate|limit|attempt|count|throttle|window" && \
|
|
323
|
-
echo "SEC: in-memory rate limiter detected in $f — resets on every container restart, not shared across blue/green replicas. Use Redis (via Upstash, ioredis, or the existing Redis client) for distributed rate limiting."
|
|
324
|
-
done <<< "$TS_FILES"
|
|
325
|
-
```
|
|
326
|
-
**Confidence**: `LIKELY` — in-memory store may be intentional for non-critical paths; flag for verification.
|
|
327
|
-
**Severity**: MEDIUM — bypass requires restarting the container or hitting a different replica; not trivially exploitable but provides no protection in blue/green deploys.
|
|
328
|
-
**Evidence**: In-memory Maps/objects used for rate limiting reset on every container restart and are not shared across replicas in blue/green deployments. An attacker can bypass limits by targeting the other replica or waiting for a deploy cycle.
|
|
329
|
-
|
|
330
|
-
**F. Cookie security flags** (when TypeScript files are in diff — trigger: `$TS_FILES` non-empty)
|
|
331
|
-
```bash
|
|
332
|
-
while IFS= read -r f; do
|
|
333
|
-
[ -z "$f" ] && continue
|
|
334
|
-
grep -nE "httpOnly:\s*false|secure:\s*false|sameSite.*['\"]none['\"]" "$f" 2>/dev/null && \
|
|
335
|
-
echo "SEC: cookie in $f has weakened security flag. httpOnly: false exposes cookie to JS (XSS risk). secure: false allows transmission over HTTP. sameSite: none without Secure enables CSRF."
|
|
336
|
-
done <<< "$TS_FILES"
|
|
337
|
-
```
|
|
338
|
-
**Confidence**: `CONFIRMED` — explicit `false`/`none` values are objective findings.
|
|
339
|
-
**Severity**: HIGH for `httpOnly: false` (direct XSS vector for session cookies); MEDIUM for `secure: false` or `sameSite: none`.
|
|
340
|
-
**Evidence**: Explicitly weakened cookie security flags are directly exploitable: `httpOnly: false` exposes the cookie value to any JavaScript running in the page (XSS-to-session-theft); `secure: false` allows the cookie to be sent over plaintext HTTP; `sameSite: none` without `Secure` enables cross-site request forgery.
|
|
341
|
-
|
|
342
|
-
**G. Fallback credential scan in config files** (when config files are in diff — trigger: `$CONFIG_FILES` non-empty)
|
|
343
|
-
```bash
|
|
344
|
-
while IFS= read -r f; do
|
|
345
|
-
[ -z "$f" ] && continue
|
|
346
|
-
# Shell-style fallback: ${VAR:-default} — flag non-trivial fallback values that may be credentials
|
|
347
|
-
grep -nE "\$\{[A-Z_]+:-[^}]{4,}\}" "$f" | grep -ivE "(localhost|127\.0\.0\.1|0\.0\.0\.0|true|false|^[0-9]+$|/tmp|/var|/etc)" && \
|
|
348
|
-
echo "SEC: potential credential fallback in $f — verify fallback value is safe if env var is absent"
|
|
349
|
-
done <<< "$CONFIG_FILES"
|
|
350
|
-
```
|
|
351
|
-
For each hit: read the fallback value.
|
|
352
|
-
- Contains `$apr1$`, `$2b$`, `$bcrypt$`, `$argon2`, or hash-like long alphanumeric with `$` separators: **CONFIRMED HIGH** — htpasswd/bcrypt placeholder; active if env var is absent.
|
|
353
|
-
- Is `admin`, `password`, `changeme`, `secret`, `test`, or similar weak string: **CONFIRMED HIGH** — default credential.
|
|
354
|
-
- Is empty string (`${VAR:-}` or `${VAR:-""}`): **CONFIRMED HIGH** if field controls authentication.
|
|
355
|
-
|
|
356
|
-
**Required cross-checks when a credential fallback is found**: (1) Is the env var required in `app/env_validation.py`? (2) Is it in `scripts/decrypt-secrets.sh` ENV_MAPPING? (3) Is it marked required in `.env.example`? If ANY missing: the insecure fallback may be active in some environments.
|
|
357
|
-
**Confidence**: `CONFIRMED` — fallback value is objectively verifiable.
|
|
358
|
-
**Severity**: HIGH — insecure fallback credential is equivalent to a hardcoded credential for any environment missing the env var.
|
|
359
|
-
**Evidence**: Config files commonly use `${VAR:-placeholder}` patterns during development, where the placeholder is a sample credential or hash. If the env var is absent in any environment (staging, non-production, first deploy before secrets are injected), the placeholder becomes the active credential. Also covered by INFRA agent item 13 (deployment context + ENV_MAPPING cross-check); this item extends coverage to PRs not classified as INFRA domain.
|
|
360
|
-
|
|
361
|
-
**H. Runtime UID × volume ownership** (when any Dockerfile is in diff AND diff contains `USER`, `su-exec`, `gosu`, or `setuid` — trigger: `$DOCKERFILE_FILES` non-empty AND UID change detected)
|
|
362
|
-
```bash
|
|
363
|
-
UID_CHANGE=$(while IFS= read -r f; do
|
|
364
|
-
[ -z "$f" ] && continue
|
|
365
|
-
gh pr diff [PR_NUMBER] -- "$f" 2>/dev/null | grep -E '^\+.*(USER\s+[^0]|su-exec|gosu|setuid)' | grep -v '^+++' && break
|
|
366
|
-
done <<< "$DOCKERFILE_FILES")
|
|
367
|
-
if [ -n "$UID_CHANGE" ]; then
|
|
368
|
-
echo "UID change detected — cross-referencing named volume mounts..."
|
|
369
|
-
COMPOSE_FILES=$(find . -maxdepth 2 -name "docker-compose*.yml" 2>/dev/null | sort)
|
|
370
|
-
while IFS= read -r cf; do
|
|
371
|
-
[ -z "$cf" ] && continue
|
|
372
|
-
grep -A2 "volumes:" "$cf" | grep -E "^\s+\w.*:" | grep -v "^--$" && echo " ^ in $cf"
|
|
373
|
-
done <<< "$COMPOSE_FILES"
|
|
374
|
-
fi
|
|
375
|
-
```
|
|
376
|
-
For each named volume found: determine its container mount point. Named volumes are created as **root-owned** by Docker at first use — a privilege drop (root → UID N) without a `chown` before the drop will silently break any service path that calls `mkdir`, `write_bytes`, `open(... 'w')`, or `os.makedirs` under that mount point.
|
|
377
|
-
|
|
378
|
-
**Check**: Does the entrypoint script (or equivalent) run `chown -R <user>:<group> <mount_point>` BEFORE the `exec su-exec` / `exec gosu` call? If not:
|
|
379
|
-
- Search the service codebase for filesystem write operations under the volume mount point:
|
|
380
|
-
```bash
|
|
381
|
-
grep -rn "mkdir\|write_bytes\|open(.*['\"]w\|makedirs\|shutil\.copy\|shutil\.move" services/{SERVICE}/ --include="*.py" | grep -v __pycache__
|
|
382
|
-
```
|
|
383
|
-
- If any write path is found under the volume mount: **CONFIRMED BLOCKING** — the service will fail with `[Errno 13] Permission denied` on first write attempt after the privilege drop.
|
|
384
|
-
|
|
385
|
-
**Confidence**: `CONFIRMED` — Docker named volume ownership is an objective, verifiable fact (volumes are root-owned at creation).
|
|
386
|
-
**Severity**: HIGH — filesystem writes fail silently at runtime with `PermissionError`; no startup crash, so the failure only surfaces when the code path executes.
|
|
387
|
-
**Evidence**: When a container's runtime user changes from root to a non-root UID, Docker named volumes retain their existing ownership (root:root). Any code path that writes to a path under a named volume mount point will fail with `[Errno 13] Permission denied`. The fix is to add `chown -R <user>:<group> <mount_point>` to the entrypoint script before the privilege-drop `exec` call. <!-- Added: forge#323 -->
|
|
388
|
-
|
|
389
|
-
## Step 3: Code Quality Scan
|
|
390
|
-
1. **Error handling**: Exceptions swallowed silently, generic except clauses
|
|
391
|
-
2. **Resource leaks**: Unclosed connections, files, async tasks not awaited
|
|
392
|
-
3. **Logic errors**: Off-by-one, wrong operators, missing returns
|
|
393
|
-
4. **Type safety**: Optional types accessed without None check
|
|
394
|
-
5. **Runtime correctness**: Will this code actually execute without errors? Trace every construct to its runtime behavior — duplicate column names in SQL, mismatched function signatures, invalid regex, malformed templates. If something looks "redundant", ask whether the redundancy causes an error downstream (e.g., duplicate columns + DISTINCT ON = ambiguous reference).
|
|
395
|
-
6. **Python scoping hazards**: Local `import X` inside a function makes `X` a local variable for the ENTIRE function. If any code above that import references `X`, it will crash with `UnboundLocalError`. Check every function-scoped import against all references to that name in the same scope. This is a CONFIRMED CRITICAL — it's a guaranteed runtime crash.
|
|
396
|
-
7. **Callback/callable signature mismatch**: When the diff passes a `lambda`, function reference, or callable as a keyword argument to a library/framework function (e.g., `prepared_statement_name_func=lambda _: ""`, `key=lambda x, y: ...`, `default=lambda: ...`), verify the callback's parameter count matches what the library expects. Check the library's default value for that parameter or its documentation. A lambda with wrong arity is a guaranteed `TypeError` at runtime — this is CONFIRMED CRITICAL. Wrong arity cannot be caught by static analysis or `py_compile` — it only fails at the call site. The review agent must verify signature, not just quote the lambda and approve.
|
|
397
|
-
8. **Fix PR input-format coverage** (when PR title contains "fix" and the diff patches a specific input format failure): When a fix PR addresses one documented failure mode (e.g., blank string → `[]`), search `.env.example` for ALL documented input format examples for the affected env var or config field. For each documented format that the fix does NOT handle, flag as CONFIRMED HIGH.
|
|
398
|
-
```bash
|
|
399
|
-
# Find the env var(s) touched by the fix
|
|
400
|
-
FIXED_VARS=$(gh pr diff [PR_NUMBER] | grep -oE "ENABLED_[A-Z_]+|[A-Z_]{3,}(?=.*decode_complex_value|.*list\[str\]|.*List\[str\])" | sort -u)
|
|
401
|
-
# For each var, read its .env.example comment for documented format examples
|
|
402
|
-
for var in $FIXED_VARS; do
|
|
403
|
-
grep -A5 "$var" .env.example 2>/dev/null | grep -iE "comma.separated|csv|json|space.separated|pipe.separated|semicolon.separated"
|
|
404
|
-
done
|
|
405
|
-
# Then read the fix itself — what format(s) does it handle?
|
|
406
|
-
gh pr diff [PR_NUMBER] | grep -E "^\+" | grep -iE "json\.loads|split\(','\)|split\(' '\)|csv|,\.join"
|
|
407
|
-
```
|
|
408
|
-
Compare documented formats against handled formats. Any documented format not handled by the fix is a gap. A fix that addresses one documented failure mode (e.g., blank string) while leaving another documented format (e.g., CSV) unhandled is an incomplete fix. <!-- Added: forge#190 -->
|
|
409
|
-
9. **Scope creep detection**: Compare the PR title/description scope against the actual diff size. When a fix PR's diff contains significantly more logic than its stated scope implies, there is a high risk that the extra code introduces bugs the reviewer is not primed to look for — and that the builder agent added context from a different branch or a different issue.
|
|
410
|
-
```bash
|
|
411
|
-
# Count lines added in the diff (excluding whitespace-only lines and file headers)
|
|
412
|
-
DIFF_LINES_ADDED=$(gh pr diff [PR_NUMBER] | grep -c '^+[^+]' 2>/dev/null || echo 0)
|
|
413
|
-
PR_TITLE=$(gh pr view [PR_NUMBER] --json title --jq '.title')
|
|
414
|
-
PR_BODY=$(gh pr view [PR_NUMBER] --json body --jq '.body')
|
|
415
|
-
```
|
|
416
|
-
**Flag as POSSIBLE (MEDIUM) if ALL of the following are true**:
|
|
417
|
-
- PR title contains "fix" (indicating a targeted bug fix, not a feature or refactor)
|
|
418
|
-
- The diff adds more than 40 non-whitespace lines
|
|
419
|
-
- The added lines include logic blocks (functions, classes, conditionals) not directly related to the stated fix
|
|
420
|
-
- The added logic is NOT mentioned in the PR description
|
|
421
|
-
|
|
422
|
-
**What to check**: Read the diff in full. For each block of added code that is NOT directly implementing the stated fix, ask: "Is this block explained by the PR title or description?" If not, flag it. Specifically look for:
|
|
423
|
-
- New import statements for modules not referenced in the fix
|
|
424
|
-
- New function definitions beyond what the fix requires
|
|
425
|
-
- Feature gate blocks added to a bug-fix PR
|
|
426
|
-
- Conditional blocks that check for feature flags, subscriptions, or plan tiers in a PR that claims to fix an unrelated error
|
|
427
|
-
|
|
428
|
-
**Confidence**: `POSSIBLE` (reviewer cannot always know the full intent — flag for human review, do not hard-block). If the extra code is importing a module from a non-existent path, upgrade to `CONFIRMED HIGH`.
|
|
429
|
-
|
|
430
|
-
When a builder agent working in a milestone-branch worktree adds code to a fast-lane fix PR, it can introduce imports of milestone-only modules or feature gate logic unrelated to the stated fix. These additions pass local compilation (the module exists in the worktree) but crash at runtime (the module doesn't exist on the base branch). <!-- Added: forge#277 -->
|
|
431
|
-
|
|
432
|
-
10. **Architectural fit check**: For any PR that introduces new files in service directories (`services/`, `worker/`, `scripts/`, `tools/`, or project-specific tooling directories like `reddit-bot/`, `bots/`, `cli/`), check whether a scheduled or automated system already handles the same functional capability.
|
|
433
|
-
```bash
|
|
434
|
-
# Get list of new files introduced by the PR
|
|
435
|
-
gh pr diff [PR_NUMBER] --name-only | grep -E "^(services|worker|scripts|tools|reddit-bot|bots|cli)/"
|
|
436
|
-
# For each new file, identify the primary capability (posting, syncing, notifying, etc.)
|
|
437
|
-
# Then search for existing implementations:
|
|
438
|
-
grep -rn "{capability_keyword}" services/ --include="*.py" -l 2>/dev/null | head -10
|
|
439
|
-
grep -rn "scheduler\|celery\|cron\|nightly\|periodic\|schedule" services/ --include="*.py" -l 2>/dev/null | head -10
|
|
440
|
-
```
|
|
441
|
-
**Flag as POSSIBLE (HIGH) if**: A new file implements a capability (e.g., posting to Reddit, sending emails, syncing data) AND a search of existing services finds a scheduler, runner, or automated pipeline that already performs the same action. The risk: parallel systems create split ownership, dual authentication surface, and non-deterministic execution order.
|
|
442
|
-
|
|
443
|
-
**What to look for**:
|
|
444
|
-
- New file posts to a platform (Reddit, Twitter, Slack) → check if an existing service has a scheduler or crosspost runner for that platform
|
|
445
|
-
- New file sends emails or notifications → check if a notification service or Herald already handles this
|
|
446
|
-
- New file syncs or exports data → check if a worker or periodic job already owns this data flow
|
|
447
|
-
- New file queries an external API on a schedule → check if an existing scheduler already polls that API
|
|
448
|
-
|
|
449
|
-
**Confidence**: `POSSIBLE` — reviewer may not have full knowledge of all service capabilities. Flag for human verification. Escalate to `CONFIRMED HIGH` only if the existing code path is unambiguously doing the same thing (e.g., both files call the same third-party API endpoint to perform the same action).
|
|
450
|
-
|
|
451
|
-
A new standalone script that duplicates a capability owned by an existing service creates split ownership: two code paths performing the same action, each with separate authentication surfaces and no coordination. The review passing on syntax and call signatures does not catch this — the question is whether the new file should exist at all. <!-- Added: forge#279 -->
|
|
452
|
-
|
|
453
|
-
11. **Pattern-completion check** *(conditional — when PR title contains "fix" AND the diff removes or narrows a condition or field gate in a router file)*: When a fix PR changes a condition that guards a function call or restricts a set of fields, grep for the same original (unfixed) condition in sibling files within the same router directory. A fix that corrects one router but leaves identical gates in sibling routers is incomplete — users of those endpoints still receive the original error.
|
|
454
|
-
```bash
|
|
455
|
-
# Identify the router directory of the changed file
|
|
456
|
-
ROUTER_DIR=$(gh pr diff [PR_NUMBER] --name-only | grep -E "routers/.*\.py$" | head -1 | xargs dirname 2>/dev/null)
|
|
457
|
-
|
|
458
|
-
# Extract removed condition lines from the diff
|
|
459
|
-
REMOVED_CONDITIONS=$(gh pr diff [PR_NUMBER] | grep -E '^-.*\bif\b.*\bor\b|^-.*\bif\b.*(and|:)\s*$' | grep -v '^---' | sed 's/^-//' | head -5)
|
|
460
|
-
|
|
461
|
-
if [ -n "$ROUTER_DIR" ] && [ -n "$REMOVED_CONDITIONS" ]; then
|
|
462
|
-
echo "$REMOVED_CONDITIONS" | while IFS= read -r condition; do
|
|
463
|
-
# Extract a key term from the condition to search for
|
|
464
|
-
PATTERN=$(echo "$condition" | grep -oP '\b[a-z_]{5,}\b' | grep -v "^if$\|^or$\|^and$\|^not$\|^request$\|^self$" | head -1)
|
|
465
|
-
if [ -n "$PATTERN" ]; then
|
|
466
|
-
# Find changed files to exclude
|
|
467
|
-
CHANGED=$(gh pr diff [PR_NUMBER] --name-only | tr '\n' ' ')
|
|
468
|
-
# Search sibling files for the same pattern in a conditional context
|
|
469
|
-
grep -rn "\b${PATTERN}\b" "$ROUTER_DIR" --include="*.py" | grep -E "\bif\b" | while read -r match; do
|
|
470
|
-
FILE=$(echo "$match" | cut -d: -f1)
|
|
471
|
-
# Skip files already changed in this PR
|
|
472
|
-
echo "$CHANGED" | grep -q "$FILE" && continue
|
|
473
|
-
echo "SEC: Residual unfixed gate pattern '$PATTERN' found in $match — sibling router may have the same bug as this fix addressed. Verify the condition is correct for all fields in scope."
|
|
474
|
-
done
|
|
475
|
-
fi
|
|
476
|
-
done
|
|
477
|
-
fi
|
|
478
|
-
```
|
|
479
|
-
**Flag as CONFIRMED (HIGH) if**: The same condition pattern exists in a sibling router file AND that file is NOT in the PR diff. The sibling file has the same structural bug — users of that endpoint are still affected.
|
|
480
|
-
|
|
481
|
-
**Flag as POSSIBLE (MEDIUM) if**: The pattern match is found but the surrounding context is ambiguous (e.g., the field appears in a different kind of condition or with different semantics).
|
|
482
|
-
|
|
483
|
-
A condition fix that is limited to the most visible endpoint (the one generating observed errors) leaves users of sibling endpoints experiencing the original bug. The fix is structurally complete only when all callers of the gated operation have been verified. <!-- Added: forge#383 -->
|
|
484
|
-
|
|
485
|
-
## Step 4: Post Findings
|
|
486
|
-
```bash
|
|
487
|
-
gh pr comment [PR_NUMBER] --body "$(cat <<'EOF'
|
|
488
|
-
## Security & Code Quality Scan
|
|
489
|
-
|
|
490
|
-
### Risk Level: [LOW/MEDIUM/HIGH/CRITICAL]
|
|
491
|
-
|
|
492
|
-
### Security Findings
|
|
493
|
-
| Severity | Confidence | Finding | File:Line | Evidence |
|
|
494
|
-
|----------|------------|---------|-----------|----------|
|
|
495
|
-
| ... | ... | ... | ... | ... |
|
|
496
|
-
|
|
497
|
-
### Code Quality Issues
|
|
498
|
-
| Category | Issue | File:Line | Impact |
|
|
499
|
-
|----------|-------|-----------|--------|
|
|
500
|
-
| ... | ... | ... | ... |
|
|
501
|
-
|
|
502
|
-
### Files Reviewed
|
|
503
|
-
[List all files checked]
|
|
504
|
-
|
|
505
|
-
---
|
|
506
|
-
*Automated security & quality scan*
|
|
507
|
-
|
|
508
|
-
<!-- REVIEW-FINDINGS-START -->
|
|
509
|
-
<!-- FINDING:SEC-1|CONFIDENCE|SEVERITY|file.py:line|Summary -->
|
|
510
|
-
<!-- (add one FINDING line per issue found — include ALL confidence levels) -->
|
|
511
|
-
<!-- REVIEW-FINDINGS-END -->
|
|
512
|
-
EOF
|
|
513
|
-
)"
|
|
514
|
-
```
|
|
515
|
-
|
|
516
|
-
**Structured Findings**: Include the structured findings block above at the end of your comment. Your prefix: `SEC`. See the Structured Findings Protocol section above for format rules.
|
|
517
|
-
```
|
|
518
|
-
|
|
519
|
-
### Coverage Matrix — SEC Agent
|
|
520
|
-
|
|
521
|
-
| Defect Category | Check Item(s) | Status | Ref |
|
|
522
|
-
|----------------|---------------|--------|-----|
|
|
523
|
-
| SQL/command/template injection | Step 2.1 | COVERED | #210–#323 |
|
|
524
|
-
| SSRF via user-controlled URLs | Step 2.2 | COVERED | |
|
|
525
|
-
| Path traversal | Step 2.3 | COVERED | |
|
|
526
|
-
| Unsafe deserialization | Step 2.4 | COVERED | |
|
|
527
|
-
| Hardcoded secrets | Step 2.5 | COVERED | |
|
|
528
|
-
| XSS | Step 2.6 | COVERED | |
|
|
529
|
-
| Token privilege confusion | Step 2.7 | COVERED | |
|
|
530
|
-
| Unbounded resource consumption | Step 2.8 | COVERED | |
|
|
531
|
-
| API information disclosure | Step 2.9 (a/b/c) | COVERED | #302 |
|
|
532
|
-
| Port bind address exposure | Step 2.5A | COVERED | #296 |
|
|
533
|
-
| Missing container USER directive | Step 2.5B | COVERED | #296 |
|
|
534
|
-
| Response header info leakage | Step 2.5C | COVERED | #302 |
|
|
535
|
-
| CI security scan gating | Step 2.5D | COVERED | #296 |
|
|
536
|
-
| In-memory rate limiter bypass | Step 2.5E | COVERED | |
|
|
537
|
-
| Cookie security flags | Step 2.5F | COVERED | |
|
|
538
|
-
| Credential fallbacks in config | Step 2.5G | COVERED | #301 |
|
|
539
|
-
| UID x volume ownership | Step 2.5H | COVERED | #323 |
|
|
540
|
-
| Error handling gaps | Step 3.1 | COVERED | |
|
|
541
|
-
| Resource leaks | Step 3.2 | COVERED | |
|
|
542
|
-
| Logic errors / off-by-one | Step 3.3 | COVERED | |
|
|
543
|
-
| Type safety (Optional without None check) | Step 3.4 | COVERED | |
|
|
544
|
-
| Runtime correctness (duplicate cols, bad regex) | Step 3.5 | COVERED | |
|
|
545
|
-
| Python scoping hazards (local import) | Step 3.6 | COVERED | |
|
|
546
|
-
| Callback signature mismatch | Step 3.7 | COVERED | #277 |
|
|
547
|
-
| Fix PR input-format coverage | Step 3.8 | COVERED | #190 |
|
|
548
|
-
| Scope creep detection | Step 3.9 | COVERED | #277 |
|
|
549
|
-
| Architectural fit (parallel systems) | Step 3.10 | COVERED | #279 |
|
|
550
|
-
| Pattern-completion for condition-fix PRs (sibling router check) | Step 3.11 | COVERED | #383 |
|
|
551
|
-
| Supply chain / dependency confusion | — | GAP | |
|
|
552
|
-
| Timing side-channels in auth | — | GAP | |
|
|
553
|
-
|
|
554
|
-
---
|
|
555
|
-
|
|
556
|
-
### Agent: Auth Conventions Auditor
|
|
557
|
-
|
|
558
|
-
**Trigger**: AUTH domain detected
|
|
559
|
-
**Type**: `security-exploit-auditor` | **Model**: `sonnet`
|
|
560
|
-
|
|
561
|
-
**Prompt template:**
|
|
562
|
-
```
|
|
563
|
-
You are auditing PR #[PR_NUMBER] for authentication and authorization correctness in [PROJECT_NAME].
|
|
564
|
-
|
|
565
|
-
## CRITICAL PROJECT CONVENTIONS — You MUST enforce these:
|
|
566
|
-
|
|
567
|
-
[DOMAIN_CONTEXT]
|
|
568
|
-
|
|
569
|
-
### How to Check
|
|
570
|
-
```bash
|
|
571
|
-
# Find auth patterns in changed files
|
|
572
|
-
grep -rn "auth\|Depends\|get_current\|verify_token\|require_auth" [CHANGED_FILES]
|
|
573
|
-
|
|
574
|
-
# Cross-reference with route path to determine if correct
|
|
575
|
-
grep -rn "@router\.(get\|post\|put\|delete)\|app\.(get\|post\|put\|delete)" [CHANGED_FILES]
|
|
576
|
-
```
|
|
577
|
-
|
|
578
|
-
### What to Verify
|
|
579
|
-
1. Every new endpoint has an auth dependency (no unauthenticated endpoints without justification)
|
|
580
|
-
2. The CORRECT auth dependency is used based on route type (refer to project conventions in [DOMAIN_CONTEXT] above — if absent, derive from surrounding code patterns)
|
|
581
|
-
3. Multi-tenancy: resource access checks `user_id` ownership BEFORE returning data
|
|
582
|
-
4. Rate limiting is applied to public-facing endpoints
|
|
583
|
-
5. No IDOR vulnerabilities (different error codes for 404 vs 403 leak existence)
|
|
584
|
-
6. Header forwarding trust model: when the diff touches Next.js middleware, route handlers, or FastAPI rate-limiting middleware, verify that attacker-controlled `X-Forwarded-For` headers cannot spoof the IP used for rate limit keying
|
|
585
|
-
|
|
586
|
-
```bash
|
|
587
|
-
# Check if Next.js proxy layer strips or validates X-Forwarded-For before forwarding
|
|
588
|
-
grep -nE "x-forwarded-for|X-Forwarded-For|headers.*forwarded|forwarded.*headers" \
|
|
589
|
-
web/src/middleware.ts web/src/app/api/ 2>/dev/null
|
|
590
|
-
# If found without a strip/validate step: CONFIRMED HIGH — complete rate limit bypass
|
|
591
|
-
# Safe pattern: headers.delete("x-forwarded-for") or rightmost-IP-only extraction
|
|
592
|
-
|
|
593
|
-
# Check if FastAPI backend trusts X-Forwarded-For for rate limit keying
|
|
594
|
-
grep -rn "X-Forwarded-For\|x_forwarded_for\|forwarded_for\|client_ip\|real_ip" \
|
|
595
|
-
services/api/app/ | head -10
|
|
596
|
-
# If found without trusted-proxy-range validation: CONFIRMED HIGH
|
|
597
|
-
# The backend must only trust X-Forwarded-For when the immediate caller is in a known trusted proxy range
|
|
598
|
-
```
|
|
599
|
-
|
|
600
|
-
**Flag as CONFIRMED HIGH** if: X-Forwarded-For is forwarded from client to backend without stripping or rightmost-only extraction AND the backend keys rate limits on that header value.
|
|
601
|
-
|
|
602
|
-
7. JWT secret/algorithm separation: when the diff touches JWT signing, verification, or token generation, verify that different token classes (user session, admin-proxy, API key) use distinct signing secrets or asymmetric algorithms — not a shared secret differentiated only by claims
|
|
603
|
-
|
|
604
|
-
```bash
|
|
605
|
-
# Check signing configuration across token classes
|
|
606
|
-
grep -nE "SECRET|algorithm|HS256|RS256|verify|decode" \
|
|
607
|
-
services/api/app/security/auth.py web/src/lib/auth.ts | head -20
|
|
608
|
-
# Flag if: multiple token classes reference the same secret variable AND the only differentiator is a claim (e.g., aud, role)
|
|
609
|
-
# Safe pattern: admin tokens use a separate ADMIN_JWT_SECRET or RS256 (asymmetric)
|
|
610
|
-
# so that a user-level token CANNOT be escalated to admin by claim manipulation
|
|
611
|
-
```
|
|
612
|
-
|
|
613
|
-
**Flag as CONFIRMED HIGH** if: two or more token classes with different privilege levels share the same signing secret and algorithm, with only claim values (e.g., `aud="admin-proxy"`) preventing privilege escalation.
|
|
614
|
-
|
|
615
|
-
## Context
|
|
616
|
-
- Files changed: [AUTH_RELEVANT_FILES]
|
|
617
|
-
- PR description: [PR_BODY]
|
|
618
|
-
|
|
619
|
-
## Steps
|
|
620
|
-
1. Read the diff: `gh pr diff [PR_NUMBER]`
|
|
621
|
-
2. For each endpoint, verify auth dependency matches route type
|
|
622
|
-
3. Check ownership queries for multi-tenancy
|
|
623
|
-
4. Search for patterns: `grep -rn "Depends(get_current" services/api/app/routers/`
|
|
624
|
-
5. Compare against existing router patterns for consistency
|
|
625
|
-
6. If diff touches `web/src/middleware.ts`, Next.js route handlers, or FastAPI rate-limit middleware: run header forwarding check (item 6 grep commands above)
|
|
626
|
-
7. If diff touches `services/api/app/security/auth.py`, `web/src/lib/auth.ts`, or any JWT signing/verification code: run JWT secret separation check (item 7 grep commands above)
|
|
627
|
-
|
|
628
|
-
## Post Findings
|
|
629
|
-
```bash
|
|
630
|
-
gh pr comment [PR_NUMBER] --body "$(cat <<'EOF'
|
|
631
|
-
## Auth Conventions Audit
|
|
632
|
-
|
|
633
|
-
### Endpoints Reviewed
|
|
634
|
-
| Endpoint | Method | Auth Dep Used | Expected Auth Dep | Correct? | Ownership Check |
|
|
635
|
-
|----------|--------|--------------|-------------------|----------|-----------------|
|
|
636
|
-
| /path | POST | CurrentUser | SessionUser | NO — VIOLATION | Yes/No/N/A |
|
|
637
|
-
|
|
638
|
-
### Convention Violations
|
|
639
|
-
[List any violations of the SessionUser/CurrentUser convention with file:line]
|
|
640
|
-
|
|
641
|
-
### Multi-Tenancy Check
|
|
642
|
-
[For each resource access, show the ownership check or flag missing]
|
|
643
|
-
|
|
644
|
-
### Header Forwarding Trust Model (Item 6)
|
|
645
|
-
[If proxy/middleware files touched: state whether X-Forwarded-For is stripped/validated before rate-limit IP keying, or SKIPPED if no middleware changes in this PR]
|
|
646
|
-
|
|
647
|
-
### JWT Secret Separation (Item 7)
|
|
648
|
-
[If auth.py or JWT signing files touched: state whether each token class uses a distinct secret/algorithm, or SKIPPED if no JWT configuration changes in this PR]
|
|
649
|
-
|
|
650
|
-
### Files Reviewed
|
|
651
|
-
[List all auth-related files checked]
|
|
652
|
-
|
|
653
|
-
---
|
|
654
|
-
*Auth conventions audit*
|
|
655
|
-
|
|
656
|
-
<!-- REVIEW-FINDINGS-START -->
|
|
657
|
-
<!-- FINDING:AUTH-1|CONFIDENCE|SEVERITY|file.py:line|Summary -->
|
|
658
|
-
<!-- (add one FINDING line per issue found — include ALL confidence levels) -->
|
|
659
|
-
<!-- REVIEW-FINDINGS-END -->
|
|
660
|
-
EOF
|
|
661
|
-
)"
|
|
662
|
-
```
|
|
663
|
-
|
|
664
|
-
**Structured Findings**: Include the structured findings block above at the end of your comment. Your prefix: `AUTH`. See the Structured Findings Protocol section above for format rules.
|
|
665
|
-
```
|
|
666
|
-
|
|
667
|
-
### Coverage Matrix — AUTH Agent
|
|
668
|
-
|
|
669
|
-
| Defect Category | Check Item(s) | Status | Ref |
|
|
670
|
-
|----------------|---------------|--------|-----|
|
|
671
|
-
| Missing auth dependency on endpoint | Item 1 | COVERED | |
|
|
672
|
-
| Wrong auth dependency (SessionUser vs CurrentUser) | Item 2 | COVERED | |
|
|
673
|
-
| Multi-tenancy / IDOR | Item 3 | COVERED | |
|
|
674
|
-
| Missing rate limiting on public endpoints | Item 4 | COVERED | |
|
|
675
|
-
| IDOR via error code differentiation | Item 5 | COVERED | |
|
|
676
|
-
| X-Forwarded-For trust model bypass | Item 6 | COVERED | #299 |
|
|
677
|
-
| JWT shared-secret privilege escalation | Item 7 | COVERED | #299 |
|
|
678
|
-
| OAuth state parameter CSRF | — | GAP | |
|
|
679
|
-
| Session fixation / session invalidation on privilege change | — | GAP | |
|
|
680
|
-
|
|
681
|
-
---
|
|
682
|
-
|
|
683
|
-
### Agent: Billing Integrity Auditor
|
|
684
|
-
|
|
685
|
-
**Trigger**: BILLING domain detected
|
|
686
|
-
**Type**: `codebase-explorer` | **Model**: `sonnet`
|
|
687
|
-
|
|
688
|
-
**Prompt template:**
|
|
689
|
-
```
|
|
690
|
-
You are auditing PR #[PR_NUMBER] for billing integrity in [PROJECT_NAME].
|
|
691
|
-
|
|
692
|
-
CRITICAL: Any billing bug = revenue loss or user overcharging.
|
|
693
|
-
|
|
694
|
-
## Project Billing Architecture
|
|
695
|
-
|
|
696
|
-
[DOMAIN_CONTEXT]
|
|
697
|
-
|
|
698
|
-
If no billing context is configured above, derive the billing flow from the changed files: trace credit check → debit → execution → reconciliation paths.
|
|
699
|
-
|
|
700
|
-
## What to Verify
|
|
701
|
-
1. **Trace the full flow**: credit check → pre-debit → execution → reconciliation
|
|
702
|
-
2. **tier_used accuracy**: Where is `tier_used` set? Is it the final tier or intermediate?
|
|
703
|
-
3. **No double-charging**: Verify pre-debit and reconciliation don't overlap
|
|
704
|
-
4. **Failure handling**: What happens to credits when a scrape fails?
|
|
705
|
-
5. **Idempotency**: Can a retry cause double-debit?
|
|
706
|
-
6. **Free scrape paths**: Is there any code path that bypasses billing entirely?
|
|
707
|
-
7. **Gate regression check**: If the PR contains or preserves a feature gate that restricts endpoint access (e.g., `if "feature_name" not in features`, tier checks, balance thresholds blocking a route), verify the gate existed in the base branch BEFORE the commits being reviewed. Run `git show origin/{base}:{file} | grep -n "gate_pattern"` to check. If the gate was introduced by the same commit chain being fixed — not an independent historical addition — flag it as a potential rogue gate with HIGH severity: the correct fix is to fully revert the gate block, not to patch around it. A rogue gate silently restricts access for all users below a tier or balance threshold without any intentional review of that restriction. This finding is **informational — not a merge blocker**, but must appear in the Findings table so it can be tracked as a follow-up. <!-- Added: forge#278 -->
|
|
708
|
-
|
|
709
|
-
## MANDATORY Before Reporting
|
|
710
|
-
- Search `grep -rn "reconcil" services/worker/` before claiming "no reconciliation"
|
|
711
|
-
- Trace `tier_used` variable before claiming "wrong tier charged"
|
|
712
|
-
- Read the job completion handler in queues.py before claiming "credits not refunded"
|
|
713
|
-
|
|
714
|
-
## Post Findings
|
|
715
|
-
```bash
|
|
716
|
-
gh pr comment [PR_NUMBER] --body "$(cat <<'EOF'
|
|
717
|
-
## Billing Integrity Audit
|
|
718
|
-
|
|
719
|
-
### Billing Impact: [NONE/LOW/MEDIUM/HIGH]
|
|
720
|
-
|
|
721
|
-
### Flow Traced
|
|
722
|
-
1. Credit check: [file:line]
|
|
723
|
-
2. Pre-debit: [file:line]
|
|
724
|
-
3. Execution: [file:line]
|
|
725
|
-
4. Tier determination: [file:line] — tier_used = [how set]
|
|
726
|
-
5. Reconciliation: [file:line]
|
|
727
|
-
|
|
728
|
-
### Findings
|
|
729
|
-
| Issue | Confidence | Evidence | Revenue Impact |
|
|
730
|
-
|-------|------------|----------|----------------|
|
|
731
|
-
| ... | CONFIRMED/LIKELY | [code path] | [estimated impact] |
|
|
732
|
-
|
|
733
|
-
### What I Verified
|
|
734
|
-
- [ ] Traced credit deduction from API to worker
|
|
735
|
-
- [ ] Found reconciliation logic at [file:line]
|
|
736
|
-
- [ ] Verified tier_used source
|
|
737
|
-
- [ ] Checked idempotency
|
|
738
|
-
- [ ] Checked feature gates for rogue-gate regression (item 7)
|
|
739
|
-
|
|
740
|
-
### Files Reviewed
|
|
741
|
-
[List all billing-related files checked]
|
|
742
|
-
|
|
743
|
-
---
|
|
744
|
-
*Billing integrity audit*
|
|
745
|
-
|
|
746
|
-
<!-- REVIEW-FINDINGS-START -->
|
|
747
|
-
<!-- FINDING:BILL-1|CONFIDENCE|SEVERITY|file.py:line|Summary -->
|
|
748
|
-
<!-- (add one FINDING line per issue found — include ALL confidence levels) -->
|
|
749
|
-
<!-- REVIEW-FINDINGS-END -->
|
|
750
|
-
EOF
|
|
751
|
-
)"
|
|
752
|
-
```
|
|
753
|
-
|
|
754
|
-
**Structured Findings**: Include the structured findings block above at the end of your comment. Your prefix: `BILL`. See the Structured Findings Protocol section above for format rules.
|
|
755
|
-
```
|
|
756
|
-
|
|
757
|
-
### Coverage Matrix — BILL Agent
|
|
758
|
-
|
|
759
|
-
| Defect Category | Check Item(s) | Status | Ref |
|
|
760
|
-
|----------------|---------------|--------|-----|
|
|
761
|
-
| Credit check → debit → reconciliation flow integrity | Item 1 | COVERED | |
|
|
762
|
-
| tier_used accuracy (wrong tier charged) | Item 2 | COVERED | |
|
|
763
|
-
| Double-charging (pre-debit + reconciliation overlap) | Item 3 | COVERED | |
|
|
764
|
-
| Credit loss on scrape failure | Item 4 | COVERED | |
|
|
765
|
-
| Retry idempotency (double-debit) | Item 5 | COVERED | |
|
|
766
|
-
| Free scrape bypass paths | Item 6 | COVERED | |
|
|
767
|
-
| Rogue feature gate regression | Item 7 | COVERED | #278 |
|
|
768
|
-
| Webhook handler completeness | — | PARTIAL | #297 |
|
|
769
|
-
| Spend limit / budget cap accuracy | — | PARTIAL | #297 |
|
|
770
|
-
| Promo/voucher redemption integrity | — | GAP | |
|
|
771
|
-
|
|
772
|
-
---
|
|
773
|
-
|
|
774
|
-
### Agent: Concurrency & Race Condition Auditor
|
|
775
|
-
|
|
776
|
-
**Trigger**: CONCURRENCY domain detected OR BILLING domain detected
|
|
777
|
-
**Type**: `general-purpose` | **Model**: `sonnet`
|
|
778
|
-
|
|
779
|
-
**Prompt template:**
|
|
780
|
-
```
|
|
781
|
-
You are auditing PR #[PR_NUMBER] for concurrency bugs and race conditions in [PROJECT_NAME].
|
|
782
|
-
|
|
783
|
-
CRITICAL: This is a billing system. Double-spend = revenue loss.
|
|
784
|
-
|
|
785
|
-
## What to Look For
|
|
786
|
-
1. **Read-modify-write without locks**: `balance = get(); if balance >= cost: deduct(cost)`
|
|
787
|
-
2. **Non-atomic Redis operations**: Multiple Redis calls that should be a pipeline/transaction
|
|
788
|
-
3. **Missing FOR UPDATE**: DB queries that read-then-write without row locks
|
|
789
|
-
4. **Shared state in async**: Global/module-level state modified by concurrent requests
|
|
790
|
-
5. **Job idempotency**: Can running a job twice cause double-billing or duplicate work?
|
|
791
|
-
6. **Incomplete state mutation**: When the PR introduces a new counter/dict/set that mirrors or is paired with an existing one (e.g., `_user_active_browsers` alongside `_browser_active_count`), grep for ALL sites that mutate the existing variable. Each site must also handle the new variable, or document why not. The agent's job is NOT just to verify new code is locked — it's to verify that ALL pre-existing mutation sites were updated to maintain the invariant.
|
|
792
|
-
7. **Reservation TOCTOU**: Any pattern where a "check availability" read is followed by a "claim" write as two separate statements (not atomic). Required safe patterns:
|
|
793
|
-
- `SELECT ... FOR UPDATE SKIP LOCKED` (advisory lock on the row during the transaction)
|
|
794
|
-
- `UPDATE ... WHERE reserved_by IS NULL RETURNING id` (single atomic claim — no separate read)
|
|
795
|
-
- Database UNIQUE constraint on the reservation column (prevents duplicate claims at DB level)
|
|
796
|
-
|
|
797
|
-
Search for check-then-claim patterns in promo, voucher, or coupon redemption code:
|
|
798
|
-
```bash
|
|
799
|
-
grep -n "reserved_by\|voucher.*claim\|promo.*redeem\|coupon.*use" <billing_files> | head -20
|
|
800
|
-
# If found without FOR UPDATE or RETURNING in same transaction: CONFIRMED HIGH
|
|
801
|
-
```
|
|
802
|
-
If `WHERE reserved_by IS NULL` appears in a SELECT that is followed by a separate UPDATE (not in the same atomic statement), this is a CONFIRMED HIGH finding — two concurrent sessions can both pass the read check before either writes.
|
|
803
|
-
|
|
804
|
-
8. **Cross-service flag staleness**: When a discount or pricing flag (`is_byop`, `has_active_subscription`, `byop_discount`) is set by the API layer at job submission and read by the Worker layer at billing time, verify the flag is re-validated at debit time — not trusted from the queued job payload.
|
|
805
|
-
|
|
806
|
-
Search for discount flags passed through Redis/job payloads:
|
|
807
|
-
```bash
|
|
808
|
-
grep -rn "is_byop\|byop_discount\|discount.*flag\|flag.*discount" services/api/app/ services/worker/worker/ | head -20
|
|
809
|
-
# If the flag flows through a job payload and is not re-validated at billing: CONFIRMED HIGH
|
|
810
|
-
```
|
|
811
|
-
A race window exists when a flag is checked at submission but consumed at billing: the underlying condition (e.g., proxy existence, subscription status) may have changed between the two operations. The fix must re-validate the flag atomically at the point of debit, not rely on a stale value from the job payload.
|
|
812
|
-
|
|
813
|
-
## Safe Patterns in This Codebase
|
|
814
|
-
```bash
|
|
815
|
-
# Search for existing protections
|
|
816
|
-
grep -rn "with_for_update\|FOR UPDATE" services/
|
|
817
|
-
grep -rn "MULTI\|pipeline\|transaction" services/
|
|
818
|
-
grep -rn "distributed_lock\|acquire_lock" services/
|
|
819
|
-
```
|
|
820
|
-
|
|
821
|
-
## Verify State Completeness
|
|
822
|
-
For every new state variable (counter, dict, set) introduced by the PR:
|
|
823
|
-
1. Identify its "sibling" — the existing state variable it mirrors or is paired with
|
|
824
|
-
2. `grep -n "sibling_variable_name"` in the same module/file
|
|
825
|
-
3. For each mutation site of the sibling: verify the new variable is also mutated (or explicitly excluded with justification)
|
|
826
|
-
4. Flag any site that mutates the sibling but NOT the new variable — this is a CONFIRMED finding (invariant violation, not a style issue)
|
|
827
|
-
|
|
828
|
-
Example: PR adds `_user_active_browsers`. Sibling is `_browser_active_count`. Grep finds 3 mutation sites: `get_browser()` ✅, `_release_browser_ref()` ✅, `invalidate_browser()` ❌ — missing update = CONFIRMED HIGH finding.
|
|
829
|
-
|
|
830
|
-
## Cancellation Safety — Prove, Don't Reason
|
|
831
|
-
When reviewing `asyncio.shield`, `asyncio.wait_for`, or `Task.cancel()` patterns:
|
|
832
|
-
|
|
833
|
-
1. NEVER assert "this is correct" without identifying the specific CancelledError delivery point
|
|
834
|
-
2. For `asyncio.shield`: the outer `await asyncio.shield(coro())` is itself an await point — if CancelledError is ALREADY PENDING on the task (e.g., injected by `asyncio.wait_for` timeout), it fires HERE before the inner coroutine starts. The shield does not protect against a pending cancellation.
|
|
835
|
-
3. For `asyncio.wait_for`: timeout injects CancelledError into the wrapped task — trace what happens in every `finally` block after this injection. Does any `finally` block contain an `await`? That await is also a cancellation point.
|
|
836
|
-
4. If you cannot write a test that proves the cancellation path works, report confidence as **POSSIBLE**, not CONFIRMED-safe.
|
|
837
|
-
5. Flag any `asyncio.shield` usage with: "Requires test: simulate CancelledError pending before shield await"
|
|
838
|
-
|
|
839
|
-
## Verify Before Claiming
|
|
840
|
-
- Check if the code you're analyzing is already protected by locks/transactions elsewhere
|
|
841
|
-
- Read the FULL function scope, not just the diff
|
|
842
|
-
- For state completeness: read ALL mutation sites of the sibling variable, not just the ones modified in the PR diff
|
|
843
|
-
- For asyncio patterns: identify the exact await point where CancelledError fires — do not reason about "what the code intends"
|
|
844
|
-
|
|
845
|
-
## Post Findings
|
|
846
|
-
```bash
|
|
847
|
-
gh pr comment [PR_NUMBER] --body "$(cat <<'EOF'
|
|
848
|
-
## Concurrency & Race Condition Audit
|
|
849
|
-
|
|
850
|
-
### Risk Level: [LOW/MEDIUM/HIGH/CRITICAL]
|
|
851
59
|
|
|
852
|
-
|
|
853
|
-
| Pattern | Location | Protected? | Confidence | Evidence |
|
|
854
|
-
|---------|----------|------------|------------|----------|
|
|
855
|
-
| read-modify-write | file:line | No | CONFIRMED | [code path] |
|
|
60
|
+
## Loading Instructions for Phase 3C
|
|
856
61
|
|
|
857
|
-
|
|
858
|
-
- FOR UPDATE: [found/not found]
|
|
859
|
-
- Redis transactions: [found/not found]
|
|
860
|
-
- Distributed locks: [found/not found]
|
|
861
|
-
- Idempotency keys: [found/not found]
|
|
62
|
+
Replace the single `Read: $FORGE_HOME/commands/review-pr-agents.md` call with:
|
|
862
63
|
|
|
863
|
-
### Asyncio Cancellation Safety
|
|
864
|
-
| Pattern | Location | Outer await cancellation-safe? | Confidence | Evidence |
|
|
865
|
-
|---------|----------|-------------------------------|------------|----------|
|
|
866
|
-
| asyncio.shield | file:line | Yes/No/POSSIBLE — [explain CancelledError delivery point] | CONFIRMED/POSSIBLE | [code path] |
|
|
867
|
-
|
|
868
|
-
### State Completeness Check
|
|
869
|
-
| New Variable | Sibling Variable | Mutation Sites Found | All Sites Updated? |
|
|
870
|
-
|-------------|-----------------|---------------------|-------------------|
|
|
871
|
-
| [new_var] | [sibling_var] | [count] | Yes/No — [missing sites if No] |
|
|
872
|
-
|
|
873
|
-
### Files Reviewed
|
|
874
|
-
[List files checked]
|
|
875
|
-
|
|
876
|
-
---
|
|
877
|
-
*Concurrency audit*
|
|
878
|
-
|
|
879
|
-
<!-- REVIEW-FINDINGS-START -->
|
|
880
|
-
<!-- FINDING:CONC-1|CONFIDENCE|SEVERITY|file.py:line|Summary -->
|
|
881
|
-
<!-- (add one FINDING line per issue found — include ALL confidence levels) -->
|
|
882
|
-
<!-- REVIEW-FINDINGS-END -->
|
|
883
|
-
EOF
|
|
884
|
-
)"
|
|
885
|
-
```
|
|
886
|
-
|
|
887
|
-
**Structured Findings**: Include the structured findings block above at the end of your comment. Your prefix: `CONC`. See the Structured Findings Protocol section above for format rules.
|
|
888
|
-
```
|
|
889
|
-
|
|
890
|
-
### Coverage Matrix — CONC Agent
|
|
891
|
-
|
|
892
|
-
| Defect Category | Check Item(s) | Status | Ref |
|
|
893
|
-
|----------------|---------------|--------|-----|
|
|
894
|
-
| Read-modify-write without locks | Item 1 | COVERED | |
|
|
895
|
-
| Non-atomic Redis operations | Item 2 | COVERED | |
|
|
896
|
-
| Missing FOR UPDATE on read-then-write | Item 3 | COVERED | |
|
|
897
|
-
| Shared state in async handlers | Item 4 | COVERED | |
|
|
898
|
-
| Job idempotency (double-billing) | Item 5 | COVERED | |
|
|
899
|
-
| Incomplete state mutation (new counter/dict) | Item 6 | COVERED | |
|
|
900
|
-
| Reservation TOCTOU (check-then-claim) | Item 7 | COVERED | #298 |
|
|
901
|
-
| Cross-service flag staleness | Item 8 | COVERED | #298 |
|
|
902
|
-
| asyncio.shield cancellation safety | Cancellation Safety | COVERED | |
|
|
903
|
-
| Distributed lock timeout / deadlock | — | GAP | |
|
|
904
|
-
| Connection pool exhaustion under concurrency | — | GAP | |
|
|
905
|
-
|
|
906
|
-
---
|
|
907
|
-
|
|
908
|
-
### Agent: Scraper Logic Auditor
|
|
909
|
-
|
|
910
|
-
**Trigger**: SCRAPING domain detected
|
|
911
|
-
**Type**: `codebase-explorer` | **Model**: `sonnet`
|
|
912
|
-
|
|
913
|
-
**Prompt template:**
|
|
914
|
-
```
|
|
915
|
-
You are auditing PR #[PR_NUMBER] for domain-specific logic correctness in [PROJECT_NAME].
|
|
916
|
-
|
|
917
|
-
## Project Domain Architecture
|
|
918
|
-
|
|
919
|
-
[DOMAIN_CONTEXT]
|
|
920
|
-
|
|
921
|
-
If no domain context is configured above, derive the architecture from the changed files: read the primary consumer/worker/handler files to understand the execution model.
|
|
922
|
-
|
|
923
|
-
## What to Verify
|
|
924
|
-
1. **Tier selection**: Is the starting tier correct? Does escalation logic work?
|
|
925
|
-
2. **Tier-billing alignment**: Does `tier_used` match what's charged?
|
|
926
|
-
3. **Timeout handling**: Are timeouts configured correctly per tier?
|
|
927
|
-
4. **Error propagation**: Do scrape failures propagate correctly?
|
|
928
|
-
5. **Playbook authority**: Does `playbook_min_tier_is_authoritative` work as expected?
|
|
929
|
-
6. **Content validation**: Does escalation trigger on empty/blocked content?
|
|
930
|
-
7. **Detection keyword consistency**: If the PR modifies `CHALLENGE_KEYWORDS`, `COMMON_COOKIES`, or ANY frozenset/tuple/list of string identifiers used for bot detection, WAF markers, or anti-detection cookies in `anti_detection/`, `consumers/`, or `browser/`:
|
|
931
|
-
- **Cross-reference sibling sets**: List all other detection-related constants in the worker service (`grep -rn "^[A-Z_]* = frozenset\|^[A-Z_]* = tuple\|^[A-Z_]* = \[" services/worker/worker/`). For each entry in the modified set, verify it appears in all relevant sibling sets. Flag any entry present in one set but absent from a functionally related sibling (e.g., `ak_bmsc` in `CHALLENGE_KEYWORDS` but not in `COMMON_COOKIES`).
|
|
932
|
-
- **Intra-file comment/code drift**: Read the full file. For any identifier mentioned in comments, docstrings, or commit messages (e.g., `_abck`, `ak_bmsc`) that is NOT present in the actual set literal, flag it as a likely typo or omission. This is CONFIRMED HIGH — comments describe the intended state; the code contradicts it.
|
|
933
|
-
- **Vendor documentation alignment**: If the PR mentions a specific vendor (Akamai, Cloudflare, DataDome, etc.), grep for the vendor's known markers across the entire worker service and verify the modified set is complete.
|
|
934
|
-
8. **Capacity constants**: Any hardcoded numeric constant defining memory limits, pool sizes, timeouts, or thresholds (e.g., `_BROWSER_ESTIMATED_MB = 200`, `MAX_BROWSERS = 5`, `POOL_SIZE = 10`):
|
|
935
|
-
- Does the PR or commit message cite a measurement source (e.g., "observed peak 744-890MB", "benchmarked at N req/s", linked issue/PR)?
|
|
936
|
-
- Search git history for prior art that may contradict the constant:
|
|
937
|
-
```bash
|
|
938
|
-
# Search git log for related measurements — adapt keywords to the constant's domain
|
|
939
|
-
git log --all -20 --oneline --grep="memory" --grep="browser" 2>/dev/null | head -10
|
|
940
|
-
git log --all -20 --oneline --grep="$(echo {CONSTANT_NAME} | tr '_' ' ' | tr '[:upper:]' '[:lower:]')" 2>/dev/null | head -5
|
|
941
|
-
# Check recent PRs that touched the same file for prior measurement context
|
|
942
|
-
git log --oneline --all -20 -- {CHANGED_FILE} 2>/dev/null | head -10
|
|
943
|
-
```
|
|
944
|
-
- If no measurement source is cited and git history contains contradicting data (e.g., a prior PR documenting real observed values): flag as **CONFIRMED** with note "Hardcoded constant contradicts measured data in git history — verify against production metrics before deploy"
|
|
945
|
-
- If no measurement source is cited and no contradicting git history found: flag as **POSSIBLE** with note "Hardcoded constant without measurement annotation — verify against production metrics"
|
|
946
|
-
- If a measurement source IS cited (comment, commit message, or linked issue): no flag needed
|
|
947
|
-
9. **API gate semantic correctness** (conditional — trigger: PR modifies any router file that gates existing request parameters behind a new condition):
|
|
948
|
-
When a PR adds or changes a condition that gates a set of request fields behind a resource requirement (API key, feature flag, tier check), **verify independently for EACH parameter in the condition** whether it actually requires the gated resource:
|
|
949
|
-
- For each field in the gate condition (`if request.X or request.Y:`, `if X and Y`): ask "Was this field previously handled without this gate? What is the behavioral change for existing users who send this field without satisfying the gate?"
|
|
950
|
-
- Distinguish resource-dependent fields (e.g., those that invoke an external LLM or BYOK key at execution time) from resource-independent fields (those processed deterministically without the gated resource). A gate that covers both classes is incorrect — the resource-independent field must be routed separately.
|
|
951
|
-
- Inspect the pre-existing code path for each field: search the router file and worker for the field's pre-change handling (`grep -rn "request\\.{field_name}" services/api/ services/worker/`). If the pre-existing path did NOT require the gated resource, the inclusion of that field in the gate is a **CONFIRMED HIGH** behavioral regression for existing callers.
|
|
952
|
-
- If different fields in the condition reach different execution paths (one LLM-dependent, one not), the condition must be split — gate only the fields that require the resource.
|
|
953
|
-
|
|
954
|
-
```bash
|
|
955
|
-
# Identify gate conditions in changed router files
|
|
956
|
-
gh pr diff [PR_NUMBER] -- "services/api/app/routers/*.py" | grep "^\+" | grep -E "if request\.|if.*or.*request\." | head -20
|
|
957
|
-
|
|
958
|
-
# For each gated field, check its pre-existing handling path
|
|
959
|
-
# (adapt FIELD_NAME to each field appearing in the changed condition)
|
|
960
|
-
git log --all --oneline -10 -- services/api/app/routers/ | head -10
|
|
961
|
-
grep -rn "FIELD_NAME" services/api/app/ services/worker/worker/ | grep -v "^\s*#" | head -20
|
|
962
|
-
```
|
|
963
|
-
<!-- Added: forge#382 -->
|
|
964
|
-
10. **Cross-component gate tracing** (conditional — trigger: PR touches both a router file that injects a field into a job payload AND a worker file that reads that field at execution time):
|
|
965
|
-
When a worker-layer review identifies that a resource field (API key, LLM key, BYOK credential) is resolved at use-time from a job payload, the review MUST also trace to the API-layer gate condition that controls injection of that field. A gate can fail at the API layer for logic that executes at the worker layer — both components must be reviewed together.
|
|
966
|
-
- Find the API router that creates the job payload and injects the field. Read the condition that controls whether the field is populated.
|
|
967
|
-
- Verify the gate condition is semantically correct for ALL request fields that flow through it — not just the primary feature field.
|
|
968
|
-
- If the API gate fires before job enqueue and the worker gate fires after dequeue, a misconfigured API gate will silently reject valid requests before the worker ever sees them.
|
|
969
|
-
|
|
970
|
-
```bash
|
|
971
|
-
# Find all sites where the key/credential field is injected into the job payload
|
|
972
|
-
grep -rn "byok_api_key\|llm_key\|extraction_key\|api_key" services/api/app/routers/ | grep -v "^\s*#" | head -20
|
|
973
|
-
|
|
974
|
-
# For each injection site, read the surrounding conditional context (±15 lines)
|
|
975
|
-
# Then cross-reference: what request fields trigger the injection condition?
|
|
976
|
-
# Are ALL those fields genuinely dependent on the injected resource?
|
|
977
|
-
|
|
978
|
-
# Check the worker consumption site for the same field
|
|
979
|
-
grep -rn "byok_api_key\|llm_key\|extraction_key\|api_key" services/worker/worker/ | grep -v "^\s*#" | head -20
|
|
980
|
-
```
|
|
981
|
-
<!-- Added: forge#382 -->
|
|
982
|
-
|
|
983
|
-
```bash
|
|
984
|
-
# Find all detection-related string constants in the worker service
|
|
985
|
-
grep -rn "^[A-Z_]* = frozenset\(\|^[A-Z_]* = tuple(\|^[A-Z_]* = \[" services/worker/worker/ 2>/dev/null
|
|
986
|
-
|
|
987
|
-
# For each modified set, check if its entries appear in sibling detection files
|
|
988
|
-
for f in [CHANGED_ANTI_DETECTION_FILES]; do
|
|
989
|
-
grep -oE "'[a-z_][a-z0-9_-]{3,}'" "$f" | tr -d "'" | sort -u | while read entry; do
|
|
990
|
-
grep -rl "'$entry'\|\"$entry\"" services/worker/worker/ | grep -v "^$f$" | grep '\.py$' || \
|
|
991
|
-
echo "SCRP: '$entry' in $f not found in any sibling worker file — verify if cross-module sync needed"
|
|
992
|
-
done
|
|
993
|
-
# Check comment vs set drift
|
|
994
|
-
grep -oE "#.*\`_?[a-z][a-z0-9_-]+\`" "$f" | grep -oE "\`[^']+\`" | tr -d '`' | while read ref; do
|
|
995
|
-
grep -q "'$ref'\|\"$ref\"" "$f" || \
|
|
996
|
-
echo "SCRP: '$ref' mentioned in comment in $f but absent from string sets — possible typo or omission (HIGH)"
|
|
997
|
-
done
|
|
998
|
-
done
|
|
999
|
-
```
|
|
1000
|
-
|
|
1001
|
-
## Cross-Service Consistency
|
|
1002
|
-
- API-side: Job creation, initial tier selection
|
|
1003
|
-
- Worker-side: Actual execution, tier escalation, result handling
|
|
1004
|
-
- Verify consistency between services
|
|
1005
|
-
|
|
1006
|
-
## Post Findings
|
|
1007
|
-
```bash
|
|
1008
|
-
gh pr comment [PR_NUMBER] --body "$(cat <<'EOF'
|
|
1009
|
-
## Scraper Logic Audit
|
|
1010
|
-
|
|
1011
|
-
### Components Affected: [list the specific service/worker/api components touched — e.g. job-queue/rate-limiter/validation/gate-semantics/cross-component-gate]
|
|
1012
|
-
|
|
1013
|
-
### Tier Flow Analysis
|
|
1014
|
-
[Trace the tier selection and escalation with file:line references]
|
|
1015
|
-
|
|
1016
|
-
### Detection Keyword Consistency
|
|
1017
|
-
[If any detection set was modified: table of sibling sets checked, entries present/absent in each, comment/code drift found]
|
|
1018
|
-
| Set Name | File | Entries | Missing from Sibling? | Comment Drift? |
|
|
1019
|
-
|----------|------|---------|----------------------|----------------|
|
|
1020
|
-
| CHALLENGE_KEYWORDS | file:line | N entries | [list missing] | [yes/no] |
|
|
1021
|
-
|
|
1022
|
-
### Capacity Constants
|
|
1023
|
-
[If any capacity constants were added or modified: list each constant, its value, whether a measurement source was cited, and any contradicting git history]
|
|
1024
|
-
| Constant | Value | Measurement Source Cited? | Git History Finding |
|
|
1025
|
-
|----------|-------|--------------------------|---------------------|
|
|
1026
|
-
| _BROWSER_ESTIMATED_MB | 200 | No | PR #4958 documented 744-890MB observed peak |
|
|
1027
|
-
|
|
1028
|
-
### API Gate Semantic Correctness
|
|
1029
|
-
[If any router file gates request fields behind a new condition: for each field in the condition, state whether it requires the gated resource and whether the pre-existing behavior is preserved. "N/A — no gate condition changes in diff" is acceptable if no gate conditions changed.]
|
|
1030
|
-
|
|
1031
|
-
### Cross-Component Gate Tracing
|
|
1032
|
-
[If worker layer injects or reads a resource field from the job payload: state the API-layer gate condition that controls injection, the fields it gates, and whether each field genuinely requires the gated resource. "N/A — no cross-component key/gate flow in diff" is acceptable if not triggered.]
|
|
1033
|
-
|
|
1034
|
-
### Findings
|
|
1035
|
-
| Category | Issue | Location | Confidence |
|
|
1036
|
-
|----------|-------|----------|------------|
|
|
1037
|
-
| Tier logic | [issue] | file:line | CONFIRMED |
|
|
1038
|
-
| Detection keywords | [issue] | file:line | CONFIRMED |
|
|
1039
|
-
| Capacity constant | [issue] | file:line | POSSIBLE |
|
|
1040
|
-
| API gate semantics | [issue] | file:line | CONFIRMED |
|
|
1041
|
-
| Cross-component gate | [issue] | file:line | CONFIRMED |
|
|
1042
|
-
|
|
1043
|
-
### Cross-Service Consistency
|
|
1044
|
-
[If changes span services, verify they're consistent]
|
|
1045
|
-
|
|
1046
|
-
### Files Reviewed
|
|
1047
|
-
[List files checked]
|
|
1048
|
-
|
|
1049
|
-
---
|
|
1050
|
-
*Scraper logic audit*
|
|
1051
|
-
|
|
1052
|
-
<!-- REVIEW-FINDINGS-START -->
|
|
1053
|
-
<!-- FINDING:SCRP-1|CONFIDENCE|SEVERITY|file.py:line|Summary -->
|
|
1054
|
-
<!-- (add one FINDING line per issue found — include ALL confidence levels) -->
|
|
1055
|
-
<!-- REVIEW-FINDINGS-END -->
|
|
1056
|
-
EOF
|
|
1057
|
-
)"
|
|
1058
|
-
```
|
|
1059
|
-
|
|
1060
|
-
**Structured Findings**: Include the structured findings block above at the end of your comment. Your prefix: `SCRP`. See the Structured Findings Protocol section above for format rules.
|
|
1061
|
-
```
|
|
1062
|
-
|
|
1063
|
-
### Coverage Matrix — SCRP Agent
|
|
1064
|
-
|
|
1065
|
-
| Defect Category | Check Item(s) | Status | Ref |
|
|
1066
|
-
|----------------|---------------|--------|-----|
|
|
1067
|
-
| Tier selection correctness | Item 1 | COVERED | |
|
|
1068
|
-
| Tier-billing alignment | Item 2 | COVERED | |
|
|
1069
|
-
| Timeout configuration per tier | Item 3 | COVERED | |
|
|
1070
|
-
| Error propagation on scrape failure | Item 4 | COVERED | |
|
|
1071
|
-
| Playbook authority logic | Item 5 | COVERED | |
|
|
1072
|
-
| Content validation / escalation trigger | Item 6 | COVERED | |
|
|
1073
|
-
| Detection keyword cross-set consistency | Item 7 | COVERED | |
|
|
1074
|
-
| Capacity constant measurement verification | Item 8 | COVERED | |
|
|
1075
|
-
| Cross-service consistency — tier logic (API vs Worker) | Cross-Service section | COVERED | |
|
|
1076
|
-
| API gate semantic correctness per parameter | Item 9 | COVERED | forge#382 |
|
|
1077
|
-
| Cross-component gate tracing (API gate ↔ worker injection) | Item 10 | COVERED | forge#382 |
|
|
1078
|
-
| Playwright resource cleanup on failure | — | GAP | |
|
|
1079
|
-
| Domain playbook override conflicts | — | GAP | |
|
|
1080
|
-
|
|
1081
|
-
---
|
|
1082
|
-
|
|
1083
|
-
### Agent: Frontend Quality Auditor
|
|
1084
|
-
|
|
1085
|
-
**Trigger**: WEB service touched
|
|
1086
|
-
**Type**: `codebase-explorer` | **Model**: `sonnet`
|
|
1087
|
-
|
|
1088
|
-
**Prompt template:**
|
|
1089
|
-
```
|
|
1090
|
-
You are auditing PR #[PR_NUMBER] for frontend quality in [PROJECT_NAME].
|
|
1091
|
-
|
|
1092
|
-
## Project Frontend Conventions
|
|
1093
|
-
|
|
1094
|
-
[DOMAIN_CONTEXT]
|
|
1095
|
-
|
|
1096
|
-
If no frontend context is configured above, derive conventions from the changed files: check package.json for framework versions and look at existing patterns in surrounding components.
|
|
1097
|
-
|
|
1098
|
-
## What to Check
|
|
1099
|
-
1. **Server vs Client components**: Is 'use client' used appropriately? Could this be a Server Component instead?
|
|
1100
|
-
2. **React lifecycle**: Missing useEffect cleanup (subscriptions, timers, AbortControllers not cancelled on unmount). This is the #1 frontend defect — causes memory leaks and stale state.
|
|
1101
|
-
3. **useCallback/useMemo deps**: Dependencies must be stable values (ids, primitives), NOT full objects (SWR responses, arrays). An unstable dep causes infinite re-render loops.
|
|
1102
|
-
4. **Data fetching**: Loading/error states handled, stale data invalidated, race conditions in parallel fetches
|
|
1103
|
-
5. **Type safety**: Zod schemas match API response types, no `any` types. Check that component props match what the parent passes — wrong prop types (e.g., passing a component reference instead of a rendered element) cause build failures.
|
|
1104
|
-
6. **Component imports**: Verify imported components actually exist. Check `@/components/ui/*` imports against what's installed via shadcn. Missing components pass `tsc` if types exist but fail `next build`.
|
|
1105
|
-
7. **XSS**: User input rendered with `dangerouslySetInnerHTML` without sanitization
|
|
1106
|
-
8. **Accessibility**: Interactive elements have proper labels, keyboard navigation works
|
|
1107
|
-
9. **Performance**: Large bundles, unnecessary client-side rendering, missing dynamic imports
|
|
1108
|
-
10. **Hook provider scope**: For any new `useX()` hook call added to a component, verify the hook is safe to call outside its provider. If `useX` internally calls `useContext(XContext)` and throws when context is undefined (i.e., the hook contains `if (!ctx) throw new Error(...)`), check ALL mount sites of the component across the codebase. If the component is used in both authenticated routes (e.g., `app/dashboard/`) AND public routes (e.g., `app/(public)/`, `app/playground/`) that do not wrap children in the provider, the hook call will crash the public route with a React error boundary cascade. Flag as CONFIRMED HIGH if a public mount site exists without the provider. <!-- Added: forge#381 -->
|
|
1109
|
-
|
|
1110
|
-
## Post Findings
|
|
1111
|
-
```bash
|
|
1112
|
-
gh pr comment [PR_NUMBER] --body "$(cat <<'EOF'
|
|
1113
|
-
## Frontend Quality Audit
|
|
1114
|
-
|
|
1115
|
-
### Quality Level: [GOOD/ACCEPTABLE/NEEDS WORK]
|
|
1116
|
-
|
|
1117
|
-
### Findings
|
|
1118
|
-
| Category | Issue | File:Line | Severity |
|
|
1119
|
-
|----------|-------|-----------|----------|
|
|
1120
|
-
| React | missing useEffect dep | file:line | MEDIUM |
|
|
1121
|
-
| A11y | button without label | file:line | LOW |
|
|
1122
|
-
|
|
1123
|
-
### Component Architecture
|
|
1124
|
-
[Server vs Client component usage assessment]
|
|
1125
|
-
|
|
1126
|
-
### Files Reviewed
|
|
1127
|
-
[List files checked]
|
|
1128
|
-
|
|
1129
|
-
---
|
|
1130
|
-
*Frontend quality audit*
|
|
1131
|
-
|
|
1132
|
-
<!-- REVIEW-FINDINGS-START -->
|
|
1133
|
-
<!-- FINDING:FE-1|CONFIDENCE|SEVERITY|file.py:line|Summary -->
|
|
1134
|
-
<!-- (add one FINDING line per issue found — include ALL confidence levels) -->
|
|
1135
|
-
<!-- REVIEW-FINDINGS-END -->
|
|
1136
|
-
EOF
|
|
1137
|
-
)"
|
|
1138
|
-
```
|
|
1139
|
-
|
|
1140
|
-
**Structured Findings**: Include the structured findings block above at the end of your comment. Your prefix: `FE`. See the Structured Findings Protocol section above for format rules.
|
|
1141
|
-
```
|
|
1142
|
-
|
|
1143
|
-
### Coverage Matrix — FE Agent
|
|
1144
|
-
|
|
1145
|
-
| Defect Category | Check Item(s) | Status | Ref |
|
|
1146
|
-
|----------------|---------------|--------|-----|
|
|
1147
|
-
| Unnecessary 'use client' directive | Item 1 | COVERED | |
|
|
1148
|
-
| Missing useEffect cleanup (memory leaks) | Item 2 | COVERED | |
|
|
1149
|
-
| Unstable useCallback/useMemo dependencies | Item 3 | COVERED | |
|
|
1150
|
-
| Missing loading/error states | Item 4 | COVERED | |
|
|
1151
|
-
| Type safety (Zod schema / prop mismatch) | Item 5 | COVERED | |
|
|
1152
|
-
| Missing component imports (shadcn) | Item 6 | COVERED | |
|
|
1153
|
-
| XSS via dangerouslySetInnerHTML | Item 7 | COVERED | |
|
|
1154
|
-
| Accessibility (labels, keyboard nav) | Item 8 | COVERED | |
|
|
1155
|
-
| Performance (bundle size, dynamic imports) | Item 9 | COVERED | |
|
|
1156
|
-
| Hook outside provider scope (crash on public routes) | Item 10 | COVERED | forge#381 |
|
|
1157
|
-
| Server Component data leakage to client | — | GAP | |
|
|
1158
|
-
| Next.js App Router caching pitfalls | — | GAP | |
|
|
1159
|
-
|
|
1160
|
-
---
|
|
1161
|
-
|
|
1162
|
-
### Agent: API Design & Consistency Auditor
|
|
1163
|
-
|
|
1164
|
-
**Trigger**: New or modified routers/routes, OR SDK/OpenAPI files changed (`sdk/`, `openapi*.json`, `openapi-versions/`)
|
|
1165
|
-
**Type**: `general-purpose` | **Model**: `sonnet`
|
|
1166
|
-
|
|
1167
|
-
**Prompt template:**
|
|
1168
|
-
```
|
|
1169
|
-
You are auditing PR #[PR_NUMBER] for API design consistency in [PROJECT_NAME].
|
|
1170
|
-
|
|
1171
|
-
## Project API Conventions
|
|
1172
|
-
|
|
1173
|
-
[DOMAIN_CONTEXT]
|
|
1174
|
-
|
|
1175
|
-
If no API context is configured above, derive conventions from the changed files: read the main router/application entrypoint to understand registration patterns, and check neighboring endpoints for error format and naming conventions.
|
|
1176
|
-
|
|
1177
|
-
## What to Check
|
|
1178
|
-
1. **Registration**: Is the new router registered in main.py?
|
|
1179
|
-
2. **Schema completeness**: Request/response models defined? Types correct?
|
|
1180
|
-
3. **Error handling**: Consistent error responses? HTTPException with proper status codes?
|
|
1181
|
-
4. **Endpoint naming**: RESTful conventions? Consistent with existing endpoints?
|
|
1182
|
-
5. **Query parameters**: Validated? Reasonable defaults?
|
|
1183
|
-
6. **Response format**: Consistent with other endpoints in the same domain?
|
|
1184
|
-
7. **External response type-safety**: Any code that consumes an external API response (HTTP client calls, SDK calls, third-party service responses) must guard against unexpected shapes. Flag direct dict/attribute access on an external response without a prior `isinstance` check or `None` guard — the response may be a dict, list, `None`, or an error string depending on the upstream service's behavior. Look for patterns like `data["key"]` or `data.field` immediately after `requests.get(...)`, `httpx.get(...)`, `await client.get(...)`, or similar calls without a guard. Exception: internal Pydantic-validated models are safe; only flag unvalidated external payloads.
|
|
1185
|
-
8. **Code generator field coverage**: Any function that builds a code snippet, SDK usage example, or serialized representation of a model must include ALL fields defined in the relevant Pydantic model or schema. Flag generators where the field list is hardcoded rather than derived from the model, or where the model has gained new fields that the generator does not emit. Search for functions named `generate_*`, `build_snippet_*`, `get_code_*`, `example_*`, or similar that reference a model class — compare their emitted fields against the model's field list.
|
|
1186
|
-
9. **New-field propagation**: When a PR adds a new field to a Pydantic model or database schema, verify that all downstream consumers are updated: serializers, snippet generators, SDK example builders, and any function that enumerates the model's fields. Search for all locations that reference the model class by name and check whether they handle the new field.
|
|
1187
|
-
|
|
1188
|
-
10. **Cross-PR SDK/schema consistency** (MANDATORY when PR touches `sdk/`, `openapi*.json`, or `openapi-versions/`): Documentation PRs that update SDK or OpenAPI files must be checked against recently-merged PRs to the same base branch. A concurrent schema PR may have already changed the API behavior being documented — producing contradictory docs that tell users to use methods the API now rejects.
|
|
1189
|
-
|
|
1190
|
-
```bash
|
|
1191
|
-
# Get the PR's base branch
|
|
1192
|
-
BASE=$(gh pr view [PR_NUMBER] --json baseRefName --jq '.baseRefName')
|
|
1193
|
-
|
|
1194
|
-
# Find PRs merged to this base in the last 48 hours
|
|
1195
|
-
CUTOFF="$(date -u -d '48 hours ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -v-48H +%Y-%m-%dT%H:%M:%SZ)"
|
|
1196
|
-
RECENT_MERGED=$(gh pr list --base "$BASE" --state merged --limit 20 \
|
|
1197
|
-
--json number,title,mergedAt,files \
|
|
1198
|
-
2>/dev/null | jq --arg cutoff "$CUTOFF" \
|
|
1199
|
-
'.[] | select(.mergedAt > $cutoff) | {number, title, schema_files: [.files[].path | select(test("schemas?/|scrape\\.py|models\\.py"))]}' \
|
|
1200
|
-
2>/dev/null | head -40)
|
|
1201
|
-
|
|
1202
|
-
echo "Recent merged PRs to $BASE (last 48h): $RECENT_MERGED"
|
|
1203
|
-
|
|
1204
|
-
# For each recent PR that touched schema files, check if Literal types changed
|
|
1205
|
-
for PR_NUM in $(echo "$RECENT_MERGED" | grep -oP '"number":\s*\K\d+'); do
|
|
1206
|
-
SCHEMA_DIFF=$(gh pr diff "$PR_NUM" 2>/dev/null | grep -E '^\+.*Literal\[|^\-.*Literal\[' | grep -v '^\+\+\+\|^---')
|
|
1207
|
-
if [ -n "$SCHEMA_DIFF" ]; then
|
|
1208
|
-
echo "SCHEMA CHANGE in PR #$PR_NUM (already merged to $BASE):"
|
|
1209
|
-
echo "$SCHEMA_DIFF"
|
|
1210
|
-
echo "--- Verify this PR's SDK/OpenAPI docs are consistent with the above Literal change ---"
|
|
1211
|
-
fi
|
|
1212
|
-
done
|
|
1213
|
-
```
|
|
1214
|
-
|
|
1215
|
-
For each schema Literal change found in recently-merged PRs:
|
|
1216
|
-
- Read the diff of THIS PR (PR [PR_NUMBER]) to see what values the SDK/OpenAPI files now document
|
|
1217
|
-
- If this PR's SDK documentation still lists values that the schema change removed (e.g., SDK JSDoc says `DELETE` is supported but schema now only accepts `GET`/`POST`) → this is a **CONFIRMED HIGH** finding
|
|
1218
|
-
- If this PR's SDK documentation says DELETE is "use with caution" but the schema already returns 422 for DELETE → this is a **CONFIRMED HIGH** finding (false reassurance)
|
|
1219
|
-
- Pattern: `sdk/*/client.py` `_valid_methods` list, TypeScript JSDoc `@param` literals, `openapi*.json` enum arrays — all must be consistent with the API schema's current `Literal[...]` type
|
|
1220
|
-
|
|
1221
|
-
## Cross-Reference
|
|
1222
|
-
```bash
|
|
1223
|
-
# See how existing routers are structured
|
|
1224
|
-
grep -rn "APIRouter\|include_router" services/api/app/
|
|
1225
|
-
# Check existing patterns
|
|
1226
|
-
ls services/api/app/routers/
|
|
1227
|
-
# Find external response consumers (unguarded dict/attribute access after HTTP calls)
|
|
1228
|
-
grep -rn "\.get\|\.post\|\.put\|\.delete\|httpx\|requests\." services/api/app/ | grep -v "test_\|#"
|
|
1229
|
-
# Find code/snippet generators
|
|
1230
|
-
grep -rn "def generate_\|def build_snippet\|def get_code_\|def.*example\b" services/api/app/ services/worker/
|
|
1231
|
-
# Find all locations that reference a modified model class
|
|
1232
|
-
grep -rn "{ModelClassName}" services/api/app/ services/worker/ web/src/
|
|
1233
|
-
# SDK method lists (for cross-PR check #10)
|
|
1234
|
-
grep -rn "_valid_methods\|Literal\[" sdk/ 2>/dev/null | head -20
|
|
1235
|
-
grep -rn "Literal\[" services/api/app/schemas/ 2>/dev/null | head -20
|
|
1236
64
|
```
|
|
1237
|
-
|
|
1238
|
-
|
|
1239
|
-
```bash
|
|
1240
|
-
gh pr comment [PR_NUMBER] --body "$(cat <<'EOF'
|
|
1241
|
-
## API Design & Consistency Audit
|
|
1242
|
-
|
|
1243
|
-
### New/Modified Endpoints
|
|
1244
|
-
| Endpoint | Method | Registered? | Schema? | Auth? | Consistent? |
|
|
1245
|
-
|----------|--------|-------------|---------|-------|-------------|
|
|
1246
|
-
| /path | POST | Yes/No | Yes/No | Yes/No | Yes/No |
|
|
1247
|
-
|
|
1248
|
-
### Consistency Issues
|
|
1249
|
-
[Any deviations from established patterns]
|
|
1250
|
-
|
|
1251
|
-
### Files Reviewed
|
|
1252
|
-
[List files checked]
|
|
1253
|
-
|
|
1254
|
-
---
|
|
1255
|
-
*API design audit*
|
|
1256
|
-
|
|
1257
|
-
<!-- REVIEW-FINDINGS-START -->
|
|
1258
|
-
<!-- FINDING:API-1|CONFIDENCE|SEVERITY|file.py:line|Summary -->
|
|
1259
|
-
<!-- (add one FINDING line per issue found — include ALL confidence levels) -->
|
|
1260
|
-
<!-- REVIEW-FINDINGS-END -->
|
|
1261
|
-
EOF
|
|
1262
|
-
)"
|
|
1263
|
-
```
|
|
1264
|
-
|
|
1265
|
-
**Structured Findings**: Include the structured findings block above at the end of your comment. Your prefix: `API`. See the Structured Findings Protocol section above for format rules.
|
|
1266
|
-
```
|
|
1267
|
-
|
|
1268
|
-
### Coverage Matrix — API Agent
|
|
1269
|
-
|
|
1270
|
-
| Defect Category | Check Item(s) | Status | Ref |
|
|
1271
|
-
|----------------|---------------|--------|-----|
|
|
1272
|
-
| Unregistered router | Item 1 | COVERED | |
|
|
1273
|
-
| Missing request/response schema | Item 2 | COVERED | |
|
|
1274
|
-
| Inconsistent error handling | Item 3 | COVERED | |
|
|
1275
|
-
| Non-RESTful endpoint naming | Item 4 | COVERED | |
|
|
1276
|
-
| Unvalidated query parameters | Item 5 | COVERED | |
|
|
1277
|
-
| Inconsistent response format | Item 6 | COVERED | |
|
|
1278
|
-
| External response type-safety | Item 7 | COVERED | |
|
|
1279
|
-
| Code generator field coverage drift | Item 8 | COVERED | |
|
|
1280
|
-
| New-field propagation to consumers | Item 9 | COVERED | |
|
|
1281
|
-
| Cross-PR SDK/schema consistency | Item 10 | COVERED | #190 |
|
|
1282
|
-
| API versioning contract breaks (v1 vs v2) | — | GAP | |
|
|
1283
|
-
| Pagination contract consistency | — | GAP | |
|
|
1284
|
-
|
|
1285
|
-
---
|
|
1286
|
-
|
|
1287
|
-
### Agent: Database & Migration Auditor
|
|
1288
|
-
|
|
1289
|
-
**Trigger**: DATABASE domain detected
|
|
1290
|
-
**Type**: `general-purpose` | **Model**: `sonnet`
|
|
1291
|
-
|
|
1292
|
-
**Prompt template:**
|
|
1293
|
-
```
|
|
1294
|
-
You are auditing PR #[PR_NUMBER] for database changes in [PROJECT_NAME].
|
|
1295
|
-
|
|
1296
|
-
[DOMAIN_CONTEXT]
|
|
1297
|
-
|
|
1298
|
-
## What to Check
|
|
1299
|
-
1. **SQL correctness** (CHECK FIRST — a query that errors at runtime breaks the whole page):
|
|
1300
|
-
- **Ambiguous references**: `SELECT t.col, t.*` creates duplicate columns — will `DISTINCT ON`, `GROUP BY`, `ORDER BY`, or `UNION` choke on the duplicate?
|
|
1301
|
-
- **Column visibility**: CTEs and subqueries that produce duplicate column names, then are referenced by `SELECT *`
|
|
1302
|
-
- **Type mismatches**: Comparing UUID to text, integer to string without cast
|
|
1303
|
-
- **Invalid aggregations**: Non-aggregated columns in SELECT with GROUP BY
|
|
1304
|
-
- **Mental-execute the query**: Read the full SQL top-to-bottom. What columns does each CTE produce? What does the final SELECT see? Would PostgreSQL accept this or throw an error?
|
|
1305
|
-
2. **Migration safety**: Can this run on a live database without downtime?
|
|
1306
|
-
- `ALTER TABLE ... ADD COLUMN ... NOT NULL` without `DEFAULT` → table lock + failure on existing rows
|
|
1307
|
-
- `DROP TABLE/COLUMN` without `IF EXISTS` → fails on fresh DBs
|
|
1308
|
-
- Large table operations → may lock for minutes
|
|
1309
|
-
3. **Reversibility**: Is there a rollback path?
|
|
1310
|
-
4. **Index usage**: New queries without indexes on filtered/joined columns? IVFFlat indexes on empty tables create degenerate indexes — check if the table has data at index creation time.
|
|
1311
|
-
5. **Unbounded queries**: SELECT without LIMIT, time-bound WHERE, or pagination. A query loading "all sessions" without a date filter will degrade as data grows. Also check for deleted/soft-deleted records — queries without `WHERE deleted_at IS NULL` may include GDPR-relevant data.
|
|
1312
|
-
6. **N+1 queries**: Loop fetching rows one at a time instead of batch?
|
|
1313
|
-
7. **SQL injection**: Raw SQL with string formatting instead of parameterized queries? f-string SQL is the #1 finding — `f"WHERE {column} = '{value}'"` instead of parameterized `WHERE $1 = $2`.
|
|
1314
|
-
8. **Migration number collisions (CRITICAL — full-tree scan required)**:
|
|
1315
|
-
This check is structurally different from the others — it must scan the **entire `infra/migrations/` directory**, not just files in the PR diff. Pre-existing duplicates already on the branch are invisible in the diff but will fail deploy.
|
|
1316
|
-
Steps:
|
|
1317
|
-
a. List ALL `*.sql` files in `infra/migrations/` on the PR's target branch (use `git ls-tree` or `ls`)
|
|
1318
|
-
b. Extract the 4-digit numeric prefix from each filename (the leading digits before the first `_`)
|
|
1319
|
-
c. Identify any prefix that appears more than once
|
|
1320
|
-
d. Cross-reference against the `GRANDFATHERED_DUPLICATES` list in `scripts/validate-migration-order.sh` — these are known-safe legacy duplicates
|
|
1321
|
-
e. **DEPLOY GATE — CRITICAL**: If ANY non-grandfathered duplicate prefix exists → flag as **CRITICAL BLOCKER** and reject the PR. **Do NOT apply migration runner reasoning here.** The deploy gate (`deploy-production.yml`) runs `validate-migration-order.sh` with no `continue-on-error` — it hard-fails on any non-grandfathered duplicate regardless of whether the migration runner executes files correctly. The runner may handle duplicate filenames fine; the deploy script does not. A PR that passes this reasoning trap ("unique filenames, so the runner is safe") will still halt production deploy and waste a 30-minute CI compute cycle. The only safe classification is CRITICAL BLOCKER.
|
|
1322
|
-
f. Additionally: if the PR adds new migration files, verify their prefixes don't collide with existing files in the directory
|
|
1323
|
-
9. **FK and CHECK constraints**: New tables should have appropriate foreign keys and CHECK constraints. Missing FK allows orphaned rows; missing CHECK allows invalid enum values.
|
|
1324
|
-
10. **asyncpg gotchas**:
|
|
1325
|
-
- Must use `CAST(:param AS type)` for nullable params (asyncpg can't infer NULL types)
|
|
1326
|
-
- `::jsonb` after `:param` conflicts with SQLAlchemy binding — use `CAST(:param AS jsonb)`
|
|
1327
|
-
|
|
1328
|
-
## Post Findings
|
|
1329
|
-
```bash
|
|
1330
|
-
gh pr comment [PR_NUMBER] --body "$(cat <<'EOF'
|
|
1331
|
-
## Database & Migration Audit
|
|
1332
|
-
|
|
1333
|
-
### Migration Safety: [SAFE/CAUTION/DANGEROUS]
|
|
1334
|
-
|
|
1335
|
-
### Findings
|
|
1336
|
-
| Issue | Location | Severity | Evidence |
|
|
1337
|
-
|-------|----------|----------|----------|
|
|
1338
|
-
| ... | file:line | HIGH | [explanation] |
|
|
1339
|
-
|
|
1340
|
-
### Migration Review
|
|
1341
|
-
[For each SQL file: what it does, is it safe, is it reversible?]
|
|
1342
|
-
|
|
1343
|
-
### Files Reviewed
|
|
1344
|
-
[List files checked]
|
|
1345
|
-
|
|
1346
|
-
---
|
|
1347
|
-
*Database audit*
|
|
1348
|
-
|
|
1349
|
-
<!-- REVIEW-FINDINGS-START -->
|
|
1350
|
-
<!-- FINDING:DB-1|CONFIDENCE|SEVERITY|file.py:line|Summary -->
|
|
1351
|
-
<!-- (add one FINDING line per issue found — include ALL confidence levels) -->
|
|
1352
|
-
<!-- REVIEW-FINDINGS-END -->
|
|
1353
|
-
EOF
|
|
1354
|
-
)"
|
|
1355
|
-
```
|
|
1356
|
-
|
|
1357
|
-
**Structured Findings**: Include the structured findings block above at the end of your comment. Your prefix: `DB`. See the Structured Findings Protocol section above for format rules.
|
|
1358
|
-
```
|
|
1359
|
-
|
|
1360
|
-
### Coverage Matrix — DB Agent
|
|
1361
|
-
|
|
1362
|
-
| Defect Category | Check Item(s) | Status | Ref |
|
|
1363
|
-
|----------------|---------------|--------|-----|
|
|
1364
|
-
| Ambiguous column references / duplicate cols | Item 1 | COVERED | |
|
|
1365
|
-
| Type mismatches in SQL (UUID vs text) | Item 1 | COVERED | |
|
|
1366
|
-
| Migration safety (NOT NULL without DEFAULT) | Item 2 | COVERED | |
|
|
1367
|
-
| Migration reversibility | Item 3 | COVERED | |
|
|
1368
|
-
| Missing indexes on filtered/joined columns | Item 4 | COVERED | |
|
|
1369
|
-
| Unbounded queries (no LIMIT, no date filter) | Item 5 | COVERED | |
|
|
1370
|
-
| N+1 query patterns | Item 6 | COVERED | |
|
|
1371
|
-
| SQL injection (f-string SQL) | Item 7 | COVERED | |
|
|
1372
|
-
| Migration number collisions | Item 8 | COVERED | #222 |
|
|
1373
|
-
| Missing FK / CHECK constraints | Item 9 | COVERED | |
|
|
1374
|
-
| asyncpg parameter casting | Item 10 | COVERED | |
|
|
1375
|
-
| ORM field rename without migration | — | GAP | #240 |
|
|
1376
|
-
| Ghost migration (rename-to-fill-gap) | — | GAP | #227 |
|
|
1377
|
-
|
|
1378
|
-
---
|
|
1379
|
-
|
|
1380
|
-
### Agent: Infrastructure & Deploy Safety Auditor
|
|
1381
|
-
|
|
1382
|
-
**Trigger**: INFRA service touched
|
|
1383
|
-
**Type**: `general-purpose` | **Model**: `sonnet`
|
|
1384
|
-
|
|
1385
|
-
**Prompt template:**
|
|
1386
|
-
```
|
|
1387
|
-
You are auditing PR #[PR_NUMBER] for infrastructure and deployment safety in [PROJECT_NAME].
|
|
1388
|
-
|
|
1389
|
-
## Deployment Architecture
|
|
1390
|
-
|
|
1391
|
-
[DOMAIN_CONTEXT]
|
|
1392
|
-
|
|
1393
|
-
If no infra context is configured above, derive the deployment model from the changed files: read docker-compose files, GitHub Actions workflows, and entrypoint scripts to understand the deployment pipeline.
|
|
1394
|
-
|
|
1395
|
-
## What to Check
|
|
1396
|
-
1. **Downtime risk**: Will this change cause container restarts during deploy?
|
|
1397
|
-
1b. **Database container restart risk** (when `docker-compose*.yml` changes touch the `postgres` or `redis` service — `command:`, `image:`, `volumes:`, `environment:`):
|
|
1398
|
-
Changing any of these fields forces container recreation on `docker-compose up`. For stateful services this is **HIGH risk by default** — regardless of how simple the config change appears.
|
|
1399
|
-
**Auto-escalate Deploy Risk to HIGH** when this trigger fires. Then verify ALL of the following:
|
|
1400
|
-
- Is `full_page_writes = on` in the PG config? (Protects against partial page writes if shutdown is not clean)
|
|
1401
|
-
- Is `fsync = on`? (Ensures writes are durable to disk)
|
|
1402
|
-
- Is `stop_grace_period` set in docker-compose and sufficient (≥30s for Postgres)? If absent, Docker sends SIGKILL after 10s default — too short for PG to flush WAL and checkpoint.
|
|
1403
|
-
- Are there likely active long-running transactions (migrations, bulk imports, analytics queries) that will be interrupted?
|
|
1404
|
-
- Is the deploy blue-green for THIS container, or will PG/Redis be hard-restarted? (Typically stateful containers are NOT blue-green — they are singletons that get recreated in-place.)
|
|
1405
|
-
**Recommendation**: Postgres/Redis restarts should NOT happen as a side effect of a routine deploy. Recommend:
|
|
1406
|
-
- Schedule the restart during a **maintenance window** (low-traffic hours)
|
|
1407
|
-
- OR split the change: deploy the stateful config change as a **separate manual operation** with monitoring
|
|
1408
|
-
- Verify backups are current before any PG restart
|
|
1409
|
-
A Postgres container restart under active write load can corrupt btree indexes and bypass UNIQUE constraints — the database may return to a consistent state but with structural damage that only surfaces on subsequent writes or queries. <!-- Added: forge#146 -->
|
|
1410
|
-
1c. **Functional equivalence on config mechanism change** (when a stateful service's `command:` or `entrypoint:` changes structurally — e.g., inline args → script, or vice versa):
|
|
1411
|
-
The old and new configurations MUST produce the same runtime behavior. Verify:
|
|
1412
|
-
- Extract the effective config from the OLD approach (parse `command:` args or previous entrypoint)
|
|
1413
|
-
- Extract the effective config from the NEW approach (parse new entrypoint/script + env vars)
|
|
1414
|
-
- Diff them — flag any settings that changed, disappeared, or gained new defaults
|
|
1415
|
-
- **Env var delivery mechanism**: When `command:` uses `${VAR}` (Compose interpolation, host-side parse time) and the new approach uses an entrypoint script that reads `${VAR}` (shell, container runtime), the var MUST be explicitly passed via `environment:` or `env_file:`. Compose interpolation does NOT inject the var into the container — it resolves it before the container starts. If the entrypoint reads `$VAR` but no `environment:` block passes it, the var will be empty inside the container. **This is a CONFIRMED BLOCKING finding.**
|
|
1416
|
-
Compose interpolation (`${VAR}` in `command:`) resolves the variable at parse time on the host before the container starts — it does NOT inject the variable into the container's environment. If the new entrypoint script reads `$VAR` but no `environment:` block passes it, the variable will be empty inside the container. This is a silent misconfiguration that only surfaces at deploy time. <!-- Added: forge#185 -->
|
|
1417
|
-
2. **Blue-green compatibility**: Do old and new versions work simultaneously?
|
|
1418
|
-
- API responses compatible?
|
|
1419
|
-
- Database schema works with both versions?
|
|
1420
|
-
- Redis keys compatible?
|
|
1421
|
-
- Queue message formats compatible?
|
|
1422
|
-
3. **Rollback path**: Can we revert if something goes wrong?
|
|
1423
|
-
4. **Prerequisites**: New env vars, secrets, DNS changes needed before deploy?
|
|
1424
|
-
5. **In-flight requests**: Will active scrape jobs, webhooks, or billing transactions be affected?
|
|
1425
|
-
6. **Health endpoint response contract breakage** (when the PR diff includes changes to route handlers for health, status, ping, liveness, or readiness endpoints — search for changed Python functions decorating paths matching `/health`, `/status`, `/ping`, `/ready`, `/live`, `/readiness`, `/liveness`):
|
|
1426
|
-
A PR that changes the response body shape of a health/status endpoint can silently break any downstream consumer that pattern-matches on the old response format — deploy scripts, CI healthchecks, docker-compose healthcheck definitions, monitoring configs, and Traefik probes. These consumers are rarely in the same file as the endpoint handler and are often missed during code review.
|
|
1427
|
-
|
|
1428
|
-
**Scan all consumer locations for references to the old response format:**
|
|
1429
|
-
```bash
|
|
1430
|
-
# Identify old response field values from the diff
|
|
1431
|
-
OLD_VALUES=$(gh pr diff [PR_NUMBER] | grep "^\-" | grep -oP '"status"\s*:\s*"\K[^"]+' | sort -u)
|
|
1432
|
-
|
|
1433
|
-
# Search all non-application consumer locations for those old values
|
|
1434
|
-
while IFS= read -r val; do
|
|
1435
|
-
[ -z "$val" ] && continue
|
|
1436
|
-
echo "=== Searching for old response value: $val ==="
|
|
1437
|
-
grep -rn "$val" scripts/ infra/ .github/ traefik/ 2>/dev/null | grep -v "^Binary"
|
|
1438
|
-
grep -rn "$val" docker-compose*.yml 2>/dev/null
|
|
1439
|
-
done <<< "$OLD_VALUES"
|
|
1440
|
-
|
|
1441
|
-
# Also search for endpoint path references in consumer locations
|
|
1442
|
-
HEALTH_PATHS=$(gh pr diff [PR_NUMBER] | grep "^\+" | grep -oP '@\w+\.(?:get|post)\("\K[^"]+(?:health|status|ping|ready|live|readiness|liveness)[^"]*')
|
|
1443
|
-
while IFS= read -r path; do
|
|
1444
|
-
[ -z "$path" ] && continue
|
|
1445
|
-
grep -rn "$path" scripts/ infra/ .github/ traefik/ docker-compose*.yml 2>/dev/null | grep -v "^Binary"
|
|
1446
|
-
done <<< "$HEALTH_PATHS"
|
|
1447
|
-
```
|
|
1448
|
-
|
|
1449
|
-
**For each hit, classify:**
|
|
1450
|
-
- `grep -q "old_value"`, `if echo "$resp" | grep "old_value"`, jq filter on a now-missing field, or string comparison against an old response key value: **CONFIRMED BLOCKING** — the consumer will fail silently or return a false negative after deploy
|
|
1451
|
-
- Variable set from response field, then used in a conditional: **CONFIRMED BLOCKING** — behavior change is guaranteed
|
|
1452
|
-
- Log or comment referencing old value: **OK** — cosmetic only, no behavioral impact
|
|
1453
|
-
|
|
1454
|
-
**Severity**: CONFIRMED BLOCKING for any behavioral consumer. A deploy that rolls out the new response format while a deploy script still checks for the old format will cause the deploy's own health gate to reject healthy containers, forcing an automatic rollback.
|
|
1455
|
-
|
|
1456
|
-
**Recommended fix**: Update ALL behavioral consumers in the same PR as the endpoint change. If a consumer is in a deploy script or CI workflow, updating it after the endpoint change has already shipped creates a window where every deploy will roll back.
|
|
1457
|
-
|
|
1458
|
-
A response contract change that updates only the endpoint while leaving downstream consumers on the old format is equivalent to a breaking API change deployed without a migration path — the breakage surfaces at the worst possible time: during a production deploy. <!-- Added: forge#321 -->
|
|
1459
|
-
|
|
1460
|
-
7. **Secret delivery chain** (when `decrypt-secrets.sh`, `.secrets/prod.enc.yaml`, or deploy workflows change):
|
|
1461
|
-
Trace the FULL path a secret takes from SOPS to a running container. Read these files and verify consistency:
|
|
1462
|
-
- `.secrets/prod.enc.yaml` — the SOPS key name under its section (e.g., `oauth.discord_client_id`)
|
|
1463
|
-
- `scripts/decrypt-secrets.sh` — the ENV_MAPPING tuple must match the SOPS path (e.g., `("oauth", "discord_client_id")`)
|
|
1464
|
-
- `.github/workflows/deploy-production.yml` — the SCP `target:` path and the merge step's `PROJECT=` variable
|
|
1465
|
-
- `.github/workflows/hotfix-deploy.yml` — same paths, must be consistent with main deploy
|
|
1466
|
-
- `docker-compose.prod.yml` — `env_file:` must point to the same `.env.production` the merge step writes to
|
|
1467
|
-
**Critical check**: If the workflow appends `/app` to `PRODUCTION_PROJECT_PATH`, and that secret already includes `/app`, secrets will silently go to a wrong directory. Verify the SCP target resolves to the SAME file that `docker-compose.prod.yml` env_file reads.
|
|
1468
|
-
**Do NOT just verify naming consistency** — verify the filesystem paths are correct end-to-end.
|
|
1469
|
-
7b. **Shell metacharacter safety in .env writers** (when `decrypt-secrets.sh` or any script that writes values to `.env` or shell-sourceable config files is touched):
|
|
1470
|
-
Secret values can contain any character — passwords, API keys, and OAuth tokens frequently contain `;`, `!`, `|`, `&`, `(`, `)`, `{`, `}`, backticks, and other bash metacharacters. If a script that generates `.env` files uses an **allowlist-based quoting approach** (checking for a specific subset of special characters before deciding whether to quote), it will silently write unquoted values for any character not in the list. When the deploy pipeline sources that file, bash interprets the metacharacter as a command separator, causing exit 127 or command-not-found errors.
|
|
1471
|
-
|
|
1472
|
-
**What to look for**: In the diff for any script that writes `KEY=VALUE` lines to `.env` or shell-sourceable files:
|
|
1473
|
-
- Search for quoting logic: `if any(c in value for c in ...)`, character-set checks, or regex allowlists
|
|
1474
|
-
- If the script decides whether to quote based on whether the value contains characters from a hardcoded list: this is an allowlist-based approach
|
|
1475
|
-
- Check whether the allowlist includes ALL bash metacharacters: `;`, `!`, `|`, `&`, `(`, `)`, `{`, `}`, `` ` ``, `<`, `>`, `\n`, `\t`, `$`, `"`, `'`, `#`, `\`, and space. If ANY of these are missing from the allowlist: **CONFIRMED HIGH**
|
|
1476
|
-
|
|
1477
|
-
**Safe pattern**: The only safe approach is **unconditional double-quoting** of all values. The output line must always be `KEY="VALUE"` — never `KEY=VALUE` without quotes. If the script conditionally skips quoting for any value, flag it.
|
|
1478
|
-
|
|
1479
|
-
**Severity classification**:
|
|
1480
|
-
- Allowlist-based quoting with incomplete metacharacter set: **CONFIRMED HIGH** — any secret containing a missing metachar breaks `source .env.production` at deploy time
|
|
1481
|
-
- `source`/`.` command on an env file without `set -e` or error handling: **LIKELY MEDIUM** — a bad value causes a silent no-op instead of a visible deploy failure
|
|
1482
|
-
|
|
1483
|
-
**Recommended fix**: Replace the conditional quoting block with unconditional double-quoting:
|
|
1484
|
-
```python
|
|
1485
|
-
# FRAGILE — allowlist will miss metacharacters
|
|
1486
|
-
if any(c in value for c in (' ', '$', '"', "'", '\n', '\t', '#', '\\')):
|
|
1487
|
-
f.write(f'{key}="{value}"\n')
|
|
1488
|
-
else:
|
|
1489
|
-
f.write(f'{key}={value}\n')
|
|
1490
|
-
|
|
1491
|
-
# SAFE — always double-quote
|
|
1492
|
-
f.write(f'{key}="{value}"\n')
|
|
1493
|
-
```
|
|
1494
|
-
|
|
1495
|
-
An allowlist-based quoting approach in a secret-decryption script will silently fail for any metacharacter not in the list. Secret values (passwords, API keys, OAuth tokens) can contain any character — allowlists are guaranteed to be incomplete. The only safe approach is always-quote. <!-- Added: forge#286 -->
|
|
1496
|
-
8. **Cross-domain service interactions** (when shell scripts curl/wget internal services):
|
|
1497
|
-
For every `curl` or `wget` command in changed shell scripts that targets an internal service (API, worker, Redis, Postgres), trace the HTTP request through the target application's middleware stack:
|
|
1498
|
-
- Read `services/api/app/main.py` — check `TrustedHostMiddleware` allowed_hosts list. Does the curl's `Host` header value match an allowed entry? (Default `Host` is the URL hostname — a bare container IP like `172.18.0.x` is NOT `localhost`.)
|
|
1499
|
-
- Check auth requirements — does the target endpoint require API keys or session tokens?
|
|
1500
|
-
- Check the URL path — does the endpoint actually exist at that path?
|
|
1501
|
-
- **If the curl would be rejected** (wrong Host header, missing auth, wrong path): flag as CONFIRMED finding. A health check or monitoring script that gets 400/401/404 instead of 200 produces false alerts.
|
|
1502
|
-
**Do NOT assume a curl will succeed just because the URL is syntactically valid** — read the target service's code to verify.
|
|
1503
|
-
9. **Sibling workflow drift** (ALWAYS check for staging→main PRs; also check when ANY `.github/workflows/*.yml` file changes):
|
|
1504
|
-
Multiple workflows contain jobs with the same logical purpose (e.g., `test-api` exists in both `ci.yml` and `deploy-production.yml`).
|
|
1505
|
-
**For staging→main PRs, ALWAYS read both `ci.yml` and `deploy-production.yml` and compare shared jobs — even if neither file changed.** Pre-existing drift is the most dangerous kind: a PR can be approved with green CI while `deploy-production.yml` is missing env vars or install steps that `ci.yml` has, causing failures only at deploy time — not during testing.
|
|
1506
|
-
- Read both files. For each shared job (test-api, test-web): compare PYTHONPATH values on every step, dependency install steps (count + content), step names present in one but not the other
|
|
1507
|
-
- **Any env var, path, install step, or command present in one but missing from the sibling is CONFIRMED BLOCKING drift** — CI passes but deploy fails
|
|
1508
|
-
- Common siblings: `ci.yml` ↔ `deploy-production.yml` (share `test-api`, `test-web` jobs), `deploy-production.yml` ↔ `hotfix-deploy.yml` (share build/deploy logic)
|
|
1509
|
-
**This is the #1 cause of "CI passed but deploy failed" incidents.** Do not skip this check.
|
|
1510
|
-
10. **Deploy scope awareness** (ALWAYS run — checks whether ALL changed services will actually be deployed):
|
|
1511
|
-
This project may have multiple deploy pipelines. Check which pipelines cover which services:
|
|
1512
|
-
|
|
1513
|
-
```bash
|
|
1514
|
-
# List all GitHub Actions workflows
|
|
1515
|
-
ls .github/workflows/
|
|
1516
|
-
# List all paths changed in this PR
|
|
1517
|
-
gh pr diff [PR_NUMBER] --name-only
|
|
1518
|
-
```
|
|
1519
|
-
|
|
1520
|
-
For each changed path: read the `.github/workflows/` directory to determine which workflow deploys it and whether it's auto-triggered or requires manual action. If any changed service path is NOT covered by an auto-triggered workflow on merge:
|
|
1521
|
-
- Flag as CONFIRMED BLOCKING finding
|
|
1522
|
-
- Verdict: `BLOCK — Deploy prerequisite: manually trigger [workflow] for [service names] before or after this deploy`
|
|
1523
|
-
- Rationale: Services not covered by the auto-deploy pipeline will not be updated on merge, leaving a broken contract between deployed and un-deployed layers.
|
|
1524
|
-
|
|
1525
|
-
If `[DOMAIN_CONTEXT]` above contains a deploy pipeline table, use it as the authoritative source for path → pipeline mapping.
|
|
1526
|
-
11. **Config field type/doc-comment consistency** (when the diff introduces new pydantic-settings fields):
|
|
1527
|
-
When a new env var is introduced as a pydantic-settings field with a collection type (`list[str]`, `List[str]`, `set[str]`, `Set[str]`), cross-reference the field's type annotation against the format hint documented in `.env.example`. **pydantic-settings v2 parses collection-type fields via `json.loads()` — they require JSON array format like `["a","b"]`, NOT comma-separated format like `a,b`**. A doc comment saying "comma-separated" on a `list[str]` field is a CONFIRMED HIGH finding — it guarantees a startup crash for anyone following the documented format.
|
|
1528
|
-
```bash
|
|
1529
|
-
# Find new pydantic-settings fields with collection types in the diff
|
|
1530
|
-
gh pr diff [PR_NUMBER] | grep "^\+" | grep -E ":\s*(list|List|set|Set)\[str\]" | grep -oP "[A-Z_]{3,}(?=\s*:)" | sort -u
|
|
1531
|
-
# For each field, find its .env.example entry and read the format documentation
|
|
1532
|
-
for field in $COLLECTION_FIELDS; do
|
|
1533
|
-
echo "=== $field ==="
|
|
1534
|
-
grep -A5 "$field" .env.example 2>/dev/null
|
|
1535
|
-
done
|
|
1536
|
-
```
|
|
1537
|
-
If the `.env.example` comment says "comma-separated", "CSV", or shows a bare `a,b,c` example value for a `list[str]` field: **CONFIRMED HIGH** — the documented format crashes pydantic-settings v2 on startup. The fix is to update `.env.example` to show JSON format (`["a","b","c"]`) as primary, OR add a custom settings source that handles CSV as a fallback.
|
|
1538
|
-
pydantic-settings v2 parses `list[str]` fields via `json.loads()` — it requires JSON array format. A `.env.example` comment saying "comma-separated" on a `list[str]` field is a guaranteed startup crash for any operator who follows the documented format. The type annotation and the documented format must agree. <!-- Added: forge#190 -->
|
|
1539
|
-
12. **appleboy/ssh-action Go template safety** (when `.github/workflows/*.yml` changes include `appleboy/ssh-action` steps):
|
|
1540
|
-
The `appleboy/ssh-action` action processes every `script:` field through Go's `text/template` engine **client-side, before SSH transmission**. Any `{{` in the script — including in comments, `docker ps --format` strings, and `docker inspect --format` strings — is interpreted as a Go template directive. If the expression does not resolve against the action's data context (which is an empty map for `appleboy/ssh-action`), the step exits with status 1 **before the script reaches the remote shell**. Shell error handlers (`|| fallback`, `2>/dev/null`, `set -e`) are completely bypassed. If `continue-on-error: true` is set, GitHub Actions will report overall success — masking the failure silently.
|
|
1541
|
-
|
|
1542
|
-
**Scan ALL `{{` occurrences in every `script:` block of every `appleboy/ssh-action` step in changed files:**
|
|
1543
|
-
```bash
|
|
1544
|
-
while IFS= read -r f; do
|
|
1545
|
-
[ -z "$f" ] && continue
|
|
1546
|
-
if grep -q "appleboy/ssh-action" "$f"; then
|
|
1547
|
-
echo "=== appleboy/ssh-action found in $f ==="
|
|
1548
|
-
# Show all {{ occurrences with line numbers
|
|
1549
|
-
grep -n "{{" "$f"
|
|
1550
|
-
fi
|
|
1551
|
-
done < <(gh pr diff [PR_NUMBER] --name-only | grep -E '\.github/workflows/.*\.yml$')
|
|
1552
|
-
```
|
|
1553
|
-
|
|
1554
|
-
**For every `{{` found, classify and flag:**
|
|
1555
|
-
- `{{index .X Y}}` or any function call form: **CONFIRMED BLOCKING** — Go template function calls fail on empty context
|
|
1556
|
-
- `{{.FieldName}}` (field accessor, e.g., `{{.Names}}`, `{{.Status}}`): **CONFIRMED BLOCKING** — field accessors fail when the field does not exist on the data context. Do NOT assume field accessors are safe just because they look simpler than function calls. Both forms crash the template engine on an empty context.
|
|
1557
|
-
- `{{ range ... }}`, `{{ if ... }}`, `{{ with ... }}`: **CONFIRMED BLOCKING** — control flow directives
|
|
1558
|
-
|
|
1559
|
-
**Safe replacements:**
|
|
1560
|
-
- `docker inspect --format '{{index .RepoTags 0}}'` → `docker inspect IMAGE | jq -r '.[0].RepoTags[0]'`
|
|
1561
|
-
- `docker ps --format '{{.Names}}'` → `docker ps --format json | jq -r '.Names'`
|
|
1562
|
-
- `docker ps --format '{{.Status}}'` → `docker ps --format json | jq -r '.Status'`
|
|
1563
|
-
|
|
1564
|
-
Both function call forms (`{{index .X Y}}`) and field accessor forms (`{{.FieldName}}`) fail on `appleboy/ssh-action`'s empty data context. A partial fix that replaces only one form while leaving the other is insufficient — search for ALL `{{` occurrences, not just the specific form that triggered the initial investigation. <!-- Added: forge#226 -->
|
|
1565
|
-
|
|
1566
|
-
13. **Insecure config file defaults** (when any `traefik/`, `infra/nginx/`, `infra/`, or CI config file is in the diff):
|
|
1567
|
-
Config files that use shell-style variable substitution with fallback values (`${VAR:-default}`) are safe only when the fallback itself is safe. The INFRA agent checks whether the env var is delivered to the container (items 7 and 7b) — but does NOT check whether the fallback value is a credential placeholder that becomes active if the env var is absent in production.
|
|
1568
|
-
|
|
1569
|
-
**Scan changed config files for credential fallbacks:**
|
|
1570
|
-
```bash
|
|
1571
|
-
CONFIG_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "(traefik/|infra/nginx/|infra/|\.github/workflows/).*(\.ya?ml|\.toml|\.json|\.conf|\.ini)$" | head -20)
|
|
1572
|
-
while IFS= read -r f; do
|
|
1573
|
-
[ -z "$f" ] && continue
|
|
1574
|
-
# Shell-style fallback: ${VAR:-default} — filter out safe non-credential defaults
|
|
1575
|
-
grep -nE "\$\{[A-Z_]+:-[^}]{4,}\}" "$f" | grep -ivE "(localhost|127\.0\.0\.1|0\.0\.0\.0|true|false|^[0-9]+$|/tmp|/var|/etc)" | head -20
|
|
1576
|
-
done <<< "$CONFIG_FILES"
|
|
1577
|
-
```
|
|
1578
|
-
|
|
1579
|
-
**For each hit, classify the fallback value:**
|
|
1580
|
-
- Fallback matches `\$apr1\$`, `\$2b\$`, `\$bcrypt\$`, `\$argon2`, or any hash-like string (long alphanumeric with `$` separators): **CONFIRMED HIGH**
|
|
1581
|
-
- Rationale: htpasswd/bcrypt/apr1 entries are placeholder credentials. If the env var is absent in any environment, the placeholder is active and may be guessable if derived from a weak password.
|
|
1582
|
-
- Required: Replace with `${VAR}` (no fallback) AND verify the var is required in `env_validation.py`.
|
|
1583
|
-
- Fallback is an empty string (`${VAR:-}` or `${VAR:-""}`): **CONFIRMED HIGH** if the field controls authentication
|
|
1584
|
-
- Rationale: empty auth = no auth for the service protected by the config.
|
|
1585
|
-
- Fallback is a weak/example credential string (`admin`, `password`, `changeme`, `secret`, `test`, `demo`): **CONFIRMED HIGH**
|
|
1586
|
-
- Rationale: default credentials are the first thing an attacker tries; a placeholder that ships as default is permanently vulnerable unless the env var is proven present in all environments.
|
|
1587
|
-
- Fallback is a non-credential value (port number, hostname, path): **OK** — no action needed.
|
|
1588
|
-
|
|
1589
|
-
**Cross-check when a credential fallback is found:**
|
|
1590
|
-
1. Is the corresponding env var required (not optional) in `app/env_validation.py` (or equivalent startup validation)?
|
|
1591
|
-
2. Is the env var in `scripts/decrypt-secrets.sh` ENV_MAPPING (delivered from SOPS to `.env.production`)?
|
|
1592
|
-
3. Is the env var documented with a "REQUIRED" note in `.env.example`?
|
|
1593
|
-
|
|
1594
|
-
If ANY of these three are missing: the env var may legitimately be absent in some environments, making the insecure fallback active. Flag as **CONFIRMED HIGH** with all three cross-check results.
|
|
1595
|
-
|
|
1596
|
-
**Recommended fix**: Remove the fallback entirely (`${VAR}` not `${VAR:-placeholder}`) and make the env var required in `env_validation.py`. If the service cannot start without the credential, a startup crash is safer than running with a default credential.
|
|
1597
|
-
|
|
1598
|
-
The INFRA agent's secret delivery chain verification (env var present in ENV_MAPPING) does NOT verify whether the fallback value is safe. A config file can pass all delivery chain checks while still containing an active placeholder credential for any environment where the env var is absent. This check targets that gap. <!-- Added: forge#301 -->
|
|
1599
|
-
|
|
1600
|
-
14. **Host-side database tool invocations in deploy scripts** (when `.github/workflows/*.yml`, `scripts/deploy*.sh`, or any deploy entrypoint script in the diff invokes `psql`, `createdb`, or `pg_dump` directly on the host):
|
|
1601
|
-
Host-side `psql`, `createdb`, and `pg_dump` connect to PostgreSQL via the host's port binding (`localhost:5432` or `127.0.0.1:5432`). In SSH-based deploy contexts, the host-side port mapping to PostgreSQL's Docker container is unreliable — the listening address may differ from what the SSH session can reach, and the mapping is not guaranteed to be present at all in hardened host configurations. The safe pattern is to use `docker exec <postgres-container>` for all database operations in deploy scripts.
|
|
1602
|
-
|
|
1603
|
-
To find the correct container name, grep the project's docker-compose files:
|
|
1604
|
-
```bash
|
|
1605
|
-
grep -n "container_name" docker-compose*.yml | grep -i "postgres\|pg\|db"
|
|
1606
|
-
```
|
|
1607
|
-
Use the container name found (referred to as `[DB_CONTAINER]` in examples below).
|
|
1608
|
-
|
|
1609
|
-
**Scan for host-side database tool invocations:**
|
|
1610
|
-
```bash
|
|
1611
|
-
DEPLOY_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "(\.github/workflows/.*\.yml$|scripts/deploy.*\.sh$|docker-entrypoint.*\.sh$)")
|
|
1612
|
-
while IFS= read -r f; do
|
|
1613
|
-
[ -z "$f" ] && continue
|
|
1614
|
-
grep -n "\bpsql\b\|\bcreatedb\b\|\bpg_dump\b\|\bpg_restore\b" "$f" | grep -v "docker exec" | head -20
|
|
1615
|
-
done <<< "$DEPLOY_FILES"
|
|
1616
|
-
```
|
|
1617
|
-
|
|
1618
|
-
**For each match, classify:**
|
|
1619
|
-
- `psql`, `createdb`, `pg_dump`, or `pg_restore` called directly (not via `docker exec`): **CONFIRMED HIGH**
|
|
1620
|
-
- Rationale: Host-side port mappings to PostgreSQL's Docker container are unreliable in SSH deploy contexts. A deploy step that fails to connect leaves post-deploy database setup incomplete — silently if `continue-on-error: true` is set, or rolls back a successful deploy if not.
|
|
1621
|
-
- The safe pattern: `docker exec [DB_CONTAINER] psql -U $POSTGRES_USER -d $POSTGRES_DB -c "..."` or `docker exec [DB_CONTAINER] createdb -U $POSTGRES_USER $DB_NAME`
|
|
1622
|
-
- `psql`/`createdb`/`pg_dump` called inside `docker exec [DB_CONTAINER] ...`: **OK** — this is the safe pattern.
|
|
1623
|
-
|
|
1624
|
-
**Recommended fix**: Replace host-side invocations with `docker exec [DB_CONTAINER]` equivalents:
|
|
1625
|
-
- `psql -h localhost -U $USER -d $DB -c "..."` → `docker exec [DB_CONTAINER] psql -U $USER -d $DB -c "..."`
|
|
1626
|
-
- `createdb -h localhost -U $USER $DB_NAME` → `docker exec [DB_CONTAINER] createdb -U $USER $DB_NAME`
|
|
1627
|
-
- `pg_dump -h localhost -U $USER $DB` → `docker exec [DB_CONTAINER] pg_dump -U $USER $DB`
|
|
1628
|
-
|
|
1629
|
-
Host-side port bindings to Docker containers are unreliable in SSH deploy contexts. Deploy scripts that invoke database tools directly cannot depend on `localhost:5432` being reachable; the same operation via `docker exec` bypasses the host network entirely and always succeeds when the container is running. <!-- Added: forge#322 -->
|
|
1630
|
-
|
|
1631
|
-
15. **External tool config schema validation** (when any config file consumed by an external tool is in the diff — trigger: files under `traefik/`, `infra/nginx/`, `k8s/`, `terraform/`, `*.conf`, `*.toml`, or any `docker-compose*.yml` with non-trivial changes to service definitions):
|
|
1632
|
-
|
|
1633
|
-
**Principle: Logical correctness ≠ structural correctness.** A config can have the right values in the wrong nesting and be silently rejected by the external tool — producing zero error logs in some tools (e.g. Traefik v3 ignores unrecognized nesting silently). The INFRA agent must verify both dimensions:
|
|
1634
|
-
- **Logical correctness** (covered by existing checks): Are the values coherent? Do names match across files? Is the wiring consistent with the deploy pipeline?
|
|
1635
|
-
- **Structural correctness** (this check): Does the YAML/TOML/HCL structure match what the tool's actual schema expects? Are directives nested at the correct depth? Do required parent keys exist?
|
|
1636
|
-
|
|
1637
|
-
**For each external tool config file in the diff**:
|
|
1638
|
-
|
|
1639
|
-
1. **Identify the tool and version**: Read the `image:` tag in `docker-compose*.yml` for the tool (e.g., `traefik:v3.1`, `nginx:1.25`). If the tool version cannot be determined from the diff, note it as unknown.
|
|
1640
|
-
|
|
1641
|
-
2. **Reason about the schema**: Ask — "Does this change's structure match what `{tool} {version}` actually expects?" Key questions:
|
|
1642
|
-
- Is this directive/key nested under the correct parent section?
|
|
1643
|
-
- Does this version of the tool use a different config syntax than a prior version? (Version migrations often change required nesting — e.g., Traefik v2 → v3 moved many config keys to different parent sections)
|
|
1644
|
-
- Are any required sibling keys missing from the changed section?
|
|
1645
|
-
- Does the tool expect this value at file scope, or under a named block?
|
|
1646
|
-
|
|
1647
|
-
3. **Cross-reference against sibling config files**: If the project uses dynamic config loading (e.g., `traefik/dynamic/`), verify that a key defined in the static config is not also required in the dynamic config (or vice versa) for the feature to activate.
|
|
1648
|
-
|
|
1649
|
-
4. **Flag structural mismatches**: If any key or nesting does not match the tool's expected schema for the declared version, flag as **CONFIRMED HIGH**. Structural config mismatches are silently rejected at runtime — no startup crash, no error log, the feature simply does not activate.
|
|
1650
|
-
|
|
1651
|
-
```bash
|
|
1652
|
-
# Identify tool config files in the diff
|
|
1653
|
-
CONFIG_TOOL_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E \
|
|
1654
|
-
"(traefik/|infra/nginx/|k8s/|terraform/|.*\.conf$|.*\.toml$)" | head -20)
|
|
1655
|
-
COMPOSE_INFRA_FILES=$(gh pr diff [PR_NUMBER] --name-only | grep -E "docker-compose.*\.yml$")
|
|
1656
|
-
|
|
1657
|
-
# For each config file: read the full file (not just the diff) to understand the structure
|
|
1658
|
-
while IFS= read -r f; do
|
|
1659
|
-
[ -z "$f" ] && continue
|
|
1660
|
-
echo "=== Reading full config structure: $f ==="
|
|
1661
|
-
cat "$f"
|
|
1662
|
-
done <<< "$CONFIG_TOOL_FILES"
|
|
1663
|
-
|
|
1664
|
-
# Identify tool version from docker-compose (for version-aware schema reasoning)
|
|
1665
|
-
while IFS= read -r f; do
|
|
1666
|
-
[ -z "$f" ] && continue
|
|
1667
|
-
grep -E "image:\s*(traefik|nginx|haproxy|envoy|caddy|terraform):" "$f" 2>/dev/null && echo " ^ in $f"
|
|
1668
|
-
done <<< "$COMPOSE_INFRA_FILES"
|
|
1669
|
-
```
|
|
1670
|
-
|
|
1671
|
-
**Severity classification**:
|
|
1672
|
-
- Config key present but under wrong parent section: **CONFIRMED HIGH** — key is silently ignored; feature does not activate
|
|
1673
|
-
- Required sibling key missing from changed section: **CONFIRMED HIGH** — partial config is silently incomplete
|
|
1674
|
-
- Version mismatch (config uses v2 syntax on v3 tool): **CONFIRMED HIGH** — v3 silently ignores v2 keys in most tools
|
|
1675
|
-
- Structural ambiguity (key could plausibly be correct but version cannot be determined): **LIKELY MEDIUM** — flag for human verification with schema reference
|
|
1676
|
-
|
|
1677
|
-
**Do NOT rely solely on logical correctness checks when structural correctness has not been verified.** A config that names services correctly, uses coherent values, and matches the deploy script's expectations can still be silently rejected if the nesting does not match the tool's schema. Both dimensions must be verified independently. <!-- Added: forge#1104 -->
|
|
1678
|
-
|
|
1679
|
-
## Post Findings
|
|
1680
|
-
```bash
|
|
1681
|
-
gh pr comment [PR_NUMBER] --body "$(cat <<'EOF'
|
|
1682
|
-
## Infrastructure & Deploy Safety Audit
|
|
1683
|
-
|
|
1684
|
-
### Deploy Risk: [LOW/MEDIUM/HIGH/CRITICAL]
|
|
1685
|
-
|
|
1686
|
-
### Downtime Risk
|
|
1687
|
-
| Change | Restart Required? | Impact | Mitigation |
|
|
1688
|
-
|--------|-------------------|--------|------------|
|
|
1689
|
-
| ... | Yes/No | [duration] | [strategy] |
|
|
1690
|
-
|
|
1691
|
-
### Blue-Green Compatibility
|
|
1692
|
-
| Component | Old↔New Compatible? | Issue |
|
|
1693
|
-
|-----------|---------------------|-------|
|
|
1694
|
-
| API responses | Yes/No | ... |
|
|
1695
|
-
| Database | Yes/No | ... |
|
|
1696
|
-
| Redis | Yes/No | ... |
|
|
1697
|
-
|
|
1698
|
-
### Prerequisites
|
|
1699
|
-
- [ ] [Any env vars, secrets, DNS changes needed before deploy]
|
|
1700
|
-
|
|
1701
|
-
### Secret Delivery Chain
|
|
1702
|
-
[If decrypt-secrets.sh, SOPS, or deploy workflows changed — otherwise write "N/A"]
|
|
1703
|
-
| Step | File | Value | Consistent? |
|
|
1704
|
-
|------|------|-------|-------------|
|
|
1705
|
-
| SOPS key | `.secrets/prod.enc.yaml` | [section.key_name] | — |
|
|
1706
|
-
| ENV_MAPPING | `scripts/decrypt-secrets.sh` | [("section", "key_name")] | Yes/No |
|
|
1707
|
-
| SCP target | `deploy-production.yml` | [resolved path] | Yes/No |
|
|
1708
|
-
| Merge PROJECT | `deploy-production.yml` | [resolved path] | Yes/No |
|
|
1709
|
-
| env_file | `docker-compose.prod.yml` | [.env.production] | Yes/No |
|
|
1710
|
-
| Hotfix SCP | `hotfix-deploy.yml` | [resolved path] | Yes/No |
|
|
1711
|
-
|
|
1712
|
-
### Rollout Recommendation
|
|
1713
|
-
**Strategy**: [Full deploy / Canary / Low-traffic window]
|
|
1714
|
-
|
|
1715
|
-
### Files Reviewed
|
|
1716
|
-
[List files checked]
|
|
1717
|
-
|
|
1718
|
-
---
|
|
1719
|
-
*Infrastructure safety audit*
|
|
1720
|
-
|
|
1721
|
-
<!-- REVIEW-FINDINGS-START -->
|
|
1722
|
-
<!-- FINDING:INFRA-1|CONFIDENCE|SEVERITY|file.py:line|Summary -->
|
|
1723
|
-
<!-- (add one FINDING line per issue found — include ALL confidence levels) -->
|
|
1724
|
-
<!-- REVIEW-FINDINGS-END -->
|
|
1725
|
-
EOF
|
|
1726
|
-
)"
|
|
1727
|
-
```
|
|
1728
|
-
|
|
1729
|
-
**Structured Findings**: Include the structured findings block above at the end of your comment. Your prefix: `INFRA`. See the Structured Findings Protocol section above for format rules.
|
|
65
|
+
Read: $FORGE_HOME/commands/review-pr-agents/protocols.md
|
|
66
|
+
Read: $FORGE_HOME/commands/review-pr-agents/<persona>.md (one per selected agent)
|
|
1730
67
|
```
|
|
1731
68
|
|
|
1732
|
-
|
|
69
|
+
Each persona file contains:
|
|
70
|
+
- The agent's trigger condition and type
|
|
71
|
+
- Its full prompt template with all check items
|
|
72
|
+
- Its coverage matrix
|
|
1733
73
|
|
|
1734
|
-
|
|
1735
|
-
|
|
1736
|
-
|
|
1737
|
-
|
|
1738
|
-
|
|
1739
|
-
| Blue-green deployment compatibility | Item 2 | COVERED | |
|
|
1740
|
-
| Rollback path verification | Item 3 | COVERED | |
|
|
1741
|
-
| Missing deploy prerequisites (env/secrets/DNS) | Item 4 | COVERED | |
|
|
1742
|
-
| In-flight request impact | Item 5 | COVERED | |
|
|
1743
|
-
| Health endpoint response contract breakage | Item 6 | COVERED | #321 |
|
|
1744
|
-
| Secret delivery chain (SOPS → container) | Item 7 | COVERED | |
|
|
1745
|
-
| Shell metacharacter safety in .env writers | Item 7b | COVERED | #286 |
|
|
1746
|
-
| Cross-domain service interactions (curl/wget) | Item 8 | COVERED | |
|
|
1747
|
-
| Sibling workflow drift (CI vs deploy) | Item 9 | COVERED | #222 |
|
|
1748
|
-
| Deploy scope awareness (auto vs manual trigger) | Item 10 | COVERED | #239 |
|
|
1749
|
-
| Config field type/doc-comment consistency | Item 11 | COVERED | #190 |
|
|
1750
|
-
| appleboy/ssh-action Go template safety | Item 12 | COVERED | #226 |
|
|
1751
|
-
| Insecure config file defaults | Item 13 | COVERED | #301 |
|
|
1752
|
-
| Host-side database tool invocations | Item 14 | COVERED | #322 |
|
|
1753
|
-
| External tool config schema validation (structural correctness) | Item 15 | COVERED | #1104 |
|
|
1754
|
-
| Docker image tag pinning (mutable tags) | — | GAP | |
|
|
1755
|
-
| Network policy / inter-container isolation | — | GAP | |
|
|
74
|
+
The `protocols.md` file contains:
|
|
75
|
+
- Per-Agent Input Scoping rules
|
|
76
|
+
- Tool-Result Truncation Discipline
|
|
77
|
+
- Evidence-Based Review Protocol (all agents follow)
|
|
78
|
+
- Structured Findings Protocol (machine-readable findings block format)
|