forgedock 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +244 -0
- package/README.md +251 -0
- package/bin/forgedock.mjs +232 -0
- package/commands/analytics.md +777 -0
- package/commands/audit-agents.md +410 -0
- package/commands/audit.md +410 -0
- package/commands/autopilot.md +432 -0
- package/commands/cleanup.md +264 -0
- package/commands/deploy-info.md +292 -0
- package/commands/failure-recon.md +375 -0
- package/commands/forge-stats.md +183 -0
- package/commands/geo-audit.md +343 -0
- package/commands/incident-response.md +384 -0
- package/commands/issue.md +506 -0
- package/commands/milestone.md +599 -0
- package/commands/orchestrate.md +1044 -0
- package/commands/pipeline-health.md +2343 -0
- package/commands/qa-sweep.md +237 -0
- package/commands/quality-gate.md +777 -0
- package/commands/review-pr-agents.md +1711 -0
- package/commands/review-pr-staging.md +325 -0
- package/commands/review-pr.md +888 -0
- package/commands/rollback.md +253 -0
- package/commands/security-audit.md +529 -0
- package/commands/sync-ecosystem.md +527 -0
- package/commands/validate.md +131 -0
- package/commands/work-on/build/architect.md +291 -0
- package/commands/work-on/build/context.md +349 -0
- package/commands/work-on/build/implement.md +280 -0
- package/commands/work-on/build/validate.md +292 -0
- package/commands/work-on/build.md +262 -0
- package/commands/work-on/close.md +350 -0
- package/commands/work-on/decompose.md +254 -0
- package/commands/work-on/investigate.md +437 -0
- package/commands/work-on/review.md +245 -0
- package/commands/work-on-monolithic.md +298 -0
- package/commands/work-on.md +1089 -0
- package/docs/CODEX.md +92 -0
- package/docs/WORKFLOW.md +104 -0
- package/package.json +45 -0
- package/scripts/verify-env-vars.sh +201 -0
- package/scripts/verify-host-headers.sh +117 -0
- package/scripts/verify-route-registration.sh +118 -0
- package/scripts/verify-sops-chain.sh +163 -0
|
@@ -0,0 +1,777 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: Pre-commit quality check — catches defects the review would flag, so the builder can fix them before committing
|
|
3
|
+
argument-hint: (invoked by /work-on, not directly)
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# /quality-gate — Pre-Commit Quality Check
|
|
7
|
+
|
|
8
|
+
**Input**: $ARGUMENTS (worktree path and changed files list from the builder)
|
|
9
|
+
|
|
10
|
+
You are a quality gate that runs AFTER implementation but BEFORE commit. Your job is to scan the builder's code for the same defects the review agents would catch — so the builder can fix them before the PR even exists.
|
|
11
|
+
|
|
12
|
+
You do NOT post to GitHub. You do NOT create issues. You return findings directly to the builder agent that spawned you.
|
|
13
|
+
|
|
14
|
+
**Agent model policy**: Default `model: "sonnet"`. If Sonnet is rate-limited, fall back to `model: "opus"`.
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Input Format
|
|
19
|
+
|
|
20
|
+
The builder passes:
|
|
21
|
+
```
|
|
22
|
+
WORKTREE_PATH: /path/to/worktree
|
|
23
|
+
CHANGED_FILES: file1.py file2.tsx file3.sql
|
|
24
|
+
PR_BASE: staging|milestone/slug
|
|
25
|
+
TASK_TYPE: bug-fix|feature|refactor|investigation
|
|
26
|
+
ISSUE_NUMBER: #123
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
## Step 1: Read the diff
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
cd {WORKTREE_PATH}
|
|
33
|
+
git diff --cached --name-only 2>/dev/null || git diff HEAD --name-only
|
|
34
|
+
git diff --cached 2>/dev/null || git diff HEAD
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Read EVERY changed file in full — not just the diff. Context matters.
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## Step 1.5: Determine applicable domains
|
|
42
|
+
|
|
43
|
+
Before running checks, classify the changed files into domains. This avoids running irrelevant checks (e.g., database checks on a frontend-only change).
|
|
44
|
+
|
|
45
|
+
**Domain classification rules:**
|
|
46
|
+
|
|
47
|
+
| File pattern | Domain(s) triggered |
|
|
48
|
+
|---|---|
|
|
49
|
+
| `*.py` in `routers/` or `route.ts` handlers | AUTH |
|
|
50
|
+
| `*.py`, `*.ts`, `*.tsx`, `*.sh` (any code file) | SECURITY (always) |
|
|
51
|
+
| Files containing `os.getenv` or `process.env` | DEPLOY |
|
|
52
|
+
| `*.sql` or files in `migrations/` | DATABASE |
|
|
53
|
+
| `*.tsx`, `*.ts` in `web/src/` (excluding `route.ts`) | FRONTEND |
|
|
54
|
+
| `*.tsx`, `*.ts` client components with fetch/useSWR | PROXY |
|
|
55
|
+
| `*.sh` files or scripts with `curl`/`wget` | SHELL |
|
|
56
|
+
| `*.py` in `anti_detection/`, `consumers/`, `browser/`, or `shared/detection/` | STRING_SETS |
|
|
57
|
+
| `*.py` containing `asyncio.shield`, `asyncio.wait_for`, or `Task.cancel` | CONCURRENCY |
|
|
58
|
+
| `*.py` files with new module-level `dict`, `set`, `Counter`, `Lock`, or `defaultdict` assignments | STATE |
|
|
59
|
+
| `*.py` in `services/worker/`, `infra/`, or `browser/` with new assignments to variables containing `MB`, `SIZE`, `MAX`, `LIMIT`, `THRESHOLD`, or `TIMEOUT` | CAPACITY |
|
|
60
|
+
| `*.yml` in `.github/workflows/` | WORKFLOW |
|
|
61
|
+
| `*.py` with new `from app.` import statements added outside `try:` blocks | IMPORT_RESOLUTION |
|
|
62
|
+
| `Dockerfile*` or `entrypoint*.sh` with added/changed `USER`, `su-exec`, `gosu`, or `setuid` | INFRA |
|
|
63
|
+
| `*.py` in `routers/` where the diff removes a line containing an or-gate condition (lines starting with `-` containing `if.*or`) | ROUTER_BUG |
|
|
64
|
+
|
|
65
|
+
**Apply the classification:**
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
# Initialize domain flags
|
|
69
|
+
DOMAINS=""
|
|
70
|
+
|
|
71
|
+
for f in {CHANGED_FILES}; do
|
|
72
|
+
case "$f" in
|
|
73
|
+
*.sql|*migrations/*) DOMAINS="$DOMAINS DATABASE" ;;
|
|
74
|
+
esac
|
|
75
|
+
case "$f" in
|
|
76
|
+
*router*|*route.ts) DOMAINS="$DOMAINS AUTH" ;;
|
|
77
|
+
esac
|
|
78
|
+
case "$f" in
|
|
79
|
+
*.tsx|*.ts|*.jsx)
|
|
80
|
+
echo "$f" | grep -qv 'route\.ts$' && DOMAINS="$DOMAINS FRONTEND PROXY"
|
|
81
|
+
;;
|
|
82
|
+
esac
|
|
83
|
+
case "$f" in
|
|
84
|
+
*.sh) DOMAINS="$DOMAINS SHELL" ;;
|
|
85
|
+
esac
|
|
86
|
+
case "$f" in
|
|
87
|
+
*anti_detection/*|*consumers/*|*browser/*|*shared/detection/*)
|
|
88
|
+
echo "$f" | grep -qE '\.py$' && DOMAINS="$DOMAINS STRING_SETS"
|
|
89
|
+
;;
|
|
90
|
+
esac
|
|
91
|
+
case "$f" in
|
|
92
|
+
.github/workflows/*.yml) DOMAINS="$DOMAINS WORKFLOW" ;;
|
|
93
|
+
esac
|
|
94
|
+
done
|
|
95
|
+
|
|
96
|
+
# Check for asyncio concurrency patterns in Python files
|
|
97
|
+
grep -lE "asyncio\.shield|asyncio\.wait_for|Task\.cancel" {CHANGED_FILES} 2>/dev/null | grep -qE '\.py$' && DOMAINS="$DOMAINS CONCURRENCY"
|
|
98
|
+
|
|
99
|
+
# Check for new module-level state variable assignments in Python files
|
|
100
|
+
grep -lE "^\w+ = (\{\}|\[\]|set\(\)|Lock\(\)|defaultdict|Counter)" {CHANGED_FILES} 2>/dev/null | grep -qE '\.py$' && DOMAINS="$DOMAINS STATE"
|
|
101
|
+
|
|
102
|
+
# Check for capacity constants in worker/infra/browser Python files
|
|
103
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$' | grep -E 'services/worker/|infra/|browser/'); do
|
|
104
|
+
git diff HEAD -- "$f" 2>/dev/null | grep -qE '^\+[A-Za-z_]\w*(MB|SIZE|MAX|LIMIT|THRESHOLD|TIMEOUT)\w*\s*=' && DOMAINS="$DOMAINS CAPACITY" && break
|
|
105
|
+
done
|
|
106
|
+
|
|
107
|
+
# Check for env var usage in changed files
|
|
108
|
+
grep -lE "os\.getenv|process\.env" {CHANGED_FILES} 2>/dev/null && DOMAINS="$DOMAINS DEPLOY"
|
|
109
|
+
|
|
110
|
+
# Check for new app.* imports added outside try: blocks in Python files
|
|
111
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$'); do
|
|
112
|
+
NEW_IMPORTS=$(git diff HEAD -- "$f" 2>/dev/null | grep -E '^\+\s*(from app\.|import app\.)' | grep -v '^+++')
|
|
113
|
+
if [ -n "$NEW_IMPORTS" ]; then
|
|
114
|
+
DOMAINS="$DOMAINS IMPORT_RESOLUTION"
|
|
115
|
+
break
|
|
116
|
+
fi
|
|
117
|
+
done
|
|
118
|
+
|
|
119
|
+
# Check for runtime UID changes in Dockerfiles or entrypoint scripts
|
|
120
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -iE 'Dockerfile|entrypoint.*\.sh'); do
|
|
121
|
+
git diff HEAD -- "$f" 2>/dev/null | grep -E '^\+.*(USER\s+[^0]|su-exec|gosu|setuid)' | grep -v '^+++' | grep -q . && DOMAINS="$DOMAINS INFRA" && break
|
|
122
|
+
done
|
|
123
|
+
|
|
124
|
+
# Check for gate condition narrowing in router Python files (condition removal = fix may be incomplete across siblings)
|
|
125
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$' | grep -E 'router'); do
|
|
126
|
+
git diff HEAD -- "$f" 2>/dev/null | grep -E '^-.*\bif\b.*\bor\b' | grep -v '^---' | grep -q . && DOMAINS="$DOMAINS ROUTER_BUG" && break
|
|
127
|
+
done
|
|
128
|
+
|
|
129
|
+
# SECURITY is always included for any code file
|
|
130
|
+
DOMAINS="SECURITY $DOMAINS"
|
|
131
|
+
|
|
132
|
+
# Deduplicate
|
|
133
|
+
DOMAINS=$(echo "$DOMAINS" | tr ' ' '\n' | sort -u | tr '\n' ' ')
|
|
134
|
+
echo "Applicable domains: $DOMAINS"
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
**Only run checks in Step 2 that match the identified domains.** Skip all others entirely.
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## Step 2: Run domain checks
|
|
142
|
+
|
|
143
|
+
Run ONLY the checks whose domain was identified in Step 1.5. Skip checks for domains not present in the `DOMAINS` list.
|
|
144
|
+
|
|
145
|
+
- **2A (Security)**: Run if `SECURITY` in DOMAINS *(always — included for all code files)*
|
|
146
|
+
- **2B (Auth)**: Run if `AUTH` in DOMAINS
|
|
147
|
+
- **2C (Deploy chain)**: Run if `DEPLOY` in DOMAINS
|
|
148
|
+
- **2D (Database)**: Run if `DATABASE` in DOMAINS
|
|
149
|
+
- **2E (Frontend lifecycle)**: Run if `FRONTEND` in DOMAINS
|
|
150
|
+
- **2F (Frontend proxy)**: Run if `PROXY` in DOMAINS
|
|
151
|
+
- **2G (Cross-service integration)**: Run if `SHELL` in DOMAINS
|
|
152
|
+
- **2H (Asyncio cancellation)**: Run if `CONCURRENCY` in DOMAINS
|
|
153
|
+
- **2I (State completeness)**: Run if `STATE` in DOMAINS
|
|
154
|
+
- **2J (String literal consistency)**: Run if `STRING_SETS` in DOMAINS
|
|
155
|
+
- **2K (Capacity constant validation)**: Run if `CAPACITY` in DOMAINS
|
|
156
|
+
- **2L (GitHub Actions template safety)**: Run if `WORKFLOW` in DOMAINS
|
|
157
|
+
- **2M (Import resolution)**: Run if `IMPORT_RESOLUTION` in DOMAINS
|
|
158
|
+
- **2N (Runtime UID × filesystem write)**: Run if `INFRA` in DOMAINS
|
|
159
|
+
- **2O (Residual pattern check)**: Run if `ROUTER_BUG` in DOMAINS
|
|
160
|
+
|
|
161
|
+
### 2A: Security (ALL files)
|
|
162
|
+
|
|
163
|
+
For every changed file, check:
|
|
164
|
+
1. **SQL injection**: Any string formatting/f-strings in SQL queries? Must use parameterized queries (`$1`, `:param`).
|
|
165
|
+
2. **SSRF**: User-controlled URLs passed to `httpx`, `requests`, `fetch` without allowlist validation?
|
|
166
|
+
3. **Path traversal**: User input used in file paths without sanitization?
|
|
167
|
+
4. **Hardcoded secrets**: API keys, passwords, tokens in code (not env vars)?
|
|
168
|
+
5. **XSS**: User input rendered with `dangerouslySetInnerHTML`?
|
|
169
|
+
6. **Command injection**: User input in `subprocess`, `exec`, `eval`?
|
|
170
|
+
|
|
171
|
+
```bash
|
|
172
|
+
# Quick automated checks
|
|
173
|
+
for f in {CHANGED_FILES}; do
|
|
174
|
+
# f-string SQL
|
|
175
|
+
grep -nE "f['\"].*SELECT|f['\"].*INSERT|f['\"].*UPDATE|f['\"].*DELETE|f['\"].*WHERE" "$f" 2>/dev/null && echo "SEC: f-string SQL in $f"
|
|
176
|
+
# Hardcoded secrets
|
|
177
|
+
grep -nE "(api_key|secret|password|token)\s*=\s*(f?['\"]|\`)[^{'\"\`]" "$f" 2>/dev/null | grep -vE "(example|placeholder|test|mock)" && echo "SEC: possible hardcoded secret in $f"
|
|
178
|
+
# In-memory rate limiter (TypeScript only) — resets on restart, no cross-replica enforcement
|
|
179
|
+
# Pattern: in-memory Maps/objects used for rate limiting reset on container restart and are not shared across replicas
|
|
180
|
+
case "$f" in
|
|
181
|
+
*.ts|*.tsx)
|
|
182
|
+
grep -nE "new Map\(\)|=\s*\{\}|= new Map" "$f" 2>/dev/null | grep -iE "rate|limit|attempt|count|throttle|window" && \
|
|
183
|
+
echo "SEC: in-memory rate limiter in $f — resets on container restart, not shared across blue/green replicas. Use Redis for distributed rate limiting."
|
|
184
|
+
# Cookie security flags — explicit false/none values are CONFIRMED findings
|
|
185
|
+
# Pattern: explicitly weakened cookie security flags expose sessions to XSS, plaintext interception, or CSRF
|
|
186
|
+
grep -nE "httpOnly:\s*false|secure:\s*false|sameSite.*['\"]none['\"]" "$f" 2>/dev/null && \
|
|
187
|
+
echo "SEC: cookie security flag weakened in $f — httpOnly: false exposes cookie to JS (XSS); secure: false allows HTTP transmission; sameSite: none without Secure enables CSRF."
|
|
188
|
+
;;
|
|
189
|
+
esac
|
|
190
|
+
done
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
### 2B: Auth conventions (Python routers, TypeScript route handlers)
|
|
194
|
+
|
|
195
|
+
For new/modified endpoints:
|
|
196
|
+
1. Which auth dependency is used — `SessionUser` or `CurrentUser`?
|
|
197
|
+
2. Does the route path match? Dashboard routes (`/api/dashboard/*`) must use `SessionUser`. Public API routes (`/api/v1/*`) must use `CurrentUser`.
|
|
198
|
+
3. Does the endpoint check resource ownership (`user_id`) before returning data?
|
|
199
|
+
|
|
200
|
+
```bash
|
|
201
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E 'router|route'); do
|
|
202
|
+
grep -nE "CurrentUser|SessionUser|Depends\(get_" "$f" 2>/dev/null
|
|
203
|
+
grep -nE "@router\.(get|post|put|delete)" "$f" 2>/dev/null
|
|
204
|
+
done
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
### 2C: Deploy chain (when new env vars are introduced)
|
|
208
|
+
|
|
209
|
+
```bash
|
|
210
|
+
NEW_ENVS=$(grep -rnE "os\.getenv\(|process\.env\." {CHANGED_FILES} 2>/dev/null | grep -oP "(os\.getenv\(['\"]|process\.env\.)(\w+)" | sed 's/os.getenv(["'"'"']//;s/process.env.//' | sort -u)
|
|
211
|
+
if [ -n "$NEW_ENVS" ]; then
|
|
212
|
+
for var in $NEW_ENVS; do
|
|
213
|
+
# Check if it exists in .env.example
|
|
214
|
+
grep -q "$var" .env.example 2>/dev/null || echo "DEPLOY: $var not in .env.example"
|
|
215
|
+
# Check if it's in decrypt-secrets.sh ENV_MAPPING
|
|
216
|
+
grep -q "$var" scripts/decrypt-secrets.sh 2>/dev/null || echo "DEPLOY: $var not in decrypt-secrets.sh ENV_MAPPING"
|
|
217
|
+
done
|
|
218
|
+
fi
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
### 2D: Database quality (SQL migration files)
|
|
222
|
+
|
|
223
|
+
For each `.sql` file:
|
|
224
|
+
1. **NOT NULL without DEFAULT**: `ALTER TABLE ... ADD COLUMN ... NOT NULL` without `DEFAULT` locks table and fails on existing rows
|
|
225
|
+
2. **DROP without IF EXISTS**: Fails on fresh databases
|
|
226
|
+
3. **Missing indexes**: New columns used in WHERE/JOIN without indexes
|
|
227
|
+
4. **Unbounded queries**: SELECT without LIMIT or time-bound WHERE clause
|
|
228
|
+
5. **Migration number collision**: Check against origin branch
|
|
229
|
+
|
|
230
|
+
```bash
|
|
231
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep '\.sql$'); do
|
|
232
|
+
grep -nE "ADD COLUMN.*NOT NULL" "$f" | grep -v "DEFAULT" && echo "DB: NOT NULL without DEFAULT in $f"
|
|
233
|
+
grep -nE "DROP (TABLE|COLUMN|INDEX)" "$f" | grep -v "IF EXISTS" && echo "DB: DROP without IF EXISTS in $f"
|
|
234
|
+
grep -nE "SELECT.*FROM" "$f" | grep -v "LIMIT" | grep -v "WHERE.*created_at" && echo "DB: possibly unbounded query in $f"
|
|
235
|
+
done
|
|
236
|
+
|
|
237
|
+
# Migration prefix collision scan (Deploy Gate — HIGH)
|
|
238
|
+
# Runs whenever any SQL file changed. Scans the full infra/migrations/ tree for duplicate
|
|
239
|
+
# numeric prefixes not in GRANDFATHERED_DUPLICATES. Duplicates not in the allowlist WILL
|
|
240
|
+
# hard-fail deploy-production.yml's validate-migration-order.sh gate.
|
|
241
|
+
MIGRATIONS_DIR="{WORKTREE_PATH}/infra/migrations"
|
|
242
|
+
if [ -d "$MIGRATIONS_DIR" ]; then
|
|
243
|
+
# Extract grandfathered prefixes from validate-migration-order.sh if it exists
|
|
244
|
+
GRANDFATHER_SCRIPT="{WORKTREE_PATH}/scripts/validate-migration-order.sh"
|
|
245
|
+
GRANDFATHERED=""
|
|
246
|
+
if [ -f "$GRANDFATHER_SCRIPT" ]; then
|
|
247
|
+
GRANDFATHERED=$(grep -oE '[0-9]{4}' "$GRANDFATHER_SCRIPT" | sort -u | tr '\n' ' ')
|
|
248
|
+
fi
|
|
249
|
+
# Find duplicate numeric prefixes across all migration files
|
|
250
|
+
DUPLICATE_PREFIXES=$(ls "$MIGRATIONS_DIR"/*.sql 2>/dev/null \
|
|
251
|
+
| sed 's|.*/\([0-9]*\)_.*|\1|' \
|
|
252
|
+
| sort | uniq -d)
|
|
253
|
+
for prefix in $DUPLICATE_PREFIXES; do
|
|
254
|
+
if echo "$GRANDFATHERED" | grep -qw "$prefix"; then
|
|
255
|
+
echo "DB: migration prefix $prefix is duplicated but grandfathered (INFO)"
|
|
256
|
+
else
|
|
257
|
+
echo "DB-COLLISION | HIGH | infra/migrations/ | Duplicate migration prefix $prefix is NOT grandfathered — will hard-fail deploy-production.yml validate-migration-order.sh gate. Classify as CRITICAL BLOCKER."
|
|
258
|
+
fi
|
|
259
|
+
done
|
|
260
|
+
fi
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
### 2E: Frontend lifecycle (React/TSX files)
|
|
264
|
+
|
|
265
|
+
For each `.tsx`/`.ts` client component:
|
|
266
|
+
1. **useEffect cleanup**: Does every useEffect that creates subscriptions/timers return a cleanup function?
|
|
267
|
+
2. **useCallback deps**: Are dependencies stable values (ids, primitives) not full objects (SWR responses)?
|
|
268
|
+
3. **Missing error/loading states**: Does data fetching handle loading and error?
|
|
269
|
+
4. **Component unmount**: Are async operations cancelled on unmount?
|
|
270
|
+
5. **Button type attribute**: Do all `<button>` elements have an explicit `type` attribute?
|
|
271
|
+
6. **Polling loop guard**: Do all polling/retry loops have a max-iteration or timeout guard?
|
|
272
|
+
7. **Rules of Hooks**: Do function components avoid early returns before hook calls?
|
|
273
|
+
|
|
274
|
+
```bash
|
|
275
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.tsx?$' | grep -v 'route\.ts'); do
|
|
276
|
+
# Check for useEffect without cleanup (basic heuristic)
|
|
277
|
+
grep -c "useEffect" "$f" 2>/dev/null
|
|
278
|
+
grep -c "return.*cleanup\|return.*\(\) =>" "$f" 2>/dev/null
|
|
279
|
+
|
|
280
|
+
# FE-5: button missing explicit type attribute
|
|
281
|
+
# Handles both single-line (<button onClick=...>) and multi-line JSX (type= on next line).
|
|
282
|
+
# Greps a 2-line window: flag only if neither the button line nor the following line contains type=
|
|
283
|
+
grep -nE "<button\b" "$f" 2>/dev/null | while IFS=: read -r lineno rest; do
|
|
284
|
+
window=$(sed -n "${lineno},$((lineno+1))p" "$f" 2>/dev/null)
|
|
285
|
+
echo "$window" | grep -qE 'type\s*=' || \
|
|
286
|
+
echo "FE-5 | MEDIUM | $f:$lineno | <button> missing explicit type= attribute — add type=\"button\" or type=\"submit\" to prevent accidental form submission"
|
|
287
|
+
done
|
|
288
|
+
|
|
289
|
+
# FE-6: polling/retry loop without max-iteration guard
|
|
290
|
+
# Triggered by setTimeout/setInterval/requestAnimationFrame used in a recursive or loop context.
|
|
291
|
+
# Checks for absence of a guard variable (maxRetries, maxAttempts, attempt, count, retries, MAX_)
|
|
292
|
+
# within 10 lines of the polling call.
|
|
293
|
+
grep -nE "\b(setTimeout|setInterval|requestAnimationFrame)\b" "$f" 2>/dev/null | while IFS=: read -r lineno rest; do
|
|
294
|
+
start=$((lineno > 5 ? lineno - 5 : 1))
|
|
295
|
+
window=$(sed -n "${start},$((lineno+10))p" "$f" 2>/dev/null)
|
|
296
|
+
# Skip if a guard variable or clearTimeout/clearInterval is nearby
|
|
297
|
+
echo "$window" | grep -qiE '\b(maxRetries|maxAttempts|attempts|retries|retryCount|count|MAX_RETRIES|MAX_ATTEMPTS|clearTimeout|clearInterval)\b' || \
|
|
298
|
+
echo "FE-6 | MEDIUM | $f:$lineno | polling/retry call without visible max-iteration guard — add a retry counter or timeout limit to prevent infinite loops"
|
|
299
|
+
done
|
|
300
|
+
|
|
301
|
+
# FE-7: Rules of Hooks — early return before hook call (conservative, false positives possible)
|
|
302
|
+
# Flags only unconditional top-level return statements that appear before any use* call in the
|
|
303
|
+
# same function component. Does NOT flag returns inside if/else/switch blocks.
|
|
304
|
+
# Note: this is a heuristic — complex components with valid conditional logic may false-positive.
|
|
305
|
+
# Severity: MEDIUM. The review agent verifies with full AST context.
|
|
306
|
+
awk '
|
|
307
|
+
/^(export\s+)?(default\s+)?function\s+[A-Z]/ { in_component=1; found_hook=0; depth=0 }
|
|
308
|
+
in_component && /\{/ { depth++ }
|
|
309
|
+
in_component && /\}/ { depth--; if (depth <= 0) { in_component=0 } }
|
|
310
|
+
in_component && depth==1 && /^\s*return\b/ && !found_hook {
|
|
311
|
+
print FILENAME ":" NR ": FE-7 | MEDIUM | early return before first hook call — verify this does not split hook execution (Rules of Hooks)"
|
|
312
|
+
}
|
|
313
|
+
in_component && /\buse[A-Z]/ { found_hook=1 }
|
|
314
|
+
' FILENAME="$f" "$f" 2>/dev/null
|
|
315
|
+
done
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
### 2F: Frontend proxy wiring (client-side fetch calls)
|
|
319
|
+
|
|
320
|
+
```bash
|
|
321
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.(tsx?|jsx?)$' | grep -v 'route\.ts$'); do
|
|
322
|
+
grep -nE '(fetch|useSWR|apiFetch)\s*[(<]\s*[`"'"'"']/api/v1/' "$f" 2>/dev/null && echo "PROXY: direct /api/v1/ call in $f — must use /api/ proxy"
|
|
323
|
+
done
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
### 2G: Cross-service integration (shell scripts with curl/wget)
|
|
327
|
+
|
|
328
|
+
For shell scripts that make HTTP requests to internal services:
|
|
329
|
+
1. Read the target service's middleware to verify the Host header will be accepted
|
|
330
|
+
2. Check that auth tokens are included if the endpoint requires them
|
|
331
|
+
3. Verify the URL path actually exists
|
|
332
|
+
|
|
333
|
+
### 2G.5: Hardcoded localhost in DB connection functions (shell scripts)
|
|
334
|
+
|
|
335
|
+
**Triggered when**: changed `*.sh` files contain DB connection patterns (`PGPASSWORD`, `createdb`, `psql`, `pg_dump`, `pg_restore`).
|
|
336
|
+
|
|
337
|
+
**Why this matters**: Every DB connection function in a deploy script must use `${POSTGRES_HOST:-localhost}` (or equivalent env var interpolation), not a bare `"localhost"` string literal. A hardcoded `"localhost"` works in local development but fails in any environment where PostgreSQL runs on a separate host. The pattern is reliably grep-detectable. <!-- Added: forge#303 -->
|
|
338
|
+
|
|
339
|
+
```bash
|
|
340
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.sh$'); do
|
|
341
|
+
# Only check files that contain DB connection tools
|
|
342
|
+
if grep -qE "PGPASSWORD|createdb|psql|pg_dump|pg_restore" "$f" 2>/dev/null; then
|
|
343
|
+
# Look for new/modified lines with hardcoded localhost in variable assignments
|
|
344
|
+
# Match: db_host="localhost", db_host='localhost', db_host=localhost (unquoted),
|
|
345
|
+
# local db_host=localhost (bash local keyword + unquoted) in DB context
|
|
346
|
+
# Do NOT match: ${POSTGRES_HOST:-localhost} or ${DB_HOST:-localhost} (env var with fallback)
|
|
347
|
+
HARDCODED=$(git diff HEAD -- "$f" 2>/dev/null | grep '^+' | grep -v '^+++' \
|
|
348
|
+
| grep -E '\b(local\s+)?(db_host|host|DB_HOST)\s*=\s*["\047]?localhost["\047]?' \
|
|
349
|
+
| grep -vE '\$\{[A-Z_]+:-localhost\}|\$[A-Z_]+')
|
|
350
|
+
if [ -n "$HARDCODED" ]; then
|
|
351
|
+
echo "SHELL-1 | HIGH | $f | Hardcoded 'localhost' in DB connection function — use \${POSTGRES_HOST:-localhost} pattern instead. Hardcoded localhost fails in production where PostgreSQL runs on a separate host."
|
|
352
|
+
echo "$HARDCODED"
|
|
353
|
+
fi
|
|
354
|
+
fi
|
|
355
|
+
done
|
|
356
|
+
```
|
|
357
|
+
|
|
358
|
+
Report any hit as **HIGH** — the DB connection will fail in any environment where PostgreSQL is not on `127.0.0.1:5432`.
|
|
359
|
+
|
|
360
|
+
### 2H: Asyncio cancellation safety (Python async code)
|
|
361
|
+
|
|
362
|
+
**Triggered when**: changed Python files contain `asyncio.shield`, `asyncio.wait_for`, or `Task.cancel`.
|
|
363
|
+
|
|
364
|
+
**Why this matters**: `asyncio.shield()` is one of the most misused async primitives — it fails silently when `CancelledError` is already pending on the outer task. `asyncio.wait_for` with `finally` blocks can deadlock if the finally body awaits a cancellable operation. These bugs are hard to reproduce and often only surface under load or connection teardown.
|
|
365
|
+
|
|
366
|
+
**Severity**: MEDIUM (flags for review, not auto-blocking). The review agent's concurrency auditor verifies with full context.
|
|
367
|
+
|
|
368
|
+
```bash
|
|
369
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$'); do
|
|
370
|
+
# asyncio.shield usage — outer await may be reached with pending CancelledError
|
|
371
|
+
grep -nE "asyncio\.shield" "$f" 2>/dev/null && \
|
|
372
|
+
echo "ASYNC: asyncio.shield in $f — verify outer await isn't reachable with pending CancelledError. Prefer loop.create_task() for fire-and-forget."
|
|
373
|
+
|
|
374
|
+
# asyncio.wait_for — verify all finally blocks don't await cancellable operations
|
|
375
|
+
grep -nE "asyncio\.wait_for" "$f" 2>/dev/null && \
|
|
376
|
+
echo "ASYNC: asyncio.wait_for in $f — verify all finally blocks handle CancelledError without awaiting cancellable operations"
|
|
377
|
+
|
|
378
|
+
# await inside except/finally BaseException blocks — where shield failures manifest
|
|
379
|
+
grep -nE "^[[:space:]]+(except BaseException|finally):" "$f" 2>/dev/null | while IFS= read -r line; do
|
|
380
|
+
LINENO=$(echo "$line" | cut -d: -f1)
|
|
381
|
+
sed -n "$((LINENO+1)),$((LINENO+5))p" "$f" | grep -q "await" && \
|
|
382
|
+
echo "ASYNC: await in except/finally block at $f:$LINENO — may fail if CancelledError is pending"
|
|
383
|
+
done
|
|
384
|
+
done
|
|
385
|
+
```
|
|
386
|
+
|
|
387
|
+
|
|
388
|
+
### 2I: State completeness (module-level counters/dicts/sets/locks)
|
|
389
|
+
|
|
390
|
+
**Triggered when**: changed Python files introduce new module-level `dict`, `set`, `Counter`, `Lock`, or `defaultdict` assignments (detected via the STATE domain flag above).
|
|
391
|
+
|
|
392
|
+
**Why this matters**: When a new state variable is added alongside an existing one (e.g., `_user_active_browsers` alongside `_browser_active_count`), every function that mutates the existing variable is a potential mutation site for the new variable. Missing even one site causes silent data divergence — the kind of bug that only surfaces under concurrent load or connection teardown.
|
|
393
|
+
|
|
394
|
+
For each new module-level state variable in the diff:
|
|
395
|
+
1. Identify its sibling — the existing state variable it mirrors or is paired with (same file, similar type and naming pattern)
|
|
396
|
+
2. Find all mutation sites of the sibling in the same module:
|
|
397
|
+
```bash
|
|
398
|
+
grep -n "sibling_name\[" <module_file>
|
|
399
|
+
grep -n "sibling_name\.pop\|sibling_name\.get\|sibling_name\s*=" <module_file>
|
|
400
|
+
```
|
|
401
|
+
3. For each sibling mutation site NOT already in the current diff, flag it:
|
|
402
|
+
```
|
|
403
|
+
STATE: {sibling} mutated at {file}:{line} but {new_var} not handled — verify intentional
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
```bash
|
|
407
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$'); do
|
|
408
|
+
# Find new module-level state variables introduced in the diff
|
|
409
|
+
NEW_VARS=$(git diff HEAD -- "$f" 2>/dev/null | grep -E '^\+[A-Za-z_]\w* = (\{\}|\[\]|set\(\)|Lock\(\)|defaultdict|Counter)' | grep -oE '^[+][A-Za-z_]\w*' | tr -d '+' | sort -u)
|
|
410
|
+
for new_var in $NEW_VARS; do
|
|
411
|
+
# Find existing sibling variables (same file, similar type) — look at non-diff lines
|
|
412
|
+
SIBLINGS=$(grep -nE "^[A-Za-z_]\w+ = (\{\}|\[\]|set\(\)|Lock\(\)|defaultdict|Counter)" "$f" 2>/dev/null | grep -v "$new_var" | grep -oE "^[0-9]+:[A-Za-z_]\w+" | head -5)
|
|
413
|
+
for sibling_entry in $SIBLINGS; do
|
|
414
|
+
sibling=$(echo "$sibling_entry" | cut -d: -f2)
|
|
415
|
+
# Find all mutation sites of this sibling
|
|
416
|
+
grep -nE "${sibling}\[|${sibling}\.pop|${sibling}\.get|${sibling}\s*=" "$f" 2>/dev/null | while read -r match; do
|
|
417
|
+
line_num=$(echo "$match" | cut -d: -f1)
|
|
418
|
+
# Check if this line is in the diff (already handled)
|
|
419
|
+
git diff HEAD -- "$f" 2>/dev/null | grep -qE "^\+.*$(echo "$match" | cut -d: -f2-)" || \
|
|
420
|
+
echo "STATE: $sibling mutated at $f:$line_num but $new_var not handled — verify intentional"
|
|
421
|
+
done
|
|
422
|
+
done
|
|
423
|
+
done
|
|
424
|
+
done
|
|
425
|
+
```
|
|
426
|
+
|
|
427
|
+
**Severity**: MEDIUM — missing a mutation site is usually a latent bug, not an immediate crash, but it causes incorrect behavior under concurrent load.
|
|
428
|
+
|
|
429
|
+
---
|
|
430
|
+
|
|
431
|
+
### 2J: String literal consistency (anti-detection/scraping code)
|
|
432
|
+
|
|
433
|
+
**Triggered when**: changed files include Python files in `anti_detection/`, `consumers/`, `browser/`, or `shared/detection/` directories.
|
|
434
|
+
|
|
435
|
+
**Why this matters**: Detection keyword sets (e.g., `CHALLENGE_KEYWORDS`, `COMMON_COOKIES`) across related modules share a domain truth — if Akamai adds a new cookie marker, it must appear in ALL related sets. A typo or omission in one set silently breaks detection for that service path. This check catches both inter-file inconsistencies and intra-file comment/code drift.
|
|
436
|
+
|
|
437
|
+
For each changed file in the anti-detection/scraping domains:
|
|
438
|
+
|
|
439
|
+
**1. Identify modified string identifier sets** (frozensets, tuples, or lists of string literals used for detection):
|
|
440
|
+
|
|
441
|
+
```bash
|
|
442
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$' | grep -E 'anti_detection/|consumers/|browser/|shared/detection/'); do
|
|
443
|
+
echo "=== String sets in $f ==="
|
|
444
|
+
# Find named constants that are frozenset/tuple/list of strings
|
|
445
|
+
grep -nE "^[A-Z_]+ = (frozenset|tuple|set)\(\[?['\"]|^[A-Z_]+ = \(['\"]|^[A-Z_]+ = \[['\"]" "$f" 2>/dev/null
|
|
446
|
+
done
|
|
447
|
+
```
|
|
448
|
+
|
|
449
|
+
**2. For each string in a modified set, check if it appears in sibling detection files:**
|
|
450
|
+
|
|
451
|
+
```bash
|
|
452
|
+
SERVICE_DIR=$(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E 'anti_detection/|consumers/|browser/' | head -1 | grep -oE '^services/[^/]+/[^/]+' || echo "services/worker/worker")
|
|
453
|
+
|
|
454
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$' | grep -E 'anti_detection/|consumers/|browser/|shared/detection/'); do
|
|
455
|
+
# Extract string values from detection sets in this file
|
|
456
|
+
STRINGS=$(grep -oE "'[a-z_][a-z0-9_-]*'" "$f" 2>/dev/null | tr -d "'" | sort -u)
|
|
457
|
+
for s in $STRINGS; do
|
|
458
|
+
# Check if this string appears in at least one other file in the service
|
|
459
|
+
FOUND=$(grep -rl "\"$s\"\|'$s'" $SERVICE_DIR/ 2>/dev/null | grep -v "^$f$" | grep '\.py$' | head -1)
|
|
460
|
+
[ -z "$FOUND" ] && echo "STR: '$s' in $f appears NOWHERE else in $SERVICE_DIR — possible typo or dead entry"
|
|
461
|
+
done
|
|
462
|
+
done
|
|
463
|
+
```
|
|
464
|
+
|
|
465
|
+
**3. Check for identifiers mentioned in comments/docstrings of the SAME file that are NOT in the actual set** (catches the `abck` vs `_abck` class of bug):
|
|
466
|
+
|
|
467
|
+
```bash
|
|
468
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$' | grep -E 'anti_detection/|consumers/|browser/|shared/detection/'); do
|
|
469
|
+
echo "=== Comment/code drift in $f ==="
|
|
470
|
+
# Extract string literals from actual sets (remove quotes)
|
|
471
|
+
SET_STRINGS=$(grep -oE "frozenset\(\[([^]]+)\]\)" "$f" 2>/dev/null | grep -oE "'[^']+'" | tr -d "'" | sort -u)
|
|
472
|
+
SET_STRINGS="$SET_STRINGS $(grep -oE "\(([^)]*'[^']*'[^)]*)\)" "$f" 2>/dev/null | grep -oE "'[a-z_][a-z0-9_-]*'" | tr -d "'" | sort -u)"
|
|
473
|
+
|
|
474
|
+
# Find identifiers mentioned in comments (after # or in docstrings)
|
|
475
|
+
COMMENT_REFS=$(grep -oE "#.*\`[a-z_][a-z0-9_-]+\`|\"\"\".*[a-z_][a-z0-9_-]+.*\"\"\"" "$f" 2>/dev/null | grep -oE "\`[a-z_][a-z0-9_-]+\`|_[a-z][a-z0-9_]+" | tr -d '`' | sort -u)
|
|
476
|
+
|
|
477
|
+
for ref in $COMMENT_REFS; do
|
|
478
|
+
echo "$SET_STRINGS" | grep -q "^${ref}$" || echo "STR: '$ref' mentioned in comment in $f but NOT found in any string set — possible omission or typo"
|
|
479
|
+
done
|
|
480
|
+
done
|
|
481
|
+
```
|
|
482
|
+
|
|
483
|
+
**4. Cross-reference sibling sets in related modules** (the primary inter-file check):
|
|
484
|
+
|
|
485
|
+
```bash
|
|
486
|
+
# Find all detection-related Python files in the same service
|
|
487
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$' | grep -E 'anti_detection/|consumers/|browser/'); do
|
|
488
|
+
SIBLINGS=$(find $(dirname "$f")/../ -name "*.py" -path "*/anti_detection/*" -o -name "*.py" -path "*/consumers/*" -o -name "*.py" -path "*/browser/*" 2>/dev/null | grep -v "^$f$" | head -10)
|
|
489
|
+
|
|
490
|
+
# Extract detection set names from the changed file
|
|
491
|
+
SET_NAMES=$(grep -oE "^[A-Z_]+ = (frozenset|tuple|set)" "$f" 2>/dev/null | grep -oE "^[A-Z_]+")
|
|
492
|
+
|
|
493
|
+
for sib in $SIBLINGS; do
|
|
494
|
+
# Extract strings from sibling detection sets
|
|
495
|
+
SIB_STRINGS=$(grep -oE "'[a-z_][a-z0-9_-]*'" "$sib" 2>/dev/null | tr -d "'" | sort -u)
|
|
496
|
+
|
|
497
|
+
# Find strings in the changed file's sets that don't appear in the sibling
|
|
498
|
+
for s in $(grep -oE "'[a-z_][a-z0-9_-]*'" "$f" 2>/dev/null | tr -d "'" | sort -u); do
|
|
499
|
+
# Skip very short strings (likely not identifiers)
|
|
500
|
+
[ ${#s} -lt 4 ] && continue
|
|
501
|
+
echo "$SIB_STRINGS" | grep -q "^${s}$" || \
|
|
502
|
+
echo "STR: '$s' in $f not found in sibling file $sib — verify if cross-module consistency is required"
|
|
503
|
+
done
|
|
504
|
+
done
|
|
505
|
+
done
|
|
506
|
+
```
|
|
507
|
+
|
|
508
|
+
**Flag as findings**:
|
|
509
|
+
- Strings appearing nowhere else in the service: possible typo or dead entry (MEDIUM)
|
|
510
|
+
- Strings in comments not present in actual set: possible omission or typo (HIGH — same-file drift is almost always a bug)
|
|
511
|
+
- Strings in one detection set missing from a known sibling set (same service, related domain): possible inter-module inconsistency (MEDIUM)
|
|
512
|
+
|
|
513
|
+
---
|
|
514
|
+
|
|
515
|
+
### 2K: Capacity constant validation (worker/infra code)
|
|
516
|
+
|
|
517
|
+
**Triggered when**: changed Python files in `services/worker/`, `infra/`, or `browser/` introduce new assignments to variables whose names contain `MB`, `SIZE`, `MAX`, `LIMIT`, `THRESHOLD`, or `TIMEOUT`.
|
|
518
|
+
|
|
519
|
+
**Why this matters**: Hardcoded capacity constants (e.g., `_BROWSER_ESTIMATED_MB = 200`) are design-time guesses that drift from production reality. When used in guard conditions (e.g., "5 × 200 MB = 1000 MB peak"), a wrong constant directly limits system throughput and may never be caught until a slot fails to spawn. A constant is correctly-typed and correctly-used but factually wrong — logic checks pass, the bug ships.
|
|
520
|
+
|
|
521
|
+
For each new or modified capacity constant in the diff:
|
|
522
|
+
|
|
523
|
+
```bash
|
|
524
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$' | grep -E 'services/worker/|infra/|browser/'); do
|
|
525
|
+
# Find new capacity constant assignments introduced by this diff
|
|
526
|
+
NEW_CAPS=$(git diff HEAD -- "$f" 2>/dev/null | grep -E '^\+[A-Za-z_]\w*(MB|SIZE|MAX|LIMIT|THRESHOLD|TIMEOUT)\w*\s*=' | grep -oE '^[+][A-Za-z_]\w+' | tr -d '+' | sort -u)
|
|
527
|
+
for cap_var in $NEW_CAPS; do
|
|
528
|
+
cap_val=$(grep -oE "^${cap_var}\s*=\s*[0-9]+" "$f" 2>/dev/null | grep -oE '[0-9]+$')
|
|
529
|
+
|
|
530
|
+
# Check if a measurement annotation exists in the same file (inline comment on same line or preceding line)
|
|
531
|
+
annotated=$(grep -E "^${cap_var}\s*=" "$f" 2>/dev/null | grep -iE "measured|validated|observed|benchmark|production|empirical|PR #[0-9]+|issue #[0-9]+")
|
|
532
|
+
if [ -z "$annotated" ]; then
|
|
533
|
+
echo "CAP: ${cap_var}=${cap_val} in $f — no measurement annotation found (add 'VALIDATED: <source>' comment or link issue/PR)"
|
|
534
|
+
fi
|
|
535
|
+
|
|
536
|
+
# Search git log for prior art that may contradict this value
|
|
537
|
+
# Use the variable name keywords to search — strip common prefixes/suffixes
|
|
538
|
+
keywords=$(echo "$cap_var" | tr '_' ' ' | tr '[:upper:]' '[:lower:]' | sed 's/\b\(max\|min\|limit\|size\|mb\|threshold\|timeout\|estimated\|browser\|pool\|worker\)\b/ /g' | tr -s ' ')
|
|
539
|
+
git log --all -15 --oneline --grep="memory" 2>/dev/null | head -5
|
|
540
|
+
git log --all -15 --oneline --grep="$(echo $keywords | cut -d' ' -f1)" 2>/dev/null | head -5
|
|
541
|
+
done
|
|
542
|
+
done
|
|
543
|
+
```
|
|
544
|
+
|
|
545
|
+
**Flag as findings**:
|
|
546
|
+
- Capacity constant with no measurement annotation: MEDIUM — "Hardcoded constant without measurement source — add `VALIDATED: <source>` comment or link issue/PR"
|
|
547
|
+
- Capacity constant where git history contains contradicting measured values: HIGH — "Hardcoded constant contradicts measured data in git history — verify before deploy"
|
|
548
|
+
|
|
549
|
+
**Note**: This check does NOT fail constants that cite a source. A comment like `# VALIDATED: observed 200-250MB per browser under load (2026-03-01)` or a linked PR is sufficient to pass.
|
|
550
|
+
|
|
551
|
+
---
|
|
552
|
+
|
|
553
|
+
### 2L: GitHub Actions template safety (workflow files with appleboy/ssh-action)
|
|
554
|
+
|
|
555
|
+
**Triggered when**: changed files include `.github/workflows/*.yml` files.
|
|
556
|
+
|
|
557
|
+
**Why this matters**: `appleboy/ssh-action` processes every `script:` field through Go's `text/template` engine **client-side, before SSH transmission**. Any `{{` in the script — including in comments, `docker ps --format` strings, and `docker inspect --format` strings — is interpreted as a Go template directive. If the expression does not resolve against the action's data context (which is an empty map for `appleboy/ssh-action`), the step exits with status 1 **before the script reaches the remote shell**. Shell error handlers (`|| fallback`, `2>/dev/null`, `set -e`) are completely bypassed. `continue-on-error: true` masks the failure silently, producing apparent deploy success with no remote shell output.
|
|
558
|
+
|
|
559
|
+
```bash
|
|
560
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.github/workflows/.*\.yml$'); do
|
|
561
|
+
if grep -q "appleboy/ssh-action" "$f"; then
|
|
562
|
+
TEMPLATE_HITS=$(grep -n "{{" "$f")
|
|
563
|
+
if [ -n "$TEMPLATE_HITS" ]; then
|
|
564
|
+
echo "WORKFLOW-1 | HIGH | $f | Go template syntax found in file using appleboy/ssh-action"
|
|
565
|
+
echo "$TEMPLATE_HITS"
|
|
566
|
+
echo "ACTION REQUIRED: Replace docker inspect --format '{{...}}' and docker ps --format '{{...}}'"
|
|
567
|
+
echo "with jq equivalents. ANY {{ in an appleboy/ssh-action script crashes the action before"
|
|
568
|
+
echo "the script reaches the remote shell. Both function calls ({{index .X Y}}) AND field"
|
|
569
|
+
echo "accessors ({{.Names}}, {{.Status}}) are CONFIRMED BLOCKING — both fail on empty context."
|
|
570
|
+
echo ""
|
|
571
|
+
echo "Safe replacements:"
|
|
572
|
+
echo " docker inspect IMAGE | jq -r '.[0].RepoTags[0]'"
|
|
573
|
+
echo " docker ps --format json | jq -r '.Names'"
|
|
574
|
+
echo " docker ps --format json | jq -r '.Status'"
|
|
575
|
+
fi
|
|
576
|
+
fi
|
|
577
|
+
done
|
|
578
|
+
```
|
|
579
|
+
|
|
580
|
+
Report any hit as **CONFIRMED HIGH** — the step will exit 1 before reaching the remote shell.
|
|
581
|
+
|
|
582
|
+
Go template directives in `appleboy/ssh-action` `script:` blocks are preprocessed client-side before the script reaches SSH — they fail on an empty data context, crashing the action before any shell error handler can catch it. Both function call forms (`{{index .X Y}}`) and field accessors (`{{.Names}}`) fail. A partial fix that only replaces one form while leaving the other is insufficient. <!-- Added: forge#226 -->
|
|
583
|
+
|
|
584
|
+
### 2M: Import resolution (Python files with new app.* imports)
|
|
585
|
+
|
|
586
|
+
**Triggered when**: changed Python files introduce new `from app.` or `import app.` statements outside `try:` blocks.
|
|
587
|
+
|
|
588
|
+
**Why this matters**: A builder agent with access to milestone branch context can import `app.billing.subscriptions` in a fast-lane PR that targets `staging` — where that module does not exist. Every request crashes with `ModuleNotFoundError`. The module exists on disk in the builder's worktree (checked out from the milestone branch) but NOT in `git ls-files` for the base branch. `python -m py_compile` passes because the builder's environment has the module; the runtime environment does not.
|
|
589
|
+
|
|
590
|
+
```bash
|
|
591
|
+
PR_BASE="${PR_BASE:-staging}"
|
|
592
|
+
|
|
593
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$'); do
|
|
594
|
+
# Extract new app.* import statements added in the diff
|
|
595
|
+
# Only consider lines added (starting with +), skip diff headers (+++)
|
|
596
|
+
NEW_IMPORTS=$(git diff HEAD -- "$f" 2>/dev/null | grep -E '^\+\s*(from app\.|import app\.)' | grep -v '^+++' | sed 's/^+//')
|
|
597
|
+
|
|
598
|
+
# Use while IFS= read -r to preserve full import lines (word-splitting breaks multi-word imports)
|
|
599
|
+
echo "$NEW_IMPORTS" | while IFS= read -r import_line; do
|
|
600
|
+
# Skip empty lines (produced when NEW_IMPORTS is empty)
|
|
601
|
+
[ -z "$import_line" ] && continue
|
|
602
|
+
|
|
603
|
+
# Skip imports inside try: blocks — these are intentional conditional imports
|
|
604
|
+
# Check the preceding 5 lines of the file for an unindented try: statement
|
|
605
|
+
LINE_NUM=$(grep -nF "$import_line" "$f" 2>/dev/null | tail -1 | cut -d: -f1)
|
|
606
|
+
if [ -n "$LINE_NUM" ]; then
|
|
607
|
+
START=$((LINE_NUM > 5 ? LINE_NUM - 5 : 1))
|
|
608
|
+
CONTEXT=$(sed -n "${START},$((LINE_NUM-1))p" "$f" 2>/dev/null)
|
|
609
|
+
echo "$CONTEXT" | grep -qE '^\s*try\s*:' && continue
|
|
610
|
+
fi
|
|
611
|
+
|
|
612
|
+
# Derive the module path from the import statement
|
|
613
|
+
# "from app.billing.subscriptions import X" → "app/billing/subscriptions.py"
|
|
614
|
+
MODULE=$(echo "$import_line" | grep -oP '(?<=from |import )app[\w.]+' | head -1 | tr '.' '/')
|
|
615
|
+
|
|
616
|
+
if [ -n "$MODULE" ]; then
|
|
617
|
+
# Check if the module exists in the base branch file tree
|
|
618
|
+
# Try both module.py and module/__init__.py
|
|
619
|
+
EXISTS_AS_FILE=$(git ls-files --with-tree="origin/${PR_BASE}" "${MODULE}.py" 2>/dev/null | head -1)
|
|
620
|
+
EXISTS_AS_PKG=$(git ls-files --with-tree="origin/${PR_BASE}" "${MODULE}/__init__.py" 2>/dev/null | head -1)
|
|
621
|
+
|
|
622
|
+
if [ -z "$EXISTS_AS_FILE" ] && [ -z "$EXISTS_AS_PKG" ]; then
|
|
623
|
+
echo "IMPORT-1 | HIGH | $f | Cross-lane import: '${import_line}' — module '${MODULE}' does not exist on origin/${PR_BASE}. This will crash at runtime with ModuleNotFoundError. Verify the module is available on the base branch, use a try/except ImportError fallback, or remove the import."
|
|
624
|
+
fi
|
|
625
|
+
fi
|
|
626
|
+
done
|
|
627
|
+
done
|
|
628
|
+
```
|
|
629
|
+
|
|
630
|
+
**Flag as findings**:
|
|
631
|
+
- Import of `app.*` module not present in base branch tree: **HIGH** — guaranteed `ModuleNotFoundError` at runtime
|
|
632
|
+
|
|
633
|
+
**Regression guard**:
|
|
634
|
+
- Does NOT flag `try: from app.X import Y` patterns — these are intentional conditional imports
|
|
635
|
+
- Does NOT flag non-`app.*` imports (third-party libraries, `shared/`, stdlib)
|
|
636
|
+
- Does NOT block imports from modules that genuinely exist on the base branch
|
|
637
|
+
|
|
638
|
+
A builder agent working in a milestone-branch worktree has access to milestone-only modules. A fast-lane PR importing `app.*` modules that only exist on the milestone branch will pass `py_compile` (the module is present in the worktree) but crash at runtime on the base branch with `ModuleNotFoundError` on every request. <!-- Added: forge#277 -->
|
|
639
|
+
|
|
640
|
+
### 2N: Runtime UID × filesystem write check (Dockerfile or entrypoint with USER/su-exec/gosu changes)
|
|
641
|
+
|
|
642
|
+
**Triggered when**: INFRA domain is set (Dockerfile or entrypoint script introduces `USER`, `su-exec`, `gosu`, or `setuid` change).
|
|
643
|
+
|
|
644
|
+
**Why this matters**: When a container's runtime user changes from root to a non-root UID, Docker named volumes retain their existing root ownership. Any service code path that calls `mkdir`, `write_bytes`, `open(... 'w')`, or `os.makedirs` under a volume mount point will fail with `[Errno 13] Permission denied`. This failure is silent at startup — it only surfaces when the write code path executes, potentially hours or days after deploy.
|
|
645
|
+
|
|
646
|
+
```bash
|
|
647
|
+
# Step 1: Identify the service directory from the changed Dockerfile/entrypoint path
|
|
648
|
+
# e.g. services/worker/Dockerfile → service root = services/worker/
|
|
649
|
+
SERVICE_DIR=""
|
|
650
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -iE 'Dockerfile|entrypoint.*\.sh'); do
|
|
651
|
+
SERVICE_DIR=$(dirname "$f")
|
|
652
|
+
break
|
|
653
|
+
done
|
|
654
|
+
|
|
655
|
+
if [ -n "$SERVICE_DIR" ]; then
|
|
656
|
+
# Step 2: Find volume mount points for this service in any docker-compose*.yml
|
|
657
|
+
COMPOSE_FILES=$(find . -maxdepth 3 -name "docker-compose*.yml" 2>/dev/null | sort)
|
|
658
|
+
VOLUME_MOUNTS=""
|
|
659
|
+
for cf in $COMPOSE_FILES; do
|
|
660
|
+
# Extract named volume mount points (lines like: - volume_name:/container/path)
|
|
661
|
+
MOUNTS=$(grep -oE '[a-z_-]+:/[^: ]+' "$cf" 2>/dev/null | grep -v '^#' | cut -d: -f2)
|
|
662
|
+
VOLUME_MOUNTS="$VOLUME_MOUNTS $MOUNTS"
|
|
663
|
+
done
|
|
664
|
+
|
|
665
|
+
# Step 3: Grep the service directory for filesystem write operations
|
|
666
|
+
WRITE_OPS=$(grep -rn "mkdir\|write_bytes\|open(.*['\"]w\|makedirs\|shutil\.copy\|shutil\.move" \
|
|
667
|
+
"$SERVICE_DIR" --include="*.py" 2>/dev/null | grep -v __pycache__ | head -20)
|
|
668
|
+
|
|
669
|
+
if [ -n "$WRITE_OPS" ]; then
|
|
670
|
+
echo "INFRA: filesystem write operations found in $SERVICE_DIR:"
|
|
671
|
+
echo "$WRITE_OPS"
|
|
672
|
+
echo ""
|
|
673
|
+
if [ -n "$VOLUME_MOUNTS" ]; then
|
|
674
|
+
echo "Named volume mount points in docker-compose*.yml: $VOLUME_MOUNTS"
|
|
675
|
+
echo "INFRA-1: Verify entrypoint runs 'chown -R <user>:<group> <mount>' BEFORE privilege drop."
|
|
676
|
+
fi
|
|
677
|
+
fi
|
|
678
|
+
fi
|
|
679
|
+
```
|
|
680
|
+
|
|
681
|
+
**Flag as findings**:
|
|
682
|
+
- Filesystem write operations found in service directory AND named volume mounts exist AND no `chown` before privilege drop in entrypoint: **HIGH** — `PermissionError` on first write after UID drop. Named volumes are root-owned by default; a privilege drop without `chown` silently breaks all write paths under the mount point. <!-- Added: forge#323 -->
|
|
683
|
+
|
|
684
|
+
### 2O: Residual Pattern Check (router Python files with gate condition narrowing)
|
|
685
|
+
|
|
686
|
+
**Triggered when**: ROUTER_BUG domain is set (a router `*.py` file has a diff that removes or narrows a gate condition).
|
|
687
|
+
|
|
688
|
+
**Why this matters**: A bug fix that narrows a gate condition in one router file may leave identical unfixed conditions in sibling router files in the same directory. The fix is incomplete if the original (unfixed) pattern still exists elsewhere — even though the changed file is now correct. Grep-detectable at gate time.
|
|
689
|
+
|
|
690
|
+
```bash
|
|
691
|
+
PR_BASE="${PR_BASE:-staging}"
|
|
692
|
+
|
|
693
|
+
for f in $(echo {CHANGED_FILES} | tr ' ' '\n' | grep -E '\.py$' | grep -E 'router'); do
|
|
694
|
+
# Extract removed condition lines from the diff (lines starting with -)
|
|
695
|
+
REMOVED_CONDITIONS=$(git diff HEAD -- "$f" 2>/dev/null | grep -E '^-.*\bif\b' | grep -v '^---' | sed 's/^-//' | sed 's/^\s*//' | head -5)
|
|
696
|
+
|
|
697
|
+
if [ -n "$REMOVED_CONDITIONS" ]; then
|
|
698
|
+
ROUTER_DIR=$(dirname "$f")
|
|
699
|
+
echo "Checking for residual unfixed patterns in $ROUTER_DIR siblings of $f..."
|
|
700
|
+
|
|
701
|
+
# For each removed condition line, search sibling files for the same pattern
|
|
702
|
+
echo "$REMOVED_CONDITIONS" | while IFS= read -r condition; do
|
|
703
|
+
# Extract the key pattern from the condition (e.g., field name or function call)
|
|
704
|
+
PATTERN=$(echo "$condition" | grep -oP '(?<=if\s)[\w.]+|[\w.]+\(|[\w.]+\s+or' | head -1 | tr -d ' ')
|
|
705
|
+
if [ -n "$PATTERN" ]; then
|
|
706
|
+
MATCHES=$(grep -rn "$PATTERN" "$ROUTER_DIR" --include="*.py" | grep -v "^${f}:" | grep -v "^\s*#" | head -10)
|
|
707
|
+
if [ -n "$MATCHES" ]; then
|
|
708
|
+
echo "ROUTER-1 | HIGH | $f | Residual unfixed pattern '$PATTERN' found in sibling file(s) — fix may be incomplete:"
|
|
709
|
+
echo "$MATCHES" | while IFS= read -r match; do
|
|
710
|
+
echo " $match"
|
|
711
|
+
done
|
|
712
|
+
fi
|
|
713
|
+
fi
|
|
714
|
+
done
|
|
715
|
+
fi
|
|
716
|
+
done
|
|
717
|
+
```
|
|
718
|
+
|
|
719
|
+
**Flag as findings**:
|
|
720
|
+
- Original condition pattern found in sibling router files after the fix was applied: **HIGH** — the fix is incomplete; the same bug class exists in unlisted sibling files and will continue to produce the original error for users of those endpoints. <!-- Added: forge#383 -->
|
|
721
|
+
|
|
722
|
+
**False-positive prevention**:
|
|
723
|
+
- Only runs on files in a `routers/` path
|
|
724
|
+
- Only triggers when a condition line was actually removed in the diff (not just modified)
|
|
725
|
+
- Reports file:line matches from sibling files only (excludes the fixed file itself)
|
|
726
|
+
|
|
727
|
+
## Step 3: Compile findings
|
|
728
|
+
|
|
729
|
+
Format findings as a structured list:
|
|
730
|
+
|
|
731
|
+
```
|
|
732
|
+
QUALITY GATE FINDINGS:
|
|
733
|
+
[PASS] No findings — code is clean.
|
|
734
|
+
|
|
735
|
+
OR:
|
|
736
|
+
|
|
737
|
+
[FINDINGS] N issues detected:
|
|
738
|
+
|
|
739
|
+
1. SEC-1 | HIGH | file.py:45 | f-string SQL interpolation — use parameterized query
|
|
740
|
+
2. DEPLOY-1 | MEDIUM | NEW_VAR not in decrypt-secrets.sh ENV_MAPPING
|
|
741
|
+
3. DB-1 | HIGH | migration.sql:12 | NOT NULL without DEFAULT — will lock table
|
|
742
|
+
4. FE-1 | MEDIUM | component.tsx:89 | useEffect without cleanup — will leak on unmount
|
|
743
|
+
5. PROXY-1 | HIGH | page.tsx:23 | direct /api/v1/ call — must use proxy route
|
|
744
|
+
6. SHELL-1 | HIGH | deploy.sh | Hardcoded 'localhost' in DB connection function — use ${POSTGRES_HOST:-localhost} pattern
|
|
745
|
+
7. INFRA-1 | HIGH | Dockerfile | Filesystem write ops found under named volume mount — add chown before privilege drop in entrypoint
|
|
746
|
+
8. ROUTER-1 | HIGH | routers/handler.py | Residual unfixed gate condition pattern found in sibling router file routers/other_handler.py:N — fix may be incomplete across all callers
|
|
747
|
+
```
|
|
748
|
+
|
|
749
|
+
**Severity classification:**
|
|
750
|
+
- **HIGH**: Will cause a runtime error, security vulnerability, or build failure
|
|
751
|
+
- **MEDIUM**: Will cause incorrect behavior, performance degradation, or deploy issue
|
|
752
|
+
- **LOW**: Style issue, missing optimization, or defensive improvement
|
|
753
|
+
|
|
754
|
+
**Return only HIGH and MEDIUM findings.** LOW findings are noise at this stage — the review can catch those.
|
|
755
|
+
|
|
756
|
+
---
|
|
757
|
+
|
|
758
|
+
## Step 4: Return to builder
|
|
759
|
+
|
|
760
|
+
Return the findings list directly. Do NOT post to GitHub. Do NOT create issues. The builder will fix the findings and re-run the gate if needed.
|
|
761
|
+
|
|
762
|
+
**CRITICAL — continuation directive**: Your output is consumed by a calling pipeline agent. You MUST append the continuation block below to your output in ALL cases (both PASS and FINDINGS). Without it, the calling agent treats your output as a terminal signal and stops the pipeline.
|
|
763
|
+
|
|
764
|
+
If 0 findings:
|
|
765
|
+
```
|
|
766
|
+
QUALITY GATE: PASS — no issues detected.
|
|
767
|
+
|
|
768
|
+
<!-- FORGE:PHASE_COMPLETE — Quality gate is an intermediate check. See work-on.md Universal Phase Dispatcher for next phase. Not terminal — calling agent must continue. -->
|
|
769
|
+
```
|
|
770
|
+
|
|
771
|
+
If findings exist:
|
|
772
|
+
```
|
|
773
|
+
QUALITY GATE FINDINGS: N issues detected.
|
|
774
|
+
{findings list}
|
|
775
|
+
|
|
776
|
+
<!-- FORGE:PHASE_COMPLETE — Quality gate is an intermediate check. See work-on.md Universal Phase Dispatcher for next phase. Not terminal — calling agent must continue. -->
|
|
777
|
+
```
|