proofctl 0.3.1__tar.gz → 0.4.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- proofctl-0.4.0/PKG-INFO +527 -0
- proofctl-0.4.0/README.md +496 -0
- proofctl-0.4.0/proofctl/checkers/quality.py +1448 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/secrets.py +256 -0
- proofctl-0.4.0/proofctl/checkers/security.py +2932 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/terraform.py +12 -1
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/yaml_checker.py +57 -1
- proofctl-0.4.0/proofctl.egg-info/PKG-INFO +527 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl.egg-info/SOURCES.txt +1 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/pyproject.toml +1 -1
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_phase3_audit_fixes.py +28 -0
- proofctl-0.4.0/tests/test_quality.py +595 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_secrets.py +150 -0
- proofctl-0.4.0/tests/test_security_external.py +531 -0
- proofctl-0.3.1/PKG-INFO +0 -496
- proofctl-0.3.1/README.md +0 -465
- proofctl-0.3.1/proofctl/checkers/quality.py +0 -701
- proofctl-0.3.1/proofctl/checkers/security.py +0 -1534
- proofctl-0.3.1/proofctl.egg-info/PKG-INFO +0 -496
- proofctl-0.3.1/tests/test_quality.py +0 -135
- {proofctl-0.3.1 → proofctl-0.4.0}/LICENSE +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/__init__.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/_globmatch.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/baseline.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/__init__.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/base.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/dockerfile.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/hcl_utils.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/imports.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/leakage.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/llm_integration.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/placeholders.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/shell_checker.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/terragrunt.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/checkers/variants.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/cli.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/config.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/engine.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/fixer.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/models.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/registry.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/reporters/__init__.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/reporters/html_reporter.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/reporters/json_reporter.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl/reporters/terminal.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl.egg-info/dependency_links.txt +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl.egg-info/entry_points.txt +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl.egg-info/requires.txt +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/proofctl.egg-info/top_level.txt +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/setup.cfg +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_baseline.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_config_proofctl.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_dockerfile.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_engine.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_fixer.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_imports.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_leakage.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_llm.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_local_packages.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_phase1_audit_fixes.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_phase2_audit_fixes.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_phase4_audit_fixes.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_phase5_llm_imports_fixes.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_phase5_p_l_v_fixes.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_phase5_quality_fixes.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_phase6_audit_fixes.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_phase8_audit_fixes.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_placeholders.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_quality_ext.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_security.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_security_ext.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_suppression.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_terraform.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_terraform_ext.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_terragrunt.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_variants.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_yaml.py +0 -0
- {proofctl-0.3.1 → proofctl-0.4.0}/tests/test_yaml_k8s_gha.py +0 -0
proofctl-0.4.0/PKG-INFO
ADDED
|
@@ -0,0 +1,527 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: proofctl
|
|
3
|
+
Version: 0.4.0
|
|
4
|
+
Summary: Zero-dependency linter for Python, Terraform, Dockerfiles, Kubernetes, and GitHub Actions — catches AI slop and security misconfigurations pre-commit
|
|
5
|
+
Author-email: Kolawolu Odunola <kolawolu.o@gmail.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/kolawoluu/proofctl
|
|
8
|
+
Project-URL: Repository, https://github.com/kolawoluu/proofctl
|
|
9
|
+
Project-URL: Bug Tracker, https://github.com/kolawoluu/proofctl/issues
|
|
10
|
+
Keywords: linter,security,terraform,kubernetes,ai,static-analysis,pre-commit
|
|
11
|
+
Classifier: Development Status :: 4 - Beta
|
|
12
|
+
Classifier: Environment :: Console
|
|
13
|
+
Classifier: Intended Audience :: Developers
|
|
14
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
15
|
+
Classifier: Operating System :: OS Independent
|
|
16
|
+
Classifier: Programming Language :: Python :: 3
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
20
|
+
Classifier: Topic :: Security
|
|
21
|
+
Requires-Python: >=3.11
|
|
22
|
+
Description-Content-Type: text/markdown
|
|
23
|
+
License-File: LICENSE
|
|
24
|
+
Requires-Dist: typer[all]>=0.12
|
|
25
|
+
Requires-Dist: rich>=13
|
|
26
|
+
Requires-Dist: pyyaml>=6
|
|
27
|
+
Provides-Extra: dev
|
|
28
|
+
Requires-Dist: pytest>=8; extra == "dev"
|
|
29
|
+
Requires-Dist: pytest-mock>=3; extra == "dev"
|
|
30
|
+
Dynamic: license-file
|
|
31
|
+
|
|
32
|
+
# proofctl
|
|
33
|
+
|
|
34
|
+
A linter built for the DevOps stack — Python, Terraform, Kubernetes, Helm, Dockerfiles, GitHub Actions, Shell — that catches the specific kinds of mistakes AI coding tools introduce. **264 rules**, zero dependencies, runs in under a second.
|
|
35
|
+
|
|
36
|
+
```
|
|
37
|
+
proofctl check .
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
PROOFCTL-S-001 module/auth.py:42 SQL injection via string format in .execute() ERROR
|
|
42
|
+
PROOFCTL-Q-001 module/models.py:17 Mutable default argument — list assigned at def time ERROR
|
|
43
|
+
PROOFCTL-S-AI-006 module/loaders.py:88 Empty `except Exception:` swallows all errors silently WARNING
|
|
44
|
+
PROOFCTL-TF-A024 infra/sg.tf:14 SG rule 0.0.0.0/0 ingress on port 22 ERROR
|
|
45
|
+
PROOFCTL-YAML-022 k8s/deployment.yaml:8 Workload uses default namespace WARNING
|
|
46
|
+
PROOFCTL-YAML-007 .github/workflows/release.yml:23 Action not pinned to SHA ERROR
|
|
47
|
+
PROOFCTL-SECRET-001 infra/lambda.tf:8 AWS access key (AKIA...) ERROR
|
|
48
|
+
──────────────────────────────────────────────────────────────────
|
|
49
|
+
7 findings (4 ERROR, 3 WARNING) — exit 2
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## Why proofctl
|
|
55
|
+
|
|
56
|
+
LLM coding assistants ship code that compiles and looks correct but fails in predictable ways:
|
|
57
|
+
|
|
58
|
+
- **Insecure** — SQL/command injection, hardcoded credentials, weak crypto, JWT signature bypass, public S3 buckets, K8s default namespaces.
|
|
59
|
+
- **Broken** — hallucinated imports, phantom methods, type-hint vs implementation mismatch, `pass`-bodied functions shipped as real implementations.
|
|
60
|
+
- **Fragile** — broad `except Exception:`, no timeouts, mutable defaults, defensive guards for impossible states, unused configuration kwargs.
|
|
61
|
+
- **Wasteful** — single-method classes that should be functions, near-duplicate files, sprawling kwargs surface.
|
|
62
|
+
|
|
63
|
+
proofctl runs as a pre-commit linter and catches these patterns before they merge. The rule set is **measured**, not just intuited — see [proofctl-eval/BENCHMARK.md](https://github.com/kolawoluu/proofctl-eval/blob/main/BENCHMARK.md) for per-rule signal-lift numbers across:
|
|
64
|
+
|
|
65
|
+
- 17 paired AI-vs-human prompts (Layer D)
|
|
66
|
+
- 788 real AI-attributed commits across 107 individual hobbyist Python repos + 153 individual infrastructure repos (Layer E)
|
|
67
|
+
- 42 hand-labeled slop scenarios (Layer F)
|
|
68
|
+
|
|
69
|
+
The top-firing rules on real AI-authored production code are **Q-002 broad except**, **Q-005 complexity**, **Q-006 long functions**, **YAML-007/GHA-006 GitHub Actions hygiene**, and **YAML-GHA-003 missing permissions** — confirmed across both synth and real-world corpora.
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## What proofctl scans
|
|
74
|
+
|
|
75
|
+
| Language | What it catches |
|
|
76
|
+
|---|---|
|
|
77
|
+
| Python | Security (SQLi, injection, eval, JWT bypass, weak crypto), quality (mutable defaults, broad except, complexity, long functions), AI-slop patterns (echo docstrings, single-method classes, stale kwargs), import hallucinations |
|
|
78
|
+
| Terraform / HCL | AWS / GCP / Azure resource hardening (200+ rules across IAM, encryption, public exposure, audit logs), AI-slop patterns (deprecated resources, AWS region in GCP, placeholder values), required_version, sensitive outputs |
|
|
79
|
+
| Kubernetes YAML | Pod Security Standards (runAsNonRoot, allowPrivilegeEscalation, readOnlyRootFilesystem, drop ALL caps, seccomp), data-exfiltration vectors (default namespace, hostPath, low UID, RBAC wildcards, automountServiceAccountToken), image hygiene (latest tag, digest pinning) |
|
|
80
|
+
| Helm / Mustache | Same K8s checks via Helm-templated YAML |
|
|
81
|
+
| Dockerfile | USER, base image pinning, secret leakage, root-running, OCI labels, HEALTHCHECK, AI-generated antipatterns |
|
|
82
|
+
| GitHub Actions | SHA-pinning, expression injection, missing permissions, missing timeouts, secret env-var leakage |
|
|
83
|
+
| Shell | Common mistake detection (set -euo, dangerous globbing) |
|
|
84
|
+
| Secrets (any text file) | AWS / GCP / GitHub / Slack / Stripe / private keys / JWTs / hardcoded passwords / K8s Secret YAML / HCL heredocs |
|
|
85
|
+
|
|
86
|
+
---
|
|
87
|
+
|
|
88
|
+
## Installation
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
pip install proofctl
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
**Requires:** Python 3.11+
|
|
95
|
+
|
|
96
|
+
No project dependencies are added — proofctl only scans, never imports your code.
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
## Quick start
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
proofctl check . # scan current directory
|
|
104
|
+
proofctl check src/api/auth.py # scan a single file
|
|
105
|
+
proofctl check . --min-severity ERROR # only show ERROR
|
|
106
|
+
proofctl check . --fail-on ERROR # CI gate: exit non-zero only on ERROR
|
|
107
|
+
proofctl check . --changed-only # only files changed in this branch
|
|
108
|
+
proofctl check . --families S,Q,SECRET # restrict rule families
|
|
109
|
+
proofctl check . --format html --output report.html
|
|
110
|
+
proofctl rules # list all rules
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## Pre-commit integration
|
|
116
|
+
|
|
117
|
+
```yaml
|
|
118
|
+
# .pre-commit-config.yaml
|
|
119
|
+
repos:
|
|
120
|
+
- repo: https://github.com/kolawoluu/proofctl
|
|
121
|
+
rev: v0.3.1
|
|
122
|
+
hooks:
|
|
123
|
+
- id: proofctl
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Or local:
|
|
127
|
+
|
|
128
|
+
```yaml
|
|
129
|
+
repos:
|
|
130
|
+
- repo: local
|
|
131
|
+
hooks:
|
|
132
|
+
- id: proofctl
|
|
133
|
+
name: proofctl
|
|
134
|
+
language: system
|
|
135
|
+
entry: proofctl check
|
|
136
|
+
args: [--no-pypi, --fail-on, ERROR]
|
|
137
|
+
pass_filenames: false
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## CI integration (GitHub Actions)
|
|
143
|
+
|
|
144
|
+
```yaml
|
|
145
|
+
name: proofctl
|
|
146
|
+
on: [push, pull_request]
|
|
147
|
+
|
|
148
|
+
jobs:
|
|
149
|
+
proofctl:
|
|
150
|
+
runs-on: ubuntu-latest
|
|
151
|
+
steps:
|
|
152
|
+
- uses: actions/checkout@v4
|
|
153
|
+
- uses: actions/setup-python@v5
|
|
154
|
+
with:
|
|
155
|
+
python-version: "3.12"
|
|
156
|
+
- run: pip install proofctl
|
|
157
|
+
|
|
158
|
+
# --new-only compares against the committed baseline snapshot —
|
|
159
|
+
# CI fails only on regressions introduced in this PR.
|
|
160
|
+
- run: proofctl check . --no-pypi --fail-on ERROR --new-only
|
|
161
|
+
|
|
162
|
+
- name: Generate HTML report
|
|
163
|
+
if: failure()
|
|
164
|
+
run: proofctl check . --no-pypi --format html --output proofctl-report.html || true
|
|
165
|
+
|
|
166
|
+
- uses: actions/upload-artifact@v4
|
|
167
|
+
if: failure()
|
|
168
|
+
with:
|
|
169
|
+
name: proofctl-report
|
|
170
|
+
path: proofctl-report.html
|
|
171
|
+
retention-days: 14
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Create a baseline snapshot for an existing project so CI only flags regressions:
|
|
175
|
+
|
|
176
|
+
```bash
|
|
177
|
+
proofctl baseline .
|
|
178
|
+
git add .proofctl-baseline.json
|
|
179
|
+
git commit -m "chore: add proofctl baseline"
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## Rules — 255 total across 14 families
|
|
185
|
+
|
|
186
|
+
### Python — Quality (17 rules)
|
|
187
|
+
|
|
188
|
+
| Rule | Severity | Catches |
|
|
189
|
+
|------|----------|---------|
|
|
190
|
+
| Q-001 | ERROR | Mutable default argument |
|
|
191
|
+
| Q-002 | ERROR/WARNING | Bare or broad `except` with no re-raise (top AI-slop wedge — measured) |
|
|
192
|
+
| Q-003 | WARNING | Excessive `Any` or unexplained `# type: ignore` |
|
|
193
|
+
| Q-004 | INFO | Single-implementation abstraction |
|
|
194
|
+
| Q-005 | WARNING/ERROR | Cyclomatic complexity (>10 / >20) |
|
|
195
|
+
| Q-006 | WARNING/ERROR | Function body too long (>50 / >100 lines) |
|
|
196
|
+
| Q-007 | WARNING/ERROR | Too many parameters (>5 / >7) |
|
|
197
|
+
| Q-008 | WARNING | Boolean literal as positional argument |
|
|
198
|
+
| Q-009 | INFO | `print()` in non-test code |
|
|
199
|
+
| Q-AI-001 | WARNING | Confabulated enum error message ("must be one of [a,b,c]" without membership check) |
|
|
200
|
+
| Q-AI-002 | ERROR | Return-type annotation contradicts return value (`-> int: return f"x: {x}"`) |
|
|
201
|
+
| Q-AI-003 | WARNING | Stale parameter not forwarded to inner call |
|
|
202
|
+
| Q-AI-004 | INFO | Single-method class (over-engineered abstraction) |
|
|
203
|
+
| Q-AI-005 | WARNING | Defensive guard for impossible state |
|
|
204
|
+
| Q-AI-006 | WARNING/ERROR | `except Exception: pass` — generic exception swallow |
|
|
205
|
+
| Q-AI-007 | INFO | Sprawling unused kwargs (4+ defaulted, ≥2 unreferenced) |
|
|
206
|
+
| Q-AI-008 | INFO | Verbose echo docstring on trivial accessor |
|
|
207
|
+
|
|
208
|
+
### Python — Security (24 rules)
|
|
209
|
+
|
|
210
|
+
| Rule | Severity | Catches | Authority |
|
|
211
|
+
|------|----------|---------|-----------|
|
|
212
|
+
| S-001 | ERROR | SQL injection via string formatting | OWASP A03, CWE-89 |
|
|
213
|
+
| S-002 | ERROR | Command injection (`shell=True`) | OWASP A03, CWE-78 |
|
|
214
|
+
| S-003 | ERROR | Unsafe deserialization | OWASP A08, CWE-502 |
|
|
215
|
+
| S-004 | ERROR | Weak crypto (md5/sha1 — `hashlib.md5`, `from hashlib import md5`, `Crypto.Hash.MD5.new()`) | OWASP A02, CWE-327 |
|
|
216
|
+
| S-005 | ERROR | `eval()`/`exec()` on non-literal input | CWE-95 |
|
|
217
|
+
| S-006 | WARNING | Missing `timeout=` on HTTP calls | CWE-400 |
|
|
218
|
+
| S-007 | ERROR | JWT signature bypass | OWASP A07 |
|
|
219
|
+
| S-008 | ERROR | Path traversal — extends to `os.remove`, `shutil.*`, `tarfile`, `zipfile`, archive `extractall` | CWE-22 |
|
|
220
|
+
| S-009 | ERROR | Insecure UUID for security tokens | CWE-338 |
|
|
221
|
+
| S-010 | WARNING | Info exposure — `traceback.format_exc()` / `str(e)` in responses + secrets in logs | OWASP A09, CWE-200 |
|
|
222
|
+
| S-011 | ERROR/WARNING | Insecure cipher mode (ECB, ARC4, TripleDES) + RSA/DSA keysize <2048 + static IV | OWASP A02 |
|
|
223
|
+
| S-012 | WARNING | Regex injection / ReDoS | CWE-625 |
|
|
224
|
+
| S-013 | WARNING | Open redirect — Flask/Django/FastAPI `redirect()` + `Location` header from user input | CWE-601 |
|
|
225
|
+
| S-014 | ERROR | TLS verification disabled (`verify=False`) | OWASP A02, CWE-295 |
|
|
226
|
+
| S-015 | WARNING | `random.*` used for security-sensitive value | CWE-338 |
|
|
227
|
+
| S-016 | WARNING/INFO | Subprocess with partial executable path | CWE-426 |
|
|
228
|
+
| **S-017** | ERROR | **XSS** — request taint into HTML sinks (Flask f-string return, HTMLResponse, `mark_safe`/`Markup`) | OWASP A03, CWE-79 |
|
|
229
|
+
| **S-018** | ERROR | **SSRF** — request taint into HTTP-client URLs (requests/httpx/urllib) | OWASP A10, CWE-918 |
|
|
230
|
+
| **S-019** | WARNING | **Unrestricted file upload** — Flask `request.files.save()` without `secure_filename` + extension allowlist | OWASP A04, CWE-434 |
|
|
231
|
+
| **S-020** | ERROR | **Insufficiently protected creds** — password in URL / HTTP basic auth / plaintext into DB | OWASP A02, CWE-522 |
|
|
232
|
+
| **S-021** | WARNING | **Incorrect file permissions** — `chmod(0o777|0o666)`, `umask(0)` | CWE-732 |
|
|
233
|
+
| **S-022** | ERROR | **XPath injection** — f-string / `%` / `+` into `xpath()` / `lxml.etree.XPath` | OWASP A03, CWE-643 |
|
|
234
|
+
| **S-023** | WARNING | **CSRF** — `@csrf_exempt` on POST handler | OWASP A01, CWE-352 |
|
|
235
|
+
| **S-024** | WARNING | **Missing authorization** — admin/internal/delete paths without auth decorator | OWASP A01, CWE-862 |
|
|
236
|
+
|
|
237
|
+
### Secrets (13 rules) — scans every text file
|
|
238
|
+
|
|
239
|
+
| Rule | Severity | Catches |
|
|
240
|
+
|------|----------|---------|
|
|
241
|
+
| SECRET-001 | ERROR | AWS access key (`AKIA...`) |
|
|
242
|
+
| SECRET-002 | ERROR | AWS secret key |
|
|
243
|
+
| SECRET-003 | ERROR | GitHub token (`ghp_*`, `gho_*`, `ghs_*`) |
|
|
244
|
+
| SECRET-004 | ERROR | Slack token (`xoxb/p/o/a-...`) |
|
|
245
|
+
| SECRET-005 | ERROR | Private key marker (`-----BEGIN ... PRIVATE KEY-----`) |
|
|
246
|
+
| SECRET-006 | WARNING | JWT token |
|
|
247
|
+
| SECRET-007 | ERROR | Generic high-entropy `password=`/`secret=`/`api_key=` assignment — Python AST visitor catches kwargs in calls, module-level `*_PASSWORD = "..."`, tuple-auth `auth=("u", "p")` |
|
|
248
|
+
| SECRET-008 | ERROR | Google API key (`AIza...`) |
|
|
249
|
+
| SECRET-009 | ERROR | Stripe key (`sk_live_*`, `pk_live_*`) |
|
|
250
|
+
| SECRET-010 | ERROR | Hashicorp Terraform admin password (`administrator_login_password`) |
|
|
251
|
+
| SECRET-011 | ERROR | `kind: Secret` YAML with non-empty `data:` / `stringData:` |
|
|
252
|
+
| SECRET-012 | ERROR | Secret patterns inside HCL heredocs (`user_data = <<EOF`) |
|
|
253
|
+
| **SECRET-013** | ERROR | **Connection string with embedded credentials** (`postgres://user:pass@host`, mysql, redis, mongodb, amqp, sqlserver) |
|
|
254
|
+
|
|
255
|
+
### Python — Other (12 rules)
|
|
256
|
+
|
|
257
|
+
| Family | Rules | What it covers |
|
|
258
|
+
|---|---|---|
|
|
259
|
+
| Placeholder (P) | P-001..003 | Unimplemented functions, TODOs, commented-out code |
|
|
260
|
+
| Leakage (L) | L-001..003 | JS-in-Python idioms, PII patterns, cross-language slop |
|
|
261
|
+
| Imports (I) | I-001, I-002 | Hallucinated imports, high-risk packages |
|
|
262
|
+
| Methods (M) | M-001 | Method too complex |
|
|
263
|
+
| Variants (V) | V-001, V-002 | Near-duplicate functions / files |
|
|
264
|
+
| LLM Guardrails (LLM) | LLM-001..005 | OWASP LLM Top-10 (prompt injection, max_tokens, agent loops) |
|
|
265
|
+
|
|
266
|
+
### Terraform — General + AI-slop (15 + 11 rules)
|
|
267
|
+
|
|
268
|
+
`TF-T001..T015`: required_version constraints, sensitive outputs, hardcoded secrets, wildcard IAM, mutable git refs, etc.
|
|
269
|
+
|
|
270
|
+
`TF-AI-001..011`: placeholder values, AWS region in GCP, deprecated resource types (`aws_alb` → `aws_lb`), inline lifecycle blocks removed in provider v4, etc. (AI-specific patterns no other linter looks for)
|
|
271
|
+
|
|
272
|
+
### Terraform — AWS (40 rules)
|
|
273
|
+
|
|
274
|
+
`TF-A001..A040`. Highlights:
|
|
275
|
+
|
|
276
|
+
- **A001** EC2 IMDSv2 not enforced
|
|
277
|
+
- **A002** S3 missing public-access-block
|
|
278
|
+
- **A007** EBS not encrypted
|
|
279
|
+
- **A012** VPC missing flow logs
|
|
280
|
+
- **A015** KMS rotation not enabled
|
|
281
|
+
- **A016** IAM policy with wildcard Resource/Action
|
|
282
|
+
- **A017–A023** RDS hardening (encryption, IAM auth, deletion protection, audit logs)
|
|
283
|
+
- **A024** SG rule with `0.0.0.0/0` ingress on sensitive ports
|
|
284
|
+
- **A025** EKS public endpoint without CIDR restriction
|
|
285
|
+
- **A027** Neptune cluster unencrypted
|
|
286
|
+
- **A029** DynamoDB without `server_side_encryption`
|
|
287
|
+
- **A030** ElastiCache transit/at-rest encryption disabled
|
|
288
|
+
- **A034** IAM policy attached directly to user
|
|
289
|
+
- **A035** IAM role with wildcard Principal
|
|
290
|
+
- **A036/A037** CloudTrail validation / multi-region
|
|
291
|
+
- **A038** Default SG with non-empty rules
|
|
292
|
+
- **A039** Subnet auto-assigns public IP
|
|
293
|
+
- **A040** Neptune cluster without CMK
|
|
294
|
+
|
|
295
|
+
### Terraform — GCP (50+ rules)
|
|
296
|
+
|
|
297
|
+
`TF-G001..G046`. Highlights:
|
|
298
|
+
|
|
299
|
+
- **G001..G016** GKE hardening (auto-repair, node pool config, network policy, master auth networks, binary auth)
|
|
300
|
+
- **G017–G019** GCS public access prevention, uniform access, versioning
|
|
301
|
+
- **G020** `google_service_account_key` (key in TF state)
|
|
302
|
+
- **G021/G026** Broad IAM roles / `allUsers` IAM bindings
|
|
303
|
+
- **G023..G025** KMS rotation, prevent_destroy, destroy window
|
|
304
|
+
- **G027/G028** Cloud Function / Cloud Run with public access
|
|
305
|
+
- **G029** Subnet missing VPC Flow Logs
|
|
306
|
+
- **G030** BigQuery dataset with public access
|
|
307
|
+
- **G031** Pub/Sub without CMEK
|
|
308
|
+
- **G032/G033** GCS access logging, private_ip_google_access
|
|
309
|
+
- **G034..G041** GKE deep (legacy ABAC, private nodes, metadata server, secure boot, default SA)
|
|
310
|
+
- **G042..G046** Compute firewall public ingress, Cloud SQL SSL, public IPs, IP forwarding
|
|
311
|
+
|
|
312
|
+
### Terraform — Azure (14 rules)
|
|
313
|
+
|
|
314
|
+
`TF-AZ001..AZ014`. Storage public blob, SQL public network, SSL enforcement, AKS IP/RBAC, Key Vault purge protection, role wildcards, App Service HTTPS, VM encryption-at-host, NSG SSH/RDP from internet, managed disk CMK.
|
|
315
|
+
|
|
316
|
+
### Terraform — Mechanics + Lifecycle (12 rules)
|
|
317
|
+
|
|
318
|
+
`TF-M001..M007`: SG inline-vs-rule mixing, `force_destroy`, `prevent_destroy`, mutable module refs, etc.
|
|
319
|
+
`TF-V001..V005`: lifecycle antipatterns.
|
|
320
|
+
|
|
321
|
+
### Terragrunt (6 rules)
|
|
322
|
+
|
|
323
|
+
`TG-001..TG-006`: `var.*` in HCL inputs, missing `mock_outputs`, invalid `if_exists`, unencrypted remote state.
|
|
324
|
+
|
|
325
|
+
### Dockerfile (15 rules)
|
|
326
|
+
|
|
327
|
+
`DF-001..DF-010` plus AI-specific `DF-AI-002..005`. Covers latest tags, root user, secret leakage, `--no-install-recommends`, OCI labels, `HEALTHCHECK`, legacy `ENV KEY VALUE` form.
|
|
328
|
+
|
|
329
|
+
### Kubernetes YAML (26 rules)
|
|
330
|
+
|
|
331
|
+
`YAML-001..018, 022..026`. Pod Security Standards, hostPath, default namespace, image digest pinning, runAsUser, RBAC wildcards, NET_RAW. `YAML-AI-001..006` covers removed K8s API versions and deprecated annotations.
|
|
332
|
+
|
|
333
|
+
### GitHub Actions (`YAML-GHA-*`)
|
|
334
|
+
|
|
335
|
+
SHA pinning, expression injection, missing permissions, secret env-var leakage, missing timeouts. **Top wedge family on infrastructure repos** per BENCHMARK.md — AI tools writing CI workflows consistently miss these.
|
|
336
|
+
|
|
337
|
+
### Shell (8 rules)
|
|
338
|
+
|
|
339
|
+
`SH-001..008`. Dangerous globbing, missing `set -euo pipefail`, etc.
|
|
340
|
+
|
|
341
|
+
For a complete machine-readable list:
|
|
342
|
+
|
|
343
|
+
```bash
|
|
344
|
+
proofctl rules
|
|
345
|
+
proofctl rules --family TF
|
|
346
|
+
proofctl rules --severity ERROR
|
|
347
|
+
```
|
|
348
|
+
|
|
349
|
+
---
|
|
350
|
+
|
|
351
|
+
## Suppressing findings
|
|
352
|
+
|
|
353
|
+
Inline (single line):
|
|
354
|
+
|
|
355
|
+
```python
|
|
356
|
+
result = subprocess.run(cmd, shell=True) # proofctl: ignore[PROOFCTL-S-002]
|
|
357
|
+
```
|
|
358
|
+
|
|
359
|
+
```hcl
|
|
360
|
+
resource "aws_s3_bucket" "logs" {
|
|
361
|
+
acl = "public-read" # proofctl: ignore[PROOFCTL-TF-AI-003] // or # comment style
|
|
362
|
+
}
|
|
363
|
+
```
|
|
364
|
+
|
|
365
|
+
Project-wide in `.proofctl.yaml`:
|
|
366
|
+
|
|
367
|
+
```yaml
|
|
368
|
+
disable:
|
|
369
|
+
- PROOFCTL-Q-004
|
|
370
|
+
```
|
|
371
|
+
|
|
372
|
+
---
|
|
373
|
+
|
|
374
|
+
## Configuration
|
|
375
|
+
|
|
376
|
+
Place `.proofctl.yaml` in the project root:
|
|
377
|
+
|
|
378
|
+
```yaml
|
|
379
|
+
exclude_paths:
|
|
380
|
+
- "**/.venv/**"
|
|
381
|
+
- "**/migrations/**"
|
|
382
|
+
- "**/.terraform/**"
|
|
383
|
+
|
|
384
|
+
disable:
|
|
385
|
+
- PROOFCTL-Q-004
|
|
386
|
+
- PROOFCTL-DF-AI-002
|
|
387
|
+
|
|
388
|
+
severity_overrides:
|
|
389
|
+
PROOFCTL-P-002: ERROR # promote TODOs to ERROR
|
|
390
|
+
|
|
391
|
+
rules:
|
|
392
|
+
PROOFCTL-P-002:
|
|
393
|
+
allowed_formats:
|
|
394
|
+
- 'TODO\(#\d+\)' # allow TODO(#123) format
|
|
395
|
+
exclude_paths:
|
|
396
|
+
- tests/
|
|
397
|
+
|
|
398
|
+
PROOFCTL-Q-003:
|
|
399
|
+
any_threshold: 5
|
|
400
|
+
|
|
401
|
+
PROOFCTL-V-002:
|
|
402
|
+
similarity_threshold: 0.85
|
|
403
|
+
min_file_lines: 30
|
|
404
|
+
|
|
405
|
+
PROOFCTL-I-001:
|
|
406
|
+
local_namespaces:
|
|
407
|
+
- mycompany_
|
|
408
|
+
- internal_
|
|
409
|
+
|
|
410
|
+
PROOFCTL-TF-T013:
|
|
411
|
+
required_labels:
|
|
412
|
+
- environment
|
|
413
|
+
- team
|
|
414
|
+
- cost_center
|
|
415
|
+
```
|
|
416
|
+
|
|
417
|
+
---
|
|
418
|
+
|
|
419
|
+
## Commands
|
|
420
|
+
|
|
421
|
+
### `proofctl check`
|
|
422
|
+
|
|
423
|
+
```
|
|
424
|
+
proofctl check [PATH] [OPTIONS]
|
|
425
|
+
|
|
426
|
+
Options:
|
|
427
|
+
--format / -f terminal | json | html (default: terminal)
|
|
428
|
+
--output / -o Write report to file
|
|
429
|
+
--changed-only Only scan git-changed files
|
|
430
|
+
--families P,Q,S,L,I,M,V,LLM,TF,TG,DF,YAML,SH,SECRET
|
|
431
|
+
--min-severity INFO | WARNING | ERROR
|
|
432
|
+
--fail-on Exit non-zero at this severity (default: WARNING)
|
|
433
|
+
--no-pypi Skip PyPI lookups (faster, works offline)
|
|
434
|
+
--new-only Suppress findings present in .proofctl-baseline.json
|
|
435
|
+
--fix Auto-fix fixable findings (Q-001 mutable defaults)
|
|
436
|
+
```
|
|
437
|
+
|
|
438
|
+
### `proofctl baseline`
|
|
439
|
+
|
|
440
|
+
Snapshot the current findings so future `--new-only` runs only surface regressions.
|
|
441
|
+
|
|
442
|
+
```bash
|
|
443
|
+
proofctl baseline .
|
|
444
|
+
git add .proofctl-baseline.json && git commit -m "chore: proofctl baseline"
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
### `proofctl rules`
|
|
448
|
+
|
|
449
|
+
```bash
|
|
450
|
+
proofctl rules # list all
|
|
451
|
+
proofctl rules --family S # filter by family
|
|
452
|
+
proofctl rules --severity ERROR
|
|
453
|
+
```
|
|
454
|
+
|
|
455
|
+
---
|
|
456
|
+
|
|
457
|
+
## Exit codes
|
|
458
|
+
|
|
459
|
+
| Code | Meaning |
|
|
460
|
+
|------|---------|
|
|
461
|
+
| `0` | No findings |
|
|
462
|
+
| `1` | Findings present, none at or above `--fail-on` threshold |
|
|
463
|
+
| `2` | One or more findings at or above `--fail-on` threshold |
|
|
464
|
+
|
|
465
|
+
---
|
|
466
|
+
|
|
467
|
+
## Output formats
|
|
468
|
+
|
|
469
|
+
**Terminal** (default) — rich-coloured table with file:line, rule, severity, hint.
|
|
470
|
+
|
|
471
|
+
**JSON** — machine-readable for CI:
|
|
472
|
+
|
|
473
|
+
```json
|
|
474
|
+
{
|
|
475
|
+
"schema_version": 1,
|
|
476
|
+
"summary": { "total": 4, "ERROR": 3, "WARNING": 1 },
|
|
477
|
+
"findings": [
|
|
478
|
+
{
|
|
479
|
+
"file": "module/auth.py",
|
|
480
|
+
"line": 42,
|
|
481
|
+
"col": 8,
|
|
482
|
+
"rule_id": "PROOFCTL-S-001",
|
|
483
|
+
"rule_name": "SQL injection",
|
|
484
|
+
"severity": "ERROR",
|
|
485
|
+
"message": "SQL injection via string format in .execute()",
|
|
486
|
+
"hint": "Use parameterised queries: cursor.execute(sql, params)",
|
|
487
|
+
"authority": "OWASP A03:2021, CWE-89"
|
|
488
|
+
}
|
|
489
|
+
]
|
|
490
|
+
}
|
|
491
|
+
```
|
|
492
|
+
|
|
493
|
+
**HTML** — self-contained single-file dashboard with severity summary cards and a filterable table.
|
|
494
|
+
|
|
495
|
+
---
|
|
496
|
+
|
|
497
|
+
## How proofctl rules earn their place
|
|
498
|
+
|
|
499
|
+
Rules in proofctl don't ship on intuition. The companion repo [proofctl-eval](https://github.com/kolawoluu/proofctl-eval) measures every rule's per-rule **signal lift** — fire rate on AI-generated code divided by fire rate on human-written reference code — across three corpora:
|
|
500
|
+
|
|
501
|
+
- **Layer D** Synthesised: 17 paired prompts × Claude Sonnet 4.5 vs hand-written reference. Cheap to refresh (~$0.10), useful for catching anti-signal patterns.
|
|
502
|
+
- **Layer E** Real: 788 real AI-attributed commits across 107 individual hobby Python repos + 153 individual infrastructure repos. Diff-against-parent gives clean attribution of what AI introduced. The gold-standard corpus.
|
|
503
|
+
- **Layer F** Goats: 42 hand-labeled scenarios where we know which rules *should* fire. Recall test bed.
|
|
504
|
+
|
|
505
|
+
A rule with **lift ≥ 5×** is an AI-slop wedge — it fires disproportionately on AI output. A rule with lift `< 1×` fires more on humans (anti-signal — usually indicates a bug). The benchmark is rerun on every release; rules with collapsed lift get demoted, anti-signal rules get debugged.
|
|
506
|
+
|
|
507
|
+
Top measured wedges on real production AI commits (Layer E, May 2026):
|
|
508
|
+
|
|
509
|
+
| Rule | What it catches | Real-world fires |
|
|
510
|
+
|---|---|---|
|
|
511
|
+
| Q-002 | Broad except handler | +7 / 200 commits |
|
|
512
|
+
| Q-005 | Cyclomatic complexity | +6 |
|
|
513
|
+
| Q-006 | Function too long | +5 |
|
|
514
|
+
| Q-007 | Too many parameters | +4 |
|
|
515
|
+
| YAML-007 | GHA action not SHA-pinned | +7 |
|
|
516
|
+
| YAML-GHA-006 | GHA missing timeout-minutes | +7 |
|
|
517
|
+
| YAML-GHA-003 | GHA missing/permissive permissions | +6 |
|
|
518
|
+
| Q-AI-006 | Generic exception swallow | +3 |
|
|
519
|
+
| Q-AI-004 | Single-method class | +1 |
|
|
520
|
+
|
|
521
|
+
See [BENCHMARK.md](https://github.com/kolawoluu/proofctl-eval/blob/main/BENCHMARK.md) for full numbers and methodology.
|
|
522
|
+
|
|
523
|
+
---
|
|
524
|
+
|
|
525
|
+
## License
|
|
526
|
+
|
|
527
|
+
MIT
|