@intentsolutionsio/penetration-tester 2.0.0 → 3.0.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +8 -3
- package/README.md +8 -0
- package/commands/pentest.md +5 -0
- package/package.json +8 -3
- package/skills/analyzing-tls-config/SKILL.md +221 -0
- package/skills/analyzing-tls-config/references/AUTHORIZATION.md +133 -0
- package/skills/analyzing-tls-config/references/PLAYBOOK.md +267 -0
- package/skills/analyzing-tls-config/references/THEORY.md +128 -0
- package/skills/analyzing-tls-config/scripts/analyze_tls.py +415 -0
- package/skills/auditing-cors-policy/SKILL.md +186 -0
- package/skills/auditing-cors-policy/references/PLAYBOOK.md +220 -0
- package/skills/auditing-cors-policy/references/THEORY.md +142 -0
- package/skills/auditing-cors-policy/scripts/audit_cors.py +350 -0
- package/skills/auditing-npm-dependencies/SKILL.md +254 -0
- package/skills/auditing-npm-dependencies/references/PLAYBOOK.md +175 -0
- package/skills/auditing-npm-dependencies/references/THEORY.md +122 -0
- package/skills/auditing-npm-dependencies/scripts/audit_npm.py +408 -0
- package/skills/auditing-python-dependencies/SKILL.md +251 -0
- package/skills/auditing-python-dependencies/references/PLAYBOOK.md +193 -0
- package/skills/auditing-python-dependencies/references/THEORY.md +122 -0
- package/skills/auditing-python-dependencies/scripts/audit_python.py +459 -0
- package/skills/checking-http-security-headers/SKILL.md +176 -0
- package/skills/checking-http-security-headers/references/PLAYBOOK.md +212 -0
- package/skills/checking-http-security-headers/references/THEORY.md +137 -0
- package/skills/checking-http-security-headers/scripts/check_headers.py +362 -0
- package/skills/checking-license-compliance/SKILL.md +225 -0
- package/skills/checking-license-compliance/references/PLAYBOOK.md +161 -0
- package/skills/checking-license-compliance/references/THEORY.md +152 -0
- package/skills/checking-license-compliance/scripts/check_licenses.py +461 -0
- package/skills/composing-vulnerability-report/SKILL.md +212 -0
- package/skills/composing-vulnerability-report/references/PLAYBOOK.md +180 -0
- package/skills/composing-vulnerability-report/references/THEORY.md +178 -0
- package/skills/composing-vulnerability-report/scripts/compose_report.py +396 -0
- package/skills/confirming-pentest-authorization/SKILL.md +247 -0
- package/skills/confirming-pentest-authorization/references/PLAYBOOK.md +189 -0
- package/skills/confirming-pentest-authorization/references/THEORY.md +167 -0
- package/skills/confirming-pentest-authorization/scripts/check_authorization.py +457 -0
- package/skills/defining-pentest-scope/SKILL.md +227 -0
- package/skills/defining-pentest-scope/references/PLAYBOOK.md +238 -0
- package/skills/defining-pentest-scope/references/THEORY.md +170 -0
- package/skills/defining-pentest-scope/scripts/define_scope.py +472 -0
- package/skills/detecting-command-injection-patterns/SKILL.md +144 -0
- package/skills/detecting-command-injection-patterns/references/PLAYBOOK.md +302 -0
- package/skills/detecting-command-injection-patterns/references/THEORY.md +206 -0
- package/skills/detecting-command-injection-patterns/scripts/scan_cmdi.py +290 -0
- package/skills/detecting-debug-endpoints/SKILL.md +207 -0
- package/skills/detecting-debug-endpoints/references/PLAYBOOK.md +402 -0
- package/skills/detecting-debug-endpoints/references/THEORY.md +218 -0
- package/skills/detecting-debug-endpoints/scripts/probe_debug.py +518 -0
- package/skills/detecting-directory-listing/SKILL.md +206 -0
- package/skills/detecting-directory-listing/references/PLAYBOOK.md +277 -0
- package/skills/detecting-directory-listing/references/THEORY.md +203 -0
- package/skills/detecting-directory-listing/scripts/probe_directory_listing.py +180 -0
- package/skills/detecting-eval-exec-usage/SKILL.md +128 -0
- package/skills/detecting-eval-exec-usage/references/PLAYBOOK.md +306 -0
- package/skills/detecting-eval-exec-usage/references/THEORY.md +159 -0
- package/skills/detecting-eval-exec-usage/scripts/scan_eval.py +223 -0
- package/skills/detecting-exposed-secrets-files/SKILL.md +179 -0
- package/skills/detecting-exposed-secrets-files/references/PLAYBOOK.md +274 -0
- package/skills/detecting-exposed-secrets-files/references/THEORY.md +174 -0
- package/skills/detecting-exposed-secrets-files/scripts/probe_secrets.py +207 -0
- package/skills/detecting-insecure-deserialization/SKILL.md +148 -0
- package/skills/detecting-insecure-deserialization/references/PLAYBOOK.md +333 -0
- package/skills/detecting-insecure-deserialization/references/THEORY.md +199 -0
- package/skills/detecting-insecure-deserialization/scripts/scan_deserialization.py +250 -0
- package/skills/detecting-sql-injection-patterns/SKILL.md +161 -0
- package/skills/detecting-sql-injection-patterns/references/PLAYBOOK.md +317 -0
- package/skills/detecting-sql-injection-patterns/references/THEORY.md +261 -0
- package/skills/detecting-sql-injection-patterns/scripts/scan_sqli.py +354 -0
- package/skills/detecting-ssl-cert-issues/SKILL.md +182 -0
- package/skills/detecting-ssl-cert-issues/references/PLAYBOOK.md +203 -0
- package/skills/detecting-ssl-cert-issues/references/THEORY.md +133 -0
- package/skills/detecting-ssl-cert-issues/scripts/check_cert_chain.py +481 -0
- package/skills/detecting-weak-cryptography/SKILL.md +147 -0
- package/skills/detecting-weak-cryptography/references/PLAYBOOK.md +466 -0
- package/skills/detecting-weak-cryptography/references/THEORY.md +194 -0
- package/skills/detecting-weak-cryptography/scripts/scan_weak_crypto.py +417 -0
- package/skills/fingerprinting-server-software/SKILL.md +191 -0
- package/skills/fingerprinting-server-software/references/PLAYBOOK.md +337 -0
- package/skills/fingerprinting-server-software/references/THEORY.md +183 -0
- package/skills/fingerprinting-server-software/scripts/fingerprint_server.py +347 -0
- package/skills/generating-executive-summary/SKILL.md +261 -0
- package/skills/generating-executive-summary/references/PLAYBOOK.md +201 -0
- package/skills/generating-executive-summary/references/THEORY.md +195 -0
- package/skills/generating-executive-summary/scripts/exec_summary.py +538 -0
- package/skills/mapping-findings-to-owasp-top10/SKILL.md +235 -0
- package/skills/mapping-findings-to-owasp-top10/references/PLAYBOOK.md +193 -0
- package/skills/mapping-findings-to-owasp-top10/references/THEORY.md +160 -0
- package/skills/mapping-findings-to-owasp-top10/scripts/map_owasp.py +540 -0
- package/skills/performing-penetration-testing/SKILL.md +282 -190
- package/skills/performing-penetration-testing/references/OWASP_TOP_10.md +22 -0
- package/skills/performing-penetration-testing/references/REMEDIATION_PLAYBOOK.md +46 -0
- package/skills/performing-penetration-testing/references/SECURITY_HEADERS.md +41 -0
- package/skills/performing-penetration-testing/scripts/code_security_scanner.py +144 -79
- package/skills/performing-penetration-testing/scripts/dependency_auditor.py +116 -93
- package/skills/performing-penetration-testing/scripts/security_scanner.py +574 -446
- package/skills/probing-dangerous-http-methods/SKILL.md +182 -0
- package/skills/probing-dangerous-http-methods/references/PLAYBOOK.md +234 -0
- package/skills/probing-dangerous-http-methods/references/THEORY.md +145 -0
- package/skills/probing-dangerous-http-methods/scripts/probe_methods.py +263 -0
- package/skills/recording-pentest-engagement/SKILL.md +253 -0
- package/skills/recording-pentest-engagement/references/PLAYBOOK.md +203 -0
- package/skills/recording-pentest-engagement/references/THEORY.md +195 -0
- package/skills/recording-pentest-engagement/scripts/record_engagement.py +461 -0
- package/skills/scanning-for-hardcoded-secrets/SKILL.md +215 -0
- package/skills/scanning-for-hardcoded-secrets/references/PLAYBOOK.md +325 -0
- package/skills/scanning-for-hardcoded-secrets/references/THEORY.md +175 -0
- package/skills/scanning-for-hardcoded-secrets/scripts/scan_secrets.py +395 -0
- package/skills/tracing-transitive-vulnerabilities/SKILL.md +235 -0
- package/skills/tracing-transitive-vulnerabilities/references/PLAYBOOK.md +233 -0
- package/skills/tracing-transitive-vulnerabilities/references/THEORY.md +138 -0
- package/skills/tracing-transitive-vulnerabilities/scripts/trace_vulns.py +484 -0
|
@@ -0,0 +1,461 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""recording-pentest-engagement — chain-of-custody packager.
|
|
3
|
+
|
|
4
|
+
Walks an engagement directory, builds a SHA-256 manifest of every file,
|
|
5
|
+
optionally signs the manifest with GPG, optionally creates a tarball of the
|
|
6
|
+
directory + manifest. Emits Findings via lib/finding.py for any inconsistency
|
|
7
|
+
between the manifest and the on-disk state.
|
|
8
|
+
|
|
9
|
+
Usage:
|
|
10
|
+
python3 record_engagement.py PATH [--output FILE] [--format json|jsonl|markdown]
|
|
11
|
+
[--min-severity sev] [--manifest FILE]
|
|
12
|
+
[--tar FILE] [--sign] [--signer KEY]
|
|
13
|
+
[--exclude GLOB]
|
|
14
|
+
"""
|
|
15
|
+
|
|
16
|
+
from __future__ import annotations
|
|
17
|
+
|
|
18
|
+
import argparse
|
|
19
|
+
import fnmatch
|
|
20
|
+
import hashlib
|
|
21
|
+
import json
|
|
22
|
+
import os
|
|
23
|
+
import shutil
|
|
24
|
+
import subprocess
|
|
25
|
+
import sys
|
|
26
|
+
import tarfile
|
|
27
|
+
from datetime import datetime, timezone
|
|
28
|
+
from pathlib import Path
|
|
29
|
+
from typing import Any
|
|
30
|
+
|
|
31
|
+
# --- lib/ import -------------------------------------------------------------
|
|
32
|
+
_LIB_ROOT = Path(__file__).resolve().parents[3]
|
|
33
|
+
sys.path.insert(0, str(_LIB_ROOT))
|
|
34
|
+
|
|
35
|
+
from lib.finding import Finding, Severity # noqa: E402
|
|
36
|
+
from lib import report # noqa: E402
|
|
37
|
+
|
|
38
|
+
|
|
39
|
+
SKILL_ID = "recording-pentest-engagement"
|
|
40
|
+
CATEGORY = "evidence-chain"
|
|
41
|
+
DEFAULT_EXCLUDES = (
|
|
42
|
+
# fnmatch-style patterns. fnmatch's `*` crosses directory separators on
|
|
43
|
+
# POSIX (Linux + macOS), so a bare name plus a `*/<name>` variant
|
|
44
|
+
# together cover top-level and any nested instance. The pack's CI runs
|
|
45
|
+
# on ubuntu-latest only; on Windows, fnmatch's `*` semantics differ and
|
|
46
|
+
# these patterns would miss deeply-nested matches. Reviewer note
|
|
47
|
+
# (PR #837 review): if Windows support ever lands, swap to a
|
|
48
|
+
# per-depth pattern set or pathspec-based matching.
|
|
49
|
+
"manifest.sha256",
|
|
50
|
+
"*/manifest.sha256",
|
|
51
|
+
"manifest.sha256.asc",
|
|
52
|
+
"*/manifest.sha256.asc",
|
|
53
|
+
".DS_Store",
|
|
54
|
+
"*/.DS_Store",
|
|
55
|
+
"*__pycache__*",
|
|
56
|
+
)
|
|
57
|
+
|
|
58
|
+
|
|
59
|
+
# --- Helpers ----------------------------------------------------------------
|
|
60
|
+
|
|
61
|
+
|
|
62
|
+
def _f(
|
|
63
|
+
severity: Severity,
|
|
64
|
+
title: str,
|
|
65
|
+
target: str,
|
|
66
|
+
detail: str,
|
|
67
|
+
remediation: str,
|
|
68
|
+
evidence: tuple[tuple[str, Any], ...] = (),
|
|
69
|
+
) -> Finding:
|
|
70
|
+
return Finding(
|
|
71
|
+
skill_id=SKILL_ID,
|
|
72
|
+
title=title,
|
|
73
|
+
severity=severity,
|
|
74
|
+
target=target,
|
|
75
|
+
detail=detail,
|
|
76
|
+
remediation=remediation,
|
|
77
|
+
evidence=evidence,
|
|
78
|
+
)
|
|
79
|
+
|
|
80
|
+
|
|
81
|
+
def _matches_any(path: str, patterns: list[str]) -> bool:
|
|
82
|
+
return any(fnmatch.fnmatch(path, pat) for pat in patterns)
|
|
83
|
+
|
|
84
|
+
|
|
85
|
+
def _sha256_file(path: Path) -> str:
|
|
86
|
+
h = hashlib.sha256()
|
|
87
|
+
with open(path, "rb") as fh:
|
|
88
|
+
for chunk in iter(lambda: fh.read(65536), b""):
|
|
89
|
+
h.update(chunk)
|
|
90
|
+
return h.hexdigest()
|
|
91
|
+
|
|
92
|
+
|
|
93
|
+
# --- Manifest building ------------------------------------------------------
|
|
94
|
+
|
|
95
|
+
|
|
96
|
+
def walk_files(root: Path, excludes: list[str]) -> tuple[list[Path], list[Finding]]:
|
|
97
|
+
"""Return (file_list, findings) where findings flag operational issues."""
|
|
98
|
+
files: list[Path] = []
|
|
99
|
+
findings: list[Finding] = []
|
|
100
|
+
if not root.exists():
|
|
101
|
+
findings.append(
|
|
102
|
+
_f(
|
|
103
|
+
Severity.CRITICAL,
|
|
104
|
+
f"engagement directory missing: {root}",
|
|
105
|
+
str(root),
|
|
106
|
+
f"Path {root} does not exist; cannot record engagement.",
|
|
107
|
+
"Create the engagement directory and re-run.",
|
|
108
|
+
)
|
|
109
|
+
)
|
|
110
|
+
return files, findings
|
|
111
|
+
|
|
112
|
+
for dirpath, _dirnames, filenames in os.walk(root):
|
|
113
|
+
for name in filenames:
|
|
114
|
+
p = Path(dirpath) / name
|
|
115
|
+
rel = p.relative_to(root)
|
|
116
|
+
rel_str = str(rel)
|
|
117
|
+
if _matches_any(rel_str, excludes):
|
|
118
|
+
continue
|
|
119
|
+
try:
|
|
120
|
+
if p.is_symlink():
|
|
121
|
+
findings.append(
|
|
122
|
+
_f(
|
|
123
|
+
Severity.MEDIUM,
|
|
124
|
+
f"symlink in tree: {rel_str}",
|
|
125
|
+
str(p),
|
|
126
|
+
"Symlinks break archive portability and complicate integrity verification.",
|
|
127
|
+
"Replace the symlink with the actual file, or exclude "
|
|
128
|
+
"the directory containing the symlink.",
|
|
129
|
+
)
|
|
130
|
+
)
|
|
131
|
+
continue
|
|
132
|
+
if p.stat().st_size == 0:
|
|
133
|
+
findings.append(
|
|
134
|
+
_f(
|
|
135
|
+
Severity.INFO,
|
|
136
|
+
f"empty file in tree: {rel_str}",
|
|
137
|
+
str(p),
|
|
138
|
+
"0-byte file; possibly an export error or placeholder.",
|
|
139
|
+
"Verify the file is intentional; remove if not.",
|
|
140
|
+
)
|
|
141
|
+
)
|
|
142
|
+
except (OSError, PermissionError) as e:
|
|
143
|
+
findings.append(
|
|
144
|
+
_f(
|
|
145
|
+
Severity.HIGH,
|
|
146
|
+
f"cannot stat {rel_str}",
|
|
147
|
+
str(p),
|
|
148
|
+
f"OSError: {e}",
|
|
149
|
+
"Resolve permissions; re-run.",
|
|
150
|
+
)
|
|
151
|
+
)
|
|
152
|
+
continue
|
|
153
|
+
files.append(p)
|
|
154
|
+
return files, findings
|
|
155
|
+
|
|
156
|
+
|
|
157
|
+
def compute_manifest(root: Path, files: list[Path]) -> tuple[list[tuple[str, str]], list[Finding]]:
|
|
158
|
+
"""Return (manifest_entries, findings) where entries are (hash, relpath)."""
|
|
159
|
+
entries: list[tuple[str, str]] = []
|
|
160
|
+
findings: list[Finding] = []
|
|
161
|
+
for f in sorted(files):
|
|
162
|
+
try:
|
|
163
|
+
digest = _sha256_file(f)
|
|
164
|
+
except (OSError, PermissionError) as e:
|
|
165
|
+
findings.append(
|
|
166
|
+
_f(
|
|
167
|
+
Severity.HIGH,
|
|
168
|
+
f"cannot read {f.relative_to(root)}",
|
|
169
|
+
str(f),
|
|
170
|
+
f"Cannot hash file: {e}",
|
|
171
|
+
"Resolve permissions; re-run.",
|
|
172
|
+
)
|
|
173
|
+
)
|
|
174
|
+
continue
|
|
175
|
+
entries.append((digest, str(f.relative_to(root))))
|
|
176
|
+
return entries, findings
|
|
177
|
+
|
|
178
|
+
|
|
179
|
+
def write_manifest(entries: list[tuple[str, str]], manifest_path: Path) -> None:
|
|
180
|
+
lines = [f"{digest} {rel}\n" for digest, rel in entries]
|
|
181
|
+
manifest_path.write_text("".join(lines), encoding="utf-8")
|
|
182
|
+
|
|
183
|
+
|
|
184
|
+
# --- Manifest verification (when one already exists) -----------------------
|
|
185
|
+
|
|
186
|
+
|
|
187
|
+
def load_existing_manifest(manifest_path: Path) -> list[tuple[str, str]] | None:
|
|
188
|
+
if not manifest_path.exists():
|
|
189
|
+
return None
|
|
190
|
+
out: list[tuple[str, str]] = []
|
|
191
|
+
for line in manifest_path.read_text(encoding="utf-8").splitlines():
|
|
192
|
+
line = line.rstrip("\n")
|
|
193
|
+
if not line.strip():
|
|
194
|
+
continue
|
|
195
|
+
# standard sha256sum format: HASH<sp><sp>PATH
|
|
196
|
+
if " " in line:
|
|
197
|
+
digest, rel = line.split(" ", 1)
|
|
198
|
+
elif " " in line:
|
|
199
|
+
parts = line.split()
|
|
200
|
+
digest = parts[0]
|
|
201
|
+
rel = " ".join(parts[1:])
|
|
202
|
+
else:
|
|
203
|
+
continue
|
|
204
|
+
out.append((digest.strip(), rel.strip()))
|
|
205
|
+
return out
|
|
206
|
+
|
|
207
|
+
|
|
208
|
+
def verify_against_existing(
|
|
209
|
+
root: Path, computed: list[tuple[str, str]], existing: list[tuple[str, str]]
|
|
210
|
+
) -> list[Finding]:
|
|
211
|
+
findings: list[Finding] = []
|
|
212
|
+
existing_map = dict((rel, digest) for digest, rel in existing)
|
|
213
|
+
computed_map = dict((rel, digest) for digest, rel in computed)
|
|
214
|
+
|
|
215
|
+
for rel, comp_digest in computed_map.items():
|
|
216
|
+
if rel not in existing_map:
|
|
217
|
+
findings.append(
|
|
218
|
+
_f(
|
|
219
|
+
Severity.HIGH,
|
|
220
|
+
f"file not in existing manifest: {rel}",
|
|
221
|
+
str(root / rel),
|
|
222
|
+
"File present on disk but absent from the existing manifest. "
|
|
223
|
+
"Either the file was added after manifest creation, or the "
|
|
224
|
+
"manifest is stale.",
|
|
225
|
+
"Re-generate the manifest to include the new file (acceptable "
|
|
226
|
+
"for in-progress engagements; never acceptable post-closeout).",
|
|
227
|
+
)
|
|
228
|
+
)
|
|
229
|
+
elif existing_map[rel] != comp_digest:
|
|
230
|
+
findings.append(
|
|
231
|
+
_f(
|
|
232
|
+
Severity.CRITICAL,
|
|
233
|
+
f"hash mismatch: {rel}",
|
|
234
|
+
str(root / rel),
|
|
235
|
+
f"Manifest says {existing_map[rel]}, computed {comp_digest}. "
|
|
236
|
+
f"The file has been modified since the manifest was created.",
|
|
237
|
+
"Investigate the modification. If legitimate (post-engagement "
|
|
238
|
+
"edit), the original archive's integrity claim is broken; "
|
|
239
|
+
"create a NEW archive with a fresh manifest and document the "
|
|
240
|
+
"modification reason.",
|
|
241
|
+
)
|
|
242
|
+
)
|
|
243
|
+
|
|
244
|
+
for rel in existing_map.keys() - computed_map.keys():
|
|
245
|
+
findings.append(
|
|
246
|
+
_f(
|
|
247
|
+
Severity.HIGH,
|
|
248
|
+
f"manifest entry for missing file: {rel}",
|
|
249
|
+
str(root / rel),
|
|
250
|
+
"Manifest lists a file that doesn't exist on disk.",
|
|
251
|
+
"Either restore the file or generate a fresh manifest without it.",
|
|
252
|
+
)
|
|
253
|
+
)
|
|
254
|
+
|
|
255
|
+
return findings
|
|
256
|
+
|
|
257
|
+
|
|
258
|
+
# --- Findings sanity (cross-reference) --------------------------------------
|
|
259
|
+
|
|
260
|
+
|
|
261
|
+
def scan_findings_for_external_refs(root: Path) -> list[Finding]:
|
|
262
|
+
"""If findings JSONs reference paths, verify those paths are inside root."""
|
|
263
|
+
out: list[Finding] = []
|
|
264
|
+
findings_dir = root / "findings"
|
|
265
|
+
if not findings_dir.is_dir():
|
|
266
|
+
return out
|
|
267
|
+
for f in findings_dir.glob("**/*.json"):
|
|
268
|
+
try:
|
|
269
|
+
data = json.loads(f.read_text(encoding="utf-8"))
|
|
270
|
+
except (OSError, json.JSONDecodeError):
|
|
271
|
+
continue
|
|
272
|
+
records = data if isinstance(data, list) else data.get("findings", []) if isinstance(data, dict) else []
|
|
273
|
+
for rec in records:
|
|
274
|
+
if not isinstance(rec, dict):
|
|
275
|
+
continue
|
|
276
|
+
evidence = rec.get("evidence", {})
|
|
277
|
+
if isinstance(evidence, dict):
|
|
278
|
+
for k, v in evidence.items():
|
|
279
|
+
if not isinstance(v, str):
|
|
280
|
+
continue
|
|
281
|
+
if v.startswith("/") and root.as_posix() not in v:
|
|
282
|
+
out.append(
|
|
283
|
+
_f(
|
|
284
|
+
Severity.MEDIUM,
|
|
285
|
+
f"finding references out-of-tree path: {v}",
|
|
286
|
+
str(f),
|
|
287
|
+
f"Finding evidence['{k}'] = {v} points outside the "
|
|
288
|
+
f"engagement directory tree at {root}.",
|
|
289
|
+
"Either copy the referenced file into the engagement "
|
|
290
|
+
"tree (so it's archived) or change the finding to "
|
|
291
|
+
"reference an in-tree path.",
|
|
292
|
+
)
|
|
293
|
+
)
|
|
294
|
+
return out
|
|
295
|
+
|
|
296
|
+
|
|
297
|
+
# --- GPG signing ------------------------------------------------------------
|
|
298
|
+
|
|
299
|
+
|
|
300
|
+
def sign_manifest(manifest_path: Path, signer: str | None) -> tuple[bool, str]:
|
|
301
|
+
if not shutil.which("gpg"):
|
|
302
|
+
return False, "gpg not installed"
|
|
303
|
+
cmd = ["gpg", "--armor", "--detach-sign"]
|
|
304
|
+
if signer:
|
|
305
|
+
cmd.extend(["--local-user", signer])
|
|
306
|
+
cmd.extend(["--output", str(manifest_path) + ".asc", str(manifest_path)])
|
|
307
|
+
try:
|
|
308
|
+
proc = subprocess.run(cmd, capture_output=True, text=True, timeout=60, check=False) # noqa: S603
|
|
309
|
+
except (subprocess.TimeoutExpired, FileNotFoundError) as e:
|
|
310
|
+
return False, f"gpg invocation failed: {e}"
|
|
311
|
+
if proc.returncode != 0:
|
|
312
|
+
return False, f"gpg exited {proc.returncode}: {proc.stderr.strip()[:200]}"
|
|
313
|
+
return True, ""
|
|
314
|
+
|
|
315
|
+
|
|
316
|
+
# --- Tar packaging ----------------------------------------------------------
|
|
317
|
+
|
|
318
|
+
|
|
319
|
+
def create_tar(root: Path, tar_path: Path) -> tuple[bool, str]:
|
|
320
|
+
try:
|
|
321
|
+
tar_path.parent.mkdir(parents=True, exist_ok=True)
|
|
322
|
+
with tarfile.open(tar_path, "w:gz") as tar:
|
|
323
|
+
tar.add(root, arcname=root.name)
|
|
324
|
+
except (OSError, tarfile.TarError) as e:
|
|
325
|
+
return False, f"tar creation failed: {e}"
|
|
326
|
+
return True, ""
|
|
327
|
+
|
|
328
|
+
|
|
329
|
+
# --- CLI ---------------------------------------------------------------------
|
|
330
|
+
|
|
331
|
+
|
|
332
|
+
def _build_arg_parser() -> argparse.ArgumentParser:
|
|
333
|
+
p = argparse.ArgumentParser(description=__doc__.split("\n")[0])
|
|
334
|
+
p.add_argument("path", help="Engagement directory")
|
|
335
|
+
p.add_argument("--output", default=None)
|
|
336
|
+
p.add_argument("--format", default="markdown", choices=["json", "jsonl", "markdown"])
|
|
337
|
+
p.add_argument(
|
|
338
|
+
"--min-severity",
|
|
339
|
+
default="info",
|
|
340
|
+
choices=["info", "low", "medium", "high", "critical"],
|
|
341
|
+
)
|
|
342
|
+
p.add_argument("--manifest", default=None)
|
|
343
|
+
p.add_argument("--tar", default=None)
|
|
344
|
+
p.add_argument("--sign", action="store_true")
|
|
345
|
+
p.add_argument("--signer", default=None)
|
|
346
|
+
p.add_argument("--exclude", action="append", default=[])
|
|
347
|
+
return p
|
|
348
|
+
|
|
349
|
+
|
|
350
|
+
def _filter_min_severity(findings: list[Finding], min_sev: str) -> list[Finding]:
|
|
351
|
+
floor = Severity(min_sev).numeric
|
|
352
|
+
return [f for f in findings if f.severity.numeric >= floor]
|
|
353
|
+
|
|
354
|
+
|
|
355
|
+
def main(argv: list[str] | None = None) -> int:
|
|
356
|
+
args = _build_arg_parser().parse_args(argv)
|
|
357
|
+
root = Path(args.path).resolve()
|
|
358
|
+
excludes = list(DEFAULT_EXCLUDES) + list(args.exclude)
|
|
359
|
+
|
|
360
|
+
files, walk_findings = walk_files(root, excludes)
|
|
361
|
+
if not files and walk_findings:
|
|
362
|
+
report.emit(walk_findings, args.output, args.format, scan_target=str(root))
|
|
363
|
+
return 1
|
|
364
|
+
|
|
365
|
+
entries, hash_findings = compute_manifest(root, files)
|
|
366
|
+
|
|
367
|
+
manifest_path = Path(args.manifest).resolve() if args.manifest else (root / "manifest.sha256")
|
|
368
|
+
existing = load_existing_manifest(manifest_path)
|
|
369
|
+
verify_findings: list[Finding] = []
|
|
370
|
+
if existing is not None:
|
|
371
|
+
verify_findings = verify_against_existing(root, entries, existing)
|
|
372
|
+
|
|
373
|
+
# Always rewrite the manifest to current state (it's authoritative now)
|
|
374
|
+
try:
|
|
375
|
+
write_manifest(entries, manifest_path)
|
|
376
|
+
except OSError as e:
|
|
377
|
+
walk_findings.append(
|
|
378
|
+
_f(
|
|
379
|
+
Severity.HIGH,
|
|
380
|
+
f"cannot write manifest at {manifest_path}",
|
|
381
|
+
str(manifest_path),
|
|
382
|
+
f"OSError: {e}",
|
|
383
|
+
"Resolve permissions; re-run.",
|
|
384
|
+
)
|
|
385
|
+
)
|
|
386
|
+
|
|
387
|
+
ref_findings = scan_findings_for_external_refs(root)
|
|
388
|
+
|
|
389
|
+
sign_findings: list[Finding] = []
|
|
390
|
+
if args.sign:
|
|
391
|
+
ok, msg = sign_manifest(manifest_path, args.signer)
|
|
392
|
+
if ok:
|
|
393
|
+
sign_findings.append(
|
|
394
|
+
_f(
|
|
395
|
+
Severity.INFO,
|
|
396
|
+
"manifest signed",
|
|
397
|
+
str(manifest_path) + ".asc",
|
|
398
|
+
"Manifest signed with GPG detached signature.",
|
|
399
|
+
"Distribute the .asc alongside the manifest; verify with "
|
|
400
|
+
"`gpg --verify manifest.sha256.asc manifest.sha256`.",
|
|
401
|
+
)
|
|
402
|
+
)
|
|
403
|
+
else:
|
|
404
|
+
sign_findings.append(
|
|
405
|
+
_f(
|
|
406
|
+
Severity.HIGH,
|
|
407
|
+
"manifest signing failed",
|
|
408
|
+
str(manifest_path),
|
|
409
|
+
msg,
|
|
410
|
+
"Sign the manifest manually after resolving the GPG issue, or skip --sign.",
|
|
411
|
+
)
|
|
412
|
+
)
|
|
413
|
+
|
|
414
|
+
tar_findings: list[Finding] = []
|
|
415
|
+
if args.tar:
|
|
416
|
+
ok, msg = create_tar(root, Path(args.tar).resolve())
|
|
417
|
+
if ok:
|
|
418
|
+
tar_findings.append(
|
|
419
|
+
_f(
|
|
420
|
+
Severity.INFO,
|
|
421
|
+
"archive created",
|
|
422
|
+
args.tar,
|
|
423
|
+
f"Archive written: {args.tar}",
|
|
424
|
+
"Hand off to legal / archive / customer per the engagement closeout protocol.",
|
|
425
|
+
)
|
|
426
|
+
)
|
|
427
|
+
else:
|
|
428
|
+
tar_findings.append(
|
|
429
|
+
_f(
|
|
430
|
+
Severity.HIGH,
|
|
431
|
+
"archive creation failed",
|
|
432
|
+
args.tar,
|
|
433
|
+
msg,
|
|
434
|
+
"Resolve the tar error and re-run with --tar.",
|
|
435
|
+
)
|
|
436
|
+
)
|
|
437
|
+
|
|
438
|
+
all_findings = walk_findings + hash_findings + verify_findings + ref_findings + sign_findings + tar_findings
|
|
439
|
+
if not all_findings:
|
|
440
|
+
all_findings = [
|
|
441
|
+
_f(
|
|
442
|
+
Severity.INFO,
|
|
443
|
+
"engagement archive is internally consistent",
|
|
444
|
+
str(root),
|
|
445
|
+
f"Manifest covers {len(entries)} files. No integrity issues detected.",
|
|
446
|
+
"No action required. Distribute the manifest (and optionally "
|
|
447
|
+
".tar.gz) per the engagement closeout protocol.",
|
|
448
|
+
evidence=(
|
|
449
|
+
("file_count", len(entries)),
|
|
450
|
+
("manifest", str(manifest_path)),
|
|
451
|
+
("recorded_at", datetime.now(timezone.utc).isoformat()),
|
|
452
|
+
),
|
|
453
|
+
)
|
|
454
|
+
]
|
|
455
|
+
all_findings = _filter_min_severity(all_findings, args.min_severity)
|
|
456
|
+
report.emit(all_findings, args.output, args.format, scan_target=str(root))
|
|
457
|
+
return report.exit_code(all_findings)
|
|
458
|
+
|
|
459
|
+
|
|
460
|
+
if __name__ == "__main__":
|
|
461
|
+
sys.exit(main())
|
|
@@ -0,0 +1,215 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: scanning-for-hardcoded-secrets
|
|
3
|
+
description: |
|
|
4
|
+
Scan a source-code tree for hardcoded credentials embedded in source
|
|
5
|
+
files: AWS access keys, GitHub tokens, Stripe keys, Slack tokens,
|
|
6
|
+
Anthropic API keys, OpenAI keys, JWT signing secrets, generic
|
|
7
|
+
base64-encoded passwords, RSA / SSH private keys, and high-entropy
|
|
8
|
+
string literals that pattern-match common credential shapes.
|
|
9
|
+
Use when: pre-commit gate before pushing a feature branch, audit
|
|
10
|
+
before SOC2, post-incident scan after a leak, or inheriting a
|
|
11
|
+
codebase you didn't write.
|
|
12
|
+
Threshold: any source file contains a string that matches a
|
|
13
|
+
canonical credential regex (AWS AKIA prefix, GitHub ghp_ prefix,
|
|
14
|
+
etc.) OR a string with Shannon entropy above 4.5 in a field
|
|
15
|
+
context (key=, token:, secret=).
|
|
16
|
+
Trigger with: "scan secrets", "credential scan", "find hardcoded
|
|
17
|
+
keys", "leak check".
|
|
18
|
+
allowed-tools:
|
|
19
|
+
- Read
|
|
20
|
+
- Bash(python3:*)
|
|
21
|
+
- Glob
|
|
22
|
+
- Grep
|
|
23
|
+
disallowed-tools:
|
|
24
|
+
- Bash(rm:*)
|
|
25
|
+
- Bash(curl:*)
|
|
26
|
+
- Bash(wget:*)
|
|
27
|
+
- Write(.env)
|
|
28
|
+
- Edit(.env)
|
|
29
|
+
version: 3.0.0-dev
|
|
30
|
+
author: Jeremy Longshore <jeremy@intentsolutions.io>
|
|
31
|
+
license: MIT
|
|
32
|
+
compatibility: Designed for Claude Code
|
|
33
|
+
tags:
|
|
34
|
+
- security
|
|
35
|
+
- static-analysis
|
|
36
|
+
- secrets
|
|
37
|
+
- pentest
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
# Scanning for Hardcoded Secrets
|
|
41
|
+
|
|
42
|
+
## Overview
|
|
43
|
+
|
|
44
|
+
The single most common cause of credential breach in 2026 remains
|
|
45
|
+
hardcoded secrets in source code. Engineers paste an API key into a
|
|
46
|
+
config file "just for testing," forget to remove it, commit the
|
|
47
|
+
file. The credential is now in the repository's history forever
|
|
48
|
+
(`git rebase` doesn't help if anyone cloned in between) and
|
|
49
|
+
extractable by anyone who reaches the repo: contractors,
|
|
50
|
+
ex-employees, attackers via `.git/` directory exposure (see skill
|
|
51
|
+
six), GitHub bot scrapers crawling public repos.
|
|
52
|
+
|
|
53
|
+
The cost of detection-after-commit is near-zero (free tools exist:
|
|
54
|
+
gitleaks, trufflehog, this skill). The cost of detection-before-commit
|
|
55
|
+
is also near-zero (pre-commit hooks). The cost of remediation after
|
|
56
|
+
the fact is rotating every credential exposed + auditing for
|
|
57
|
+
exploitation + potentially notifying customers of breach. The
|
|
58
|
+
asymmetry is severe, the discipline is the only constraint.
|
|
59
|
+
|
|
60
|
+
This skill scans a filesystem tree, matching against a canonical
|
|
61
|
+
regex library covering the credential shapes attackers and bots
|
|
62
|
+
search for first.
|
|
63
|
+
|
|
64
|
+
## When the skill produces findings
|
|
65
|
+
|
|
66
|
+
| Finding | Severity | Threshold | Affected control |
|
|
67
|
+
|---|---|---|---|
|
|
68
|
+
| AWS access key (AKIA-prefix) | **CRITICAL** | Literal `AKIA[0-9A-Z]{16}` in any file | CWE-798 |
|
|
69
|
+
| AWS secret access key | **CRITICAL** | 40-char base64 in `aws_secret_access_key` field context | CWE-798 |
|
|
70
|
+
| GitHub personal access token | **CRITICAL** | `ghp_[A-Za-z0-9]{36}` or `gho_`, `ghu_`, `ghs_`, `ghr_` | CWE-798 |
|
|
71
|
+
| GitHub app installation token | **CRITICAL** | `ghs_[A-Za-z0-9]{36}` | CWE-798 |
|
|
72
|
+
| Stripe live key | **CRITICAL** | `sk_live_[A-Za-z0-9]{24,}` | CWE-798 |
|
|
73
|
+
| Stripe test key | **MEDIUM** | `sk_test_[A-Za-z0-9]{24,}` | CWE-798 |
|
|
74
|
+
| Anthropic API key | **CRITICAL** | `sk-ant-api03-[A-Za-z0-9_-]{93}` or similar | CWE-798 |
|
|
75
|
+
| OpenAI API key | **CRITICAL** | `sk-(proj-)?[A-Za-z0-9_-]{40,}` | CWE-798 |
|
|
76
|
+
| Slack bot token | **CRITICAL** | `xoxb-[A-Za-z0-9-]+` | CWE-798 |
|
|
77
|
+
| Slack user token | **CRITICAL** | `xoxp-[A-Za-z0-9-]+` | CWE-798 |
|
|
78
|
+
| Google API key | **HIGH** | `AIza[A-Za-z0-9_-]{35}` | CWE-798 |
|
|
79
|
+
| RSA / OpenSSH private key | **CRITICAL** | BEGIN PRIVATE KEY header (RSA, OPENSSH, EC, DSA variants) | CWE-321 |
|
|
80
|
+
| JWT secret | **HIGH** | Long string in `jwt_secret`, `JWT_SECRET`, `signing_secret` field | CWE-321 |
|
|
81
|
+
| Generic password literal | **HIGH** | `password = "..."` with non-placeholder value | CWE-798 |
|
|
82
|
+
| High-entropy string in key/token field | **MEDIUM** | Shannon entropy ≥ 4.5 in `key:`/`token:` field context | CWE-798 |
|
|
83
|
+
| `.env`-shaped KEY=VALUE in non-`.env` file | **HIGH** | Multiple `[A-Z_]+=` lines in `.py`/`.js`/`.md` files | CWE-200 |
|
|
84
|
+
|
|
85
|
+
## Prerequisites
|
|
86
|
+
|
|
87
|
+
- Python 3.9+
|
|
88
|
+
- Target source-code tree on local filesystem
|
|
89
|
+
|
|
90
|
+
## Instructions
|
|
91
|
+
|
|
92
|
+
### Step 1 — Identify the scan target
|
|
93
|
+
|
|
94
|
+
This skill scans a filesystem path. No authorization gate (it
|
|
95
|
+
operates on local source code, not network targets).
|
|
96
|
+
|
|
97
|
+
### Step 2 — Run the scanner
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
python3 ${CLAUDE_PLUGIN_ROOT}/skills/scanning-for-hardcoded-secrets/scripts/scan_secrets.py /path/to/repo
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Options:
|
|
104
|
+
|
|
105
|
+
```
|
|
106
|
+
Usage: scan_secrets.py PATH [OPTIONS]
|
|
107
|
+
|
|
108
|
+
Options:
|
|
109
|
+
--output FILE Write findings to FILE (default: stdout)
|
|
110
|
+
--format FMT json | jsonl | markdown (default: markdown)
|
|
111
|
+
--min-severity SEV (default: info)
|
|
112
|
+
--include-tests Include files under tests/, test/, __tests__/, spec/
|
|
113
|
+
(default: excluded to reduce false positives)
|
|
114
|
+
--git-history N Also scan the last N git commits' diffs (default: 0
|
|
115
|
+
= working tree only)
|
|
116
|
+
--exclude GLOB Skip files matching glob (repeatable)
|
|
117
|
+
--entropy-only Only flag entropy-based findings (skip regex)
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
The scanner walks the tree, applies the regex library to every
|
|
121
|
+
file's contents, and emits a Finding per match with file path, line
|
|
122
|
+
number, severity, and the redacted matched text.
|
|
123
|
+
|
|
124
|
+
### Step 3 — Interpret findings
|
|
125
|
+
|
|
126
|
+
CRITICAL = the matched string is a real credential shape that
|
|
127
|
+
upstream tools auto-extract. Rotate the credential immediately.
|
|
128
|
+
Audit logs for any API call against that credential since the
|
|
129
|
+
commit landed.
|
|
130
|
+
|
|
131
|
+
HIGH = pattern strongly suggests credential but requires manual
|
|
132
|
+
verification (the literal might be a placeholder or test fixture).
|
|
133
|
+
|
|
134
|
+
MEDIUM / LOW = entropy-based heuristic that needs human review.
|
|
135
|
+
|
|
136
|
+
### Step 4 — Remediation
|
|
137
|
+
|
|
138
|
+
For any confirmed real credential:
|
|
139
|
+
|
|
140
|
+
1. **Rotate immediately.** Don't wait to refactor; the leak window
|
|
141
|
+
is between when the commit landed and when you rotate.
|
|
142
|
+
2. **Audit usage.** Check provider's API logs for any unfamiliar
|
|
143
|
+
calls against that credential since the leak commit timestamp.
|
|
144
|
+
3. **Remove from source.** Move to environment variables, secrets
|
|
145
|
+
manager, or a runtime-provisioned secret. See
|
|
146
|
+
`references/PLAYBOOK.md` for per-language patterns.
|
|
147
|
+
4. **Scrub history if reasonable.** `git filter-repo` or `BFG
|
|
148
|
+
Repo-Cleaner` can purge the secret from history, but only if you
|
|
149
|
+
can force-push and coordinate with every clone-holder. For
|
|
150
|
+
public repos, history-scrub is often not worth the disruption
|
|
151
|
+
compared to just rotating.
|
|
152
|
+
|
|
153
|
+
## Examples
|
|
154
|
+
|
|
155
|
+
### Example 1 — Pre-commit gate
|
|
156
|
+
|
|
157
|
+
```bash
|
|
158
|
+
# .git/hooks/pre-commit (or via pre-commit framework)
|
|
159
|
+
python3 plugins/security/penetration-tester/skills/scanning-for-hardcoded-secrets/scripts/scan_secrets.py \
|
|
160
|
+
--min-severity high --format json . | jq -e 'length == 0' \
|
|
161
|
+
|| { echo "Secrets detected. Fix before commit."; exit 1; }
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### Example 2 — CI scan on every push
|
|
165
|
+
|
|
166
|
+
```yaml
|
|
167
|
+
- name: Hardcoded-secrets scan
|
|
168
|
+
run: |
|
|
169
|
+
python3 plugins/security/penetration-tester/skills/scanning-for-hardcoded-secrets/scripts/scan_secrets.py \
|
|
170
|
+
. --min-severity high --format json --output secrets-scan.json
|
|
171
|
+
- run: |
|
|
172
|
+
if jq 'length > 0' secrets-scan.json | grep -q true; then
|
|
173
|
+
echo "::error::Hardcoded secret detected"
|
|
174
|
+
exit 1
|
|
175
|
+
fi
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
### Example 3 — Audit inherited codebase
|
|
179
|
+
|
|
180
|
+
```bash
|
|
181
|
+
python3 ${CLAUDE_PLUGIN_ROOT}/skills/scanning-for-hardcoded-secrets/scripts/scan_secrets.py \
|
|
182
|
+
/path/to/acquired-repo --include-tests --min-severity medium
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
`--include-tests` is important here because legacy test fixtures
|
|
186
|
+
often contain real credentials someone forgot to redact.
|
|
187
|
+
|
|
188
|
+
## Output
|
|
189
|
+
|
|
190
|
+
JSON / JSONL / Markdown per `lib/report.py`. Exit codes: 0 clean, 1
|
|
191
|
+
high/critical, 2 error.
|
|
192
|
+
|
|
193
|
+
Matched strings are partially redacted in output (first 4 + last 4
|
|
194
|
+
chars visible, middle redacted) to avoid the scanner output itself
|
|
195
|
+
becoming a leak surface.
|
|
196
|
+
|
|
197
|
+
## Error Handling
|
|
198
|
+
|
|
199
|
+
- **False positive on placeholder strings** like `<YOUR_KEY_HERE>` →
|
|
200
|
+
the scanner skips strings containing `<`, `>`, `EXAMPLE`,
|
|
201
|
+
`PLACEHOLDER`, `YOUR_`, `XXXX` (configurable).
|
|
202
|
+
- **Binary file in tree** → skipped (the scanner reads only text
|
|
203
|
+
files by content-type sniffing).
|
|
204
|
+
- **Large file** → files >5 MB are skipped (avoids scanning compiled
|
|
205
|
+
artifacts and lockfiles).
|
|
206
|
+
|
|
207
|
+
## Resources
|
|
208
|
+
|
|
209
|
+
- `references/THEORY.md` — Per-credential-family threat model, why
|
|
210
|
+
each provider's keys are extracted by bots first, history-scrub
|
|
211
|
+
decision framework, entropy-detection theory
|
|
212
|
+
- `references/PLAYBOOK.md` — Per-language migration patterns
|
|
213
|
+
(Python dotenv, Node .env+dotenv, Ruby Rails credentials, Go
|
|
214
|
+
envconfig), provider rotation procedures, GitHub secret-scanning
|
|
215
|
+
integration
|