feed-the-machine 1.6.0 → 1.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +170 -170
- package/bin/brain.py +1340 -0
- package/bin/convert_claude_skills_to_codex.py +490 -0
- package/bin/generate-manifest.mjs +463 -463
- package/bin/harden_codex_skills.py +141 -0
- package/bin/install.mjs +491 -491
- package/bin/migrate-eng-buddy-data.py +875 -0
- package/bin/playbook_engine/__init__.py +1 -0
- package/bin/playbook_engine/conftest.py +8 -0
- package/bin/playbook_engine/extractor.py +33 -0
- package/bin/playbook_engine/manager.py +102 -0
- package/bin/playbook_engine/models.py +84 -0
- package/bin/playbook_engine/registry.py +35 -0
- package/bin/playbook_engine/test_extractor.py +72 -0
- package/bin/playbook_engine/test_integration.py +129 -0
- package/bin/playbook_engine/test_manager.py +85 -0
- package/bin/playbook_engine/test_models.py +166 -0
- package/bin/playbook_engine/test_registry.py +67 -0
- package/bin/playbook_engine/test_tracer.py +86 -0
- package/bin/playbook_engine/tracer.py +93 -0
- package/bin/tasks_db.py +456 -0
- package/docs/HOOKS.md +243 -243
- package/docs/INBOX.md +233 -233
- package/ftm/SKILL.md +125 -122
- package/ftm-audit/SKILL.md +623 -623
- package/ftm-audit/references/protocols/PROJECT-PATTERNS.md +91 -91
- package/ftm-audit/references/protocols/RUNTIME-WIRING.md +66 -66
- package/ftm-audit/references/protocols/WIRING-CONTRACTS.md +135 -135
- package/ftm-audit/references/strategies/AUTO-FIX-STRATEGIES.md +69 -69
- package/ftm-audit/references/templates/REPORT-FORMAT.md +96 -96
- package/ftm-audit/scripts/run-knip.sh +23 -23
- package/ftm-audit.yml +2 -2
- package/ftm-brainstorm/SKILL.md +1003 -498
- package/ftm-brainstorm/evals/evals.json +180 -100
- package/ftm-brainstorm/evals/promptfoo.yaml +109 -109
- package/ftm-brainstorm/references/agent-prompts.md +552 -224
- package/ftm-brainstorm/references/plan-template.md +209 -121
- package/ftm-brainstorm.yml +2 -2
- package/ftm-browse/SKILL.md +454 -454
- package/ftm-browse/daemon/browser-manager.ts +206 -206
- package/ftm-browse/daemon/bun.lock +30 -30
- package/ftm-browse/daemon/cli.ts +347 -347
- package/ftm-browse/daemon/commands.ts +410 -410
- package/ftm-browse/daemon/main.ts +357 -357
- package/ftm-browse/daemon/package.json +17 -17
- package/ftm-browse/daemon/server.ts +189 -189
- package/ftm-browse/daemon/snapshot.ts +519 -519
- package/ftm-browse/daemon/tsconfig.json +22 -22
- package/ftm-browse.yml +4 -4
- package/ftm-capture/SKILL.md +370 -370
- package/ftm-capture.yml +4 -4
- package/ftm-codex-gate/SKILL.md +361 -361
- package/ftm-codex-gate.yml +2 -2
- package/ftm-config/SKILL.md +422 -345
- package/ftm-config.default.yml +125 -82
- package/ftm-config.yml +44 -2
- package/ftm-council/SKILL.md +416 -416
- package/ftm-council/references/prompts/CLAUDE-INVESTIGATION.md +60 -60
- package/ftm-council/references/prompts/CODEX-INVESTIGATION.md +58 -58
- package/ftm-council/references/prompts/GEMINI-INVESTIGATION.md +58 -58
- package/ftm-council/references/prompts/REBUTTAL-TEMPLATE.md +57 -57
- package/ftm-council/references/protocols/PREREQUISITES.md +47 -47
- package/ftm-council/references/protocols/STEP-0-FRAMING.md +46 -46
- package/ftm-council.yml +2 -2
- package/ftm-dashboard/SKILL.md +163 -163
- package/ftm-dashboard.yml +4 -4
- package/ftm-debug/SKILL.md +1037 -1037
- package/ftm-debug/references/phases/PHASE-0-INTAKE.md +58 -58
- package/ftm-debug/references/phases/PHASE-1-TRIAGE.md +46 -46
- package/ftm-debug/references/phases/PHASE-2-WAR-ROOM-AGENTS.md +279 -279
- package/ftm-debug/references/phases/PHASE-3-TO-6-EXECUTION.md +436 -436
- package/ftm-debug/references/protocols/BLACKBOARD.md +86 -86
- package/ftm-debug/references/protocols/EDGE-CASES.md +103 -103
- package/ftm-debug.yml +2 -2
- package/ftm-diagram/SKILL.md +277 -277
- package/ftm-diagram.yml +2 -2
- package/ftm-executor/SKILL.md +777 -777
- package/ftm-executor/references/STYLE-TEMPLATE.md +73 -73
- package/ftm-executor/references/phases/PHASE-0-VERIFICATION.md +62 -62
- package/ftm-executor/references/phases/PHASE-2-AGENT-ASSEMBLY.md +34 -34
- package/ftm-executor/references/phases/PHASE-3-WORKTREES.md +38 -38
- package/ftm-executor/references/phases/PHASE-4-5-AUDIT.md +72 -72
- package/ftm-executor/references/phases/PHASE-4-DISPATCH.md +66 -66
- package/ftm-executor/references/phases/PHASE-5-5-CODEX-GATE.md +73 -73
- package/ftm-executor/references/protocols/DOCUMENTATION-BOOTSTRAP.md +36 -36
- package/ftm-executor/references/protocols/MODEL-PROFILE.md +59 -59
- package/ftm-executor/references/protocols/PROGRESS-TRACKING.md +66 -66
- package/ftm-executor/runtime/ftm-runtime.mjs +252 -252
- package/ftm-executor/runtime/package.json +8 -8
- package/ftm-executor.yml +2 -2
- package/ftm-git/SKILL.md +441 -441
- package/ftm-git/evals/evals.json +26 -26
- package/ftm-git/evals/promptfoo.yaml +75 -75
- package/ftm-git/hooks/post-commit-experience.sh +92 -92
- package/ftm-git/references/patterns/SECRET-PATTERNS.md +104 -104
- package/ftm-git/references/protocols/REMEDIATION.md +139 -139
- package/ftm-git/scripts/pre-commit-secrets.sh +110 -110
- package/ftm-git.yml +2 -2
- package/ftm-inbox/backend/__pycache__/main.cpython-314.pyc +0 -0
- package/ftm-inbox/backend/adapters/_retry.py +64 -64
- package/ftm-inbox/backend/adapters/base.py +230 -230
- package/ftm-inbox/backend/adapters/freshservice.py +104 -104
- package/ftm-inbox/backend/adapters/gmail.py +125 -125
- package/ftm-inbox/backend/adapters/jira.py +136 -136
- package/ftm-inbox/backend/adapters/registry.py +192 -192
- package/ftm-inbox/backend/adapters/slack.py +110 -110
- package/ftm-inbox/backend/db/connection.py +54 -54
- package/ftm-inbox/backend/db/schema.py +78 -78
- package/ftm-inbox/backend/executor/__init__.py +7 -7
- package/ftm-inbox/backend/executor/engine.py +149 -149
- package/ftm-inbox/backend/executor/step_runner.py +98 -98
- package/ftm-inbox/backend/main.py +103 -103
- package/ftm-inbox/backend/models/__init__.py +1 -1
- package/ftm-inbox/backend/models/unified_task.py +36 -36
- package/ftm-inbox/backend/planner/__init__.py +6 -6
- package/ftm-inbox/backend/planner/__pycache__/__init__.cpython-314.pyc +0 -0
- package/ftm-inbox/backend/planner/__pycache__/generator.cpython-314.pyc +0 -0
- package/ftm-inbox/backend/planner/__pycache__/schema.cpython-314.pyc +0 -0
- package/ftm-inbox/backend/planner/generator.py +127 -127
- package/ftm-inbox/backend/planner/schema.py +34 -34
- package/ftm-inbox/backend/requirements.txt +5 -5
- package/ftm-inbox/backend/routes/__pycache__/plan.cpython-314.pyc +0 -0
- package/ftm-inbox/backend/routes/execute.py +186 -186
- package/ftm-inbox/backend/routes/health.py +52 -52
- package/ftm-inbox/backend/routes/inbox.py +68 -68
- package/ftm-inbox/backend/routes/plan.py +271 -271
- package/ftm-inbox/bin/launchagent.mjs +91 -91
- package/ftm-inbox/bin/setup.mjs +188 -188
- package/ftm-inbox/bin/start.sh +10 -10
- package/ftm-inbox/bin/status.sh +17 -17
- package/ftm-inbox/bin/stop.sh +8 -8
- package/ftm-inbox/config.example.yml +55 -55
- package/ftm-inbox/package-lock.json +2898 -2898
- package/ftm-inbox/package.json +26 -26
- package/ftm-inbox/postcss.config.js +6 -6
- package/ftm-inbox/src/app.css +199 -199
- package/ftm-inbox/src/app.html +18 -18
- package/ftm-inbox/src/lib/api.ts +166 -166
- package/ftm-inbox/src/lib/components/ExecutionLog.svelte +81 -81
- package/ftm-inbox/src/lib/components/InboxFeed.svelte +143 -143
- package/ftm-inbox/src/lib/components/PlanStep.svelte +271 -271
- package/ftm-inbox/src/lib/components/PlanView.svelte +206 -206
- package/ftm-inbox/src/lib/components/StreamPanel.svelte +99 -99
- package/ftm-inbox/src/lib/components/TaskCard.svelte +190 -190
- package/ftm-inbox/src/lib/components/ui/EmptyState.svelte +63 -63
- package/ftm-inbox/src/lib/components/ui/KawaiiCard.svelte +86 -86
- package/ftm-inbox/src/lib/components/ui/PillButton.svelte +106 -106
- package/ftm-inbox/src/lib/components/ui/StatusBadge.svelte +67 -67
- package/ftm-inbox/src/lib/components/ui/StreamDrawer.svelte +149 -149
- package/ftm-inbox/src/lib/components/ui/ThemeToggle.svelte +80 -80
- package/ftm-inbox/src/lib/theme.ts +47 -47
- package/ftm-inbox/src/routes/+layout.svelte +76 -76
- package/ftm-inbox/src/routes/+page.svelte +401 -401
- package/ftm-inbox/svelte.config.js +12 -12
- package/ftm-inbox/tailwind.config.ts +63 -63
- package/ftm-inbox/tsconfig.json +13 -13
- package/ftm-inbox/vite.config.ts +6 -6
- package/ftm-intent/SKILL.md +241 -241
- package/ftm-intent.yml +2 -2
- package/ftm-manifest.json +3794 -3794
- package/ftm-map/SKILL.md +291 -291
- package/ftm-map/scripts/db.py +712 -712
- package/ftm-map/scripts/index.py +415 -415
- package/ftm-map/scripts/parser.py +224 -224
- package/ftm-map/scripts/queries/go-tags.scm +20 -20
- package/ftm-map/scripts/queries/javascript-tags.scm +35 -35
- package/ftm-map/scripts/queries/python-tags.scm +31 -31
- package/ftm-map/scripts/queries/ruby-tags.scm +19 -19
- package/ftm-map/scripts/queries/rust-tags.scm +37 -37
- package/ftm-map/scripts/queries/typescript-tags.scm +41 -41
- package/ftm-map/scripts/query.py +301 -301
- package/ftm-map/scripts/ranker.py +377 -377
- package/ftm-map/scripts/requirements.txt +5 -5
- package/ftm-map/scripts/setup-hooks.sh +27 -27
- package/ftm-map/scripts/setup.sh +56 -56
- package/ftm-map/scripts/test_db.py +364 -364
- package/ftm-map/scripts/test_parser.py +174 -174
- package/ftm-map/scripts/test_query.py +183 -183
- package/ftm-map/scripts/test_ranker.py +199 -199
- package/ftm-map/scripts/views.py +591 -591
- package/ftm-map.yml +2 -2
- package/ftm-mind/SKILL.md +201 -1943
- package/ftm-mind/evals/promptfoo.yaml +142 -142
- package/ftm-mind/references/blackboard-protocol.md +110 -0
- package/ftm-mind/references/blackboard-schema.md +328 -328
- package/ftm-mind/references/complexity-guide.md +110 -110
- package/ftm-mind/references/complexity-sizing.md +138 -0
- package/ftm-mind/references/decide-act-protocol.md +172 -0
- package/ftm-mind/references/direct-execution.md +51 -0
- package/ftm-mind/references/environment-discovery.md +77 -0
- package/ftm-mind/references/event-registry.md +319 -319
- package/ftm-mind/references/mcp-inventory.md +300 -296
- package/ftm-mind/references/ops-routing.md +47 -0
- package/ftm-mind/references/orient-protocol.md +234 -0
- package/ftm-mind/references/personality.md +40 -0
- package/ftm-mind/references/protocols/COMPLEXITY-SIZING.md +72 -72
- package/ftm-mind/references/protocols/MCP-HEURISTICS.md +32 -32
- package/ftm-mind/references/protocols/PLAN-APPROVAL.md +80 -80
- package/ftm-mind/references/reflexion-protocol.md +249 -249
- package/ftm-mind/references/routing/SCENARIOS.md +22 -22
- package/ftm-mind/references/routing-scenarios.md +35 -35
- package/ftm-mind.yml +2 -2
- package/ftm-ops.yml +4 -0
- package/ftm-pause/SKILL.md +395 -395
- package/ftm-pause/references/protocols/SKILL-RESTORE-PROTOCOLS.md +186 -186
- package/ftm-pause/references/protocols/VALIDATION.md +80 -80
- package/ftm-pause.yml +2 -2
- package/ftm-researcher/SKILL.md +275 -275
- package/ftm-researcher/evals/agent-diversity.yaml +17 -17
- package/ftm-researcher/evals/synthesis-quality.yaml +12 -12
- package/ftm-researcher/evals/trigger-accuracy.yaml +39 -39
- package/ftm-researcher/references/adaptive-search.md +116 -116
- package/ftm-researcher/references/agent-prompts.md +193 -193
- package/ftm-researcher/references/council-integration.md +193 -193
- package/ftm-researcher/references/output-format.md +203 -203
- package/ftm-researcher/references/synthesis-pipeline.md +165 -165
- package/ftm-researcher/scripts/score_credibility.py +234 -234
- package/ftm-researcher/scripts/validate_research.py +92 -92
- package/ftm-researcher.yml +2 -2
- package/ftm-resume/SKILL.md +518 -518
- package/ftm-resume/references/protocols/VALIDATION.md +172 -172
- package/ftm-resume.yml +2 -2
- package/ftm-retro/SKILL.md +380 -380
- package/ftm-retro/references/protocols/SCORING-RUBRICS.md +89 -89
- package/ftm-retro/references/templates/REPORT-FORMAT.md +109 -109
- package/ftm-retro.yml +2 -2
- package/ftm-routine/SKILL.md +170 -170
- package/ftm-routine.yml +4 -4
- package/ftm-state/blackboard/capabilities.json +5 -5
- package/ftm-state/blackboard/capabilities.schema.json +27 -27
- package/ftm-state/blackboard/context.json +37 -23
- package/ftm-state/blackboard/experiences/doom-statusline-fix.json +26 -0
- package/ftm-state/blackboard/experiences/hackathon-pages-site.json +26 -0
- package/ftm-state/blackboard/experiences/hindsight-sso-kickoff.json +42 -0
- package/ftm-state/blackboard/experiences/index.json +58 -9
- package/ftm-state/blackboard/experiences/learning-ragnarok-api-access.json +23 -0
- package/ftm-state/blackboard/experiences/nordlayer-members-auto-assign.json +26 -0
- package/ftm-state/blackboard/experiences/saml2aws-stale-session-fix.json +41 -0
- package/ftm-state/blackboard/patterns.json +6 -6
- package/ftm-state/schemas/context.schema.json +130 -130
- package/ftm-state/schemas/experience-index.schema.json +77 -77
- package/ftm-state/schemas/experience.schema.json +78 -78
- package/ftm-state/schemas/patterns.schema.json +44 -44
- package/ftm-upgrade/SKILL.md +194 -194
- package/ftm-upgrade/scripts/check-version.sh +76 -76
- package/ftm-upgrade/scripts/upgrade.sh +143 -143
- package/ftm-upgrade.yml +2 -2
- package/ftm-verify.yml +2 -2
- package/ftm.yml +2 -2
- package/hooks/ftm-auto-log.sh +137 -0
- package/hooks/ftm-blackboard-enforcer.sh +93 -93
- package/hooks/ftm-discovery-reminder.sh +90 -90
- package/hooks/ftm-drafts-gate.sh +61 -61
- package/hooks/ftm-event-logger.mjs +107 -107
- package/hooks/ftm-install-hooks.sh +240 -0
- package/hooks/ftm-learning-capture.sh +117 -0
- package/hooks/ftm-map-autodetect.sh +79 -79
- package/hooks/ftm-pending-sync-check.sh +22 -22
- package/hooks/ftm-plan-gate.sh +92 -92
- package/hooks/ftm-post-commit-trigger.sh +57 -57
- package/hooks/ftm-post-compaction.sh +138 -0
- package/hooks/ftm-pre-compaction.sh +147 -0
- package/hooks/ftm-session-end.sh +52 -0
- package/hooks/ftm-session-snapshot.sh +213 -0
- package/hooks/settings-template.json +81 -81
- package/install.sh +363 -363
- package/package.json +84 -84
- package/uninstall.sh +25 -25
package/ftm-git/evals/evals.json
CHANGED
|
@@ -1,26 +1,26 @@
|
|
|
1
|
-
{
|
|
2
|
-
"skill_name": "ftm-git",
|
|
3
|
-
"evals": [
|
|
4
|
-
{
|
|
5
|
-
"id": 1,
|
|
6
|
-
"name": "python-stripe-aws",
|
|
7
|
-
"prompt": "I just finished the payments module. commit this and push it up",
|
|
8
|
-
"expected_output": "Should detect sk_live_ Stripe key, AKIA AWS access key, and AWS secret. Should extract all three to .env, refactor payments.py to use os.environ/os.getenv, add .env to .gitignore, and block the commit until remediated.",
|
|
9
|
-
"files": ["test-fixtures/payments.py"]
|
|
10
|
-
},
|
|
11
|
-
{
|
|
12
|
-
"id": 2,
|
|
13
|
-
"name": "js-config-unignored-env",
|
|
14
|
-
"prompt": "hey can you scan this repo for any secrets before I share it with the new contractor",
|
|
15
|
-
"expected_output": "Should find Google API key, Slack bot token, Slack webhook, and SendGrid key in config.js. Should also flag .env not being in .gitignore. Should extract secrets to .env, refactor config.js to use process.env, ensure .gitignore covers .env files.",
|
|
16
|
-
"files": ["test-fixtures/config.js", "test-fixtures/.env"]
|
|
17
|
-
},
|
|
18
|
-
{
|
|
19
|
-
"id": 3,
|
|
20
|
-
"name": "yaml-google-key-in-ci",
|
|
21
|
-
"prompt": "push to main",
|
|
22
|
-
"expected_output": "Should detect Google API key hardcoded in deploy.yml env block, the RSA private key, and the Slack webhook URL. Should flag that CI files need to use GitHub Actions secrets (${{ secrets.VAR }}) instead of hardcoded values. Should block the push.",
|
|
23
|
-
"files": ["test-fixtures/deploy.yml"]
|
|
24
|
-
}
|
|
25
|
-
]
|
|
26
|
-
}
|
|
1
|
+
{
|
|
2
|
+
"skill_name": "ftm-git",
|
|
3
|
+
"evals": [
|
|
4
|
+
{
|
|
5
|
+
"id": 1,
|
|
6
|
+
"name": "python-stripe-aws",
|
|
7
|
+
"prompt": "I just finished the payments module. commit this and push it up",
|
|
8
|
+
"expected_output": "Should detect sk_live_ Stripe key, AKIA AWS access key, and AWS secret. Should extract all three to .env, refactor payments.py to use os.environ/os.getenv, add .env to .gitignore, and block the commit until remediated.",
|
|
9
|
+
"files": ["test-fixtures/payments.py"]
|
|
10
|
+
},
|
|
11
|
+
{
|
|
12
|
+
"id": 2,
|
|
13
|
+
"name": "js-config-unignored-env",
|
|
14
|
+
"prompt": "hey can you scan this repo for any secrets before I share it with the new contractor",
|
|
15
|
+
"expected_output": "Should find Google API key, Slack bot token, Slack webhook, and SendGrid key in config.js. Should also flag .env not being in .gitignore. Should extract secrets to .env, refactor config.js to use process.env, ensure .gitignore covers .env files.",
|
|
16
|
+
"files": ["test-fixtures/config.js", "test-fixtures/.env"]
|
|
17
|
+
},
|
|
18
|
+
{
|
|
19
|
+
"id": 3,
|
|
20
|
+
"name": "yaml-google-key-in-ci",
|
|
21
|
+
"prompt": "push to main",
|
|
22
|
+
"expected_output": "Should detect Google API key hardcoded in deploy.yml env block, the RSA private key, and the Slack webhook URL. Should flag that CI files need to use GitHub Actions secrets (${{ secrets.VAR }}) instead of hardcoded values. Should block the push.",
|
|
23
|
+
"files": ["test-fixtures/deploy.yml"]
|
|
24
|
+
}
|
|
25
|
+
]
|
|
26
|
+
}
|
|
@@ -1,75 +1,75 @@
|
|
|
1
|
-
description: "ftm-git secret detection and remediation"
|
|
2
|
-
|
|
3
|
-
prompts:
|
|
4
|
-
- "{{input}}"
|
|
5
|
-
|
|
6
|
-
providers:
|
|
7
|
-
- id: "exec:claude --print"
|
|
8
|
-
label: "claude-code"
|
|
9
|
-
|
|
10
|
-
tests:
|
|
11
|
-
# Eval 1: python-stripe-aws
|
|
12
|
-
- description: "detects Stripe live key and AWS credentials, extracts to .env, blocks commit"
|
|
13
|
-
vars:
|
|
14
|
-
input: "I just finished the payments module. commit this and push it up"
|
|
15
|
-
assert:
|
|
16
|
-
- type: regex
|
|
17
|
-
value: "sk_live_|stripe|Stripe"
|
|
18
|
-
description: "Detects the Stripe live key"
|
|
19
|
-
- type: regex
|
|
20
|
-
value: "AKIA|AWS|aws"
|
|
21
|
-
description: "Detects the AWS access key"
|
|
22
|
-
- type: regex
|
|
23
|
-
value: "\\.env"
|
|
24
|
-
description: "References .env file for secret extraction"
|
|
25
|
-
- type: regex
|
|
26
|
-
value: "\\.gitignore"
|
|
27
|
-
description: "References .gitignore to prevent committing secrets"
|
|
28
|
-
- type: regex
|
|
29
|
-
value: "block|blocked|cannot commit|will not commit|stop|halt|remediat"
|
|
30
|
-
description: "Blocks the commit until secrets are remediated"
|
|
31
|
-
|
|
32
|
-
# Eval 2: js-config-unignored-env
|
|
33
|
-
- description: "detects multiple secrets in config.js, flags missing .gitignore entry, extracts to .env"
|
|
34
|
-
vars:
|
|
35
|
-
input: "hey can you scan this repo for any secrets before I share it with the new contractor"
|
|
36
|
-
assert:
|
|
37
|
-
- type: regex
|
|
38
|
-
value: "Google|GOOGLE|AIza"
|
|
39
|
-
description: "Detects the Google API key"
|
|
40
|
-
- type: regex
|
|
41
|
-
value: "Slack|slack|xoxb"
|
|
42
|
-
description: "Detects the Slack token"
|
|
43
|
-
- type: regex
|
|
44
|
-
value: "SendGrid|sendgrid|SG\\."
|
|
45
|
-
description: "Detects the SendGrid key"
|
|
46
|
-
- type: regex
|
|
47
|
-
value: "\\.env"
|
|
48
|
-
description: "References .env file for secret extraction"
|
|
49
|
-
- type: regex
|
|
50
|
-
value: "\\.gitignore"
|
|
51
|
-
description: "Flags .gitignore coverage issue"
|
|
52
|
-
- type: regex
|
|
53
|
-
value: "process\\.env"
|
|
54
|
-
description: "Recommends using process.env in refactored code"
|
|
55
|
-
|
|
56
|
-
# Eval 3: yaml-google-key-in-ci
|
|
57
|
-
- description: "detects hardcoded secrets in CI yaml, recommends GitHub Actions secrets, blocks push"
|
|
58
|
-
vars:
|
|
59
|
-
input: "push to main"
|
|
60
|
-
assert:
|
|
61
|
-
- type: regex
|
|
62
|
-
value: "Google|GOOGLE|AIza|deploy\\.yml|CI|ci"
|
|
63
|
-
description: "Detects the Google API key in the CI file"
|
|
64
|
-
- type: regex
|
|
65
|
-
value: "RSA|private key|BEGIN RSA|-----BEGIN"
|
|
66
|
-
description: "Detects the RSA private key"
|
|
67
|
-
- type: regex
|
|
68
|
-
value: "Slack|slack|webhook"
|
|
69
|
-
description: "Detects the Slack webhook URL"
|
|
70
|
-
- type: regex
|
|
71
|
-
value: "secrets\\.|\\$\\{\\{|GitHub Actions|github actions|Actions secret"
|
|
72
|
-
description: "Recommends using GitHub Actions secrets instead of hardcoded values"
|
|
73
|
-
- type: regex
|
|
74
|
-
value: "block|blocked|cannot push|will not push|stop|halt|remediat"
|
|
75
|
-
description: "Blocks the push until secrets are remediated"
|
|
1
|
+
description: "ftm-git secret detection and remediation"
|
|
2
|
+
|
|
3
|
+
prompts:
|
|
4
|
+
- "{{input}}"
|
|
5
|
+
|
|
6
|
+
providers:
|
|
7
|
+
- id: "exec:claude --print"
|
|
8
|
+
label: "claude-code"
|
|
9
|
+
|
|
10
|
+
tests:
|
|
11
|
+
# Eval 1: python-stripe-aws
|
|
12
|
+
- description: "detects Stripe live key and AWS credentials, extracts to .env, blocks commit"
|
|
13
|
+
vars:
|
|
14
|
+
input: "I just finished the payments module. commit this and push it up"
|
|
15
|
+
assert:
|
|
16
|
+
- type: regex
|
|
17
|
+
value: "sk_live_|stripe|Stripe"
|
|
18
|
+
description: "Detects the Stripe live key"
|
|
19
|
+
- type: regex
|
|
20
|
+
value: "AKIA|AWS|aws"
|
|
21
|
+
description: "Detects the AWS access key"
|
|
22
|
+
- type: regex
|
|
23
|
+
value: "\\.env"
|
|
24
|
+
description: "References .env file for secret extraction"
|
|
25
|
+
- type: regex
|
|
26
|
+
value: "\\.gitignore"
|
|
27
|
+
description: "References .gitignore to prevent committing secrets"
|
|
28
|
+
- type: regex
|
|
29
|
+
value: "block|blocked|cannot commit|will not commit|stop|halt|remediat"
|
|
30
|
+
description: "Blocks the commit until secrets are remediated"
|
|
31
|
+
|
|
32
|
+
# Eval 2: js-config-unignored-env
|
|
33
|
+
- description: "detects multiple secrets in config.js, flags missing .gitignore entry, extracts to .env"
|
|
34
|
+
vars:
|
|
35
|
+
input: "hey can you scan this repo for any secrets before I share it with the new contractor"
|
|
36
|
+
assert:
|
|
37
|
+
- type: regex
|
|
38
|
+
value: "Google|GOOGLE|AIza"
|
|
39
|
+
description: "Detects the Google API key"
|
|
40
|
+
- type: regex
|
|
41
|
+
value: "Slack|slack|xoxb"
|
|
42
|
+
description: "Detects the Slack token"
|
|
43
|
+
- type: regex
|
|
44
|
+
value: "SendGrid|sendgrid|SG\\."
|
|
45
|
+
description: "Detects the SendGrid key"
|
|
46
|
+
- type: regex
|
|
47
|
+
value: "\\.env"
|
|
48
|
+
description: "References .env file for secret extraction"
|
|
49
|
+
- type: regex
|
|
50
|
+
value: "\\.gitignore"
|
|
51
|
+
description: "Flags .gitignore coverage issue"
|
|
52
|
+
- type: regex
|
|
53
|
+
value: "process\\.env"
|
|
54
|
+
description: "Recommends using process.env in refactored code"
|
|
55
|
+
|
|
56
|
+
# Eval 3: yaml-google-key-in-ci
|
|
57
|
+
- description: "detects hardcoded secrets in CI yaml, recommends GitHub Actions secrets, blocks push"
|
|
58
|
+
vars:
|
|
59
|
+
input: "push to main"
|
|
60
|
+
assert:
|
|
61
|
+
- type: regex
|
|
62
|
+
value: "Google|GOOGLE|AIza|deploy\\.yml|CI|ci"
|
|
63
|
+
description: "Detects the Google API key in the CI file"
|
|
64
|
+
- type: regex
|
|
65
|
+
value: "RSA|private key|BEGIN RSA|-----BEGIN"
|
|
66
|
+
description: "Detects the RSA private key"
|
|
67
|
+
- type: regex
|
|
68
|
+
value: "Slack|slack|webhook"
|
|
69
|
+
description: "Detects the Slack webhook URL"
|
|
70
|
+
- type: regex
|
|
71
|
+
value: "secrets\\.|\\$\\{\\{|GitHub Actions|github actions|Actions secret"
|
|
72
|
+
description: "Recommends using GitHub Actions secrets instead of hardcoded values"
|
|
73
|
+
- type: regex
|
|
74
|
+
value: "block|blocked|cannot push|will not push|stop|halt|remediat"
|
|
75
|
+
description: "Blocks the push until secrets are remediated"
|
|
@@ -1,92 +1,92 @@
|
|
|
1
|
-
#!/usr/bin/env bash
|
|
2
|
-
|
|
3
|
-
# FTM Post-Commit Experience Recorder
|
|
4
|
-
# Ensures every commit produces at least a minimal experience entry.
|
|
5
|
-
# Only creates an entry if one hasn't been recorded recently (2+ min gap).
|
|
6
|
-
|
|
7
|
-
set -euo pipefail
|
|
8
|
-
|
|
9
|
-
STATE_DIR="$HOME/.claude/ftm-state/blackboard"
|
|
10
|
-
EXPERIENCES_DIR="$STATE_DIR/experiences"
|
|
11
|
-
INDEX_FILE="$EXPERIENCES_DIR/index.json"
|
|
12
|
-
|
|
13
|
-
# Ensure directories exist
|
|
14
|
-
mkdir -p "$EXPERIENCES_DIR"
|
|
15
|
-
|
|
16
|
-
# Check if an experience was recorded in the last 2 minutes
|
|
17
|
-
RECENT_THRESHOLD=$(($(date +%s) - 120))
|
|
18
|
-
LATEST_EXPERIENCE=""
|
|
19
|
-
|
|
20
|
-
if [ -d "$EXPERIENCES_DIR" ]; then
|
|
21
|
-
LATEST_EXPERIENCE=$(find "$EXPERIENCES_DIR" -name "*.json" -not -name "index.json" -newer /dev/null -maxdepth 1 2>/dev/null | sort -r | head -1)
|
|
22
|
-
fi
|
|
23
|
-
|
|
24
|
-
if [ -n "$LATEST_EXPERIENCE" ]; then
|
|
25
|
-
# Check if the latest experience file was modified within the last 2 minutes
|
|
26
|
-
if [ "$(uname)" = "Darwin" ]; then
|
|
27
|
-
FILE_TIME=$(stat -f %m "$LATEST_EXPERIENCE" 2>/dev/null || echo 0)
|
|
28
|
-
else
|
|
29
|
-
FILE_TIME=$(stat -c %Y "$LATEST_EXPERIENCE" 2>/dev/null || echo 0)
|
|
30
|
-
fi
|
|
31
|
-
|
|
32
|
-
if [ "$FILE_TIME" -gt "$RECENT_THRESHOLD" ]; then
|
|
33
|
-
# Experience was recently recorded by the LLM — skip
|
|
34
|
-
exit 0
|
|
35
|
-
fi
|
|
36
|
-
fi
|
|
37
|
-
|
|
38
|
-
# Extract commit metadata
|
|
39
|
-
COMMIT_HASH=$(git rev-parse --short HEAD)
|
|
40
|
-
COMMIT_MSG=$(git log -1 --pretty=%s)
|
|
41
|
-
COMMIT_DATE=$(date +%Y-%m-%d)
|
|
42
|
-
COMMIT_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ)
|
|
43
|
-
FILES_CHANGED=$(git diff-tree --no-commit-id --name-only -r HEAD | tr '\n' ', ' | sed 's/,$//')
|
|
44
|
-
BRANCH=$(git rev-parse --abbrev-ref HEAD)
|
|
45
|
-
|
|
46
|
-
# Generate a slug from commit message
|
|
47
|
-
SLUG=$(echo "$COMMIT_MSG" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g' | sed 's/--*/-/g' | cut -c1-50)
|
|
48
|
-
FILENAME="${COMMIT_DATE}_${SLUG}.json"
|
|
49
|
-
|
|
50
|
-
# Don't create duplicate if file already exists
|
|
51
|
-
if [ -f "$EXPERIENCES_DIR/$FILENAME" ]; then
|
|
52
|
-
exit 0
|
|
53
|
-
fi
|
|
54
|
-
|
|
55
|
-
# Create minimal experience entry
|
|
56
|
-
cat > "$EXPERIENCES_DIR/$FILENAME" << EXPEOF
|
|
57
|
-
{
|
|
58
|
-
"task_type": "commit",
|
|
59
|
-
"description": "$COMMIT_MSG",
|
|
60
|
-
"source": "git-hook",
|
|
61
|
-
"timestamp": "$COMMIT_TIME",
|
|
62
|
-
"commit_hash": "$COMMIT_HASH",
|
|
63
|
-
"branch": "$BRANCH",
|
|
64
|
-
"files_changed": "$FILES_CHANGED",
|
|
65
|
-
"complexity_estimated": "micro",
|
|
66
|
-
"complexity_actual": "micro",
|
|
67
|
-
"outcome": "success",
|
|
68
|
-
"confidence": 0.5,
|
|
69
|
-
"tags": ["auto-recorded", "git-commit"],
|
|
70
|
-
"lessons": []
|
|
71
|
-
}
|
|
72
|
-
EXPEOF
|
|
73
|
-
|
|
74
|
-
# Update index.json
|
|
75
|
-
# Read existing index, add new entry, write back
|
|
76
|
-
if [ -f "$INDEX_FILE" ]; then
|
|
77
|
-
# Use node for reliable JSON manipulation (available in FTM environments)
|
|
78
|
-
node -e "
|
|
79
|
-
const fs = require('fs');
|
|
80
|
-
const idx = JSON.parse(fs.readFileSync('$INDEX_FILE', 'utf-8'));
|
|
81
|
-
idx.entries.push({
|
|
82
|
-
file: '$FILENAME',
|
|
83
|
-
task_type: 'commit',
|
|
84
|
-
tags: ['auto-recorded', 'git-commit'],
|
|
85
|
-
timestamp: '$COMMIT_TIME',
|
|
86
|
-
confidence: 0.5
|
|
87
|
-
});
|
|
88
|
-
idx.metadata.total_count = idx.entries.length;
|
|
89
|
-
idx.metadata.last_updated = '$COMMIT_TIME';
|
|
90
|
-
fs.writeFileSync('$INDEX_FILE', JSON.stringify(idx, null, 2));
|
|
91
|
-
" 2>/dev/null || true
|
|
92
|
-
fi
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
|
|
3
|
+
# FTM Post-Commit Experience Recorder
|
|
4
|
+
# Ensures every commit produces at least a minimal experience entry.
|
|
5
|
+
# Only creates an entry if one hasn't been recorded recently (2+ min gap).
|
|
6
|
+
|
|
7
|
+
set -euo pipefail
|
|
8
|
+
|
|
9
|
+
STATE_DIR="$HOME/.claude/ftm-state/blackboard"
|
|
10
|
+
EXPERIENCES_DIR="$STATE_DIR/experiences"
|
|
11
|
+
INDEX_FILE="$EXPERIENCES_DIR/index.json"
|
|
12
|
+
|
|
13
|
+
# Ensure directories exist
|
|
14
|
+
mkdir -p "$EXPERIENCES_DIR"
|
|
15
|
+
|
|
16
|
+
# Check if an experience was recorded in the last 2 minutes
|
|
17
|
+
RECENT_THRESHOLD=$(($(date +%s) - 120))
|
|
18
|
+
LATEST_EXPERIENCE=""
|
|
19
|
+
|
|
20
|
+
if [ -d "$EXPERIENCES_DIR" ]; then
|
|
21
|
+
LATEST_EXPERIENCE=$(find "$EXPERIENCES_DIR" -name "*.json" -not -name "index.json" -newer /dev/null -maxdepth 1 2>/dev/null | sort -r | head -1)
|
|
22
|
+
fi
|
|
23
|
+
|
|
24
|
+
if [ -n "$LATEST_EXPERIENCE" ]; then
|
|
25
|
+
# Check if the latest experience file was modified within the last 2 minutes
|
|
26
|
+
if [ "$(uname)" = "Darwin" ]; then
|
|
27
|
+
FILE_TIME=$(stat -f %m "$LATEST_EXPERIENCE" 2>/dev/null || echo 0)
|
|
28
|
+
else
|
|
29
|
+
FILE_TIME=$(stat -c %Y "$LATEST_EXPERIENCE" 2>/dev/null || echo 0)
|
|
30
|
+
fi
|
|
31
|
+
|
|
32
|
+
if [ "$FILE_TIME" -gt "$RECENT_THRESHOLD" ]; then
|
|
33
|
+
# Experience was recently recorded by the LLM — skip
|
|
34
|
+
exit 0
|
|
35
|
+
fi
|
|
36
|
+
fi
|
|
37
|
+
|
|
38
|
+
# Extract commit metadata
|
|
39
|
+
COMMIT_HASH=$(git rev-parse --short HEAD)
|
|
40
|
+
COMMIT_MSG=$(git log -1 --pretty=%s)
|
|
41
|
+
COMMIT_DATE=$(date +%Y-%m-%d)
|
|
42
|
+
COMMIT_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ)
|
|
43
|
+
FILES_CHANGED=$(git diff-tree --no-commit-id --name-only -r HEAD | tr '\n' ', ' | sed 's/,$//')
|
|
44
|
+
BRANCH=$(git rev-parse --abbrev-ref HEAD)
|
|
45
|
+
|
|
46
|
+
# Generate a slug from commit message
|
|
47
|
+
SLUG=$(echo "$COMMIT_MSG" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g' | sed 's/--*/-/g' | cut -c1-50)
|
|
48
|
+
FILENAME="${COMMIT_DATE}_${SLUG}.json"
|
|
49
|
+
|
|
50
|
+
# Don't create duplicate if file already exists
|
|
51
|
+
if [ -f "$EXPERIENCES_DIR/$FILENAME" ]; then
|
|
52
|
+
exit 0
|
|
53
|
+
fi
|
|
54
|
+
|
|
55
|
+
# Create minimal experience entry
|
|
56
|
+
cat > "$EXPERIENCES_DIR/$FILENAME" << EXPEOF
|
|
57
|
+
{
|
|
58
|
+
"task_type": "commit",
|
|
59
|
+
"description": "$COMMIT_MSG",
|
|
60
|
+
"source": "git-hook",
|
|
61
|
+
"timestamp": "$COMMIT_TIME",
|
|
62
|
+
"commit_hash": "$COMMIT_HASH",
|
|
63
|
+
"branch": "$BRANCH",
|
|
64
|
+
"files_changed": "$FILES_CHANGED",
|
|
65
|
+
"complexity_estimated": "micro",
|
|
66
|
+
"complexity_actual": "micro",
|
|
67
|
+
"outcome": "success",
|
|
68
|
+
"confidence": 0.5,
|
|
69
|
+
"tags": ["auto-recorded", "git-commit"],
|
|
70
|
+
"lessons": []
|
|
71
|
+
}
|
|
72
|
+
EXPEOF
|
|
73
|
+
|
|
74
|
+
# Update index.json
|
|
75
|
+
# Read existing index, add new entry, write back
|
|
76
|
+
if [ -f "$INDEX_FILE" ]; then
|
|
77
|
+
# Use node for reliable JSON manipulation (available in FTM environments)
|
|
78
|
+
node -e "
|
|
79
|
+
const fs = require('fs');
|
|
80
|
+
const idx = JSON.parse(fs.readFileSync('$INDEX_FILE', 'utf-8'));
|
|
81
|
+
idx.entries.push({
|
|
82
|
+
file: '$FILENAME',
|
|
83
|
+
task_type: 'commit',
|
|
84
|
+
tags: ['auto-recorded', 'git-commit'],
|
|
85
|
+
timestamp: '$COMMIT_TIME',
|
|
86
|
+
confidence: 0.5
|
|
87
|
+
});
|
|
88
|
+
idx.metadata.total_count = idx.entries.length;
|
|
89
|
+
idx.metadata.last_updated = '$COMMIT_TIME';
|
|
90
|
+
fs.writeFileSync('$INDEX_FILE', JSON.stringify(idx, null, 2));
|
|
91
|
+
" 2>/dev/null || true
|
|
92
|
+
fi
|
|
@@ -1,104 +1,104 @@
|
|
|
1
|
-
# Secret Patterns — Extended Pattern Library
|
|
2
|
-
|
|
3
|
-
Full regex pattern library for Phase 1 scanning. Tier 1 patterns are high-confidence; Tier 2 require surrounding context validation.
|
|
4
|
-
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
## Tier 1: High-Confidence Patterns (almost certainly real secrets)
|
|
8
|
-
|
|
9
|
-
These patterns have distinctive prefixes or structures that make false positives rare. Run in parallel.
|
|
10
|
-
|
|
11
|
-
```
|
|
12
|
-
# AWS
|
|
13
|
-
AKIA[0-9A-Z]{16} # AWS Access Key ID
|
|
14
|
-
amzn\.mws\.[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12} # AWS MWS
|
|
15
|
-
|
|
16
|
-
# GitHub
|
|
17
|
-
ghp_[A-Za-z0-9_]{36} # GitHub PAT (classic)
|
|
18
|
-
gho_[A-Za-z0-9_]{36} # GitHub OAuth
|
|
19
|
-
ghu_[A-Za-z0-9_]{36} # GitHub user token
|
|
20
|
-
ghs_[A-Za-z0-9_]{36} # GitHub server token
|
|
21
|
-
github_pat_[A-Za-z0-9_]{82} # GitHub fine-grained PAT
|
|
22
|
-
|
|
23
|
-
# Slack
|
|
24
|
-
xoxb-[0-9]{10,13}-[0-9]{10,13}-[a-zA-Z0-9]{24} # Slack bot token
|
|
25
|
-
xoxp-[0-9]{10,13}-[0-9]{10,13}-[a-zA-Z0-9]{24,34} # Slack user token
|
|
26
|
-
xoxa-[0-9]{10,13}-[0-9]{10,13}-[a-zA-Z0-9]{24,34} # Slack app token
|
|
27
|
-
xoxr-[0-9]{10,13}-[0-9]{10,13}-[a-zA-Z0-9]{24,34} # Slack refresh token
|
|
28
|
-
|
|
29
|
-
# Google
|
|
30
|
-
AIza[0-9A-Za-z\-_]{35} # Google API key
|
|
31
|
-
|
|
32
|
-
# Stripe
|
|
33
|
-
sk_live_[0-9a-zA-Z]{24,} # Stripe secret key (live)
|
|
34
|
-
sk_test_[0-9a-zA-Z]{24,} # Stripe secret key (test)
|
|
35
|
-
rk_live_[0-9a-zA-Z]{24,} # Stripe restricted key
|
|
36
|
-
|
|
37
|
-
# Other services
|
|
38
|
-
SG\.[A-Za-z0-9\-_]{22}\.[A-Za-z0-9\-_]{43} # SendGrid
|
|
39
|
-
SK[0-9a-fA-F]{32} # Twilio
|
|
40
|
-
npm_[A-Za-z0-9]{36} # npm token
|
|
41
|
-
pypi-[A-Za-z0-9\-_]{100,} # PyPI token
|
|
42
|
-
glpat-[A-Za-z0-9\-_]{20,} # GitLab PAT
|
|
43
|
-
-----BEGIN (RSA|DSA|EC|OPENSSH|PGP) PRIVATE KEY----- # Private keys
|
|
44
|
-
```
|
|
45
|
-
|
|
46
|
-
---
|
|
47
|
-
|
|
48
|
-
## Tier 2: Context-Dependent Patterns (need surrounding context to confirm)
|
|
49
|
-
|
|
50
|
-
These match common assignment patterns. Check that the value isn't a placeholder, empty string, or env var reference before flagging:
|
|
51
|
-
|
|
52
|
-
```
|
|
53
|
-
# Generic key/secret assignments — flag if value looks real (not placeholder)
|
|
54
|
-
(api_key|apikey|api-key)\s*[:=]\s*["']?[A-Za-z0-9\-_]{16,}["']?
|
|
55
|
-
(secret|secret_key|client_secret)\s*[:=]\s*["']?[A-Za-z0-9\-_]{16,}["']?
|
|
56
|
-
(password|passwd|pwd)\s*[:=]\s*["']?[^\s"']{8,}["']?
|
|
57
|
-
(token|access_token|auth_token)\s*[:=]\s*["']?[A-Za-z0-9\-_.]{16,}["']?
|
|
58
|
-
(database_url|db_url|connection_string)\s*[:=]\s*["']?[^\s"']{20,}["']?
|
|
59
|
-
|
|
60
|
-
# Bearer tokens in code
|
|
61
|
-
bearer\s+[A-Za-z0-9\-._~+/]{20,}
|
|
62
|
-
|
|
63
|
-
# Webhook URLs with tokens
|
|
64
|
-
https://hooks\.slack\.com/services/T[A-Z0-9]{8,}/B[A-Z0-9]{8,}/[a-zA-Z0-9]{24}
|
|
65
|
-
```
|
|
66
|
-
|
|
67
|
-
---
|
|
68
|
-
|
|
69
|
-
## What to Ignore (false positive suppression)
|
|
70
|
-
|
|
71
|
-
Skip matches that are clearly not real secrets:
|
|
72
|
-
|
|
73
|
-
- Values that are `""`, `''`, `None`, `null`, `undefined`, `TODO`, `CHANGEME`, `your-key-here`, `xxx`, `placeholder`, `example`, `test`, `dummy`, `fake`, `sample`
|
|
74
|
-
- References to environment variables: `os.environ[`, `process.env.`, `ENV[`, `${`, `os.getenv(`
|
|
75
|
-
- Lines that are comments (`#`, `//`, `/*`, `--`)
|
|
76
|
-
- Files in `node_modules/`, `.git/`, `vendor/`, `__pycache__/`, `dist/`, `build/`
|
|
77
|
-
- Files that are themselves `.env.example`, `.env.sample`, `.env.template`
|
|
78
|
-
- Lock files (`package-lock.json`, `yarn.lock`, `Gemfile.lock`, `poetry.lock`)
|
|
79
|
-
- Test fixtures where the "secret" is obviously fake (e.g., `test_api_key = "sk_test_abc123"` in a test file — but still flag `sk_live_*` in test files, those are real)
|
|
80
|
-
|
|
81
|
-
---
|
|
82
|
-
|
|
83
|
-
## Severity Classification
|
|
84
|
-
|
|
85
|
-
After validation, findings are sorted by severity:
|
|
86
|
-
|
|
87
|
-
| Severity | Meaning |
|
|
88
|
-
|---|---|
|
|
89
|
-
| **CRITICAL** | Tier 1 match (high-confidence secret) in a tracked or staged file |
|
|
90
|
-
| **HIGH** | Tier 2 confirmed match in a tracked or staged file |
|
|
91
|
-
| **MEDIUM** | `.env` file not in `.gitignore`, or secret in a fallback default |
|
|
92
|
-
| **LOW** | Secret in a gitignored file but the gitignore rule might be fragile |
|
|
93
|
-
|
|
94
|
-
---
|
|
95
|
-
|
|
96
|
-
## Per-Finding Record Format
|
|
97
|
-
|
|
98
|
-
For each finding, record:
|
|
99
|
-
- **file**: absolute path
|
|
100
|
-
- **line**: line number
|
|
101
|
-
- **pattern**: which pattern matched
|
|
102
|
-
- **tier**: 1 or 2
|
|
103
|
-
- **value_preview**: first 8 chars + `...` + last 4 chars (never log the full secret)
|
|
104
|
-
- **context**: the surrounding code (with the secret value masked)
|
|
1
|
+
# Secret Patterns — Extended Pattern Library
|
|
2
|
+
|
|
3
|
+
Full regex pattern library for Phase 1 scanning. Tier 1 patterns are high-confidence; Tier 2 require surrounding context validation.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Tier 1: High-Confidence Patterns (almost certainly real secrets)
|
|
8
|
+
|
|
9
|
+
These patterns have distinctive prefixes or structures that make false positives rare. Run in parallel.
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
# AWS
|
|
13
|
+
AKIA[0-9A-Z]{16} # AWS Access Key ID
|
|
14
|
+
amzn\.mws\.[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12} # AWS MWS
|
|
15
|
+
|
|
16
|
+
# GitHub
|
|
17
|
+
ghp_[A-Za-z0-9_]{36} # GitHub PAT (classic)
|
|
18
|
+
gho_[A-Za-z0-9_]{36} # GitHub OAuth
|
|
19
|
+
ghu_[A-Za-z0-9_]{36} # GitHub user token
|
|
20
|
+
ghs_[A-Za-z0-9_]{36} # GitHub server token
|
|
21
|
+
github_pat_[A-Za-z0-9_]{82} # GitHub fine-grained PAT
|
|
22
|
+
|
|
23
|
+
# Slack
|
|
24
|
+
xoxb-[0-9]{10,13}-[0-9]{10,13}-[a-zA-Z0-9]{24} # Slack bot token
|
|
25
|
+
xoxp-[0-9]{10,13}-[0-9]{10,13}-[a-zA-Z0-9]{24,34} # Slack user token
|
|
26
|
+
xoxa-[0-9]{10,13}-[0-9]{10,13}-[a-zA-Z0-9]{24,34} # Slack app token
|
|
27
|
+
xoxr-[0-9]{10,13}-[0-9]{10,13}-[a-zA-Z0-9]{24,34} # Slack refresh token
|
|
28
|
+
|
|
29
|
+
# Google
|
|
30
|
+
AIza[0-9A-Za-z\-_]{35} # Google API key
|
|
31
|
+
|
|
32
|
+
# Stripe
|
|
33
|
+
sk_live_[0-9a-zA-Z]{24,} # Stripe secret key (live)
|
|
34
|
+
sk_test_[0-9a-zA-Z]{24,} # Stripe secret key (test)
|
|
35
|
+
rk_live_[0-9a-zA-Z]{24,} # Stripe restricted key
|
|
36
|
+
|
|
37
|
+
# Other services
|
|
38
|
+
SG\.[A-Za-z0-9\-_]{22}\.[A-Za-z0-9\-_]{43} # SendGrid
|
|
39
|
+
SK[0-9a-fA-F]{32} # Twilio
|
|
40
|
+
npm_[A-Za-z0-9]{36} # npm token
|
|
41
|
+
pypi-[A-Za-z0-9\-_]{100,} # PyPI token
|
|
42
|
+
glpat-[A-Za-z0-9\-_]{20,} # GitLab PAT
|
|
43
|
+
-----BEGIN (RSA|DSA|EC|OPENSSH|PGP) PRIVATE KEY----- # Private keys
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## Tier 2: Context-Dependent Patterns (need surrounding context to confirm)
|
|
49
|
+
|
|
50
|
+
These match common assignment patterns. Check that the value isn't a placeholder, empty string, or env var reference before flagging:
|
|
51
|
+
|
|
52
|
+
```
|
|
53
|
+
# Generic key/secret assignments — flag if value looks real (not placeholder)
|
|
54
|
+
(api_key|apikey|api-key)\s*[:=]\s*["']?[A-Za-z0-9\-_]{16,}["']?
|
|
55
|
+
(secret|secret_key|client_secret)\s*[:=]\s*["']?[A-Za-z0-9\-_]{16,}["']?
|
|
56
|
+
(password|passwd|pwd)\s*[:=]\s*["']?[^\s"']{8,}["']?
|
|
57
|
+
(token|access_token|auth_token)\s*[:=]\s*["']?[A-Za-z0-9\-_.]{16,}["']?
|
|
58
|
+
(database_url|db_url|connection_string)\s*[:=]\s*["']?[^\s"']{20,}["']?
|
|
59
|
+
|
|
60
|
+
# Bearer tokens in code
|
|
61
|
+
bearer\s+[A-Za-z0-9\-._~+/]{20,}
|
|
62
|
+
|
|
63
|
+
# Webhook URLs with tokens
|
|
64
|
+
https://hooks\.slack\.com/services/T[A-Z0-9]{8,}/B[A-Z0-9]{8,}/[a-zA-Z0-9]{24}
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## What to Ignore (false positive suppression)
|
|
70
|
+
|
|
71
|
+
Skip matches that are clearly not real secrets:
|
|
72
|
+
|
|
73
|
+
- Values that are `""`, `''`, `None`, `null`, `undefined`, `TODO`, `CHANGEME`, `your-key-here`, `xxx`, `placeholder`, `example`, `test`, `dummy`, `fake`, `sample`
|
|
74
|
+
- References to environment variables: `os.environ[`, `process.env.`, `ENV[`, `${`, `os.getenv(`
|
|
75
|
+
- Lines that are comments (`#`, `//`, `/*`, `--`)
|
|
76
|
+
- Files in `node_modules/`, `.git/`, `vendor/`, `__pycache__/`, `dist/`, `build/`
|
|
77
|
+
- Files that are themselves `.env.example`, `.env.sample`, `.env.template`
|
|
78
|
+
- Lock files (`package-lock.json`, `yarn.lock`, `Gemfile.lock`, `poetry.lock`)
|
|
79
|
+
- Test fixtures where the "secret" is obviously fake (e.g., `test_api_key = "sk_test_abc123"` in a test file — but still flag `sk_live_*` in test files, those are real)
|
|
80
|
+
|
|
81
|
+
---
|
|
82
|
+
|
|
83
|
+
## Severity Classification
|
|
84
|
+
|
|
85
|
+
After validation, findings are sorted by severity:
|
|
86
|
+
|
|
87
|
+
| Severity | Meaning |
|
|
88
|
+
|---|---|
|
|
89
|
+
| **CRITICAL** | Tier 1 match (high-confidence secret) in a tracked or staged file |
|
|
90
|
+
| **HIGH** | Tier 2 confirmed match in a tracked or staged file |
|
|
91
|
+
| **MEDIUM** | `.env` file not in `.gitignore`, or secret in a fallback default |
|
|
92
|
+
| **LOW** | Secret in a gitignored file but the gitignore rule might be fragile |
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## Per-Finding Record Format
|
|
97
|
+
|
|
98
|
+
For each finding, record:
|
|
99
|
+
- **file**: absolute path
|
|
100
|
+
- **line**: line number
|
|
101
|
+
- **pattern**: which pattern matched
|
|
102
|
+
- **tier**: 1 or 2
|
|
103
|
+
- **value_preview**: first 8 chars + `...` + last 4 chars (never log the full secret)
|
|
104
|
+
- **context**: the surrounding code (with the secret value masked)
|