@cbartens/bkit 6.0.0 → 6.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.bkit/README.md +154 -151
- package/.bkit/core/.agents/context.md +95 -29
- package/.bkit/core/.agents/docs.md +17 -15
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-gateguard.cjs +1 -0
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-guardrails.cjs +6 -5
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-readguard.cjs +1 -0
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-rules.cjs +65 -0
- package/.bkit/core/.agents/rules.md +108 -107
- package/.bkit/core/.agents/skills.md +63 -15
- package/.bkit/core/.bkit/README.md +154 -151
- package/.bkit/core/.bkit/authoring.md +714 -84
- package/.bkit/core/.bkit/cli.md +407 -0
- package/.bkit/core/.bkit/cloud.md +281 -193
- package/.bkit/core/.bkit/comparison.md +43 -41
- package/.bkit/core/.bkit/install.md +154 -84
- package/.bkit/core/.bkit/jsonl.md +193 -130
- package/.bkit/core/.bkit/pipeline.md +114 -114
- package/.bkit/core/.bkit/prompts.md +272 -165
- package/.bkit/core/.bkit/providers.md +457 -271
- package/.bkit/core/.bkit/router.md +69 -0
- package/.bkit/core/.bkit/templates.md +529 -256
- package/.bkit/core/.bkit/trackers.md +80 -52
- package/.bkit/core/.bkit/workflows.md +589 -504
- package/.bkit/core/.claude/hooks/bkit-gateguard.cjs +2 -2
- package/.bkit/core/.claude/hooks/bkit-guardrails.cjs +3 -3
- package/.bkit/core/.claude/hooks/bkit-preflight.sh +45 -14
- package/.bkit/core/.codex/hooks/bkit-guardrails.cjs +3 -3
- package/.bkit/core/.codex/hooks/bkit-rules.cjs +57 -0
- package/.bkit/core/.cursor/hooks/bkit-guardrails.cjs +7 -7
- package/.bkit/core/.cursor/hooks/bkit-preflight.sh +47 -12
- package/.bkit/core/.opencode/plugin/bkit-rules.js +89 -0
- package/.bkit/core/AGENTS.md +5 -73
- package/.bkit/core/docs/templates/bkit-audit-accessibility.md +127 -105
- package/.bkit/core/docs/templates/bkit-audit-adoption.md +104 -70
- package/.bkit/core/docs/templates/bkit-audit-agents.md +94 -72
- package/.bkit/core/docs/templates/bkit-audit-aio-seo.md +99 -78
- package/.bkit/core/docs/templates/bkit-audit-analytics.md +140 -82
- package/.bkit/core/docs/templates/bkit-audit-annotations.md +98 -72
- package/.bkit/core/docs/templates/bkit-audit-app-performance.md +121 -96
- package/.bkit/core/docs/templates/bkit-audit-app-store.md +157 -76
- package/.bkit/core/docs/templates/bkit-audit-architecture.md +113 -71
- package/.bkit/core/docs/templates/bkit-audit-ci-performance.md +75 -72
- package/.bkit/core/docs/templates/bkit-audit-config.md +122 -76
- package/.bkit/core/docs/templates/bkit-audit-cost.md +85 -74
- package/.bkit/core/docs/templates/bkit-audit-coverage.md +75 -68
- package/.bkit/core/docs/templates/bkit-audit-database.md +101 -70
- package/.bkit/core/docs/templates/bkit-audit-dead-code.md +164 -83
- package/.bkit/core/docs/templates/bkit-audit-deferrals.md +115 -86
- package/.bkit/core/docs/templates/bkit-audit-dependencies.md +264 -88
- package/.bkit/core/docs/templates/bkit-audit-documentation.md +118 -93
- package/.bkit/core/docs/templates/bkit-audit-duplication.md +139 -86
- package/.bkit/core/docs/templates/bkit-audit-failures.md +110 -81
- package/.bkit/core/docs/templates/bkit-audit-flakes.md +99 -74
- package/.bkit/core/docs/templates/bkit-audit-gates.md +105 -77
- package/.bkit/core/docs/templates/bkit-audit-issues.md +104 -80
- package/.bkit/core/docs/templates/bkit-audit-llm.md +131 -58
- package/.bkit/core/docs/templates/bkit-audit-logs.md +123 -100
- package/.bkit/core/docs/templates/bkit-audit-models.md +98 -53
- package/.bkit/core/docs/templates/bkit-audit-privacy.md +56 -52
- package/.bkit/core/docs/templates/bkit-audit-prompts.md +63 -54
- package/.bkit/core/docs/templates/bkit-audit-red-team.md +124 -68
- package/.bkit/core/docs/templates/bkit-audit-reliability.md +95 -70
- package/.bkit/core/docs/templates/bkit-audit-repo.md +169 -73
- package/.bkit/core/docs/templates/bkit-audit-security.md +206 -96
- package/.bkit/core/docs/templates/bkit-audit-slop.md +77 -70
- package/.bkit/core/docs/templates/bkit-audit-testing.md +162 -114
- package/.bkit/core/docs/templates/bkit-audit-usability.md +113 -71
- package/.bkit/core/docs/templates/bkit-plan.md +138 -132
- package/.bkit/core/engine/checks/after/auth.ts +40 -37
- package/.bkit/core/engine/checks/after/ci-yml.ts +65 -2
- package/.bkit/core/engine/checks/after/index.ts +12 -2
- package/.bkit/core/engine/checks/after/tools.ts +80 -33
- package/.bkit/core/engine/checks/before/context-fetch.ts +227 -33
- package/.bkit/core/engine/checks/before/env.ts +1 -1
- package/.bkit/core/engine/checks/before/index.ts +7 -0
- package/.bkit/core/engine/checks/before/repo.ts +40 -0
- package/.bkit/core/engine/checks/status-holes.ts +916 -123
- package/.bkit/core/engine/checks/tips/catalogue.ts +68 -17
- package/.bkit/core/engine/checks/tips/detect.ts +32 -0
- package/.bkit/core/engine/checks/tips/index.ts +47 -6
- package/.bkit/core/engine/cli.ts +238 -84
- package/.bkit/core/engine/commands/analysis.ts +128 -76
- package/.bkit/core/engine/commands/ci/ci.ts +363 -29
- package/.bkit/core/engine/commands/ci/pins.ts +172 -0
- package/.bkit/core/engine/commands/ci/push-gate.ts +223 -0
- package/.bkit/core/engine/commands/install/uninstall.ts +21 -27
- package/.bkit/core/engine/commands/install/upgrade.ts +32 -15
- package/.bkit/core/engine/commands/land.ts +126 -11
- package/.bkit/core/engine/commands/lib.ts +303 -6
- package/.bkit/core/engine/commands/registry.ts +6 -2
- package/.bkit/core/engine/commands/setup/setup.ts +1 -0
- package/.bkit/core/engine/commands/stats.ts +91 -35
- package/.bkit/core/engine/commands/workflow/branches.ts +1477 -117
- package/.bkit/core/engine/commands/workflow/challenge.ts +114 -39
- package/.bkit/core/engine/commands/workflow/completeness.ts +109 -0
- package/.bkit/core/engine/commands/workflow/context.ts +433 -55
- package/.bkit/core/engine/commands/workflow/counters.ts +244 -39
- package/.bkit/core/engine/commands/workflow/decisions.ts +61 -17
- package/.bkit/core/engine/commands/workflow/disk.ts +237 -55
- package/.bkit/core/engine/commands/workflow/fids.ts +19 -1
- package/.bkit/core/engine/commands/workflow/issues.ts +6 -0
- package/.bkit/core/engine/commands/workflow/ledgers.ts +458 -18
- package/.bkit/core/engine/commands/workflow/locks.ts +2 -0
- package/.bkit/core/engine/commands/workflow/plan-blocks.ts +124 -15
- package/.bkit/core/engine/commands/workflow/plans.ts +257 -0
- package/.bkit/core/engine/commands/workflow/prs.ts +12 -3
- package/.bkit/core/engine/commands/workflow/repo.ts +8 -0
- package/.bkit/core/engine/commands/workflow/roadmap.ts +56 -15
- package/.bkit/core/engine/commands/workflow/run-facts.ts +22 -31
- package/.bkit/core/engine/commands/workflow/runs.ts +652 -34
- package/.bkit/core/engine/commands/workflow/runtime.ts +3 -0
- package/.bkit/core/engine/commands/workflow/skills.ts +1 -0
- package/.bkit/core/engine/commands/workflow/status.ts +200 -17
- package/.bkit/core/engine/commands/workflow/step-table.ts +75 -33
- package/.bkit/core/engine/commands/workflow/steps.ts +635 -24
- package/.bkit/core/engine/commands/workflow/summary-findings.ts +177 -0
- package/.bkit/core/engine/commands/workflow/summary.ts +192 -46
- package/.bkit/core/engine/commands/workflow/tasks.ts +409 -0
- package/.bkit/core/engine/commands/workflow/test-blocks.ts +254 -0
- package/.bkit/core/engine/commands/workflow/tests-changed.ts +201 -0
- package/.bkit/core/engine/commands/workflow/tests.ts +1285 -356
- package/.bkit/core/engine/commands/workflow/workflows.ts +228 -43
- package/.bkit/core/engine/commands/workflow/worktrees.ts +389 -47
- package/.bkit/core/engine/install/cloud-env.ts +251 -121
- package/.bkit/core/engine/install/context-ci-workflow.ts +102 -0
- package/.bkit/core/engine/install/manifest.ts +60 -12
- package/.bkit/core/engine/install/modules/antigravity.ts +3 -2
- package/.bkit/core/engine/install/modules/cloud.ts +3 -3
- package/.bkit/core/engine/install/modules/codex.ts +15 -2
- package/.bkit/core/engine/install/modules/core.ts +956 -52
- package/.bkit/core/engine/install/modules/cursor.ts +6 -6
- package/.bkit/core/engine/install/modules/opencode.ts +16 -4
- package/.bkit/core/engine/install/template-notes.ts +130 -0
- package/.bkit/core/engine/install/version-file.ts +3 -2
- package/.bkit/core/engine/install/write.ts +5 -0
- package/.bkit/core/engine/ledgers/compact.ts +124 -0
- package/.bkit/core/engine/ledgers/journal.ts +291 -0
- package/.bkit/core/engine/ledgers/status.ts +160 -9
- package/.bkit/core/engine/ledgers/telemetry.ts +158 -21
- package/.bkit/core/engine/render/hosts.ts +114 -14
- package/.bkit/core/engine/shared/bkit-latest.ts +26 -4
- package/.bkit/core/engine/shared/dep-updater.ts +60 -0
- package/.bkit/core/engine/shared/egress.ts +1 -1
- package/.bkit/core/engine/shared/package-manager.ts +9 -0
- package/.bkit/core/engine/shared/settings.ts +92 -17
- package/.bkit/core/engine/shared/timeouts.ts +2 -0
- package/.bkit/core/engine/tools/cli-doc.ts +111 -0
- package/.bkit/core/engine/tools/grammar-density.ts +1 -1
- package/.bkit/core/engine/tools/instructions-sources.ts +15 -17
- package/.bkit/core/engine/tools/instructions.ts +94 -43
- package/.bkit/core/hooks/antigravity-hook.js +25 -10
- package/.bkit/core/hooks/gateguard-engine.js +53 -29
- package/.bkit/core/hooks/guardrails-shell.js +4 -4
- package/.bkit/core/hooks/guardrails-tiers.js +3 -3
- package/.bkit/core/hooks/preflight-linear.sh +9 -9
- package/.bkit/core/hooks/preflight-rules.sh +8 -0
- package/.bkit/core/hooks/readguard-engine.js +29 -10
- package/.bkit/core/hooks/session-rules-advisory.js +61 -0
- package/.bkit/core/prompts/bkit-audit-generate.md +187 -116
- package/.bkit/core/prompts/bkit-build-challenge-1.md +56 -37
- package/.bkit/core/prompts/bkit-build-challenge-2.md +65 -45
- package/.bkit/core/prompts/bkit-build-install-skills.md +38 -29
- package/.bkit/core/prompts/bkit-build-plan-convert.md +43 -32
- package/.bkit/core/prompts/bkit-build-plan-current.md +49 -39
- package/.bkit/core/prompts/bkit-build-plan-gaps-remediate.md +79 -55
- package/.bkit/core/prompts/bkit-build-plan-tasks.md +149 -0
- package/.bkit/core/prompts/bkit-build-plan-tests.md +120 -0
- package/.bkit/core/prompts/bkit-build-plan-verify.md +87 -50
- package/.bkit/core/prompts/bkit-debug-abort-handover.md +36 -35
- package/.bkit/core/prompts/bkit-debug-failure-diagnose.md +96 -64
- package/.bkit/core/prompts/bkit-debug-failure-remediate.md +54 -45
- package/.bkit/core/prompts/bkit-debug-failure-reproduce.md +42 -33
- package/.bkit/core/prompts/bkit-debug-failure-soak.md +53 -37
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-1.md +35 -30
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-2.md +34 -29
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-3.md +32 -28
- package/.bkit/core/prompts/bkit-design-generate.md +68 -52
- package/.bkit/core/prompts/bkit-design-organise.md +73 -35
- package/.bkit/core/prompts/bkit-design-review-1.md +58 -54
- package/.bkit/core/prompts/bkit-design-review-2.md +54 -50
- package/.bkit/core/prompts/bkit-design-spike.md +54 -47
- package/.bkit/core/prompts/bkit-design-update.md +42 -31
- package/.bkit/core/prompts/bkit-plan-generate.md +76 -58
- package/.bkit/core/prompts/bkit-plan-review-1.md +63 -49
- package/.bkit/core/prompts/bkit-plan-review-2.md +60 -48
- package/.bkit/core/prompts/bkit-plan-scope-check.md +53 -46
- package/.bkit/core/prompts/bkit-plan-update.md +70 -52
- package/.bkit/core/prompts/bkit-ship-conflict-resolve.md +82 -50
- package/.bkit/core/prompts/bkit-ship-deferred-sweep.md +80 -45
- package/.bkit/core/prompts/bkit-ship-update-docs.md +35 -29
- package/.bkit/core/prompts/bkit-utility-compress-markdown.md +55 -49
- package/.bkit/core/prompts/bkit-utility-update-decisions.md +76 -49
- package/.bkit/core/prompts/bkit-utility-update-deferrals.md +70 -41
- package/.bkit/core/prompts/bkit-utility-update-documentation.md +33 -30
- package/.bkit/core/prompts/bkit-utility-update-failures.md +73 -43
- package/.bkit/core/prompts/bkit-utility-update-knowledge.md +77 -50
- package/.bkit/core/prompts/bkit-utility-update-roadmap.md +77 -45
- package/.bkit/core/prompts/bkit-utility-update-tracker.md +144 -68
- package/.bkit/core/workflows/bkit-auto.md +53 -30
- package/.bkit/core/workflows/bkit-build.md +635 -304
- package/.bkit/core/workflows/bkit-debug.md +301 -148
- package/.bkit/core/workflows/bkit-design.md +167 -95
- package/.bkit/core/workflows/bkit-plan.md +172 -84
- package/.bkit/core/workflows/bkit-ship.md +299 -152
- package/.bkit/shared/.agents/plugins/bkit/hooks.json +10 -0
- package/.bkit/shared/.claude/bkit-cloud.md +12 -10
- package/.bkit/shared/.claude/settings.json +3 -0
- package/.bkit/shared/.codex/bkit-cloud.md +98 -0
- package/.bkit/shared/.codex/hooks.json +20 -0
- package/.bkit/shared/.cursor/bkit-cloud.md +3 -3
- package/.bkit/shared/.opencode/opencode.json +1 -1
- package/.bkit/shared/CLAUDE.md +5 -1
- package/.bkit/shared/GEMINI.md +4 -1
- package/.bkit/shared/docs/architecture.md +4 -7
- package/.bkit/shared/docs/data-schema.md +4 -7
- package/.bkit/shared/docs/deployments.md +239 -166
- package/.bkit/shared/docs/design.md +11 -12
- package/.bkit/shared/docs/noise.md +8 -7
- package/.bkit/shared/docs/observability.md +5 -8
- package/.bkit/shared/docs/security.md +35 -17
- package/.bkit/shared/docs/setup.md +29 -18
- package/.bkit/shared/docs/technology.md +3 -6
- package/.bkit/shared/docs/testing.md +282 -176
- package/.bkit/shared/gitignore +6 -2
- package/.bkit/templates/.claude/hooks/bkit-cloud-bootstrap.sh +15 -16
- package/.bkit/templates/.cursor/Dockerfile +12 -8
- package/.bkit/templates/.cursor/environment.json +1 -1
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-bug.yml +10 -8
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-change.yml +24 -12
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-idea.yml +3 -3
- package/.bkit/templates/.github/renovate.json5 +34 -6
- package/.bkit/templates/.github/workflows/bkit-ci.yml +53 -50
- package/.bkit/templates/.github/workflows/bkit-cloud.yml +13 -14
- package/.bkit/templates/.github/workflows/bkit-health.yml +97 -0
- package/.bkit/templates/.github/workflows/bkit-renovate.yml +5 -4
- package/.bkit/templates/.github/workflows/bkit-trivy.yml +5 -4
- package/.bkit/templates/.husky/pre-commit +2 -2
- package/.bkit/templates/.husky/pre-push +95 -35
- package/.bkit/templates/.prettierignore +6 -7
- package/.bkit/templates/.vale.ini +27 -27
- package/.bkit/templates/context/bkit-cloudrun.md +23 -15
- package/.bkit/templates/context/bkit-expo.md +70 -52
- package/.bkit/templates/context/bkit-github.md +93 -56
- package/.bkit/templates/context/bkit-inngest.md +21 -15
- package/.bkit/templates/context/bkit-maestro.md +22 -16
- package/.bkit/templates/context/bkit-playwright.md +23 -17
- package/.bkit/templates/context/bkit-resend.md +12 -10
- package/.bkit/templates/context/bkit-sentry.md +30 -20
- package/.bkit/templates/context/bkit-serena.md +8 -7
- package/.bkit/templates/context/bkit-stripe.md +14 -13
- package/.bkit/templates/context/bkit-supabase.md +16 -13
- package/.bkit/templates/context/bkit-tavily.md +14 -11
- package/.bkit/templates/context/bkit-vercel.md +27 -18
- package/.bkit/templates/docs/capabilities.md +15 -14
- package/.bkit/templates/scripts/ci/{check-accepted-divergences.mjs → bkit-check-accepted-divergences.mjs} +6 -3
- package/.bkit/templates/scripts/ci/{check-accepted-divergences.test.mjs → bkit-check-accepted-divergences.test.mjs} +3 -3
- package/.bkit/templates/scripts/ci/{check-capstone.mjs → bkit-check-capstone.mjs} +2 -2
- package/.bkit/templates/scripts/ci/{check-capstone.test.mjs → bkit-check-capstone.test.mjs} +4 -4
- package/.bkit/templates/scripts/ci/{check-dependency-audit.mjs → bkit-check-dependency-audit.mjs} +103 -18
- package/.bkit/templates/scripts/ci/{check-dependency-audit.test.mjs → bkit-check-dependency-audit.test.mjs} +31 -3
- package/.bkit/templates/scripts/ci/bkit-machine-lock.sh +387 -0
- package/.bkit/templates/scripts/ci/bkit-test-awake.mjs +75 -0
- package/.bkit/templates/scripts/ci/bkit-test-awake.test.mjs +160 -0
- package/.bkit/templates/scripts/ci/lanes/{resolve-push-base.mjs → bkit-resolve-push-base.mjs} +3 -3
- package/.bkit/templates/scripts/ci/lanes/{resolve-push-base.test.mjs → bkit-resolve-push-base.test.mjs} +4 -4
- package/.bkit/templates/scripts/cleanup/bkit-disk-prune.mjs +9 -1
- package/.bkit/templates/scripts/graphify/bkit-graph-drift.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graph-eager.py +3 -3
- package/.bkit/templates/scripts/graphify/bkit-graph-embed.py +63 -0
- package/.bkit/templates/scripts/graphify/bkit-graph-imports.py +2 -2
- package/.bkit/templates/scripts/graphify/bkit-graph-pre-push.sh +69 -18
- package/.bkit/templates/scripts/graphify/bkit-graph-repair.py +3 -3
- package/.bkit/templates/scripts/graphify/bkit-graph-surface.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graphify-graphml.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graphify-rollup.py +197 -126
- package/.bkit/templates/scripts/graphify/bkit-graphify.sh +64 -41
- package/.bkit/templates/scripts/slop/bkit-slop-type-suppression.mjs +23 -7
- package/.bkit/templates/slop-scan.config.json +9 -2
- package/CHANGELOG.md +119 -0
- package/README.md +250 -76
- package/dist/checks/after/auth.js +36 -33
- package/dist/checks/after/ci-yml.js +52 -2
- package/dist/checks/after/index.js +12 -4
- package/dist/checks/after/tools.js +73 -33
- package/dist/checks/before/context-fetch.js +218 -33
- package/dist/checks/before/env.js +1 -1
- package/dist/checks/before/index.js +6 -2
- package/dist/checks/before/repo.js +35 -1
- package/dist/checks/status-holes.js +739 -114
- package/dist/checks/tips/catalogue.js +59 -17
- package/dist/checks/tips/detect.js +30 -0
- package/dist/checks/tips/index.js +48 -6
- package/dist/cli.js +222 -84
- package/dist/commands/analysis.js +77 -53
- package/dist/commands/ci/ci.js +302 -25
- package/dist/commands/ci/pins.js +142 -0
- package/dist/commands/ci/push-gate.js +194 -0
- package/dist/commands/install/uninstall.js +23 -27
- package/dist/commands/install/upgrade.js +21 -11
- package/dist/commands/land.js +126 -10
- package/dist/commands/lib.js +244 -5
- package/dist/commands/registry.js +6 -2
- package/dist/commands/setup/setup.js +1 -0
- package/dist/commands/stats.js +69 -30
- package/dist/commands/workflow/branches.js +1232 -111
- package/dist/commands/workflow/challenge.js +90 -37
- package/dist/commands/workflow/completeness.js +97 -0
- package/dist/commands/workflow/context.js +371 -51
- package/dist/commands/workflow/counters.js +206 -40
- package/dist/commands/workflow/decisions.js +60 -17
- package/dist/commands/workflow/disk.js +213 -55
- package/dist/commands/workflow/fids.js +18 -1
- package/dist/commands/workflow/issues.js +6 -0
- package/dist/commands/workflow/ledgers.js +372 -21
- package/dist/commands/workflow/locks.js +2 -0
- package/dist/commands/workflow/plan-blocks.js +78 -14
- package/dist/commands/workflow/plans.js +205 -0
- package/dist/commands/workflow/prs.js +12 -3
- package/dist/commands/workflow/repo.js +6 -0
- package/dist/commands/workflow/roadmap.js +50 -17
- package/dist/commands/workflow/run-facts.js +23 -28
- package/dist/commands/workflow/runs.js +530 -33
- package/dist/commands/workflow/runtime.js +3 -0
- package/dist/commands/workflow/skills.js +1 -0
- package/dist/commands/workflow/status.js +166 -19
- package/dist/commands/workflow/step-table.js +61 -29
- package/dist/commands/workflow/steps.js +542 -26
- package/dist/commands/workflow/summary-findings.js +117 -0
- package/dist/commands/workflow/summary.js +154 -41
- package/dist/commands/workflow/tasks.js +378 -0
- package/dist/commands/workflow/test-blocks.js +231 -0
- package/dist/commands/workflow/tests-changed.js +172 -0
- package/dist/commands/workflow/tests.js +1013 -321
- package/dist/commands/workflow/workflows.js +191 -38
- package/dist/commands/workflow/worktrees.js +310 -42
- package/dist/install/cloud-env.js +233 -99
- package/dist/install/context-ci-workflow.js +85 -0
- package/dist/install/manifest.js +57 -12
- package/dist/install/modules/antigravity.js +3 -2
- package/dist/install/modules/cloud.js +3 -3
- package/dist/install/modules/codex.js +12 -2
- package/dist/install/modules/core.js +851 -53
- package/dist/install/modules/cursor.js +6 -6
- package/dist/install/modules/opencode.js +11 -5
- package/dist/install/template-notes.js +116 -0
- package/dist/install/version-file.js +3 -2
- package/dist/ledgers/compact.js +87 -0
- package/dist/ledgers/journal.js +233 -0
- package/dist/ledgers/status.js +115 -5
- package/dist/ledgers/telemetry.js +121 -17
- package/dist/render/hosts.js +100 -14
- package/dist/shared/bkit-latest.js +26 -4
- package/dist/shared/dep-updater.js +49 -0
- package/dist/shared/egress.js +1 -1
- package/dist/shared/package-manager.js +6 -0
- package/dist/shared/settings.js +59 -10
- package/dist/shared/timeouts.js +2 -0
- package/dist/tools/cli-doc.js +97 -0
- package/dist/tools/grammar-density.js +1 -1
- package/dist/tools/instructions-sources.js +15 -16
- package/dist/tools/instructions.js +83 -39
- package/package.json +58 -42
- package/.bkit/core/.bkit/lite.md +0 -73
- package/.bkit/core/engine/commands/workflow/workstreams.ts +0 -172
- package/.bkit/core/prompts/bkit-build-plan-execute.md +0 -124
- package/.bkit/templates/scripts/ci/bkit-local-stack-lock.sh +0 -194
- package/dist/commands/workflow/workstreams.js +0 -149
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Filler.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Hedging.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/LLMSlop.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Verbose.yml +0 -0
- /package/.bkit/templates/scripts/ci/{accepted-advisories.json → bkit-accepted-advisories.json} +0 -0
- /package/.bkit/templates/scripts/ci/{accepted-divergences.json → bkit-accepted-divergences.json} +0 -0
- /package/.bkit/templates/scripts/launchd/{launchd.env.example → bkit-launchd.env.example} +0 -0
- /package/.bkit/templates/scripts/launchd/{ai.bkit.launchd.plist → bkit-launchd.plist} +0 -0
|
@@ -1,49 +1,52 @@
|
|
|
1
|
-
<!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
|
|
2
|
-
|
|
3
1
|
<!-- bkit:start -->
|
|
4
2
|
|
|
5
3
|
{{BKIT_MANAGED_MARKER}}
|
|
6
4
|
|
|
5
|
+
<!-- Note: @bkit-audit-generate fills this template. The report lives at `docs/audits/{YYYYMM}/{slug}.md`, where `{YYYYMM}` is the UTC month of the run and `{slug}` is this file's name without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md`, where `{TIMESTAMP}` is the output of `date -u +%Y%m%dT%H%M%SZ`. NEVER overwrite an earlier report. The report is stamped with a `bkit:anchor` block that records the audited commit (`baseSha`) and branch (`baseRef`). NEVER write or edit that block by hand. -->
|
|
6
|
+
|
|
7
7
|
# Prompt Audit
|
|
8
8
|
|
|
9
9
|
> [!CAUTION]
|
|
10
|
-
> **Purpose:** Each prompt the product sends to a model
|
|
10
|
+
> **Purpose:** Each prompt the product sends to a model is clear, well structured, and written for the model that runs it, per that model maker's current prompting guide.
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> - Quote at most one line of prompt text per finding
|
|
15
|
-
> -
|
|
14
|
+
> - Quote at most one line of prompt text per finding.
|
|
15
|
+
> - NEVER copy personal data or a model completion into the report.
|
|
16
|
+
> - Never call a model to generate.
|
|
17
|
+
> - Count tokens with a local tokenizer or a provider count endpoint only.
|
|
16
18
|
|
|
17
19
|
## Universal Instructions
|
|
18
20
|
|
|
19
21
|
### Context & Goals
|
|
20
22
|
|
|
21
|
-
A prompt is written against one model maker's advice at one point in time. The model behind it changes, the advice changes, and the prompt does not.
|
|
23
|
+
A prompt is written against one model maker's advice at one point in time. The model behind it changes, the advice changes, and the prompt does not. Examples: rules pile up without reasons, emphasis written for an older model makes a newer one over-apply a rule, and the output format drifts from the parser that reads it. Nothing fails a test: the model still answers, a little worse. The failure this audit targets is a prompt that the current guide for its model says will degrade results, which nobody re-read after the model or the guide changed.
|
|
22
24
|
|
|
23
|
-
**Applies when:** the product sends a prompt to a language model at runtime (system prompt, message template, few-shot examples, tool or output-schema descriptions). Coding-agent instruction files and bkit-managed files are not product prompts.
|
|
25
|
+
**Applies when:** the product sends a prompt to a language model at runtime (system prompt, message template, few-shot examples, tool or output-schema descriptions). Coding-agent instruction files and bkit-managed files are not product prompts. **If** the product sends none → write one `N/A` row with its evidence, and never report the surface as "clean".
|
|
24
26
|
|
|
25
|
-
**Cadence:**
|
|
27
|
+
**Cadence:** Run once a quarter. Run an extra pass after a model change at any call site (`bkit-audit-models`), a new prompt set, or a maker guide update.
|
|
26
28
|
|
|
27
|
-
**Anchor Standards:** record URL, last-updated date or date read, and the model generation
|
|
29
|
+
**Anchor Standards:** For each guide, record the URL, the last-updated date or the date it was read, and the model generation it targets.
|
|
28
30
|
|
|
29
31
|
- Anthropic prompt engineering docs and the prompting best-practices page for current Claude models.
|
|
30
|
-
- OpenAI prompt engineering guide and the model-specific prompting guide for each model in use (e.g
|
|
32
|
+
- OpenAI prompt engineering guide and the model-specific prompting guide for each model in use (e.g., the cookbook guide per model generation).
|
|
31
33
|
- Google Gemini API prompt design strategies.
|
|
32
|
-
- Every other maker whose model runs a prompt (e.g
|
|
34
|
+
- Every other maker whose model runs a prompt (e.g., xAI, Mistral, Meta, DeepSeek, Qwen): its own prompting guide, or `Not Checked` when it publishes none; its model parameter docs either way.
|
|
33
35
|
- Each maker's tool-use and structured-output docs, for tool and schema descriptions.
|
|
36
|
+
- bkit's authoring rules: `.bkit/authoring.md` § All Agent-Read Files, § Prompts and Workflows and § Prompts Only, read at the installed bkit version (`.bkit/version`).
|
|
34
37
|
|
|
35
|
-
Guides disagree. The guide for the model that runs the prompt wins. A prompt that runs on models from more than one maker (fallback chain, router, several hosts)
|
|
38
|
+
Guides disagree. The guide for the model that runs the prompt wins. A prompt that runs on models from more than one maker (fallback chain, router, several hosts) follows the current guides of the makers whose models run it day to day. A fallback model's guide counts only where it does not conflict with them.
|
|
36
39
|
|
|
37
40
|
### Scope
|
|
38
41
|
|
|
39
|
-
-
|
|
40
|
-
-
|
|
41
|
-
-
|
|
42
|
-
-
|
|
43
|
-
-
|
|
44
|
-
-
|
|
45
|
-
-
|
|
46
|
-
- Deliberate departures from a guide
|
|
42
|
+
- Read every runtime prompt: system prompt, message template, few-shot set, prefill, tool descriptions and output-schema descriptions.
|
|
43
|
+
- Name the model and the guide for each prompt.
|
|
44
|
+
- Check clarity: the task, success, the context and reason behind each rule, definitions, contradictions and dead instructions.
|
|
45
|
+
- Check structure: sections, delimiters, ordering, examples, and the output format against its parser.
|
|
46
|
+
- Check model fit: emphasis, reasoning and verbosity controls, sampling parameters, prefill, and workarounds from an older model.
|
|
47
|
+
- Check tool and output-schema descriptions.
|
|
48
|
+
- Measure token cost, duplication and cache-friendly ordering.
|
|
49
|
+
- Deliberate departures from a guide are recorded once, and are not re-raised.
|
|
47
50
|
|
|
48
51
|
### Out of Scope
|
|
49
52
|
|
|
@@ -52,7 +55,7 @@ Guides disagree. The guide for the model that runs the prompt wins. A prompt tha
|
|
|
52
55
|
| --- | --- | --- |
|
|
53
56
|
| Injection boundary, tool capability, output validators, call limits, job-scope bound, evals / golden sets | LLM sweep | `bkit-audit-llm` |
|
|
54
57
|
| Model choice, deprecation, catalogue, fallback-chain membership | Model sweep | `bkit-audit-models` |
|
|
55
|
-
| Coding-agent instruction files (e.g
|
|
58
|
+
| Coding-agent instruction files (e.g., `AGENTS.md`, `CLAUDE.md`, agent definitions, skills): loading, permissions, untrusted input | Agent config sweep | `bkit-audit-agents` |
|
|
56
59
|
| Contradictions between instruction files; always-loaded size | Documentation sweep | `bkit-audit-documentation` |
|
|
57
60
|
| bkit-managed prompts and templates (`.bkit/`, `bkit-*` host agent mirrors, `bkit:start` / `bkit:end` blocks) | Fixed upstream in bkit | bkit's own run of this audit |
|
|
58
61
|
| Live hostile probes | Red team | `bkit-audit-red-team` |
|
|
@@ -63,41 +66,44 @@ Guides disagree. The guide for the model that runs the prompt wins. A prompt tha
|
|
|
63
66
|
|
|
64
67
|
#### Core
|
|
65
68
|
|
|
66
|
-
- [ ] **Permissions:** blocked
|
|
67
|
-
- [ ] **Code Search:** Steps 2, 4-9 — text search and language server for call sites (e.g
|
|
68
|
-
- [ ] **Web:** Steps 3, 10 — each maker's prompting guide (e.g
|
|
69
|
+
- [ ] **Permissions:** every blocked call is recorded.
|
|
70
|
+
- [ ] **Code Search:** Steps 2, 4-9 — text search, and the language server for call sites (e.g., ripgrep, Serena, `tsserver`). Prompts are often markdown or string literals, and tool descriptions are often strings in schema files. Language-server hits under-report. **If** a result is empty → treat it as inconclusive, corroborate it with text search, and name the tool. Fallback: text search only, and name the tool for each claim.
|
|
71
|
+
- [ ] **Web:** Steps 3, 10 — each maker's prompting guide (e.g., WebFetch, WebSearch). Fallback: record guide practice as unread; Steps 4-7 cite the previous report's guide version and say so.
|
|
69
72
|
|
|
70
73
|
#### Conditional
|
|
71
74
|
|
|
72
|
-
- [ ] **LLM Report:**
|
|
73
|
-
- [ ] **
|
|
75
|
+
- [ ] **LLM Report:** **If** a `bkit-audit-llm` report exists → Steps 2, 9 — its `Inventory` and `Evals`. Fallback: build the inventory from code, and mark eval coverage `Not Checked`.
|
|
76
|
+
- [ ] **Authoring Rules:** **If** `.bkit/authoring.md` exists → Step 5 — its principles. Fallback: mark the authoring rows `Not Checked`, never as a pass.
|
|
77
|
+
- [ ] **Token Counter:** Step 8 — a local tokenizer or a provider count endpoint (no generation). Fallback: use character count ÷ 4, labelled as an estimate.
|
|
74
78
|
|
|
75
79
|
### 🔒 Gate 1: Prerequisites Check
|
|
76
80
|
|
|
77
|
-
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
78
|
-
- [ ] **Gate 1 — Verified:**
|
|
81
|
+
- [ ] **Gaps named:** every missing item names its fallback and its `Not Checked` entry.
|
|
82
|
+
- [ ] **Gate 1 — Verified:** every item above, every Core item and every applicable Conditional item is checked.
|
|
79
83
|
|
|
80
84
|
### Probe Steps
|
|
81
85
|
|
|
82
|
-
- [ ] **Step 1 — Previous Run:** **Done when:**
|
|
83
|
-
- [ ] **Step 2 — Inventory:** Start from code: every model call site outside tests
|
|
84
|
-
- [ ] **Step 3 — Guides:**
|
|
85
|
-
- [ ] **Step 4 — Clarity:**
|
|
86
|
-
- [ ] **Step 5 — Structure:**
|
|
87
|
-
- [ ] **Step 6 — Model Fit:**
|
|
88
|
-
- [ ] **Step 7 — Tools and Schemas:**
|
|
89
|
-
- [ ] **Step 8 — Efficiency:**
|
|
90
|
-
- [ ] **Step 9 — Deliberate Choices:**
|
|
91
|
-
- [ ] **Step 10 — Standards:** Confirm current version
|
|
92
|
-
- [ ] **Step 11 — Anything Else:** (a)
|
|
86
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** `Previous Run` holds the closure rate, or the run is marked baseline.
|
|
87
|
+
- [ ] **Step 2 — Inventory:** Start from the code: every model call site outside tests, with the newest `bkit-audit-llm` `Inventory` as a cross-check, not as the source. For each prompt, record: the file and call sites · the model(s) that run it — every model in the fallback chain, resolved at the call site (choice and lifecycle belong to `bkit-audit-models`, and are not scored here) · the maker guide that applies · the task (extract, classify, write, converse, tool-calling) · the output consumer (parser, schema, human) · the eval set covering it (`bkit-audit-llm` `Evals`) · the last-changed commit · whether the model or guide changed since. Exclude bkit-managed files and coding-agent instruction files. Diff against the previous run, and read each new prompt or new model first. **Done when:** `Inventory` has a row per prompt, or `N/A` is recorded → **STOP**.
|
|
88
|
+
- [ ] **Step 3 — Guides:** For each maker in `Inventory`, read the current prompting guide and the model-specific page for each model generation in use. Record the URL, the last-updated date or the date read, and the generation targeted. Build `Guidance`: practice · source · makers it applies to · which conflict. **If** makers conflict → note which prompts run on both. **If** there is no guide, **or** its URL is dead → still read the maker's model parameter docs per generation (e.g., reasoning-effort values, rejected sampling parameters), so the Step 6 parameter checks run. **If** the parameter docs are missing → mark that generation's parameter checks `Not Checked`. **If** a guide is unread → mark it `Not Checked`, and use the previous report's reading, labelled, for that maker in Steps 4-7. **Done when:** `Guidance` has a row per practice used in Steps 4-7, and every row is sourced.
|
|
89
|
+
- [ ] **Step 4 — Clarity:** For each prompt, check that: the task and what a good answer looks like are stated up front; the audience and context are given; each constraint carries its reason where the guide asks for one; it says what to do, not only what to avoid; the terms the output depends on are defined (e.g., date format, "recent", allowed labels, "out-of-scope" in a redirect rule); a conversational prompt states its job and the reply to off-topic asks (wording only — enforcement belongs to `bkit-audit-llm` `Job Scope`); there is no contradiction within the prompt, or between the system prompt and the template; and no dead instruction names a removed field, tool or format. **Done when:** `Clarity` has a row per prompt: a verdict per check, with each failure cited.
|
|
90
|
+
- [ ] **Step 5 — Structure:** For each prompt, check that: sections and inputs are delimited the way the guide recommends (e.g., XML tags, markdown headings); instructions are kept apart from inserted data (layout only — injection belongs to `bkit-audit-llm`); long inputs sit where the guide says (e.g., documents first, question last); the output format is stated and matches its consumer, so read the parser or schema and compare field names, types and enums, and mark a human consumer `N/A`; every parsed field has an instruction (or schema description) that sets it and says what it counts; every field the instructions ask for appears in the output format; a structured-output or tool schema is used where the provider supports it and the guide prefers it; and examples are present where format or judgment matters, varied, consistent with the instructions, and not copied verbatim into output. Then check each prompt against the principles of `.bkit/authoring.md`, one row per principle. **Done when:** `Structure` has a row per prompt: a verdict per check, with the format-vs-parser and field-to-instruction diffs recorded, and a row per `.bkit/authoring.md` principle.
|
|
91
|
+
- [ ] **Step 6 — Model Fit:** For each prompt x model running it, check against that maker's current guide. Check the emphasis level (all-caps, "CRITICAL", "must", alarm emoji), because some current guides say forceful wording now makes the model over-apply a rule. First sort each capitalised absolute (or bold / emoji emphasis), by the rule it governs, into invariant (safety, permission, required field, irreversible action) or judgement call, and judge only the judgement calls. Then check: reasoning controls (the effort or thinking setting at the call site, against a hand-written "think step by step" on a model with built-in reasoning); verbosity and length controls; sampling parameters the model ignores or rejects (e.g., temperature on a reasoning model); prefill used where the model or API no longer supports it; and a workaround written for an older model that is still present after a model change. **If** a prompt runs on models from several makers → it uses a practice every guide supports, or it is a finding. **Done when:** `Model Fit` has a row per prompt x model: check · guide line · verdict.
|
|
92
|
+
- [ ] **Step 7 — Tools and Schemas:** For each tool or output schema a prompt offers, check that: the description says what it does, when to use it and when not to; each parameter's meaning, unit, format and allowed values are given; the names are unambiguous; no two tools are ones a model could confuse; and error text returned to the model says how to recover. Capability and bounds belong to `bkit-audit-llm`. **If** there are no tools or schemas → skip with evidence. **Done when:** `Tools and Schemas` has a row per tool / schema, or the skip.
|
|
93
|
+
- [ ] **Step 8 — Efficiency:** For each prompt, record: the token count of the fixed part (naming the tool); text duplicated across prompts that one shared fragment could hold; whether fixed content comes before variable content where the provider caches prompt prefixes; and text that no instruction or example needs. Spend beyond the per-call count belongs to `bkit-audit-cost`. **Done when:** `Efficiency` has a row per prompt: tokens · duplication · cache order · dead text.
|
|
94
|
+
- [ ] **Step 9 — Deliberate Choices:** For every departure from a guide found in Steps 4-8, check `docs/decisions.jsonl`, prompt comments, and the previous reports' `Deliberate Choices`. **If** it is recorded with a reason → list it once, not as a finding. Re-raise it only when the guide changed after that reason was written. **If** it is unrecorded → mark it adopt (a finding), reject (saying why, and proposing a `docs/decisions.jsonl` row under `Process Improvements`) or not applicable. Each finding names the eval that would confirm the rewrite (`bkit-audit-llm` `Evals`). **If** there is none → write `unverified`, and propose the golden case to `bkit-audit-llm`. **Done when:** `Deliberate Choices` has a row per departure: practice · decision source or verdict.
|
|
95
|
+
- [ ] **Step 10 — Standards:** Confirm the current version of each anchor guide, and read what changed since the previous report's version. Give `Standards Coverage` one row per guide recommendation that applies to a prompt in `Inventory`, each marked checked (naming the step), not applicable (saying why) or finding. Mark each recommended practice the project does not apply as adopt (with a proposed step), reject (saying why) or not applicable. **Done when:** no row is empty.
|
|
96
|
+
- [ ] **Step 11 — Anything Else:** (a) Give a one-line verdict on each change since the previous report's commit that touches prompts, call-site model settings, tool or schema descriptions, or parsers of model output. (b) In a bounded pass of about 30 minutes, note what a senior prompt engineer would flag that no step asks about. Read the maker changelogs and model release notes since the previous run, and ask what no step would catch. **Done when:** `Beyond the Steps` holds both passes, or says `none`. **If** one class of issue is seen on 2 runs → propose a step for it in `Process Improvements`.
|
|
93
97
|
|
|
94
98
|
### 🔒 Gate 2: Probes Check
|
|
95
99
|
|
|
96
|
-
- [ ] **Steps complete:** every step `[x]`, or
|
|
97
|
-
- [ ] **Evidence:** every finding cites `file:line`, the guide line it departs from (URL
|
|
98
|
-
- [ ] **No payloads:** at most one line of prompt text
|
|
99
|
-
- [ ] **
|
|
100
|
-
- [ ] **
|
|
100
|
+
- [ ] **Steps complete:** every step is `[x]`, or skipped with evidence that its surface is absent.
|
|
101
|
+
- [ ] **Evidence:** every finding cites `file:line`, the guide line it departs from (URL and date read), and the tool behind it.
|
|
102
|
+
- [ ] **No payloads:** each finding quotes at most one line of prompt text.
|
|
103
|
+
- [ ] **No personal data:** no personal data or model completion is in the report.
|
|
104
|
+
- [ ] **Read-only:** no model generation call was made.
|
|
105
|
+
- [ ] **Prompts unchanged:** no prompt or config changed.
|
|
106
|
+
- [ ] **Gate 2 — Verified:** every item above holds.
|
|
101
107
|
|
|
102
108
|
## Findings
|
|
103
109
|
|
|
@@ -109,14 +115,14 @@ Guides disagree. The guide for the model that runs the prompt wins. A prompt tha
|
|
|
109
115
|
| `MEDIUM` | Rule with no reason where the guide asks for one; examples missing where format matters, or contradicting instructions; prompt on several makers' models relying on one maker's behaviour; fixed content after variable content where the provider caches; text duplicated across prompts; dead instruction; departure from a guide not recorded |
|
|
110
116
|
| `LOW` | Wording; token waste with no behaviour change; non-write tool description gaps |
|
|
111
117
|
|
|
112
|
-
Within a severity
|
|
118
|
+
Within a severity, list write paths first, then prompts whose model changed. A departure recorded in `docs/decisions.jsonl` with a reason is not a finding.
|
|
113
119
|
|
|
114
120
|
<!-- prettier-ignore -->
|
|
115
121
|
| ID | Severity | Prompt | Location | Evidence | Expected | Verify |
|
|
116
122
|
| --- | --- | --- | --- | --- | --- | --- |
|
|
117
123
|
| PENDING | PENDING | PENDING | PENDING | PENDING | PENDING | PENDING |
|
|
118
124
|
|
|
119
|
-
|
|
125
|
+
**If** the run finds nothing → write one row: `None`. `Verify` = the eval that confirms the fix, or `unverified`.
|
|
120
126
|
|
|
121
127
|
## Report
|
|
122
128
|
|
|
@@ -189,14 +195,17 @@ PENDING
|
|
|
189
195
|
|
|
190
196
|
### 🔒 Gate 3: Report Check
|
|
191
197
|
|
|
192
|
-
- [ ] **No placeholders:** no `PENDING` left
|
|
193
|
-
- [ ] **Summary:**
|
|
194
|
-
- [ ] **
|
|
195
|
-
- [ ] **
|
|
196
|
-
- [ ] **
|
|
197
|
-
- [ ] **
|
|
198
|
-
- [ ] **
|
|
199
|
-
- [ ] **
|
|
198
|
+
- [ ] **No placeholders:** no `PENDING` is left.
|
|
199
|
+
- [ ] **Summary:** `Summary` gives the count per severity.
|
|
200
|
+
- [ ] **Prompts read:** `Summary` names the prompts read.
|
|
201
|
+
- [ ] **Guide versions:** `Summary` gives the guide versions read.
|
|
202
|
+
- [ ] **Series:** `Inventory` and `Deliberate Choices` are diffed against the previous report's, or the run is marked baseline.
|
|
203
|
+
- [ ] **Guides:** every `Model Fit` verdict names the guide and the date it was judged against.
|
|
204
|
+
- [ ] **Writes:** this run changed only this report, the template's `Project Instructions` and its own `docs/failures.jsonl` rows.
|
|
205
|
+
- [ ] **Process Improvements:** `Universal` and `Project` are both filled.
|
|
206
|
+
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID is in exactly one `Suggested Issues` row.
|
|
207
|
+
- [ ] **Next Step:** every `Suggested Issues` row has a `Next Step`.
|
|
208
|
+
- [ ] **Gate 3 — Verified:** every item above holds.
|
|
200
209
|
|
|
201
210
|
<!-- bkit:end -->
|
|
202
211
|
|
|
@@ -1,33 +1,40 @@
|
|
|
1
|
-
<!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
|
|
2
|
-
|
|
3
1
|
<!-- bkit:start -->
|
|
4
2
|
|
|
5
3
|
{{BKIT_MANAGED_MARKER}}
|
|
6
4
|
|
|
5
|
+
<!-- Note: @bkit-audit-generate fills this template. The report lives at `docs/audits/{YYYYMM}/{slug}.md`, where `{YYYYMM}` is the UTC month of the run and `{slug}` is this file's name without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md`, where `{TIMESTAMP}` is the output of `date -u +%Y%m%dT%H%M%SZ`. NEVER overwrite an earlier report. The report is stamped with a `bkit:anchor` block that records the audited commit (`baseSha`) and branch (`baseRef`). NEVER write or edit that block by hand. -->
|
|
6
|
+
|
|
7
7
|
# Red Team Audit
|
|
8
8
|
|
|
9
9
|
> [!CAUTION]
|
|
10
|
-
> **Purpose:** Active adversarial exploitation of a non-production deployment: one hostile request per probe, the request and response kept, then
|
|
10
|
+
> **Purpose:** Active adversarial exploitation of a non-production deployment: one hostile request per probe, with the request and response kept, then cleanup.
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> -
|
|
15
|
-
> -
|
|
16
|
-
> -
|
|
14
|
+
> - Probe a non-production deployment only.
|
|
15
|
+
> - NEVER probe production, the release branch, or a third party's own platform beyond the calls the app already makes.
|
|
16
|
+
> - Use synthetic identities only.
|
|
17
|
+
> - Creating and deleting those identities is the only data write, and they are deleted before the report finishes.
|
|
18
|
+
> - Send one request per probe.
|
|
19
|
+
> - Send no load, no volume fuzzing, and no spend past a single cap.
|
|
20
|
+
> - Stop at proof, and do not pivot.
|
|
21
|
+
> - NEVER copy a working token into the report.
|
|
22
|
+
> - NEVER copy a personal-data value into the report.
|
|
23
|
+
> - Record the route, the input shape and the response.
|
|
17
24
|
|
|
18
25
|
## Universal Instructions
|
|
19
26
|
|
|
20
27
|
### Context & Goals
|
|
21
28
|
|
|
22
|
-
Three
|
|
29
|
+
Three kinds of failure do not show up in a source read. A row-security denial looks like an empty result. A middleware check is not visible in the route file. A prompt injection shows only when the model runs. The failure this audit targets is a control that holds in the source and fails on a request. Examples: a mixed tenant-id list is clamped to the caller's rows instead of refused, a revoked token is still accepted, or a message makes the model delete items it does not mention. The static suspicions come from `bkit-audit-security` and `bkit-audit-llm`, and this audit sends the request.
|
|
23
30
|
|
|
24
|
-
**Applies when:** non-production deployment can be called, and the system has auth, tenancy, or a public free-text model surface.
|
|
31
|
+
**Applies when:** a non-production deployment can be called, and the system has auth, tenancy, or a public free-text model surface. **If** it has none → write one `N/A` row with its evidence, and never report the surface as "clean".
|
|
25
32
|
|
|
26
|
-
**Cadence:** once before launch, then
|
|
33
|
+
**Cadence:** Run once before launch, then once a quarter. Run an extra pass after any change to authentication, a route that takes an id, the inbound pipeline, or a machine-client tool.
|
|
27
34
|
|
|
28
35
|
**Anchor Standards:**
|
|
29
36
|
|
|
30
|
-
- OWASP Web Security Testing Guide (e.g
|
|
37
|
+
- OWASP Web Security Testing Guide (e.g., WSTG v4.2; record if a newer stable release exists).
|
|
31
38
|
- Penetration Testing Execution Standard (PTES): pre-engagement, intelligence gathering, threat modelling, vulnerability analysis, exploitation, post-exploitation, reporting.
|
|
32
39
|
- When a probe drives a model: OWASP GenAI Red Teaming Guide, OWASP AI Testing Guide, MITRE ATLAS.
|
|
33
40
|
- OWASP Mobile Application Security Testing Guide (MASTG), when the native build is in scope.
|
|
@@ -35,17 +42,20 @@ Three shapes do not show up in a source read: row-security denial looks like emp
|
|
|
35
42
|
|
|
36
43
|
### Scope
|
|
37
44
|
|
|
38
|
-
-
|
|
39
|
-
-
|
|
40
|
-
- Retest
|
|
41
|
-
-
|
|
42
|
-
-
|
|
43
|
-
-
|
|
44
|
-
-
|
|
45
|
-
-
|
|
46
|
-
-
|
|
47
|
-
-
|
|
48
|
-
-
|
|
45
|
+
- Write the in-list and the out-list for this run, and what each platform's testing page allows.
|
|
46
|
+
- Diff the attack surface against the previous report, and give everything new at least one probe.
|
|
47
|
+
- Retest every earlier break whose fix has landed.
|
|
48
|
+
- Probe cross-tenant reads through the app.
|
|
49
|
+
- Probe cross-tenant reads separately through the data API with the client public key.
|
|
50
|
+
- Probe acting above a role.
|
|
51
|
+
- Probe every way a removed member could get back in.
|
|
52
|
+
- Probe authentication, session end and off-site redirects.
|
|
53
|
+
- Probe hostile inbound content: unauthorised writes, leaks, internal fetches, impersonation and caps.
|
|
54
|
+
- Probe machine-client tool misuse.
|
|
55
|
+
- Probe injection at the render, search and webhook boundaries.
|
|
56
|
+
- Cross each rate-limit and cost cap once.
|
|
57
|
+
- Probe off-topic use of public model surfaces.
|
|
58
|
+
- Clean up every identity this run created.
|
|
49
59
|
|
|
50
60
|
### Out of Scope
|
|
51
61
|
|
|
@@ -61,65 +71,107 @@ Three shapes do not show up in a source read: row-security denial looks like emp
|
|
|
61
71
|
|
|
62
72
|
### Verdicts
|
|
63
73
|
|
|
64
|
-
|
|
74
|
+
Each probe gets one verdict, on its `Probes` row.
|
|
65
75
|
|
|
66
|
-
- **HELD:** control did what it should.
|
|
67
|
-
- **BROKE:** control failed
|
|
68
|
-
- **INCONCLUSIVE:** blocked, skipped, or no clear answer. Say why.
|
|
69
|
-
- **FIXED** / **STILL BROKE:** retest of earlier `BROKE
|
|
70
|
-
- **OTHER:** fits
|
|
76
|
+
- **HELD:** the control did what it should. A held probe is a result, not a blank. **If** no test proves it → route it to `bkit-audit-testing`. It is not a finding.
|
|
77
|
+
- **BROKE:** the control failed, and that is a finding. Record the blast radius by reasoning, not by pivoting.
|
|
78
|
+
- **INCONCLUSIVE:** the probe was blocked, skipped, or gave no clear answer. Say why. It stays in `Probes`, because a skipped check otherwise reads as passed. **If** a `BROKE` has no kept response → mark it `INCONCLUSIVE` until a later run reproduces it.
|
|
79
|
+
- **FIXED** / **STILL BROKE:** use these only for a retest of an earlier `BROKE`, with the fix commit.
|
|
80
|
+
- **OTHER:** it fits no other verdict. Say why. **If** the same `OTHER` shows up on 2 runs → propose a new verdict in `Process Improvements`.
|
|
71
81
|
|
|
72
82
|
### Pre-Flight / Dependencies
|
|
73
83
|
|
|
74
84
|
#### Core
|
|
75
85
|
|
|
76
|
-
- [ ] **Permissions:** blocked
|
|
77
|
-
- [ ] **Target:** Steps 2-13 — non-production deployment of this commit, resolved by commit SHA
|
|
78
|
-
- [ ] **Identities:** Steps 2, 5-12, 15 — create and delete synthetic users and tenants,
|
|
79
|
-
- [ ] **Web:** Steps 2, 14 — each platform's penetration-testing page
|
|
86
|
+
- [ ] **Permissions:** every blocked call is recorded.
|
|
87
|
+
- [ ] **Target:** Steps 2-13 — a non-production deployment of this commit, resolved by commit SHA and confirmed serving. Fallback: none, so the probes stay `INCONCLUSIVE`. NEVER point them at production.
|
|
88
|
+
- [ ] **Identities:** Steps 2, 5-12, 15 — create and delete synthetic users and tenants, with a guard so that a non-synthetic identity cannot be deleted. Fallback: none. Do not probe with real accounts.
|
|
89
|
+
- [ ] **Web:** Steps 2, 14 — read each platform's penetration-testing page and the anchor standards. Fallback: **If** a platform's policy is unread → send no probes, and record why.
|
|
80
90
|
|
|
81
91
|
#### Conditional
|
|
82
92
|
|
|
83
|
-
- [ ] **Data API:**
|
|
84
|
-
- [ ] **Inbound Mail:**
|
|
85
|
-
- [ ] **Machine Client:**
|
|
86
|
-
- [ ] **Native Build:**
|
|
87
|
-
- [ ] **Model Spend:**
|
|
93
|
+
- [ ] **Data API:** **If** the client public key plus a user token can call the data API directly → Step 6 — e.g., PostgREST, Storage, Realtime. Fallback: probe through the app routes only, and mark the direct path `Not Checked`.
|
|
94
|
+
- [ ] **Inbound Mail:** **If** the synthetic tenant has an inbound address → Step 10. Fallback: skip the step, and mark it `Not Checked`.
|
|
95
|
+
- [ ] **Machine Client:** **If** a token-gated tool endpoint exists → Steps 2, 11. Fallback: skip with evidence.
|
|
96
|
+
- [ ] **Native Build:** **If** this run includes the native client → Step 8 — cached data and sync on a device or emulator. Fallback: test the web only, and mark the device path `Not Checked`.
|
|
97
|
+
- [ ] **Model Spend:** **If** a probe drives a model → Steps 10, 13 — a budget for one crossing of one cap. Fallback: do not send the cost probe, and mark it `INCONCLUSIVE`.
|
|
88
98
|
|
|
89
99
|
### 🔒 Gate 1: Prerequisites Check
|
|
90
100
|
|
|
91
|
-
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
92
|
-
- [ ] **Gate 1 — Verified:**
|
|
101
|
+
- [ ] **Gaps named:** every missing item names its fallback and its `Not Checked` entry.
|
|
102
|
+
- [ ] **Gate 1 — Verified:** every item above, every Core item and every applicable Conditional item is checked.
|
|
93
103
|
|
|
94
104
|
### Probe Steps
|
|
95
105
|
|
|
96
|
-
- [ ] **Step 1 — Previous Run:** **Done when:**
|
|
97
|
-
- [ ] **Step 2 — Engage:** Record commit and non-production URL deployed for it
|
|
98
|
-
-
|
|
99
|
-
-
|
|
100
|
-
-
|
|
101
|
-
-
|
|
102
|
-
-
|
|
103
|
-
-
|
|
104
|
-
-
|
|
105
|
-
-
|
|
106
|
-
-
|
|
107
|
-
-
|
|
108
|
-
-
|
|
109
|
-
-
|
|
110
|
-
-
|
|
106
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** `Previous Run` holds the closure rate, or the run is marked baseline.
|
|
107
|
+
- [ ] **Step 2 — Engage:** Record the commit and the non-production URL deployed for it.
|
|
108
|
+
- Use the deployment the live lane resolves for that SHA, not a branch alias, which may be stale.
|
|
109
|
+
- Confirm it is serving.
|
|
110
|
+
- Write the scope in `Scope`:
|
|
111
|
+
- The in-list covers: the non-production web URL (e.g., preview deploy, staging), the API host, the machine-client endpoint, the inbound mail domain, the data API via the client public key, the background-job serve endpoint, and the native build if it is tested.
|
|
112
|
+
- The out-list covers: production, the release branch, the persistent demo tenant, every third party's own platform, and anything not on the in-list.
|
|
113
|
+
- Read the current penetration-testing page of each platform the probes touch: the app host, the database, and the native vendor when the native build is in scope.
|
|
114
|
+
- Record what each allows and the plan tier.
|
|
115
|
+
- **If** a page forbids a probe → do not send it, and mark it `INCONCLUSIVE`.
|
|
116
|
+
- Create two synthetic tenants, attacker and victim, each with the roles the product distinguishes.
|
|
117
|
+
- Issue a machine-client token for each owner where the product has one, on a plan the feature allows.
|
|
118
|
+
- Seed the victim with a few records, one recurring record and an attachment.
|
|
119
|
+
- Do not touch the persistent demo tenant.
|
|
120
|
+
- Read the newest `bkit-audit-security` and `bkit-audit-llm` reports.
|
|
121
|
+
- Give each finding they could not prove a probe.
|
|
122
|
+
- Give each of the model audit's prompt-only rules a probe.
|
|
123
|
+
- Give each trust boundary the threat model names at least one probe.
|
|
124
|
+
- **If** there is no threat model → say so in `Scope`.
|
|
125
|
+
- Before a destructive probe, confirm that no other live test holds the target database, and record the window. Do not wait for a named approver.
|
|
126
|
+
- **If** a later probe degrades the target, touches non-synthetic data, or reaches a third party → stop the remaining probes, and record the stop in `Scope`.
|
|
127
|
+
- Each later probe gets a `Probes` row with: the request with tokens redacted, the response status, the relevant body and the verdict. A `BROKE` row adds the blast radius by reasoning, not by pivoting.
|
|
128
|
+
|
|
129
|
+
**Done when:** `Scope` holds the URL, the commit, the in-list, the out-list, the platform rules, the plan tier, the identities (no tokens) and the suspicion list.
|
|
130
|
+
|
|
131
|
+
- [ ] **Step 3 — Surface:** List the attack surface at this commit: routes, server actions, machine-client tools, webhooks, and the response headers of each in-scope origin. Diff it against the previous report's list. Give everything new at least one probe. **Done when:** `Surface` holds the list and the diff.
|
|
132
|
+
- [ ] **Step 4 — Retest:** Re-run every earlier `BROKE` whose fix has landed, from its recorded reproduction. Mark it `FIXED` or `STILL BROKE`, with the fix commit. Confirm that the regression test the prior run named exists. **If** it does not → route it to `bkit-audit-testing`. **Done when:** `Retests` has a row per prior break, or says `none`.
|
|
133
|
+
- [ ] **Step 5 — Cross-Tenant:** As the attacker, request the victim's records by id on every route that takes one: items, attachments, download URLs, members, invitations and machine-client read tools. A mixed list of own and foreign ids must be refused, not clamped to the attacker's rows. Expected: denial or not-found, and no victim field in any part of the response, errors included. **Done when:** `Probes` has a row per route, in the Step 2 row shape.
|
|
134
|
+
- [ ] **Step 6 — Direct API:** Step 5 goes through the app, and an attacker can skip the app. With the attacker's token and the client public key, ask the data API for the victim's tenant: each tenant-scoped table, privileged routines, GraphQL if enabled, file-storage list and download, and realtime subscribe. Expected: no victim row, object or event. **If** a denial looks like an empty result → send the same request once as the victim, to show it would have returned data. **Done when:** `Probes` has a row per surface, including the victim-control request, or the step is skipped with evidence that the direct API is not exposed.
|
|
135
|
+
- [ ] **Step 7 — Roles:** Inside the attacker tenant, have the low role try what only a higher role may do: invite, remove a member, change the plan, issue a machine-client token, delete a record that is not theirs, and delete the tenant. Check each against the role matrix. **Done when:** `Probes` has a row per operation.
|
|
136
|
+
- [ ] **Step 8 — Removed:** The realistic attacker is a former tenant member. Remove the attacker's second member, then try every way back in: the web session, native cached data and sync, a machine-client token they issued, invitation links they sent, signed attachment URLs they were given, a capability URL (a calendar feed or equivalent), and mail to the tenant address. Expected: every path fails on the next request, and mail from a removed member gets no more trust than a stranger's. **Done when:** `Probes` has a row per path.
|
|
137
|
+
- [ ] **Step 9 — Authentication:** Probe an expired token and a revoked one. A revoked machine token must fail on the next call, with no grace period unless the product records one. A token for a plan that excludes the feature must be refused. Probe a cookie-authenticated write with no `Origin`, and one with a foreign `Origin`. Probe redirect parameters that point off-site. Check that sign-out-all ends every other session. **Done when:** `Probes` has a row per case.
|
|
138
|
+
- [ ] **Step 10 — Inbound:** Send mail to the synthetic tenant's inbound address, from the attacker's address, trying to make the model: delete or change records the message does not mention; leak other records into the reply; fetch an internal address (`http://169.254.169.254/`, localhost, private ranges); claim to be a member (by address or display name) from a domain that fails or lacks SPF and DKIM; and exceed the link and attachment caps. Use a payload corpus (e.g., promptfoo, garak), and record which. On a shared inbound provider, use the environment tag the project documents, so the message cannot land in production. Expected: only the records the message genuinely describes are written, nothing else changes, no internal fetch happens, and every cap fires. Check the action log for what was written. The cost probe crosses one limit, once. **Done when:** `Probes` has a row per attempt, with action-log evidence, or `INCONCLUSIVE` where model spend is unavailable.
|
|
139
|
+
- [ ] **Step 11 — Tools:** With the attacker's token, act as a client that ignores the rules: set fields the allow-list excludes, pass the victim's ids, change a recurring record without the required scope, send oversized arguments, and repeat a create to test the idempotency key. **Done when:** `Probes` has a row per case, or the step is skipped with evidence that there are no machine-client tools.
|
|
140
|
+
- [ ] **Step 12 — Boundaries:** Put script and markup in text fields, and check how they render on the web, in the native client and in a reply. Send injection through the search query and the filters that reach the search routine. Webhook endpoints: no signature must get a denial, and a wrong signature must get the same denial plus an error log. Rate limits: the documented thresholds must fire, then stop. **Done when:** `Probes` has a row per boundary.
|
|
141
|
+
- [ ] **Step 13 — Off-Topic:** For each public free-text model surface (from `bkit-audit-llm` `Job Scope`, or from `Surface` when that report lists none), act as an anonymous or synthetic visitor and send one ask per probe: general coding (e.g., "write a Python function that…"), homework, an essay, and an off-topic ask wrapped in an on-topic frame (e.g., "for my trip, write a script that…"). Then send one greeting as a control. Expected: a redirect per the surface's stated job and no off-topic answer, while the greeting gets a normal reply. **If** an off-topic ask is answered → mark it `BROKE`, and record what was meant to bound it (prompt, grounding gate, classifier). **Done when:** `Probes` has a row per surface x ask, or the step is skipped with evidence that there is no public free-text model surface.
|
|
142
|
+
- [ ] **Step 14 — Standards:** Confirm the current version of each anchor standard, and read what changed since the previous report's version. Give `Standards Coverage` one row per WSTG category (INFO, CONF, IDNT, ATHN, ATHZ, SESS, INPV, ERRH, CRYP, BUSL, CLNT, APIT), and one per ATLAS tactic a model probe touched, each marked checked (naming the step), not applicable (saying why) or finding. Give each bkit anchor (a doc section, rule or issue shape) one row, marked followed (with the step or evidence), not applicable (saying why) or finding. A departure with no recorded reason (e.g., a `docs/decisions.jsonl` row or an accepted-divergence entry) is a finding. Mark each recommended practice the project does not apply as adopt (with a proposed step), reject (saying why) or not applicable. **Done when:** no row is empty.
|
|
143
|
+
- [ ] **Step 15 — Anything Else:** (a) Give a one-line verdict on each change since the previous report's commit that touches the attack surface: routes, server actions, machine-client tools, webhooks, response headers, inbound handling, and what the client public key can reach. The Step 3 diff is the starting list.
|
|
144
|
+
- (b) In a bounded pass of about 30 minutes, note what a senior penetration tester would flag that no step asks about. Read what happened since the previous run (incidents, `bkit-audit-logs` reports, new `docs/failures.jsonl` rows, vendor status incidents), and ask what no step would catch.
|
|
145
|
+
- Probes these passes send follow the Rules, use identities made for this run, and go in `Probes`.
|
|
146
|
+
- Then delete every identity this run created with the guarded delete, which also reaps any tenant it would strand.
|
|
147
|
+
- Confirm that the orphan sweep finds nothing left.
|
|
148
|
+
- Record where the full request and response evidence is kept, for how long, and the redaction applied.
|
|
149
|
+
- `Expected` names where the finding goes, and recommends the safer fix.
|
|
150
|
+
- This run files nothing. It does write the `docs/failures.jsonl` rows.
|
|
151
|
+
- Route each result as follows:
|
|
152
|
+
- `BROKE` → a plan from this report, a `docs/failures.jsonl` row, and a regression test (ideally at the deployed boundary) so that `bkit-audit-testing` keeps it fixed.
|
|
153
|
+
- `HELD` with no test → `bkit-audit-testing`.
|
|
154
|
+
- A model result → `bkit-audit-llm`, against the finding it tested.
|
|
155
|
+
- `STILL BROKE` → the original finding, with the retest and the fix commit.
|
|
156
|
+
- `INCONCLUSIVE` → the next run's Step 2 suspicion list, with what blocked it.
|
|
157
|
+
- `OTHER` → this report, and a second run proposes a verdict.
|
|
158
|
+
- A vulnerability in the database product itself, not in the project's use of it → the vendor, per that vendor's testing page, plus this report.
|
|
159
|
+
- A stop condition → the remaining probes stop, and the stop is recorded in `Scope`.
|
|
160
|
+
|
|
161
|
+
**Done when:** `Beyond the Steps` holds both passes, or says `none`. `Cleanup` confirms that every identity this run created is gone. **If** one class of issue is seen on 2 runs → propose a step for it in `Process Improvements`.
|
|
111
162
|
|
|
112
163
|
### 🔒 Gate 2: Probes Check
|
|
113
164
|
|
|
114
|
-
- [ ] **Steps complete:** every step `[x]`, or
|
|
115
|
-
- [ ] **Evidence:** every finding cites
|
|
116
|
-
- [ ] **Target:** no probe
|
|
117
|
-
- [ ] **Cleanup:** every identity this run created is deleted
|
|
118
|
-
- [ ] **
|
|
165
|
+
- [ ] **Steps complete:** every step is `[x]`, or skipped with evidence that its surface is absent.
|
|
166
|
+
- [ ] **Evidence:** every finding cites evidence someone can check again, and the tool that produced it.
|
|
167
|
+
- [ ] **Target:** no probe was sent to production, the release branch, a third-party platform, or a non-synthetic identity.
|
|
168
|
+
- [ ] **Cleanup:** every identity this run created is deleted.
|
|
169
|
+
- [ ] **No tokens:** no token value appears in the report.
|
|
170
|
+
- [ ] **Gate 2 — Verified:** every item above holds.
|
|
119
171
|
|
|
120
172
|
## Findings
|
|
121
173
|
|
|
122
|
-
`HELD` probe stays in `Probes`. Every `BROKE` is finding, shares probe ID. `Expected` carries WSTG id, CWE, likelihood and impact
|
|
174
|
+
A `HELD` probe stays in `Probes`. Every `BROKE` is a finding, and shares its probe ID. `Expected` carries the WSTG id, the CWE, and the reasoning for likelihood and impact. A model finding also names its OWASP Top 10 for LLM / agentic applications category and its MITRE ATLAS technique, the vocabulary `bkit-audit-llm` uses.
|
|
123
175
|
|
|
124
176
|
<!-- prettier-ignore -->
|
|
125
177
|
| Severity | Use |
|
|
@@ -129,14 +181,14 @@ One per probe, on the `Probes` row.
|
|
|
129
181
|
| `MEDIUM` | A mixed id list is clamped instead of refused; a rate limit or size cap does not fire; markup renders unsanitised on one surface; a public model surface answers an off-topic ask |
|
|
130
182
|
| `LOW` | Status differs from expected, and the response exposes no data and grants no action |
|
|
131
183
|
|
|
132
|
-
Within a severity
|
|
184
|
+
Within a severity, list cross-tenant findings first. A break on the non-production target keeps the severity it would have in production.
|
|
133
185
|
|
|
134
186
|
<!-- prettier-ignore -->
|
|
135
187
|
| ID | Severity | Verdict | Location | Evidence | Expected |
|
|
136
188
|
| --- | --- | --- | --- | --- | --- |
|
|
137
189
|
| PENDING | PENDING | PENDING | PENDING | PENDING | PENDING |
|
|
138
190
|
|
|
139
|
-
|
|
191
|
+
**If** the run finds nothing → write one row: `None`.
|
|
140
192
|
|
|
141
193
|
## Report
|
|
142
194
|
|
|
@@ -197,13 +249,17 @@ PENDING
|
|
|
197
249
|
|
|
198
250
|
### 🔒 Gate 3: Report Check
|
|
199
251
|
|
|
200
|
-
- [ ] **No placeholders:** no `PENDING` left
|
|
201
|
-
- [ ] **Summary:**
|
|
202
|
-
- [ ] **
|
|
203
|
-
- [ ] **
|
|
204
|
-
- [ ] **
|
|
205
|
-
- [ ] **
|
|
206
|
-
- [ ] **
|
|
252
|
+
- [ ] **No placeholders:** no `PENDING` is left.
|
|
253
|
+
- [ ] **Summary:** `Summary` gives the count per severity.
|
|
254
|
+
- [ ] **Probe counts:** `Summary` gives the probe counts (`HELD` / `BROKE` / `INCONCLUSIVE`).
|
|
255
|
+
- [ ] **Cleanup stated:** `Cleanup` states that every identity this run created is deleted.
|
|
256
|
+
- [ ] **Evidence kept:** `Cleanup` states where the redacted evidence is kept.
|
|
257
|
+
- [ ] **Writes:** this run changed only this report, the template's `Project Instructions` and its own `docs/failures.jsonl` rows.
|
|
258
|
+
- [ ] **Throwaways removed:** every throwaway worktree or remote target is removed.
|
|
259
|
+
- [ ] **Process Improvements:** `Universal` and `Project` are both filled.
|
|
260
|
+
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID is in exactly one `Suggested Issues` row.
|
|
261
|
+
- [ ] **Next Step:** every `Suggested Issues` row has a `Next Step`.
|
|
262
|
+
- [ ] **Gate 3 — Verified:** every item above holds.
|
|
207
263
|
|
|
208
264
|
<!-- bkit:end -->
|
|
209
265
|
|