@cbartens/bkit 5.5.0 → 6.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.bkit/README.md +30 -20
- package/.bkit/core/.agents/docs.md +1 -0
- package/.bkit/core/.agents/rules.md +51 -26
- package/.bkit/core/.bkit/README.md +29 -19
- package/.bkit/core/.bkit/authoring.md +117 -0
- package/.bkit/core/.bkit/cloud.md +11 -11
- package/.bkit/core/.bkit/comparison.md +5 -5
- package/.bkit/core/.bkit/install.md +20 -20
- package/.bkit/core/.bkit/jsonl.md +202 -122
- package/.bkit/core/.bkit/lite.md +1 -1
- package/.bkit/core/.bkit/pipeline.md +49 -52
- package/.bkit/core/.bkit/prompts.md +106 -167
- package/.bkit/core/.bkit/providers.md +29 -32
- package/.bkit/core/.bkit/templates.md +44 -18
- package/.bkit/core/.bkit/trackers.md +60 -28
- package/.bkit/core/.bkit/workflows.md +399 -359
- package/.bkit/core/AGENTS.md +16 -22
- package/.bkit/core/docs/templates/bkit-audit-accessibility.md +14 -38
- package/.bkit/core/docs/templates/bkit-audit-adoption.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-agents.md +19 -43
- package/.bkit/core/docs/templates/bkit-audit-aio-seo.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-analytics.md +14 -38
- package/.bkit/core/docs/templates/bkit-audit-annotations.md +11 -35
- package/.bkit/core/docs/templates/bkit-audit-app-performance.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-app-store.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-architecture.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-ci-performance.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-config.md +18 -42
- package/.bkit/core/docs/templates/bkit-audit-cost.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-coverage.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-database.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-dead-code.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-deferrals.md +15 -38
- package/.bkit/core/docs/templates/bkit-audit-dependencies.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-documentation.md +11 -35
- package/.bkit/core/docs/templates/bkit-audit-duplication.md +18 -42
- package/.bkit/core/docs/templates/bkit-audit-failures.md +13 -37
- package/.bkit/core/docs/templates/bkit-audit-flakes.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-gates.md +14 -38
- package/.bkit/core/docs/templates/bkit-audit-issues.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-llm.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-logs.md +14 -38
- package/.bkit/core/docs/templates/bkit-audit-models.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-privacy.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-prompts.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-red-team.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-reliability.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-repo.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-security.md +13 -37
- package/.bkit/core/docs/templates/bkit-audit-slop.md +14 -38
- package/.bkit/core/docs/templates/bkit-audit-testing.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-usability.md +16 -40
- package/.bkit/core/docs/templates/bkit-plan.md +97 -226
- package/.bkit/core/engine/checks/after/ci-yml.ts +39 -30
- package/.bkit/core/engine/checks/after/github.ts +101 -72
- package/.bkit/core/engine/checks/after/index.ts +10 -12
- package/.bkit/core/engine/checks/after/suppression.ts +8 -5
- package/.bkit/core/engine/checks/after/tools.ts +47 -92
- package/.bkit/core/engine/checks/before/env.ts +2 -2
- package/.bkit/core/engine/checks/before/files.ts +5 -4
- package/.bkit/core/engine/checks/before/index.ts +18 -1
- package/.bkit/core/engine/checks/before/repo.ts +168 -2
- package/.bkit/core/engine/checks/tips/catalogue.ts +16 -5
- package/.bkit/core/engine/checks/tips/detect.ts +34 -0
- package/.bkit/core/engine/checks/tips/index.ts +10 -7
- package/.bkit/core/engine/cli.ts +20 -53
- package/.bkit/core/engine/commands/analysis.ts +171 -140
- package/.bkit/core/engine/commands/ci/ci.ts +6 -6
- package/.bkit/core/engine/commands/comply.ts +77 -103
- package/.bkit/core/engine/commands/install/doctor.ts +2 -2
- package/.bkit/core/engine/commands/install/install.ts +57 -18
- package/.bkit/core/engine/commands/install/uninstall.ts +65 -28
- package/.bkit/core/engine/commands/land.ts +18 -12
- package/.bkit/core/engine/commands/lib.ts +79 -17
- package/.bkit/core/engine/commands/registry.ts +4 -0
- package/.bkit/core/engine/commands/setup/setup.ts +7 -7
- package/.bkit/core/engine/commands/stats.ts +204 -0
- package/.bkit/core/engine/commands/tips.ts +10 -5
- package/.bkit/core/engine/commands/workflow/branches.ts +741 -62
- package/.bkit/core/engine/commands/workflow/challenge.ts +673 -0
- package/.bkit/core/engine/commands/workflow/context.ts +853 -29
- package/.bkit/core/engine/commands/workflow/counters.ts +267 -84
- package/.bkit/core/engine/commands/workflow/decisions.ts +541 -0
- package/.bkit/core/engine/commands/workflow/disk.ts +2 -2
- package/.bkit/core/engine/commands/workflow/fids.ts +793 -33
- package/.bkit/core/engine/commands/workflow/issues.ts +427 -105
- package/.bkit/core/engine/commands/workflow/ledgers.ts +884 -27
- package/.bkit/core/engine/commands/workflow/locks.ts +26 -2
- package/.bkit/core/engine/commands/workflow/plan-blocks.ts +582 -0
- package/.bkit/core/engine/commands/workflow/plan-versions.ts +21 -0
- package/.bkit/core/engine/commands/workflow/prs.ts +166 -30
- package/.bkit/core/engine/commands/workflow/repo.ts +59 -7
- package/.bkit/core/engine/commands/workflow/roadmap.ts +16 -7
- package/.bkit/core/engine/commands/workflow/run-facts.ts +501 -0
- package/.bkit/core/engine/commands/workflow/runs.ts +179 -49
- package/.bkit/core/engine/commands/workflow/runtime.ts +22 -1
- package/.bkit/core/engine/commands/workflow/status.ts +142 -29
- package/.bkit/core/engine/commands/workflow/step-table.ts +339 -0
- package/.bkit/core/engine/commands/workflow/steps.ts +213 -88
- package/.bkit/core/engine/commands/workflow/summary.ts +398 -0
- package/.bkit/core/engine/commands/workflow/tests.ts +1013 -148
- package/.bkit/core/engine/commands/workflow/tracker.ts +1195 -0
- package/.bkit/core/engine/commands/workflow/workflows.ts +984 -207
- package/.bkit/core/engine/commands/workflow/workstreams.ts +11 -5
- package/.bkit/core/engine/commands/workflow/worktrees.ts +276 -22
- package/.bkit/core/engine/install/manifest.ts +11 -9
- package/.bkit/core/engine/install/modules/cloud.ts +21 -13
- package/.bkit/core/engine/install/modules/core.ts +325 -7
- package/.bkit/core/engine/install/modules/opencode.ts +2 -2
- package/.bkit/core/engine/install/version-file.ts +6 -2
- package/.bkit/core/engine/ledgers/migrate.ts +92 -0
- package/.bkit/core/engine/ledgers/status.ts +92 -15
- package/.bkit/core/engine/ledgers/telemetry.ts +1261 -254
- package/.bkit/core/engine/render/hosts.ts +4 -0
- package/.bkit/core/engine/render/models.ts +12 -0
- package/.bkit/core/engine/shared/branches.ts +4 -1
- package/.bkit/core/engine/shared/egress.ts +7 -0
- package/.bkit/core/engine/shared/json.ts +25 -7
- package/.bkit/core/engine/shared/settings.ts +202 -33
- package/.bkit/core/engine/shared/text.ts +361 -0
- package/.bkit/core/engine/tools/grammar-density.ts +215 -0
- package/.bkit/core/engine/tools/instructions-sources.ts +834 -0
- package/.bkit/core/engine/tools/instructions.ts +1751 -0
- package/.bkit/core/hooks/gateguard-engine.js +24 -23
- package/.bkit/core/hooks/guardrails-shell.js +2 -3
- package/.bkit/core/hooks/guardrails-tiers.js +7 -1
- package/.bkit/core/hooks/readguard-engine.js +2 -2
- package/.bkit/core/prompts/bkit-audit-generate.md +62 -25
- package/.bkit/core/prompts/bkit-build-challenge-1.md +40 -17
- package/.bkit/core/prompts/bkit-build-challenge-2.md +44 -17
- package/.bkit/core/prompts/bkit-build-install-skills.md +26 -13
- package/.bkit/core/prompts/bkit-build-plan-convert.md +74 -0
- package/.bkit/core/prompts/bkit-build-plan-current.md +26 -13
- package/.bkit/core/prompts/bkit-build-plan-execute.md +74 -26
- package/.bkit/core/prompts/bkit-build-plan-gaps-remediate.md +64 -23
- package/.bkit/core/prompts/bkit-build-plan-verify.md +57 -33
- package/.bkit/core/prompts/bkit-debug-abort-handover.md +30 -14
- package/.bkit/core/prompts/bkit-debug-failure-diagnose.md +34 -14
- package/.bkit/core/prompts/bkit-debug-failure-remediate.md +34 -17
- package/.bkit/core/prompts/bkit-debug-failure-reproduce.md +26 -13
- package/.bkit/core/prompts/bkit-debug-failure-soak.md +31 -16
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-1.md +21 -11
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-2.md +20 -11
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-3.md +20 -11
- package/.bkit/core/prompts/bkit-design-generate.md +33 -13
- package/.bkit/core/prompts/bkit-design-organise.md +25 -12
- package/.bkit/core/prompts/bkit-design-review-1.md +31 -15
- package/.bkit/core/prompts/bkit-design-review-2.md +28 -13
- package/.bkit/core/prompts/bkit-design-spike.md +30 -13
- package/.bkit/core/prompts/bkit-design-update.md +33 -19
- package/.bkit/core/prompts/bkit-plan-generate.md +75 -19
- package/.bkit/core/prompts/bkit-plan-review-1.md +50 -27
- package/.bkit/core/prompts/bkit-plan-review-2.md +59 -28
- package/.bkit/core/prompts/bkit-plan-scope-check.md +109 -0
- package/.bkit/core/prompts/bkit-plan-update.md +61 -24
- package/.bkit/core/prompts/bkit-ship-conflict-resolve.md +27 -15
- package/.bkit/core/prompts/bkit-ship-deferred-sweep.md +31 -16
- package/.bkit/core/prompts/bkit-ship-update-docs.md +30 -15
- package/.bkit/core/prompts/bkit-utility-compress-markdown.md +24 -14
- package/.bkit/core/prompts/bkit-utility-update-decisions.md +32 -14
- package/.bkit/core/prompts/bkit-utility-update-deferrals.md +29 -14
- package/.bkit/core/prompts/bkit-utility-update-documentation.md +28 -13
- package/.bkit/core/prompts/bkit-utility-update-failures.md +31 -14
- package/.bkit/core/prompts/bkit-utility-update-knowledge.md +33 -14
- package/.bkit/core/prompts/bkit-utility-update-roadmap.md +38 -17
- package/.bkit/core/prompts/bkit-utility-update-tracker.md +43 -20
- package/.bkit/core/workflows/bkit-auto.md +54 -46
- package/.bkit/core/workflows/bkit-build.md +577 -458
- package/.bkit/core/workflows/bkit-debug.md +348 -333
- package/.bkit/core/workflows/bkit-design.md +201 -237
- package/.bkit/core/workflows/bkit-plan.md +210 -321
- package/.bkit/core/workflows/bkit-ship.md +328 -278
- package/.bkit/shared/.bkit/.env.example +2 -2
- package/.bkit/shared/.claude/settings.json +11 -0
- package/.bkit/shared/.cursor/mcp.json +1 -1
- package/.bkit/shared/.cursor/rules/bkit-rules.mdc +1 -1
- package/.bkit/shared/.gitattributes +1 -1
- package/.bkit/shared/.mcp.json +1 -1
- package/.bkit/shared/.opencode/opencode.json +11 -0
- package/.bkit/shared/CLAUDE.md +2 -2
- package/.bkit/shared/GEMINI.md +2 -2
- package/.bkit/shared/docs/architecture.md +1 -1
- package/.bkit/shared/docs/data-schema.md +1 -1
- package/.bkit/shared/docs/deployments.md +16 -28
- package/.bkit/shared/docs/design.md +1 -1
- package/.bkit/shared/docs/noise.md +1 -1
- package/.bkit/shared/docs/observability.md +2 -2
- package/.bkit/shared/docs/security.md +1 -1
- package/.bkit/shared/docs/setup.md +1 -1
- package/.bkit/shared/docs/technology.md +1 -1
- package/.bkit/shared/docs/testing.md +27 -10
- package/.bkit/templates/.claude/hooks/bkit-cloud-bootstrap.sh +1 -1
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-change.yml +26 -5
- package/.bkit/templates/.github/workflows/bkit-ci.yml +22 -84
- package/.bkit/templates/.github/workflows/bkit-cloud.yml +21 -19
- package/.bkit/templates/.husky/pre-push +52 -13
- package/.bkit/templates/.prettierignore +7 -0
- package/.bkit/templates/scripts/ci/bkit-local-stack-lock.sh +194 -0
- package/.bkit/templates/scripts/ci/check-capstone.mjs +2 -4
- package/.bkit/templates/scripts/ci/check-capstone.test.mjs +4 -23
- package/.bkit/templates/scripts/ci/lanes/resolve-push-base.mjs +104 -23
- package/.bkit/templates/scripts/ci/lanes/resolve-push-base.test.mjs +103 -6
- package/.bkit/templates/scripts/cleanup/bkit-disk-prune.mjs +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graph-pre-push.sh +3 -1
- package/.bkit/templates/scripts/prettier/bkit-insert-ignore.mjs +49 -29
- package/CHANGELOG.md +142 -0
- package/README.md +58 -34
- package/dist/checks/after/ci-yml.js +39 -29
- package/dist/checks/after/github.js +99 -72
- package/dist/checks/after/index.js +11 -13
- package/dist/checks/after/suppression.js +8 -5
- package/dist/checks/after/tools.js +40 -83
- package/dist/checks/before/env.js +2 -2
- package/dist/checks/before/files.js +5 -3
- package/dist/checks/before/index.js +15 -2
- package/dist/checks/before/repo.js +163 -2
- package/dist/checks/tips/catalogue.js +14 -5
- package/dist/checks/tips/detect.js +32 -0
- package/dist/checks/tips/index.js +7 -5
- package/dist/cli.js +19 -47
- package/dist/commands/analysis.js +109 -100
- package/dist/commands/ci/ci.js +6 -6
- package/dist/commands/comply.js +54 -91
- package/dist/commands/install/doctor.js +2 -2
- package/dist/commands/install/install.js +50 -19
- package/dist/commands/install/uninstall.js +59 -28
- package/dist/commands/land.js +16 -12
- package/dist/commands/lib.js +51 -16
- package/dist/commands/registry.js +4 -0
- package/dist/commands/setup/setup.js +7 -7
- package/dist/commands/stats.js +136 -0
- package/dist/commands/tips.js +7 -3
- package/dist/commands/workflow/branches.js +656 -56
- package/dist/commands/workflow/challenge.js +556 -0
- package/dist/commands/workflow/context.js +709 -30
- package/dist/commands/workflow/counters.js +208 -75
- package/dist/commands/workflow/decisions.js +441 -0
- package/dist/commands/workflow/disk.js +2 -2
- package/dist/commands/workflow/fids.js +689 -33
- package/dist/commands/workflow/issues.js +362 -99
- package/dist/commands/workflow/ledgers.js +762 -27
- package/dist/commands/workflow/locks.js +24 -2
- package/dist/commands/workflow/plan-blocks.js +437 -0
- package/dist/commands/workflow/plan-versions.js +22 -0
- package/dist/commands/workflow/prs.js +142 -32
- package/dist/commands/workflow/repo.js +52 -7
- package/dist/commands/workflow/roadmap.js +9 -7
- package/dist/commands/workflow/run-facts.js +426 -0
- package/dist/commands/workflow/runs.js +144 -43
- package/dist/commands/workflow/runtime.js +12 -1
- package/dist/commands/workflow/status.js +130 -29
- package/dist/commands/workflow/step-table.js +276 -0
- package/dist/commands/workflow/steps.js +171 -80
- package/dist/commands/workflow/summary.js +275 -0
- package/dist/commands/workflow/tests.js +834 -136
- package/dist/commands/workflow/tracker.js +983 -0
- package/dist/commands/workflow/workflows.js +834 -195
- package/dist/commands/workflow/workstreams.js +11 -5
- package/dist/commands/workflow/worktrees.js +221 -22
- package/dist/install/manifest.js +11 -9
- package/dist/install/modules/cloud.js +22 -13
- package/dist/install/modules/core.js +326 -7
- package/dist/install/modules/opencode.js +2 -2
- package/dist/install/version-file.js +6 -2
- package/dist/ledgers/migrate.js +81 -0
- package/dist/ledgers/status.js +61 -8
- package/dist/ledgers/telemetry.js +905 -124
- package/dist/render/hosts.js +4 -0
- package/dist/render/models.js +12 -0
- package/dist/shared/branches.js +3 -1
- package/dist/shared/egress.js +7 -0
- package/dist/shared/json.js +20 -7
- package/dist/shared/settings.js +147 -26
- package/dist/shared/text.js +333 -0
- package/dist/tools/grammar-density.js +183 -0
- package/dist/tools/instructions-sources.js +721 -0
- package/dist/tools/instructions.js +1579 -0
- package/package.json +114 -23
- package/.bkit/core/engine/commands/harvest.ts +0 -459
- package/.bkit/core/engine/tools/wording.ts +0 -748
- package/.bkit/core/prompts/bkit-build-challenge-reconcile.md +0 -69
- package/.bkit/core/prompts/bkit-build-challenge-remediate.md +0 -71
- package/.bkit/core/prompts/bkit-build-challenge-tests.md +0 -74
- package/.bkit/core/prompts/bkit-build-summary.md +0 -94
- package/.bkit/core/prompts/bkit-debug-summary.md +0 -96
- package/.bkit/core/prompts/bkit-design-summary.md +0 -107
- package/.bkit/core/prompts/bkit-plan-complexity-check.md +0 -67
- package/.bkit/core/prompts/bkit-plan-complexity-split.md +0 -88
- package/.bkit/core/prompts/bkit-plan-review-3.md +0 -93
- package/.bkit/core/prompts/bkit-plan-scope-check-1.md +0 -85
- package/.bkit/core/prompts/bkit-plan-scope-check-2.md +0 -85
- package/.bkit/core/prompts/bkit-plan-summary.md +0 -107
- package/.bkit/core/prompts/bkit-ship-summary.md +0 -78
- package/.bkit/templates/scripts/ci/lanes/check-tree-cache.mjs +0 -138
- package/.bkit/templates/scripts/ci/lanes/check-tree-cache.test.mjs +0 -181
- package/dist/commands/harvest.js +0 -353
- package/dist/tools/wording.js +0 -680
|
@@ -11,14 +11,6 @@
|
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> - Read-only audit; never edit code, config, or data.
|
|
15
|
-
> - `Universal Instructions` apply to all projects; `Project Instructions` apply only to this project.
|
|
16
|
-
> - Local audit instructions / overrides → `Project Instructions` below the bkit-managed block; universal instructions stay in-block.
|
|
17
|
-
> - Any tools mentioned in `Universal Instructions` are examples only; swap for actual tools in final output; drop inapplicable.
|
|
18
|
-
> - Use dense token-efficient phrasing; no filler / hedging / restating; tables > prose, bullets > paragraphs.
|
|
19
|
-
> - Read `.agents/docs.md` → load docs for workflow / step / task; skip missing; cite docs inline where required.
|
|
20
|
-
> - Every claim cites this run's evidence (`file:line`, or command / query and its output) and tool behind it; a previous report's figure, list, or status is a lead, not evidence.
|
|
21
|
-
> - Read this document once at session start; do not trust session memory; do not re-read on every loop iteration.
|
|
22
14
|
> - NEVER print or copy a secret value; names, presence, dates, and truncated SHA-256 fingerprints only.
|
|
23
15
|
> - Dashboards, CLIs, and APIs read-only; NEVER change a setting, secret, or DNS record.
|
|
24
16
|
> - Every drift claim names the environment and where each side was read.
|
|
@@ -29,11 +21,11 @@
|
|
|
29
21
|
|
|
30
22
|
Code is reviewed; dashboards are not. Dashboard changes (hosting, database, payments, error tracker) leave no diff, review, or history → next environment built from docs differs. Target failure: **drift nobody sees until it fails** — e.g. variable in production missing in preview (preview passes on fallback production never takes, or reverse); required check waiting on renamed job; domain / membership / secret lapsing on a date with no deploy. Each = short outage, cheap to prevent, slow to diagnose (nothing changed).
|
|
31
23
|
|
|
32
|
-
**Applies when:** project deploys to
|
|
24
|
+
**Applies when:** project deploys to >=1 hosted environment, or holds settings outside the repo (variables, dashboards, repository settings, DNS). None → one `N/A` row with evidence, never "clean".
|
|
33
25
|
|
|
34
26
|
**Cadence:** monthly. Extra pass before any release, and after any change made in a dashboard.
|
|
35
27
|
|
|
36
|
-
**Anchor Standards:**
|
|
28
|
+
**Anchor Standards:**
|
|
37
29
|
|
|
38
30
|
- The Twelve-Factor App: III. Config and X. Dev/prod parity.
|
|
39
31
|
- Source-host CI security and ruleset docs (e.g. GitHub Actions secure-use reference, GitHub rulesets REST API).
|
|
@@ -72,17 +64,11 @@ Code is reviewed; dashboards are not. Dashboard changes (hosting, database, paym
|
|
|
72
64
|
|
|
73
65
|
### Pre-Flight / Dependencies
|
|
74
66
|
|
|
75
|
-
**Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
|
|
76
|
-
|
|
77
|
-
**Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
|
|
78
|
-
|
|
79
67
|
#### Core
|
|
80
68
|
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
- [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied (here or in any step) → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`; run every step the remaining tools allow; work needing blocked call → `Not Checked`. never stop the run for a blocked call; never retry or reword a blocked call to get past it.
|
|
69
|
+
- [ ] **Permissions:** blocked calls recorded
|
|
84
70
|
- [ ] **Variable Listing:** Steps 2, 3 — names per environment from every host, CI, and secret store (e.g. `vercel env ls`, `gh secret list` / `gh variable list`, `gcloud secrets list`, `eas env:list`). Fallback: repo-declared names only; environments `Not Checked`.
|
|
85
|
-
- [ ] **Platform Read:** Steps 4
|
|
71
|
+
- [ ] **Platform Read:** Steps 4-6 — read-only dashboard / CLI / management API per platform (e.g. Vercel API, Supabase Management API, Stripe CLI, Sentry API). Fallback: repo and docs only; platform side `Not Checked`.
|
|
86
72
|
- [ ] **Audit Logs:** Step 6 — per-platform settings-change history (e.g. Vercel team activity, Supabase org audit log, GitHub org / repo audit log). Fallback: snapshot diff only; set-and-reverted changes `Not Checked`.
|
|
87
73
|
- [ ] **Repository API:** Step 7 — rulesets, protection, security settings (e.g. `gh api`, GitLab API). Fallback: none; items `Not Checked`.
|
|
88
74
|
- [ ] **DNS and TLS:** Steps 8, 11 — e.g. `dig`, RDAP / `whois`, `openssl s_client`. Fallback: online checker; name it.
|
|
@@ -90,26 +76,22 @@ Always verify.
|
|
|
90
76
|
|
|
91
77
|
#### Conditional
|
|
92
78
|
|
|
93
|
-
Verify only when applicable.
|
|
94
|
-
|
|
95
79
|
- [ ] **Built Bundle:** Build-time public variables → Step 3 — preview build's client bundle to search. Fallback: source search only; bundle `Not Checked`.
|
|
96
|
-
- [ ] **Secret Fingerprints:** Secret held in
|
|
80
|
+
- [ ] **Secret Fingerprints:** Secret held in >=2 stores → Step 3 — read-only value compare by SHA-256 (e.g. repo sync tooling's drift mode `node scripts/secrets/bkit-secrets.mjs drift` from bkit package template `.bkit/templates/scripts/secrets/`; or hash of value read in-process). Fallback: names + `updated_at` only; values `Not Checked`.
|
|
97
81
|
- [ ] **Registrar Account:** Owned domains → Steps 8, 11 — renewal, auto-renew, lock, MFA, contact (e.g. registrar dashboard / API). Fallback: RDAP for lock and expiry; rest `Not Checked`.
|
|
98
82
|
- [ ] **Test Devices:** Native app with deep / invite links → Step 9 — device or emulator with and without the app. Fallback: `.well-known` validation only; link opening `Not Checked`.
|
|
99
83
|
- [ ] **Mailbox-Provider Tools:** Sends email → Step 10 — e.g. Google Postmaster Tools, email-provider dashboard / API. Fallback: headers of one sent message; rates `Not Checked`.
|
|
100
84
|
|
|
101
85
|
### 🔒 Gate 1: Prerequisites Check
|
|
102
86
|
|
|
103
|
-
**Checkpoint.** Verify and check `[x]` all applicable boxes above, then continue to `Probe Steps` without asking.
|
|
104
|
-
|
|
105
87
|
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
106
|
-
- [ ] **Gate 1
|
|
88
|
+
- [ ] **Gate 1 — Verified:** All Core items checked. All applicable Conditional items checked.
|
|
107
89
|
|
|
108
90
|
### Probe Steps
|
|
109
91
|
|
|
110
|
-
- [ ] **Step 1 — Previous Run:**
|
|
111
|
-
- [ ] **Step 2 — Variables:** Every variable declared (env schema / loader, if any) or read in code (e.g. `process.env.X`, `os.environ[...]`) → where set: example env files, each hosting environment (e.g. development / preview / production), CI secrets and variables, secret manager, mobile build service (where used). One matrix, names only. Missing from an environment that runs code reading it = finding unless a doc says why. Set where nothing declares it → `bkit-audit-dead-code`. **Done when:** `Variables` matrix: variable
|
|
112
|
-
- [ ] **Step 3 — Variable Handling:** Per matrix variable: (a) build-time public prefix (e.g. `NEXT_PUBLIC_*`, `EXPO_PUBLIC_*`, `VITE_*`) inlined into shipped bundle → never a secret; change needs rebuild, not env change. Search preview build's client bundle for server-only names and secret-key prefixes (e.g. `sk_live_`); presence only. (b) Secret held as host's secret type (unreadable back from dashboard / CLI). Preview holding production secret (e.g. DB service-role key, live payments key) = finding. (c) Missing / malformed value rejected at startup (e.g. env schema loader) in every environment, not first use; silent default differing between environments = finding. (d) Feature flags: owner, date added, value per environment; preview
|
|
92
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** closure rate in `Previous Run`, or run marked baseline.
|
|
93
|
+
- [ ] **Step 2 — Variables:** Every variable declared (env schema / loader, if any) or read in code (e.g. `process.env.X`, `os.environ[...]`) → where set: example env files, each hosting environment (e.g. development / preview / production), CI secrets and variables, secret manager, mobile build service (where used). One matrix, names only. Missing from an environment that runs code reading it = finding unless a doc says why. Set where nothing declares it → `bkit-audit-dead-code`. **Done when:** `Variables` matrix: variable x environment, names only.
|
|
94
|
+
- [ ] **Step 3 — Variable Handling:** Per matrix variable: (a) build-time public prefix (e.g. `NEXT_PUBLIC_*`, `EXPO_PUBLIC_*`, `VITE_*`) inlined into shipped bundle → never a secret; change needs rebuild, not env change. Search preview build's client bundle for server-only names and secret-key prefixes (e.g. `sk_live_`); presence only. (b) Secret held as host's secret type (unreadable back from dashboard / CLI). Preview holding production secret (e.g. DB service-role key, live payments key) = finding. (c) Missing / malformed value rejected at startup (e.g. env schema loader) in every environment, not first use; silent default differing between environments = finding. (d) Feature flags: owner, date added, value per environment; preview != production with no doc saying why = drift; same everywhere → `bkit-audit-dead-code`. (e) Value parity: secret held in >=2 stores for one environment (env file, hosting, CI, secret manager, mobile build service, function secrets) → same value everywhere; name present everywhere proves nothing (rotated in one store, stale in the rest). Compare SHA-256 fingerprints, never values: readable store → hash live value in-process; write-only store (e.g. GitHub Actions secrets) → fingerprint at last recorded push vs source, and `updated_at` vs that push (later = changed outside the sync). Store holding a name the deployed code no longer reads (renamed, stale) = finding. Unprovable → `Not Checked`, never pass. (f) Source: every secret has one declared source every store is compared against (e.g. sync config with one row per secret naming every store); copy no API reaches (pasted into a provider dashboard, e.g. webhook signing secret) → listed in a copy inventory with how to update it. No declared source, or unlisted hand-set copy = finding; no sync tooling → `Process Improvements` proposes one (e.g. bkit package template `.bkit/templates/scripts/secrets/`). Push path bypassing the sync → `bkit-audit-security`. **Done when:** every matrix variable answers (a)-(f) in `Variables`.
|
|
113
95
|
- [ ] **Step 4 — Platform Settings:** Per platform in use (e.g. hosting: Vercel, Netlify; DB / auth: Supabase, RDS, Auth0; mobile build: EAS), only settings the platform has; none → `N/A`. Dashboard vs repo / docs — hosting: build command, function regions / durations, deployment protection, deployment checks; DB + auth: config file (if any) vs each hosted project (e.g. auth settings, redirect URLs, email templates, rate limits, extensions, scheduled DB jobs vs docs); background jobs: registered per environment vs code registry; webhooks (payments, email): endpoints, subscribed events, inbound routes vs docs; error tracker: alert rules, monitors vs docs; mobile: build profiles / environments vs build config. Auth (config file, dev, production), where offered: site URL, redirect allow-list (no wildcard preview URLs in production), token / OTP expiry, leaked-password protection, SMTP sender, auth rate limits. Mobile config vs builds: runtime version, OTA-update channel per branch (if OTA updates ship, e.g. EAS Update), bundle ids — mismatch breaks OTA silently. Classify: in code / documented / recorded nowhere; recorded nowhere = finding even when value is right (next environment will not get it). **Done when:** `Platform Settings` row per setting: platform · environment · where recorded · verdict.
|
|
114
96
|
- [ ] **Step 5 — Runtimes and Crons:** Runtime version (e.g. `.nvmrc`, `engines`, `.python-version`) vs hosting, CI, mobile build images; DB major version (e.g. platform config file) vs each hosted environment; platform CLI version pinned. Every scheduled job wherever defined (hosting crons, job-runner cron triggers, DB scheduler, scheduled CI workflows) → timezone each runs in; job defined in two places runs twice = finding. **Done when:** `Platform Settings` holds row per runtime and per schedule.
|
|
115
97
|
- [ ] **Step 6 — Dashboard Changes:** Today's diff misses a setting changed and changed back → read each platform audit log since previous report; list each settings change: who, what, recorded in repo / doc? Snapshot dashboard config as secret-free JSON (e.g. hosting project API; DB management API: auth config, network restrictions, SSL enforcement, pooler; source-host API: rulesets); confirm no secret value; diff vs previous report's `Snapshot`, or committed project copy if any. No previous → baseline. **Done when:** `Dashboard Changes` lists every settings change; `Snapshot` holds JSON and diff, or marked baseline.
|
|
@@ -117,19 +99,17 @@ Verify only when applicable.
|
|
|
117
99
|
- [ ] **Step 8 — DNS:** Every domain and subdomain: registrar, renewal date, auto-renew, each record vs where its provider says it should point (e.g. hosting, email). Record for service no longer used = finding (dangling subdomain → takeover). Mail-authentication policy → `bkit-audit-security`; here drift and expiry only. CAA present → authorises every CA the hosts issue from (e.g. `letsencrypt.org`), else renewal fails silently; shrinking certificate lifetimes bite sooner. Registrar hardening: transfer lock (`clientTransferProhibited`), account MFA, registrant email deliverable and off-domain, DNSSEC on or off with written reason. Inbound mail: MX → intended provider; MTA-STS and TLS reporting set (testing mode, then enforce); DMARC `rua` mailbox exists and is read. **Done when:** `DNS` row per domain and record with verdict.
|
|
118
100
|
- [ ] **Step 9 — Links:** Native app with deep / invite links → web serves `/.well-known/apple-app-site-association` and `/.well-known/assetlinks.json`; each valid and names bundle id and signing identity of current build (`Platform Settings`). Open invite link on device with app (opens app) and without (reaches working web page). Broken file breaks every link silently. **Done when:** `Links` row per file and device case.
|
|
119
101
|
- [ ] **Step 10 — Deliverability:** Sends email → sending domains vs mailbox-provider sender requirements; record page thresholds. Spam rate (e.g. Google Postmaster Tools) vs bulk-sender limit. One-click unsubscribe (RFC 8058 `List-Unsubscribe-Post` header + visible link) on every non-transactional message; transactional (sign-in links, receipts) exempt. Bounces, complaints, unsubscribes suppressed; nothing re-adds them. Transactional separate from marketing (subdomain / stream) so marketing complaint cannot sink sign-in mail. **Done when:** `Deliverability` row per requirement: threshold · measurement · verdict.
|
|
120
|
-
- [ ] **Step 11 — Expiry:** Every credential / account with a date: domains, non-auto-renewing certificates, OAuth client secrets, expiring API keys, source-host tokens, build-service tokens, annual paid plans. Store signing credentials and developer memberships → listed by reference to `bkit-audit-app-store`. Per row: date, owner, what breaks. Due
|
|
102
|
+
- [ ] **Step 11 — Expiry:** Every credential / account with a date: domains, non-auto-renewing certificates, OAuth client secrets, expiring API keys, source-host tokens, build-service tokens, annual paid plans. Store signing credentials and developer memberships → listed by reference to `bkit-audit-app-store`. Per row: date, owner, what breaks. Due <=60 days = finding; unknown date = finding. Each non-auto-renewing certificate: issuer and days left measured live (e.g. `openssl s_client`, TLS checker), not read from dashboard; cover custom DB domains, inbound mail hosts, API hosts off main host. Hand-renewed certificate = finding whatever its date; propose automation. **Done when:** `Expiry Calendar` sorted by date, every row with a date or an unknown-date finding.
|
|
121
103
|
- [ ] **Step 12 — Standards:** Confirm current version per anchor standard; read changes since previous report's version. `Standards Coverage`: one row per Twelve-Factor III and X requirement, CI secure-use recommendation, and platform checklist item → checked (name step) / not applicable (why) / finding. Per bkit anchor (doc section, rule, or issue shape): one row → followed (name step or evidence) / not applicable (why) / finding; departure with no recorded reason (e.g. `docs/decisions.jsonl` row, accepted-divergence entry) = finding. Unapplied recommended practice → adopt (propose step) / reject (why) / not applicable. **Done when:** no empty row.
|
|
122
104
|
- [ ] **Step 13 — Anything Else:** (a) Everything changed since previous report's commit touching config (declared variables and example env files, hosting / DB / mobile config files, workflows, and rulesets, scheduled jobs, DNS records, every `Dashboard Changes` entry) → one-line verdict each; dashboard setting changed that no step lists = finding, propose adding it to `Platform Settings`. (b) Bounded ~30 min pass: what a senior platform engineer flags that no step asks; read what happened since previous run (incidents, `bkit-audit-logs` reports, new `docs/failures.jsonl` rows, vendor status incidents) — what would no step catch? **Done when:** `Beyond the Steps` holds both passes, or `none`. Class seen on 2 runs → propose step in `Process Improvements`.
|
|
123
105
|
|
|
124
106
|
### 🔒 Gate 2: Probes Check
|
|
125
107
|
|
|
126
|
-
**Checkpoint.** Verify and check `[x]` all boxes below, then continue to `Findings` without asking.
|
|
127
|
-
|
|
128
108
|
- [ ] **Steps complete:** every step `[x]`, or explicit skip with evidence the surface is absent.
|
|
129
109
|
- [ ] **Evidence:** every finding cites re-checkable evidence and the tool behind it.
|
|
130
110
|
- [ ] **No secrets:** no secret value anywhere in the report, incl. `Snapshot`; fingerprints truncated SHA-256 only.
|
|
131
111
|
- [ ] **Remote untouched:** no dashboard setting, secret, or DNS record changed.
|
|
132
|
-
- [ ] **Gate 2
|
|
112
|
+
- [ ] **Gate 2 — Verified:** Every checkbox above independently verified and checked off.
|
|
133
113
|
|
|
134
114
|
## Findings
|
|
135
115
|
|
|
@@ -137,11 +117,11 @@ Verify only when applicable.
|
|
|
137
117
|
| Severity | Use |
|
|
138
118
|
| --- | --- |
|
|
139
119
|
| `CRITICAL` | Secret in a public build-time variable or client bundle; production secret reachable from preview; dangling record on a live domain; expired domain / certificate / credential still in use |
|
|
140
|
-
| `HIGH` | Required check that no longer reports; anything expiring
|
|
141
|
-
| `MEDIUM` | Setting recorded nowhere; secret with no declared source; hand-set secret copy missing from the inventory; silent default differing between environments; flag drift with no doc; expiry 31
|
|
120
|
+
| `HIGH` | Required check that no longer reports; anything expiring <=30 days; variable missing from an environment that runs code reading it; secret value differing between stores of one environment, or code reading a name no store holds; wildcard redirect in production auth; job scheduled in two places; bypass wider than needed or repository security setting off; CAA blocking the issuing CA; broken `.well-known` file or invite link |
|
|
121
|
+
| `MEDIUM` | Setting recorded nowhere; secret with no declared source; hand-set secret copy missing from the inventory; silent default differing between environments; flag drift with no doc; expiry 31-60 days or unknown; hand-renewed certificate; secret not held as secret type; runtime version drift; registrar hardening gap; deliverability rule not met |
|
|
142
122
|
| `LOW` | Doc describing a setting wrongly while the value is right; naming drift changing no behaviour |
|
|
143
123
|
|
|
144
|
-
|
|
124
|
+
Within a severity: secrets / production first.
|
|
145
125
|
|
|
146
126
|
<!-- prettier-ignore -->
|
|
147
127
|
| ID | Severity | Location | Evidence | Expected |
|
|
@@ -225,17 +205,13 @@ PENDING
|
|
|
225
205
|
|
|
226
206
|
### 🔒 Gate 3: Report Check
|
|
227
207
|
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
- [ ] **No placeholders:** Findings table and every `Report` section have no `PENDING`.
|
|
208
|
+
- [ ] **No placeholders:** no `PENDING` left
|
|
231
209
|
- [ ] **Summary:** names count per severity, environments, and dashboards read.
|
|
232
210
|
- [ ] **Series:** `Snapshot` diffed against previous report's, or marked baseline.
|
|
233
|
-
- [ ] **Writes:** only this report
|
|
234
|
-
- [ ] **Process Improvements:** `Universal`
|
|
211
|
+
- [ ] **Writes:** only this report, template `Project Instructions`, and ledger rows changed; throwaway worktrees / remote targets removed.
|
|
212
|
+
- [ ] **Process Improvements:** `Universal` and `Project` filled
|
|
235
213
|
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID in exactly one row; every row has a `Next Step`.
|
|
236
|
-
- [ ] **Gate 3
|
|
237
|
-
|
|
238
|
-
> `Project Instructions` below `bkit:end` are consumer-owned — `bkit install` never overwrites them, `bkit check` drift stays clean. Add project-specific tools, paths, commands, and checks there, keyed by section or step title; never results.
|
|
214
|
+
- [ ] **Gate 3 — Verified:** Every checkbox above independently verified and checked off.
|
|
239
215
|
|
|
240
216
|
<!-- bkit:end -->
|
|
241
217
|
|
|
@@ -11,14 +11,6 @@
|
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> - Read-only audit; never edit code, config, or data.
|
|
15
|
-
> - `Universal Instructions` apply to all projects; `Project Instructions` apply only to this project.
|
|
16
|
-
> - Local audit instructions / overrides → `Project Instructions` below the bkit-managed block; universal instructions stay in-block.
|
|
17
|
-
> - Any tools mentioned in `Universal Instructions` are examples only; swap for actual tools in final output; drop inapplicable.
|
|
18
|
-
> - Use dense token-efficient phrasing; no filler / hedging / restating; tables > prose, bullets > paragraphs.
|
|
19
|
-
> - Read `.agents/docs.md` → load docs for workflow / step / task; skip missing; cite docs inline where required.
|
|
20
|
-
> - Every claim cites this run's evidence (`file:line`, or command / query and its output) and tool behind it; a previous report's figure, list, or status is a lead, not evidence.
|
|
21
|
-
> - Read this document once at session start; do not trust session memory; do not re-read on every loop iteration.
|
|
22
14
|
> - never change a plan, cancel a resource, or edit a billing setting.
|
|
23
15
|
> - NEVER copy an invoice number, payment-card detail, account id, or secret into the report.
|
|
24
16
|
|
|
@@ -34,7 +26,7 @@
|
|
|
34
26
|
|
|
35
27
|
**Cadence:** monthly, after invoices close. Extra pass when a vendor or a paid plan is added or changed, a spend alert fires, or at launch.
|
|
36
28
|
|
|
37
|
-
**Anchor Standards:**
|
|
29
|
+
**Anchor Standards:**
|
|
38
30
|
|
|
39
31
|
- FinOps Framework (e.g. the current edition). Capabilities this audit uses: Allocation, Anomaly Management, Budgeting, Forecasting, Unit Economics, and Rate Optimization, across Public Cloud, SaaS, and AI.
|
|
40
32
|
- FOCUS (FinOps Open Cost and Usage Specification); record the version.
|
|
@@ -74,41 +66,31 @@ One per vendor, and one per paid resource inside a vendor that has several.
|
|
|
74
66
|
|
|
75
67
|
### Pre-Flight / Dependencies
|
|
76
68
|
|
|
77
|
-
**Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
|
|
78
|
-
|
|
79
|
-
**Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
|
|
80
|
-
|
|
81
69
|
#### Core
|
|
82
70
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
- [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied (here or in any step) → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`; run every step the remaining tools allow; work needing blocked call → `Not Checked`. never stop the run for a blocked call; never retry or reword a blocked call to get past it.
|
|
86
|
-
- [ ] **Bills:** Steps 3–5, 8 — invoices or billing export (e.g. FOCUS), read-only plan/usage/cap pages (e.g. host dashboard, cloud billing console, vendor API). Fallback: unread vendor = `ASK`; tax, foreign exchange, and card fees `Not Checked`.
|
|
71
|
+
- [ ] **Permissions:** blocked calls recorded
|
|
72
|
+
- [ ] **Bills:** Steps 3-5, 8 — invoices or billing export (e.g. FOCUS), read-only plan/usage/cap pages (e.g. host dashboard, cloud billing console, vendor API). Fallback: unread vendor = `ASK`; tax, foreign exchange, and card fees `Not Checked`.
|
|
87
73
|
- [ ] **Code Search:** Steps 3, 6, 7 — SDKs, outbound hosts, reference checks (e.g. ripgrep, language-server references). Fallback: docs and invoices only; `REMOVE` proof marked incomplete.
|
|
88
74
|
- [ ] **Web:** Step 9 — fetch anchor standards. Fallback: versions recorded as unread.
|
|
89
75
|
|
|
90
76
|
#### Conditional
|
|
91
77
|
|
|
92
|
-
Verify only when applicable.
|
|
93
|
-
|
|
94
78
|
- [ ] **Payment Reports:** Processor or store fees → Steps 3, 5 — fee and commission reports (e.g. card processor, app-store proceeds). Fallback: `Not Checked`.
|
|
95
79
|
- [ ] **Legacy Accounts:** Migration map names replaced platform → Step 3 — confirm those subscriptions are not still billing. Fallback: `ASK`.
|
|
96
80
|
|
|
97
81
|
### 🔒 Gate 1: Prerequisites Check
|
|
98
82
|
|
|
99
|
-
**Checkpoint.** Verify and check `[x]` all applicable boxes above, then continue to `Probe Steps` without asking.
|
|
100
|
-
|
|
101
83
|
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
102
|
-
- [ ] **Gate 1
|
|
84
|
+
- [ ] **Gate 1 — Verified:** All Core items checked. All applicable Conditional items checked.
|
|
103
85
|
|
|
104
86
|
### Probe Steps
|
|
105
87
|
|
|
106
|
-
- [ ] **Step 1 — Previous Run:**
|
|
107
|
-
- [ ] **Step 2 — Detect:** Find paid vendors, usage-billed APIs, or payment / store fees (manifests, outbound hosts, vendor docs). **Done when:** evidence in `Checked`, or `N/A` and STOP
|
|
88
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** closure rate in `Previous Run`, or run marked baseline.
|
|
89
|
+
- [ ] **Step 2 — Detect:** Find paid vendors, usage-billed APIs, or payment / store fees (manifests, outbound hosts, vendor docs). **Done when:** evidence in `Checked`, or `N/A` and **STOP**.
|
|
108
90
|
- [ ] **Step 3 — Inventory:** Build list from code (SDKs, outbound hosts), docs (vendor list, secrets list), and owner-shared invoices; compare. In code, not docs = finding (doc fix → `bkit-audit-documentation`). On invoice, nowhere in code = `REMOVE` candidate. Count non-vendor-plan fees (card-processor, app-store commission): grow with paying accounts; no vendor dashboard. Replaced platform still billing after replacement shipped = `REMOVE` candidate. **Done when:** `Vendors` lists every vendor with its sources.
|
|
109
|
-
- [ ] **Step 4 — Plans:** Per vendor: plan, billing model (flat, per seat, per use), spend last three months, usage driver, plan limit, proximity to limit. Split by environment; preview/dev billed like production = finding. Per usage-billed vendor: preview/E2E/CI vs production share, by feature where vendor can tell. Log audit environment split → start from `bkit-audit-logs`. E2E-driven usage on usage-billed vendor = `RIGHT-SIZE` candidate. Every free-tier resource production/CI depends on; behavior at limit (e.g. free DB pauses, free monitor stops). Free tier failing hard = finding even at $0 today. Compare monthly vs annual, prepaid credits, startup programmes, committed-use discounts; propose commitment only if usage stable
|
|
110
|
-
- [ ] **Step 5 — Unit Cost:** Total monthly cost ÷ active accounts and active users; split fixed/variable. Per metered feature, cost per action (e.g. per model assist, per inbound message). Series vs previous report: without unit cost, bill health is undecidable as usage grows. vs revenue: price
|
|
111
|
-
- [ ] **Step 6 — Drivers:** Per usage-billed service: code driving the meter and worst case (retry, tenant fan-out, cron frequency). Multiplier with no upper bound = finding. Model-call multiplier also names `bkit-audit-llm` (in-code limits); other unbounded multipliers → `docs/failures.jsonl` row, which the audit agent writes. Egress/data transfer own line: grow with users, unnamed in code (DB egress, cached egress, host data transfer, origin transfer, storage transforms, log-drain egress); top routes/buckets by bytes. Observability metered: log drains, host observability events, replay, trace sample rates in code. From drivers, forecast monthly cost at
|
|
91
|
+
- [ ] **Step 4 — Plans:** Per vendor: plan, billing model (flat, per seat, per use), spend last three months, usage driver, plan limit, proximity to limit. Split by environment; preview/dev billed like production = finding. Per usage-billed vendor: preview/E2E/CI vs production share, by feature where vendor can tell. Log audit environment split → start from `bkit-audit-logs`. E2E-driven usage on usage-billed vendor = `RIGHT-SIZE` candidate. Every free-tier resource production/CI depends on; behavior at limit (e.g. free DB pauses, free monitor stops). Free tier failing hard = finding even at $0 today. Compare monthly vs annual, prepaid credits, startup programmes, committed-use discounts; propose commitment only if usage stable >=3 months. Record billing currency, card FX fees, sales tax/VAT/GST/reverse-charge correctness (invoice + card statement, not dashboards). **Done when:** `Vendors` row per vendor: verdict · plan · billing model · three-month spend · change since previous report · driver · limit · environment split. Total spend included.
|
|
92
|
+
- [ ] **Step 5 — Unit Cost:** Total monthly cost ÷ active accounts and active users; split fixed/variable. Per metered feature, cost per action (e.g. per model assist, per inbound message). Series vs previous report: without unit cost, bill health is undecidable as usage grows. vs revenue: price x assumed conversion, less processor fees and store commission, vs cost per account. Account costing more than it pays = finding with assumptions. **Done when:** `Unit Cost` holds figures, trend, assumptions.
|
|
93
|
+
- [ ] **Step 6 — Drivers:** Per usage-billed service: code driving the meter and worst case (retry, tenant fan-out, cron frequency). Multiplier with no upper bound = finding. Model-call multiplier also names `bkit-audit-llm` (in-code limits); other unbounded multipliers → `docs/failures.jsonl` row, which the audit agent writes. Egress/data transfer own line: grow with users, unnamed in code (DB egress, cached egress, host data transfer, origin transfer, storage transforms, log-drain egress); top routes/buckets by bytes. Observability metered: log drains, host observability events, replay, trace sample rates in code. From drivers, forecast monthly cost at 10x and 100x today's active accounts; name first vendor whose tier breaks and month at current growth. CI minutes → `bkit-audit-ci-performance`; do not analyse here. **Done when:** `Drivers` row per meter; `Forecast` holds 10x, 100x, first tier break, month.
|
|
112
94
|
- [ ] **Step 7 — Leftovers:** Seats for people who no longer need them — cost only here; access → `bkit-audit-security`. Idle resources: projects, DBs, monitors, buckets, domains from experiments. Duplicates: two vendors, one job. **Done when:** `Leftovers` row per hit with monthly amount and verdict.
|
|
113
95
|
- [ ] **Step 8 — Caps:** Per usage-billed vendor: spend cap or budget alert, amount, recipients. No cap and no alert = finding even when spend is small. When hit:
|
|
114
96
|
- Notify, stop spend, or pause service; whether hit = outage. Notify-only may leave production serving; pause may take production down (e.g. host spend limit notifies unless production deploys paused). Record check frequency; lag → overshoot.
|
|
@@ -119,17 +101,15 @@ Verify only when applicable.
|
|
|
119
101
|
Anomalies since previous report: how found (alert or invoice), time to find, cost avoided. Cap that only notifies, omits line items, or pauses production → owner decision on notify/cap/pause with what each would stop; this run does not set it. **Done when:** `Caps` row per usage-billed vendor, and anomaly list.
|
|
120
102
|
|
|
121
103
|
- [ ] **Step 9 — Standards:** Confirm current version per anchor standard; read changes since previous report's version. `Standards Coverage`: one row per FinOps capability (Allocation, Anomaly Management, Budgeting, Forecasting, Unit Economics, and Rate Optimization) → checked (name step) / not applicable (why) / finding. Score each capability crawl, walk, or run; name next to raise. Per bkit anchor (doc section, rule, or issue shape): one row → followed (name step or evidence) / not applicable (why) / finding; departure with no recorded reason (e.g. `docs/decisions.jsonl` row, accepted-divergence entry) = finding. Unapplied recommended practice → adopt (propose step) / reject (why) / not applicable. **Done when:** no empty row.
|
|
122
|
-
- [ ] **Step 10 — Anything Else:** (a) Everything changed since previous report's commit touching cost (SDKs, outbound hosts, vendors, plans, crons, schedules, fan-outs, model calls, sample rates, environment, seat changes) → one-line verdict each. (b) Bounded ~30 min pass: what senior FinOps flags that no step asks; read since previous run (incidents, log-audit reports under `docs/audits/`, new `docs/failures.jsonl` rows, vendor status incidents) — what would no step catch? Include invoice charges Steps 4
|
|
104
|
+
- [ ] **Step 10 — Anything Else:** (a) Everything changed since previous report's commit touching cost (SDKs, outbound hosts, vendors, plans, crons, schedules, fan-outs, model calls, sample rates, environment, seat changes) → one-line verdict each. (b) Bounded ~30 min pass: what senior FinOps flags that no step asks; read since previous run (incidents, log-audit reports under `docs/audits/`, new `docs/failures.jsonl` rows, vendor status incidents) — what would no step catch? Include invoice charges Steps 4-8 do not explain (new line item, fee, tax, FX, overage). **Done when:** `Beyond the Steps` holds both passes, or `none`. Class seen on 2 runs → propose step in `Process Improvements`.
|
|
123
105
|
|
|
124
106
|
### 🔒 Gate 2: Probes Check
|
|
125
107
|
|
|
126
|
-
**Checkpoint.** Verify and check `[x]` all boxes below, then continue to `Findings` without asking.
|
|
127
|
-
|
|
128
108
|
- [ ] **Steps complete:** every step `[x]`, or explicit skip with evidence the surface is absent.
|
|
129
109
|
- [ ] **Evidence:** every finding cites re-checkable evidence and the tool behind it.
|
|
130
110
|
- [ ] **No billing writes:** no plan, resource, cap, or billing setting changed.
|
|
131
111
|
- [ ] **No account secrets:** no invoice number, card detail, account id, or secret in the report.
|
|
132
|
-
- [ ] **Gate 2
|
|
112
|
+
- [ ] **Gate 2 — Verified:** Every checkbox above independently verified and checked off.
|
|
133
113
|
|
|
134
114
|
## Findings
|
|
135
115
|
|
|
@@ -141,7 +121,7 @@ Verify only when applicable.
|
|
|
141
121
|
| `MEDIUM` | Plan no longer fits (`RIGHT-SIZE`); preview, end-to-end, or CI share of a usage bill; free tier that fails hard; cap that only notifies, omits line items, or pauses production; duplicate vendor; tax or foreign exchange applied wrongly |
|
|
142
122
|
| `LOW` | Docs omit a vendor the code calls; a seat whose access is otherwise in scope for the security sweep; a currency note with no wrong charge |
|
|
143
123
|
|
|
144
|
-
|
|
124
|
+
Within a severity: larger monthly amount first. No cap and no alert on a usage-billed vendor is at least `HIGH`, even when today's spend is small.
|
|
145
125
|
|
|
146
126
|
<!-- prettier-ignore -->
|
|
147
127
|
| ID | Severity | Verdict | Location | Evidence | Expected |
|
|
@@ -213,17 +193,13 @@ PENDING
|
|
|
213
193
|
|
|
214
194
|
### 🔒 Gate 3: Report Check
|
|
215
195
|
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
- [ ] **No placeholders:** Findings table and every `Report` section have no `PENDING`.
|
|
196
|
+
- [ ] **No placeholders:** no `PENDING` left
|
|
219
197
|
- [ ] **Summary:** names count per severity and total monthly spend.
|
|
220
198
|
- [ ] **Series:** `Vendors` spend, `Unit Cost`, and `Forecast` beside previous report's numbers, or marked baseline.
|
|
221
|
-
- [ ] **Writes:** only this report
|
|
222
|
-
- [ ] **Process Improvements:** `Universal`
|
|
199
|
+
- [ ] **Writes:** only this report, template `Project Instructions`, and ledger rows changed; throwaway worktrees / remote targets removed.
|
|
200
|
+
- [ ] **Process Improvements:** `Universal` and `Project` filled
|
|
223
201
|
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID in exactly one row; every row has a `Next Step`.
|
|
224
|
-
- [ ] **Gate 3
|
|
225
|
-
|
|
226
|
-
> `Project Instructions` below `bkit:end` are consumer-owned — `bkit install` never overwrites them, `bkit check` drift stays clean. Add project-specific tools, paths, commands, and checks there, keyed by section or step title; never results.
|
|
202
|
+
- [ ] **Gate 3 — Verified:** Every checkbox above independently verified and checked off.
|
|
227
203
|
|
|
228
204
|
<!-- bkit:end -->
|
|
229
205
|
|
|
@@ -11,14 +11,6 @@
|
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> - Read-only audit; never edit code, config, or data.
|
|
15
|
-
> - `Universal Instructions` apply to all projects; `Project Instructions` apply only to this project.
|
|
16
|
-
> - Local audit instructions / overrides → `Project Instructions` below the bkit-managed block; universal instructions stay in-block.
|
|
17
|
-
> - Any tools mentioned in `Universal Instructions` are examples only; swap for actual tools in final output; drop inapplicable.
|
|
18
|
-
> - Use dense token-efficient phrasing; no filler / hedging / restating; tables > prose, bullets > paragraphs.
|
|
19
|
-
> - Read `.agents/docs.md` → load docs for workflow / step / task; skip missing; cite docs inline where required.
|
|
20
|
-
> - Every claim cites this run's evidence (`file:line`, or command / query and its output) and tool behind it; a previous report's figure, list, or status is a lead, not evidence.
|
|
21
|
-
> - Read this document once at session start; do not trust session memory; do not re-read on every loop iteration.
|
|
22
14
|
> - never edit workflows, floor files, coverage config, or the gate script; the report states the edit.
|
|
23
15
|
> - never lower a floor, add an exclusion, or add an ignore hint.
|
|
24
16
|
> - NEVER run a tool that rewrites source to measure mutants or coverage. Read artifacts already produced.
|
|
@@ -33,7 +25,7 @@ Static floor: coverage climbs, gate far below, large drop passes green. One-run
|
|
|
33
25
|
|
|
34
26
|
**Cadence:** monthly; extra pass after coverage lift, runner/provider version change, include/exclude/measured-glob change, or new package/app under gated glob.
|
|
35
27
|
|
|
36
|
-
**Anchor Standards:**
|
|
28
|
+
**Anchor Standards:**
|
|
37
29
|
|
|
38
30
|
- Coverage practice for a gate (e.g. Google, "Code Coverage Best Practices"; Ivanković et al., "Code Coverage at Google").
|
|
39
31
|
- Coverage guide for pinned runner and provider (e.g. Vitest coverage, including `coverage.include` and AST-aware remapping; Jest coverage; Istanbul). Record major pinned.
|
|
@@ -44,7 +36,7 @@ Static floor: coverage climbs, gate far below, large drop passes green. One-run
|
|
|
44
36
|
### Scope
|
|
45
37
|
|
|
46
38
|
- Floor history: no floor dropped, no new exclusion, no new ignore hint.
|
|
47
|
-
- Sample
|
|
39
|
+
- Sample >=10 full runs per gated surface; ratchet from lowest, never one run.
|
|
48
40
|
- Denominator: glob-should-cover files vs summary files.
|
|
49
41
|
- Identical-code variance across two runs; cause (tests vs tooling).
|
|
50
42
|
- Recommended floor per gated surface; surfaces move independently.
|
|
@@ -64,15 +56,9 @@ Static floor: coverage climbs, gate far below, large drop passes green. One-run
|
|
|
64
56
|
|
|
65
57
|
### Pre-Flight / Dependencies
|
|
66
58
|
|
|
67
|
-
**Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
|
|
68
|
-
|
|
69
|
-
**Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
|
|
70
|
-
|
|
71
59
|
#### Core
|
|
72
60
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
- [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied (here or in any step) → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`; run every step the remaining tools allow; work needing blocked call → `Not Checked`. never stop the run for a blocked call; never retry or reword a blocked call to get past it.
|
|
61
|
+
- [ ] **Permissions:** blocked calls recorded
|
|
76
62
|
- [ ] **CI History:** Steps 4, 5, 7, 10 — branch coverage from full CI runs, e.g. `gh run list` / `gh run download`, GitLab pipelines, Codecov. Fallback: local re-runs fill gap; mark every local observation.
|
|
77
63
|
- [ ] **Coverage Artifacts:** Steps 6, 7, 11, 12 — summary and per-file coverage, e.g. `coverage-summary.json`, `coverage-final.json`, lcov. Fallback: percentages only; file-level claims `Not Checked`.
|
|
78
64
|
- [ ] **Git History:** Steps 3, 9, 15 — `git log -L` / `git log -p` on floor values and coverage config. Fallback: current tree only; history `Not Checked`.
|
|
@@ -80,27 +66,23 @@ Always verify.
|
|
|
80
66
|
|
|
81
67
|
#### Conditional
|
|
82
68
|
|
|
83
|
-
Verify only when applicable.
|
|
84
|
-
|
|
85
69
|
- [ ] **Local Runner:** Fewer than ten CI observations for a gated surface → Steps 4, 7 — re-run that surface's coverage command in detached worktree, removed before report finishes. Fallback: do not tighten that surface; shortfall in `Not Checked`.
|
|
86
70
|
- [ ] **Mutation Score:** Mutation report or `bkit-audit-testing` report exists → Step 9 — latest score beside each gated surface. Fallback: movement without mutation comparison; record in `Not Checked`. Do not start a run that rewrites source.
|
|
87
71
|
- [ ] **Patch Source:** Changed-file coverage reported → Step 12 — e.g. PR coverage comment changed-file table, Codecov patch status. Fallback: diff plus per-file coverage; if neither, `Not Checked`.
|
|
88
72
|
|
|
89
73
|
### 🔒 Gate 1: Prerequisites Check
|
|
90
74
|
|
|
91
|
-
**Checkpoint.** Verify and check `[x]` all applicable boxes above, then continue to `Probe Steps` without asking.
|
|
92
|
-
|
|
93
75
|
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
94
|
-
- [ ] **Gate 1
|
|
76
|
+
- [ ] **Gate 1 — Verified:** All Core items checked. All applicable Conditional items checked.
|
|
95
77
|
|
|
96
78
|
### Probe Steps
|
|
97
79
|
|
|
98
|
-
- [ ] **Step 1 — Previous Run:**
|
|
99
|
-
- [ ] **Step 2 — Detect:** Find coverage floor gate (CI job, script, or threshold failing below committed number) and where number is stored. List hard-gated vs report-only surfaces. Metric = branch coverage unless recorded decision names another; line, statement, function percentages are not floors. None → `N/A` and STOP. **Done when:** gate path, metric, surface list in `Checked`, or `N/A` and STOP
|
|
80
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** closure rate in `Previous Run`, or run marked baseline.
|
|
81
|
+
- [ ] **Step 2 — Detect:** Find coverage floor gate (CI job, script, or threshold failing below committed number) and where number is stored. List hard-gated vs report-only surfaces. Metric = branch coverage unless recorded decision names another; line, statement, function percentages are not floors. None → `N/A` and STOP. **Done when:** gate path, metric, surface list in `Checked`, or `N/A` and **STOP**.
|
|
100
82
|
- [ ] **Step 3 — History:** Since previous report's commit: no floor dropped (`git log -L` on each floor value and coverage include / exclude). Floor never moves down; drop is fixed, not accommodated. Diff exclusions. Count ignore hints under each gated glob, then and now (e.g. `v8 ignore`, `istanbul ignore`, `c8 ignore`, `# pragma: no cover`). Each new exclusion and ignore hint, with commit = finding. **Done when:** `History` lists every floor change, new exclusion, and new ignore hint, or `none`.
|
|
101
|
-
- [ ] **Step 4 — Sample:** Per gated surface,
|
|
83
|
+
- [ ] **Step 4 — Sample:** Per gated surface, >=10 full runs: same commit, or most recent full CI runs on default branch. CI history first; local re-run only to fill gap; mark local observations. Record every run id. No partial lane. Never ratchet off single run. **Done when:** `Sample` holds >=10 observations per gated surface, or named shortfall and that floor not tightened.
|
|
102
84
|
- [ ] **Step 5 — Lowest:** Per gated surface, record every observation; ratchet from lowest, not latest. Write covered/total beside percentage. Spread = max − min. Spread above two points = finding (non-deterministic coverage); `Variance` finds cause; `Floor` does not tighten until spread fixed. **Done when:** `Sample` row per surface: observations · covered/total · lowest · spread.
|
|
103
|
-
- [ ] **Step 6 — Denominator:** Runner omits file no test imports unless include list names it (e.g. Vitest
|
|
85
|
+
- [ ] **Step 6 — Denominator:** Runner omits file no test imports unless include list names it (e.g. Vitest >=4) → compare summary file count with `git ls-files` for glob surface should cover; list source files missing from report. Missing file inflates percentage = finding. Other runner → equivalent omission, or skip with evidence. **Done when:** missing files in `Denominator`, or skip recorded.
|
|
104
86
|
- [ ] **Step 7 — Variance:** Identical-code coverage should match. Two runs on same commit → diff per-file coverage (e.g. `coverage-final.json`); list files whose covered branches differ. Conditional or timing-dependent tests → `bkit-audit-flakes`. **Done when:** file list in `Variance`, or identical and recorded.
|
|
105
87
|
- [ ] **Step 8 — Floor:** Recommended floor = lowest observation, rounded down, minus one point. Spread above two points keeps current floor. Recommended below current floor → leave floor; drop = finding. Surfaces ratchet separately; do not nudge surface whose tests did not change. **Done when:** `Recommendation` holds current · lowest · recommended · move? per gated surface.
|
|
106
88
|
- [ ] **Step 9 — Movement:** Name tests that caused lift; durable (every tier invocation) vs conditional lane. Ratchet with no named cause = finding. Before crediting tests, record runner, coverage-provider package, and provider per run in `Sample`; diff lockfile between oldest and newest. Provider change (e.g. AST-aware remapping) moves branch counts with no test change → do not credit tests. Latest mutation score beside each gated surface. Coverage up, mutation flat = finding (covers code, checks nothing → `bkit-audit-testing`). **Done when:** `Movement` names cause per surface, or tooling change explaining it.
|
|
@@ -113,13 +95,11 @@ Verify only when applicable.
|
|
|
113
95
|
|
|
114
96
|
### 🔒 Gate 2: Probes Check
|
|
115
97
|
|
|
116
|
-
**Checkpoint.** Verify and check `[x]` all boxes below, then continue to `Findings` without asking.
|
|
117
|
-
|
|
118
98
|
- [ ] **Steps complete:** every step `[x]`, or explicit skip with evidence the surface is absent.
|
|
119
99
|
- [ ] **Evidence:** every finding cites re-checkable evidence and the tool behind it.
|
|
120
100
|
- [ ] **Floors untouched:** no workflow, floor file, coverage config, or gate script changed.
|
|
121
|
-
- [ ] **Sample:** every gated surface has
|
|
122
|
-
- [ ] **Gate 2
|
|
101
|
+
- [ ] **Sample:** every gated surface has >=10 full-run observations, or named shortfall and its floor not tightened; no partial lane in the sample.
|
|
102
|
+
- [ ] **Gate 2 — Verified:** Every checkbox above independently verified and checked off.
|
|
123
103
|
|
|
124
104
|
## Findings
|
|
125
105
|
|
|
@@ -131,7 +111,7 @@ Verify only when applicable.
|
|
|
131
111
|
| `MEDIUM` | Recommended floor below current and the drop left in place; coverage up while the mutation score stays flat; report-only surface with no stay-or-gate decision; high-risk module with uncovered branches; change merged below the patch bar; movement with no named cause |
|
|
132
112
|
| `LOW` | Comment or prose in a floor file that does not change the number; report-only drift inside a recorded stay-report-only decision |
|
|
133
113
|
|
|
134
|
-
|
|
114
|
+
Within a severity: gated surfaces, then exclusions.
|
|
135
115
|
|
|
136
116
|
<!-- prettier-ignore -->
|
|
137
117
|
| ID | Severity | Location | Evidence | Expected |
|
|
@@ -215,17 +195,13 @@ PENDING
|
|
|
215
195
|
|
|
216
196
|
### 🔒 Gate 3: Report Check
|
|
217
197
|
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
- [ ] **No placeholders:** Findings table and every `Report` section have no `PENDING`.
|
|
198
|
+
- [ ] **No placeholders:** no `PENDING` left
|
|
221
199
|
- [ ] **Summary:** names count per severity, and old floor → recommended floor per gated surface.
|
|
222
200
|
- [ ] **Series:** `Sample` spreads and the floor series beside previous report's numbers, or marked baseline.
|
|
223
|
-
- [ ] **Writes:** only this report
|
|
224
|
-
- [ ] **Process Improvements:** `Universal`
|
|
201
|
+
- [ ] **Writes:** only this report, template `Project Instructions`, and ledger rows changed; throwaway worktrees / remote targets removed.
|
|
202
|
+
- [ ] **Process Improvements:** `Universal` and `Project` filled
|
|
225
203
|
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID in exactly one row; every row has a `Next Step`.
|
|
226
|
-
- [ ] **Gate 3
|
|
227
|
-
|
|
228
|
-
> `Project Instructions` below `bkit:end` are consumer-owned — `bkit install` never overwrites them, `bkit check` drift stays clean. Add project-specific tools, paths, commands, and checks there, keyed by section or step title; never results.
|
|
204
|
+
- [ ] **Gate 3 — Verified:** Every checkbox above independently verified and checked off.
|
|
229
205
|
|
|
230
206
|
<!-- bkit:end -->
|
|
231
207
|
|
|
@@ -11,14 +11,6 @@
|
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> - Read-only audit; never edit code, config, or data.
|
|
15
|
-
> - `Universal Instructions` apply to all projects; `Project Instructions` apply only to this project.
|
|
16
|
-
> - Local audit instructions / overrides → `Project Instructions` below the bkit-managed block; universal instructions stay in-block.
|
|
17
|
-
> - Any tools mentioned in `Universal Instructions` are examples only; swap for actual tools in final output; drop inapplicable.
|
|
18
|
-
> - Use dense token-efficient phrasing; no filler / hedging / restating; tables > prose, bullets > paragraphs.
|
|
19
|
-
> - Read `.agents/docs.md` → load docs for workflow / step / task; skip missing; cite docs inline where required.
|
|
20
|
-
> - Every claim cites this run's evidence (`file:line`, or command / query and its output) and tool behind it; a previous report's figure, list, or status is a lead, not evidence.
|
|
21
|
-
> - Read this document once at session start; do not trust session memory; do not re-read on every loop iteration.
|
|
22
14
|
> - Remote catalogue queries ONLY use `SELECT` (non-relational: read-only reads); writes run locally, or in explicitly rolled-back transactions; ALWAYS check remote unchanged.
|
|
23
15
|
> - NEVER push schema to remote by hand; NEVER hard-delete; restore target = throwaway, deleted before report finishes.
|
|
24
16
|
> - Every claim names the environment it ran against.
|
|
@@ -29,11 +21,11 @@
|
|
|
29
21
|
|
|
30
22
|
Migrations and integration tests exercise queries; neither verifies privileges, policies, or constraints match code assumptions. Target failure: **a control that reads as enforced in migration text and is not enforced in the live catalogue** — e.g. `REVOKE` undone by platform default privileges, tenant table with no row-level policy, `CHECK` evaluating `NULL` (not `FALSE`) for the shape it should reject. Only the catalogue knows → query it.
|
|
31
23
|
|
|
32
|
-
**Applies when:** migrations, schema file, or DB client in a manifest. None → one `N/A` row with evidence, never "clean". Relational DB → all steps. Non-relational store (e.g. Firestore, DynamoDB, MongoDB) → Steps 1
|
|
24
|
+
**Applies when:** migrations, schema file, or DB client in a manifest. None → one `N/A` row with evidence, never "clean". Relational DB → all steps. Non-relational store (e.g. Firestore, DynamoDB, MongoDB) → Steps 1-4, 6-9, 12-13, 17-19 against its own mechanism (e.g. Firestore security rules, IAM policies, admin SDK vs client SDK); Steps 5, 10-11, 14-16 only where it has an equivalent, else `N/A` with evidence.
|
|
33
25
|
|
|
34
26
|
**Cadence:** monthly. Extra pass when a migration adds a privileged function, changes access policy, or adds a tenancy-scoped table.
|
|
35
27
|
|
|
36
|
-
**Anchor Standards:**
|
|
28
|
+
**Anchor Standards:**
|
|
37
29
|
|
|
38
30
|
- Platform production checklist and security guide (e.g. Supabase).
|
|
39
31
|
- Platform advisors and inspection CLI (e.g. Supabase Advisors, `supabase inspect db`); record CLI version.
|
|
@@ -64,40 +56,30 @@ Migrations and integration tests exercise queries; neither verifies privileges,
|
|
|
64
56
|
|
|
65
57
|
### Pre-Flight / Dependencies
|
|
66
58
|
|
|
67
|
-
**Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
|
|
68
|
-
|
|
69
|
-
**Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
|
|
70
|
-
|
|
71
59
|
#### Core
|
|
72
60
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
- [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied (here or in any step) → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`; run every step the remaining tools allow; work needing blocked call → `Not Checked`. never stop the run for a blocked call; never retry or reword a blocked call to get past it.
|
|
76
|
-
- [ ] **Catalogue Read:** Steps 3–16 — read-only SQL (or store's read-only API) per environment, e.g. platform MCP `execute_sql`, `psql` as read-only role. Fallback: local DB from migrations; remote claims = hypotheses.
|
|
61
|
+
- [ ] **Permissions:** blocked calls recorded
|
|
62
|
+
- [ ] **Catalogue Read:** Steps 3-16 — read-only SQL (or store's read-only API) per environment, e.g. platform MCP `execute_sql`, `psql` as read-only role. Fallback: local DB from migrations; remote claims = hypotheses.
|
|
77
63
|
- [ ] **Local DB:** Steps 2, 10, 14 — replayable from migrations. Fallback: rolled-back transactions and remote re-check.
|
|
78
64
|
- [ ] **Web:** Step 18 — fetch anchor standards. Fallback: versions recorded as unread.
|
|
79
65
|
|
|
80
66
|
#### Conditional
|
|
81
67
|
|
|
82
|
-
Verify only when applicable.
|
|
83
|
-
|
|
84
68
|
- [ ] **Platform Advisors:** Managed platform → Step 3 — e.g. Supabase MCP `get_advisors`. Fallback: equivalent catalogue queries.
|
|
85
69
|
- [ ] **Platform Settings:** Managed platform → Steps 4, 17 — network, MFA, owners, backups, point-in-time recovery, e.g. dashboard, management API. Fallback: none; items `Not Checked`.
|
|
86
|
-
- [ ] **Inspection CLI:** Platforms with one → Steps 14
|
|
70
|
+
- [ ] **Inspection CLI:** Platforms with one → Steps 14-16 — e.g. `supabase inspect db`. Fallback: engine statistics views, e.g. `pg_stat_*`.
|
|
87
71
|
- [ ] **Migration Linter:** SQL migrations → Step 10 — e.g. Squawk. Fallback: apply its rules by reading.
|
|
88
72
|
- [ ] **Restore Target:** Hosted DB → Step 17 — permission and tier to create throwaway project. Fallback: none; absence = finding.
|
|
89
73
|
|
|
90
74
|
### 🔒 Gate 1: Prerequisites Check
|
|
91
75
|
|
|
92
|
-
**Checkpoint.** Verify and check `[x]` all applicable boxes above, then continue to `Probe Steps` without asking.
|
|
93
|
-
|
|
94
76
|
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
95
|
-
- [ ] **Gate 1
|
|
77
|
+
- [ ] **Gate 1 — Verified:** All Core items checked. All applicable Conditional items checked.
|
|
96
78
|
|
|
97
79
|
### Probe Steps
|
|
98
80
|
|
|
99
|
-
- [ ] **Step 1 — Previous Run:**
|
|
100
|
-
- [ ] **Step 2 — Detect:** Find migrations / schema file / DB client. Confirm local DB matches migrations before trusting local measurements. Schema doc absent, migrations exist → missing doc = finding; still read migrations. **Done when:** evidence in `Checked`, or `N/A` and STOP
|
|
81
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** closure rate in `Previous Run`, or run marked baseline.
|
|
82
|
+
- [ ] **Step 2 — Detect:** Find migrations / schema file / DB client. Confirm local DB matches migrations before trusting local measurements. Schema doc absent, migrations exist → missing doc = finding; still read migrations. **Done when:** evidence in `Checked`, or `N/A` and **STOP**.
|
|
101
83
|
- [ ] **Step 3 — Advisors:** Run platform security and performance advisors; each row = candidate, not finding. One line per lint the platform's advisor runs, even when clean (e.g. Supabase: unindexed FKs, duplicate indexes, per-row auth in policy, multiple permissive policies, mutable function `search_path`, extension in public schema, exposed table without row-level security, owner-rights view). No advisor → engine-appropriate subset by catalogue query; rest `N/A`. Cite platform fix pattern per finding. **Done when:** every row dispositioned in `Advisors`.
|
|
102
84
|
- [ ] **Step 4 — Checklist:** Walk platform production checklist against production, not memory: transport encryption enforced, network restrictions, org MFA, >1 owner, point-in-time recovery where size / RPO needs it, compute sized for `Connections` peak. **Done when:** every item met / not met (finding) / not applicable (why).
|
|
103
85
|
- [ ] **Step 5 — Inventory:** Count tables, migrations, extensions. Applied vs directory, both directions — drift invalidates downstream. Hosted major vs platform support window; extensions blocking next major. **Done when:** counts, drift, version in `Environment`.
|
|
@@ -118,13 +100,11 @@ Verify only when applicable.
|
|
|
118
100
|
|
|
119
101
|
### 🔒 Gate 2: Probes Check
|
|
120
102
|
|
|
121
|
-
**Checkpoint.** Verify and check `[x]` all boxes below, then continue to `Findings` without asking.
|
|
122
|
-
|
|
123
103
|
- [ ] **Steps complete:** every step `[x]`, or explicit skip with evidence the surface is absent.
|
|
124
104
|
- [ ] **Evidence:** every finding cites re-checkable evidence and the tool behind it.
|
|
125
105
|
- [ ] **Environments:** every finding names the environment it ran against.
|
|
126
106
|
- [ ] **Remote untouched:** no remote database changed; throwaway restore target deleted.
|
|
127
|
-
- [ ] **Gate 2
|
|
107
|
+
- [ ] **Gate 2 — Verified:** Every checkbox above independently verified and checked off.
|
|
128
108
|
|
|
129
109
|
## Findings
|
|
130
110
|
|
|
@@ -132,11 +112,11 @@ Verify only when applicable.
|
|
|
132
112
|
| Severity | Use |
|
|
133
113
|
| --- | --- |
|
|
134
114
|
| `CRITICAL` | Tenant data readable / writable across tenants or by anonymous role; anonymous-executable privileged function without identity check; write failing silently every call; migration dropping data without backfill |
|
|
135
|
-
| `HIGH` | Tenancy table with no enforcing layer (no row policy, no app-layer scope); public bucket / owner-rights view exposing tenant data; constraint that cannot fire; lock-unsafe migration on live table; no proven restore, no recovery targets, or no off-platform backup; applied migrations
|
|
136
|
-
| `MEDIUM` | Missing index the query shape needs; housekeeping table without purge; vacuum lag; sequence >50%; role without timeouts; derived data without agreement check; schema doc
|
|
115
|
+
| `HIGH` | Tenancy table with no enforcing layer (no row policy, no app-layer scope); public bucket / owner-rights view exposing tenant data; constraint that cannot fire; lock-unsafe migration on live table; no proven restore, no recovery targets, or no off-platform backup; applied migrations != directory |
|
|
116
|
+
| `MEDIUM` | Missing index the query shape needs; housekeeping table without purge; vacuum lag; sequence >50%; role without timeouts; derived data without agreement check; schema doc != migrations |
|
|
137
117
|
| `LOW` | Deny-all table without policies; naming drift changing neither access nor results |
|
|
138
118
|
|
|
139
|
-
|
|
119
|
+
Within a severity: security / data first.
|
|
140
120
|
|
|
141
121
|
<!-- prettier-ignore -->
|
|
142
122
|
| ID | Severity | Location | Evidence | Expected |
|
|
@@ -224,17 +204,13 @@ PENDING
|
|
|
224
204
|
|
|
225
205
|
### 🔒 Gate 3: Report Check
|
|
226
206
|
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
- [ ] **No placeholders:** Findings table and every `Report` section have no `PENDING`.
|
|
207
|
+
- [ ] **No placeholders:** no `PENDING` left
|
|
230
208
|
- [ ] **Summary:** names count per severity and environments measured.
|
|
231
209
|
- [ ] **Series:** every series beside previous report's numbers, or marked baseline.
|
|
232
|
-
- [ ] **Writes:** only this report
|
|
233
|
-
- [ ] **Process Improvements:** `Universal`
|
|
210
|
+
- [ ] **Writes:** only this report, template `Project Instructions`, and ledger rows changed; throwaway worktrees / remote targets removed.
|
|
211
|
+
- [ ] **Process Improvements:** `Universal` and `Project` filled
|
|
234
212
|
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID in exactly one row; every row has a `Next Step`.
|
|
235
|
-
- [ ] **Gate 3
|
|
236
|
-
|
|
237
|
-
> `Project Instructions` below `bkit:end` are consumer-owned — `bkit install` never overwrites them, `bkit check` drift stays clean. Add project-specific tools, paths, commands, and checks there, keyed by section or step title; never results.
|
|
213
|
+
- [ ] **Gate 3 — Verified:** Every checkbox above independently verified and checked off.
|
|
238
214
|
|
|
239
215
|
<!-- bkit:end -->
|
|
240
216
|
|