@cbartens/bkit 6.0.0 → 6.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.bkit/README.md +154 -151
- package/.bkit/core/.agents/context.md +95 -29
- package/.bkit/core/.agents/docs.md +17 -15
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-gateguard.cjs +1 -0
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-guardrails.cjs +6 -5
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-readguard.cjs +1 -0
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-rules.cjs +65 -0
- package/.bkit/core/.agents/rules.md +108 -107
- package/.bkit/core/.agents/skills.md +63 -15
- package/.bkit/core/.bkit/README.md +154 -151
- package/.bkit/core/.bkit/authoring.md +714 -84
- package/.bkit/core/.bkit/cli.md +407 -0
- package/.bkit/core/.bkit/cloud.md +281 -193
- package/.bkit/core/.bkit/comparison.md +43 -41
- package/.bkit/core/.bkit/install.md +154 -84
- package/.bkit/core/.bkit/jsonl.md +193 -130
- package/.bkit/core/.bkit/pipeline.md +114 -114
- package/.bkit/core/.bkit/prompts.md +272 -165
- package/.bkit/core/.bkit/providers.md +457 -271
- package/.bkit/core/.bkit/router.md +69 -0
- package/.bkit/core/.bkit/templates.md +529 -256
- package/.bkit/core/.bkit/trackers.md +80 -52
- package/.bkit/core/.bkit/workflows.md +589 -504
- package/.bkit/core/.claude/hooks/bkit-gateguard.cjs +2 -2
- package/.bkit/core/.claude/hooks/bkit-guardrails.cjs +3 -3
- package/.bkit/core/.claude/hooks/bkit-preflight.sh +45 -14
- package/.bkit/core/.codex/hooks/bkit-guardrails.cjs +3 -3
- package/.bkit/core/.codex/hooks/bkit-rules.cjs +57 -0
- package/.bkit/core/.cursor/hooks/bkit-guardrails.cjs +7 -7
- package/.bkit/core/.cursor/hooks/bkit-preflight.sh +47 -12
- package/.bkit/core/.opencode/plugin/bkit-rules.js +89 -0
- package/.bkit/core/AGENTS.md +5 -73
- package/.bkit/core/docs/templates/bkit-audit-accessibility.md +127 -105
- package/.bkit/core/docs/templates/bkit-audit-adoption.md +104 -70
- package/.bkit/core/docs/templates/bkit-audit-agents.md +94 -72
- package/.bkit/core/docs/templates/bkit-audit-aio-seo.md +99 -78
- package/.bkit/core/docs/templates/bkit-audit-analytics.md +140 -82
- package/.bkit/core/docs/templates/bkit-audit-annotations.md +98 -72
- package/.bkit/core/docs/templates/bkit-audit-app-performance.md +121 -96
- package/.bkit/core/docs/templates/bkit-audit-app-store.md +157 -76
- package/.bkit/core/docs/templates/bkit-audit-architecture.md +113 -71
- package/.bkit/core/docs/templates/bkit-audit-ci-performance.md +75 -72
- package/.bkit/core/docs/templates/bkit-audit-config.md +122 -76
- package/.bkit/core/docs/templates/bkit-audit-cost.md +85 -74
- package/.bkit/core/docs/templates/bkit-audit-coverage.md +75 -68
- package/.bkit/core/docs/templates/bkit-audit-database.md +101 -70
- package/.bkit/core/docs/templates/bkit-audit-dead-code.md +164 -83
- package/.bkit/core/docs/templates/bkit-audit-deferrals.md +115 -86
- package/.bkit/core/docs/templates/bkit-audit-dependencies.md +264 -88
- package/.bkit/core/docs/templates/bkit-audit-documentation.md +118 -93
- package/.bkit/core/docs/templates/bkit-audit-duplication.md +139 -86
- package/.bkit/core/docs/templates/bkit-audit-failures.md +110 -81
- package/.bkit/core/docs/templates/bkit-audit-flakes.md +99 -74
- package/.bkit/core/docs/templates/bkit-audit-gates.md +105 -77
- package/.bkit/core/docs/templates/bkit-audit-issues.md +104 -80
- package/.bkit/core/docs/templates/bkit-audit-llm.md +131 -58
- package/.bkit/core/docs/templates/bkit-audit-logs.md +123 -100
- package/.bkit/core/docs/templates/bkit-audit-models.md +98 -53
- package/.bkit/core/docs/templates/bkit-audit-privacy.md +56 -52
- package/.bkit/core/docs/templates/bkit-audit-prompts.md +63 -54
- package/.bkit/core/docs/templates/bkit-audit-red-team.md +124 -68
- package/.bkit/core/docs/templates/bkit-audit-reliability.md +95 -70
- package/.bkit/core/docs/templates/bkit-audit-repo.md +169 -73
- package/.bkit/core/docs/templates/bkit-audit-security.md +206 -96
- package/.bkit/core/docs/templates/bkit-audit-slop.md +77 -70
- package/.bkit/core/docs/templates/bkit-audit-testing.md +162 -114
- package/.bkit/core/docs/templates/bkit-audit-usability.md +113 -71
- package/.bkit/core/docs/templates/bkit-plan.md +138 -132
- package/.bkit/core/engine/checks/after/auth.ts +40 -37
- package/.bkit/core/engine/checks/after/ci-yml.ts +65 -2
- package/.bkit/core/engine/checks/after/index.ts +12 -2
- package/.bkit/core/engine/checks/after/tools.ts +80 -33
- package/.bkit/core/engine/checks/before/context-fetch.ts +227 -33
- package/.bkit/core/engine/checks/before/env.ts +1 -1
- package/.bkit/core/engine/checks/before/index.ts +7 -0
- package/.bkit/core/engine/checks/before/repo.ts +40 -0
- package/.bkit/core/engine/checks/status-holes.ts +916 -123
- package/.bkit/core/engine/checks/tips/catalogue.ts +68 -17
- package/.bkit/core/engine/checks/tips/detect.ts +32 -0
- package/.bkit/core/engine/checks/tips/index.ts +47 -6
- package/.bkit/core/engine/cli.ts +238 -84
- package/.bkit/core/engine/commands/analysis.ts +128 -76
- package/.bkit/core/engine/commands/ci/ci.ts +363 -29
- package/.bkit/core/engine/commands/ci/pins.ts +172 -0
- package/.bkit/core/engine/commands/ci/push-gate.ts +223 -0
- package/.bkit/core/engine/commands/install/uninstall.ts +21 -27
- package/.bkit/core/engine/commands/install/upgrade.ts +32 -15
- package/.bkit/core/engine/commands/land.ts +126 -11
- package/.bkit/core/engine/commands/lib.ts +303 -6
- package/.bkit/core/engine/commands/registry.ts +6 -2
- package/.bkit/core/engine/commands/setup/setup.ts +1 -0
- package/.bkit/core/engine/commands/stats.ts +91 -35
- package/.bkit/core/engine/commands/workflow/branches.ts +1477 -117
- package/.bkit/core/engine/commands/workflow/challenge.ts +114 -39
- package/.bkit/core/engine/commands/workflow/completeness.ts +109 -0
- package/.bkit/core/engine/commands/workflow/context.ts +433 -55
- package/.bkit/core/engine/commands/workflow/counters.ts +244 -39
- package/.bkit/core/engine/commands/workflow/decisions.ts +61 -17
- package/.bkit/core/engine/commands/workflow/disk.ts +237 -55
- package/.bkit/core/engine/commands/workflow/fids.ts +19 -1
- package/.bkit/core/engine/commands/workflow/issues.ts +6 -0
- package/.bkit/core/engine/commands/workflow/ledgers.ts +458 -18
- package/.bkit/core/engine/commands/workflow/locks.ts +2 -0
- package/.bkit/core/engine/commands/workflow/plan-blocks.ts +124 -15
- package/.bkit/core/engine/commands/workflow/plans.ts +257 -0
- package/.bkit/core/engine/commands/workflow/prs.ts +12 -3
- package/.bkit/core/engine/commands/workflow/repo.ts +8 -0
- package/.bkit/core/engine/commands/workflow/roadmap.ts +56 -15
- package/.bkit/core/engine/commands/workflow/run-facts.ts +22 -31
- package/.bkit/core/engine/commands/workflow/runs.ts +652 -34
- package/.bkit/core/engine/commands/workflow/runtime.ts +3 -0
- package/.bkit/core/engine/commands/workflow/skills.ts +1 -0
- package/.bkit/core/engine/commands/workflow/status.ts +200 -17
- package/.bkit/core/engine/commands/workflow/step-table.ts +75 -33
- package/.bkit/core/engine/commands/workflow/steps.ts +635 -24
- package/.bkit/core/engine/commands/workflow/summary-findings.ts +177 -0
- package/.bkit/core/engine/commands/workflow/summary.ts +192 -46
- package/.bkit/core/engine/commands/workflow/tasks.ts +409 -0
- package/.bkit/core/engine/commands/workflow/test-blocks.ts +254 -0
- package/.bkit/core/engine/commands/workflow/tests-changed.ts +201 -0
- package/.bkit/core/engine/commands/workflow/tests.ts +1285 -356
- package/.bkit/core/engine/commands/workflow/workflows.ts +228 -43
- package/.bkit/core/engine/commands/workflow/worktrees.ts +389 -47
- package/.bkit/core/engine/install/cloud-env.ts +251 -121
- package/.bkit/core/engine/install/context-ci-workflow.ts +102 -0
- package/.bkit/core/engine/install/manifest.ts +60 -12
- package/.bkit/core/engine/install/modules/antigravity.ts +3 -2
- package/.bkit/core/engine/install/modules/cloud.ts +3 -3
- package/.bkit/core/engine/install/modules/codex.ts +15 -2
- package/.bkit/core/engine/install/modules/core.ts +956 -52
- package/.bkit/core/engine/install/modules/cursor.ts +6 -6
- package/.bkit/core/engine/install/modules/opencode.ts +16 -4
- package/.bkit/core/engine/install/template-notes.ts +130 -0
- package/.bkit/core/engine/install/version-file.ts +3 -2
- package/.bkit/core/engine/install/write.ts +5 -0
- package/.bkit/core/engine/ledgers/compact.ts +124 -0
- package/.bkit/core/engine/ledgers/journal.ts +291 -0
- package/.bkit/core/engine/ledgers/status.ts +160 -9
- package/.bkit/core/engine/ledgers/telemetry.ts +158 -21
- package/.bkit/core/engine/render/hosts.ts +114 -14
- package/.bkit/core/engine/shared/bkit-latest.ts +26 -4
- package/.bkit/core/engine/shared/dep-updater.ts +60 -0
- package/.bkit/core/engine/shared/egress.ts +1 -1
- package/.bkit/core/engine/shared/package-manager.ts +9 -0
- package/.bkit/core/engine/shared/settings.ts +92 -17
- package/.bkit/core/engine/shared/timeouts.ts +2 -0
- package/.bkit/core/engine/tools/cli-doc.ts +111 -0
- package/.bkit/core/engine/tools/grammar-density.ts +1 -1
- package/.bkit/core/engine/tools/instructions-sources.ts +15 -17
- package/.bkit/core/engine/tools/instructions.ts +94 -43
- package/.bkit/core/hooks/antigravity-hook.js +25 -10
- package/.bkit/core/hooks/gateguard-engine.js +53 -29
- package/.bkit/core/hooks/guardrails-shell.js +4 -4
- package/.bkit/core/hooks/guardrails-tiers.js +3 -3
- package/.bkit/core/hooks/preflight-linear.sh +9 -9
- package/.bkit/core/hooks/preflight-rules.sh +8 -0
- package/.bkit/core/hooks/readguard-engine.js +29 -10
- package/.bkit/core/hooks/session-rules-advisory.js +61 -0
- package/.bkit/core/prompts/bkit-audit-generate.md +187 -116
- package/.bkit/core/prompts/bkit-build-challenge-1.md +56 -37
- package/.bkit/core/prompts/bkit-build-challenge-2.md +65 -45
- package/.bkit/core/prompts/bkit-build-install-skills.md +38 -29
- package/.bkit/core/prompts/bkit-build-plan-convert.md +43 -32
- package/.bkit/core/prompts/bkit-build-plan-current.md +49 -39
- package/.bkit/core/prompts/bkit-build-plan-gaps-remediate.md +79 -55
- package/.bkit/core/prompts/bkit-build-plan-tasks.md +149 -0
- package/.bkit/core/prompts/bkit-build-plan-tests.md +120 -0
- package/.bkit/core/prompts/bkit-build-plan-verify.md +87 -50
- package/.bkit/core/prompts/bkit-debug-abort-handover.md +36 -35
- package/.bkit/core/prompts/bkit-debug-failure-diagnose.md +96 -64
- package/.bkit/core/prompts/bkit-debug-failure-remediate.md +54 -45
- package/.bkit/core/prompts/bkit-debug-failure-reproduce.md +42 -33
- package/.bkit/core/prompts/bkit-debug-failure-soak.md +53 -37
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-1.md +35 -30
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-2.md +34 -29
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-3.md +32 -28
- package/.bkit/core/prompts/bkit-design-generate.md +68 -52
- package/.bkit/core/prompts/bkit-design-organise.md +73 -35
- package/.bkit/core/prompts/bkit-design-review-1.md +58 -54
- package/.bkit/core/prompts/bkit-design-review-2.md +54 -50
- package/.bkit/core/prompts/bkit-design-spike.md +54 -47
- package/.bkit/core/prompts/bkit-design-update.md +42 -31
- package/.bkit/core/prompts/bkit-plan-generate.md +76 -58
- package/.bkit/core/prompts/bkit-plan-review-1.md +63 -49
- package/.bkit/core/prompts/bkit-plan-review-2.md +60 -48
- package/.bkit/core/prompts/bkit-plan-scope-check.md +53 -46
- package/.bkit/core/prompts/bkit-plan-update.md +70 -52
- package/.bkit/core/prompts/bkit-ship-conflict-resolve.md +82 -50
- package/.bkit/core/prompts/bkit-ship-deferred-sweep.md +80 -45
- package/.bkit/core/prompts/bkit-ship-update-docs.md +35 -29
- package/.bkit/core/prompts/bkit-utility-compress-markdown.md +55 -49
- package/.bkit/core/prompts/bkit-utility-update-decisions.md +76 -49
- package/.bkit/core/prompts/bkit-utility-update-deferrals.md +70 -41
- package/.bkit/core/prompts/bkit-utility-update-documentation.md +33 -30
- package/.bkit/core/prompts/bkit-utility-update-failures.md +73 -43
- package/.bkit/core/prompts/bkit-utility-update-knowledge.md +77 -50
- package/.bkit/core/prompts/bkit-utility-update-roadmap.md +77 -45
- package/.bkit/core/prompts/bkit-utility-update-tracker.md +144 -68
- package/.bkit/core/workflows/bkit-auto.md +53 -30
- package/.bkit/core/workflows/bkit-build.md +635 -304
- package/.bkit/core/workflows/bkit-debug.md +301 -148
- package/.bkit/core/workflows/bkit-design.md +167 -95
- package/.bkit/core/workflows/bkit-plan.md +172 -84
- package/.bkit/core/workflows/bkit-ship.md +299 -152
- package/.bkit/shared/.agents/plugins/bkit/hooks.json +10 -0
- package/.bkit/shared/.claude/bkit-cloud.md +12 -10
- package/.bkit/shared/.claude/settings.json +3 -0
- package/.bkit/shared/.codex/bkit-cloud.md +98 -0
- package/.bkit/shared/.codex/hooks.json +20 -0
- package/.bkit/shared/.cursor/bkit-cloud.md +3 -3
- package/.bkit/shared/.opencode/opencode.json +1 -1
- package/.bkit/shared/CLAUDE.md +5 -1
- package/.bkit/shared/GEMINI.md +4 -1
- package/.bkit/shared/docs/architecture.md +4 -7
- package/.bkit/shared/docs/data-schema.md +4 -7
- package/.bkit/shared/docs/deployments.md +239 -166
- package/.bkit/shared/docs/design.md +11 -12
- package/.bkit/shared/docs/noise.md +8 -7
- package/.bkit/shared/docs/observability.md +5 -8
- package/.bkit/shared/docs/security.md +35 -17
- package/.bkit/shared/docs/setup.md +29 -18
- package/.bkit/shared/docs/technology.md +3 -6
- package/.bkit/shared/docs/testing.md +282 -176
- package/.bkit/shared/gitignore +6 -2
- package/.bkit/templates/.claude/hooks/bkit-cloud-bootstrap.sh +15 -16
- package/.bkit/templates/.cursor/Dockerfile +12 -8
- package/.bkit/templates/.cursor/environment.json +1 -1
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-bug.yml +10 -8
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-change.yml +24 -12
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-idea.yml +3 -3
- package/.bkit/templates/.github/renovate.json5 +34 -6
- package/.bkit/templates/.github/workflows/bkit-ci.yml +53 -50
- package/.bkit/templates/.github/workflows/bkit-cloud.yml +13 -14
- package/.bkit/templates/.github/workflows/bkit-health.yml +97 -0
- package/.bkit/templates/.github/workflows/bkit-renovate.yml +5 -4
- package/.bkit/templates/.github/workflows/bkit-trivy.yml +5 -4
- package/.bkit/templates/.husky/pre-commit +2 -2
- package/.bkit/templates/.husky/pre-push +95 -35
- package/.bkit/templates/.prettierignore +6 -7
- package/.bkit/templates/.vale.ini +27 -27
- package/.bkit/templates/context/bkit-cloudrun.md +23 -15
- package/.bkit/templates/context/bkit-expo.md +70 -52
- package/.bkit/templates/context/bkit-github.md +93 -56
- package/.bkit/templates/context/bkit-inngest.md +21 -15
- package/.bkit/templates/context/bkit-maestro.md +22 -16
- package/.bkit/templates/context/bkit-playwright.md +23 -17
- package/.bkit/templates/context/bkit-resend.md +12 -10
- package/.bkit/templates/context/bkit-sentry.md +30 -20
- package/.bkit/templates/context/bkit-serena.md +8 -7
- package/.bkit/templates/context/bkit-stripe.md +14 -13
- package/.bkit/templates/context/bkit-supabase.md +16 -13
- package/.bkit/templates/context/bkit-tavily.md +14 -11
- package/.bkit/templates/context/bkit-vercel.md +27 -18
- package/.bkit/templates/docs/capabilities.md +15 -14
- package/.bkit/templates/scripts/ci/{check-accepted-divergences.mjs → bkit-check-accepted-divergences.mjs} +6 -3
- package/.bkit/templates/scripts/ci/{check-accepted-divergences.test.mjs → bkit-check-accepted-divergences.test.mjs} +3 -3
- package/.bkit/templates/scripts/ci/{check-capstone.mjs → bkit-check-capstone.mjs} +2 -2
- package/.bkit/templates/scripts/ci/{check-capstone.test.mjs → bkit-check-capstone.test.mjs} +4 -4
- package/.bkit/templates/scripts/ci/{check-dependency-audit.mjs → bkit-check-dependency-audit.mjs} +103 -18
- package/.bkit/templates/scripts/ci/{check-dependency-audit.test.mjs → bkit-check-dependency-audit.test.mjs} +31 -3
- package/.bkit/templates/scripts/ci/bkit-machine-lock.sh +387 -0
- package/.bkit/templates/scripts/ci/bkit-test-awake.mjs +75 -0
- package/.bkit/templates/scripts/ci/bkit-test-awake.test.mjs +160 -0
- package/.bkit/templates/scripts/ci/lanes/{resolve-push-base.mjs → bkit-resolve-push-base.mjs} +3 -3
- package/.bkit/templates/scripts/ci/lanes/{resolve-push-base.test.mjs → bkit-resolve-push-base.test.mjs} +4 -4
- package/.bkit/templates/scripts/cleanup/bkit-disk-prune.mjs +9 -1
- package/.bkit/templates/scripts/graphify/bkit-graph-drift.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graph-eager.py +3 -3
- package/.bkit/templates/scripts/graphify/bkit-graph-embed.py +63 -0
- package/.bkit/templates/scripts/graphify/bkit-graph-imports.py +2 -2
- package/.bkit/templates/scripts/graphify/bkit-graph-pre-push.sh +69 -18
- package/.bkit/templates/scripts/graphify/bkit-graph-repair.py +3 -3
- package/.bkit/templates/scripts/graphify/bkit-graph-surface.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graphify-graphml.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graphify-rollup.py +197 -126
- package/.bkit/templates/scripts/graphify/bkit-graphify.sh +64 -41
- package/.bkit/templates/scripts/slop/bkit-slop-type-suppression.mjs +23 -7
- package/.bkit/templates/slop-scan.config.json +9 -2
- package/CHANGELOG.md +119 -0
- package/README.md +250 -76
- package/dist/checks/after/auth.js +36 -33
- package/dist/checks/after/ci-yml.js +52 -2
- package/dist/checks/after/index.js +12 -4
- package/dist/checks/after/tools.js +73 -33
- package/dist/checks/before/context-fetch.js +218 -33
- package/dist/checks/before/env.js +1 -1
- package/dist/checks/before/index.js +6 -2
- package/dist/checks/before/repo.js +35 -1
- package/dist/checks/status-holes.js +739 -114
- package/dist/checks/tips/catalogue.js +59 -17
- package/dist/checks/tips/detect.js +30 -0
- package/dist/checks/tips/index.js +48 -6
- package/dist/cli.js +222 -84
- package/dist/commands/analysis.js +77 -53
- package/dist/commands/ci/ci.js +302 -25
- package/dist/commands/ci/pins.js +142 -0
- package/dist/commands/ci/push-gate.js +194 -0
- package/dist/commands/install/uninstall.js +23 -27
- package/dist/commands/install/upgrade.js +21 -11
- package/dist/commands/land.js +126 -10
- package/dist/commands/lib.js +244 -5
- package/dist/commands/registry.js +6 -2
- package/dist/commands/setup/setup.js +1 -0
- package/dist/commands/stats.js +69 -30
- package/dist/commands/workflow/branches.js +1232 -111
- package/dist/commands/workflow/challenge.js +90 -37
- package/dist/commands/workflow/completeness.js +97 -0
- package/dist/commands/workflow/context.js +371 -51
- package/dist/commands/workflow/counters.js +206 -40
- package/dist/commands/workflow/decisions.js +60 -17
- package/dist/commands/workflow/disk.js +213 -55
- package/dist/commands/workflow/fids.js +18 -1
- package/dist/commands/workflow/issues.js +6 -0
- package/dist/commands/workflow/ledgers.js +372 -21
- package/dist/commands/workflow/locks.js +2 -0
- package/dist/commands/workflow/plan-blocks.js +78 -14
- package/dist/commands/workflow/plans.js +205 -0
- package/dist/commands/workflow/prs.js +12 -3
- package/dist/commands/workflow/repo.js +6 -0
- package/dist/commands/workflow/roadmap.js +50 -17
- package/dist/commands/workflow/run-facts.js +23 -28
- package/dist/commands/workflow/runs.js +530 -33
- package/dist/commands/workflow/runtime.js +3 -0
- package/dist/commands/workflow/skills.js +1 -0
- package/dist/commands/workflow/status.js +166 -19
- package/dist/commands/workflow/step-table.js +61 -29
- package/dist/commands/workflow/steps.js +542 -26
- package/dist/commands/workflow/summary-findings.js +117 -0
- package/dist/commands/workflow/summary.js +154 -41
- package/dist/commands/workflow/tasks.js +378 -0
- package/dist/commands/workflow/test-blocks.js +231 -0
- package/dist/commands/workflow/tests-changed.js +172 -0
- package/dist/commands/workflow/tests.js +1013 -321
- package/dist/commands/workflow/workflows.js +191 -38
- package/dist/commands/workflow/worktrees.js +310 -42
- package/dist/install/cloud-env.js +233 -99
- package/dist/install/context-ci-workflow.js +85 -0
- package/dist/install/manifest.js +57 -12
- package/dist/install/modules/antigravity.js +3 -2
- package/dist/install/modules/cloud.js +3 -3
- package/dist/install/modules/codex.js +12 -2
- package/dist/install/modules/core.js +851 -53
- package/dist/install/modules/cursor.js +6 -6
- package/dist/install/modules/opencode.js +11 -5
- package/dist/install/template-notes.js +116 -0
- package/dist/install/version-file.js +3 -2
- package/dist/ledgers/compact.js +87 -0
- package/dist/ledgers/journal.js +233 -0
- package/dist/ledgers/status.js +115 -5
- package/dist/ledgers/telemetry.js +121 -17
- package/dist/render/hosts.js +100 -14
- package/dist/shared/bkit-latest.js +26 -4
- package/dist/shared/dep-updater.js +49 -0
- package/dist/shared/egress.js +1 -1
- package/dist/shared/package-manager.js +6 -0
- package/dist/shared/settings.js +59 -10
- package/dist/shared/timeouts.js +2 -0
- package/dist/tools/cli-doc.js +97 -0
- package/dist/tools/grammar-density.js +1 -1
- package/dist/tools/instructions-sources.js +15 -16
- package/dist/tools/instructions.js +83 -39
- package/package.json +58 -42
- package/.bkit/core/.bkit/lite.md +0 -73
- package/.bkit/core/engine/commands/workflow/workstreams.ts +0 -172
- package/.bkit/core/prompts/bkit-build-plan-execute.md +0 -124
- package/.bkit/templates/scripts/ci/bkit-local-stack-lock.sh +0 -194
- package/dist/commands/workflow/workstreams.js +0 -149
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Filler.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Hedging.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/LLMSlop.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Verbose.yml +0 -0
- /package/.bkit/templates/scripts/ci/{accepted-advisories.json → bkit-accepted-advisories.json} +0 -0
- /package/.bkit/templates/scripts/ci/{accepted-divergences.json → bkit-accepted-divergences.json} +0 -0
- /package/.bkit/templates/scripts/launchd/{launchd.env.example → bkit-launchd.env.example} +0 -0
- /package/.bkit/templates/scripts/launchd/{ai.bkit.launchd.plist → bkit-launchd.plist} +0 -0
|
@@ -1,134 +1,142 @@
|
|
|
1
|
-
<!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
|
|
2
|
-
|
|
3
1
|
<!-- bkit:start -->
|
|
4
2
|
|
|
5
3
|
{{BKIT_MANAGED_MARKER}}
|
|
6
4
|
|
|
5
|
+
<!-- Note: @bkit-audit-generate fills this template. The report lives at `docs/audits/{YYYYMM}/{slug}.md`, where `{YYYYMM}` is the UTC month of the run and `{slug}` is this file's name without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md`, where `{TIMESTAMP}` is the output of `date -u +%Y%m%dT%H%M%SZ`. NEVER overwrite an earlier report. The report is stamped with a `bkit:anchor` block that records the audited commit (`baseSha`) and branch (`baseRef`). NEVER write or edit that block by hand. -->
|
|
6
|
+
|
|
7
7
|
# Cost Audit
|
|
8
8
|
|
|
9
9
|
> [!CAUTION]
|
|
10
|
-
> **Purpose:** Judge every vendor's plan against usage, bill
|
|
10
|
+
> **Purpose:** Judge every vendor's plan against its usage, find what drives the bill, and check whether a spike would be noticed before the invoice.
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> -
|
|
15
|
-
> - NEVER copy an invoice number, payment-card detail, account id
|
|
14
|
+
> - Never change a plan, cancel a resource or edit a billing setting.
|
|
15
|
+
> - NEVER copy an invoice number, payment-card detail, account id or secret into the report.
|
|
16
16
|
|
|
17
17
|
## Universal Instructions
|
|
18
18
|
|
|
19
19
|
### Context & Goals
|
|
20
20
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
- A cut is never only saving: report what it would stop (e.g. uptime monitor disabled for billing never alerts).
|
|
24
|
-
|
|
25
|
-
**Applies when:** paid vendors, usage-billed APIs, or payment / store fees. None → one `N/A` row with evidence, never "clean".
|
|
21
|
+
Spend spans many vendors, each on its own plan, and nobody sees the whole bill. Tiers chosen before launch stop fitting, free tiers end without a deploy, and usage-billed services spike from one bug (a retry loop, a cron fan-out over every tenant, an uncapped scraper). A cut is never only a saving, so report what it would stop (e.g., an uptime monitor disabled for billing never alerts).
|
|
22
|
+
**Applies when:** the project has paid vendors, usage-billed APIs, or payment or store fees. **If** it has none → write one `N/A` row with its evidence, and never report the surface as "clean".
|
|
26
23
|
|
|
27
|
-
**Cadence:**
|
|
24
|
+
**Cadence:** Run once a month, after invoices close. Run an extra pass when a vendor or a paid plan is added or changed, when a spend alert fires, or at launch.
|
|
28
25
|
|
|
29
26
|
**Anchor Standards:**
|
|
30
27
|
|
|
31
|
-
- FinOps Framework (e.g
|
|
28
|
+
- FinOps Framework (e.g., the current edition). Capabilities this audit uses: Allocation, Anomaly Management, Budgeting, Forecasting, Unit Economics, and Rate Optimization, across Public Cloud, SaaS, and AI.
|
|
32
29
|
- FOCUS (FinOps Open Cost and Usage Specification); record the version.
|
|
33
|
-
- Vendor spend-control docs (e.g
|
|
30
|
+
- Vendor spend-control docs (e.g., host spend management, database spend cap, model-gateway guardrails, error-tracker spike protection); record the date read.
|
|
34
31
|
- bkit block of `docs/deployments.md` (bkit version from `.bkit/version`): § Disable GitHub Code Quality billing
|
|
35
32
|
|
|
36
33
|
### Scope
|
|
37
34
|
|
|
38
|
-
-
|
|
39
|
-
-
|
|
40
|
-
-
|
|
41
|
-
-
|
|
42
|
-
-
|
|
43
|
-
-
|
|
44
|
-
-
|
|
35
|
+
- List every vendor, plan and environment, including the fees that are not a vendor plan.
|
|
36
|
+
- Find the usage drivers in code, egress and metered observability.
|
|
37
|
+
- Compare unit cost with revenue.
|
|
38
|
+
- Find seats, idle resources and duplicate vendors.
|
|
39
|
+
- Check spend caps and alerts: whether a spike is noticed, and what a cap does when it is hit.
|
|
40
|
+
- Find legacy subscriptions still billing after a replacement shipped.
|
|
41
|
+
- Check currency, foreign exchange and tax.
|
|
45
42
|
|
|
46
43
|
### Out of Scope
|
|
47
44
|
|
|
48
45
|
<!-- prettier-ignore -->
|
|
49
46
|
| Excluded item | Why | Where it belongs |
|
|
50
47
|
| --- | --- | --- |
|
|
51
|
-
| Model-call limits in code | LLM
|
|
52
|
-
| Why CI minutes are high | CI performance
|
|
53
|
-
| Whether a seat's access is still justified |
|
|
54
|
-
| Editing docs to add a vendor |
|
|
55
|
-
| Changing a plan, cap, or billing setting |
|
|
48
|
+
| Model-call limits in code | The LLM audit owns them | `bkit-audit-llm` |
|
|
49
|
+
| Why CI minutes are high | The CI performance audit owns it | `bkit-audit-ci-performance` |
|
|
50
|
+
| Whether a seat's access is still justified | The security audit owns it | `bkit-audit-security` |
|
|
51
|
+
| Editing docs to add a vendor | The documentation audit owns it | `bkit-audit-documentation` |
|
|
52
|
+
| Changing a plan, cap, or billing setting | The audit only reports | A plan made from this report |
|
|
56
53
|
|
|
57
54
|
### Verdicts
|
|
58
55
|
|
|
59
|
-
|
|
56
|
+
Give one verdict per vendor, and one per paid resource inside a vendor that has several.
|
|
60
57
|
|
|
61
|
-
- **KEEP:** used, and the plan fits. Say what uses it.
|
|
62
|
-
- **RIGHT-SIZE:** used, but plan
|
|
63
|
-
- **REMOVE:** billed, nothing uses it.
|
|
64
|
-
- **ASK:** cannot tell
|
|
65
|
-
- **OTHER:** fits
|
|
58
|
+
- **KEEP:** the vendor is used, and the plan fits. Say what uses it.
|
|
59
|
+
- **RIGHT-SIZE:** the vendor is used, but the plan or tier no longer fits: it has unused headroom, or is near a hard limit. State the change, the monthly saving and what it would stop.
|
|
60
|
+
- **REMOVE:** the vendor is billed, and nothing uses it. The proof is that no code calls it and no workflow needs it (wider dead code goes to `bkit-audit-dead-code`). Use language-server references where available, and text search. An empty result is inconclusive, so re-run an empty result once before giving `REMOVE`.
|
|
61
|
+
- **ASK:** the evidence cannot tell, usually because a dashboard is unread. Name who can tell. An unread bill is `ASK`, never cheap.
|
|
62
|
+
- **OTHER:** it fits no other verdict. Say why. **If** the same `OTHER` shows up on 2 runs → propose a new verdict in `Process Improvements`.
|
|
66
63
|
|
|
67
64
|
### Pre-Flight / Dependencies
|
|
68
65
|
|
|
69
66
|
#### Core
|
|
70
67
|
|
|
71
|
-
- [ ] **Permissions:** blocked
|
|
72
|
-
- [ ] **Bills:** Steps 3-5, 8 — invoices or billing export (e.g
|
|
73
|
-
- [ ] **Code Search:** Steps 3, 6, 7 — SDKs, outbound hosts
|
|
74
|
-
- [ ] **Web:** Step 9 — fetch anchor standards. Fallback: versions
|
|
68
|
+
- [ ] **Permissions:** every blocked call is recorded.
|
|
69
|
+
- [ ] **Bills:** Steps 3-5, 8 — invoices or a billing export (e.g., FOCUS), and read-only plan, usage and cap pages (e.g., the host dashboard, the cloud billing console, a vendor API). Fallback: give an unread vendor `ASK`, and mark tax, foreign exchange and card fees `Not Checked`.
|
|
70
|
+
- [ ] **Code Search:** Steps 3, 6, 7 — SDKs, outbound hosts and reference checks (e.g., ripgrep, language-server references). Fallback: use docs and invoices only, and mark the `REMOVE` proof incomplete.
|
|
71
|
+
- [ ] **Web:** Step 9 — fetch the anchor standards. Fallback: record the versions as unread.
|
|
75
72
|
|
|
76
73
|
#### Conditional
|
|
77
74
|
|
|
78
|
-
- [ ] **Payment Reports:**
|
|
79
|
-
- [ ] **Legacy Accounts:**
|
|
75
|
+
- [ ] **Payment Reports:** **If** the project pays processor or store fees → Steps 3, 5 — fee and commission reports (e.g., the card processor, app-store proceeds). Fallback: `Not Checked`.
|
|
76
|
+
- [ ] **Legacy Accounts:** **If** the migration map names a replaced platform → Step 3 — confirm that its subscriptions are not still billing. Fallback: give it `ASK`.
|
|
80
77
|
|
|
81
78
|
### 🔒 Gate 1: Prerequisites Check
|
|
82
79
|
|
|
83
|
-
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
84
|
-
- [ ] **Gate 1 — Verified:**
|
|
80
|
+
- [ ] **Gaps named:** every missing item names its fallback and its `Not Checked` entry.
|
|
81
|
+
- [ ] **Gate 1 — Verified:** every item above, every Core item and every applicable Conditional item is checked.
|
|
85
82
|
|
|
86
83
|
### Probe Steps
|
|
87
84
|
|
|
88
|
-
- [ ] **Step 1 — Previous Run:** **Done when:**
|
|
89
|
-
- [ ] **Step 2 — Detect:** Find paid vendors, usage-billed APIs, or payment
|
|
90
|
-
- [ ] **Step 3 — Inventory:** Build list from code (SDKs, outbound hosts), docs (vendor list, secrets list)
|
|
91
|
-
- [ ] **Step 4 — Plans:**
|
|
92
|
-
-
|
|
93
|
-
-
|
|
94
|
-
-
|
|
95
|
-
-
|
|
96
|
-
-
|
|
97
|
-
-
|
|
98
|
-
-
|
|
99
|
-
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
- [ ] **Step
|
|
104
|
-
- [ ] **Step
|
|
85
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** `Previous Run` holds the closure rate, or the run is marked baseline.
|
|
86
|
+
- [ ] **Step 2 — Detect:** Find paid vendors, usage-billed APIs, or payment or store fees (manifests, outbound hosts, vendor docs). **Done when:** `Checked` holds evidence, or `N/A` is recorded → **STOP**.
|
|
87
|
+
- [ ] **Step 3 — Inventory:** Build the vendor list from code (SDKs, outbound hosts), docs (vendor list, secrets list) and the invoices the owner shared, and compare the sources. A vendor in code but not in docs is a finding (the doc fix goes to `bkit-audit-documentation`). A vendor on an invoice but nowhere in code is a `REMOVE` candidate. Count the fees that are not a vendor plan (card-processor fees, app-store commission): they grow with paying accounts and have no vendor dashboard. A replaced platform still billing after its replacement shipped is a `REMOVE` candidate. **Done when:** `Vendors` lists every vendor with its sources.
|
|
88
|
+
- [ ] **Step 4 — Plans:** For each vendor, record the plan, the billing model (flat, per seat, per use), the spend over the last three months, the usage driver, the plan limit and how close usage is to it.
|
|
89
|
+
- Split by environment. Preview or dev billed like production is a finding.
|
|
90
|
+
- For each usage-billed vendor, record the share used by preview, E2E and CI against production, by feature where the vendor can tell. E2E-driven usage on a usage-billed vendor is a `RIGHT-SIZE` candidate.
|
|
91
|
+
- For the logging environment split, start from `bkit-audit-logs`.
|
|
92
|
+
- List every free-tier resource that production or CI depends on, and its behaviour at the limit (e.g., a free DB pauses, a free monitor stops). A free tier that fails hard is a finding, even at $0 today.
|
|
93
|
+
- Compare monthly with annual billing, prepaid credits, startup programmes and committed-use discounts.
|
|
94
|
+
- **If** usage has been stable for >=3 months → propose a commitment.
|
|
95
|
+
- **Else** → propose none.
|
|
96
|
+
- Record the billing currency, card FX fees, and whether sales tax, VAT, GST or reverse charge is applied correctly, from the invoice and the card statement, not dashboards.
|
|
97
|
+
|
|
98
|
+
**Done when:** `Vendors` holds one row per vendor, with verdict, plan, billing model, three-month spend, change since the previous report, driver, limit and environment split, plus the total spend.
|
|
99
|
+
|
|
100
|
+
- [ ] **Step 5 — Unit Cost:** Divide the total monthly cost by active accounts and by active users, and split it into fixed and variable. For each metered feature, record the cost per action (e.g., per model assist, per inbound message). Compare the series with the previous report, because without unit cost the bill's health cannot be judged as usage grows. Against revenue, compare price x assumed conversion, less processor fees and store commission, with the cost per account. An account that costs more than it pays is a finding, stated with its assumptions. **Done when:** `Unit Cost` holds the figures, the trend and the assumptions.
|
|
101
|
+
- [ ] **Step 6 — Drivers:** For each usage-billed service, find the code that drives the meter and its worst case (retry, tenant fan-out, cron frequency). A multiplier with no upper bound is a finding. A model-call multiplier also names `bkit-audit-llm` (in-code limits). Other unbounded multipliers get a `docs/failures.jsonl` row, which this run writes. Give egress and data transfer their own line, because they grow with users and are unnamed in code (DB egress, cached egress, host data transfer, origin transfer, storage transforms, log-drain egress), and list the top routes or buckets by bytes. Check metered observability: log drains, host observability events, replay, and trace sample rates in code. From the drivers, forecast the monthly cost at 10x and 100x today's active accounts, and name the first vendor whose tier breaks and the month it breaks at current growth. CI minutes go to `bkit-audit-ci-performance`, so do not analyse them here. **Done when:** `Drivers` holds one row per meter, and `Forecast` holds 10x, 100x, the first tier break and its month.
|
|
102
|
+
- [ ] **Step 7 — Leftovers:** Find seats for people who no longer need them, and record only their cost here. Access review goes to `bkit-audit-security`. Find idle resources: projects, DBs, monitors, buckets and domains left from experiments. Find duplicates: two vendors doing one job. **Done when:** `Leftovers` holds one row per hit, with its monthly amount and verdict.
|
|
103
|
+
- [ ] **Step 8 — Caps:** For each usage-billed vendor, record the spend cap or budget alert, its amount and its recipients. No cap and no alert is a finding, even when spend is small. Record what happens when the cap is hit:
|
|
104
|
+
- Whether it notifies, stops spend or pauses service, and whether a hit means an outage. Notify-only may leave production serving, and pause may take production down (e.g., a host spend limit notifies unless production deploys are paused). Record the check frequency, because lag leads to overshoot.
|
|
105
|
+
- The line items the cap does not cover. Name them from each vendor's spend-cap doc (e.g., compute, log drains).
|
|
106
|
+
- Model-gateway keys (e.g., a per-key spend limit with reset, a guardrail budget). Check that keys are separate per environment or feature, so preview or one feature cannot drain production. In-code limits go to `bkit-audit-llm`, and the key-level budget stays here. No spend limit, or one key shared across environments, is a finding.
|
|
107
|
+
- Error tracker: spike protection per project, and a notification when it trips (protection turned on does not notify by itself). Check that ingest keys are rate-limited and that an on-demand budget is set. Quota exhaustion blinds `bkit-audit-logs`.
|
|
108
|
+
|
|
109
|
+
List the anomalies since the previous report: how each was found (alert or invoice), the time to find it, and the cost avoided. **If** a cap only notifies, omits line items or pauses production → record it as an owner decision between notify, cap and pause, with what each would stop. This run does not set it. **Done when:** `Caps` holds one row per usage-billed vendor, and the anomaly list.
|
|
110
|
+
|
|
111
|
+
- [ ] **Step 9 — Standards:** Confirm the current version of each anchor standard, and read what changed since the previous report's version. Give `Standards Coverage` one row per FinOps capability (Allocation, Anomaly Management, Budgeting, Forecasting, Unit Economics, and Rate Optimization), each marked checked (naming the step), not applicable (saying why) or finding. Score each capability crawl, walk or run, and name the next one to raise. Give each bkit anchor (a doc section, rule or issue shape) one row, marked followed (with the step or evidence), not applicable (saying why) or finding. A departure with no recorded reason (e.g., a `docs/decisions.jsonl` row or an accepted-divergence entry) is a finding. Mark each recommended practice the project does not apply as adopt (with a proposed step), reject (saying why) or not applicable. **Done when:** no row is empty.
|
|
112
|
+
- [ ] **Step 10 — Anything Else:** (a) Give a one-line verdict on each change since the previous report's commit that touches cost: SDKs, outbound hosts, vendors, plans, crons, schedules, fan-outs, model calls, sample rates, environment and seat changes. (b) In a bounded pass of about 30 minutes, note what a senior FinOps practitioner would flag that no step asks about. Read what happened since the previous run (incidents, log-audit reports under `docs/audits/`, new `docs/failures.jsonl` rows, vendor status incidents), and ask what no step would catch. Include the invoice charges that Steps 4-8 do not explain (a new line item, fee, tax, FX or overage). **Done when:** `Beyond the Steps` holds both passes, or says `none`. **If** one class of issue is seen on 2 runs → propose a step for it in `Process Improvements`.
|
|
105
113
|
|
|
106
114
|
### 🔒 Gate 2: Probes Check
|
|
107
115
|
|
|
108
|
-
- [ ] **Steps complete:** every step `[x]`, or
|
|
109
|
-
- [ ] **Evidence:** every finding cites
|
|
110
|
-
- [ ] **No billing writes:** no plan, resource, cap
|
|
111
|
-
- [ ] **No account secrets:** no invoice number, card detail, account id
|
|
112
|
-
- [ ] **Gate 2 — Verified:**
|
|
116
|
+
- [ ] **Steps complete:** every step is `[x]`, or skipped with evidence that its surface is absent.
|
|
117
|
+
- [ ] **Evidence:** every finding cites evidence someone can check again, and the tool that produced it.
|
|
118
|
+
- [ ] **No billing writes:** no plan, resource, cap or billing setting changed.
|
|
119
|
+
- [ ] **No account secrets:** the report holds no invoice number, card detail, account id or secret.
|
|
120
|
+
- [ ] **Gate 2 — Verified:** every item above holds.
|
|
113
121
|
|
|
114
122
|
## Findings
|
|
115
123
|
|
|
116
124
|
<!-- prettier-ignore -->
|
|
117
125
|
| Severity | Use |
|
|
118
126
|
| --- | --- |
|
|
119
|
-
| `CRITICAL` | A cap or free-tier limit that stops production with no workaround
|
|
120
|
-
| `HIGH` |
|
|
121
|
-
| `MEDIUM` |
|
|
122
|
-
| `LOW` | Docs omit a vendor the code calls
|
|
127
|
+
| `CRITICAL` | A cap or free-tier limit that stops production with no workaround. An unbounded meter that can already exhaust the account with no way to stop it |
|
|
128
|
+
| `HIGH` | A usage-billed vendor with no cap and no alert. An account that costs more than it pays. A forecast that breaks a tier inside the planning window. A legacy subscription still billing after its replacement is live |
|
|
129
|
+
| `MEDIUM` | A plan that no longer fits (`RIGHT-SIZE`). A preview, end-to-end, or CI share of a usage bill. A free tier that fails hard. A cap that only notifies, omits line items, or pauses production. A duplicate vendor. Tax or foreign exchange applied wrongly |
|
|
130
|
+
| `LOW` | Docs that omit a vendor the code calls. A seat whose access is otherwise in scope for the security audit. A currency note with no wrong charge |
|
|
123
131
|
|
|
124
|
-
Within a severity
|
|
132
|
+
Within a severity, list the larger monthly amount first. No cap and no alert on a usage-billed vendor is at least `HIGH`, even when today's spend is small.
|
|
125
133
|
|
|
126
134
|
<!-- prettier-ignore -->
|
|
127
135
|
| ID | Severity | Verdict | Location | Evidence | Expected |
|
|
128
136
|
| --- | --- | --- | --- | --- | --- |
|
|
129
137
|
| PENDING | PENDING | PENDING | PENDING | PENDING | PENDING |
|
|
130
138
|
|
|
131
|
-
|
|
139
|
+
**If** the run finds nothing → write one row: `None`.
|
|
132
140
|
|
|
133
141
|
## Report
|
|
134
142
|
|
|
@@ -193,13 +201,16 @@ PENDING
|
|
|
193
201
|
|
|
194
202
|
### 🔒 Gate 3: Report Check
|
|
195
203
|
|
|
196
|
-
- [ ] **No placeholders:** no `PENDING` left
|
|
197
|
-
- [ ] **Summary:**
|
|
198
|
-
- [ ] **
|
|
199
|
-
- [ ] **
|
|
200
|
-
- [ ] **
|
|
201
|
-
- [ ] **
|
|
202
|
-
- [ ] **
|
|
204
|
+
- [ ] **No placeholders:** no `PENDING` is left.
|
|
205
|
+
- [ ] **Summary:** `Summary` gives the count per severity.
|
|
206
|
+
- [ ] **Monthly spend:** `Summary` gives the total monthly spend.
|
|
207
|
+
- [ ] **Series:** the report shows `Vendors` spend, `Unit Cost`, and `Forecast` beside the previous report's numbers, or marks the run as baseline.
|
|
208
|
+
- [ ] **Writes:** this run changed only this report, the template's `Project Instructions` and its own `docs/failures.jsonl` rows.
|
|
209
|
+
- [ ] **Throwaways removed:** every throwaway worktree or remote target is removed.
|
|
210
|
+
- [ ] **Process Improvements:** `Universal` and `Project` are both filled.
|
|
211
|
+
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID is in exactly one `Suggested Issues` row.
|
|
212
|
+
- [ ] **Next Step:** every `Suggested Issues` row has a `Next Step`.
|
|
213
|
+
- [ ] **Gate 3 — Verified:** every item above holds.
|
|
203
214
|
|
|
204
215
|
<!-- bkit:end -->
|
|
205
216
|
|
|
@@ -1,124 +1,128 @@
|
|
|
1
|
-
<!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
|
|
2
|
-
|
|
3
1
|
<!-- bkit:start -->
|
|
4
2
|
|
|
5
3
|
{{BKIT_MANAGED_MARKER}}
|
|
6
4
|
|
|
5
|
+
<!-- Note: @bkit-audit-generate fills this template. The report lives at `docs/audits/{YYYYMM}/{slug}.md`, where `{YYYYMM}` is the UTC month of the run and `{slug}` is this file's name without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md`, where `{TIMESTAMP}` is the output of `date -u +%Y%m%dT%H%M%SZ`. NEVER overwrite an earlier report. The report is stamped with a `bkit:anchor` block that records the audited commit (`baseSha`) and branch (`baseRef`). NEVER write or edit that block by hand. -->
|
|
6
|
+
|
|
7
7
|
# Coverage Floor
|
|
8
8
|
|
|
9
9
|
> [!CAUTION]
|
|
10
|
-
> **Purpose:**
|
|
10
|
+
> **Purpose:** The ratchet of a CI coverage-floor gate: whether each committed floor still sits under the measured branch coverage, and the next floor each gated surface should move to.
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> -
|
|
15
|
-
> -
|
|
16
|
-
> -
|
|
14
|
+
> - Never edit workflows, floor files, coverage config or the gate script.
|
|
15
|
+
> - State each edit in the report instead.
|
|
16
|
+
> - Never lower a floor, add an exclusion or add an ignore hint.
|
|
17
|
+
> - NEVER run a tool that rewrites source to measure mutants or coverage.
|
|
18
|
+
> - Read the artifacts already produced.
|
|
17
19
|
|
|
18
20
|
## Universal Instructions
|
|
19
21
|
|
|
20
22
|
### Context & Goals
|
|
21
23
|
|
|
22
|
-
|
|
24
|
+
A static floor fails one way: coverage climbs, the gate stays far below it, and a large drop still passes green. Tightening off one run fails another way: branch coverage on identical code moves between runs, so the gate flakes, and a flaky gate gets disabled. The failure this audit targets is a floor left far under measured coverage, or a floor ratcheted onto noise.
|
|
23
25
|
|
|
24
|
-
**Applies when:** coverage floor gate exists.
|
|
26
|
+
**Applies when:** a coverage floor gate exists. **If** none exists → write one `N/A` row with its evidence, and never report the surface as "clean".
|
|
25
27
|
|
|
26
|
-
**Cadence:**
|
|
28
|
+
**Cadence:** Run once a month. Run an extra pass after a coverage lift, a runner or provider version change, an include, exclude or measured-glob change, or a new package or app under a gated glob.
|
|
27
29
|
|
|
28
30
|
**Anchor Standards:**
|
|
29
31
|
|
|
30
|
-
- Coverage practice for a gate (e.g
|
|
31
|
-
- Coverage guide for pinned runner and provider (e.g
|
|
32
|
-
- Patch (diff) coverage (e.g
|
|
33
|
-
- Ratchet tooling (e.g
|
|
32
|
+
- Coverage practice for a gate (e.g., Google, "Code Coverage Best Practices"; Ivanković et al., "Code Coverage at Google").
|
|
33
|
+
- Coverage guide for pinned runner and provider (e.g., Vitest coverage, including `coverage.include` and AST-aware remapping; Jest coverage; Istanbul). Record major pinned.
|
|
34
|
+
- Patch (diff) coverage (e.g., Codecov patch status; PR coverage-comment changed-file mode). Record action version.
|
|
35
|
+
- Ratchet tooling (e.g., Betterer). Record version read.
|
|
34
36
|
- Goodhart's law: coverage is a signal, not a goal.
|
|
35
37
|
|
|
36
38
|
### Scope
|
|
37
39
|
|
|
38
|
-
-
|
|
39
|
-
- Sample >=10 full runs per gated surface
|
|
40
|
-
-
|
|
41
|
-
-
|
|
42
|
-
-
|
|
43
|
-
-
|
|
44
|
-
-
|
|
45
|
-
-
|
|
40
|
+
- Check the floor history: no floor dropped, no new exclusion and no new ignore hint.
|
|
41
|
+
- Sample >=10 full runs per gated surface.
|
|
42
|
+
- Ratchet from the lowest observation, never from one run.
|
|
43
|
+
- Compare the denominator: the files the glob should cover against the files in the summary.
|
|
44
|
+
- Measure the variance on identical code across two runs, and its cause (tests or tooling).
|
|
45
|
+
- Recommend a floor for each gated surface, each surface's floor on its own.
|
|
46
|
+
- List the report-only surfaces, including a named surface that emits no coverage.
|
|
47
|
+
- Report the high-risk modules, as report-only.
|
|
48
|
+
- Check the patch coverage of the changes merged since the previous report.
|
|
46
49
|
|
|
47
50
|
### Out of Scope
|
|
48
51
|
|
|
49
52
|
<!-- prettier-ignore -->
|
|
50
53
|
| Excluded item | Why | Where it belongs |
|
|
51
54
|
| --- | --- | --- |
|
|
52
|
-
| Whether tests check the right behaviour, can fail, or run |
|
|
53
|
-
| Conditional or timing-dependent tests behind a spread |
|
|
54
|
-
| Patch coverage as a new required check |
|
|
55
|
-
| Changing floors, exclusions, ignore hints, or workflows |
|
|
55
|
+
| Whether tests check the right behaviour, can fail, or run | The testing audit owns test quality | `bkit-audit-testing` |
|
|
56
|
+
| Conditional or timing-dependent tests behind a spread | The flakes audit owns them | `bkit-audit-flakes` |
|
|
57
|
+
| Patch coverage as a new required check | The gates audit owns gate ranking | `bkit-audit-gates` |
|
|
58
|
+
| Changing floors, exclusions, ignore hints, or workflows | The audit only reports | A plan made from this report |
|
|
56
59
|
|
|
57
60
|
### Pre-Flight / Dependencies
|
|
58
61
|
|
|
59
62
|
#### Core
|
|
60
63
|
|
|
61
|
-
- [ ] **Permissions:** blocked
|
|
62
|
-
- [ ] **CI History:** Steps 4, 5, 7, 10 — branch coverage from full CI runs
|
|
63
|
-
- [ ] **Coverage Artifacts:** Steps 6, 7, 11, 12 — summary and per-file coverage
|
|
64
|
-
- [ ] **Git History:** Steps 3, 9, 15 — `git log -L` / `git log -p` on floor values and coverage config. Fallback: current tree only
|
|
65
|
-
- [ ] **Web:** Step 14 — fetch anchor standards. Fallback: versions
|
|
64
|
+
- [ ] **Permissions:** every blocked call is recorded.
|
|
65
|
+
- [ ] **CI History:** Steps 4, 5, 7, 10 — branch coverage from full CI runs (e.g., `gh run list` / `gh run download`, GitLab pipelines, Codecov). Fallback: fill the gap with local re-runs, and mark every local observation.
|
|
66
|
+
- [ ] **Coverage Artifacts:** Steps 6, 7, 11, 12 — summary and per-file coverage (e.g., `coverage-summary.json`, `coverage-final.json`, lcov). Fallback: use percentages only, and mark file-level claims `Not Checked`.
|
|
67
|
+
- [ ] **Git History:** Steps 3, 9, 15 — `git log -L` / `git log -p` on the floor values and coverage config. Fallback: use the current tree only, and mark history `Not Checked`.
|
|
68
|
+
- [ ] **Web:** Step 14 — fetch the anchor standards. Fallback: record the versions as unread.
|
|
66
69
|
|
|
67
70
|
#### Conditional
|
|
68
71
|
|
|
69
|
-
- [ ] **Local Runner:**
|
|
70
|
-
- [ ] **Mutation Score:**
|
|
71
|
-
- [ ] **Patch Source:**
|
|
72
|
+
- [ ] **Local Runner:** **If** a gated surface has fewer than ten CI observations, **or** CI keeps no per-file coverage artifact → Steps 4, 7 — re-run that surface's coverage command in a detached worktree, removed before the report finishes. Fallback: do not tighten that surface, and record the shortfall in `Not Checked`.
|
|
73
|
+
- [ ] **Mutation Score:** **If** a mutation report or a `bkit-audit-testing` report exists → Step 9 — the latest score beside each gated surface. Fallback: report movement without a mutation comparison, and record that in `Not Checked`. Do not start a run that rewrites source.
|
|
74
|
+
- [ ] **Patch Source:** **If** changed-file coverage is reported → Step 12 — the changed-file coverage (e.g., the PR coverage comment's changed-file table, Codecov patch status). Fallback: use the diff plus per-file coverage. **If** neither exists → mark it `Not Checked`.
|
|
72
75
|
|
|
73
76
|
### 🔒 Gate 1: Prerequisites Check
|
|
74
77
|
|
|
75
|
-
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
76
|
-
- [ ] **Gate 1 — Verified:**
|
|
78
|
+
- [ ] **Gaps named:** every missing item names its fallback and its `Not Checked` entry.
|
|
79
|
+
- [ ] **Gate 1 — Verified:** every item above, every Core item and every applicable Conditional item is checked.
|
|
77
80
|
|
|
78
81
|
### Probe Steps
|
|
79
82
|
|
|
80
|
-
- [ ] **Step 1 — Previous Run:** **Done when:**
|
|
81
|
-
- [ ] **Step 2 — Detect:** Find coverage floor gate (CI job, script
|
|
82
|
-
- [ ] **Step 3 — History:** Since previous report's commit
|
|
83
|
-
- [ ] **Step 4 — Sample:**
|
|
84
|
-
- [ ] **Step 5 — Lowest:**
|
|
85
|
-
- [ ] **Step 6 — Denominator:**
|
|
86
|
-
- [ ] **Step 7 — Variance:**
|
|
87
|
-
- [ ] **Step 8 — Floor:**
|
|
88
|
-
- [ ] **Step 9 — Movement:** Name tests that caused lift
|
|
89
|
-
- [ ] **Step 10 — Report-Only:** List report-only surfaces and numbers. Not gating is part of record:
|
|
90
|
-
- [ ] **Step 11 — Modules:**
|
|
91
|
-
- [ ] **Step 12 — Patch:**
|
|
92
|
-
- [ ] **Step 13 — Recommend:**
|
|
93
|
-
- [ ] **Step 14 — Standards:** Confirm current version
|
|
94
|
-
- [ ] **Step 15 — Anything Else:** (a)
|
|
83
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** `Previous Run` holds the closure rate, or the run is marked baseline.
|
|
84
|
+
- [ ] **Step 2 — Detect:** Find the coverage floor gate (a CI job, script or threshold that fails below a committed number) and where the number is stored. List the hard-gated and the report-only surfaces. The metric is branch coverage, unless a recorded decision names another. Line, statement and function percentages are not floors. **If** there is no gate → write `N/A` and **STOP**. **Done when:** `Checked` holds gate path, metric, surface list, or `N/A` is recorded → **STOP**.
|
|
85
|
+
- [ ] **Step 3 — History:** Since the previous report's commit, check that no floor dropped (`git log -L` on each floor value and on the coverage include and exclude). A floor never moves down: a drop is fixed, not accommodated. Diff the exclusions. Count the ignore hints under each gated glob, then and now (e.g., `v8 ignore`, `istanbul ignore`, `c8 ignore`, `# pragma: no cover`). Each new exclusion and each new ignore hint is a finding, with its commit. **Done when:** `History` lists every floor change, new exclusion and new ignore hint, or `none`.
|
|
86
|
+
- [ ] **Step 4 — Sample:** For each gated surface, take >=10 full runs: on the same commit, or the most recent full CI runs on the default branch, else on the integration branch. Record which source each run came from. Use CI history first, and re-run locally only to fill a gap. Mark the local observations. Record every run id. Include no partial lane. Never ratchet off a single run. **Done when:** `Sample` holds >=10 observations per gated surface, or names the shortfall and leaves that floor untightened.
|
|
87
|
+
- [ ] **Step 5 — Lowest:** For each gated surface, record every observation, and ratchet from the lowest, not the latest. Write covered/total beside each percentage. Spread = max − min. A spread above two points is a finding (non-deterministic coverage). `Variance` finds the cause, and `Floor` does not tighten until the spread is fixed. **Done when:** `Sample` holds one row per surface, with the observations, covered/total, the lowest and the spread.
|
|
88
|
+
- [ ] **Step 6 — Denominator:** **If** the runner omits a file that no test imports unless the include list names it (e.g., Vitest >=4) → compare the summary's file count with `git ls-files` for the glob the surface should cover, and list the source files missing from the report. A missing file inflates the percentage, which is a finding. **If** the project uses another runner → check its equivalent omission, or skip with evidence. **Done when:** `Denominator` holds the missing files, or the skip is recorded.
|
|
89
|
+
- [ ] **Step 7 — Variance:** Coverage on identical code should match. Take two runs on the same commit, diff the per-file coverage (e.g., `coverage-final.json`), and list the files whose covered branches differ. Conditional or timing-dependent tests go to `bkit-audit-flakes`. **Done when:** `Variance` holds the file list, or records that the runs are identical.
|
|
90
|
+
- [ ] **Step 8 — Floor:** The recommended floor = the lowest observation, rounded down, minus one point. **If** the spread is above two points → keep the current floor. **If** the recommended floor is below the current floor → leave the floor as it is, and record the drop as a finding. Surfaces ratchet separately, so do not nudge a surface whose tests did not change. **Done when:** `Recommendation` holds the current, lowest and recommended floor, and whether to move, for each gated surface.
|
|
91
|
+
- [ ] **Step 9 — Movement:** Name the tests that caused a lift, and whether they are durable (run on every tier invocation) or in a conditional lane. A ratchet with no named cause is a finding. Before crediting tests, record the runner, the coverage-provider package and the provider for each run in `Sample`, and diff the lockfile between the oldest and newest runs. **If** the provider changed (e.g., AST-aware remapping) → branch counts move with no test change, so do not credit the tests with that movement. Put the latest mutation score beside each gated surface. Coverage up with mutation flat is a finding, because the tests cover code but check nothing, and it goes to `bkit-audit-testing`. **Done when:** `Movement` names the cause for each surface, or the tooling change that explains it.
|
|
92
|
+
- [ ] **Step 10 — Report-Only:** List the report-only surfaces and their numbers. Not gating is part of the record: each surface stays report-only (saying why) or has a date to decide. A surface reported across runs with no decision is a finding. A project-named surface with no coverage is a finding, not an omission. Record any E2E-collected coverage, or server and route code that no unit test reaches (e.g., V8/Istanbul through a browser runner and merger). **If** there is none → say so. **Done when:** `Report-Only` holds one row per surface, with the number, the decision and whether coverage is collected.
|
|
93
|
+
- [ ] **Step 11 — Modules:** For each project-named high-risk area, record the branch coverage and the absolute number of uncovered branches. Name the files, so the next run measures the same set. This is report-only and does not move the floor. **Done when:** `Modules` holds one row per area, with its files, branch % and uncovered branches.
|
|
94
|
+
- [ ] **Step 12 — Patch:** List every change merged since the previous report whose changed lines had branch coverage below a stated bar, and state the bar (default 90% per commit, unless the project states another). Patch coverage names the change that dropped coverage, which the aggregate gate does not. A change below the bar is a finding, and a gate candidate for `bkit-audit-gates`. **Done when:** `Patch` lists every below-bar change, or `none`, and states the bar.
|
|
95
|
+
- [ ] **Step 13 — Recommend:** For each surface, state old floor → new floor and the edit (file and value). Do not apply it. A workflow change in the edit needs explicit human confirmation. Note whether the previous report's recommended floor matches the tree. For each surface, write one line on what red means: often an untested file under the glob, not removed coverage. **If** CI does not print the recommended next floor (the lowest of the sample, rounded down, minus one point) beside the current one → propose that in `Process Improvements`. **Done when:** `Recommendation` holds the edit text and the red-means line, and the tree is unchanged.
|
|
96
|
+
- [ ] **Step 14 — Standards:** Confirm the current version of each anchor standard, and read what changed since the previous report's version. Give `Standards Coverage` one row per anchor practice for a coverage gate (e.g., the per-commit bar, include-list guidance, patch coverage, ratchet tooling), each marked checked (naming the step), not applicable (saying why) or finding. Mark each recommended practice the project does not apply as adopt (with a proposed step), reject (saying why) or not applicable. **Done when:** no row is empty.
|
|
97
|
+
- [ ] **Step 15 — Anything Else:** (a) Give a one-line verdict on each change since the previous report's commit that touches coverage measurement: runner and provider versions, include and exclude, ignore hints, measured globs, floor files, the gate script, the merge script, the floor step, and new packages and apps. (b) In a bounded pass of about 30 minutes, note what a senior test engineer owning CI quality gates would flag that no step asks about. Read what happened since the previous run (incidents, `bkit-audit-logs` reports, new `docs/failures.jsonl` rows, vendor status), and ask what no step would catch. **Done when:** `Beyond the Steps` holds both passes, or says `none`. **If** one class of issue is seen on 2 runs → propose a step for it in `Process Improvements`.
|
|
95
98
|
|
|
96
99
|
### 🔒 Gate 2: Probes Check
|
|
97
100
|
|
|
98
|
-
- [ ] **Steps complete:** every step `[x]`, or
|
|
99
|
-
- [ ] **Evidence:** every finding cites
|
|
100
|
-
- [ ] **Floors untouched:** no workflow, floor file, coverage config
|
|
101
|
-
- [ ] **Sample:** every gated surface has >=10 full-run observations, or named shortfall
|
|
102
|
-
- [ ] **
|
|
101
|
+
- [ ] **Steps complete:** every step is `[x]`, or skipped with evidence that its surface is absent.
|
|
102
|
+
- [ ] **Evidence:** every finding cites evidence someone can check again, and the tool that produced it.
|
|
103
|
+
- [ ] **Floors untouched:** no workflow, floor file, coverage config or gate script changed.
|
|
104
|
+
- [ ] **Sample:** every gated surface has >=10 full-run observations, or a named shortfall with its floor not tightened.
|
|
105
|
+
- [ ] **Full runs only:** the sample holds no partial lane.
|
|
106
|
+
- [ ] **Gate 2 — Verified:** every item above holds.
|
|
103
107
|
|
|
104
108
|
## Findings
|
|
105
109
|
|
|
106
110
|
<!-- prettier-ignore -->
|
|
107
111
|
| Severity | Use |
|
|
108
112
|
| --- | --- |
|
|
109
|
-
| `CRITICAL` |
|
|
110
|
-
| `HIGH` |
|
|
111
|
-
| `MEDIUM` |
|
|
112
|
-
| `LOW` |
|
|
113
|
+
| `CRITICAL` | The floor gate was removed, or a gated surface was dropped from it with no replacement. The gate passes when the coverage summary is missing |
|
|
114
|
+
| `HIGH` | A floor was lowered. A new exclusion or ignore hint under a gated glob. A spread above two points. Source files in the glob missing from the summary |
|
|
115
|
+
| `MEDIUM` | The recommended floor is below the current one and the drop is left in place. Coverage up while the mutation score stays flat. A report-only surface with no stay-or-gate decision. A high-risk module with uncovered branches. A change merged below the patch bar. Movement with no named cause |
|
|
116
|
+
| `LOW` | A comment or prose in a floor file that does not change the number. Report-only drift inside a recorded stay-report-only decision |
|
|
113
117
|
|
|
114
|
-
Within a severity
|
|
118
|
+
Within a severity, list gated surfaces first, then exclusions.
|
|
115
119
|
|
|
116
120
|
<!-- prettier-ignore -->
|
|
117
121
|
| ID | Severity | Location | Evidence | Expected |
|
|
118
122
|
| --- | --- | --- | --- | --- |
|
|
119
123
|
| PENDING | PENDING | PENDING | PENDING | PENDING |
|
|
120
124
|
|
|
121
|
-
|
|
125
|
+
**If** the run finds nothing → write one row: `None`.
|
|
122
126
|
|
|
123
127
|
## Report
|
|
124
128
|
|
|
@@ -195,13 +199,16 @@ PENDING
|
|
|
195
199
|
|
|
196
200
|
### 🔒 Gate 3: Report Check
|
|
197
201
|
|
|
198
|
-
- [ ] **No placeholders:** no `PENDING` left
|
|
199
|
-
- [ ] **Summary:**
|
|
200
|
-
- [ ] **
|
|
201
|
-
- [ ] **
|
|
202
|
-
- [ ] **
|
|
203
|
-
- [ ] **
|
|
204
|
-
- [ ] **
|
|
202
|
+
- [ ] **No placeholders:** no `PENDING` is left.
|
|
203
|
+
- [ ] **Summary:** `Summary` gives the count per severity.
|
|
204
|
+
- [ ] **Floors:** `Summary` gives old floor → recommended floor for each gated surface.
|
|
205
|
+
- [ ] **Series:** the report shows `Sample` spreads and the floor series beside the previous report's numbers, or marks the run as baseline.
|
|
206
|
+
- [ ] **Writes:** this run changed only this report, the template's `Project Instructions` and its own `docs/failures.jsonl` rows.
|
|
207
|
+
- [ ] **Throwaways removed:** every throwaway worktree or remote target is removed.
|
|
208
|
+
- [ ] **Process Improvements:** `Universal` and `Project` are both filled.
|
|
209
|
+
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID is in exactly one `Suggested Issues` row.
|
|
210
|
+
- [ ] **Next Step:** every `Suggested Issues` row has a `Next Step`.
|
|
211
|
+
- [ ] **Gate 3 — Verified:** every item above holds.
|
|
205
212
|
|
|
206
213
|
<!-- bkit:end -->
|
|
207
214
|
|