@cbartens/bkit 5.5.1 → 6.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.bkit/README.md +167 -154
- package/.bkit/core/.agents/context.md +95 -29
- package/.bkit/core/.agents/docs.md +18 -15
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-gateguard.cjs +1 -0
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-guardrails.cjs +6 -5
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-readguard.cjs +1 -0
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-rules.cjs +65 -0
- package/.bkit/core/.agents/rules.md +112 -102
- package/.bkit/core/.agents/skills.md +63 -15
- package/.bkit/core/.bkit/README.md +167 -154
- package/.bkit/core/.bkit/authoring.md +747 -0
- package/.bkit/core/.bkit/cli.md +407 -0
- package/.bkit/core/.bkit/cloud.md +283 -195
- package/.bkit/core/.bkit/comparison.md +45 -43
- package/.bkit/core/.bkit/install.md +166 -95
- package/.bkit/core/.bkit/jsonl.md +339 -196
- package/.bkit/core/.bkit/pipeline.md +142 -145
- package/.bkit/core/.bkit/prompts.md +295 -264
- package/.bkit/core/.bkit/providers.md +460 -277
- package/.bkit/core/.bkit/router.md +69 -0
- package/.bkit/core/.bkit/templates.md +530 -256
- package/.bkit/core/.bkit/trackers.md +116 -65
- package/.bkit/core/.bkit/workflows.md +742 -618
- package/.bkit/core/.claude/hooks/bkit-gateguard.cjs +2 -2
- package/.bkit/core/.claude/hooks/bkit-guardrails.cjs +3 -3
- package/.bkit/core/.claude/hooks/bkit-preflight.sh +45 -14
- package/.bkit/core/.codex/hooks/bkit-guardrails.cjs +3 -3
- package/.bkit/core/.codex/hooks/bkit-rules.cjs +57 -0
- package/.bkit/core/.cursor/hooks/bkit-guardrails.cjs +7 -7
- package/.bkit/core/.cursor/hooks/bkit-preflight.sh +47 -12
- package/.bkit/core/.opencode/plugin/bkit-rules.js +89 -0
- package/.bkit/core/AGENTS.md +6 -80
- package/.bkit/core/docs/templates/bkit-audit-accessibility.md +128 -130
- package/.bkit/core/docs/templates/bkit-audit-adoption.md +104 -94
- package/.bkit/core/docs/templates/bkit-audit-agents.md +96 -98
- package/.bkit/core/docs/templates/bkit-audit-aio-seo.md +100 -103
- package/.bkit/core/docs/templates/bkit-audit-analytics.md +141 -107
- package/.bkit/core/docs/templates/bkit-audit-annotations.md +99 -97
- package/.bkit/core/docs/templates/bkit-audit-app-performance.md +122 -121
- package/.bkit/core/docs/templates/bkit-audit-app-store.md +158 -101
- package/.bkit/core/docs/templates/bkit-audit-architecture.md +114 -96
- package/.bkit/core/docs/templates/bkit-audit-ci-performance.md +76 -97
- package/.bkit/core/docs/templates/bkit-audit-config.md +123 -101
- package/.bkit/core/docs/templates/bkit-audit-cost.md +86 -99
- package/.bkit/core/docs/templates/bkit-audit-coverage.md +76 -93
- package/.bkit/core/docs/templates/bkit-audit-database.md +102 -95
- package/.bkit/core/docs/templates/bkit-audit-dead-code.md +165 -108
- package/.bkit/core/docs/templates/bkit-audit-deferrals.md +115 -109
- package/.bkit/core/docs/templates/bkit-audit-dependencies.md +265 -113
- package/.bkit/core/docs/templates/bkit-audit-documentation.md +119 -118
- package/.bkit/core/docs/templates/bkit-audit-duplication.md +140 -111
- package/.bkit/core/docs/templates/bkit-audit-failures.md +111 -106
- package/.bkit/core/docs/templates/bkit-audit-flakes.md +100 -99
- package/.bkit/core/docs/templates/bkit-audit-gates.md +107 -103
- package/.bkit/core/docs/templates/bkit-audit-issues.md +105 -105
- package/.bkit/core/docs/templates/bkit-audit-llm.md +134 -85
- package/.bkit/core/docs/templates/bkit-audit-logs.md +124 -125
- package/.bkit/core/docs/templates/bkit-audit-models.md +99 -78
- package/.bkit/core/docs/templates/bkit-audit-privacy.md +56 -76
- package/.bkit/core/docs/templates/bkit-audit-prompts.md +63 -78
- package/.bkit/core/docs/templates/bkit-audit-red-team.md +125 -93
- package/.bkit/core/docs/templates/bkit-audit-reliability.md +97 -96
- package/.bkit/core/docs/templates/bkit-audit-repo.md +170 -98
- package/.bkit/core/docs/templates/bkit-audit-security.md +207 -121
- package/.bkit/core/docs/templates/bkit-audit-slop.md +78 -95
- package/.bkit/core/docs/templates/bkit-audit-testing.md +163 -139
- package/.bkit/core/docs/templates/bkit-audit-usability.md +114 -96
- package/.bkit/core/docs/templates/bkit-plan.md +158 -281
- package/.bkit/core/engine/checks/after/auth.ts +40 -37
- package/.bkit/core/engine/checks/after/ci-yml.ts +104 -31
- package/.bkit/core/engine/checks/after/github.ts +95 -17
- package/.bkit/core/engine/checks/after/index.ts +21 -8
- package/.bkit/core/engine/checks/after/suppression.ts +8 -5
- package/.bkit/core/engine/checks/after/tools.ts +125 -123
- package/.bkit/core/engine/checks/before/context-fetch.ts +227 -33
- package/.bkit/core/engine/checks/before/env.ts +3 -3
- package/.bkit/core/engine/checks/before/files.ts +5 -4
- package/.bkit/core/engine/checks/before/index.ts +25 -1
- package/.bkit/core/engine/checks/before/repo.ts +208 -2
- package/.bkit/core/engine/checks/status-holes.ts +916 -123
- package/.bkit/core/engine/checks/tips/catalogue.ts +82 -20
- package/.bkit/core/engine/checks/tips/detect.ts +66 -0
- package/.bkit/core/engine/checks/tips/index.ts +52 -11
- package/.bkit/core/engine/cli.ts +251 -130
- package/.bkit/core/engine/commands/analysis.ts +282 -199
- package/.bkit/core/engine/commands/ci/ci.ts +363 -29
- package/.bkit/core/engine/commands/ci/pins.ts +172 -0
- package/.bkit/core/engine/commands/ci/push-gate.ts +223 -0
- package/.bkit/core/engine/commands/comply.ts +77 -103
- package/.bkit/core/engine/commands/install/doctor.ts +2 -2
- package/.bkit/core/engine/commands/install/install.ts +57 -18
- package/.bkit/core/engine/commands/install/uninstall.ts +86 -55
- package/.bkit/core/engine/commands/install/upgrade.ts +32 -15
- package/.bkit/core/engine/commands/land.ts +139 -18
- package/.bkit/core/engine/commands/lib.ts +381 -22
- package/.bkit/core/engine/commands/registry.ts +10 -2
- package/.bkit/core/engine/commands/setup/setup.ts +8 -7
- package/.bkit/core/engine/commands/stats.ts +260 -0
- package/.bkit/core/engine/commands/tips.ts +10 -5
- package/.bkit/core/engine/commands/workflow/branches.ts +2215 -176
- package/.bkit/core/engine/commands/workflow/challenge.ts +748 -0
- package/.bkit/core/engine/commands/workflow/completeness.ts +109 -0
- package/.bkit/core/engine/commands/workflow/context.ts +1249 -47
- package/.bkit/core/engine/commands/workflow/counters.ts +483 -95
- package/.bkit/core/engine/commands/workflow/decisions.ts +585 -0
- package/.bkit/core/engine/commands/workflow/disk.ts +238 -56
- package/.bkit/core/engine/commands/workflow/fids.ts +811 -33
- package/.bkit/core/engine/commands/workflow/issues.ts +413 -64
- package/.bkit/core/engine/commands/workflow/ledgers.ts +1333 -36
- package/.bkit/core/engine/commands/workflow/locks.ts +28 -2
- package/.bkit/core/engine/commands/workflow/plan-blocks.ts +691 -0
- package/.bkit/core/engine/commands/workflow/plan-versions.ts +21 -0
- package/.bkit/core/engine/commands/workflow/plans.ts +257 -0
- package/.bkit/core/engine/commands/workflow/prs.ts +178 -33
- package/.bkit/core/engine/commands/workflow/repo.ts +66 -8
- package/.bkit/core/engine/commands/workflow/roadmap.ts +72 -22
- package/.bkit/core/engine/commands/workflow/run-facts.ts +492 -0
- package/.bkit/core/engine/commands/workflow/runs.ts +827 -79
- package/.bkit/core/engine/commands/workflow/runtime.ts +25 -1
- package/.bkit/core/engine/commands/workflow/skills.ts +1 -0
- package/.bkit/core/engine/commands/workflow/status.ts +329 -33
- package/.bkit/core/engine/commands/workflow/step-table.ts +381 -0
- package/.bkit/core/engine/commands/workflow/steps.ts +841 -105
- package/.bkit/core/engine/commands/workflow/summary-findings.ts +177 -0
- package/.bkit/core/engine/commands/workflow/summary.ts +544 -0
- package/.bkit/core/engine/commands/workflow/tasks.ts +409 -0
- package/.bkit/core/engine/commands/workflow/test-blocks.ts +254 -0
- package/.bkit/core/engine/commands/workflow/tests-changed.ts +201 -0
- package/.bkit/core/engine/commands/workflow/tests.ts +2124 -330
- package/.bkit/core/engine/commands/workflow/tracker.ts +1195 -0
- package/.bkit/core/engine/commands/workflow/workflows.ts +1197 -235
- package/.bkit/core/engine/commands/workflow/worktrees.ts +543 -24
- package/.bkit/core/engine/install/cloud-env.ts +251 -121
- package/.bkit/core/engine/install/context-ci-workflow.ts +102 -0
- package/.bkit/core/engine/install/manifest.ts +67 -17
- package/.bkit/core/engine/install/modules/antigravity.ts +3 -2
- package/.bkit/core/engine/install/modules/cloud.ts +24 -16
- package/.bkit/core/engine/install/modules/codex.ts +15 -2
- package/.bkit/core/engine/install/modules/core.ts +1196 -45
- package/.bkit/core/engine/install/modules/cursor.ts +6 -6
- package/.bkit/core/engine/install/modules/opencode.ts +18 -6
- package/.bkit/core/engine/install/template-notes.ts +130 -0
- package/.bkit/core/engine/install/version-file.ts +9 -4
- package/.bkit/core/engine/install/write.ts +5 -0
- package/.bkit/core/engine/ledgers/compact.ts +124 -0
- package/.bkit/core/engine/ledgers/journal.ts +291 -0
- package/.bkit/core/engine/ledgers/migrate.ts +92 -0
- package/.bkit/core/engine/ledgers/status.ts +249 -21
- package/.bkit/core/engine/ledgers/telemetry.ts +1397 -253
- package/.bkit/core/engine/render/hosts.ts +118 -14
- package/.bkit/core/engine/render/models.ts +12 -0
- package/.bkit/core/engine/shared/bkit-latest.ts +26 -4
- package/.bkit/core/engine/shared/branches.ts +4 -1
- package/.bkit/core/engine/shared/dep-updater.ts +60 -0
- package/.bkit/core/engine/shared/egress.ts +8 -1
- package/.bkit/core/engine/shared/json.ts +25 -7
- package/.bkit/core/engine/shared/package-manager.ts +9 -0
- package/.bkit/core/engine/shared/settings.ts +243 -31
- package/.bkit/core/engine/shared/text.ts +361 -0
- package/.bkit/core/engine/shared/timeouts.ts +2 -0
- package/.bkit/core/engine/tools/cli-doc.ts +111 -0
- package/.bkit/core/engine/tools/grammar-density.ts +215 -0
- package/.bkit/core/engine/tools/instructions-sources.ts +832 -0
- package/.bkit/core/engine/tools/instructions.ts +1802 -0
- package/.bkit/core/hooks/antigravity-hook.js +25 -10
- package/.bkit/core/hooks/gateguard-engine.js +54 -29
- package/.bkit/core/hooks/guardrails-shell.js +4 -5
- package/.bkit/core/hooks/guardrails-tiers.js +10 -4
- package/.bkit/core/hooks/preflight-linear.sh +9 -9
- package/.bkit/core/hooks/preflight-rules.sh +8 -0
- package/.bkit/core/hooks/readguard-engine.js +29 -10
- package/.bkit/core/hooks/session-rules-advisory.js +61 -0
- package/.bkit/core/prompts/bkit-audit-generate.md +202 -94
- package/.bkit/core/prompts/bkit-build-challenge-1.md +76 -34
- package/.bkit/core/prompts/bkit-build-challenge-2.md +85 -38
- package/.bkit/core/prompts/bkit-build-install-skills.md +50 -28
- package/.bkit/core/prompts/bkit-build-plan-convert.md +85 -0
- package/.bkit/core/prompts/bkit-build-plan-current.md +59 -36
- package/.bkit/core/prompts/bkit-build-plan-gaps-remediate.md +101 -36
- package/.bkit/core/prompts/bkit-build-plan-tasks.md +149 -0
- package/.bkit/core/prompts/bkit-build-plan-tests.md +120 -0
- package/.bkit/core/prompts/bkit-build-plan-verify.md +104 -43
- package/.bkit/core/prompts/bkit-debug-abort-handover.md +47 -32
- package/.bkit/core/prompts/bkit-debug-failure-diagnose.md +109 -57
- package/.bkit/core/prompts/bkit-debug-failure-remediate.md +65 -39
- package/.bkit/core/prompts/bkit-debug-failure-reproduce.md +53 -31
- package/.bkit/core/prompts/bkit-debug-failure-soak.md +65 -34
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-1.md +46 -31
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-2.md +45 -31
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-3.md +43 -30
- package/.bkit/core/prompts/bkit-design-generate.md +80 -46
- package/.bkit/core/prompts/bkit-design-organise.md +83 -32
- package/.bkit/core/prompts/bkit-design-review-1.md +70 -50
- package/.bkit/core/prompts/bkit-design-review-2.md +65 -46
- package/.bkit/core/prompts/bkit-design-spike.md +64 -40
- package/.bkit/core/prompts/bkit-design-update.md +53 -28
- package/.bkit/core/prompts/bkit-plan-generate.md +101 -29
- package/.bkit/core/prompts/bkit-plan-review-1.md +78 -41
- package/.bkit/core/prompts/bkit-plan-review-2.md +78 -35
- package/.bkit/core/prompts/bkit-plan-scope-check.md +116 -0
- package/.bkit/core/prompts/bkit-plan-update.md +86 -31
- package/.bkit/core/prompts/bkit-ship-conflict-resolve.md +94 -50
- package/.bkit/core/prompts/bkit-ship-deferred-sweep.md +91 -41
- package/.bkit/core/prompts/bkit-ship-update-docs.md +45 -24
- package/.bkit/core/prompts/bkit-utility-compress-markdown.md +65 -49
- package/.bkit/core/prompts/bkit-utility-update-decisions.md +87 -42
- package/.bkit/core/prompts/bkit-utility-update-deferrals.md +81 -37
- package/.bkit/core/prompts/bkit-utility-update-documentation.md +43 -25
- package/.bkit/core/prompts/bkit-utility-update-failures.md +84 -37
- package/.bkit/core/prompts/bkit-utility-update-knowledge.md +88 -42
- package/.bkit/core/prompts/bkit-utility-update-roadmap.md +92 -39
- package/.bkit/core/prompts/bkit-utility-update-tracker.md +159 -60
- package/.bkit/core/workflows/bkit-auto.md +89 -58
- package/.bkit/core/workflows/bkit-build.md +967 -517
- package/.bkit/core/workflows/bkit-debug.md +555 -387
- package/.bkit/core/workflows/bkit-design.md +319 -283
- package/.bkit/core/workflows/bkit-plan.md +319 -342
- package/.bkit/core/workflows/bkit-ship.md +525 -328
- package/.bkit/shared/.agents/plugins/bkit/hooks.json +10 -0
- package/.bkit/shared/.bkit/.env.example +2 -2
- package/.bkit/shared/.claude/bkit-cloud.md +12 -10
- package/.bkit/shared/.claude/settings.json +14 -0
- package/.bkit/shared/.codex/bkit-cloud.md +98 -0
- package/.bkit/shared/.codex/hooks.json +20 -0
- package/.bkit/shared/.cursor/bkit-cloud.md +3 -3
- package/.bkit/shared/.cursor/mcp.json +1 -1
- package/.bkit/shared/.cursor/rules/bkit-rules.mdc +1 -1
- package/.bkit/shared/.gitattributes +1 -1
- package/.bkit/shared/.mcp.json +1 -1
- package/.bkit/shared/.opencode/opencode.json +12 -1
- package/.bkit/shared/CLAUDE.md +6 -2
- package/.bkit/shared/GEMINI.md +5 -2
- package/.bkit/shared/docs/architecture.md +4 -7
- package/.bkit/shared/docs/data-schema.md +4 -7
- package/.bkit/shared/docs/deployments.md +241 -168
- package/.bkit/shared/docs/design.md +11 -12
- package/.bkit/shared/docs/noise.md +8 -7
- package/.bkit/shared/docs/observability.md +5 -8
- package/.bkit/shared/docs/security.md +35 -17
- package/.bkit/shared/docs/setup.md +29 -18
- package/.bkit/shared/docs/technology.md +3 -6
- package/.bkit/shared/docs/testing.md +289 -174
- package/.bkit/shared/gitignore +6 -2
- package/.bkit/templates/.claude/hooks/bkit-cloud-bootstrap.sh +15 -16
- package/.bkit/templates/.cursor/Dockerfile +12 -8
- package/.bkit/templates/.cursor/environment.json +1 -1
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-bug.yml +10 -8
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-change.yml +45 -12
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-idea.yml +3 -3
- package/.bkit/templates/.github/renovate.json5 +34 -6
- package/.bkit/templates/.github/workflows/bkit-ci.yml +54 -51
- package/.bkit/templates/.github/workflows/bkit-cloud.yml +34 -33
- package/.bkit/templates/.github/workflows/bkit-health.yml +97 -0
- package/.bkit/templates/.github/workflows/bkit-renovate.yml +5 -4
- package/.bkit/templates/.github/workflows/bkit-trivy.yml +5 -4
- package/.bkit/templates/.husky/pre-commit +2 -2
- package/.bkit/templates/.husky/pre-push +96 -36
- package/.bkit/templates/.prettierignore +6 -0
- package/.bkit/templates/.vale.ini +27 -27
- package/.bkit/templates/context/bkit-cloudrun.md +23 -15
- package/.bkit/templates/context/bkit-expo.md +70 -52
- package/.bkit/templates/context/bkit-github.md +93 -56
- package/.bkit/templates/context/bkit-inngest.md +21 -15
- package/.bkit/templates/context/bkit-maestro.md +22 -16
- package/.bkit/templates/context/bkit-playwright.md +23 -17
- package/.bkit/templates/context/bkit-resend.md +12 -10
- package/.bkit/templates/context/bkit-sentry.md +30 -20
- package/.bkit/templates/context/bkit-serena.md +8 -7
- package/.bkit/templates/context/bkit-stripe.md +14 -13
- package/.bkit/templates/context/bkit-supabase.md +16 -13
- package/.bkit/templates/context/bkit-tavily.md +14 -11
- package/.bkit/templates/context/bkit-vercel.md +27 -18
- package/.bkit/templates/docs/capabilities.md +15 -14
- package/.bkit/templates/scripts/ci/{check-accepted-divergences.mjs → bkit-check-accepted-divergences.mjs} +6 -3
- package/.bkit/templates/scripts/ci/{check-accepted-divergences.test.mjs → bkit-check-accepted-divergences.test.mjs} +3 -3
- package/.bkit/templates/scripts/ci/{check-capstone.mjs → bkit-check-capstone.mjs} +2 -2
- package/.bkit/templates/scripts/ci/{check-capstone.test.mjs → bkit-check-capstone.test.mjs} +4 -4
- package/.bkit/templates/scripts/ci/{check-dependency-audit.mjs → bkit-check-dependency-audit.mjs} +103 -18
- package/.bkit/templates/scripts/ci/{check-dependency-audit.test.mjs → bkit-check-dependency-audit.test.mjs} +31 -3
- package/.bkit/templates/scripts/ci/bkit-machine-lock.sh +387 -0
- package/.bkit/templates/scripts/ci/bkit-test-awake.mjs +75 -0
- package/.bkit/templates/scripts/ci/bkit-test-awake.test.mjs +160 -0
- package/.bkit/templates/scripts/ci/lanes/{resolve-push-base.mjs → bkit-resolve-push-base.mjs} +3 -3
- package/.bkit/templates/scripts/ci/lanes/{resolve-push-base.test.mjs → bkit-resolve-push-base.test.mjs} +4 -4
- package/.bkit/templates/scripts/cleanup/bkit-disk-prune.mjs +10 -2
- package/.bkit/templates/scripts/graphify/bkit-graph-drift.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graph-eager.py +3 -3
- package/.bkit/templates/scripts/graphify/bkit-graph-embed.py +63 -0
- package/.bkit/templates/scripts/graphify/bkit-graph-imports.py +2 -2
- package/.bkit/templates/scripts/graphify/bkit-graph-pre-push.sh +69 -18
- package/.bkit/templates/scripts/graphify/bkit-graph-repair.py +3 -3
- package/.bkit/templates/scripts/graphify/bkit-graph-surface.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graphify-graphml.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graphify-rollup.py +197 -126
- package/.bkit/templates/scripts/graphify/bkit-graphify.sh +64 -41
- package/.bkit/templates/scripts/slop/bkit-slop-type-suppression.mjs +23 -7
- package/.bkit/templates/slop-scan.config.json +9 -2
- package/CHANGELOG.md +231 -0
- package/README.md +278 -89
- package/dist/checks/after/auth.js +36 -33
- package/dist/checks/after/ci-yml.js +91 -30
- package/dist/checks/after/github.js +91 -17
- package/dist/checks/after/index.js +22 -11
- package/dist/checks/after/suppression.js +8 -5
- package/dist/checks/after/tools.js +111 -114
- package/dist/checks/before/context-fetch.js +218 -33
- package/dist/checks/before/env.js +3 -3
- package/dist/checks/before/files.js +5 -3
- package/dist/checks/before/index.js +20 -3
- package/dist/checks/before/repo.js +197 -2
- package/dist/checks/status-holes.js +739 -114
- package/dist/checks/tips/catalogue.js +70 -19
- package/dist/checks/tips/detect.js +62 -0
- package/dist/checks/tips/index.js +51 -9
- package/dist/cli.js +234 -124
- package/dist/commands/analysis.js +174 -141
- package/dist/commands/ci/ci.js +302 -25
- package/dist/commands/ci/pins.js +142 -0
- package/dist/commands/ci/push-gate.js +194 -0
- package/dist/commands/comply.js +54 -91
- package/dist/commands/install/doctor.js +2 -2
- package/dist/commands/install/install.js +50 -19
- package/dist/commands/install/uninstall.js +82 -55
- package/dist/commands/install/upgrade.js +21 -11
- package/dist/commands/land.js +137 -17
- package/dist/commands/lib.js +295 -21
- package/dist/commands/registry.js +10 -2
- package/dist/commands/setup/setup.js +8 -7
- package/dist/commands/stats.js +175 -0
- package/dist/commands/tips.js +7 -3
- package/dist/commands/workflow/branches.js +1873 -152
- package/dist/commands/workflow/challenge.js +609 -0
- package/dist/commands/workflow/completeness.js +97 -0
- package/dist/commands/workflow/context.js +1045 -46
- package/dist/commands/workflow/counters.js +385 -86
- package/dist/commands/workflow/decisions.js +484 -0
- package/dist/commands/workflow/disk.js +214 -56
- package/dist/commands/workflow/fids.js +706 -33
- package/dist/commands/workflow/issues.js +349 -59
- package/dist/commands/workflow/ledgers.js +1123 -37
- package/dist/commands/workflow/locks.js +26 -2
- package/dist/commands/workflow/plan-blocks.js +501 -0
- package/dist/commands/workflow/plan-versions.js +22 -0
- package/dist/commands/workflow/plans.js +205 -0
- package/dist/commands/workflow/prs.js +154 -35
- package/dist/commands/workflow/repo.js +58 -8
- package/dist/commands/workflow/roadmap.js +58 -23
- package/dist/commands/workflow/run-facts.js +421 -0
- package/dist/commands/workflow/runs.js +670 -72
- package/dist/commands/workflow/runtime.js +15 -1
- package/dist/commands/workflow/skills.js +1 -0
- package/dist/commands/workflow/status.js +282 -34
- package/dist/commands/workflow/step-table.js +308 -0
- package/dist/commands/workflow/steps.js +703 -96
- package/dist/commands/workflow/summary-findings.js +117 -0
- package/dist/commands/workflow/summary.js +388 -0
- package/dist/commands/workflow/tasks.js +378 -0
- package/dist/commands/workflow/test-blocks.js +231 -0
- package/dist/commands/workflow/tests-changed.js +172 -0
- package/dist/commands/workflow/tests.js +1695 -305
- package/dist/commands/workflow/tracker.js +983 -0
- package/dist/commands/workflow/workflows.js +1014 -222
- package/dist/commands/workflow/worktrees.js +432 -26
- package/dist/install/cloud-env.js +233 -99
- package/dist/install/context-ci-workflow.js +85 -0
- package/dist/install/manifest.js +64 -17
- package/dist/install/modules/antigravity.js +3 -2
- package/dist/install/modules/cloud.js +25 -16
- package/dist/install/modules/codex.js +12 -2
- package/dist/install/modules/core.js +1090 -48
- package/dist/install/modules/cursor.js +6 -6
- package/dist/install/modules/opencode.js +13 -7
- package/dist/install/template-notes.js +116 -0
- package/dist/install/version-file.js +9 -4
- package/dist/ledgers/compact.js +87 -0
- package/dist/ledgers/journal.js +233 -0
- package/dist/ledgers/migrate.js +81 -0
- package/dist/ledgers/status.js +176 -13
- package/dist/ledgers/telemetry.js +1010 -125
- package/dist/render/hosts.js +104 -14
- package/dist/render/models.js +12 -0
- package/dist/shared/bkit-latest.js +26 -4
- package/dist/shared/branches.js +3 -1
- package/dist/shared/dep-updater.js +49 -0
- package/dist/shared/egress.js +8 -1
- package/dist/shared/json.js +20 -7
- package/dist/shared/package-manager.js +6 -0
- package/dist/shared/settings.js +181 -24
- package/dist/shared/text.js +333 -0
- package/dist/shared/timeouts.js +2 -0
- package/dist/tools/cli-doc.js +97 -0
- package/dist/tools/grammar-density.js +183 -0
- package/dist/tools/instructions-sources.js +720 -0
- package/dist/tools/instructions.js +1623 -0
- package/package.json +156 -51
- package/.bkit/core/.bkit/lite.md +0 -73
- package/.bkit/core/engine/commands/harvest.ts +0 -459
- package/.bkit/core/engine/commands/workflow/workstreams.ts +0 -166
- package/.bkit/core/engine/tools/wording.ts +0 -748
- package/.bkit/core/prompts/bkit-build-challenge-reconcile.md +0 -69
- package/.bkit/core/prompts/bkit-build-challenge-remediate.md +0 -71
- package/.bkit/core/prompts/bkit-build-challenge-tests.md +0 -74
- package/.bkit/core/prompts/bkit-build-plan-execute.md +0 -76
- package/.bkit/core/prompts/bkit-build-summary.md +0 -94
- package/.bkit/core/prompts/bkit-debug-summary.md +0 -96
- package/.bkit/core/prompts/bkit-design-summary.md +0 -107
- package/.bkit/core/prompts/bkit-plan-complexity-check.md +0 -67
- package/.bkit/core/prompts/bkit-plan-complexity-split.md +0 -88
- package/.bkit/core/prompts/bkit-plan-review-3.md +0 -93
- package/.bkit/core/prompts/bkit-plan-scope-check-1.md +0 -85
- package/.bkit/core/prompts/bkit-plan-scope-check-2.md +0 -85
- package/.bkit/core/prompts/bkit-plan-summary.md +0 -107
- package/.bkit/core/prompts/bkit-ship-summary.md +0 -78
- package/.bkit/templates/scripts/ci/bkit-local-stack-lock.sh +0 -194
- package/dist/commands/harvest.js +0 -353
- package/dist/commands/workflow/workstreams.js +0 -143
- package/dist/tools/wording.js +0 -680
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Filler.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Hedging.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/LLMSlop.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Verbose.yml +0 -0
- /package/.bkit/templates/scripts/ci/{accepted-advisories.json → bkit-accepted-advisories.json} +0 -0
- /package/.bkit/templates/scripts/ci/{accepted-divergences.json → bkit-accepted-divergences.json} +0 -0
- /package/.bkit/templates/scripts/launchd/{launchd.env.example → bkit-launchd.env.example} +0 -0
- /package/.bkit/templates/scripts/launchd/{ai.bkit.launchd.plist → bkit-launchd.plist} +0 -0
|
@@ -1,9 +1,9 @@
|
|
|
1
|
-
<!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
|
|
2
|
-
|
|
3
1
|
<!-- bkit:start -->
|
|
4
2
|
|
|
5
3
|
{{BKIT_MANAGED_MARKER}}
|
|
6
4
|
|
|
5
|
+
<!-- Note: @bkit-audit-generate fills this template. The report lives at `docs/audits/{YYYYMM}/{slug}.md`, where `{YYYYMM}` is the UTC month of the run and `{slug}` is this file's name without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md`, where `{TIMESTAMP}` is the output of `date -u +%Y%m%dT%H%M%SZ`. NEVER overwrite an earlier report. The report is stamped with a `bkit:anchor` block that records the audited commit (`baseSha`) and branch (`baseRef`). NEVER write or edit that block by hand. -->
|
|
6
|
+
|
|
7
7
|
# Testing Audit
|
|
8
8
|
|
|
9
9
|
> [!CAUTION]
|
|
@@ -11,181 +11,205 @@
|
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> -
|
|
15
|
-
> -
|
|
16
|
-
> -
|
|
17
|
-
> -
|
|
18
|
-
> - Use dense token-efficient phrasing; no filler / hedging / restating; tables > prose, bullets > paragraphs.
|
|
19
|
-
> - Read `.agents/docs.md` → load docs for workflow / step / task; skip missing; cite docs inline where required.
|
|
20
|
-
> - Every claim cites this run's evidence (`file:line`, or command / query and its output) and tool behind it; a previous report's figure, list, or status is a lead, not evidence.
|
|
21
|
-
> - Read this document once at session start; do not trust session memory; do not re-read on every loop iteration.
|
|
22
|
-
> - NEVER weaken, skip, delete, or quarantine a test to get green. A test that fails because it found a real bug is working.
|
|
23
|
-
> - Mutation and clock probes restore the tree before the report finishes: in-place edits reverted, throwaway setup removed, `git status` clean on touched paths. Restore fails → STOP, restore, record as finding.
|
|
24
|
-
> - Do not run suites that send mail, take payment, or write production tenants. Unit and local / throwaway targets only.
|
|
14
|
+
> - Never weaken, skip, delete or quarantine a test to get green. A test that fails because it found a real bug is working.
|
|
15
|
+
> - Mutation and clock probes restore the tree before the report finishes: revert in-place edits, remove throwaway setup, and leave `git status` clean on the touched paths.
|
|
16
|
+
> - **If** a restore fails → **STOP**, restore the tree, and record a finding.
|
|
17
|
+
> - NEVER run suites that send mail, take payment or write to production tenants. Run unit tests and local or throwaway targets only.
|
|
25
18
|
|
|
26
19
|
## Universal Instructions
|
|
27
20
|
|
|
28
21
|
### Context & Goals
|
|
29
22
|
|
|
30
|
-
|
|
23
|
+
A green suite means the tests passed. It does not mean they could fail, that every promise has a test, or that the lanes run them. Coverage percentage misses both: a line can be covered by a test that asserts nothing, and a percentage cannot show behaviour that should exist but has no test. The failure this audit targets is a test that stays green when the behaviour it names is broken, or a promise with no test at the tier that would notice.
|
|
31
24
|
|
|
32
|
-
**Applies when:** test runner, test files
|
|
25
|
+
**Applies when:** the project has a test runner, test files or a CI test lane. **If** it has none → write one `N/A` row with its evidence, and never report the surface as "clean".
|
|
33
26
|
|
|
34
|
-
**Cadence:**
|
|
27
|
+
**Cadence:** Run once a month, covering every product area and every package or app, and record the mutation cost per package. **If** a package's mutation tool cannot finish → mark it `Not Checked` with that cost, and never omit it. Run an extra pass when the test toolchain's major version or config changes (run mutation with force, because incremental mode misses those changes), when a new route, worker or external service lands, or when `bkit-audit-logs` or `bkit-audit-flakes` reports a bug no test caught.
|
|
35
28
|
|
|
36
|
-
**Anchor Standards:**
|
|
29
|
+
**Anchor Standards:**
|
|
37
30
|
|
|
38
|
-
- Risk-based testing (e.g
|
|
39
|
-
- Test-size model (e.g
|
|
40
|
-
- Mutation tool, incl. incremental mode and disable comments (e.g
|
|
41
|
-
- Property-based testing (e.g
|
|
42
|
-
- Contract testing (e.g
|
|
43
|
-
- Database tests (e.g
|
|
31
|
+
- Risk-based testing (e.g., ISO/IEC/IEEE 29119 Part 1:2022 and Part 2:2021, ISTQB Foundation Level syllabus).
|
|
32
|
+
- Test-size model (e.g., Testing Trophy, Google test sizes); unversioned → record the date read.
|
|
33
|
+
- Mutation tool, incl. incremental mode and disable comments (e.g., StrykerJS, mutmut); record the version.
|
|
34
|
+
- Property-based testing (e.g., fast-check, Hypothesis); record the version read.
|
|
35
|
+
- Contract testing (e.g., Pact, schema snapshots); record the version read.
|
|
36
|
+
- Database tests (e.g., pgTAP, a catalog check against the deployed schema); record the date read.
|
|
44
37
|
- OWASP ASVS and the OWASP API Security Top 10; record the edition read.
|
|
45
|
-
- Visual and accessibility snapshots (e.g
|
|
46
|
-
- Test-smell catalogue (e.g
|
|
47
|
-
- Test lint (e.g
|
|
48
|
-
- Network mock that fails closed (e.g
|
|
49
|
-
- bkit block of `docs/testing.md` (bkit version from `.bkit/version`): § Validated means executed, not "didn't fail"; § A guard asserts the rule, not the sentence that explains it; § Which tier does a behaviour need?; § Test first; § What each tier must do; § Design assertions are a dimension, not a lane; § Schema-invariant DB checks (e.g. RLS coverage); § Provision backing services fresh; § Optional hardening (adopt when the cost is justified); § Coverage on PRs
|
|
38
|
+
- Visual and accessibility snapshots (e.g., Playwright `toHaveScreenshot` / `toMatchAriaSnapshot`, device screenshots); record the runner version.
|
|
39
|
+
- Test-smell catalogue (e.g., xUnit Test Patterns, Meszaros 2007).
|
|
40
|
+
- Test lint (e.g., `@vitest/eslint-plugin`, `eslint-plugin-playwright`, `eslint-plugin-testing-library`, or Oxlint's `vitest` + `typescript` plugins); record each version. typescript-eslint type-aware rules need a TypeScript version it supports; newer → Oxlint `--type-aware` (oxlint-tsgolint).
|
|
41
|
+
- Network mock that fails closed (e.g., MSW); record the major read.
|
|
42
|
+
- bkit block of `docs/testing.md` (bkit version from `.bkit/version`): § Validated means executed, not "didn't fail"; § A guard asserts the rule, not the sentence that explains it; § Which tier does a behaviour need?; § Test first; § Tests lock; § Local test sleep prevention; § What each tier must do; § Design assertions are a dimension, not a lane; § Schema-invariant DB checks (e.g. RLS coverage); § Provision backing services fresh; § Optional hardening (adopt when the cost is justified); § Coverage on PRs
|
|
50
43
|
|
|
51
44
|
### Scope
|
|
52
45
|
|
|
53
|
-
-
|
|
54
|
-
-
|
|
55
|
-
-
|
|
56
|
-
-
|
|
46
|
+
- Build the should-test list: promises, never-events, platforms, roles, failure paths, and data that outlives the request.
|
|
47
|
+
- Check that each item is tested at the right tier, not only at some tier.
|
|
48
|
+
- Check that database rules and boundary contracts have tests, not only the guards in front of them.
|
|
49
|
+
- Run mutation testing to see whether the tests go red.
|
|
50
|
+
- Search for the known cannot-fail shapes.
|
|
51
|
+
- Check the lint and setup that should catch those shapes.
|
|
52
|
+
- Check that every test file has a lane.
|
|
53
|
+
- Check timezone and date boundaries.
|
|
54
|
+
- Find duplicated test procedures.
|
|
57
55
|
|
|
58
56
|
### Out of Scope
|
|
59
57
|
|
|
60
58
|
<!-- prettier-ignore -->
|
|
61
59
|
| Excluded item | Why | Where it belongs |
|
|
62
60
|
| --- | --- | --- |
|
|
63
|
-
| Branch-coverage percentage and the CI floor |
|
|
64
|
-
| Flake diagnosis | Flake triage | `bkit-audit-flakes` |
|
|
65
|
-
| Log sweeps | Log triage | `bkit-audit-logs` |
|
|
66
|
-
| Turning a check into a CI gate |
|
|
67
|
-
| Dead code beyond a mutant no test runs |
|
|
68
|
-
| Product-code duplication |
|
|
69
|
-
| Catalogue privileges and policy correctness |
|
|
70
|
-
| Changing tests or code |
|
|
61
|
+
| Branch-coverage percentage and the CI floor | The coverage sweep owns them | `bkit-audit-coverage` |
|
|
62
|
+
| Flake diagnosis | Flake triage owns it | `bkit-audit-flakes` |
|
|
63
|
+
| Log sweeps | Log triage owns them | `bkit-audit-logs` |
|
|
64
|
+
| Turning a check into a CI gate | The gates sweep owns it | `bkit-audit-gates` |
|
|
65
|
+
| Dead code beyond a mutant no test runs | The dead-code sweep owns it | `bkit-audit-dead-code` |
|
|
66
|
+
| Product-code duplication | The duplication sweep owns it | `bkit-audit-duplication` |
|
|
67
|
+
| Catalogue privileges and policy correctness | The database sweep owns them | `bkit-audit-database` |
|
|
68
|
+
| Changing tests or code | The audit only reports | A plan made from this report |
|
|
71
69
|
|
|
72
70
|
### Pre-Flight / Dependencies
|
|
73
71
|
|
|
74
|
-
**Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
|
|
75
|
-
|
|
76
|
-
**Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
|
|
77
|
-
|
|
78
72
|
#### Core
|
|
79
73
|
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
- [ ] **
|
|
83
|
-
- [ ] **
|
|
84
|
-
- [ ] **Test Runner:** Steps 3, 6, 9, 10 — execute or read reports, e.g. Vitest, Jest, pytest. Fallback: static read only; counts, mutation, clock runs `Not Checked`.
|
|
85
|
-
- [ ] **Web:** Step 13 — fetch anchor standards. Fallback: versions recorded as unread.
|
|
74
|
+
- [ ] **Permissions:** every blocked call is recorded.
|
|
75
|
+
- [ ] **Git:** Steps 1, 14 — `git rev-parse`, and the diff since the previous report. Fallback: mark the run as baseline, and mark the ages and the what-changed pass `Not Checked`.
|
|
76
|
+
- [ ] **Test Runner:** Steps 3, 6, 9, 10 — run the tests or read their reports (e.g., Vitest, Jest, pytest). Fallback: read statically only, and mark the counts, mutation runs and clock runs `Not Checked`.
|
|
77
|
+
- [ ] **Web:** Step 13 — fetch the anchor standards. Fallback: record the versions as unread.
|
|
86
78
|
|
|
87
79
|
#### Conditional
|
|
88
80
|
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
- [ ] **
|
|
92
|
-
- [ ] **
|
|
93
|
-
- [ ] **
|
|
94
|
-
- [ ] **Test Lint:** Test files linted → Step 8 — e.g. `@vitest/eslint-plugin`, `eslint-plugin-playwright`, Oxlint `vitest` / `typescript` plugins. Fallback: text search for the shapes.
|
|
95
|
-
- [ ] **Language Server:** Typed language → Step 11 — references, e.g. Serena, `tsserver`. Fallback: text search; name it.
|
|
81
|
+
- [ ] **Mutation Tool:** **If** a tool exists for the language → Step 6 — mutation runs (e.g., Stryker, mutmut). Fallback: break the code the tests protect by hand, and mark the scores `Not Checked`.
|
|
82
|
+
- [ ] **E2E Runner:** **If** there is browser or device E2E → Steps 3, 7, 9 — E2E runs (e.g., Playwright, Cypress, Maestro). Fallback: cover unit and integration tests only.
|
|
83
|
+
- [ ] **DB Tests:** **If** there are row-level policies, buckets or data migrations → Step 4 — database tests (e.g., pgTAP, `supabase db test`, a deployed-schema check). Fallback: read the existing tests, and treat the live matrix as a hypothesis.
|
|
84
|
+
- [ ] **Test Lint:** **If** test files are linted → Step 8 — test lint rules (e.g., `@vitest/eslint-plugin`, `eslint-plugin-playwright`, Oxlint `vitest` / `typescript` plugins). Fallback: text search for the shapes.
|
|
85
|
+
- [ ] **Language Server:** **If** the language is typed → Step 11 — references (e.g., Serena, `tsserver`). Fallback: text search, and name it.
|
|
96
86
|
|
|
97
87
|
### 🔒 Gate 1: Prerequisites Check
|
|
98
88
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
102
|
-
- [ ] **Gate 1 - Verified:** All Core items checked. All applicable Conditional items checked.
|
|
89
|
+
- [ ] **Gaps named:** every missing item names its fallback and its `Not Checked` entry.
|
|
90
|
+
- [ ] **Gate 1 — Verified:** every item above, every Core item and every applicable Conditional item is checked.
|
|
103
91
|
|
|
104
92
|
### Probe Steps
|
|
105
93
|
|
|
106
|
-
- [ ] **Step 1 — Previous Run:**
|
|
107
|
-
- [ ] **Step 2 — Scope:**
|
|
108
|
-
- [ ] **Step 3 — Behaviours:** Start from product should
|
|
109
|
-
- Screens and routes
|
|
110
|
-
- Unattended work
|
|
111
|
-
- Security rules
|
|
112
|
-
- OWASP ASVS and API Security Top 10 items security doc may omit (cross-tenant object access, open redirect, SSRF
|
|
113
|
-
-
|
|
114
|
-
-
|
|
115
|
-
- Each external-service boundary
|
|
116
|
-
- Multiply every row
|
|
117
|
-
-
|
|
118
|
-
- Rank gaps: another tenant's data, lost data
|
|
119
|
-
-
|
|
120
|
-
-
|
|
121
|
-
- [ ] **Step 4 — Database Rules:**
|
|
122
|
-
- [ ] **Step 5 — Contracts:**
|
|
123
|
-
- [ ] **Step 6 — Mutation:** Run mutation tool on every package
|
|
124
|
-
- **Bug in the code
|
|
125
|
-
- **Real gap
|
|
126
|
-
- **No test at all
|
|
127
|
-
- **Caught elsewhere
|
|
128
|
-
- **Equivalent
|
|
129
|
-
- **Not worth a test
|
|
130
|
-
- **Tool misreport
|
|
131
|
-
- **Other
|
|
132
|
-
|
|
133
|
-
Do not trust tool numbers until checked.
|
|
134
|
-
|
|
135
|
-
- [ ] **Step 7 — Shapes:** Search every package for
|
|
136
|
-
-
|
|
137
|
-
-
|
|
138
|
-
-
|
|
139
|
-
-
|
|
140
|
-
- Setup
|
|
141
|
-
-
|
|
142
|
-
-
|
|
143
|
-
- Absence
|
|
144
|
-
-
|
|
145
|
-
-
|
|
146
|
-
-
|
|
147
|
-
-
|
|
148
|
-
-
|
|
149
|
-
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
- [ ] **Step
|
|
157
|
-
-
|
|
158
|
-
-
|
|
159
|
-
-
|
|
94
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** `Previous Run` holds the closure rate, or the run is marked baseline.
|
|
95
|
+
- [ ] **Step 2 — Scope:** Cover every product area in Steps 3-5, because behaviour usually crosses packages. Cover every package or app in Steps 6-11. **If** an area is shared with a native client → cover every platform, because one defect can have a different cause on each. **Done when:** `Scope` holds every area, every package and the platforms, and `Checked` holds the tools and commands.
|
|
96
|
+
- [ ] **Step 3 — Behaviours:** Start from what the product should do, not from the code. The code finds untested code, not promises with no test or unguarded failures. Build rows from:
|
|
97
|
+
- Screens and routes: each action is one row, and finishing an action is a separate row from opening it.
|
|
98
|
+
- Unattended work: workers, crons, inbound, webhooks and tool APIs.
|
|
99
|
+
- Security rules: cross-tenant read and write, a role acting above itself, and signed-out access.
|
|
100
|
+
- OWASP ASVS and API Security Top 10 items the security doc may omit (cross-tenant object access, open redirect, SSRF through a user-supplied URL, mass assignment, rate limiting, CSRF, webhook signature bypass). Give each one a tier.
|
|
101
|
+
- The decisions ledger: a decision with no test can be undone unnoticed.
|
|
102
|
+
- The failures ledger: each fixed row needs a test that goes red if the fix is reverted. For each fixed `HIGH` / `CRITICAL` row, revert the fix as a Step 6 hand mutant. A test named for the row is not proof. **If** the fix is not found (e.g., `git log -S` / `--grep` on the row key or file), **or** cannot be reverted locally (an applied migration, a remote setting) → mark the row `Not Checked` with the reason, never guarded.
|
|
103
|
+
- Each external-service boundary: it needs a live test that reaches the real service.
|
|
104
|
+
- Multiply every row by every platform (native is never out of scope when a native client exists), by every role that can and cannot act, by the failure path (the request fails, a service is down, the input is invalid, the user cancels halfway), and by data that outlives the request (deletes, retention, billing, storage writes).
|
|
105
|
+
- For each row, set the tier (unit, integration, live, web E2E, device E2E or none) against the project rubric. A tier below the behaviour (a mock for a real external API) is a gap. A tier above it (only web or device E2E where unit or integration suffices) is a finding, because it is a source of slowness and flakes. Record the test count and runtime per tier per package (trophy against pyramid). For each core screen, check for a visual or accessibility snapshot on web, or a device screenshot on native, and list the screens with neither.
|
|
106
|
+
- Rank the gaps: another tenant's data, lost data and money first, then a broken core action, then cosmetic.
|
|
107
|
+
- Look for recurring shapes: an action that is started but never finished (confirm on destructive or session-wide actions); a failing request that still reports success; a security guard that is the only stop (e.g., an app check because the DB client skips RLS), listed even if the code is correct today; and settings that live only in config (triggers, schedules, concurrency, lint globs, auth mount paths), where a blank value breaks production unnoticed.
|
|
108
|
+
- **If** a behaviour-to-test registry exists → start from it and add to it, and do not rebuild it. Coverage reports show untested code, but cannot produce this list. **Done when:** `Gaps` holds the ranked gaps, the tiers and the per-tier counts.
|
|
109
|
+
- [ ] **Step 4 — Database Rules:** App guard tests cannot catch a policy regression in a migration. **If** the database has row-level policies (e.g., Postgres RLS, Firestore rules), storage buckets, definer-rights functions or data-changing migrations → for each row-policy table, bucket and definer-rights function, check for a DB test (e.g., pgTAP), or a DB-backed integration test, that asserts allow and deny per role and operation (select, insert, update, delete). Record the gaps in the matrix. **If** a migration transforms rows, not only the schema → check for a replay test on seeded rows that asserts the invariants: row counts, no orphaned foreign keys, and no nulls in a new `NOT NULL` column. **Done when:** `Database Rules` holds the matrix, or the step is skipped with evidence.
|
|
110
|
+
- [ ] **Step 5 — Contracts:** **If** there are external services or older clients in the field → check that fixtures are typed to the provider SDK and the pinned API version, age each payload against its version, and check that a webhook with a bad or expired signature is tested and rejected. A contract or snapshot pins the API and validator shapes the oldest supported client needs. Without one, a breaking server change ships green. **Done when:** `Contracts` has its rows, or the step is skipped with evidence.
|
|
111
|
+
- [ ] **Step 6 — Mutation:** Run the mutation tool on every package, one small change at a time (flip a comparison, delete a line, blank a string). Setup and traps are in the project docs. During in-place mutation, nothing else builds, lints, tests or commits the package under mutation, and the tree must be clean afterwards. **If** it is not → **STOP** and restore. Give every unnoticed change exactly one verdict:
|
|
112
|
+
- **Bug in the code:** the change exposed real wrong behaviour. Route it as an app bug.
|
|
113
|
+
- **Real gap:** a test exists, but does not check this. Name the missing assertion.
|
|
114
|
+
- **No test at all:** no unit test runs this code. Send the tier that should own it to Step 3.
|
|
115
|
+
- **Caught elsewhere:** an integration, live or E2E test would catch it. Name the test. "Probably the E2E" is not a verdict.
|
|
116
|
+
- **Equivalent:** there is no observable difference. Report the tool's disable comment to add, so the next run does not judge it again. Adding it is the next step.
|
|
117
|
+
- **Not worth a test:** log text, error messages.
|
|
118
|
+
- **Tool misreport:** re-check it by hand.
|
|
119
|
+
- **Other:** say why. **If** the same `Other` comes up on 2 runs → propose a verdict for it in `Process Improvements`.
|
|
120
|
+
|
|
121
|
+
Do not trust the tool's numbers until they are checked. A runner with zero tests per mutant reports every mutant in the suite as survived. A test file that crashes on load counts as survived, not caught. Integration-named tests that mock everything can sit outside both the unit and mutation runs. Tests that read source as text do not see the mutation. Re-check every zero-tests survivor by hand. For code the tool cannot reach (browser specs, device flows, lint guard tests), break it by hand, run the test, watch it go red, and restore. **If** in-place edits are blocked (e.g., a shared checkout) → do the hand mutants in a throwaway worktree, set up per `.agents/rules.md` § New worktree. A red caused by a missing gitignored input there is not a caught mutant. Record two scores, total and covered (covered excludes mutants no test runs), by package and by the product risk area from Step 3. A total dominated by **No test at all** hides weak existing tests. List the pure functions where boundary mutants survive (comparisons, dates, timezones, permission checks, validators), and recommend one property-based test for each. Record the cost: wall-clock, CPU-seconds, peak memory and triage agents. Whether a CI check is viable goes to `bkit-audit-gates`, based on those numbers. Run in incremental mode, keeping the previous incremental file, and do a full run (e.g., Stryker `--force`) every few runs and after a dependency, config or snapshot change. Record the mode. **Done when:** `Scores` holds both scores, the verdicts, the property candidates, the cost and the mode, or the step is skipped with evidence.
|
|
122
|
+
|
|
123
|
+
- [ ] **Step 7 — Shapes:** Search every package for tests that cannot fail. Each of these shapes has been green while checking nothing:
|
|
124
|
+
- A wait that does not wait: a visibility check with an ignored timeout (e.g., `isVisible({ timeout })`), or a response wait nobody reads.
|
|
125
|
+
- An assertion that matches anything: a bare regex in an object matcher (e.g., Vitest `toMatchObject`), or two constants compared instead of calling the code.
|
|
126
|
+
- A mock that hides the thing under test: a hand-written mock of a shared module, or a fixture shape production never produces.
|
|
127
|
+
- A test of a copy, not the code: a hand-written copy of a guard, or two apps with the same code compared with each other.
|
|
128
|
+
- Setup that skips the code: a fake DB that is always empty, so the branch under test never runs.
|
|
129
|
+
- A test that locks in a bug: it asserts wrong behaviour that no decision wanted.
|
|
130
|
+
- A positive control that cannot fail: it asserts absence using a selector or text that never existed.
|
|
131
|
+
- Absence asserted too early: it asserts that something is absent before it could have appeared, with no wait after the trigger.
|
|
132
|
+
- A listener attached after the event it observes, so it never sees the event.
|
|
133
|
+
- An absence guard that outlived its writer: it asserts that a file, path or name is not written, shipped or listed, yet no source writes, copies or templates it any more (a retired or deprecated name), and no migration or cleanup removes it. Search the sources for each named path. Also flag absence samples (headings, strings) that appear nowhere, because the guard then passes whatever leaks. Keep secret and credential guards, and removal checks for a migration that still runs.
|
|
134
|
+
- A guard tested in a different setup from runtime (cwd, config, env).
|
|
135
|
+
- A string scan cut short: a comment stripper that cuts at a marker also cuts a marker inside strings.
|
|
136
|
+
- The wrong surface: a write goes to one place, and the assertion reads another. For every write, the assertion must read the surface the write meant to change. Derived copies go to `bkit-audit-database`.
|
|
137
|
+
- A status-only assertion: it checks the response code only, on an operation that reports success when it failed. Assert what was stored.
|
|
138
|
+
- Two clients mocked as one: user-scoped and admin clients differ, and one shared mock cannot see a write through the wrong one. Mocks must keep dependencies that differ at runtime different.
|
|
139
|
+
- "Did not throw" as the only check on a boundary (webhook, external API, DB write). Not throwing != the work happened. Step 6 usually confirms it.
|
|
140
|
+
- An exit status swallowed by a wrapper: a script or pipe around the runner exits 0 whatever the runner returns (e.g., a pipe into `tee` without `pipefail`).
|
|
141
|
+
|
|
142
|
+
**If** you find a new shape → propose it under `Process Improvements`, and do not edit this list during the run. **Done when:** `Shapes` holds every hit, or says `none`.
|
|
143
|
+
|
|
144
|
+
- [ ] **Step 8 — Guards:** A shape that tools can catch belongs in a lint rule or a setup file, not in a per-run search.
|
|
145
|
+
- Confirm the rules are enabled for test files, and record the violation count for each test-stack lint rule that catches a shape (e.g., JS: `vitest/expect-expect`, `vitest/valid-expect`, `vitest/no-conditional-in-test`, `vitest/no-conditional-expect`, `playwright/missing-playwright-await`, `playwright/no-wait-for-timeout`, `playwright/prefer-web-first-assertions`, `playwright/no-networkidle`, `playwright/no-force-option`, `@typescript-eslint/no-floating-promises` (Oxlint: `typescript/no-floating-promises` with `--type-aware`); Python: `flake8-pytest-style`).
|
|
146
|
+
- **If** there is no test-stack linter → text search for the shapes, and say so.
|
|
147
|
+
- **If** a rule would catch a Step 7 shape and is off → record a finding, with `bkit-audit-gates` as its destination.
|
|
148
|
+
- Confirm that the network mock fails a test on an unhandled request (e.g., MSW `onUnhandledRequest: 'error'`). Otherwise an unmocked call reaches the real service or hangs.
|
|
149
|
+
- Check that fixtures and factories are typed to the generated DB types and schema validators, where the project has them.
|
|
150
|
+
- Grep the tests for type-check escapes (e.g., TS `as any`, `as unknown as`; Python `cast(Any, ...)`, `# type: ignore`), because they let a fixture drift to a shape production never produces.
|
|
151
|
+
- **If** the language is untyped → `N/A`.
|
|
152
|
+
- Count the snapshot assertions and their size.
|
|
153
|
+
- List the snapshot files (e.g., `.snap`) changed in the same commit as the code they cover since the previous report's commit. A large snapshot, or a wholesale `-u` update, behaves like an assertion that matches anything.
|
|
154
|
+
|
|
155
|
+
**Done when:** `Guards` holds the counts and hits.
|
|
156
|
+
|
|
157
|
+
- [ ] **Step 9 — Runs:** Check that every test file has a lane, including test files the runner does not collect (e.g., `*.test.sh`). Find names that put a file outside every run (see Step 6). Find suites that skip when setup fails, instead of failing. Find skips (e.g., `test.skip`, `fixme`, `describe.skip`, `pytest.mark.skip`) and env-dependent skips. Take skip counts per file from the CI runner log too (e.g., job log `N skipped` lines), not only from local runs, because a tool missing on the runner skips only there. **If** the runner prints totals only → use a per-file reporter (e.g., verbose, JUnit). **If** there is no CI, **or** the log has expired → mark the CI half `Not Checked`. Find stale build output the runner now collects (e.g., Vitest 4 and later does not ignore `dist/` by default). List the 20 slowest tests per tier from the runner reports. A slow test is the next one someone skips. **Done when:** `Runs` has its rows.
|
|
158
|
+
- [ ] **Step 10 — Time:** **If** there is date, calendar or timezone logic → run the unit tests with `TZ` set to a non-UTC zone that observes DST (e.g., `Pacific/Auckland`, `America/St_Johns`), on the process before workers spawn. Then run the date-touching suites with the clock pinned near a month end and a DST change (e.g., `vi.setSystemTime`, Playwright `page.clock`), and remove the throwaway setup afterwards. A clock pin covers in-process time only. Suites that spawn subprocesses (e.g., `git`, `node`) need the child clock set too (e.g., `GIT_AUTHOR_DATE` / `GIT_COMMITTER_DATE`, a faketime wrapper). **If** the child clock cannot be set → mark the subprocess half `Not Checked`, and still record the in-process result. Record the tests whose result changes. List the suites that never pin the clock. **Done when:** `Time` holds the results, or the step is skipped with evidence.
|
|
159
|
+
- [ ] **Step 11 — Copies:** Find the same test procedure in several places (open a dialog, create a user, read credentials). List the copies that disagree first. Copies that agree need a single owner only if they are likely to drift. Once one owner is created, the next step adds a guard that fails on a new copy. A code graph finds copied functions, and copied statement sequences need a syntax-tree scan. Product-code duplication goes to `bkit-audit-duplication`. **Done when:** `Copies` holds the groups, or says `none`.
|
|
160
|
+
- [ ] **Step 12 — Route:** Give no finding without evidence someone can check again: `file:line`, or a query, run id or measurement, plus the tool.
|
|
161
|
+
- An empty language-server result is inconclusive, so corroborate it with text search.
|
|
162
|
+
- The report names where each finding goes.
|
|
163
|
+
- This run writes the `docs/failures.jsonl` rows. Sending to every other destination is the next step's job.
|
|
164
|
+
- Destinations:
|
|
165
|
+
- an app bug → a `docs/failures.jsonl` row and an issue, marked as security when it is one
|
|
166
|
+
- a small missing test → the test to write
|
|
167
|
+
- a large missing test → a follow-up issue
|
|
168
|
+
- a cannot-fail test → the fix, a positive control (break the code, watch it go red, restore) and a `docs/failures.jsonl` row
|
|
169
|
+
- a behaviour with no recorded intent → a question for the maintainer, and do not guess from the code
|
|
170
|
+
- a tier too high → the lower-tier test to write, and the higher-tier test stays until the new one has been seen red
|
|
171
|
+
- an equivalent mutant → the disable comment to add
|
|
172
|
+
- a timezone or date result:
|
|
173
|
+
- **If** the app is wrong → an app bug
|
|
174
|
+
- **Else** → a test fix and a `docs/failures.jsonl` row
|
|
175
|
+
- a flaky test → `bkit-audit-flakes`
|
|
176
|
+
- moving the coverage floor → `bkit-audit-coverage`
|
|
177
|
+
- a check as a CI gate → `bkit-audit-gates`
|
|
178
|
+
- dead code → `bkit-audit-dead-code`
|
|
179
|
+
|
|
180
|
+
**Done when:** every finding's Expected names its destination.
|
|
181
|
+
|
|
182
|
+
- [ ] **Step 13 — Standards:** Confirm the current version of each anchor standard, and read what changed since the previous report's version. Give `Standards Coverage` one row per OWASP API Security Top 10 item and ASVS chapter (the attack rows from Step 3), and one per test smell in the catalogue (the shapes from Step 7), each marked checked (naming the step), not applicable (saying why) or finding. Give each bkit anchor (a doc section, rule or issue shape) one row, marked followed (with the step or evidence), not applicable (saying why) or finding. A departure with no recorded reason (e.g., a `docs/decisions.jsonl` row or an accepted-divergence entry) is a finding. Mark each recommended practice the project does not apply as adopt (with a proposed step), reject (saying why) or not applicable. Do not change the steps during the run. **Done when:** no row is empty.
|
|
183
|
+
- [ ] **Step 14 — Anything Else:** (a) Give a one-line verdict on each change since the previous report's commit that touches the suite: test files and helpers, fixtures and factories, unit, browser and device runner config, CI test lanes, and the routes, workers, migrations and external services the tests cover. (b) In a bounded pass of about 30 minutes, note what a senior test engineer would flag outside the steps. Read what happened since the previous run (incidents, `bkit-audit-logs`, new `docs/failures.jsonl` rows, vendor status), and ask what no step would catch. **Done when:** `Beyond the Steps` holds both passes, or says `none`. **If** one class of issue is seen on 2 runs → propose a step for it in `Process Improvements`.
|
|
160
184
|
|
|
161
185
|
### 🔒 Gate 2: Probes Check
|
|
162
186
|
|
|
163
|
-
**
|
|
164
|
-
|
|
165
|
-
- [ ] **
|
|
166
|
-
- [ ] **
|
|
167
|
-
- [ ] **
|
|
168
|
-
- [ ] **Scores:** total and covered
|
|
169
|
-
- [ ] **Gate 2
|
|
187
|
+
- [ ] **Steps complete:** every step is `[x]`, or skipped with evidence that its surface is absent.
|
|
188
|
+
- [ ] **Evidence:** every finding cites evidence someone can check again, and the tool that produced it.
|
|
189
|
+
- [ ] **Mutations restored:** every mutation is restored.
|
|
190
|
+
- [ ] **Clock setup removed:** the throwaway clock setup is removed.
|
|
191
|
+
- [ ] **No test weakened:** no test was weakened, skipped or deleted.
|
|
192
|
+
- [ ] **Scores:** `Scores` records the total and covered scores, or mutation is explicitly skipped with evidence.
|
|
193
|
+
- [ ] **Gate 2 — Verified:** every item above holds.
|
|
170
194
|
|
|
171
195
|
## Findings
|
|
172
196
|
|
|
173
197
|
<!-- prettier-ignore -->
|
|
174
198
|
| Severity | Use |
|
|
175
199
|
| --- | --- |
|
|
176
|
-
| `CRITICAL` |
|
|
177
|
-
| `HIGH` | Core behaviour with no test, or a test that cannot go red
|
|
178
|
-
| `MEDIUM` |
|
|
179
|
-
| `LOW` |
|
|
200
|
+
| `CRITICAL` | An untested or cannot-fail guard on cross-tenant access, money, or a workflow with no workaround. The lane that should run that guard does not |
|
|
201
|
+
| `HIGH` | Core behaviour with no test, or with a test that cannot go red. A security guard with no test. An untyped external contract, or untested signature rejection. A timezone-dependent result hiding a real bug |
|
|
202
|
+
| `MEDIUM` | A tier too low or too high. A missing failure path. A lint rule turned off that would catch a cannot-fail shape. Duplicated procedures that disagree. A snapshot that matches anything. A suite that skips itself |
|
|
203
|
+
| `LOW` | An equivalent mutant without a disable comment. A cosmetic screen without a visual assertion. Duplicates that agree |
|
|
180
204
|
|
|
181
|
-
|
|
205
|
+
Within a severity, list security and data findings first.
|
|
182
206
|
|
|
183
207
|
<!-- prettier-ignore -->
|
|
184
208
|
| ID | Severity | Location | Evidence | Expected |
|
|
185
209
|
| --- | --- | --- | --- | --- |
|
|
186
210
|
| PENDING | PENDING | PENDING | PENDING | PENDING |
|
|
187
211
|
|
|
188
|
-
|
|
212
|
+
**If** the run finds nothing → write one row: `None`.
|
|
189
213
|
|
|
190
214
|
## Report
|
|
191
215
|
|
|
@@ -266,17 +290,17 @@ PENDING
|
|
|
266
290
|
|
|
267
291
|
### 🔒 Gate 3: Report Check
|
|
268
292
|
|
|
269
|
-
**
|
|
270
|
-
|
|
271
|
-
- [ ] **
|
|
272
|
-
- [ ] **
|
|
273
|
-
- [ ] **Series:** test count and runtime per tier, and both scores, beside previous report's numbers, or
|
|
274
|
-
- [ ] **Writes:** only this report
|
|
275
|
-
- [ ] **
|
|
276
|
-
- [ ] **
|
|
277
|
-
- [ ] **
|
|
278
|
-
|
|
279
|
-
|
|
293
|
+
- [ ] **No placeholders:** no `PENDING` is left.
|
|
294
|
+
- [ ] **Summary:** `Summary` gives the count per severity.
|
|
295
|
+
- [ ] **Top finding:** `Summary` names the single most important finding.
|
|
296
|
+
- [ ] **Mutation scores:** `Summary` gives both mutation scores per package, or records the skip.
|
|
297
|
+
- [ ] **Series:** the report shows test count and runtime per tier, and both scores, beside the previous report's numbers, or marks the run as baseline.
|
|
298
|
+
- [ ] **Writes:** this run changed only this report, the template's `Project Instructions` and its own `docs/failures.jsonl` rows.
|
|
299
|
+
- [ ] **Throwaways removed:** every throwaway worktree or remote target is removed.
|
|
300
|
+
- [ ] **Process Improvements:** `Universal` and `Project` are both filled.
|
|
301
|
+
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID is in exactly one `Suggested Issues` row.
|
|
302
|
+
- [ ] **Next Step:** every `Suggested Issues` row has a `Next Step`.
|
|
303
|
+
- [ ] **Gate 3 — Verified:** every item above holds.
|
|
280
304
|
|
|
281
305
|
<!-- bkit:end -->
|
|
282
306
|
|