@cbartens/bkit 5.5.1 → 6.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.bkit/README.md +167 -154
- package/.bkit/core/.agents/context.md +95 -29
- package/.bkit/core/.agents/docs.md +18 -15
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-gateguard.cjs +1 -0
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-guardrails.cjs +6 -5
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-readguard.cjs +1 -0
- package/.bkit/core/.agents/plugins/bkit/hooks/bkit-rules.cjs +65 -0
- package/.bkit/core/.agents/rules.md +112 -102
- package/.bkit/core/.agents/skills.md +63 -15
- package/.bkit/core/.bkit/README.md +167 -154
- package/.bkit/core/.bkit/authoring.md +747 -0
- package/.bkit/core/.bkit/cli.md +407 -0
- package/.bkit/core/.bkit/cloud.md +283 -195
- package/.bkit/core/.bkit/comparison.md +45 -43
- package/.bkit/core/.bkit/install.md +166 -95
- package/.bkit/core/.bkit/jsonl.md +339 -196
- package/.bkit/core/.bkit/pipeline.md +142 -145
- package/.bkit/core/.bkit/prompts.md +295 -264
- package/.bkit/core/.bkit/providers.md +460 -277
- package/.bkit/core/.bkit/router.md +69 -0
- package/.bkit/core/.bkit/templates.md +530 -256
- package/.bkit/core/.bkit/trackers.md +116 -65
- package/.bkit/core/.bkit/workflows.md +742 -618
- package/.bkit/core/.claude/hooks/bkit-gateguard.cjs +2 -2
- package/.bkit/core/.claude/hooks/bkit-guardrails.cjs +3 -3
- package/.bkit/core/.claude/hooks/bkit-preflight.sh +45 -14
- package/.bkit/core/.codex/hooks/bkit-guardrails.cjs +3 -3
- package/.bkit/core/.codex/hooks/bkit-rules.cjs +57 -0
- package/.bkit/core/.cursor/hooks/bkit-guardrails.cjs +7 -7
- package/.bkit/core/.cursor/hooks/bkit-preflight.sh +47 -12
- package/.bkit/core/.opencode/plugin/bkit-rules.js +89 -0
- package/.bkit/core/AGENTS.md +6 -80
- package/.bkit/core/docs/templates/bkit-audit-accessibility.md +128 -130
- package/.bkit/core/docs/templates/bkit-audit-adoption.md +104 -94
- package/.bkit/core/docs/templates/bkit-audit-agents.md +96 -98
- package/.bkit/core/docs/templates/bkit-audit-aio-seo.md +100 -103
- package/.bkit/core/docs/templates/bkit-audit-analytics.md +141 -107
- package/.bkit/core/docs/templates/bkit-audit-annotations.md +99 -97
- package/.bkit/core/docs/templates/bkit-audit-app-performance.md +122 -121
- package/.bkit/core/docs/templates/bkit-audit-app-store.md +158 -101
- package/.bkit/core/docs/templates/bkit-audit-architecture.md +114 -96
- package/.bkit/core/docs/templates/bkit-audit-ci-performance.md +76 -97
- package/.bkit/core/docs/templates/bkit-audit-config.md +123 -101
- package/.bkit/core/docs/templates/bkit-audit-cost.md +86 -99
- package/.bkit/core/docs/templates/bkit-audit-coverage.md +76 -93
- package/.bkit/core/docs/templates/bkit-audit-database.md +102 -95
- package/.bkit/core/docs/templates/bkit-audit-dead-code.md +165 -108
- package/.bkit/core/docs/templates/bkit-audit-deferrals.md +115 -109
- package/.bkit/core/docs/templates/bkit-audit-dependencies.md +265 -113
- package/.bkit/core/docs/templates/bkit-audit-documentation.md +119 -118
- package/.bkit/core/docs/templates/bkit-audit-duplication.md +140 -111
- package/.bkit/core/docs/templates/bkit-audit-failures.md +111 -106
- package/.bkit/core/docs/templates/bkit-audit-flakes.md +100 -99
- package/.bkit/core/docs/templates/bkit-audit-gates.md +107 -103
- package/.bkit/core/docs/templates/bkit-audit-issues.md +105 -105
- package/.bkit/core/docs/templates/bkit-audit-llm.md +134 -85
- package/.bkit/core/docs/templates/bkit-audit-logs.md +124 -125
- package/.bkit/core/docs/templates/bkit-audit-models.md +99 -78
- package/.bkit/core/docs/templates/bkit-audit-privacy.md +56 -76
- package/.bkit/core/docs/templates/bkit-audit-prompts.md +63 -78
- package/.bkit/core/docs/templates/bkit-audit-red-team.md +125 -93
- package/.bkit/core/docs/templates/bkit-audit-reliability.md +97 -96
- package/.bkit/core/docs/templates/bkit-audit-repo.md +170 -98
- package/.bkit/core/docs/templates/bkit-audit-security.md +207 -121
- package/.bkit/core/docs/templates/bkit-audit-slop.md +78 -95
- package/.bkit/core/docs/templates/bkit-audit-testing.md +163 -139
- package/.bkit/core/docs/templates/bkit-audit-usability.md +114 -96
- package/.bkit/core/docs/templates/bkit-plan.md +158 -281
- package/.bkit/core/engine/checks/after/auth.ts +40 -37
- package/.bkit/core/engine/checks/after/ci-yml.ts +104 -31
- package/.bkit/core/engine/checks/after/github.ts +95 -17
- package/.bkit/core/engine/checks/after/index.ts +21 -8
- package/.bkit/core/engine/checks/after/suppression.ts +8 -5
- package/.bkit/core/engine/checks/after/tools.ts +125 -123
- package/.bkit/core/engine/checks/before/context-fetch.ts +227 -33
- package/.bkit/core/engine/checks/before/env.ts +3 -3
- package/.bkit/core/engine/checks/before/files.ts +5 -4
- package/.bkit/core/engine/checks/before/index.ts +25 -1
- package/.bkit/core/engine/checks/before/repo.ts +208 -2
- package/.bkit/core/engine/checks/status-holes.ts +916 -123
- package/.bkit/core/engine/checks/tips/catalogue.ts +82 -20
- package/.bkit/core/engine/checks/tips/detect.ts +66 -0
- package/.bkit/core/engine/checks/tips/index.ts +52 -11
- package/.bkit/core/engine/cli.ts +251 -130
- package/.bkit/core/engine/commands/analysis.ts +282 -199
- package/.bkit/core/engine/commands/ci/ci.ts +363 -29
- package/.bkit/core/engine/commands/ci/pins.ts +172 -0
- package/.bkit/core/engine/commands/ci/push-gate.ts +223 -0
- package/.bkit/core/engine/commands/comply.ts +77 -103
- package/.bkit/core/engine/commands/install/doctor.ts +2 -2
- package/.bkit/core/engine/commands/install/install.ts +57 -18
- package/.bkit/core/engine/commands/install/uninstall.ts +86 -55
- package/.bkit/core/engine/commands/install/upgrade.ts +32 -15
- package/.bkit/core/engine/commands/land.ts +139 -18
- package/.bkit/core/engine/commands/lib.ts +381 -22
- package/.bkit/core/engine/commands/registry.ts +10 -2
- package/.bkit/core/engine/commands/setup/setup.ts +8 -7
- package/.bkit/core/engine/commands/stats.ts +260 -0
- package/.bkit/core/engine/commands/tips.ts +10 -5
- package/.bkit/core/engine/commands/workflow/branches.ts +2215 -176
- package/.bkit/core/engine/commands/workflow/challenge.ts +748 -0
- package/.bkit/core/engine/commands/workflow/completeness.ts +109 -0
- package/.bkit/core/engine/commands/workflow/context.ts +1249 -47
- package/.bkit/core/engine/commands/workflow/counters.ts +483 -95
- package/.bkit/core/engine/commands/workflow/decisions.ts +585 -0
- package/.bkit/core/engine/commands/workflow/disk.ts +238 -56
- package/.bkit/core/engine/commands/workflow/fids.ts +811 -33
- package/.bkit/core/engine/commands/workflow/issues.ts +413 -64
- package/.bkit/core/engine/commands/workflow/ledgers.ts +1333 -36
- package/.bkit/core/engine/commands/workflow/locks.ts +28 -2
- package/.bkit/core/engine/commands/workflow/plan-blocks.ts +691 -0
- package/.bkit/core/engine/commands/workflow/plan-versions.ts +21 -0
- package/.bkit/core/engine/commands/workflow/plans.ts +257 -0
- package/.bkit/core/engine/commands/workflow/prs.ts +178 -33
- package/.bkit/core/engine/commands/workflow/repo.ts +66 -8
- package/.bkit/core/engine/commands/workflow/roadmap.ts +72 -22
- package/.bkit/core/engine/commands/workflow/run-facts.ts +492 -0
- package/.bkit/core/engine/commands/workflow/runs.ts +827 -79
- package/.bkit/core/engine/commands/workflow/runtime.ts +25 -1
- package/.bkit/core/engine/commands/workflow/skills.ts +1 -0
- package/.bkit/core/engine/commands/workflow/status.ts +329 -33
- package/.bkit/core/engine/commands/workflow/step-table.ts +381 -0
- package/.bkit/core/engine/commands/workflow/steps.ts +841 -105
- package/.bkit/core/engine/commands/workflow/summary-findings.ts +177 -0
- package/.bkit/core/engine/commands/workflow/summary.ts +544 -0
- package/.bkit/core/engine/commands/workflow/tasks.ts +409 -0
- package/.bkit/core/engine/commands/workflow/test-blocks.ts +254 -0
- package/.bkit/core/engine/commands/workflow/tests-changed.ts +201 -0
- package/.bkit/core/engine/commands/workflow/tests.ts +2124 -330
- package/.bkit/core/engine/commands/workflow/tracker.ts +1195 -0
- package/.bkit/core/engine/commands/workflow/workflows.ts +1197 -235
- package/.bkit/core/engine/commands/workflow/worktrees.ts +543 -24
- package/.bkit/core/engine/install/cloud-env.ts +251 -121
- package/.bkit/core/engine/install/context-ci-workflow.ts +102 -0
- package/.bkit/core/engine/install/manifest.ts +67 -17
- package/.bkit/core/engine/install/modules/antigravity.ts +3 -2
- package/.bkit/core/engine/install/modules/cloud.ts +24 -16
- package/.bkit/core/engine/install/modules/codex.ts +15 -2
- package/.bkit/core/engine/install/modules/core.ts +1196 -45
- package/.bkit/core/engine/install/modules/cursor.ts +6 -6
- package/.bkit/core/engine/install/modules/opencode.ts +18 -6
- package/.bkit/core/engine/install/template-notes.ts +130 -0
- package/.bkit/core/engine/install/version-file.ts +9 -4
- package/.bkit/core/engine/install/write.ts +5 -0
- package/.bkit/core/engine/ledgers/compact.ts +124 -0
- package/.bkit/core/engine/ledgers/journal.ts +291 -0
- package/.bkit/core/engine/ledgers/migrate.ts +92 -0
- package/.bkit/core/engine/ledgers/status.ts +249 -21
- package/.bkit/core/engine/ledgers/telemetry.ts +1397 -253
- package/.bkit/core/engine/render/hosts.ts +118 -14
- package/.bkit/core/engine/render/models.ts +12 -0
- package/.bkit/core/engine/shared/bkit-latest.ts +26 -4
- package/.bkit/core/engine/shared/branches.ts +4 -1
- package/.bkit/core/engine/shared/dep-updater.ts +60 -0
- package/.bkit/core/engine/shared/egress.ts +8 -1
- package/.bkit/core/engine/shared/json.ts +25 -7
- package/.bkit/core/engine/shared/package-manager.ts +9 -0
- package/.bkit/core/engine/shared/settings.ts +243 -31
- package/.bkit/core/engine/shared/text.ts +361 -0
- package/.bkit/core/engine/shared/timeouts.ts +2 -0
- package/.bkit/core/engine/tools/cli-doc.ts +111 -0
- package/.bkit/core/engine/tools/grammar-density.ts +215 -0
- package/.bkit/core/engine/tools/instructions-sources.ts +832 -0
- package/.bkit/core/engine/tools/instructions.ts +1802 -0
- package/.bkit/core/hooks/antigravity-hook.js +25 -10
- package/.bkit/core/hooks/gateguard-engine.js +54 -29
- package/.bkit/core/hooks/guardrails-shell.js +4 -5
- package/.bkit/core/hooks/guardrails-tiers.js +10 -4
- package/.bkit/core/hooks/preflight-linear.sh +9 -9
- package/.bkit/core/hooks/preflight-rules.sh +8 -0
- package/.bkit/core/hooks/readguard-engine.js +29 -10
- package/.bkit/core/hooks/session-rules-advisory.js +61 -0
- package/.bkit/core/prompts/bkit-audit-generate.md +202 -94
- package/.bkit/core/prompts/bkit-build-challenge-1.md +76 -34
- package/.bkit/core/prompts/bkit-build-challenge-2.md +85 -38
- package/.bkit/core/prompts/bkit-build-install-skills.md +50 -28
- package/.bkit/core/prompts/bkit-build-plan-convert.md +85 -0
- package/.bkit/core/prompts/bkit-build-plan-current.md +59 -36
- package/.bkit/core/prompts/bkit-build-plan-gaps-remediate.md +101 -36
- package/.bkit/core/prompts/bkit-build-plan-tasks.md +149 -0
- package/.bkit/core/prompts/bkit-build-plan-tests.md +120 -0
- package/.bkit/core/prompts/bkit-build-plan-verify.md +104 -43
- package/.bkit/core/prompts/bkit-debug-abort-handover.md +47 -32
- package/.bkit/core/prompts/bkit-debug-failure-diagnose.md +109 -57
- package/.bkit/core/prompts/bkit-debug-failure-remediate.md +65 -39
- package/.bkit/core/prompts/bkit-debug-failure-reproduce.md +53 -31
- package/.bkit/core/prompts/bkit-debug-failure-soak.md +65 -34
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-1.md +46 -31
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-2.md +45 -31
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-3.md +43 -30
- package/.bkit/core/prompts/bkit-design-generate.md +80 -46
- package/.bkit/core/prompts/bkit-design-organise.md +83 -32
- package/.bkit/core/prompts/bkit-design-review-1.md +70 -50
- package/.bkit/core/prompts/bkit-design-review-2.md +65 -46
- package/.bkit/core/prompts/bkit-design-spike.md +64 -40
- package/.bkit/core/prompts/bkit-design-update.md +53 -28
- package/.bkit/core/prompts/bkit-plan-generate.md +101 -29
- package/.bkit/core/prompts/bkit-plan-review-1.md +78 -41
- package/.bkit/core/prompts/bkit-plan-review-2.md +78 -35
- package/.bkit/core/prompts/bkit-plan-scope-check.md +116 -0
- package/.bkit/core/prompts/bkit-plan-update.md +86 -31
- package/.bkit/core/prompts/bkit-ship-conflict-resolve.md +94 -50
- package/.bkit/core/prompts/bkit-ship-deferred-sweep.md +91 -41
- package/.bkit/core/prompts/bkit-ship-update-docs.md +45 -24
- package/.bkit/core/prompts/bkit-utility-compress-markdown.md +65 -49
- package/.bkit/core/prompts/bkit-utility-update-decisions.md +87 -42
- package/.bkit/core/prompts/bkit-utility-update-deferrals.md +81 -37
- package/.bkit/core/prompts/bkit-utility-update-documentation.md +43 -25
- package/.bkit/core/prompts/bkit-utility-update-failures.md +84 -37
- package/.bkit/core/prompts/bkit-utility-update-knowledge.md +88 -42
- package/.bkit/core/prompts/bkit-utility-update-roadmap.md +92 -39
- package/.bkit/core/prompts/bkit-utility-update-tracker.md +159 -60
- package/.bkit/core/workflows/bkit-auto.md +89 -58
- package/.bkit/core/workflows/bkit-build.md +967 -517
- package/.bkit/core/workflows/bkit-debug.md +555 -387
- package/.bkit/core/workflows/bkit-design.md +319 -283
- package/.bkit/core/workflows/bkit-plan.md +319 -342
- package/.bkit/core/workflows/bkit-ship.md +525 -328
- package/.bkit/shared/.agents/plugins/bkit/hooks.json +10 -0
- package/.bkit/shared/.bkit/.env.example +2 -2
- package/.bkit/shared/.claude/bkit-cloud.md +12 -10
- package/.bkit/shared/.claude/settings.json +14 -0
- package/.bkit/shared/.codex/bkit-cloud.md +98 -0
- package/.bkit/shared/.codex/hooks.json +20 -0
- package/.bkit/shared/.cursor/bkit-cloud.md +3 -3
- package/.bkit/shared/.cursor/mcp.json +1 -1
- package/.bkit/shared/.cursor/rules/bkit-rules.mdc +1 -1
- package/.bkit/shared/.gitattributes +1 -1
- package/.bkit/shared/.mcp.json +1 -1
- package/.bkit/shared/.opencode/opencode.json +12 -1
- package/.bkit/shared/CLAUDE.md +6 -2
- package/.bkit/shared/GEMINI.md +5 -2
- package/.bkit/shared/docs/architecture.md +4 -7
- package/.bkit/shared/docs/data-schema.md +4 -7
- package/.bkit/shared/docs/deployments.md +241 -168
- package/.bkit/shared/docs/design.md +11 -12
- package/.bkit/shared/docs/noise.md +8 -7
- package/.bkit/shared/docs/observability.md +5 -8
- package/.bkit/shared/docs/security.md +35 -17
- package/.bkit/shared/docs/setup.md +29 -18
- package/.bkit/shared/docs/technology.md +3 -6
- package/.bkit/shared/docs/testing.md +289 -174
- package/.bkit/shared/gitignore +6 -2
- package/.bkit/templates/.claude/hooks/bkit-cloud-bootstrap.sh +15 -16
- package/.bkit/templates/.cursor/Dockerfile +12 -8
- package/.bkit/templates/.cursor/environment.json +1 -1
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-bug.yml +10 -8
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-change.yml +45 -12
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-idea.yml +3 -3
- package/.bkit/templates/.github/renovate.json5 +34 -6
- package/.bkit/templates/.github/workflows/bkit-ci.yml +54 -51
- package/.bkit/templates/.github/workflows/bkit-cloud.yml +34 -33
- package/.bkit/templates/.github/workflows/bkit-health.yml +97 -0
- package/.bkit/templates/.github/workflows/bkit-renovate.yml +5 -4
- package/.bkit/templates/.github/workflows/bkit-trivy.yml +5 -4
- package/.bkit/templates/.husky/pre-commit +2 -2
- package/.bkit/templates/.husky/pre-push +96 -36
- package/.bkit/templates/.prettierignore +6 -0
- package/.bkit/templates/.vale.ini +27 -27
- package/.bkit/templates/context/bkit-cloudrun.md +23 -15
- package/.bkit/templates/context/bkit-expo.md +70 -52
- package/.bkit/templates/context/bkit-github.md +93 -56
- package/.bkit/templates/context/bkit-inngest.md +21 -15
- package/.bkit/templates/context/bkit-maestro.md +22 -16
- package/.bkit/templates/context/bkit-playwright.md +23 -17
- package/.bkit/templates/context/bkit-resend.md +12 -10
- package/.bkit/templates/context/bkit-sentry.md +30 -20
- package/.bkit/templates/context/bkit-serena.md +8 -7
- package/.bkit/templates/context/bkit-stripe.md +14 -13
- package/.bkit/templates/context/bkit-supabase.md +16 -13
- package/.bkit/templates/context/bkit-tavily.md +14 -11
- package/.bkit/templates/context/bkit-vercel.md +27 -18
- package/.bkit/templates/docs/capabilities.md +15 -14
- package/.bkit/templates/scripts/ci/{check-accepted-divergences.mjs → bkit-check-accepted-divergences.mjs} +6 -3
- package/.bkit/templates/scripts/ci/{check-accepted-divergences.test.mjs → bkit-check-accepted-divergences.test.mjs} +3 -3
- package/.bkit/templates/scripts/ci/{check-capstone.mjs → bkit-check-capstone.mjs} +2 -2
- package/.bkit/templates/scripts/ci/{check-capstone.test.mjs → bkit-check-capstone.test.mjs} +4 -4
- package/.bkit/templates/scripts/ci/{check-dependency-audit.mjs → bkit-check-dependency-audit.mjs} +103 -18
- package/.bkit/templates/scripts/ci/{check-dependency-audit.test.mjs → bkit-check-dependency-audit.test.mjs} +31 -3
- package/.bkit/templates/scripts/ci/bkit-machine-lock.sh +387 -0
- package/.bkit/templates/scripts/ci/bkit-test-awake.mjs +75 -0
- package/.bkit/templates/scripts/ci/bkit-test-awake.test.mjs +160 -0
- package/.bkit/templates/scripts/ci/lanes/{resolve-push-base.mjs → bkit-resolve-push-base.mjs} +3 -3
- package/.bkit/templates/scripts/ci/lanes/{resolve-push-base.test.mjs → bkit-resolve-push-base.test.mjs} +4 -4
- package/.bkit/templates/scripts/cleanup/bkit-disk-prune.mjs +10 -2
- package/.bkit/templates/scripts/graphify/bkit-graph-drift.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graph-eager.py +3 -3
- package/.bkit/templates/scripts/graphify/bkit-graph-embed.py +63 -0
- package/.bkit/templates/scripts/graphify/bkit-graph-imports.py +2 -2
- package/.bkit/templates/scripts/graphify/bkit-graph-pre-push.sh +69 -18
- package/.bkit/templates/scripts/graphify/bkit-graph-repair.py +3 -3
- package/.bkit/templates/scripts/graphify/bkit-graph-surface.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graphify-graphml.py +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graphify-rollup.py +197 -126
- package/.bkit/templates/scripts/graphify/bkit-graphify.sh +64 -41
- package/.bkit/templates/scripts/slop/bkit-slop-type-suppression.mjs +23 -7
- package/.bkit/templates/slop-scan.config.json +9 -2
- package/CHANGELOG.md +231 -0
- package/README.md +278 -89
- package/dist/checks/after/auth.js +36 -33
- package/dist/checks/after/ci-yml.js +91 -30
- package/dist/checks/after/github.js +91 -17
- package/dist/checks/after/index.js +22 -11
- package/dist/checks/after/suppression.js +8 -5
- package/dist/checks/after/tools.js +111 -114
- package/dist/checks/before/context-fetch.js +218 -33
- package/dist/checks/before/env.js +3 -3
- package/dist/checks/before/files.js +5 -3
- package/dist/checks/before/index.js +20 -3
- package/dist/checks/before/repo.js +197 -2
- package/dist/checks/status-holes.js +739 -114
- package/dist/checks/tips/catalogue.js +70 -19
- package/dist/checks/tips/detect.js +62 -0
- package/dist/checks/tips/index.js +51 -9
- package/dist/cli.js +234 -124
- package/dist/commands/analysis.js +174 -141
- package/dist/commands/ci/ci.js +302 -25
- package/dist/commands/ci/pins.js +142 -0
- package/dist/commands/ci/push-gate.js +194 -0
- package/dist/commands/comply.js +54 -91
- package/dist/commands/install/doctor.js +2 -2
- package/dist/commands/install/install.js +50 -19
- package/dist/commands/install/uninstall.js +82 -55
- package/dist/commands/install/upgrade.js +21 -11
- package/dist/commands/land.js +137 -17
- package/dist/commands/lib.js +295 -21
- package/dist/commands/registry.js +10 -2
- package/dist/commands/setup/setup.js +8 -7
- package/dist/commands/stats.js +175 -0
- package/dist/commands/tips.js +7 -3
- package/dist/commands/workflow/branches.js +1873 -152
- package/dist/commands/workflow/challenge.js +609 -0
- package/dist/commands/workflow/completeness.js +97 -0
- package/dist/commands/workflow/context.js +1045 -46
- package/dist/commands/workflow/counters.js +385 -86
- package/dist/commands/workflow/decisions.js +484 -0
- package/dist/commands/workflow/disk.js +214 -56
- package/dist/commands/workflow/fids.js +706 -33
- package/dist/commands/workflow/issues.js +349 -59
- package/dist/commands/workflow/ledgers.js +1123 -37
- package/dist/commands/workflow/locks.js +26 -2
- package/dist/commands/workflow/plan-blocks.js +501 -0
- package/dist/commands/workflow/plan-versions.js +22 -0
- package/dist/commands/workflow/plans.js +205 -0
- package/dist/commands/workflow/prs.js +154 -35
- package/dist/commands/workflow/repo.js +58 -8
- package/dist/commands/workflow/roadmap.js +58 -23
- package/dist/commands/workflow/run-facts.js +421 -0
- package/dist/commands/workflow/runs.js +670 -72
- package/dist/commands/workflow/runtime.js +15 -1
- package/dist/commands/workflow/skills.js +1 -0
- package/dist/commands/workflow/status.js +282 -34
- package/dist/commands/workflow/step-table.js +308 -0
- package/dist/commands/workflow/steps.js +703 -96
- package/dist/commands/workflow/summary-findings.js +117 -0
- package/dist/commands/workflow/summary.js +388 -0
- package/dist/commands/workflow/tasks.js +378 -0
- package/dist/commands/workflow/test-blocks.js +231 -0
- package/dist/commands/workflow/tests-changed.js +172 -0
- package/dist/commands/workflow/tests.js +1695 -305
- package/dist/commands/workflow/tracker.js +983 -0
- package/dist/commands/workflow/workflows.js +1014 -222
- package/dist/commands/workflow/worktrees.js +432 -26
- package/dist/install/cloud-env.js +233 -99
- package/dist/install/context-ci-workflow.js +85 -0
- package/dist/install/manifest.js +64 -17
- package/dist/install/modules/antigravity.js +3 -2
- package/dist/install/modules/cloud.js +25 -16
- package/dist/install/modules/codex.js +12 -2
- package/dist/install/modules/core.js +1090 -48
- package/dist/install/modules/cursor.js +6 -6
- package/dist/install/modules/opencode.js +13 -7
- package/dist/install/template-notes.js +116 -0
- package/dist/install/version-file.js +9 -4
- package/dist/ledgers/compact.js +87 -0
- package/dist/ledgers/journal.js +233 -0
- package/dist/ledgers/migrate.js +81 -0
- package/dist/ledgers/status.js +176 -13
- package/dist/ledgers/telemetry.js +1010 -125
- package/dist/render/hosts.js +104 -14
- package/dist/render/models.js +12 -0
- package/dist/shared/bkit-latest.js +26 -4
- package/dist/shared/branches.js +3 -1
- package/dist/shared/dep-updater.js +49 -0
- package/dist/shared/egress.js +8 -1
- package/dist/shared/json.js +20 -7
- package/dist/shared/package-manager.js +6 -0
- package/dist/shared/settings.js +181 -24
- package/dist/shared/text.js +333 -0
- package/dist/shared/timeouts.js +2 -0
- package/dist/tools/cli-doc.js +97 -0
- package/dist/tools/grammar-density.js +183 -0
- package/dist/tools/instructions-sources.js +720 -0
- package/dist/tools/instructions.js +1623 -0
- package/package.json +156 -51
- package/.bkit/core/.bkit/lite.md +0 -73
- package/.bkit/core/engine/commands/harvest.ts +0 -459
- package/.bkit/core/engine/commands/workflow/workstreams.ts +0 -166
- package/.bkit/core/engine/tools/wording.ts +0 -748
- package/.bkit/core/prompts/bkit-build-challenge-reconcile.md +0 -69
- package/.bkit/core/prompts/bkit-build-challenge-remediate.md +0 -71
- package/.bkit/core/prompts/bkit-build-challenge-tests.md +0 -74
- package/.bkit/core/prompts/bkit-build-plan-execute.md +0 -76
- package/.bkit/core/prompts/bkit-build-summary.md +0 -94
- package/.bkit/core/prompts/bkit-debug-summary.md +0 -96
- package/.bkit/core/prompts/bkit-design-summary.md +0 -107
- package/.bkit/core/prompts/bkit-plan-complexity-check.md +0 -67
- package/.bkit/core/prompts/bkit-plan-complexity-split.md +0 -88
- package/.bkit/core/prompts/bkit-plan-review-3.md +0 -93
- package/.bkit/core/prompts/bkit-plan-scope-check-1.md +0 -85
- package/.bkit/core/prompts/bkit-plan-scope-check-2.md +0 -85
- package/.bkit/core/prompts/bkit-plan-summary.md +0 -107
- package/.bkit/core/prompts/bkit-ship-summary.md +0 -78
- package/.bkit/templates/scripts/ci/bkit-local-stack-lock.sh +0 -194
- package/dist/commands/harvest.js +0 -353
- package/dist/commands/workflow/workstreams.js +0 -143
- package/dist/tools/wording.js +0 -680
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Filler.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Hedging.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/LLMSlop.yml +0 -0
- /package/.bkit/templates/.vale/styles/{House → Bkit}/Verbose.yml +0 -0
- /package/.bkit/templates/scripts/ci/{accepted-advisories.json → bkit-accepted-advisories.json} +0 -0
- /package/.bkit/templates/scripts/ci/{accepted-divergences.json → bkit-accepted-divergences.json} +0 -0
- /package/.bkit/templates/scripts/launchd/{launchd.env.example → bkit-launchd.env.example} +0 -0
- /package/.bkit/templates/scripts/launchd/{ai.bkit.launchd.plist → bkit-launchd.plist} +0 -0
|
@@ -1,157 +1,158 @@
|
|
|
1
|
-
<!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
|
|
2
|
-
|
|
3
1
|
<!-- bkit:start -->
|
|
4
2
|
|
|
5
3
|
{{BKIT_MANAGED_MARKER}}
|
|
6
4
|
|
|
5
|
+
<!-- Note: @bkit-audit-generate fills this template. The report lives at `docs/audits/{YYYYMM}/{slug}.md`, where `{YYYYMM}` is the UTC month of the run and `{slug}` is this file's name without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md`, where `{TIMESTAMP}` is the output of `date -u +%Y%m%dT%H%M%SZ`. NEVER overwrite an earlier report. The report is stamped with a `bkit:anchor` block that records the audited commit (`baseSha`) and branch (`baseRef`). NEVER write or edit that block by hand. -->
|
|
6
|
+
|
|
7
7
|
# Flake Triage
|
|
8
8
|
|
|
9
9
|
> [!CAUTION]
|
|
10
|
-
> **Purpose:**
|
|
10
|
+
> **Purpose:** A diagnosis of the tests that pass only on retry, or fail on one surface and pass on another, and the change that would remove each one.
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> -
|
|
15
|
-
> -
|
|
16
|
-
> -
|
|
17
|
-
> -
|
|
18
|
-
> - Use dense token-efficient phrasing; no filler / hedging / restating; tables > prose, bullets > paragraphs.
|
|
19
|
-
> - Read `.agents/docs.md` → load docs for workflow / step / task; skip missing; cite docs inline where required.
|
|
20
|
-
> - Every claim cites this run's evidence (`file:line`, or command / query and its output) and tool behind it; a previous report's figure, list, or status is a lead, not evidence.
|
|
21
|
-
> - Read this document once at session start; do not trust session memory; do not re-read on every loop iteration.
|
|
22
|
-
> - never `.skip`, quarantine, delete an assertion, or widen a timeout to green a lane. A timeout named before the trace is read is not a verdict.
|
|
23
|
-
> - Repro and repeat runs use the suite's test tenant only; NEVER production tenants, payments, or mail.
|
|
14
|
+
> - Never `.skip` a test, quarantine it, delete an assertion, or widen a timeout to green a lane.
|
|
15
|
+
> - A timeout named before the trace is read is not a verdict.
|
|
16
|
+
> - Run repros and repeat runs against the suite's test tenant only.
|
|
17
|
+
> - NEVER touch production tenants, payments or mail.
|
|
24
18
|
|
|
25
19
|
## Universal Instructions
|
|
26
20
|
|
|
27
21
|
### Context & Goals
|
|
28
22
|
|
|
29
|
-
|
|
30
|
-
- Standing danger: read flake as timing → widen timeout; reported error usually consequence; timeout bump before trace read turns real bug into slow green.
|
|
31
|
-
- Load-scaled budget on shared dev machines: sampled once at config / worker load → stale mid-run (a load ramp starves late files); re-read per test file (vitest `vi.setConfig` in a setup file, jest `jest.setTimeout` in setup), same base + cap, factor pinned 1 on CI. Freshness fix = mechanism fix, not a widen.
|
|
32
|
-
- Target failure: **a skip, quarantine, or wider timeout standing in for a named mechanism.**
|
|
23
|
+
A ledger row can record that a flake exists, but the useful part is the mechanism, and that never fits in one line. The standing danger is reading a flake as timing and widening the timeout. The reported error is usually a consequence, and a timeout bump made before the trace is read turns a real bug into a slow green. On shared dev machines, a load-scaled time budget sampled once at config or worker load goes stale mid-run, because a load ramp starves the later files. Re-read it per test file (vitest `vi.setConfig` in a setup file, jest `jest.setTimeout` in setup), with the same base and cap, and the factor pinned to 1 on CI. That freshness fix is a mechanism fix, not a widen. The failure this audit targets is a skip, a quarantine or a wider timeout standing in for a named mechanism.
|
|
33
24
|
|
|
34
|
-
**Applies when:** suite runs more than once
|
|
25
|
+
**Applies when:** the suite runs more than once, in CI or as a local repeat. **If** it does not → write one `N/A` row with its evidence, and never report the surface as "clean".
|
|
35
26
|
|
|
36
|
-
**Cadence:**
|
|
27
|
+
**Cadence:** Run once every two weeks, over every workflow's runs since the previous report. Use the same window as `bkit-audit-logs`, because both read the same CI runs. A flake is not quarantined, so it stays until it is fixed, and its age is measured, not remembered. Run an extra pass, for one flake or red run, when a test on the integration branch passes only on retry or fails on one surface and passes on another; after a change to retries, workers, timeouts, trace or a shared fixture; and after a fix lands, to measure the repeat rate for the run count `Fix` named.
|
|
37
28
|
|
|
38
|
-
**Anchor Standards:**
|
|
29
|
+
**Anchor Standards:**
|
|
39
30
|
|
|
40
31
|
- Flake root-cause taxonomy (Luo et al., "An Empirical Analysis of Flaky Tests", FSE 2014).
|
|
41
32
|
- Google, "Flaky Tests at Google and How We Mitigate Them" (Google Testing Blog, 2016).
|
|
42
|
-
- Order-dependence and root-cause location (e.g
|
|
43
|
-
- Browser-runner retries, flaky status, repeat-each, fail-on-flaky (e.g
|
|
44
|
-
- Unit-runner shuffle, seed, and retry (e.g
|
|
33
|
+
- Order-dependence and root-cause location (e.g., iDFlakies, Lam et al., ICST 2019; Ziftci and Cavalcanti, "De-Flake Your Tests", ICSME 2020).
|
|
34
|
+
- Browser-runner retries, flaky status, repeat-each, fail-on-flaky (e.g., Playwright); record the version.
|
|
35
|
+
- Unit-runner shuffle, seed, and retry (e.g., Vitest `sequence.shuffle` / `sequence.seed` / `retry`, pytest-randomly); record the version.
|
|
45
36
|
- bkit block of `docs/testing.md` (bkit version from `.bkit/version`): § Which resources may each lane touch?; § E2E on PRs: run the full suite, watch which DB it hits; § Concurrent invocations collide on disk, not just in the database
|
|
46
37
|
|
|
47
38
|
### Scope
|
|
48
39
|
|
|
49
|
-
-
|
|
50
|
-
-
|
|
51
|
-
- CI
|
|
52
|
-
-
|
|
53
|
-
-
|
|
40
|
+
- Take every runner-reported flake since the previous report, and every test that failed on one run or surface and passed on another.
|
|
41
|
+
- Measure the failure rate, read the log up to the first runner line, and read the trace.
|
|
42
|
+
- Compare CI with local, and check vendor incidents, before blaming the test.
|
|
43
|
+
- Take the fixture-sharing set from tooling, not from memory.
|
|
44
|
+
- Check timezone and date, and order dependence.
|
|
45
|
+
- Check device flakes: boot, retries, animation and artifact locations.
|
|
46
|
+
- Give each flake one mechanism, one root-cause class, one verdict, and the change that would remove it.
|
|
54
47
|
|
|
55
48
|
### Out of Scope
|
|
56
49
|
|
|
57
50
|
<!-- prettier-ignore -->
|
|
58
51
|
| Excluded item | Why | Where it belongs |
|
|
59
52
|
| --- | --- | --- |
|
|
60
|
-
| Whether the right behaviours are tested |
|
|
61
|
-
| Log content beyond the flake's run |
|
|
62
|
-
| CI annotation quality |
|
|
63
|
-
| Vendor-incident catalogue |
|
|
64
|
-
| Quarantine, skip, or a widened timeout |
|
|
65
|
-
| Applying the named fix |
|
|
53
|
+
| Whether the right behaviours are tested | The testing sweep covers it | `bkit-audit-testing` |
|
|
54
|
+
| Log content beyond the flake's run | The log triage covers it | `bkit-audit-logs` |
|
|
55
|
+
| CI annotation quality | The annotations sweep covers it | `bkit-audit-annotations` |
|
|
56
|
+
| Vendor-incident catalogue | The dependencies sweep covers it | `bkit-audit-dependencies` |
|
|
57
|
+
| Quarantine, skip, or a widened timeout | It is never an outcome | — |
|
|
58
|
+
| Applying the named fix | The audit only reports | A plan made from this report |
|
|
66
59
|
|
|
67
60
|
### Verdicts
|
|
68
61
|
|
|
69
|
-
|
|
62
|
+
Give each flake one verdict. The root-cause class (`Mechanism`) is the evidence, and the verdicts below are the outcomes. A quarantine, a `.skip` or a widened timeout is never an outcome.
|
|
70
63
|
|
|
71
|
-
- **FIX NAMED
|
|
72
|
-
- **BUG
|
|
73
|
-
- **INFRASTRUCTURE
|
|
74
|
-
- **PARKED
|
|
75
|
-
- **FIXED
|
|
76
|
-
- **not ESTABLISHED
|
|
77
|
-
- **OTHER
|
|
64
|
+
- **FIX NAMED:** the mechanism is named. The report names the change that would remove it, with the runs that prove it (`Fix`).
|
|
65
|
+
- **BUG:** a real defect is behind the flake. It gets its own issue and a `docs/failures.jsonl` row.
|
|
66
|
+
- **INFRASTRUCTURE:** a host, preview, database or vendor incident explains it. It is not charged to the test.
|
|
67
|
+
- **PARKED:** the fix waits on a decision, and the report names the decision.
|
|
68
|
+
- **FIXED:** a fix landed since the previous report, and the repeated rate measurement ran the count `Fix` named with no failure.
|
|
69
|
+
- **not ESTABLISHED:** no mechanism fits. Say what is not established and what would establish it. The flake stays open.
|
|
70
|
+
- **OTHER:** it fits no other verdict. Say why. **If** the same `OTHER` shows up on 2 runs → propose a new verdict in `Process Improvements`.
|
|
78
71
|
|
|
79
72
|
### Pre-Flight / Dependencies
|
|
80
73
|
|
|
81
|
-
**Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
|
|
82
|
-
|
|
83
|
-
**Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
|
|
84
|
-
|
|
85
74
|
#### Core
|
|
86
75
|
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
- [ ] **
|
|
90
|
-
- [ ] **
|
|
91
|
-
- [ ] **Repeat Runner:** Steps 4, 7, 8 — repeat with retries off, e.g. Playwright `--repeat-each`, pytest-repeat. Fallback: rate `Not Checked`; do not mark `FIX NAMED` on unmeasured flake unless mechanism deterministic and report says rate unknown.
|
|
92
|
-
- [ ] **Web:** Steps 6, 13 — vendor status pages and anchor standards. Fallback: incidents and versions recorded as unread.
|
|
76
|
+
- [ ] **Permissions:** every blocked call is recorded.
|
|
77
|
+
- [ ] **CI History:** Steps 2, 3 — workflow runs since the previous report (e.g., `gh run list`, the Actions API, GitLab pipelines). Log access: the `context-fetch` blocks below `bkit:end` in `.agents/context.md` are ready-made read-only readers. `bkit context list-blocks` prints each block's `context` and `body`. Set a body's `# Customise:` knobs to the audit window, and run it with `direnv exec . bash`. Fallback: local repeats only, and mark the ranked CI list `Not Checked`.
|
|
78
|
+
- [ ] **Repeat Runner:** Steps 4, 7, 8 — repeat runs with retries off (e.g., Playwright `--repeat-each`, pytest-repeat). Fallback: mark the rate `Not Checked`. Do not mark an unmeasured flake `FIX NAMED` unless its mechanism is deterministic and the report says the rate is unknown.
|
|
79
|
+
- [ ] **Web:** Steps 6, 13 — vendor status pages and the anchor standards. Fallback: record the versions as unread, and the incidents as well.
|
|
93
80
|
|
|
94
81
|
#### Conditional
|
|
95
82
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
- [ ] **
|
|
99
|
-
- [ ] **
|
|
100
|
-
- [ ] **Vendor Status:** Failure time may overlap vendor window → Step 6 — e.g. GitHub status, hosting preview, database host. Fallback: `INFRASTRUCTURE` `Not Checked`.
|
|
101
|
-
- [ ] **Device Runner:** Native flake → Step 9 — e.g. Maestro, EAS, XCUITest. Fallback: native flakes `Not Checked`.
|
|
83
|
+
- [ ] **Traces:** **If** the suite has browser E2E → Step 5 — the trace, video and screenshot of the failed attempt (e.g., Playwright trace, Cypress). Fallback: the log only, and mark geometry claims `Not Checked`.
|
|
84
|
+
- [ ] **Language Server:** **If** the language is typed → Steps 6, 11 — which specs share a fixture (e.g., Serena, `tsserver`). Fallback: text search, and name it. Guessing the sharing set misattributes the diagnosis.
|
|
85
|
+
- [ ] **Vendor Status:** **If** the failure time may overlap a vendor window → Step 6 — e.g., GitHub status, the hosting preview, the database host. Fallback: mark `INFRASTRUCTURE` `Not Checked`.
|
|
86
|
+
- [ ] **Device Runner:** **If** a flake is native → Step 9 — e.g., Maestro, EAS, XCUITest. Fallback: mark native flakes `Not Checked`.
|
|
102
87
|
|
|
103
88
|
### 🔒 Gate 1: Prerequisites Check
|
|
104
89
|
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
108
|
-
- [ ] **Gate 1 - Verified:** All Core items checked. All applicable Conditional items checked.
|
|
90
|
+
- [ ] **Gaps named:** every missing item names its fallback and its `Not Checked` entry.
|
|
91
|
+
- [ ] **Gate 1 — Verified:** every item above, every Core item and every applicable Conditional item is checked.
|
|
109
92
|
|
|
110
93
|
### Probe Steps
|
|
111
94
|
|
|
112
|
-
- [ ] **Step 1 — Previous Run:**
|
|
113
|
-
- [ ] **Step 2 — List:** Before triage, list every test with runner status flaky (failed, passed on retry) and every pass-only-on-retry, across every workflow since previous report. Also list
|
|
114
|
-
- [ ] **Step 3 — Pin:**
|
|
115
|
-
- [ ] **Step 4 — Rate:**
|
|
116
|
-
- [ ] **Step 5 — Log:**
|
|
117
|
-
-
|
|
118
|
-
-
|
|
119
|
-
-
|
|
120
|
-
-
|
|
121
|
-
-
|
|
122
|
-
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
95
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** `Previous Run` holds the closure rate, or the run is marked baseline.
|
|
96
|
+
- [ ] **Step 2 — List:** Before triage, list every test with the runner status flaky (failed, then passed on retry) and every pass-only-on-retry, across every workflow since the previous report. Also list the tests that failed on one run or surface and passed on another. With retries off, `flaky` never appears, and a red run someone noticed is input, not the list. Rank by flake rate (flaky or failed runs ÷ runs) and by the reruns caused. For each open flake, record first seen, last seen and age, and report the median age of the open flakes. A red run that is absent from the list is a detection finding. Triage in rank order. **Done when:** `Flakes` holds the ranked list, the ages and the median.
|
|
97
|
+
- [ ] **Step 3 — Pin:** For each triaged flake, record the run id, the commit, and whether the job failed itself or only downstream. Record a control run and its valid scope. An earlier red run is a control only when its failure mode differs. For example, an auth failure that redirects every lane to sign-in is a control for the tenant-minting lane only. **Done when:** `Pin` holds the run, the job and the control.
|
|
98
|
+
- [ ] **Step 4 — Rate:** Run the spec with repeat-each, retries off, at the CI worker count, and record k failures in N runs. Repeat with the CPU throttled or loaded. Record N, k, the workers and the commit. Without a measured rate, there is nothing to compare a fix against (Step 11). **Done when:** `Rate` holds N, k, the workers and the commit, or why the rate was not measured.
|
|
99
|
+
- [ ] **Step 5 — Log:** The reported error is usually a consequence.
|
|
100
|
+
- Read up to the first runner line. A closed target or a crashed browser is downstream of a test timeout. An unmatched locator can sit in a passing assertion. A row-level denial gives silent zero rows, which looks like an empty mock.
|
|
101
|
+
- Open the trace, where the runner records one.
|
|
102
|
+
- Take geometry (the cell, y, in or out of the viewport) from the trace, or from a local repro at the same viewport, never from layout reasoning. A negative bounding box, or a pointer dragged out of the viewport, is that shape.
|
|
103
|
+
- Recognise these, and do not stop at them: an assertion at the runner default beside an explicit longer timeout, and a retrying helper that burns most of the test timeout before the body runs.
|
|
104
|
+
- Check the capture settings. Every CI browser and device runner keeps the trace, or the nearest artefact (log, video, screenshot), of the failed first attempt, not only the retry. For example, Playwright `retain-on-first-failure` and `retain-on-failure-and-retries` keep it, while `on-first-retry` records the retry, which is usually the passing attempt for a flake. A missing setting is a finding.
|
|
105
|
+
- From the trace or HAR, list the third-party requests (analytics, fonts, maps), and recommend a route stub for any that block load or response.
|
|
106
|
+
|
|
107
|
+
**Done when:** `Log` holds the first runner line, the trace notes, the capture settings and the third-party requests.
|
|
108
|
+
|
|
109
|
+
- [ ] **Step 6 — Surfaces:** CI and local run different apps (e.g., a production build on a preview deploy, against a dev server reusing its process).
|
|
110
|
+
- **If** a test fails on one **and** not the other → check hydration timing, build output and a stale dev server before blaming the test.
|
|
111
|
+
- Record the CI and local `TZ`.
|
|
112
|
+
- Rule out infrastructure: compare the failure time with host status, database and vendor status, and the preview deploy and cold start when the project uses them. A flake that infrastructure explains gets the class `infrastructure` in Step 10 and the verdict `INFRASTRUCTURE`, and is not charged to the test.
|
|
113
|
+
- Make the same split inside CI. A worker-scoped fixture shared by many specs means a test that mutates the tenant fails whichever spec runs next.
|
|
114
|
+
- Take the sharing set from the language server, not from memory: which specs reference the fixture or helper, where a helper moved to, and type errors without a full typecheck. Reference counts from before the tests joined the project graph are stale.
|
|
115
|
+
- Pace the queries, because back-to-back reference queries give partial results; re-run surprisingly small sets.
|
|
116
|
+
- The language server cannot see a mock-factory key. For fixture wiring that is not typed (runner deps, CI YAML, scripts), use text search.
|
|
117
|
+
|
|
118
|
+
**Done when:** `Surfaces` holds the surface split, the infrastructure check and the sharing set.
|
|
119
|
+
|
|
120
|
+
- [ ] **Step 7 — Time:** **If** a flake touches dates → repro it locally with the clock pinned (e.g., Playwright `page.clock.setFixedTime`, `vi.setSystemTime`) at the failed date, at month end, and at a DST change. Run with the local `TZ`, then the CI one, and set `TZ` before the workers spawn. **Done when:** `Time` holds the results, or the skip with evidence that the flake does not touch dates.
|
|
121
|
+
- [ ] **Step 8 — Order:** **If** the runner is a unit runner → shuffle with a recorded seed (e.g., Vitest `--sequence.shuffle --sequence.seed=<n>`). **If** it is a browser runner → change the worker or shard count. Record the seed and the counts. This tests cross-spec contamination (e.g., the Step 6 shared fixture) by running, not only by the reference graph. **Done when:** `Order` holds the seed, the counts and the result.
|
|
122
|
+
- [ ] **Step 9 — Native:** **If** a flake is on a device runner (e.g., Maestro, EAS) → Steps 5-6 target the browser, so also check that boot finished before the flow started, device-cloud retries, and animation and keyboard timing. Record the artifact locations (screenshots, recordings, logs). **Done when:** `Native` holds the answers, or the skip with evidence.
|
|
123
|
+
- [ ] **Step 10 — Mechanism:** Teardown deletes the rows an event points at, so most diagnoses are corroborated by the mechanism, not by the row. Tell "fits" apart from "proven". Give one root-cause class: async wait, concurrency, order dependency, shared fixture state, resource leak, network, external service, time and date, randomness, unordered collection, infrastructure, or other (described). In Luo et al., async wait, concurrency and order dependency cover about 78% of classified flakes. Record the per-class counts across open flakes as a series. Before settling on a timing mechanism, run the floating-promise and missing-await rules on the spec and its helpers (e.g., `@typescript-eslint/no-floating-promises`, `playwright/missing-playwright-await`), and record the hits, so a missing `await` is not hidden behind timing. **If** a real defect is behind the flake → the verdict is `BUG`. **Done when:** `Mechanism` holds the class, what is not established, and the await hits.
|
|
124
|
+
- [ ] **Step 11 — Fix:** Name the cause and the change that removes it; making the change is the next step. Never `.skip`, quarantine, delete an assertion or widen a timeout. Before the repaired test is trusted, the next step puts the bug back, watches the test go red, restores the fix, and confirms `git diff` on the source file is empty. That is not optional for anything touching row-level access control or a cross-tenant boundary. A change to a shared helper or fixture is a signature change, so check references first. A one-spec fix made through a shared fixture causes the next unattributable failure. Putting the bug back proves a deterministic fix. A flaky fix also repeats Step 4: at rate p, about 3/p runs with no failure gives 95% confidence that the rate is below p (the rule of three), and the report states the count. Give one verdict from `Verdicts`. **Done when:** `Fix` holds the verdict, the change or why there is none, and the run count.
|
|
125
|
+
- [ ] **Step 12 — Route:** No finding goes in without evidence someone can check again: a `file:line`, or a query, run id or measurement with its tool. An empty language-server result is inconclusive, so corroborate it with text search. The report names the destinations. This run writes the `docs/failures.jsonl` rows. Sending to every other destination is the next step's job. Every flake gets a `docs/failures.jsonl` row with its class, rate and age. A named mechanism gets the recommended fix and the proof runs (Step 11). A real defect gets its own issue and a ledger row. A fix that waits on a decision gets the named decision. A mechanism the error tracker also saw goes to `bkit-audit-logs`. A retry-pass or warning that CI annotated goes to `bkit-audit-annotations`. A missing trace or capture setting, a third-party stub, or a missing `await` gets a recommended change (Steps 5, 10). **Done when:** every finding's Expected names its destination, including the ledger row.
|
|
126
|
+
- [ ] **Step 13 — Standards:** Confirm the current version of each anchor standard, and read what changed since the previous report's version. In `Standards Coverage`, give a row per Luo et al. class and per flake feature the runner offers (e.g., retries, flaky status, repeat-each, fail-on-flaky, sequence shuffle), marked checked (with the step), not applicable (saying why) or finding. Give each bkit anchor (a doc section, rule or issue shape) one row, marked followed (with the step or evidence), not applicable (saying why) or finding. A departure with no recorded reason (e.g., a `docs/decisions.jsonl` row or an accepted-divergence entry) is a finding. Mark each recommended practice the project does not apply as adopt (with a proposed step), reject (saying why) or not applicable. Do not change the steps during the run. **Done when:** no row is empty.
|
|
127
|
+
- [ ] **Step 14 — Anything Else:** (a) List everything since the previous report's commit that touches test stability (browser and unit config: retries, workers, timeouts and trace; shared fixtures and helpers; CI workflows and runners; the preview and DB for E2E; device flows), with a one-line verdict each. (b) In a bounded pass of about 30 minutes, note what a senior test-infrastructure engineer would flag that no step asks about. Read what happened since the previous run (incidents, `bkit-audit-logs`, new `docs/failures.jsonl` rows, vendor status), and note what no step would catch. **Done when:** `Beyond the Steps` holds both passes, or says `none`. **If** one class of issue is seen on 2 runs → propose a step for it in `Process Improvements`.
|
|
126
128
|
|
|
127
129
|
### 🔒 Gate 2: Probes Check
|
|
128
130
|
|
|
129
|
-
**
|
|
130
|
-
|
|
131
|
-
- [ ] **
|
|
132
|
-
- [ ] **
|
|
133
|
-
- [ ] **Verdicts:** every triaged flake has one verdict from `Verdicts`; none is skip, quarantine, or a widened timeout.
|
|
131
|
+
- [ ] **Steps complete:** every step is `[x]`, or skipped with evidence that its surface is absent.
|
|
132
|
+
- [ ] **Evidence:** every finding cites evidence someone can check again, and the tool that produced it.
|
|
133
|
+
- [ ] **Verdicts:** every triaged flake has one verdict from `Verdicts`.
|
|
134
|
+
- [ ] **No masking:** no verdict is a skip, a quarantine or a widened timeout.
|
|
134
135
|
- [ ] **Rates:** every triaged flake records N and k, or why the rate was not measured.
|
|
135
|
-
- [ ] **Gate 2
|
|
136
|
+
- [ ] **Gate 2 — Verified:** every item above holds.
|
|
136
137
|
|
|
137
138
|
## Findings
|
|
138
139
|
|
|
139
140
|
<!-- prettier-ignore -->
|
|
140
141
|
| Severity | Use |
|
|
141
142
|
| --- | --- |
|
|
142
|
-
| `CRITICAL` |
|
|
143
|
-
| `HIGH` |
|
|
144
|
-
| `MEDIUM` |
|
|
145
|
-
| `LOW` |
|
|
143
|
+
| `CRITICAL` | A flake masks a cross-tenant write, data loss, or a broken workflow with no workaround. |
|
|
144
|
+
| `HIGH` | A real defect is behind the flake. The only path is blocked, and the workaround is a rerun. |
|
|
145
|
+
| `MEDIUM` | An intermittent failure is named. The trace of the failed attempt is missing. A shared fixture can attribute a failure to the next spec. Reruns cost time repeatedly. |
|
|
146
|
+
| `LOW` | A vendor incident is not charged to the test. A timing theory is speculative, with no measured rate. |
|
|
146
147
|
|
|
147
|
-
|
|
148
|
+
Within a severity, list real defects first.
|
|
148
149
|
|
|
149
150
|
<!-- prettier-ignore -->
|
|
150
151
|
| ID | Severity | Verdict | Location | Evidence | Expected |
|
|
151
152
|
| --- | --- | --- | --- | --- | --- |
|
|
152
153
|
| PENDING | PENDING | PENDING | PENDING | PENDING | PENDING |
|
|
153
154
|
|
|
154
|
-
|
|
155
|
+
**If** the run finds nothing → write one row: `None`.
|
|
155
156
|
|
|
156
157
|
## Report
|
|
157
158
|
|
|
@@ -232,17 +233,17 @@ PENDING
|
|
|
232
233
|
|
|
233
234
|
### 🔒 Gate 3: Report Check
|
|
234
235
|
|
|
235
|
-
**
|
|
236
|
-
|
|
237
|
-
- [ ] **
|
|
238
|
-
- [ ] **
|
|
239
|
-
- [ ] **Series:** each open flake's rate and the median age beside previous report's numbers, or
|
|
240
|
-
- [ ] **Writes:** only this report
|
|
241
|
-
- [ ] **
|
|
242
|
-
- [ ] **
|
|
243
|
-
- [ ] **
|
|
244
|
-
|
|
245
|
-
|
|
236
|
+
- [ ] **No placeholders:** no `PENDING` is left.
|
|
237
|
+
- [ ] **Summary:** `Summary` gives the count per severity.
|
|
238
|
+
- [ ] **Verdict counts:** `Summary` gives the count per verdict.
|
|
239
|
+
- [ ] **Median age:** `Summary` gives the median age of open flakes.
|
|
240
|
+
- [ ] **Series:** the report shows each open flake's rate and the median age beside the previous report's numbers, or marks the run as baseline.
|
|
241
|
+
- [ ] **Writes:** this run changed only this report, the template's `Project Instructions` and its own `docs/failures.jsonl` rows.
|
|
242
|
+
- [ ] **Throwaways removed:** every throwaway worktree or remote target is removed.
|
|
243
|
+
- [ ] **Process Improvements:** `Universal` and `Project` are both filled.
|
|
244
|
+
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID is in exactly one `Suggested Issues` row.
|
|
245
|
+
- [ ] **Next Step:** every `Suggested Issues` row has a `Next Step`.
|
|
246
|
+
- [ ] **Gate 3 — Verified:** every item above holds.
|
|
246
247
|
|
|
247
248
|
<!-- bkit:end -->
|
|
248
249
|
|