@cbartens/bkit 5.5.0 → 6.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.bkit/README.md +30 -20
- package/.bkit/core/.agents/docs.md +1 -0
- package/.bkit/core/.agents/rules.md +51 -26
- package/.bkit/core/.bkit/README.md +29 -19
- package/.bkit/core/.bkit/authoring.md +117 -0
- package/.bkit/core/.bkit/cloud.md +11 -11
- package/.bkit/core/.bkit/comparison.md +5 -5
- package/.bkit/core/.bkit/install.md +20 -20
- package/.bkit/core/.bkit/jsonl.md +202 -122
- package/.bkit/core/.bkit/lite.md +1 -1
- package/.bkit/core/.bkit/pipeline.md +49 -52
- package/.bkit/core/.bkit/prompts.md +106 -167
- package/.bkit/core/.bkit/providers.md +29 -32
- package/.bkit/core/.bkit/templates.md +44 -18
- package/.bkit/core/.bkit/trackers.md +60 -28
- package/.bkit/core/.bkit/workflows.md +399 -359
- package/.bkit/core/AGENTS.md +16 -22
- package/.bkit/core/docs/templates/bkit-audit-accessibility.md +14 -38
- package/.bkit/core/docs/templates/bkit-audit-adoption.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-agents.md +19 -43
- package/.bkit/core/docs/templates/bkit-audit-aio-seo.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-analytics.md +14 -38
- package/.bkit/core/docs/templates/bkit-audit-annotations.md +11 -35
- package/.bkit/core/docs/templates/bkit-audit-app-performance.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-app-store.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-architecture.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-ci-performance.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-config.md +18 -42
- package/.bkit/core/docs/templates/bkit-audit-cost.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-coverage.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-database.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-dead-code.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-deferrals.md +15 -38
- package/.bkit/core/docs/templates/bkit-audit-dependencies.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-documentation.md +11 -35
- package/.bkit/core/docs/templates/bkit-audit-duplication.md +18 -42
- package/.bkit/core/docs/templates/bkit-audit-failures.md +13 -37
- package/.bkit/core/docs/templates/bkit-audit-flakes.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-gates.md +14 -38
- package/.bkit/core/docs/templates/bkit-audit-issues.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-llm.md +16 -40
- package/.bkit/core/docs/templates/bkit-audit-logs.md +14 -38
- package/.bkit/core/docs/templates/bkit-audit-models.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-privacy.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-prompts.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-red-team.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-reliability.md +15 -39
- package/.bkit/core/docs/templates/bkit-audit-repo.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-security.md +13 -37
- package/.bkit/core/docs/templates/bkit-audit-slop.md +14 -38
- package/.bkit/core/docs/templates/bkit-audit-testing.md +12 -36
- package/.bkit/core/docs/templates/bkit-audit-usability.md +16 -40
- package/.bkit/core/docs/templates/bkit-plan.md +97 -226
- package/.bkit/core/engine/checks/after/ci-yml.ts +39 -30
- package/.bkit/core/engine/checks/after/github.ts +101 -72
- package/.bkit/core/engine/checks/after/index.ts +10 -12
- package/.bkit/core/engine/checks/after/suppression.ts +8 -5
- package/.bkit/core/engine/checks/after/tools.ts +47 -92
- package/.bkit/core/engine/checks/before/env.ts +2 -2
- package/.bkit/core/engine/checks/before/files.ts +5 -4
- package/.bkit/core/engine/checks/before/index.ts +18 -1
- package/.bkit/core/engine/checks/before/repo.ts +168 -2
- package/.bkit/core/engine/checks/tips/catalogue.ts +16 -5
- package/.bkit/core/engine/checks/tips/detect.ts +34 -0
- package/.bkit/core/engine/checks/tips/index.ts +10 -7
- package/.bkit/core/engine/cli.ts +20 -53
- package/.bkit/core/engine/commands/analysis.ts +171 -140
- package/.bkit/core/engine/commands/ci/ci.ts +6 -6
- package/.bkit/core/engine/commands/comply.ts +77 -103
- package/.bkit/core/engine/commands/install/doctor.ts +2 -2
- package/.bkit/core/engine/commands/install/install.ts +57 -18
- package/.bkit/core/engine/commands/install/uninstall.ts +65 -28
- package/.bkit/core/engine/commands/land.ts +18 -12
- package/.bkit/core/engine/commands/lib.ts +79 -17
- package/.bkit/core/engine/commands/registry.ts +4 -0
- package/.bkit/core/engine/commands/setup/setup.ts +7 -7
- package/.bkit/core/engine/commands/stats.ts +204 -0
- package/.bkit/core/engine/commands/tips.ts +10 -5
- package/.bkit/core/engine/commands/workflow/branches.ts +741 -62
- package/.bkit/core/engine/commands/workflow/challenge.ts +673 -0
- package/.bkit/core/engine/commands/workflow/context.ts +853 -29
- package/.bkit/core/engine/commands/workflow/counters.ts +267 -84
- package/.bkit/core/engine/commands/workflow/decisions.ts +541 -0
- package/.bkit/core/engine/commands/workflow/disk.ts +2 -2
- package/.bkit/core/engine/commands/workflow/fids.ts +793 -33
- package/.bkit/core/engine/commands/workflow/issues.ts +427 -105
- package/.bkit/core/engine/commands/workflow/ledgers.ts +884 -27
- package/.bkit/core/engine/commands/workflow/locks.ts +26 -2
- package/.bkit/core/engine/commands/workflow/plan-blocks.ts +582 -0
- package/.bkit/core/engine/commands/workflow/plan-versions.ts +21 -0
- package/.bkit/core/engine/commands/workflow/prs.ts +166 -30
- package/.bkit/core/engine/commands/workflow/repo.ts +59 -7
- package/.bkit/core/engine/commands/workflow/roadmap.ts +16 -7
- package/.bkit/core/engine/commands/workflow/run-facts.ts +501 -0
- package/.bkit/core/engine/commands/workflow/runs.ts +179 -49
- package/.bkit/core/engine/commands/workflow/runtime.ts +22 -1
- package/.bkit/core/engine/commands/workflow/status.ts +142 -29
- package/.bkit/core/engine/commands/workflow/step-table.ts +339 -0
- package/.bkit/core/engine/commands/workflow/steps.ts +213 -88
- package/.bkit/core/engine/commands/workflow/summary.ts +398 -0
- package/.bkit/core/engine/commands/workflow/tests.ts +1013 -148
- package/.bkit/core/engine/commands/workflow/tracker.ts +1195 -0
- package/.bkit/core/engine/commands/workflow/workflows.ts +984 -207
- package/.bkit/core/engine/commands/workflow/workstreams.ts +11 -5
- package/.bkit/core/engine/commands/workflow/worktrees.ts +276 -22
- package/.bkit/core/engine/install/manifest.ts +11 -9
- package/.bkit/core/engine/install/modules/cloud.ts +21 -13
- package/.bkit/core/engine/install/modules/core.ts +325 -7
- package/.bkit/core/engine/install/modules/opencode.ts +2 -2
- package/.bkit/core/engine/install/version-file.ts +6 -2
- package/.bkit/core/engine/ledgers/migrate.ts +92 -0
- package/.bkit/core/engine/ledgers/status.ts +92 -15
- package/.bkit/core/engine/ledgers/telemetry.ts +1261 -254
- package/.bkit/core/engine/render/hosts.ts +4 -0
- package/.bkit/core/engine/render/models.ts +12 -0
- package/.bkit/core/engine/shared/branches.ts +4 -1
- package/.bkit/core/engine/shared/egress.ts +7 -0
- package/.bkit/core/engine/shared/json.ts +25 -7
- package/.bkit/core/engine/shared/settings.ts +202 -33
- package/.bkit/core/engine/shared/text.ts +361 -0
- package/.bkit/core/engine/tools/grammar-density.ts +215 -0
- package/.bkit/core/engine/tools/instructions-sources.ts +834 -0
- package/.bkit/core/engine/tools/instructions.ts +1751 -0
- package/.bkit/core/hooks/gateguard-engine.js +24 -23
- package/.bkit/core/hooks/guardrails-shell.js +2 -3
- package/.bkit/core/hooks/guardrails-tiers.js +7 -1
- package/.bkit/core/hooks/readguard-engine.js +2 -2
- package/.bkit/core/prompts/bkit-audit-generate.md +62 -25
- package/.bkit/core/prompts/bkit-build-challenge-1.md +40 -17
- package/.bkit/core/prompts/bkit-build-challenge-2.md +44 -17
- package/.bkit/core/prompts/bkit-build-install-skills.md +26 -13
- package/.bkit/core/prompts/bkit-build-plan-convert.md +74 -0
- package/.bkit/core/prompts/bkit-build-plan-current.md +26 -13
- package/.bkit/core/prompts/bkit-build-plan-execute.md +74 -26
- package/.bkit/core/prompts/bkit-build-plan-gaps-remediate.md +64 -23
- package/.bkit/core/prompts/bkit-build-plan-verify.md +57 -33
- package/.bkit/core/prompts/bkit-debug-abort-handover.md +30 -14
- package/.bkit/core/prompts/bkit-debug-failure-diagnose.md +34 -14
- package/.bkit/core/prompts/bkit-debug-failure-remediate.md +34 -17
- package/.bkit/core/prompts/bkit-debug-failure-reproduce.md +26 -13
- package/.bkit/core/prompts/bkit-debug-failure-soak.md +31 -16
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-1.md +21 -11
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-2.md +20 -11
- package/.bkit/core/prompts/bkit-design-brainstorm-ideate-3.md +20 -11
- package/.bkit/core/prompts/bkit-design-generate.md +33 -13
- package/.bkit/core/prompts/bkit-design-organise.md +25 -12
- package/.bkit/core/prompts/bkit-design-review-1.md +31 -15
- package/.bkit/core/prompts/bkit-design-review-2.md +28 -13
- package/.bkit/core/prompts/bkit-design-spike.md +30 -13
- package/.bkit/core/prompts/bkit-design-update.md +33 -19
- package/.bkit/core/prompts/bkit-plan-generate.md +75 -19
- package/.bkit/core/prompts/bkit-plan-review-1.md +50 -27
- package/.bkit/core/prompts/bkit-plan-review-2.md +59 -28
- package/.bkit/core/prompts/bkit-plan-scope-check.md +109 -0
- package/.bkit/core/prompts/bkit-plan-update.md +61 -24
- package/.bkit/core/prompts/bkit-ship-conflict-resolve.md +27 -15
- package/.bkit/core/prompts/bkit-ship-deferred-sweep.md +31 -16
- package/.bkit/core/prompts/bkit-ship-update-docs.md +30 -15
- package/.bkit/core/prompts/bkit-utility-compress-markdown.md +24 -14
- package/.bkit/core/prompts/bkit-utility-update-decisions.md +32 -14
- package/.bkit/core/prompts/bkit-utility-update-deferrals.md +29 -14
- package/.bkit/core/prompts/bkit-utility-update-documentation.md +28 -13
- package/.bkit/core/prompts/bkit-utility-update-failures.md +31 -14
- package/.bkit/core/prompts/bkit-utility-update-knowledge.md +33 -14
- package/.bkit/core/prompts/bkit-utility-update-roadmap.md +38 -17
- package/.bkit/core/prompts/bkit-utility-update-tracker.md +43 -20
- package/.bkit/core/workflows/bkit-auto.md +54 -46
- package/.bkit/core/workflows/bkit-build.md +577 -458
- package/.bkit/core/workflows/bkit-debug.md +348 -333
- package/.bkit/core/workflows/bkit-design.md +201 -237
- package/.bkit/core/workflows/bkit-plan.md +210 -321
- package/.bkit/core/workflows/bkit-ship.md +328 -278
- package/.bkit/shared/.bkit/.env.example +2 -2
- package/.bkit/shared/.claude/settings.json +11 -0
- package/.bkit/shared/.cursor/mcp.json +1 -1
- package/.bkit/shared/.cursor/rules/bkit-rules.mdc +1 -1
- package/.bkit/shared/.gitattributes +1 -1
- package/.bkit/shared/.mcp.json +1 -1
- package/.bkit/shared/.opencode/opencode.json +11 -0
- package/.bkit/shared/CLAUDE.md +2 -2
- package/.bkit/shared/GEMINI.md +2 -2
- package/.bkit/shared/docs/architecture.md +1 -1
- package/.bkit/shared/docs/data-schema.md +1 -1
- package/.bkit/shared/docs/deployments.md +16 -28
- package/.bkit/shared/docs/design.md +1 -1
- package/.bkit/shared/docs/noise.md +1 -1
- package/.bkit/shared/docs/observability.md +2 -2
- package/.bkit/shared/docs/security.md +1 -1
- package/.bkit/shared/docs/setup.md +1 -1
- package/.bkit/shared/docs/technology.md +1 -1
- package/.bkit/shared/docs/testing.md +27 -10
- package/.bkit/templates/.claude/hooks/bkit-cloud-bootstrap.sh +1 -1
- package/.bkit/templates/.github/ISSUE_TEMPLATE/bkit-change.yml +26 -5
- package/.bkit/templates/.github/workflows/bkit-ci.yml +22 -84
- package/.bkit/templates/.github/workflows/bkit-cloud.yml +21 -19
- package/.bkit/templates/.husky/pre-push +52 -13
- package/.bkit/templates/.prettierignore +7 -0
- package/.bkit/templates/scripts/ci/bkit-local-stack-lock.sh +194 -0
- package/.bkit/templates/scripts/ci/check-capstone.mjs +2 -4
- package/.bkit/templates/scripts/ci/check-capstone.test.mjs +4 -23
- package/.bkit/templates/scripts/ci/lanes/resolve-push-base.mjs +104 -23
- package/.bkit/templates/scripts/ci/lanes/resolve-push-base.test.mjs +103 -6
- package/.bkit/templates/scripts/cleanup/bkit-disk-prune.mjs +1 -1
- package/.bkit/templates/scripts/graphify/bkit-graph-pre-push.sh +3 -1
- package/.bkit/templates/scripts/prettier/bkit-insert-ignore.mjs +49 -29
- package/CHANGELOG.md +142 -0
- package/README.md +58 -34
- package/dist/checks/after/ci-yml.js +39 -29
- package/dist/checks/after/github.js +99 -72
- package/dist/checks/after/index.js +11 -13
- package/dist/checks/after/suppression.js +8 -5
- package/dist/checks/after/tools.js +40 -83
- package/dist/checks/before/env.js +2 -2
- package/dist/checks/before/files.js +5 -3
- package/dist/checks/before/index.js +15 -2
- package/dist/checks/before/repo.js +163 -2
- package/dist/checks/tips/catalogue.js +14 -5
- package/dist/checks/tips/detect.js +32 -0
- package/dist/checks/tips/index.js +7 -5
- package/dist/cli.js +19 -47
- package/dist/commands/analysis.js +109 -100
- package/dist/commands/ci/ci.js +6 -6
- package/dist/commands/comply.js +54 -91
- package/dist/commands/install/doctor.js +2 -2
- package/dist/commands/install/install.js +50 -19
- package/dist/commands/install/uninstall.js +59 -28
- package/dist/commands/land.js +16 -12
- package/dist/commands/lib.js +51 -16
- package/dist/commands/registry.js +4 -0
- package/dist/commands/setup/setup.js +7 -7
- package/dist/commands/stats.js +136 -0
- package/dist/commands/tips.js +7 -3
- package/dist/commands/workflow/branches.js +656 -56
- package/dist/commands/workflow/challenge.js +556 -0
- package/dist/commands/workflow/context.js +709 -30
- package/dist/commands/workflow/counters.js +208 -75
- package/dist/commands/workflow/decisions.js +441 -0
- package/dist/commands/workflow/disk.js +2 -2
- package/dist/commands/workflow/fids.js +689 -33
- package/dist/commands/workflow/issues.js +362 -99
- package/dist/commands/workflow/ledgers.js +762 -27
- package/dist/commands/workflow/locks.js +24 -2
- package/dist/commands/workflow/plan-blocks.js +437 -0
- package/dist/commands/workflow/plan-versions.js +22 -0
- package/dist/commands/workflow/prs.js +142 -32
- package/dist/commands/workflow/repo.js +52 -7
- package/dist/commands/workflow/roadmap.js +9 -7
- package/dist/commands/workflow/run-facts.js +426 -0
- package/dist/commands/workflow/runs.js +144 -43
- package/dist/commands/workflow/runtime.js +12 -1
- package/dist/commands/workflow/status.js +130 -29
- package/dist/commands/workflow/step-table.js +276 -0
- package/dist/commands/workflow/steps.js +171 -80
- package/dist/commands/workflow/summary.js +275 -0
- package/dist/commands/workflow/tests.js +834 -136
- package/dist/commands/workflow/tracker.js +983 -0
- package/dist/commands/workflow/workflows.js +834 -195
- package/dist/commands/workflow/workstreams.js +11 -5
- package/dist/commands/workflow/worktrees.js +221 -22
- package/dist/install/manifest.js +11 -9
- package/dist/install/modules/cloud.js +22 -13
- package/dist/install/modules/core.js +326 -7
- package/dist/install/modules/opencode.js +2 -2
- package/dist/install/version-file.js +6 -2
- package/dist/ledgers/migrate.js +81 -0
- package/dist/ledgers/status.js +61 -8
- package/dist/ledgers/telemetry.js +905 -124
- package/dist/render/hosts.js +4 -0
- package/dist/render/models.js +12 -0
- package/dist/shared/branches.js +3 -1
- package/dist/shared/egress.js +7 -0
- package/dist/shared/json.js +20 -7
- package/dist/shared/settings.js +147 -26
- package/dist/shared/text.js +333 -0
- package/dist/tools/grammar-density.js +183 -0
- package/dist/tools/instructions-sources.js +721 -0
- package/dist/tools/instructions.js +1579 -0
- package/package.json +114 -23
- package/.bkit/core/engine/commands/harvest.ts +0 -459
- package/.bkit/core/engine/tools/wording.ts +0 -748
- package/.bkit/core/prompts/bkit-build-challenge-reconcile.md +0 -69
- package/.bkit/core/prompts/bkit-build-challenge-remediate.md +0 -71
- package/.bkit/core/prompts/bkit-build-challenge-tests.md +0 -74
- package/.bkit/core/prompts/bkit-build-summary.md +0 -94
- package/.bkit/core/prompts/bkit-debug-summary.md +0 -96
- package/.bkit/core/prompts/bkit-design-summary.md +0 -107
- package/.bkit/core/prompts/bkit-plan-complexity-check.md +0 -67
- package/.bkit/core/prompts/bkit-plan-complexity-split.md +0 -88
- package/.bkit/core/prompts/bkit-plan-review-3.md +0 -93
- package/.bkit/core/prompts/bkit-plan-scope-check-1.md +0 -85
- package/.bkit/core/prompts/bkit-plan-scope-check-2.md +0 -85
- package/.bkit/core/prompts/bkit-plan-summary.md +0 -107
- package/.bkit/core/prompts/bkit-ship-summary.md +0 -78
- package/.bkit/templates/scripts/ci/lanes/check-tree-cache.mjs +0 -138
- package/.bkit/templates/scripts/ci/lanes/check-tree-cache.test.mjs +0 -181
- package/dist/commands/harvest.js +0 -353
- package/dist/tools/wording.js +0 -680
|
@@ -11,27 +11,19 @@
|
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> - Read-only audit; never edit code, config, or data.
|
|
15
|
-
> - `Universal Instructions` apply to all projects; `Project Instructions` apply only to this project.
|
|
16
|
-
> - Local audit instructions / overrides → `Project Instructions` below the bkit-managed block; universal instructions stay in-block.
|
|
17
|
-
> - Any tools mentioned in `Universal Instructions` are examples only; swap for actual tools in final output; drop inapplicable.
|
|
18
|
-
> - Use dense token-efficient phrasing; no filler / hedging / restating; tables > prose, bullets > paragraphs.
|
|
19
|
-
> - Read `.agents/docs.md` → load docs for workflow / step / task; skip missing; cite docs inline where required.
|
|
20
|
-
> - Every claim cites this run's evidence (`file:line`, or command / query and its output) and tool behind it; a previous report's figure, list, or status is a lead, not evidence.
|
|
21
|
-
> - Read this document once at session start; do not trust session memory; do not re-read on every loop iteration.
|
|
22
14
|
> - Generated registers / graphs → verify-only mode or throwaway copy; never regenerate a tracked file.
|
|
23
15
|
|
|
24
16
|
## Universal Instructions
|
|
25
17
|
|
|
26
18
|
### Context & Goals
|
|
27
19
|
|
|
28
|
-
Identical-signature gates catch identical copies; divergent duplication hurts — one behaviour, N implementations drifted apart, each locally correct, review and tests pass. Three shapes, invisible to others' detectors: **rule** (one expression inlined at N sites then edited — expression is not a symbol → text / structural search, not language server), **capability** (one operation, N modules or mechanisms — register, reference search, literal-mechanism search), **paired decision** (two files must change together, each owning its half, nothing tying them — co-change history). Shared home existing
|
|
20
|
+
Identical-signature gates catch identical copies; divergent duplication hurts — one behaviour, N implementations drifted apart, each locally correct, review and tests pass. Three shapes, invisible to others' detectors: **rule** (one expression inlined at N sites then edited — expression is not a symbol → text / structural search, not language server), **capability** (one operation, N modules or mechanisms — register, reference search, literal-mechanism search), **paired decision** (two files must change together, each owning its half, nothing tying them — co-change history). Shared home existing != every site calls it; documented convention does not stop drift. Until guard covers both altitudes, this sweep = prevention. Target failure: **copies of one piece of knowledge that silently disagree** — e.g. security control left `null` at call site remediation missed.
|
|
29
21
|
|
|
30
22
|
**Applies when:** source code spanning more than one module, package, or platform. None → one `N/A` row with evidence, never "clean".
|
|
31
23
|
|
|
32
24
|
**Cadence:** monthly. Extra pass when consolidation lands (re-check register row and call sites), new owner module or shared package lands, or production failure traces to divergent copies.
|
|
33
25
|
|
|
34
|
-
**Anchor Standards:**
|
|
26
|
+
**Anchor Standards:**
|
|
35
27
|
|
|
36
28
|
- Clone taxonomy Type-1 to Type-4 and clone detector at version run (e.g. jscpd clone-types guide, PMD CPD).
|
|
37
29
|
- Structural search at version run (e.g. ast-grep, Semgrep).
|
|
@@ -78,7 +70,7 @@ Identical-signature gates catch identical copies; divergent duplication hurts
|
|
|
78
70
|
|
|
79
71
|
**Confirm before `DUPLICATE`:**
|
|
80
72
|
|
|
81
|
-
- Duplicated name
|
|
73
|
+
- Duplicated name != deliberate platform split (e.g. `.native.tsx` / `.web.tsx`) or local UI handler.
|
|
82
74
|
- Bypass is gratuitous, not a surface the owner cannot serve (e.g. framework-specific request context).
|
|
83
75
|
- Similar defaults are one decision — a least-privilege read default and a signup write default are not; forcing one value breaks a flow.
|
|
84
76
|
- "Dead" / "unwired" copy has no config-only or deployed consumer (explicit worker registries, deployed functions).
|
|
@@ -88,17 +80,11 @@ Carry `COINCIDENTAL` and `RULE-OF-THREE-PENDING` forward in `Carried Forward`
|
|
|
88
80
|
|
|
89
81
|
### Pre-Flight / Dependencies
|
|
90
82
|
|
|
91
|
-
**Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
|
|
92
|
-
|
|
93
|
-
**Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
|
|
94
|
-
|
|
95
83
|
#### Core
|
|
96
84
|
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
- [ ] **
|
|
100
|
-
- [ ] **Clean Tree:** Steps 2–13 — clean `git status` at recorded commit; findings only mean something against a known tree. Fallback: detached worktree off default branch.
|
|
101
|
-
- [ ] **Text Search:** Steps 3, 5, 7–10 — inline expressions, literals, env names, non-code files, e.g. ripgrep, `git grep`. Fallback: `grep -rn`.
|
|
85
|
+
- [ ] **Permissions:** blocked calls recorded
|
|
86
|
+
- [ ] **Clean Tree:** Steps 2-13 — clean `git status` at recorded commit; findings only mean something against a known tree. Fallback: detached worktree off default branch.
|
|
87
|
+
- [ ] **Text Search:** Steps 3, 5, 7-10 — inline expressions, literals, env names, non-code files, e.g. ripgrep, `git grep`. Fallback: `grep -rn`.
|
|
102
88
|
- [ ] **Structural Search:** Steps 3, 12 — syntax patterns that skip comments / strings and catch reordered copies, e.g. ast-grep, Semgrep; ad hoc run, record resolved version. Fallback: text search; reformatted copies `Not Checked`.
|
|
103
89
|
- [ ] **Clone Detector:** Steps 4, 8 — Type-1 to Type-3 clones, e.g. jscpd, PMD CPD; ad hoc run, record resolved version. Fallback: `Clones` `Not Checked`.
|
|
104
90
|
- [ ] **Git History:** Steps 4, 10 — full history for clone baseline, blame, and co-change. Fallback: growth and coupling `Not Checked`.
|
|
@@ -106,8 +92,6 @@ Always verify.
|
|
|
106
92
|
|
|
107
93
|
#### Conditional
|
|
108
94
|
|
|
109
|
-
Verify only when applicable.
|
|
110
|
-
|
|
111
95
|
- [ ] **Language Server:** Typed language → Steps 3, 5, 13 — references on helpers and owners, e.g. Serena, `tsserver`, pyright. Fallback: text search only; caller sets = hypotheses; `one` vs `bypassed` unproven.
|
|
112
96
|
- [ ] **Module Graph:** Steps 5, 10 — duplicate symbol names, god nodes, import edges, e.g. graphify, dependency-cruiser, madge. Fallback: module-graph cross-check `Not Checked`.
|
|
113
97
|
- [ ] **Capability Register:** Register exists → Step 5 — capability → owner → status, verify-only mode. Fallback: rows derived in `Capabilities` only; no register = finding.
|
|
@@ -116,18 +100,16 @@ Verify only when applicable.
|
|
|
116
100
|
|
|
117
101
|
### 🔒 Gate 1: Prerequisites Check
|
|
118
102
|
|
|
119
|
-
**Checkpoint.** Verify and check `[x]` all applicable boxes above, then continue to `Probe Steps` without asking.
|
|
120
|
-
|
|
121
103
|
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
122
|
-
- [ ] **Gate 1
|
|
104
|
+
- [ ] **Gate 1 — Verified:** All Core items checked. All applicable Conditional items checked.
|
|
123
105
|
|
|
124
106
|
### Probe Steps
|
|
125
107
|
|
|
126
|
-
- [ ] **Step 1 — Previous Run:**
|
|
127
|
-
- [ ] **Step 2 — Detect:** List apps, packages, platforms; shared homes (utility / owner modules); capability register; duplication gates enforced and what each catches (e.g. identical-signature gate → identical copies only). **Done when:** surface map and existing gates in `Checked`, or `N/A` and STOP
|
|
108
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** closure rate in `Previous Run`, or run marked baseline.
|
|
109
|
+
- [ ] **Step 2 — Detect:** List apps, packages, platforms; shared homes (utility / owner modules); capability register; duplication gates enforced and what each catches (e.g. identical-signature gate → identical copies only). **Done when:** surface map and existing gates in `Checked`, or `N/A` and **STOP**.
|
|
128
110
|
- [ ] **Step 3 — Rule Altitude:** Hunt one conceptual operation inlined at N sites instead of called from one function, copies differing on load-bearing details: fallback chains (e.g. JS `a ?? b ?? c`, divergent terminal fallback or order); null vs falsy (e.g. JS `??` vs `||`, with / without trim, optional-chain guards); formatting (date / time / timezone, currency, name / initials, address); predicates ("is active / deleted / owner?" filters); defaults (same enum / config default hardcoded in several readers). Structural rules where possible, text search where not; keep rules under `tmp/`, paste each into report for next run. Subtraction per shared helper: references on helper (routed set) minus sites matching inline pattern (inline set) → inline-not-routed = unmigrated site. Counts before reference-graph change (e.g. project references added) = stale → re-take, never carry forward. **Done when:** `Inline Rules` row per rule: helper · routed · inline · unmigrated · verdict, with its search rule.
|
|
129
111
|
- [ ] **Step 4 — Clones:** Clone detector over source roots, machine-readable output, three passes: Type-1 exact; Type-2 renamed (ignore identifiers / literals); Type-3 near-miss (e.g. similarity 0.85, syntax-tree mode). Duplication % per package as series vs previous report. Every hit = candidate for `Verify`, not finding. Multiple platforms → counterpart listing (functions with / without counterpart per side). Growth: exact pass baselined at previous report's commit → clones new since vs consolidated since, blame naming introducer; arrival faster than consolidation = finding. **Done when:** per-package %, growth, counterpart list in `Clones`.
|
|
130
|
-
- [ ] **Step 5 — Capability Altitude:** Above file graph: capabilities, not files. Per capability: owner, callers, status, note. Status: `one` = one implementation, every caller routes through it; `bypassed` = owner exists, some callers skip it; `split` = 2+ competing implementations, no owner; `gap` = designed, not built. Three detectors — different findings: (a) literal-mechanism search for second mechanism (e.g. second SDK client constructor, rate limiter, email-send call, duplicated validation regex) — primary; symbol graph cannot see duplicated regex; (b) language-server references on each owner → caller set, `one` vs `bypassed`; (c) module-graph duplicate-symbol-name check — group callable nodes by name across distinct non-test source files, 3+ definitions = candidate; god-node list flags owner gone load-bearing. Register status
|
|
112
|
+
- [ ] **Step 5 — Capability Altitude:** Above file graph: capabilities, not files. Per capability: owner, callers, status, note. Status: `one` = one implementation, every caller routes through it; `bypassed` = owner exists, some callers skip it; `split` = 2+ competing implementations, no owner; `gap` = designed, not built. Three detectors — different findings: (a) literal-mechanism search for second mechanism (e.g. second SDK client constructor, rate limiter, email-send call, duplicated validation regex) — primary; symbol graph cannot see duplicated regex; (b) language-server references on each owner → caller set, `one` vs `bypassed`; (c) module-graph duplicate-symbol-name check — group callable nodes by name across distinct non-test source files, 3+ definitions = candidate; god-node list flags owner gone load-bearing. Register status != measured status = finding; register is not edited. **Done when:** `Capabilities` row per capability: owner · callers · measured status · register status · verdict.
|
|
131
113
|
- [ ] **Step 6 — Restated Types:** Typed language with schema source (generated DB types, migrations, IDL) → per core table: hand-written types, validation schemas, route validators restating columns, enums, constraints. Not derived from source = finding. Diff vs source: column added, renamed, or nullable on one side only = live defect. **Done when:** `Types` row per restated type: source · derived? · drift · verdict, or skip with evidence.
|
|
132
114
|
- [ ] **Step 7 — Libraries:** Literal search finds second client, not second library. List dependencies across every manifest doing one job: two date libraries, two validation libraries or two majors side by side, two HTTP clients, two markdown parsers. Dedupe check and resolution reason for every package at more than one version. Each = capability row with library as mechanism. **Done when:** `Libraries` lists every same-job pair and multi-version package with reason.
|
|
133
115
|
- [ ] **Step 8 — Outside Code:** SQL: access-policy bodies and helper functions redefined across migrations vs one function. Shared config: type-checker, linter, styling presets copied between apps vs extending shared package. i18n: same string under several keys. List formats clone detector supports; run over migrations and config; text search the rest. **Done when:** every hit = candidate for `Verify`; absent surface in `Checked`.
|
|
@@ -135,19 +117,17 @@ Verify only when applicable.
|
|
|
135
117
|
- [ ] **Step 10 — Coupling:** Per candidate set: co-change history (e.g. `git log --name-only`, code-maat). Always changed together → true duplicate, consolidate; changed separately → already diverged, decide which behaviour is right first. Reverse detector: file pairs in different packages, or one per platform, co-changing in >50% of commits with no import edge → paired-decision candidate. Comments tying two files by prose (e.g. `keep in sync`, `must match`, `mirrors`). Pair → straight to `Prevention`. **Done when:** `Coupling` row per candidate and pair: co-change share · import edge · verdict.
|
|
136
118
|
- [ ] **Step 11 — Confidence:** Name tool behind every count. Grade claim types: second implementation exists = high (every site read, not matched); exact call-site counts = medium; "no production consumer" = medium, needs in-code corroboration; "does not exist" = high only when absent from schema and all source. Language server under-reports → false `one`: pace queries; re-run surprising low / empty count before recording; empty = inconclusive, corroborate with text search; mocked-module factory keys and packages outside project graph invisible. Module graph: confirms presence, never absence; misses callers in anonymous callbacks and module-level initialisers; manifest-driven or type-only edges can fake cross-package edges and cycles; stale resolution cache after path-alias change. Language server and text search agreeing on count = strongest evidence; disagreement names file to open. Count corrected mid-run → record which, from what, by what tool. **Done when:** `Method` grades every claim type and names tool per count, incl. corrections.
|
|
137
119
|
- [ ] **Step 12 — Prevention:** Does a guard fail on re-implementation at both altitudes — inline rule expression outside its helper, and mechanism (constructor, send call, env name) outside its owner? Per-rule bespoke guard = one guard per consolidated field, forever; structural rule per pattern (owner-file ignores, rules checked in, seeded from Step 3 rules) plus literal search with owner allowlist (env names) covers both altitudes without language server. Record whether recorded decisions favour guard or sweep, and whether evidence since contradicts them. Do not build. **Done when:** `Prevention` states next step = guard or sweep, with evidence and the rules it would start from.
|
|
138
|
-
- [ ] **Step 13 — Verify:** Open every candidate file; cite `file:line` for every survivor. Apply `Verdicts` checks; re-check previous `Carried Forward` rows (third copy → `DUPLICATE`). Rank survivors by blast radius
|
|
120
|
+
- [ ] **Step 13 — Verify:** Open every candidate file; cite `file:line` for every survivor. Apply `Verdicts` checks; re-check previous `Carried Forward` rows (third copy → `DUPLICATE`). Rank survivors by blast radius x divergence x user-visibility. Per survivor name consolidation and home: rule → one function in correct shared package, inline copies removed; capability → one owner module with call contract, register row updated; wrong abstraction → inlined back; paired decision → paired-owner check or gate; drifted type or live defect → `docs/failures.jsonl` row, fix ahead of refactor; second library / version → `bkit-audit-dependencies`; deliberate bypass blocked on other work → register row naming blocker (unrecorded bypass reads `one` next run = false clean); one-off structural problem or missing module-graph evidence → `bkit-audit-architecture`. Consolidation changing shared signature → `Expected` names reference check first. **Done when:** every candidate has verdict, evidence, rank, and home.
|
|
139
121
|
- [ ] **Step 14 — Standards:** Confirm current version per anchor standard; read changes since previous report's version. `Standards Coverage`: one row per standard, per clone type (Type-1 to Type-4), and per shape (rule, capability, paired decision) → checked (name step) / not applicable (why) / finding. Unapplied recommended practice → adopt (propose step) / reject (why) / not applicable. **Done when:** no empty row.
|
|
140
122
|
- [ ] **Step 15 — Anything Else:** (a) Everything changed since previous report's commit touching duplication (new shared helpers / owner modules, shared packages, cross-platform counterparts of screen or hook, validation schemas and route validators, migrations adding tables / enums / policies, new dependencies, config presets, i18n keys) → one-line verdict each. (b) Bounded ~30 min pass: what senior engineer owning shared packages flags that no step asks; read since previous run (incidents, new `docs/failures.jsonl` rows) — what would no step catch? **Done when:** `Beyond the Steps` holds both passes, or `none`. Class seen on 2 runs → propose step in `Process Improvements`.
|
|
141
123
|
|
|
142
124
|
### 🔒 Gate 2: Probes Check
|
|
143
125
|
|
|
144
|
-
**Checkpoint.** Verify and check `[x]` all boxes below, then continue to `Findings` without asking.
|
|
145
|
-
|
|
146
126
|
- [ ] **Steps complete:** every step `[x]`, or explicit skip with evidence the surface is absent.
|
|
147
127
|
- [ ] **Evidence:** every finding cites re-checkable evidence and the tool behind it.
|
|
148
128
|
- [ ] **Sites read:** every `DUPLICATE` cites `file:line` from an opened file, not a match.
|
|
149
129
|
- [ ] **Tree clean:** no tracked file changed by this run (other writers' paths named); tracked registers / graphs unchanged.
|
|
150
|
-
- [ ] **Gate 2
|
|
130
|
+
- [ ] **Gate 2 — Verified:** Every checkbox above independently verified and checked off.
|
|
151
131
|
|
|
152
132
|
## Findings
|
|
153
133
|
|
|
@@ -156,10 +136,10 @@ Verify only when applicable.
|
|
|
156
136
|
| --- | --- |
|
|
157
137
|
| `CRITICAL` | Divergent copy disables a security / access control (e.g. rate limiter, auth, or tenancy check) or loses data; broken workflow with no workaround |
|
|
158
138
|
| `HIGH` | User-visible divergence between copies of one rule; hand-written type drifted from schema; paired decision out of step; `split` capability on a request path |
|
|
159
|
-
| `MEDIUM` | Copies consistent today (drift risk, no live bug); `bypassed` owner; register status
|
|
139
|
+
| `MEDIUM` | Copies consistent today (drift risk, no live bug); `bypassed` owner; register status != measured; wrong abstraction; two libraries / versions for one job; SQL / config / i18n copies; clone growth outpacing consolidation |
|
|
160
140
|
| `LOW` | Cosmetic copies; test-only clones; stale register note changing no status |
|
|
161
141
|
|
|
162
|
-
|
|
142
|
+
Within a severity: higher blast radius x divergence x user-visibility first.
|
|
163
143
|
|
|
164
144
|
<!-- prettier-ignore -->
|
|
165
145
|
| ID | Severity | Verdict | Location | Evidence | Expected |
|
|
@@ -247,17 +227,13 @@ PENDING
|
|
|
247
227
|
|
|
248
228
|
### 🔒 Gate 3: Report Check
|
|
249
229
|
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
- [ ] **No placeholders:** Findings table and every `Report` section have no `PENDING`.
|
|
230
|
+
- [ ] **No placeholders:** no `PENDING` left
|
|
253
231
|
- [ ] **Summary:** names count per severity, count per verdict, and the guard-or-sweep call.
|
|
254
232
|
- [ ] **Series:** per-package duplication % and clone growth beside previous report's numbers, or marked baseline.
|
|
255
|
-
- [ ] **Writes:** only this report
|
|
256
|
-
- [ ] **Process Improvements:** `Universal`
|
|
233
|
+
- [ ] **Writes:** only this report, template `Project Instructions`, and ledger rows changed; throwaway worktrees / remote targets removed.
|
|
234
|
+
- [ ] **Process Improvements:** `Universal` and `Project` filled
|
|
257
235
|
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID in exactly one row; every row has a `Next Step`.
|
|
258
|
-
- [ ] **Gate 3
|
|
259
|
-
|
|
260
|
-
> `Project Instructions` below `bkit:end` are consumer-owned — `bkit install` never overwrites them, `bkit check` drift stays clean. Add project-specific tools, paths, commands, and checks there, keyed by section or step title; never results.
|
|
236
|
+
- [ ] **Gate 3 — Verified:** Every checkbox above independently verified and checked off.
|
|
261
237
|
|
|
262
238
|
<!-- bkit:end -->
|
|
263
239
|
|
|
@@ -11,14 +11,6 @@
|
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> - Read-only audit; never edit code, config, or data.
|
|
15
|
-
> - `Universal Instructions` apply to all projects; `Project Instructions` apply only to this project.
|
|
16
|
-
> - Local audit instructions / overrides → `Project Instructions` below the bkit-managed block; universal instructions stay in-block.
|
|
17
|
-
> - Any tools mentioned in `Universal Instructions` are examples only; swap for actual tools in final output; drop inapplicable.
|
|
18
|
-
> - Use dense token-efficient phrasing; no filler / hedging / restating; tables > prose, bullets > paragraphs.
|
|
19
|
-
> - Read `.agents/docs.md` → load docs for workflow / step / task; skip missing; cite docs inline where required.
|
|
20
|
-
> - Every claim cites this run's evidence (`file:line`, or command / query and its output) and tool behind it; a previous report's figure, list, or status is a lead, not evidence.
|
|
21
|
-
> - Read this document once at session start; do not trust session memory; do not re-read on every loop iteration.
|
|
22
14
|
> - NEVER append or edit a ledger line. Close, regression, triage, and knowledge rows live in `Expected` only. `ts` is minted at append (`.bkit/jsonl.md` § Write Idiom), never written in this report.
|
|
23
15
|
> - A security row (`category: "security"`, or an auth, secrets, or data-exposure hole) carries key, verdict, and severity only. `file:line` stays in the ledger row. A public repo uses a draft repository security advisory, not this report.
|
|
24
16
|
|
|
@@ -39,7 +31,7 @@ Row text that says found by reading and must be confirmed later = hypothesis, no
|
|
|
39
31
|
|
|
40
32
|
**Cadence:** once per ship cycle, and at least monthly when no cycle ships. Extra pass when new `CRITICAL` row lands (age target 7 days), or after incident on production or preview.
|
|
41
33
|
|
|
42
|
-
**Anchor Standards:**
|
|
34
|
+
**Anchor Standards:**
|
|
43
35
|
|
|
44
36
|
- Problem management and the known-error database (e.g. ITIL 4; record the edition). The deferred and known-error review it describes (e.g. ITIL-aligned known-error monitoring).
|
|
45
37
|
- Postmortem action items (e.g. Google SRE Workbook, Postmortem Culture).
|
|
@@ -86,23 +78,15 @@ Row text awaiting confirmation = UNCONFIRMED until reproduced or read end to end
|
|
|
86
78
|
|
|
87
79
|
### Pre-Flight / Dependencies
|
|
88
80
|
|
|
89
|
-
**Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
|
|
90
|
-
|
|
91
|
-
**Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
|
|
92
|
-
|
|
93
81
|
#### Core
|
|
94
82
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
- [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied (here or in any step) → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`; run every step the remaining tools allow; work needing blocked call → `Not Checked`. never stop the run for a blocked call; never retry or reword a blocked call to get past it.
|
|
98
|
-
- [ ] **Ledger:** Steps 3–7 — `docs/failures.jsonl` and `.bkit/jsonl.md`. Fallback: file absent → one `N/A` row after Step 3 and STOP.
|
|
83
|
+
- [ ] **Permissions:** blocked calls recorded
|
|
84
|
+
- [ ] **Ledger:** Steps 3-7 — `docs/failures.jsonl` and `.bkit/jsonl.md`. Fallback: file absent → one `N/A` row after Step 3 and STOP.
|
|
99
85
|
- [ ] **Git History:** Steps 2, 5 — commits since the previous report, fix commits. Fallback: closures and regression-by-commit `Not Checked`.
|
|
100
86
|
- [ ] **Web:** Step 8 — fetch anchor standards. Fallback: versions recorded as unread.
|
|
101
87
|
|
|
102
88
|
#### Conditional
|
|
103
89
|
|
|
104
|
-
Verify only when applicable.
|
|
105
|
-
|
|
106
90
|
- [ ] **Language Server:** Typed language → Step 4 — symbol at current path, caller chain, diagnostics for one file without full typecheck, e.g. Serena, `tsserver`. Fallback: text search. Reach and close needing callers stay hypotheses until corroborated.
|
|
107
91
|
- [ ] **Error Tracker:** Production or preview events exist → Steps 4, 5 — event counts and flows after fix commit, e.g. Sentry, platform logs. Log access: `.agents/context.md` `context-fetch` blocks below `bkit:end` are ready-made read-only readers (`bkit context list-blocks` → each block's `context` and `body`); set a body's `# Customise:` knobs to the audit window; run it via `direnv exec . bash`. Fallback: code only. Event counts `Not Checked`.
|
|
108
92
|
- [ ] **Tracker:** Issues wired → Steps 4, 7 — issue carrying bug, postmortem tracking issue. Fallback: issue pointers `Not Checked`.
|
|
@@ -110,14 +94,12 @@ Verify only when applicable.
|
|
|
110
94
|
|
|
111
95
|
### 🔒 Gate 1: Prerequisites Check
|
|
112
96
|
|
|
113
|
-
**Checkpoint.** Verify and check `[x]` all applicable boxes above, then continue to `Probe Steps` without asking.
|
|
114
|
-
|
|
115
97
|
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
116
|
-
- [ ] **Gate 1
|
|
98
|
+
- [ ] **Gate 1 — Verified:** All Core items checked. All applicable Conditional items checked.
|
|
117
99
|
|
|
118
100
|
### Probe Steps
|
|
119
101
|
|
|
120
|
-
- [ ] **Step 1 — Previous Run:**
|
|
102
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** closure rate in `Previous Run`, or run marked baseline.
|
|
121
103
|
- [ ] **Step 2 — Closures:** Re-verify every FIXED and MIS-FRAMED row from previous report at `HEAD`. Each no longer holding = finding, with what changed. **Done when:** `Closures` holds every row, each failure, and failure rate, or run marked baseline.
|
|
122
104
|
- [ ] **Step 3 — Fold:** Last-row-wins per `key`, not per line. Append-only file (`.gitattributes` `merge=union`); editing a line keeps stale copy beside replacement. Highest numeric `ts` wins (`.bkit/jsonl.md` § Updating a Row). `key` whose `ts` is not a number → last line in file is latest. Open = every winning row with `status != "CLOSED"`, including `TRIAGED` (diagnosed is not fixed), `RESOLVED`, `FIXED`, `SUPERSEDED`, and rows with no `status`. Word that looks finished is not done. `RESOLVED` or `FIXED` row really done → `CLOSED` row in `Expected`, with `references` that closed it. Not skipped because word looks finished. Order: `CRITICAL`, then `HIGH`, then oldest `ts`. Compare `severity` case-insensitively. No `severity` → assign with verdict. Age from row's first `ts`: list every `CRITICAL` older than 7 days and every `HIGH` older than 30 days. Where CI check goes red while any `CRITICAL` row has `status != "CLOSED"`, report whether installed. Green check beside such row = finding (gate not reading ledger). Absent → record in `Checked`. Large open set may be partial pass → record rows verified / rows open. Winning future `ts` that would outrank close → name `key`. No later `ts` invented. **Done when:** `Trend` holds open counts, age-target breaches, gate result, and verified / open, or `N/A` and STOP.
|
|
123
105
|
- [ ] **Step 4 — Source:** No verdict before this step. Under-reporting biases toward closing live bug.
|
|
@@ -136,10 +118,10 @@ Pace reference queries. Re-run empty or surprisingly low result before close ver
|
|
|
136
118
|
|
|
137
119
|
Incident rows (seen on production or preview, not in review) need follow-ups checked. Remediation usually names more than code fix: regression test, monitor, alert, doc change. Check each landed. Fix landed, test or monitor did not → not FIXED. `Expected` names each missing follow-up and issue to open.
|
|
138
120
|
|
|
139
|
-
Each incident: written postmortem;
|
|
121
|
+
Each incident: written postmortem; >=1 action preventing or detecting repeat, with owner and tracking issue at top priority. User-affecting outage needs >=1 top-priority bug. Missing postmortem, or postmortem with only mitigation items = finding. `Expected` = issue to open. **Done when:** every row verified this pass has evidence and tool on finding. `Incidents` lists follow-ups and postmortem gaps, or `none`. Unverified rows stay counted in `Trend`; not passed.
|
|
140
122
|
|
|
141
123
|
- [ ] **Step 5 — Recurrences:** Same-`key` append recording defect seen again = recurrence (`.bkit/jsonl.md` § Updating a Row). Omit appends that only change `status`. For `category: "testing"`, flake count = `key`'s row count under that rule. Rank open rows by count. Then regressions: each row `CLOSED` since previous report → look for later row same `key`, new row same defect under another `key`, or error-tracker events on matching issue after close. Each → `Expected` = `OPEN` row to append. **Done when:** `Recurrences` ranks open rows and lists regressions, or `none`.
|
|
142
|
-
- [ ] **Step 6 — Classes:** (a) Known errors carry workaround. For every `TRIAGED` or CONFIRMED row agents keep hitting (
|
|
124
|
+
- [ ] **Step 6 — Classes:** (a) Known errors carry workaround. For every `TRIAGED` or CONFIRMED row agents keep hitting (>=2 recurrences in Step 5, or parked row whose blocker not in winning row), check `docs/knowledge.jsonl` has `ACTIVE` row with workaround. Missing → finding. `Expected` = knowledge row to append. (b) Count open and recently closed rows by `category`, package, root-cause class. Top recurring class this run = gate candidate → name for `bkit-audit-gates`. **Done when:** `Classes` holds workaround gaps, counts, gate candidate, or `none`.
|
|
143
125
|
- [ ] **Step 7 — Classify:** Every row, as Step 4 left it, gets one verdict. This run does not append or open issues. Security row → `bkit-audit-security`: key, verdict, severity only in this report. Flaky test diagnosis → `bkit-audit-flakes`. Older flake diagnoses may sit on tracker issues predating that audit. Search those too.
|
|
144
126
|
|
|
145
127
|
<!-- prettier-ignore -->
|
|
@@ -167,13 +149,11 @@ Each incident: written postmortem; ≥1 action preventing or detecting repeat, w
|
|
|
167
149
|
|
|
168
150
|
### 🔒 Gate 2: Probes Check
|
|
169
151
|
|
|
170
|
-
**Checkpoint.** Verify and check `[x]` all boxes below, then continue to `Findings` without asking.
|
|
171
|
-
|
|
172
152
|
- [ ] **Steps complete:** every step `[x]`, or explicit skip with evidence the surface is absent.
|
|
173
153
|
- [ ] **Evidence:** every finding cites re-checkable evidence and the tool behind it.
|
|
174
154
|
- [ ] **Read-only:** no ledger line, code, or config changed. Every close, regression, and knowledge row is `Expected` only.
|
|
175
155
|
- [ ] **Security:** security rows carry key, verdict, and severity only. No exploit detail, no `file:line`.
|
|
176
|
-
- [ ] **Gate 2
|
|
156
|
+
- [ ] **Gate 2 — Verified:** Every checkbox above independently verified and checked off.
|
|
177
157
|
|
|
178
158
|
## Findings
|
|
179
159
|
|
|
@@ -185,7 +165,7 @@ Each incident: written postmortem; ≥1 action preventing or detecting repeat, w
|
|
|
185
165
|
| `MEDIUM` | Mis-framed or understated. Recurrence or regression. Known error with no workaround on record. Age target breached. Hypothesis still open |
|
|
186
166
|
| `LOW` | Duplicate key. Path drift that does not change reach. Missing `severity` |
|
|
187
167
|
|
|
188
|
-
|
|
168
|
+
Within a severity: oldest `ts` first. Finding keeps severity when it becomes ledger row or issue. Row's bucket = ledger severity, or verdict assigns.
|
|
189
169
|
|
|
190
170
|
<!-- prettier-ignore -->
|
|
191
171
|
| ID | Severity | Verdict | Location | Evidence | Expected |
|
|
@@ -253,17 +233,13 @@ PENDING
|
|
|
253
233
|
|
|
254
234
|
### 🔒 Gate 3: Report Check
|
|
255
235
|
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
- [ ] **No placeholders:** Findings table and every `Report` section have no `PENDING`.
|
|
236
|
+
- [ ] **No placeholders:** no `PENDING` left
|
|
259
237
|
- [ ] **Summary:** names count per severity, age-target breach count, and rows verified / rows open.
|
|
260
238
|
- [ ] **Series:** open count and the recurrence ranking beside previous report's numbers, or marked baseline.
|
|
261
|
-
- [ ] **Writes:** only this report
|
|
262
|
-
- [ ] **Process Improvements:** `Universal`
|
|
239
|
+
- [ ] **Writes:** only this report, template `Project Instructions`, and ledger rows changed; throwaway worktrees / remote targets removed.
|
|
240
|
+
- [ ] **Process Improvements:** `Universal` and `Project` filled
|
|
263
241
|
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID in exactly one row; every row has a `Next Step`.
|
|
264
|
-
- [ ] **Gate 3
|
|
265
|
-
|
|
266
|
-
> `Project Instructions` below `bkit:end` are consumer-owned — `bkit install` never overwrites them, `bkit check` drift stays clean. Add project-specific tools, paths, commands, and checks there, keyed by section or step title; never results.
|
|
242
|
+
- [ ] **Gate 3 — Verified:** Every checkbox above independently verified and checked off.
|
|
267
243
|
|
|
268
244
|
<!-- bkit:end -->
|
|
269
245
|
|
|
@@ -11,14 +11,6 @@
|
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> - Read-only audit; never edit code, config, or data.
|
|
15
|
-
> - `Universal Instructions` apply to all projects; `Project Instructions` apply only to this project.
|
|
16
|
-
> - Local audit instructions / overrides → `Project Instructions` below the bkit-managed block; universal instructions stay in-block.
|
|
17
|
-
> - Any tools mentioned in `Universal Instructions` are examples only; swap for actual tools in final output; drop inapplicable.
|
|
18
|
-
> - Use dense token-efficient phrasing; no filler / hedging / restating; tables > prose, bullets > paragraphs.
|
|
19
|
-
> - Read `.agents/docs.md` → load docs for workflow / step / task; skip missing; cite docs inline where required.
|
|
20
|
-
> - Every claim cites this run's evidence (`file:line`, or command / query and its output) and tool behind it; a previous report's figure, list, or status is a lead, not evidence.
|
|
21
|
-
> - Read this document once at session start; do not trust session memory; do not re-read on every loop iteration.
|
|
22
14
|
> - never `.skip`, quarantine, delete an assertion, or widen a timeout to green a lane. A timeout named before the trace is read is not a verdict.
|
|
23
15
|
> - Repro and repeat runs use the suite's test tenant only; NEVER production tenants, payments, or mail.
|
|
24
16
|
|
|
@@ -35,7 +27,7 @@
|
|
|
35
27
|
|
|
36
28
|
**Cadence:** biweekly; every workflow's runs since previous report; same window as `bkit-audit-logs` (both read same CI runs). Do not quarantine; flake stays until fixed → age measured, not remembered. Extra pass (one flake or red run): pass-only-on-retry or fail one surface/pass another on integration branch; after retries/workers/timeouts/trace/shared-fixture change; after fix lands → repeat rate measurement for run count `Fix` named.
|
|
37
29
|
|
|
38
|
-
**Anchor Standards:**
|
|
30
|
+
**Anchor Standards:**
|
|
39
31
|
|
|
40
32
|
- Flake root-cause taxonomy (Luo et al., "An Empirical Analysis of Flaky Tests", FSE 2014).
|
|
41
33
|
- Google, "Flaky Tests at Google and How We Mitigate Them" (Google Testing Blog, 2016).
|
|
@@ -78,23 +70,15 @@ One per flake. Root-cause class (`Mechanism`) = evidence; these = outcomes. Quar
|
|
|
78
70
|
|
|
79
71
|
### Pre-Flight / Dependencies
|
|
80
72
|
|
|
81
|
-
**Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
|
|
82
|
-
|
|
83
|
-
**Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
|
|
84
|
-
|
|
85
73
|
#### Core
|
|
86
74
|
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
- [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied (here or in any step) → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`; run every step the remaining tools allow; work needing blocked call → `Not Checked`. never stop the run for a blocked call; never retry or reword a blocked call to get past it.
|
|
75
|
+
- [ ] **Permissions:** blocked calls recorded
|
|
90
76
|
- [ ] **CI History:** Steps 2, 3 — workflow runs since previous report, e.g. `gh run list`, Actions API, GitLab pipelines. Log access: `.agents/context.md` `context-fetch` blocks below `bkit:end` are ready-made read-only readers (`bkit context list-blocks` → each block's `context` and `body`); set a body's `# Customise:` knobs to the audit window; run it via `direnv exec . bash`. Fallback: local repeats only; ranked CI list `Not Checked`.
|
|
91
77
|
- [ ] **Repeat Runner:** Steps 4, 7, 8 — repeat with retries off, e.g. Playwright `--repeat-each`, pytest-repeat. Fallback: rate `Not Checked`; do not mark `FIX NAMED` on unmeasured flake unless mechanism deterministic and report says rate unknown.
|
|
92
78
|
- [ ] **Web:** Steps 6, 13 — vendor status pages and anchor standards. Fallback: incidents and versions recorded as unread.
|
|
93
79
|
|
|
94
80
|
#### Conditional
|
|
95
81
|
|
|
96
|
-
Verify only when applicable.
|
|
97
|
-
|
|
98
82
|
- [ ] **Traces:** Browser E2E → Step 5 — trace, video, screenshot of failed attempt, e.g. Playwright trace, Cypress. Fallback: log only; geometry claims `Not Checked`.
|
|
99
83
|
- [ ] **Language Server:** Typed language → Steps 6, 11 — which specs share fixture, e.g. Serena, `tsserver`. Fallback: text search; name it. Guessing sharing set misattributes diagnosis.
|
|
100
84
|
- [ ] **Vendor Status:** Failure time may overlap vendor window → Step 6 — e.g. GitHub status, hosting preview, database host. Fallback: `INFRASTRUCTURE` `Not Checked`.
|
|
@@ -102,14 +86,12 @@ Verify only when applicable.
|
|
|
102
86
|
|
|
103
87
|
### 🔒 Gate 1: Prerequisites Check
|
|
104
88
|
|
|
105
|
-
**Checkpoint.** Verify and check `[x]` all applicable boxes above, then continue to `Probe Steps` without asking.
|
|
106
|
-
|
|
107
89
|
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
108
|
-
- [ ] **Gate 1
|
|
90
|
+
- [ ] **Gate 1 — Verified:** All Core items checked. All applicable Conditional items checked.
|
|
109
91
|
|
|
110
92
|
### Probe Steps
|
|
111
93
|
|
|
112
|
-
- [ ] **Step 1 — Previous Run:**
|
|
94
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** closure rate in `Previous Run`, or run marked baseline.
|
|
113
95
|
- [ ] **Step 2 — List:** Before triage, list every test with runner status flaky (failed, passed on retry) and every pass-only-on-retry, across every workflow since previous report. Also list fail one run/surface, pass another: with retries off, `flaky` never appears; a noticed red run is input, not the list. Rank by flake rate (flaky or failed runs ÷ runs) and reruns caused. Per open flake: first-seen, last-seen, age. Report median age of open flakes. Red run absent from list → detection finding. Triage in rank order. **Done when:** ranked list, ages, median in `Flakes`.
|
|
114
96
|
- [ ] **Step 3 — Pin:** Per triaged flake: run id, commit, job that failed vs downstream-only. Control run and valid scope. Prior red run = control only when failure mode differs — auth failure redirecting every lane to sign-in is control for tenant-minting lane only. **Done when:** run, job, control in `Pin`.
|
|
115
97
|
- [ ] **Step 4 — Rate:** Spec with repeat-each, retries off, at CI worker count: k failures in N runs. Repeat with CPU throttled/loaded. Record N, k, workers, commit. Without measured rate, nothing to compare fix against (Step 11). **Done when:** N, k, workers, commit in `Rate`, or why not measured.
|
|
@@ -117,8 +99,8 @@ Verify only when applicable.
|
|
|
117
99
|
- [ ] **Step 6 — Surfaces:** CI vs local = different apps (e.g. production build on preview deploy vs dev server reusing process). Fail one, not other → hydration timing, build output, stale dev server before test blame. Record CI and local `TZ`. Rule out infrastructure: failure time vs host status, preview deploy/cold-start (if used), database/vendor status. Infra-explained flake → class `infrastructure` in Step 10, verdict `INFRASTRUCTURE`, not charged to test. Same split inside CI. Worker-scoped fixture shared by many specs: mutating tenant fails whichever spec runs next. Sharing set from language server, not memory: which specs reference fixture/helper; moved helper location; type errors without full typecheck. Pace queries — back-to-back reference queries → partial results; re-run surprisingly small sets. No language server for mock-factory key. Non-typed fixture wiring (runner deps, CI YAML, scripts) → text search. Stale reference counts from before tests joined project graph. **Done when:** surface split, infra check, sharing set in `Surfaces`.
|
|
118
100
|
- [ ] **Step 7 — Time:** Date-touching flake → local repro with clock pinned (e.g. Playwright `page.clock.setFixedTime`, `vi.setSystemTime`): failed date, month end, DST change. `TZ` local then CI; set `TZ` before workers spawn. **Done when:** results in `Time`, or skip with evidence flake does not touch dates.
|
|
119
101
|
- [ ] **Step 8 — Order:** Unit runner → shuffle with recorded seed (e.g. Vitest `--sequence.shuffle --sequence.seed=<n>`). Browser runner → different worker/shard count. Record seed and counts. Tests cross-spec contamination (e.g. Step 6 shared fixture) by running, not only reference graph. **Done when:** seed, counts, result in `Order`.
|
|
120
|
-
- [ ] **Step 9 — Native:** Device-runner flake (e.g. Maestro, EAS) → Steps 5
|
|
121
|
-
- [ ] **Step 10 — Mechanism:** Teardown deletes rows event points at → most diagnoses corroborated by mechanism, not row. Distinguish "fits" vs "proven". One root-cause class: async wait, concurrency, order dependency, shared fixture state, resource leak, network, external service, time and date, randomness, unordered collection, infrastructure, or other (describe). Luo et al.: async wait, concurrency, order dependency
|
|
102
|
+
- [ ] **Step 9 — Native:** Device-runner flake (e.g. Maestro, EAS) → Steps 5-6 target browser; also boot finished before flow, device-cloud retries, animation/keyboard timing. Record artifact locations (screenshots, recordings, logs). **Done when:** answers in `Native`, or skip with evidence.
|
|
103
|
+
- [ ] **Step 10 — Mechanism:** Teardown deletes rows event points at → most diagnoses corroborated by mechanism, not row. Distinguish "fits" vs "proven". One root-cause class: async wait, concurrency, order dependency, shared fixture state, resource leak, network, external service, time and date, randomness, unordered collection, infrastructure, or other (describe). Luo et al.: async wait, concurrency, order dependency ~ 78% classified. Per-class counts across open flakes = series. Before timing mechanism → floating-promise/missing-await rules on spec/helpers (e.g. `@typescript-eslint/no-floating-promises`, `playwright/missing-playwright-await`); record hits so missing `await` is not hidden behind timing. Real defect behind flake → verdict `BUG`. **Done when:** class, what is not established, await hits in `Mechanism`.
|
|
122
104
|
- [ ] **Step 11 — Fix:** Name cause and change that removes it; making change = next step. Never `.skip`, quarantine, delete assertion, widen timeout. Before trusting the repaired test, the next step puts the bug back, watches the test go red, restores, and confirms `git diff` on the source file is empty. That is not optional on anything touching row-level access control or a cross-tenant boundary. Shared helper/fixture change = signature change: references first. One-spec fix via shared fixture → next unattributable failure. Put bug back proves deterministic fix. Flaky fix also repeats Step 4: at rate p, ~3/p runs with no failure → 95% confidence rate below p (rule of three); report states count. One verdict from `Verdicts`. **Done when:** verdict, change, or why none, run count in `Fix`.
|
|
123
105
|
- [ ] **Step 12 — Route:** No finding without re-checkable evidence: `file:line`, or query/run id/measurement + tool. Language-server empty inconclusive → text search corroboration. Report names destinations (sending = next step); the audit agent writes `docs/failures.jsonl` rows. Every flake → `docs/failures.jsonl` row (class, rate, age). Named mechanism → recommended fix + proof runs (Step 11). Real defect → own issue + ledger row. Fix waits on decision → named decision. Mechanism error tracker also saw → `bkit-audit-logs`. Retry-pass/warning CI annotated → `bkit-audit-annotations`. Missing trace/capture, third-party stub, missing `await` → recommended change (Steps 5, 10). **Done when:** every finding's Expected names destination, including ledger row.
|
|
124
106
|
- [ ] **Step 13 — Standards:** Confirm current version per anchor standard; read changes since previous report's version. `Standards Coverage`: row per Luo et al. class + per flake feature the runner offers (e.g. retries, flaky status, repeat-each, fail-on-flaky, sequence shuffle) → checked (name step) / not applicable (why) / finding. Per bkit anchor (doc section, rule, or issue shape): one row → followed (name step or evidence) / not applicable (why) / finding; departure with no recorded reason (e.g. `docs/decisions.jsonl` row, accepted-divergence entry) = finding. Unapplied recommended practice → adopt (propose step) / reject (why) / not applicable. Do not change steps during run. **Done when:** no empty row.
|
|
@@ -126,13 +108,11 @@ Verify only when applicable.
|
|
|
126
108
|
|
|
127
109
|
### 🔒 Gate 2: Probes Check
|
|
128
110
|
|
|
129
|
-
**Checkpoint.** Verify and check `[x]` all boxes below, then continue to `Findings` without asking.
|
|
130
|
-
|
|
131
111
|
- [ ] **Steps complete:** every step `[x]`, or explicit skip with evidence the surface is absent.
|
|
132
112
|
- [ ] **Evidence:** every finding cites re-checkable evidence and the tool behind it.
|
|
133
113
|
- [ ] **Verdicts:** every triaged flake has one verdict from `Verdicts`; none is skip, quarantine, or a widened timeout.
|
|
134
114
|
- [ ] **Rates:** every triaged flake records N and k, or why the rate was not measured.
|
|
135
|
-
- [ ] **Gate 2
|
|
115
|
+
- [ ] **Gate 2 — Verified:** Every checkbox above independently verified and checked off.
|
|
136
116
|
|
|
137
117
|
## Findings
|
|
138
118
|
|
|
@@ -144,7 +124,7 @@ Verify only when applicable.
|
|
|
144
124
|
| `MEDIUM` | Named intermittent failure; missing trace of the failed attempt; shared fixture that can attribute a failure to the next spec; repeated rerun cost |
|
|
145
125
|
| `LOW` | Vendor incident not charged to the test; speculative timing with no measured rate |
|
|
146
126
|
|
|
147
|
-
|
|
127
|
+
Within a severity: real defects first.
|
|
148
128
|
|
|
149
129
|
<!-- prettier-ignore -->
|
|
150
130
|
| ID | Severity | Verdict | Location | Evidence | Expected |
|
|
@@ -232,17 +212,13 @@ PENDING
|
|
|
232
212
|
|
|
233
213
|
### 🔒 Gate 3: Report Check
|
|
234
214
|
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
- [ ] **No placeholders:** Findings table and every `Report` section have no `PENDING`.
|
|
215
|
+
- [ ] **No placeholders:** no `PENDING` left
|
|
238
216
|
- [ ] **Summary:** names count per severity and per verdict, and the median age of open flakes.
|
|
239
217
|
- [ ] **Series:** each open flake's rate and the median age beside previous report's numbers, or marked baseline.
|
|
240
|
-
- [ ] **Writes:** only this report
|
|
241
|
-
- [ ] **Process Improvements:** `Universal`
|
|
218
|
+
- [ ] **Writes:** only this report, template `Project Instructions`, and ledger rows changed; throwaway worktrees / remote targets removed.
|
|
219
|
+
- [ ] **Process Improvements:** `Universal` and `Project` filled
|
|
242
220
|
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID in exactly one row; every row has a `Next Step`.
|
|
243
|
-
- [ ] **Gate 3
|
|
244
|
-
|
|
245
|
-
> `Project Instructions` below `bkit:end` are consumer-owned — `bkit install` never overwrites them, `bkit check` drift stays clean. Add project-specific tools, paths, commands, and checks there, keyed by section or step title; never results.
|
|
221
|
+
- [ ] **Gate 3 — Verified:** Every checkbox above independently verified and checked off.
|
|
246
222
|
|
|
247
223
|
<!-- bkit:end -->
|
|
248
224
|
|
|
@@ -11,14 +11,6 @@
|
|
|
11
11
|
>
|
|
12
12
|
> **Rules:**
|
|
13
13
|
>
|
|
14
|
-
> - Read-only audit; never edit code, config, or data.
|
|
15
|
-
> - `Universal Instructions` apply to all projects; `Project Instructions` apply only to this project.
|
|
16
|
-
> - Local audit instructions / overrides → `Project Instructions` below the bkit-managed block; universal instructions stay in-block.
|
|
17
|
-
> - Any tools mentioned in `Universal Instructions` are examples only; swap for actual tools in final output; drop inapplicable.
|
|
18
|
-
> - Use dense token-efficient phrasing; no filler / hedging / restating; tables > prose, bullets > paragraphs.
|
|
19
|
-
> - Read `.agents/docs.md` → load docs for workflow / step / task; skip missing; cite docs inline where required.
|
|
20
|
-
> - Every claim cites this run's evidence (`file:line`, or command / query and its output) and tool behind it; a previous report's figure, list, or status is a lead, not evidence.
|
|
21
|
-
> - Read this document once at session start; do not trust session memory; do not re-read on every loop iteration.
|
|
22
14
|
> - never build, wire, or land a gate; never edit workflows, hooks, allowlists, or gate scripts.
|
|
23
15
|
> - NEVER weaken an existing gate — no skip, rule downgrade, lowered floor, or broader ignore — so a candidate looks adoptable.
|
|
24
16
|
|
|
@@ -32,7 +24,7 @@ Maintenance sweeps find breakage; none prevent the next. Sibling audits repeat
|
|
|
32
24
|
|
|
33
25
|
**Cadence:** quarterly. Extra pass when a sibling audit's `Process Improvements` proposes a new gate, or `bkit-audit-failures` names a recurring root-cause class as a gate candidate.
|
|
34
26
|
|
|
35
|
-
**Anchor Standards:**
|
|
27
|
+
**Anchor Standards:**
|
|
36
28
|
|
|
37
29
|
- Fitness functions (e.g. Ford, Parsons, Kua, and Sadalage, _Building Evolutionary Architectures_, 2nd ed.).
|
|
38
30
|
- Static-analysis review bar (e.g. Tricorder: under 10% effective false positives, and an actionable message; Sadowski et al., Communications of the ACM, 2018).
|
|
@@ -42,7 +34,7 @@ Maintenance sweeps find breakage; none prevent the next. Sibling audits repeat
|
|
|
42
34
|
- Required status checks and a merge queue (e.g. GitHub rulesets).
|
|
43
35
|
- OpenSSF Scorecard checks, at the Scorecard version read.
|
|
44
36
|
- DORA continuous integration capability.
|
|
45
|
-
- bkit block of `docs/testing.md` (bkit version from `.bkit/version`): § CI test tiers (the bkit scaffold); § Accepted divergences; § Step names say whether a red is real; § Composite actions: two rules that fail silently
|
|
37
|
+
- bkit block of `docs/testing.md` (bkit version from `.bkit/version`): § CI test tiers (the bkit scaffold); § Accepted divergences; § Step names say whether a red is real; § Composite actions: two rules that fail silently; § JUnit reports for bkit
|
|
46
38
|
- bkit block of `docs/deployments.md` (bkit version from `.bkit/version`): § Commit-message force tags; § Workflow-file limits GitHub does not enforce visibly; § Ship gate + bookkeeping
|
|
47
39
|
- bkit block of `docs/design.md` (bkit version from `.bkit/version`): § Machine-checked reference
|
|
48
40
|
|
|
@@ -72,15 +64,9 @@ One per candidate.
|
|
|
72
64
|
|
|
73
65
|
### Pre-Flight / Dependencies
|
|
74
66
|
|
|
75
|
-
**Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
|
|
76
|
-
|
|
77
|
-
**Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
|
|
78
|
-
|
|
79
67
|
#### Core
|
|
80
68
|
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
- [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied (here or in any step) → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`; run every step the remaining tools allow; work needing blocked call → `Not Checked`. never stop the run for a blocked call; never retry or reword a blocked call to get past it.
|
|
69
|
+
- [ ] **Permissions:** blocked calls recorded
|
|
84
70
|
- [ ] **Sibling Reports:** Steps 2, 3 — newest report per other `bkit-audit-*` slug under `docs/audits/`, else that template's `Process Improvements`. Fallback: templates only; a missing report = that audit `Not Checked`.
|
|
85
71
|
- [ ] **Ledgers:** Steps 3, 10, 11 — `docs/failures.jsonl`, `docs/decisions.jsonl`, `docs/deferrals.jsonl`. Fallback: incident evidence and prior rejections `Not Checked`.
|
|
86
72
|
- [ ] **Repo Gates:** Steps 7, 9, 12, 13, 15 — workflows, hook directory, gate scripts, allowlist files, e.g. `.github/workflows`, `.husky`, `.gitlab-ci.yml`, every `bkit:gate:*` script in `package.json`. Fallback: none; those items `Not Checked`.
|
|
@@ -88,24 +74,20 @@ Always verify.
|
|
|
88
74
|
|
|
89
75
|
#### Conditional
|
|
90
76
|
|
|
91
|
-
Verify only when applicable.
|
|
92
|
-
|
|
93
77
|
- [ ] **Ruleset Read:** Host exposes branch rules → Step 7 — required checks and bypass actors, e.g. `gh api` rulesets, GitLab protected branches. Fallback: workflow and hook files only; bypass actors `Not Checked`.
|
|
94
78
|
- [ ] **Run History:** A gate has already landed → Step 12 — how often it fired, e.g. `gh run list`, CI analytics. Fallback: code reading only; fire rate `Not Checked`.
|
|
95
79
|
- [ ] **Scorecard:** Repo is public, or a token is present → Steps 3, 14 — e.g. `scorecard`. Fallback: read the check list; live score `Not Checked`.
|
|
96
80
|
|
|
97
81
|
### 🔒 Gate 1: Prerequisites Check
|
|
98
82
|
|
|
99
|
-
**Checkpoint.** Verify and check `[x]` all applicable boxes above, then continue to `Probe Steps` without asking.
|
|
100
|
-
|
|
101
83
|
- [ ] **Gaps named:** every missing item names its fallback and `Not Checked` entry.
|
|
102
|
-
- [ ] **Gate 1
|
|
84
|
+
- [ ] **Gate 1 — Verified:** All Core items checked. All applicable Conditional items checked.
|
|
103
85
|
|
|
104
86
|
### Probe Steps
|
|
105
87
|
|
|
106
|
-
- [ ] **Step 1 — Previous Run:**
|
|
88
|
+
- [ ] **Step 1 — Previous Run:** **Done when:** closure rate in `Previous Run`, or run marked baseline.
|
|
107
89
|
- [ ] **Step 2 — Inventory:** Collect gate candidates from each other `bkit-audit-*` report's `Process Improvements` (else that template's), without re-deriving. Record source slug. Two audits proposing one guard = one candidate, both sources — not two work items. Re-grep of a swept class + owner-allowlist over same literals = one guard. **Done when:** `Inventory` row per candidate: source slug · one-line guard.
|
|
108
|
-
- [ ] **Step 3 — Widen:** Sibling sections are one source. Also add, record source each: (a) every defect class in `docs/failures.jsonl` on
|
|
90
|
+
- [ ] **Step 3 — Widen:** Sibling sections are one source. Also add, record source each: (a) every defect class in `docs/failures.jsonl` on >=2 rows, whether or not a sibling proposed a gate; (b) standard gates nobody proposed, each vs tree — secret-scanning push protection, code scanning (e.g. CodeQL), dependency-review check, workflow linters (e.g. `actionlint`, `zizmor`), lockfile lint, test-runner forbid-only flag (e.g. Playwright `--forbid-only`), typechecking of test files, OpenSSF Scorecard checks; (c) policy-as-code for config with no type system (e.g. JSON Schema, Conftest) — bot config, job-runner triggers and concurrency, hosting config, mobile build config, workflow YAML. Class (c) is what `bkit-audit-testing` flags. Already enforced → `Health`, not a second candidate. **Done when:** every added row in `Inventory` with source.
|
|
109
91
|
- [ ] **Step 4 — Score:** Per candidate, six axes. Fails if fail condition holds.
|
|
110
92
|
- **Determinism.** Can a machine decide pass/fail with no judgement? Fails if the verdict needs someone to read intent.
|
|
111
93
|
- **False-positive class.** Empty, a fixed allowlist, or structural? Fails if structural and unencoded.
|
|
@@ -114,7 +96,7 @@ Verify only when applicable.
|
|
|
114
96
|
- **Cost.** Runtime per change + developer wait and upkeep (baseline maintenance, version bumps). Fails if slower than the value it returns.
|
|
115
97
|
- **Actionability.** Does failure say what to change and where (`file:line` plus fix)? Fails if developer must investigate. Deterministic but cryptic gate is bypassed as fast as a noisy one.
|
|
116
98
|
|
|
117
|
-
Rank by incident evidence
|
|
99
|
+
Rank by incident evidence x determinism ÷ effort. Prefer candidate with real incident over merely tidy. **Done when:** `Scores` holds six answers, rank, and fail axes.
|
|
118
100
|
|
|
119
101
|
- [ ] **Step 5 — Disqualify:** Candidate stays audit if any hold:
|
|
120
102
|
1. **Judgement is the task.** Examples: log line that is bug or handled outcome; deferral that may still be live (`bkit-audit-deferrals`); duplication or two deliberately different decisions (`bkit-audit-duplication`). No gate decides these.
|
|
@@ -135,7 +117,7 @@ Verify only when applicable.
|
|
|
135
117
|
|
|
136
118
|
- [ ] **Step 7 — Enforcement:** Where integration branch accepts direct push, CI check on that branch reds after code landed: detective, not preventive. Per candidate: required status check on integration and release branch? who bypasses? is only preventive point a pre-push hook a verify-skip flag skips? Ranking says so when only preventive point can be bypassed. **Done when:** `Enforcement` row per candidate: branch · required? · bypass · preventive point.
|
|
137
119
|
- [ ] **Step 8 — Positive Control:** Gate that passes because it measures nothing is worse than no gate — it stops anyone looking. Control every new gate meets: put defect back, watch it red. This run does not build the gate; report states that control. Gate with no proven-red demonstration is not finished. Related cheap checks: unmatched browser-test locator can be passing assertion; row-level denial can be silent zero rows, indistinguishable from empty table (`bkit-audit-database`). Skip = hollow pass. Required job skipped by job-level condition reports success. Workflow skipped by path filter, branch filter, or skip-CI marker leaves required check pending. Per candidate: skip behaviour; how skip is kept from standing in for pass. If proven-red is convention for gate change, not merge requirement → propose in `Process Improvements`. **Done when:** `Controls` states positive control and skip behaviour per candidate.
|
|
138
|
-
- [ ] **Step 9 — Sequence:** Each candidate: report-only
|
|
120
|
+
- [ ] **Step 9 — Sequence:** Each candidate: report-only >=1 full cycle, then enforcing. This run does not land gate. Read current tree, fix or allowlist findings, flip enforcing. Name candidates that would red if enforced cold. Report-only ends when gate meets stated bar, not just when cycle passed. Per candidate, bar: minimum runs; effective false-positive rate under 10% (findings judged not real problem ÷ all findings); no unexplained reds; no flakes of its own; added p90 runtime under stated budget. Name one enforcement tier. **Clear-then-gate** default: fix or allowlist every existing finding, then enforce on all. **Ratchet** (new violations only) when clearing existing findings is >1 PR: gate fails only on findings not in committed baseline (guard baseline file, or tool's own, e.g. Betterer results file, reviewdog added-only mode). Baseline may only shrink; each entry names owner. Allowlist per-offender, keyed to identity — never rule downgrade or route exclusion. Every entry cites owner; no owner and no date = permanent exemption. Ratchet baseline same contract; never grows. **Done when:** `Sequence` names bar and tier per candidate.
|
|
139
121
|
- [ ] **Step 10 — Decision:** Any workflow change needs explicit human confirmation. Propose, do not edit. Rule forbidding weakening gate applies to new one: bar for adding is that what it catches will be fixed. Record ADOPT, REJECT, or OTHER, including `docs/decisions.jsonl` row to append for ADOPT and REJECT. Appending is next step. Rejection whose reason later stops holding must stay findable. **Done when:** every candidate in `Scores` has verdict and row text, or OTHER with what settles it; verdict counts (proposed, ADOPT, REJECT, OTHER, reverted) in `Scores`.
|
|
140
122
|
- [ ] **Step 11 — Re-Review:** Gate built and reverted is not settled. Re-read each rejection in `docs/decisions.jsonl` and prior reports; check whether reasoning still holds. Two decisions on one guard, neither retired → criterion cannot be judged = finding. Reasoning no longer holds → score again into `Scores`, with `docs/decisions.jsonl` row that would retire old decision. **Done when:** `Rejections` row per reverted or rejected gate: still holds?
|
|
141
123
|
- [ ] **Step 12 — Health:** Per landed gate, since previous report: fire rate, true-positive rate, overrides/bypasses, own flakes, runtime. Never fired = perfectly preventive or hollow — indistinguishable outside; fresh positive control (`Positive Control`) tells apart. Never fired or hollow → name that control as next step. Every `bkit:gate:*` script in `package.json` = landed gate; workflows, agents, pre-push run only these → expand each (follow every nested `pnpm <script>` to its body) → list each command. Hollow → `CRITICAL`: script missing, empty, or runs nothing that can fail (e.g. `echo`, `true`, `|| exit 0`); command exits 0 when its tool absent (e.g. `command -v … || echo skipping`) = that check hollow → name tool and install step. **Done when:** `Health` row per landed gate, or none landed; each `bkit:gate:*` row lists every command and which can skip.
|
|
@@ -145,13 +127,11 @@ Verify only when applicable.
|
|
|
145
127
|
|
|
146
128
|
### 🔒 Gate 2: Probes Check
|
|
147
129
|
|
|
148
|
-
**Checkpoint.** Verify and check `[x]` all boxes below, then continue to `Findings` without asking.
|
|
149
|
-
|
|
150
130
|
- [ ] **Steps complete:** every step `[x]`, or explicit skip with evidence the surface is absent.
|
|
151
131
|
- [ ] **Evidence:** every finding cites re-checkable evidence and the tool behind it.
|
|
152
132
|
- [ ] **Sources read:** every candidate cites the section it came from; a score off a sibling summary alone is not a score.
|
|
153
133
|
- [ ] **No gate landed:** no workflow, hook, allowlist, or gate script changed.
|
|
154
|
-
- [ ] **Gate 2
|
|
134
|
+
- [ ] **Gate 2 — Verified:** Every checkbox above independently verified and checked off.
|
|
155
135
|
|
|
156
136
|
## Findings
|
|
157
137
|
|
|
@@ -163,7 +143,7 @@ Verify only when applicable.
|
|
|
163
143
|
| `MEDIUM` | Repeat defect class with a home lane and no gate; report-only candidate past its bar with no enforce-or-stay decision; allowlist entry past its date or with no owner |
|
|
164
144
|
| `LOW` | Candidate with no incident; OTHER waiting on a tool or a decision; gate-message wording that changes no pass/fail |
|
|
165
145
|
|
|
166
|
-
|
|
146
|
+
A candidate takes the severity of the class its gate would catch. Within a severity: higher `Score` first.
|
|
167
147
|
|
|
168
148
|
<!-- prettier-ignore -->
|
|
169
149
|
| ID | Severity | Verdict | Location | Evidence | Expected |
|
|
@@ -247,17 +227,13 @@ PENDING
|
|
|
247
227
|
|
|
248
228
|
### 🔒 Gate 3: Report Check
|
|
249
229
|
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
- [ ] **No placeholders:** Findings table and every `Report` section have no `PENDING`.
|
|
230
|
+
- [ ] **No placeholders:** no `PENDING` left
|
|
253
231
|
- [ ] **Summary:** names count per severity and per verdict (ADOPT / REJECT / OTHER).
|
|
254
232
|
- [ ] **Series:** verdict counts, `Health`, and allowlist sizes beside previous report's numbers, or marked baseline.
|
|
255
|
-
- [ ] **Writes:** only this report
|
|
256
|
-
- [ ] **Process Improvements:** `Universal`
|
|
233
|
+
- [ ] **Writes:** only this report, template `Project Instructions`, and ledger rows changed; throwaway worktrees / remote targets removed.
|
|
234
|
+
- [ ] **Process Improvements:** `Universal` and `Project` filled
|
|
257
235
|
- [ ] **Suggested Issues:** every `MEDIUM`+ finding ID in exactly one row; every row has a `Next Step`.
|
|
258
|
-
- [ ] **Gate 3
|
|
259
|
-
|
|
260
|
-
> `Project Instructions` below `bkit:end` are consumer-owned — `bkit install` never overwrites them, `bkit check` drift stays clean. Add project-specific tools, paths, commands, and checks there, keyed by section or step title; never results.
|
|
236
|
+
- [ ] **Gate 3 — Verified:** Every checkbox above independently verified and checked off.
|
|
261
237
|
|
|
262
238
|
<!-- bkit:end -->
|
|
263
239
|
|