@cbartens/bkit 5.2.1 → 5.2.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (90) hide show
  1. package/.bkit/README.md +1 -1
  2. package/.bkit/core/.agents/context.md +2 -2
  3. package/.bkit/core/.agents/plugins/bkit/hooks/bkit-guardrails.cjs +7 -1
  4. package/.bkit/core/.bkit/comparison.md +114 -45
  5. package/.bkit/core/.bkit/install.md +8 -5
  6. package/.bkit/core/.bkit/jsonl.md +3 -3
  7. package/.bkit/core/.bkit/prompts.md +6 -5
  8. package/.bkit/core/.bkit/providers.md +5 -4
  9. package/.bkit/core/.bkit/templates.md +11 -10
  10. package/.bkit/core/.bkit/workflows.md +8 -6
  11. package/.bkit/core/.codex/hooks/bkit-guardrails.cjs +5 -1
  12. package/.bkit/core/.cursor/hooks/bkit-guardrails.cjs +8 -2
  13. package/.bkit/core/docs/templates/bkit-audit-accessibility.md +5 -2
  14. package/.bkit/core/docs/templates/bkit-audit-adoption.md +5 -2
  15. package/.bkit/core/docs/templates/bkit-audit-agents.md +5 -2
  16. package/.bkit/core/docs/templates/bkit-audit-aio-seo.md +5 -2
  17. package/.bkit/core/docs/templates/bkit-audit-analytics.md +5 -2
  18. package/.bkit/core/docs/templates/bkit-audit-annotations.md +6 -3
  19. package/.bkit/core/docs/templates/bkit-audit-app-performance.md +5 -2
  20. package/.bkit/core/docs/templates/bkit-audit-app-store.md +5 -2
  21. package/.bkit/core/docs/templates/bkit-audit-architecture.md +5 -2
  22. package/.bkit/core/docs/templates/bkit-audit-ci-performance.md +5 -2
  23. package/.bkit/core/docs/templates/bkit-audit-config.md +5 -2
  24. package/.bkit/core/docs/templates/bkit-audit-cost.md +5 -2
  25. package/.bkit/core/docs/templates/bkit-audit-coverage.md +5 -2
  26. package/.bkit/core/docs/templates/bkit-audit-database.md +5 -2
  27. package/.bkit/core/docs/templates/bkit-audit-dead-code.md +5 -2
  28. package/.bkit/core/docs/templates/bkit-audit-deferrals.md +5 -2
  29. package/.bkit/core/docs/templates/bkit-audit-dependencies.md +5 -2
  30. package/.bkit/core/docs/templates/bkit-audit-documentation.md +5 -2
  31. package/.bkit/core/docs/templates/bkit-audit-duplication.md +5 -2
  32. package/.bkit/core/docs/templates/bkit-audit-failures.md +6 -3
  33. package/.bkit/core/docs/templates/bkit-audit-flakes.md +6 -3
  34. package/.bkit/core/docs/templates/bkit-audit-gates.md +5 -2
  35. package/.bkit/core/docs/templates/bkit-audit-issues.md +5 -2
  36. package/.bkit/core/docs/templates/bkit-audit-llm.md +5 -2
  37. package/.bkit/core/docs/templates/bkit-audit-logs.md +16 -12
  38. package/.bkit/core/docs/templates/bkit-audit-models.md +5 -2
  39. package/.bkit/core/docs/templates/bkit-audit-privacy.md +5 -2
  40. package/.bkit/core/docs/templates/bkit-audit-prompts.md +5 -2
  41. package/.bkit/core/docs/templates/bkit-audit-red-team.md +5 -2
  42. package/.bkit/core/docs/templates/bkit-audit-reliability.md +6 -3
  43. package/.bkit/core/docs/templates/bkit-audit-repo.md +5 -2
  44. package/.bkit/core/docs/templates/bkit-audit-security.md +6 -3
  45. package/.bkit/core/docs/templates/bkit-audit-slop.md +5 -2
  46. package/.bkit/core/docs/templates/bkit-audit-testing.md +5 -2
  47. package/.bkit/core/docs/templates/bkit-audit-usability.md +5 -2
  48. package/.bkit/core/docs/templates/bkit-plan.md +1 -1
  49. package/.bkit/core/engine/checks/after/index.ts +2 -2
  50. package/.bkit/core/engine/commands/install/check.ts +3 -11
  51. package/.bkit/core/engine/commands/install/doctor.ts +2 -3
  52. package/.bkit/core/engine/commands/install/install.ts +44 -11
  53. package/.bkit/core/engine/commands/install/uninstall.ts +60 -44
  54. package/.bkit/core/engine/commands/install/upgrade.ts +8 -1
  55. package/.bkit/core/engine/commands/workflow/branches.ts +13 -4
  56. package/.bkit/core/engine/install/hosts.ts +14 -4
  57. package/.bkit/core/engine/install/manifest.ts +2 -2
  58. package/.bkit/core/engine/install/modules/core.ts +9 -6
  59. package/.bkit/core/engine/ledgers/telemetry.ts +3 -4
  60. package/.bkit/core/hooks/guardrails-read.js +4 -9
  61. package/.bkit/core/hooks/guardrails-tiers.js +312 -28
  62. package/.bkit/core/prompts/bkit-audit-generate.md +19 -5
  63. package/.bkit/shared/.claude/settings.json +19 -0
  64. package/.bkit/shared/.opencode/opencode.json +21 -1
  65. package/.bkit/shared/docs/deployments.md +14 -0
  66. package/.bkit/templates/.github/workflows/bkit-ci.yml +9 -0
  67. package/.bkit/templates/context/bkit-cloudrun.md +28 -19
  68. package/.bkit/templates/context/bkit-expo.md +30 -13
  69. package/.bkit/templates/context/bkit-inngest.md +54 -17
  70. package/.bkit/templates/context/bkit-resend.md +36 -8
  71. package/.bkit/templates/context/bkit-sentry.md +11 -4
  72. package/.bkit/templates/context/bkit-stripe.md +39 -22
  73. package/.bkit/templates/context/bkit-supabase.md +45 -20
  74. package/.bkit/templates/context/bkit-vercel.md +45 -16
  75. package/.bkit/templates/scripts/ci/accepted-advisories.json +7 -0
  76. package/.bkit/templates/scripts/ci/check-dependency-audit.mjs +200 -0
  77. package/.bkit/templates/scripts/ci/check-dependency-audit.test.mjs +127 -0
  78. package/CHANGELOG.md +45 -0
  79. package/README.md +76 -20
  80. package/dist/checks/after/index.js +1 -1
  81. package/dist/commands/install/check.js +3 -15
  82. package/dist/commands/install/doctor.js +2 -1
  83. package/dist/commands/install/install.js +42 -8
  84. package/dist/commands/install/uninstall.js +52 -45
  85. package/dist/commands/install/upgrade.js +8 -2
  86. package/dist/commands/workflow/branches.js +13 -5
  87. package/dist/install/hosts.js +11 -4
  88. package/dist/install/manifest.js +2 -2
  89. package/dist/install/modules/core.js +9 -7
  90. package/package.json +1 -1
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -65,10 +65,13 @@ Migrations and integration tests exercise queries; neither verifies privileges,
65
65
 
66
66
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
67
67
 
68
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
69
+
68
70
  #### Core
69
71
 
70
72
  Always verify.
71
73
 
74
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
72
75
  - [ ] **Catalogue Read:** Steps 3–16 — read-only SQL (or store's read-only API) per environment, e.g. platform MCP `execute_sql`, `psql` as read-only role. Fallback: local DB from migrations; remote claims = hypotheses.
73
76
  - [ ] **Local DB:** Steps 2, 10, 14 — replayable from migrations. Fallback: rolled-back transactions and remote re-check.
74
77
  - [ ] **Web:** Step 18 — fetch anchor standards. Fallback: versions recorded as unread.
@@ -92,7 +95,7 @@ Verify only when applicable.
92
95
 
93
96
  ### Probe Steps
94
97
 
95
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
98
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
96
99
  - [ ] **Step 2 — Detect:** Find migrations / schema file / DB client. Confirm local DB matches migrations before trusting local measurements. Schema doc absent, migrations exist → missing doc = finding; still read migrations. **Done when:** evidence in `Checked`, or `N/A` and STOP.
97
100
  - [ ] **Step 3 — Advisors:** Run platform security and performance advisors; each row = candidate, not finding. One line per lint the platform's advisor runs, even when clean (e.g. Supabase: unindexed FKs, duplicate indexes, per-row auth in policy, multiple permissive policies, mutable function `search_path`, extension in public schema, exposed table without row-level security, owner-rights view). No advisor → engine-appropriate subset by catalogue query; rest `N/A`. Cite platform fix pattern per finding. **Done when:** every row dispositioned in `Advisors`.
98
101
  - [ ] **Step 4 — Checklist:** Walk platform production checklist against production, not memory: transport encryption enforced, network restrictions, org MFA, >1 owner, point-in-time recovery where size / RPO needs it, compute sized for `Connections` peak. **Done when:** every item met / not met (finding) / not applicable (why).
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -75,10 +75,13 @@ Dead code = control that looks live, is not: limiter on unset env var; E2E asser
75
75
 
76
76
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
77
77
 
78
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
79
+
78
80
  #### Core
79
81
 
80
82
  Always verify.
81
83
 
84
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
82
85
  - [ ] **Clean Tree:** Steps 2–17 — clean `git status` at recorded commit; one checkout all steps. Fallback: detached worktree off integration branch; dirty-path candidates = hypotheses.
83
86
  - [ ] **Text Search:** Steps 5–14 — repo search (`git grep`, ripgrep). Fallback: none → those steps `Not Checked`.
84
87
  - [ ] **Reachability Scanner:** Steps 3–4 — ad hoc, not committed dependency (e.g. `pnpm dlx knip`, vulture). Fallback: language-server refs + text search; scanner categories `Not Checked`.
@@ -104,7 +107,7 @@ Verify only when applicable.
104
107
 
105
108
  ### Probe Steps
106
109
 
107
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
110
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
108
111
  - [ ] **Step 2 — Detectors:** Compiled language → scanner sees module exports only; dead branch/parameter inside live function → compiler + linter. Read resolved config per workspace, not base: unused-local, unused-parameter, no-unreachable-code; linter unused-vars, unreachable, always-true/false (e.g. `@typescript-eslint/no-unnecessary-condition`). Workspace override or file disable switches one off silently. Detector off = finding → `bkit-audit-gates` (gate candidate). **Done when:** `Detectors` lists each config and resolved value.
109
112
  - [ ] **Step 3 — Reachability:** Ad hoc scanner, same resolved version both runs; record version. Do not add it as a dependency, and do not wire a CI gate this run. Report baseline-config readiness. Gate before `False Positives` exclusions encoded → structural noise → disabled. Do not encode test-only code as ignore entries or negated project patterns — plugins add test files as entries; production mode separates. Check scanner stderr before trusting: plugin load failure drops entry class → false unused files. Signal: files, exports, types, duplicates. Dependency categories → `Dependencies` pass, not mixed. Same version in production mode (or strict: implies production + each workspace uses only its own deps). Export used default run, unused production → test/config-only class. Report both runs side by side. **Done when:** version, both runs, plugin-load errors in `Reachability`.
110
113
  - [ ] **Step 4 — Dependencies:** Scanner dependency mode per workspace, categories the scanner offers (e.g. knip: unused, unlisted, unresolved, unlisted binaries, catalog entries unreferenced). Before unused verdict → search name across manifest scripts, config, CI. Config-only consumers (e.g. linter, formatter, CSS presets, bundler plugins, `@types/*`) = usual false hits. Unused = install/supply-chain surface; unlisted = hoisted from another workspace. Unlisted or advisory → `bkit-audit-dependencies`. **Done when:** `Dependencies` row per candidate: name · class · config-or-CI reader · verdict.
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -93,10 +93,13 @@ Never set `unjustified` while repairing a missing or illegal `reason`. That repa
93
93
 
94
94
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
95
95
 
96
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
97
+
96
98
  #### Core
97
99
 
98
100
  Always verify.
99
101
 
102
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
100
103
  - [ ] **Ledger:** Steps 3–9 — `docs/deferrals.jsonl` and `.bkit/jsonl.md`. Fallback: file absent → one `N/A` row after Step 3 and STOP.
101
104
  - [ ] **Git History:** Steps 2, 7 — commits since the previous report, 90-day churn. Fallback: closures and churn `Not Checked`.
102
105
  - [ ] **Web:** Step 10 — fetch anchor standards. Fallback: versions recorded as unread.
@@ -118,7 +121,7 @@ Verify only when applicable.
118
121
 
119
122
  ### Probe Steps
120
123
 
121
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
124
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
122
125
  - [ ] **Step 2 — Closures:** Re-verify every DONE and SUPERSEDED row from the previous report at `HEAD`. Each that no longer holds = finding, with what changed. **Done when:** `Closures` holds every row, each failure, and the failure rate, or run marked baseline.
123
126
  - [ ] **Step 3 — Fold:** Last-row-wins per `key`, not per line. Append-only (`.gitattributes` `merge=union`); editing a line keeps stale copy beside replacement. Highest numeric `ts` wins (`.bkit/jsonl.md` § Updating a Row). Non-numeric `ts` on a `key` → last line in file is latest. Open = every winning row with `status != "CLOSED"`, including `TRIAGED`, `RESOLVED`, `BLOCKED`, `SUPERSEDED`, `CANCELLED`, free text after the word, missing `status`, and any value outside the enum. Finished-looking words ≠ done. Order: `CRITICAL`, then `HIGH`, then oldest `ts`. Compare `severity` case-insensitively. No `severity` → assign with verdict. Trend vs previous report: open count; inflow = sum `deferralsAdded` on `outcome` rows in `.bkit/stats.jsonl` since previous run where field exists; outflow = rows that became `CLOSED` since; age histogram by `ts`; share of open rows missing `severity` or `reason`. Large open set may be partial pass → record rows verified / rows open. Invisible to fold: deferral with no `key`, or recorded only in a PR description. **Done when:** `Trend` holds counts, histogram, inflow, outflow, missing-field share, verified / open, and those limits, or `N/A` and STOP.
124
127
  - [ ] **Step 4 — Defects:** Winning row of each `key`. Do not append. `Expected` = row writer appends: copy latest row, change only defective field, same `key`, `ts` minted at append, never literal and never in the future.
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -86,10 +86,13 @@ Batch verdict, in `Summary`, not per PR. The landing push's CI is the next step'
86
86
 
87
87
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
88
88
 
89
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
90
+
89
91
  #### Core
90
92
 
91
93
  Always verify.
92
94
 
95
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
93
96
  - [ ] **Git Host:** Steps 2, 3, 11, 12 — open PRs by head-ref prefix, CI runs, behind-count, e.g. `gh`, GitLab API. Fallback: none; PR steps `Not Checked`, batch verdict `FAIL`.
94
97
  - [ ] **Package Manager:** Steps 5, 8–10 — install, lockfile, registry metadata, e.g. pnpm, npm. Fallback: none; `Batch` and `Verification` `Not Checked`, batch verdict `FAIL`.
95
98
  - [ ] **Clean Worktree:** Steps 8–10 — detached worktree off the integration branch, removed before the report finishes. Fallback: none; do not build the batch in the main checkout.
@@ -112,7 +115,7 @@ Verify only when applicable.
112
115
 
113
116
  ### Probe Steps
114
117
 
115
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
118
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
116
119
  - [ ] **Step 2 — Inventory:** From previous report, list deferred PRs and why. Deferred major whose blocker cleared → first candidate. Select open PRs by head-ref prefix from updater config (`branchPrefix`, default `renovate/`; Dependabot `dependabot/`), never title or author. Token-authored PR on prefix = in scope; record author as evidence, not selector. Per PR: number, title, head ref, head SHA, mergeable, merge state, files, labels, commits behind integration branch, latest CI run (finished only; in-progress ≠ result). **Done when:** `Inventory` names prefix and config source, lists every open PR on prefix, or `none`.
117
120
  - [ ] **Step 3 — Bot Health:** No dependency updater config = CRITICAL. Examples: Renovate (`.github/renovate.json` or `.github/renovate.json5`; rules example `.bkit/templates/.github/renovate.json5`, self-hosted runner example `.bkit/templates/.github/workflows/bkit-renovate.yml`) or Dependabot (`.github/dependabot.yml`; example `.bkit/templates/.github/dependabot.yml`). See `docs/deployments.md` § Dependency updaters. Both configs = finding; keep one. Bot dashboard: gather-only (bot rewrites). Record blocked entries and will-not-auto-rebase notices → stale until person acts. Bot reads presets from default branch, not integration. Fence / group on integration branch inactive until release carries it. Base-config edits on integration apply only if bot merges base-branch config — option read from default branch. PR proposes what integration config blocks → read default-branch copy before debugging rule; promotion-lag when branches differ = signal. Validate configs as default branch has them. Dashboard + logs: config errors, PRs held by concurrency / hourly limits, awaiting approval, open config-migration PRs. Broken config ≈ quiet week. **Done when:** `Bot Health` holds blocked PRs, config read branch, validator result, held / awaiting-approval items.
118
121
  - [ ] **Step 4 — Supply Chain:** Read settings as default branch has them. Record each vs bot best-practice preset and package manager supply-chain page. Bot: minimum release age >= three days (e.g. Renovate `minimumReleaseAge` / `security:minimumReleaseAgeNpm` + `internalChecksFilter: "strict"`; Dependabot `cooldown`); actions / images digest-pinned; abandonment flagging (e.g. Renovate abandonments preset); config-migration PRs enabled, where bot offers them. Package manager, where offered (e.g. pnpm): trust policy no-downgrade, exotic subdeps blocked, explicit install-script allowlist, strict dependency builds; none offered → `N/A` with evidence. Release-age 0 on package manager = recorded trade-off where docs say so; re-deciding = person (`DECISION`). Gap: bot release age does not apply to lockfile-maintenance PRs (package manager makes changes; gate off → no cooldown) → batch with no cooldown; `Cooldown` stands in; `Screen` first. Missing / weaker setting → `Recommendations` row + line to add. **Done when:** `Supply Chain` row per setting: value · standard · verdict.
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -73,10 +73,13 @@ Docs are the record agents work from: memory points at them, plans cite them, co
73
73
 
74
74
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
75
75
 
76
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
77
+
76
78
  #### Core
77
79
 
78
80
  Always verify.
79
81
 
82
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
80
83
  - [ ] **Repo Read:** Steps 2–13 — live docs, instruction files, manifests, CI workflow. Fallback: none; unreadable tree = finding.
81
84
  - [ ] **Web:** Step 14 — fetch anchor standards. Fallback: versions recorded as unread.
82
85
 
@@ -100,7 +103,7 @@ Verify only when applicable.
100
103
 
101
104
  ### Probe Steps
102
105
 
103
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
106
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
104
107
  - [ ] **Step 2 — Scope:** No live docs → one `N/A` row with evidence and STOP. Else every live doc. Rank by drift: per live doc, commits since that doc last changed to repo paths it cites. Record ranking. Start from up-to-date default branch with clean `git status`. Historical records are citers: pointer leads nowhere = finding, fix at target. Stale fact inside them = history, not defect. **Done when:** every live doc and drift ranking in `Scope`.
105
108
  - [ ] **Step 3 — Catalogue:** `.agents/docs.md`: every file on disk under `docs/` outside `plans/`, `archive/`, and `audits/` has a row, and every row's file exists; no doc listed twice (two docs on one topic = bug, including one doc in both tables with different reading columns); each row's one-line scope still matches the doc. Root `README.md` = working entry point: what the project is, links setup doc, architecture doc, and `.agents/docs.md`, and how to report a security issue. Missing community-health file with no README substitute = finding. **Done when:** `Catalogue` names every mismatch, and every clean check.
106
109
  - [ ] **Step 4 — Mechanical:** Run prose lint, format check, and managed-block check. Then:
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -90,10 +90,13 @@ Carry `COINCIDENTAL` and `RULE-OF-THREE-PENDING` forward in `Carried Forward`
90
90
 
91
91
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
92
92
 
93
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
94
+
93
95
  #### Core
94
96
 
95
97
  Always verify.
96
98
 
99
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
97
100
  - [ ] **Clean Tree:** Steps 2–13 — clean `git status` at recorded commit; findings only mean something against a known tree. Fallback: detached worktree off default branch.
98
101
  - [ ] **Text Search:** Steps 3, 5, 7–10 — inline expressions, literals, env names, non-code files, e.g. ripgrep, `git grep`. Fallback: `grep -rn`.
99
102
  - [ ] **Structural Search:** Steps 3, 12 — syntax patterns that skip comments / strings and catch reordered copies, e.g. ast-grep, Semgrep; ad hoc run, record resolved version. Fallback: text search; reformatted copies `Not Checked`.
@@ -120,7 +123,7 @@ Verify only when applicable.
120
123
 
121
124
  ### Probe Steps
122
125
 
123
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
126
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
124
127
  - [ ] **Step 2 — Detect:** List apps, packages, platforms; shared homes (utility / owner modules); capability register; duplication gates enforced and what each catches (e.g. identical-signature gate → identical copies only). **Done when:** surface map and existing gates in `Checked`, or `N/A` and STOP.
125
128
  - [ ] **Step 3 — Rule Altitude:** Hunt one conceptual operation inlined at N sites instead of called from one function, copies differing on load-bearing details: fallback chains (e.g. JS `a ?? b ?? c`, divergent terminal fallback or order); null vs falsy (e.g. JS `??` vs `||`, with / without trim, optional-chain guards); formatting (date / time / timezone, currency, name / initials, address); predicates ("is active / deleted / owner?" filters); defaults (same enum / config default hardcoded in several readers). Structural rules where possible, text search where not; keep rules under `tmp/`, paste each into report for next run. Subtraction per shared helper: references on helper (routed set) minus sites matching inline pattern (inline set) → inline-not-routed = unmigrated site. Counts before reference-graph change (e.g. project references added) = stale → re-take, never carry forward. **Done when:** `Inline Rules` row per rule: helper · routed · inline · unmigrated · verdict, with its search rule.
126
129
  - [ ] **Step 4 — Clones:** Clone detector over source roots, machine-readable output, three passes: Type-1 exact; Type-2 renamed (ignore identifiers / literals); Type-3 near-miss (e.g. similarity 0.85, syntax-tree mode). Duplication % per package as series vs previous report. Every hit = candidate for `Verify`, not finding. Multiple platforms → counterpart listing (functions with / without counterpart per side). Growth: exact pass baselined at previous report's commit → clones new since vs consolidated since, blame naming introducer; arrival faster than consolidation = finding. **Done when:** per-package %, growth, counterpart list in `Clones`.
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -88,10 +88,13 @@ Row text awaiting confirmation = UNCONFIRMED until reproduced or read end to end
88
88
 
89
89
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
90
90
 
91
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
92
+
91
93
  #### Core
92
94
 
93
95
  Always verify.
94
96
 
97
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
95
98
  - [ ] **Ledger:** Steps 3–7 — `docs/failures.jsonl` and `.bkit/jsonl.md`. Fallback: file absent → one `N/A` row after Step 3 and STOP.
96
99
  - [ ] **Git History:** Steps 2, 5 — commits since the previous report, fix commits. Fallback: closures and regression-by-commit `Not Checked`.
97
100
  - [ ] **Web:** Step 8 — fetch anchor standards. Fallback: versions recorded as unread.
@@ -101,7 +104,7 @@ Always verify.
101
104
  Verify only when applicable.
102
105
 
103
106
  - [ ] **Language Server:** Typed language → Step 4 — symbol at current path, caller chain, diagnostics for one file without full typecheck, e.g. Serena, `tsserver`. Fallback: text search. Reach and close needing callers stay hypotheses until corroborated.
104
- - [ ] **Error Tracker:** Production or preview events exist → Steps 4, 5 — event counts and flows after fix commit, e.g. Sentry, platform logs. Fallback: code only. Event counts `Not Checked`.
107
+ - [ ] **Error Tracker:** Production or preview events exist → Steps 4, 5 — event counts and flows after fix commit, e.g. Sentry, platform logs. Log access: `.agents/context.md` `context-fetch` blocks below `bkit:end` are ready-made read-only readers (`bkit context list-blocks` → each block's `context` and `body`); set a body's `# Customise:` knobs to the audit window; run it via `direnv exec . bash`. Fallback: code only. Event counts `Not Checked`.
105
108
  - [ ] **Tracker:** Issues wired → Steps 4, 7 — issue carrying bug, postmortem tracking issue. Fallback: issue pointers `Not Checked`.
106
109
  - [ ] **CI Config:** Workflow can fail build → Step 3 — gate on open `CRITICAL` failures. Fallback: gate check `Not Checked`.
107
110
 
@@ -114,7 +117,7 @@ Verify only when applicable.
114
117
 
115
118
  ### Probe Steps
116
119
 
117
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
120
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
118
121
  - [ ] **Step 2 — Closures:** Re-verify every FIXED and MIS-FRAMED row from previous report at `HEAD`. Each no longer holding = finding, with what changed. **Done when:** `Closures` holds every row, each failure, and failure rate, or run marked baseline.
119
122
  - [ ] **Step 3 — Fold:** Last-row-wins per `key`, not per line. Append-only file (`.gitattributes` `merge=union`); editing a line keeps stale copy beside replacement. Highest numeric `ts` wins (`.bkit/jsonl.md` § Updating a Row). `key` whose `ts` is not a number → last line in file is latest. Open = every winning row with `status != "CLOSED"`, including `TRIAGED` (diagnosed is not fixed), `RESOLVED`, `FIXED`, `SUPERSEDED`, and rows with no `status`. Word that looks finished is not done. `RESOLVED` or `FIXED` row really done → `CLOSED` row in `Expected`, with `references` that closed it. Not skipped because word looks finished. Order: `CRITICAL`, then `HIGH`, then oldest `ts`. Compare `severity` case-insensitively. No `severity` → assign with verdict. Age from row's first `ts`: list every `CRITICAL` older than 7 days and every `HIGH` older than 30 days. Where CI check goes red while any `CRITICAL` row has `status != "CLOSED"`, report whether installed. Green check beside such row = finding (gate not reading ledger). Absent → record in `Checked`. Large open set may be partial pass → record rows verified / rows open. Winning future `ts` that would outrank close → name `key`. No later `ts` invented. **Done when:** `Trend` holds open counts, age-target breaches, gate result, and verified / open, or `N/A` and STOP.
120
123
  - [ ] **Step 4 — Source:** No verdict before this step. Under-reporting biases toward closing live bug.
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -79,11 +79,14 @@ One per flake. Root-cause class (`Mechanism`) = evidence; these = outcomes. Quar
79
79
 
80
80
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
81
81
 
82
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
83
+
82
84
  #### Core
83
85
 
84
86
  Always verify.
85
87
 
86
- - [ ] **CI History:** Steps 2, 3 — workflow runs since previous report, e.g. `gh run list`, Actions API, GitLab pipelines. Fallback: local repeats only; ranked CI list `Not Checked`.
88
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
89
+ - [ ] **CI History:** Steps 2, 3 — workflow runs since previous report, e.g. `gh run list`, Actions API, GitLab pipelines. Log access: `.agents/context.md` `context-fetch` blocks below `bkit:end` are ready-made read-only readers (`bkit context list-blocks` → each block's `context` and `body`); set a body's `# Customise:` knobs to the audit window; run it via `direnv exec . bash`. Fallback: local repeats only; ranked CI list `Not Checked`.
87
90
  - [ ] **Repeat Runner:** Steps 4, 7, 8 — repeat with retries off, e.g. Playwright `--repeat-each`, pytest-repeat. Fallback: rate `Not Checked`; do not mark `FIX NAMED` on unmeasured flake unless mechanism deterministic and report says rate unknown.
88
91
  - [ ] **Web:** Steps 6, 13 — vendor status pages and anchor standards. Fallback: incidents and versions recorded as unread.
89
92
 
@@ -105,7 +108,7 @@ Verify only when applicable.
105
108
 
106
109
  ### Probe Steps
107
110
 
108
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
111
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
109
112
  - [ ] **Step 2 — List:** Before triage, list every test with runner status flaky (failed, passed on retry) and every pass-only-on-retry, across every workflow since previous report. Also list fail one run/surface, pass another: with retries off, `flaky` never appears; a noticed red run is input, not the list. Rank by flake rate (flaky or failed runs ÷ runs) and reruns caused. Per open flake: first-seen, last-seen, age. Report median age of open flakes. Red run absent from list → detection finding. Triage in rank order. **Done when:** ranked list, ages, median in `Flakes`.
110
113
  - [ ] **Step 3 — Pin:** Per triaged flake: run id, commit, job that failed vs downstream-only. Control run and valid scope. Prior red run = control only when failure mode differs — auth failure redirecting every lane to sign-in is control for tenant-minting lane only. **Done when:** run, job, control in `Pin`.
111
114
  - [ ] **Step 4 — Rate:** Spec with repeat-each, retries off, at CI worker count: k failures in N runs. Repeat with CPU throttled/loaded. Record N, k, workers, commit. Without measured rate, nothing to compare fix against (Step 11). **Done when:** N, k, workers, commit in `Rate`, or why not measured.
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -71,10 +71,13 @@ One per candidate.
71
71
 
72
72
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
73
73
 
74
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
75
+
74
76
  #### Core
75
77
 
76
78
  Always verify.
77
79
 
80
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
78
81
  - [ ] **Sibling Reports:** Steps 2, 3 — newest report per other `bkit-audit-*` slug under `docs/audits/`, else that template's `Process Improvements`. Fallback: templates only; a missing report = that audit `Not Checked`.
79
82
  - [ ] **Ledgers:** Steps 3, 10, 11 — `docs/failures.jsonl`, `docs/decisions.jsonl`, `docs/deferrals.jsonl`. Fallback: incident evidence and prior rejections `Not Checked`.
80
83
  - [ ] **Repo Gates:** Steps 7, 9, 12, 13, 15 — workflows, hook directory, gate scripts, allowlist files, e.g. `.github/workflows`, `.husky`, `.gitlab-ci.yml`, every `bkit:gate:*` script in `package.json`. Fallback: none; those items `Not Checked`.
@@ -97,7 +100,7 @@ Verify only when applicable.
97
100
 
98
101
  ### Probe Steps
99
102
 
100
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
103
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
101
104
  - [ ] **Step 2 — Inventory:** Collect gate candidates from each other `bkit-audit-*` report's `Process Improvements` (else that template's), without re-deriving. Record source slug. Two audits proposing one guard = one candidate, both sources — not two work items. Re-grep of a swept class + owner-allowlist over same literals = one guard. **Done when:** `Inventory` row per candidate: source slug · one-line guard.
102
105
  - [ ] **Step 3 — Widen:** Sibling sections are one source. Also add, record source each: (a) every defect class in `docs/failures.jsonl` on ≥2 rows, whether or not a sibling proposed a gate; (b) standard gates nobody proposed, each vs tree — secret-scanning push protection, code scanning (e.g. CodeQL), dependency-review check, workflow linters (e.g. `actionlint`, `zizmor`), lockfile lint, test-runner forbid-only flag (e.g. Playwright `--forbid-only`), typechecking of test files, OpenSSF Scorecard checks; (c) policy-as-code for config with no type system (e.g. JSON Schema, Conftest) — bot config, job-runner triggers and concurrency, hosting config, mobile build config, workflow YAML. Class (c) is what `bkit-audit-testing` flags. Already enforced → `Health`, not a second candidate. **Done when:** every added row in `Inventory` with source.
103
106
  - [ ] **Step 4 — Score:** Per candidate, six axes. Fails if fail condition holds.
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -113,10 +113,13 @@ Finding `CRITICAL`–`LOW` in another audit does not set priority. Two rules do.
113
113
 
114
114
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
115
115
 
116
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
117
+
116
118
  #### Core
117
119
 
118
120
  Always verify.
119
121
 
122
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
120
123
  - [ ] **Tracker Read:** Steps 2, 5, 10 — every board item with lane, size, and priority, e.g. `gh project item-list`, Linear issues. Fallback: none; unread items `Not Checked`, proposal does not cover them.
121
124
  - [ ] **Open-Issue List:** Steps 2, 10 — every open issue, every repo / team board draws from, read separately from board, e.g. `gh issue list --state open`, Linear team issues. Fallback: none; membership `Not Checked`, proposal covers board items only.
122
125
  - [ ] **Issue Bodies:** Steps 3, 4, 6, 8 — title, labels, body, parent, milestone, blocked-by, e.g. `gh issue view`, Linear issue API. Fallback: titles only; each such rank low-confidence in `Judgement Calls`, links from structured fields only.
@@ -138,7 +141,7 @@ Verify only when applicable.
138
141
 
139
142
  ### Probe Steps
140
143
 
141
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
144
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
142
145
  - [ ] **Step 2 — Board:** Read every item. Assert read not truncated (count below limit; at limit → raise and re-read). Record lane names, size and priority field ids and option ids from live field list. Unused numeric estimate stays unused; size = estimate. Membership: diff open-issue list vs board. Open issue not on board = off-board; closed issue outside done lane = stale card; list each by number. Off-board stays in scope: Steps 3–10 treat as item with no lane, size, priority. **Done when:** item count, lanes, field ids, off-board, stale cards in `Board`, or `N/A` and STOP.
143
146
  - [ ] **Step 3 — Triage:** Read every open issue, not titles only. Short body prefix OK when summary front-loaded; open umbrella/milestone issues in full. While reading: stale = no activity 90 days and body no longer true of code/plan (standing issue never stale for age); duplicate = same change twice or one wholly inside another; `Low` on 3+ passes and still true = won't-do candidate (standing excluded); ready-lane item missing acceptance criteria, defect reproduction, or test to write = not ready (still gets size/priority). Note each Rule 1 override for Step 6 with label correction. List stale/duplicate with merge target or stale reason. Closing = owner; this run does not close. Monthly clock → stale only. **Done when:** `Triage` lists stale, duplicates, won't-do, not-ready, overrides to rank.
144
147
  - [ ] **Step 4 — Links:** Per open issue record parent, milestone, blocked-by. Blocked-by: read all three — structured links (`Link API`), body block build gates on (e.g. GitHub `###### bkit:blocked-by start` / `end`), body prose ("blocked by #N", "after #N"). Flag: (1) Parent — child listed in parent body, no link (or reverse); open child, closed parent; open non-standing parent, all children closed → `Judgement Calls`. (2) Milestone — closed / past-due milestone with open issues; child with no or later milestone than parent; body / title names milestone, field empty; open milestone, no open issues. (3) Blocked-by — prose only; structured link vs body block disagree; closed blocker still listed; blocked issue in ready or later lane, blocker open; blocker off-board; cycle. Edges → Step 5 blocker rule, Step 6 parent / child moves. **Done when:** `Links` holds per-kind counts (with / without parent, milestone, blocker) and every flag, or `none`.
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -77,10 +77,13 @@ Taxonomy names match `bkit-audit-red-team`, so a finding here and its live proof
77
77
 
78
78
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
79
79
 
80
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
81
+
80
82
  #### Core
81
83
 
82
84
  Always verify.
83
85
 
86
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
84
87
  - [ ] **Code Search:** Steps 2–12, 14 — text search and language server for callers (e.g. ripgrep, Serena, `tsserver`). Prompts often markdown; tool names often strings. Language-server hits under-report; empty inconclusive → corroborate with text search and name the tool. Fallback: text search only; name it per claim.
85
88
  - [ ] **Taxonomy Skill:** Steps 2–9, 13 — skill covering LLM and agentic top 10 editions in Anchor Standards (e.g. `owasp-security`). Categories it lacks → `Checked`. Fallback: read anchor standards directly.
86
89
  - [ ] **Web:** Steps 4, 9, 13 — tool-server spec, provider terms, anchor standards. Fallback: regime and version changes recorded as unread.
@@ -101,7 +104,7 @@ Verify only when applicable.
101
104
 
102
105
  ### Probe Steps
103
106
 
104
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
107
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
105
108
  - [ ] **Step 2 — Inventory:** Start from code, not remembered caller list. Find every model-package import outside that package and outside tests; list prompt-set directories. Load taxonomy skill; categories it lacks vs anchor editions → `Checked`. Per model call and tool, one `Inventory` row: input and who controls it · model-group name at call site (choice, deprecation, catalogue, zero-retention routing → `bkit-audit-models`; do not score here) · tools offered · what a tool call writes, and to which tenant · DB client the write runs as · tenant data in prompt · output reaching someone outside tenant. Diff table vs previous run. New call site, prompt set, or tool read first. Per surface, AI impact-assessment record with named owner (ISO/IEC 42001 6.1.4, ISO/IEC 42005, NIST AI RMF MAP). Update from this table; do not rebuild. No record = finding. Checked-in inventory failing when new call site has no row → propose under `Process Improvements`; do not add this run. **Done when:** `Inventory` lists every call and tool, or `N/A` and STOP.
106
109
  - [ ] **Step 3 — Capability:** Per tool: does caller need it, what bounds it? Model writes from untrusted inbound → which rows: only from same message/thread, or any tenant row? State outcome of instruction to delete everything matching a phrase. "Any row" = finding, ranked by one-message removal scope. Outside-tool credential tenant-wide → read-only limit possible? If not, design question in `Governance`, not defect, unless code contradicts `docs/decisions.jsonl`. Read-only tools write nothing; destructive tools marked destructive. Host skipping confirmation on read-only tools → wrong annotation removes human check. Allowlist subtracting fields still subtracts; contract test pins; new tools through same filter. Tenant id from credential row, never tool arguments. Check every tool since previous run. Similarity search scoped by tenant in query (LLM08). Every embedding query filters tenant in query itself (privileged role bypasses RLS). Embedding regenerated on source edit, deleted on row delete. **Done when:** `Capability` states inbound bound and verdict for annotations, allowlist, tenant id, embedding scope.
107
110
  - [ ] **Step 4 — Governance:** Outside model holds tool-server credential → check against protocol authorization and security best practices. Record auth model. Where spec says HTTP server with authorization should use OAuth with protected-resource metadata and audience-bound tokens, static bearer = design question in `Governance`, not defect, unless code contradicts `docs/decisions.jsonl`. Token lifecycle: default lifetime, scopes, rotation/revocation, per-tool-call logging per token. Server never passes received token downstream. Snapshot every tool name, description, annotations; diff previous run. Changed tool description = supply-chain risk (ASI04): outside model reads as instructions → needs reviewer. No tool server → skip with evidence. **Done when:** `Governance` holds auth, lifecycle, passthrough, description diff, or skip.
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -70,7 +70,7 @@ One per unresolved issue. Verdict decides everything downstream.
70
70
 
71
71
  - **GENUINE ERROR:** unhandled failure, or handled with wrong handling. Root-cause it. Fix = next step.
72
72
  - **WRONG LEVEL:** working as intended, filed at error: 2xx response, guard did its job, or docblock documents branch. Recommend level change. Do not edit call site.
73
- - **NOISE:** vendor or infrastructure, no app call site (SDK internals, tunnel egress, platform insight). Recommend tracker-side filter (e.g. `beforeSend` denylist) and warnings-doc row. Do not add filter here. Denylist = vendor frames with no call site, not app code.
73
+ - **NOISE:** vendor or infrastructure, no app call site (SDK internals, tunnel egress, platform insight). Recommend tracker-side filter (e.g. `beforeSend` denylist) and `docs/noise.md` row. Do not add filter here. Denylist = vendor frames with no call site, not app code.
74
74
  - **OTHER:** fits none. Say why and what the issue needs. Same `OTHER` on 2 runs → propose new verdict in `Process Improvements`.
75
75
 
76
76
  A test-origin tag narrows the lane. It does not make something noise, and it does not lower severity.
@@ -79,10 +79,13 @@ A test-origin tag narrows the lane. It does not make something noise, and it doe
79
79
 
80
80
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
81
81
 
82
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
83
+
82
84
  #### Core
83
85
 
84
86
  Always verify.
85
87
 
88
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
86
89
  - [ ] **Error Tracker:** Steps 3–9 — read-only issue, event, and alert APIs, e.g. tracker MCP, REST. Fallback: skip tracker steps; runtime-log steps still run; tracker sections `Not Checked`. Never "clean".
87
90
  - [ ] **Web:** Step 13 — fetch anchor standards. Fallback: versions recorded as unread.
88
91
 
@@ -90,7 +93,7 @@ Always verify.
90
93
 
91
94
  Verify only when applicable.
92
95
 
93
- - [ ] **Runtime Logs:** Hosted platforms → Step 11 — read-only log APIs, e.g. log-fetch blocks, vendor CLIs, MCPs. Skipped credential = not read, never clean. Fallback: each skipped source named in `Not Checked`.
96
+ - [ ] **Runtime Logs:** Hosted platforms → Step 11 — read-only log APIs, e.g. vendor CLIs, MCPs. Log access: `.agents/context.md` `context-fetch` blocks below `bkit:end` are ready-made read-only readers (`bkit context list-blocks` → each block's `context` and `body`); set a body's `# Customise:` knobs to the audit window; run it via `direnv exec . bash`. Skipped credential = not read, never clean. Fallback: each skipped source named in `Not Checked`.
94
97
  - [ ] **Language Server:** Typed language → Step 7 — references, e.g. Serena, `tsserver`. Fallback: text search only; name it on every blast-radius row.
95
98
  - [ ] **Symbol Maps:** Release builds → Step 4 — source maps and native symbols. Fallback: unsymbolicated = finding; call-site table marked unbuildable.
96
99
  - [ ] **User Reports:** Feedback channel exists → Step 10 — tracker user feedback, support inbox, store reviews. Fallback: `Not Checked`.
@@ -104,15 +107,16 @@ Verify only when applicable.
104
107
 
105
108
  ### Probe Steps
106
109
 
107
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
110
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
108
111
  - [ ] **Step 2 — Detect:** Find error tracker, runtime logs, or both. **Done when:** evidence in `Checked`, or `N/A` and STOP.
109
112
  - [ ] **Step 3 — Board:** Same figures every run for comparison. Window = since previous report; also trailing-30-day when that is tracker default:
110
113
  - Unresolved count, and issue short ids.
111
114
  - Regressed count.
112
115
  - Events by environment. Name each. Call out production.
113
116
  - First seen, per issue, UTC.
117
+ - Reported vs platform failures. Per platform re-reporting failures to the tracker (e.g. job runner, host): platform failure count beside tracker event count, same window. Ratio < 1 = reports lost.
114
118
 
115
- Production count > zero → reprioritize everything. No real user traffic → numbers describe CI/test traffic, not user impact; report says so. Resolve release SHA per issue (`git rev-parse`, `git branch --contains`) before branch attribution. Integration-branch HEAD event = dev-side bug, not PR-under-review defect. Release health per client (e.g. web, native): crash-free sessions and crash-free users, per release and environment; latest-release adoption. Low traffic → describes CI; report says so. Previous report expected fix → confirm held and duration. Miss in window = finding. "Shipped" fix absent from default branch = still live = finding. **Done when:** `Board` holds the four figures, release health, and those confirmations.
119
+ Production count > zero → reprioritize everything. No real user traffic → numbers describe CI/test traffic, not user impact; report says so. Resolve release SHA per issue (`git rev-parse`, `git branch --contains`) before branch attribution. Integration-branch HEAD event = dev-side bug, not PR-under-review defect. Release health per client (e.g. web, native): crash-free sessions and crash-free users, per release and environment; latest-release adoption. Low traffic → describes CI; report says so. Previous report expected fix → confirm held and duration. Miss in window = finding. "Shipped" fix absent from default branch = still live = finding. **Done when:** `Board` holds the five figures, release health, and those confirmations.
116
120
 
117
121
  - [ ] **Step 4 — Instrument:** Read tracker project settings, read-only, against anchor standards.
118
122
  - Symbolication. One recent production event and one non-production: frames show original file/line, event carries debug ids, release set. Native client → Hermes maps, dSYM, ProGuard uploads where that client exists. Unsymbolicated stack → call-site table unbuildable = finding.
@@ -132,8 +136,8 @@ Verify only when applicable.
132
136
 
133
137
  - [ ] **Step 6 — Classify:** One verdict per unresolved issue. Tracker performance issues (e.g. N+1, slow DB spans) and Web Vitals regressions: not classified. Record count on `Board`; name defect-like ones for `bkit-audit-app-performance`. Flake mechanism → note for `bkit-audit-flakes`. **Done when:** every unresolved issue has verdict in `Issues`; performance count on `Board`.
134
138
  - [ ] **Step 7 — Root Cause:** Each `GENUINE ERROR` and any issue whose verdict depends on call site. Message = symptom. Read call site → path → blast radius. Call-site table = artefact: callers sharing broken mechanism vs not. Language server, where available: stack symbol → every referencing symbol → diagnostics for that file. Pace reference queries; partial answer = indistinguishable from real. Re-run surprising low count. Empty = inconclusive → corroborate with text search; say which count from where. Ask why no test caught it. Every suite mocking SDK = second finding. Verify constraints against live system, not memory: actual predicate, rolled-back insert, host vs published IP ranges. Say what is unverifiable. Teardown often deletes rows an event points at → non-production findings corroborated by mechanism, not row. A mechanism that fits is not a confirmed cause. **Done when:** each `GENUINE ERROR` row in `Issues` has root cause, call-site table, the method that produced it, what is not established, and the file a later change would touch.
135
- - [ ] **Step 8 — Level:** Each `WRONG LEVEL`: recommend error-level log → warning, plus warnings-doc row (call site, reason). Do not edit. Do not recommend tracker denylist for app code. Before downgrading, check hypothesis that would make it real defect; record check. Read both symbols; do not search column name. Fingerprint explosion: message with varying payload (ids, arrays) needs stable grouping key and pinned test before left in stream. **Done when:** each `WRONG LEVEL` row names the level change, the hypothesis checked, any fingerprint gap, and the file a later change would touch.
136
- - [ ] **Step 9 — Disposition:** Recommend resolve, archive, or leave open and why, per issue. This run does not resolve or archive. Recommend resolve only when code that files the issue changed and shipped. Resolving ahead of fix hides only signal on rails with no other coverage. **Done when:** `Issues` states resolve, archive, or leave open for every issue.
139
+ - [ ] **Step 8 — Level:** Each `WRONG LEVEL`: recommend error-level log → warning, plus `docs/noise.md` row (call site, reason). Do not edit. Do not recommend tracker denylist for app code. Before downgrading, check hypothesis that would make it real defect; record check. Read both symbols; do not search column name. Fingerprint explosion: message with varying payload (ids, arrays) needs stable grouping key and pinned test before left in stream. **Done when:** each `WRONG LEVEL` row names the level change, the hypothesis checked, any fingerprint gap, and the file a later change would touch.
140
+ - [ ] **Step 9 — Disposition:** Recommend resolve, archive, or leave open and why, per issue. This run does not resolve or archive. Recommend resolve only when code that files the issue changed and shipped. Resolving ahead of fix hides only signal on rails with no other coverage. Archived / muted issues too: per `docs/noise.md` archive or mute row, compare window events with the row's recorded trigger. Past trigger → recommend un-archive. **Done when:** `Issues` states resolve, archive, or leave open for every issue; every archive / mute row checked against its trigger.
137
141
  - [ ] **Step 10 — Invisible:** Tracker only sees errors that reach it. Caught then neither logged nor rethrown = invisible to caller and this sweep. Quiet board ≠ evidence shape absent. Look only in code behind this sweep's issues (`bkit-audit-reliability` sweeps rest):
138
142
  - `catch` blocks that neither rethrow, return error, nor log at warn or above. Start with DB writes and outbound calls.
139
143
  - Client calls whose error is never read: result discarded, or success destructured without error.
@@ -141,22 +145,22 @@ Verify only when applicable.
141
145
 
142
146
  Text search finds candidates; read each. Deliberate best-effort (telemetry, fire-and-forget notification) OK when code says so. Real one → Expected names `docs/failures.jsonl` row to append; names `bkit-audit-database` when miss is grant, policy, or write role. Appending = next step. Users see same shape: tracker user feedback, support inbox, store reviews since previous report. User-reported failure with no matching event = this class. Next step = runtime half: swallowed DB error may appear in DB log. **Done when:** `Invisible` lists each candidate read, and its verdict.
143
147
 
144
- - [ ] **Step 11 — Runtime Logs:** Read each runtime-log source in `Project Instructions` for whole window since previous report, not tool default short window. Record retention from plan page and window part source could not cover. Retention < cadence = finding → drain or shorter cadence. Skipped reader (credential, tool, block) = not read, never clean. No reader → vendor console by hand, or not read and why. Note whether server logs share one structured shape (request id, trace id, hashed tenant id, error code) so queries are filters not text search; tracker log product or OTel log exporter in use. Per log error:
148
+ - [ ] **Step 11 — Runtime Logs:** Read each runtime-log source in `Project Instructions` and each `context=runtime` block in `.agents/context.md` for whole window since previous report, not tool default short window. Record retention from plan page and window part source could not cover. Retention < cadence = finding → drain or shorter cadence. Skipped reader (credential, tool, block) = not read, never clean. No reader → vendor console by hand, or not read and why. Note whether server logs share one structured shape (request id, trace id, hashed tenant id, error code) so queries are filters not text search; tracker log product or OTel log exporter in use. Per log error:
145
149
  - Match tracker first (timestamps; trace id joins). Covered → note platform evidence on issue. Same incident, not second finding.
146
150
  - Group by time across sources. Cluster within few minutes = one outage; write once.
147
151
  - Tracker empty → gap = finding. Noise (why), or failure that should have been reported. Second → Expected names signal: log at call site, upstream-outage classifier pattern, or health-endpoint check. Monitor/capacity alert = next step, not this run.
148
152
  - Never copy log line free text. PII-safe fields reader already selects.
149
153
 
150
- Check correlation without clock. Every inbound-request → background-work path: one trace (e.g. W3C `traceparent`, tracker trace header) spans host, job runner, DB calls, in tracker and job-runner traces. Broken hop = finding. Without it, time grouping = only join. Platform failure re-reported to tracker → counts must match; platform failure with no tracker event = broken reporter. **Done when:** `Runtime Logs` row per source: read or not (why) · window covered · retention · error count · matched tracker issues. Correlation recorded.
154
+ Check correlation without clock. Every inbound-request → background-work path: one trace (e.g. W3C `traceparent`, tracker trace header) spans host, job runner, DB calls, in tracker and job-runner traces. Broken hop = finding. Without it, time grouping = only join. Platform failure re-reported to tracker → counts must match; platform failure with no tracker event = broken reporter. **Done when:** `Runtime Logs` row per source: read or not (why) · window covered · retention · error count · matched tracker issues. Correlation recorded. `Board` holds reported-vs-platform ratio per reporting platform.
151
155
 
152
156
  - [ ] **Step 12 — Alarms:** Prior steps read what arrived. This step checks what is meant to arrive still can. Per observability doc (health endpoint, alert routing, uptime monitors):
153
157
  - Uptime monitors. List each: last check-in, last failure, still right URL/environment. Status-only assertion can pass on login wall when checker follows redirects.
154
- - Cron coverage. Every clock job health endpoint should know. Unknown job can stop silently.
158
+ - Cron coverage. Every clock job health endpoint should know. Unknown job can stop silently. Read every run, not each job's newest: run started and unfinished past its timeout = finding.
155
159
  - Alert routing. Rule exists, destination current, last real alert seen. Alert never seen fire since created = unproven, not healthy. Destination by role or doc section; no personal address.
156
160
  - Alert quality. Count fired since previous report, share that led to action, time to acknowledge. Rule fired repeatedly with no action = noise; name it. Rule with no environment filter letting non-production noise reach person = noise; name it.
157
161
  - On-call. Who gets production alerts OOH, if no ack, whether quiet hours suppress anything critical. Before launch, developer inbox OK only as written answer.
158
162
 
159
- No error tracker → monitor list and alert rules `Not Checked`; cron coverage still from repo. Before launch, much unproven for lack of traffic. Record what could not be proven and why. Missing monitor = next step, not this run. **Done when:** `Alarms` row per monitor and cron: last check-in · what could not be proven. Alert-quality numbers and on-call answer included.
163
+ No error tracker → monitor list and alert rules `Not Checked`; cron coverage still from repo. Before launch, much unproven for lack of traffic. Record what could not be proven and why. Missing monitor = next step, not this run. **Done when:** `Alarms` row per monitor and cron: last check-in · runs left unfinished · what could not be proven. Alert-quality numbers and on-call answer included.
160
164
 
161
165
  - [ ] **Step 13 — Standards:** Confirm current version per anchor standard; read changes since previous report's version. `Standards Coverage`: one row per tracker setup practice (symbolication, release, environment, release health, grouping, ownership, quotas, spike protection, scrubbing) and per trace-propagation and SLO-alerting practice → checked (name step) / not applicable (why) / finding. Unapplied recommended practice → adopt (propose step) / reject (why) / not applicable. **Done when:** no empty row.
162
166
  - [ ] **Step 14 — Anything Else:** (a) Everything changed since previous report's commit touching error reporting and logs (tracker init and integrations, scrubber, upstream-outage classifier, new error-level call sites, alert rules, uptime and cron monitors, log-fetch blocks, new outside services) → one-line verdict each. (b) Bounded ~30 min pass: what senior SRE flags that no step asks; read since previous run (incidents, new `docs/failures.jsonl` rows, vendor status incidents) — what would no step catch? Any error source this run saw that `Runtime Logs` does not name → record and propose source in `Process Improvements`. **Done when:** `Beyond the Steps` holds both passes, or `none`. Class seen on 2 runs → propose step in `Process Improvements`.
@@ -179,7 +183,7 @@ Verify only when applicable.
179
183
  | `CRITICAL` | Unhandled failure on a production path with no workaround; swallowed write that returns success; message text or credentials stored in the tracker |
180
184
  | `HIGH` | Genuine error on a rail with no other coverage; production events above zero and unread; dropped events or quota exhaustion that blinds the board; platform-only failure with no signal; retention shorter than the cadence on a source production depends on |
181
185
  | `MEDIUM` | Handled outcome filed at error; unsymbolicated stack; no owner on most unresolved issues; alert that has never fired, or fires with no action; fingerprint explosion; user report with no event |
182
- | `LOW` | Noise with a named vendor frame; test-traffic issue already tagged; warnings-doc wording drift |
186
+ | `LOW` | Noise with a named vendor frame; test-traffic issue already tagged; `docs/noise.md` wording drift |
183
187
 
184
188
  Scale per `.bkit/jsonl.md` § Severity. Within a severity: production first, then blast radius. A test-origin tag does not lower severity. A quiet board is not a pass.
185
189
 
@@ -1,4 +1,4 @@
1
- <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. `@bkit-audit-generate` fills this template. -->
1
+ <!-- Note: Reports live at `docs/audits/{YYYYMM}/{slug}.md`. `{YYYYMM}` = UTC month of this run. `{slug}` = template basename without `bkit-audit-` and `.md`. **If** that month already has `{slug}.md` or `{slug}-{TIMESTAMP}.md` → write `{slug}-{TIMESTAMP}.md` (`{TIMESTAMP}` = `date -u +%Y%m%dT%H%M%SZ`). Never overwrite. The report is stamped with a `bkit:anchor` block recording the audited commit (`baseSha`) and branch (`baseRef`); never hand-author or edit it. `@bkit-audit-generate` fills this template. -->
2
2
 
3
3
  <!-- bkit:start -->
4
4
 
@@ -68,10 +68,13 @@ Config ids look identical delisted vs choice day. Target failure: **a stale entr
68
68
 
69
69
  **Checkpoint:** Check off all core items. Check conditional items only when applicable. Each item: available → tool and version; missing → fallback, gap in `Not Checked`, fix in `Process Improvements`. Unnamed gap reads as passed. never abort for a missing capability.
70
70
 
71
+ **Credentials:** probe direnv before marking a token / env var missing: `direnv exec . sh -c '[ -n "${VAR}" ] && echo set || echo unset'`; `unset` = missing. Run credentialed CLIs as `direnv exec . <cmd>`. No direnv (cloud session) → plain env check. Report names, never values.
72
+
71
73
  #### Core
72
74
 
73
75
  Always verify.
74
76
 
77
+ - [ ] **Permissions:** Before Step 1 — one cheap read-only call per tool this run uses (every item below, every rule under `Project Instructions` `Permissions`), e.g. `gh run list --limit 1`, one MCP read, one anchor-standard fetch. Blocked / denied → record call verbatim and narrow allow rule clearing it (e.g. `Bash(gh run list *)`) in `Not Checked`, `Process Improvements`, and `Project Instructions` `Permissions`. never retry or reword a blocked call to get past it.
75
78
  - [ ] **Catalogue Read:** Steps 3, 5, 6 — live model list per provider call sites use, e.g. OpenRouter `/api/v1/models`, Anthropic models API, OpenAI models API, Google models API. Fallback: maker docs only; live-endpoint claims = hypotheses.
76
79
  - [ ] **Web:** Steps 5, 10 — maker deprecation pages, model cards, anchor standards. Fallback: versions recorded as unread.
77
80
 
@@ -93,7 +96,7 @@ Verify only when applicable.
93
96
 
94
97
  ### Probe Steps
95
98
 
96
- - [ ] **Step 1 — Previous Run:** Record `git rev-parse HEAD`. Read newest prior report for this slug under `docs/audits/`. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
99
+ - [ ] **Step 1 — Previous Run:** Audited commit = this report's `bkit:anchor` `baseSha`. Read newest prior report for this slug under `docs/audits/`; its commit = its `bkit:anchor` `baseSha`, else the HEAD its `Previous Run` recorded. Count findings fixed / still open; age of oldest open. Open after 2 runs → re-raise with age. **Done when:** closure rate in `Previous Run`, or run marked baseline.
97
100
  - [ ] **Step 2 — Detect:** Find every model call site (SDK calls, HTTP to a model API, config lists) and whether choice lives in one registry or literals at sites. Embeddings and other modalities count. Registry absent and ids inline = scatter is the finding; still audit ids. **Done when:** evidence in `Checked`, or `N/A` and STOP.
98
101
  - [ ] **Step 3 — Catalogue:** Pull live catalogue for each provider Step 2 found. Record date, URL, counts (full list, any constrained subset calls require, e.g. zero-retention). Every id proposed later must exist in this pull. **Done when:** date, source, and counts in `Catalogue`.
99
102
  - [ ] **Step 4 — Call Sites:** Derive each call site's requirements from code this run: modality, tools, and tool choice, structured output, reasoning, output cap, timeout. Do not copy a requirements table forward. Language-server references on each wrapper and each exported chain → every task routing through it; re-run empty results, corroborate with text search. Empty is inconclusive. Task added since previous report appears here only. **Done when:** `Call Sites` row per site: task · wrapper · requirements · chain · `file:line` · how derived.