hstack 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,159 @@
1
+ ---
2
+ name: hstack-kernel-fit-triage
3
+ description: |
4
+ Use this skill when the engineer wants to triage an open kernel-fit finding — either acknowledge it (intent to act, not yet promoted) or dismiss it (won't pursue). The Skill is mechanical per ADR-0001: structured-elicitation loop, proposed-diff preview, Y/n confirmation, direct `Edit` writes, one atomic commit. No subagent invoked. Promotion to an ADR is a separate Skill (`/hstack:kernel-fit-promote`); this Skill never touches ADRs or change-specs. Examples:
5
+
6
+ <example>
7
+ Context: The engineer received a Slack nudge about KF-0001 (KF-P1, Category-A claim with production paths) and wants to acknowledge it for later promotion.
8
+ user: "/hstack:kernel-fit-triage KF-0001 --action acknowledge"
9
+ assistant: "I'll read KF-0001, print it in full, show the proposed-diff (status: open → acknowledged; owner: <git-handle>; updated: today; Triage Log entry appended), wait for Y/n, then Edit + commit `kernel-fit(KF-0001): acknowledge`. No reason is required for acknowledge. If you later decide to promote, run `/hstack:kernel-fit-promote KF-0001 --slug <adr-slug>`."
10
+ <commentary>
11
+ Acknowledge does not require a `--reason`. It is the "I've seen this, I intend to act, but I'm not promoting yet" state. The Triage Log records the transition. The finding stays at `status: acknowledged` until promote or dismiss happens.
12
+ </commentary>
13
+ </example>
14
+
15
+ <example>
16
+ Context: The engineer reviewed KF-0002 and decided the pattern is engineer-error, not a kernel gap.
17
+ user: "/hstack:kernel-fit-triage KF-0002 --action dismiss --reason \"The two changes flagged actually do have downstream consumers; the change-specs simply omitted the user-stories array because the engineer copied an old template. Not a kernel-flag issue, just stale boilerplate.\""
18
+ assistant: "I'll verify the reason is ≥50 characters (KF-05), read KF-0002, print the proposed-diff (status: open → dismissed; dismissed-reason: <text>; owner; updated; Triage Log entry), wait for Y/n, then commit `kernel-fit(KF-0002): dismiss`. Dismissal is terminal — re-running on this finding will surface that it's already dismissed and exit no-op."
19
+ <commentary>
20
+ Dismissal requires a reason of ≥50 characters per KF-05. The check is mechanical at validation time; the Skill enforces it at step 2. Drive-by dismissals are exactly the failure mode the rule exists to catch. The dismissed-reason is itself audit signal — a high rate of dismissals on a given pattern means either the pattern is poorly tuned or the engineer is gaming the gate.
21
+ </commentary>
22
+ </example>
23
+
24
+ <example>
25
+ Context: The engineer tries to dismiss a finding with too-short reason.
26
+ user: "/hstack:kernel-fit-triage KF-0003 --action dismiss --reason \"not relevant\""
27
+ assistant: "I'll halt — `dismissed-reason` is 12 characters; KF-05 requires ≥50. Drive-by dismissals defeat the audit signal. Either provide a more substantive reason or leave the finding at `open` (open is a legitimate long-term state when triage is genuinely deferred)."
28
+ <commentary>
29
+ Short-reason rejection is a hard halt. The Skill does not negotiate — write a real reason or leave the finding alone. Same shape as the deferral-check in `hstack-tech-debt-wontfix`.
30
+ </commentary>
31
+ </example>
32
+
33
+ tools:
34
+ - Read
35
+ - Write
36
+ - Edit
37
+ - Grep
38
+ - Glob
39
+ - Bash
40
+ - "{{TODO-SCRIPT: hstack/scripts/validate-spec.ts — validates the finding frontmatter flip and KF-05 (dismissed-reason ≥50 chars when status: dismissed)}}"
41
+ ---
42
+
43
+ ## Purpose
44
+
45
+ `hstack-kernel-fit-triage` is the mechanical status-flip Skill for kernel-fit findings. It moves a finding from `status: open` to either `acknowledged` (intent to promote, not yet) or `dismissed` (won't pursue, with mandatory rationale). Per ADR-0001, the Skill performs the `Edit` writes directly in the main session — no subagent invocation; the value to write is fully determined by the engineer's invocation arguments.
46
+
47
+ This Skill does not promote findings to ADRs. Promotion is a separate Skill (`/hstack:kernel-fit-promote`) with its own structured flow. This Skill also does not edit ADRs, change-specs, or any artifact outside `hstack/kernel-fit/findings/`.
48
+
49
+ ## When to invoke
50
+
51
+ Invoke when a finding is at `status: open` and the engineer wants to:
52
+
53
+ - **Acknowledge** (`--action acknowledge`): record intent to act on the finding without committing to an ADR yet. Useful when the finding is real but the team needs more thinking time, or when the right kernel-change shape is unclear.
54
+ - **Dismiss** (`--action dismiss --reason <text>`): close the finding without pursuing a kernel change. The reason is mandatory (≥50 characters per KF-05) and becomes part of the audit trail.
55
+
56
+ For findings already at `acknowledged`, re-invoking with `--action acknowledge` is a no-op; re-invoking with `--action dismiss --reason <text>` is permitted (acknowledged → dismissed is a valid transition when the engineer reconsiders). Findings at terminal status (`dismissed`, `promoted`, `superseded`, `archived`) are immutable from this Skill's perspective — the Skill halts.
57
+
58
+ ## Inputs
59
+
60
+ - `<finding-id>` (required, positional): the finding id, e.g. `KF-0001-category-a-claim-spans-production` or the short form `KF-0001` (the Skill resolves the latter to the matching file via glob).
61
+ - `--action <acknowledge | dismiss>` (required): the triage action.
62
+ - `--reason <text>` (required when `--action dismiss`; forbidden when `--action acknowledge`): the dismissal rationale. Must be ≥50 characters per KF-05.
63
+
64
+ ## Preconditions
65
+
66
+ - `hstack/kernel-fit/findings/<finding-id>*.md` exists. If missing, halt.
67
+ - The finding is at `status: open` (or `acknowledged` when transitioning to `dismissed`). If at any other status, halt with the current status named and the explanation that the status is terminal.
68
+ - `--action` is one of `acknowledge | dismiss` (controlled enum).
69
+ - When `--action dismiss`, `--reason` is non-empty and ≥50 characters.
70
+ - When `--action acknowledge`, `--reason` is absent (the Skill rejects redundant reasons to keep the audit signal clean — acknowledge reasons live in the next promote / dismiss invocation if needed).
71
+
72
+ ## Orchestration steps
73
+
74
+ 1. **Resolve the finding file.** Glob `hstack/kernel-fit/findings/<finding-id>*.md`. If zero matches, halt. If multiple matches (shouldn't happen with the immutable-id rule, but defense in depth), halt and ask the engineer to disambiguate.
75
+
76
+ 2. **Validate inputs against preconditions.** Walk the precondition checks above. On any failure, halt with the named reason. For the `--reason` length check (`dismiss`), surface the actual character count in the halt message so the engineer can size their next attempt.
77
+
78
+ 3. **Print the finding in full.** Read the resolved file and print its full body to the conversation. The engineer should re-read before committing to the triage action.
79
+
80
+ 4. **Compose the frontmatter edit.** Compute the exact frontmatter changes:
81
+
82
+ For `--action acknowledge`:
83
+ - `status: open → acknowledged`
84
+ - `owner: <git-handle>` (read from git config; fall back to the engineer's hstack config owner)
85
+ - `updated: <today>` (ISO date)
86
+
87
+ For `--action dismiss`:
88
+ - `status: open → dismissed` (or `acknowledged → dismissed`)
89
+ - `dismissed-reason: <text>` (the engineer's `--reason` verbatim)
90
+ - `owner: <git-handle>`
91
+ - `updated: <today>`
92
+
93
+ 5. **Compose the Triage Log append.** Compute the entry to append to the `## Triage Log` section:
94
+
95
+ For acknowledge:
96
+ ```
97
+ - `status: open → acknowledged` on <today> by <owner>. Triggered by `/hstack:kernel-fit-triage <id> --action acknowledge`.
98
+ ```
99
+
100
+ For dismiss:
101
+ ```
102
+ - `status: <prev> → dismissed` on <today> by <owner>. Reason: <dismissed-reason>.
103
+ ```
104
+
105
+ Defensive Triage Log check: if `## Triage Log` is not present in the file (legacy findings authored before the template included this section), append `\n## Triage Log\n` to the end of the file first.
106
+
107
+ 6. **Print the proposed diff.** Show the engineer the exact frontmatter changes and the exact Triage Log entry that will land. This is the kernel's mechanical-operations confirmation gate — until `validate-spec.ts` ships, the proposed-diff preview is the only contract check between the `Edit` and the commit.
108
+
109
+ 7. **Confirm.** Print "Apply triage to <finding-id>? (Y/n)". Default Yes. On `n`, abort without writing.
110
+
111
+ 8. **Edit + validate + commit.** On `Y`:
112
+ - `Edit` the file: frontmatter changes + Triage Log append.
113
+ - Run `{{TODO-SCRIPT: hstack/scripts/validate-spec.ts}}` against the file. KF-01 through KF-05 must pass; specifically KF-05 (`dismissed-reason` non-null and ≥50 chars when `status: dismissed`) gates dismissal.
114
+ - On validation pass: `git add` the file and commit with message `kernel-fit(<finding-id>): <action>` (e.g. `kernel-fit(KF-0001): acknowledge`).
115
+ - On validation failure: halt; revert via `git checkout -- <finding-file>`. Report the failing rule to the engineer.
116
+
117
+ 9. **Confirm completion.** Print "Triage applied: <finding-id> is now `<new-status>`. Next steps: `/hstack:kernel-fit-promote <finding-id> --slug <adr-slug>` to elevate to an ADR; or leave at <new-status> and let the next `/hstack:kernel-fit-scan` re-evaluate."
118
+
119
+ ## Outputs
120
+
121
+ - One edit to `hstack/kernel-fit/findings/<finding-id>*.md` — frontmatter flip + Triage Log append.
122
+ - One commit. Message: `kernel-fit(<finding-id>): <action>`.
123
+
124
+ ## Auto-commit triggers
125
+
126
+ - One commit at the status flip. The kernel's auto-commit-at-status-transition rule applies.
127
+
128
+ ## Idempotency contract
129
+
130
+ - Re-running with `--action acknowledge` on a finding already at `acknowledged`: no-op. The Skill prints "already acknowledged" and exits without writing.
131
+ - Re-running with `--action dismiss` on a finding already at `dismissed`: no-op. The Skill prints "already dismissed" + the existing `dismissed-reason` and exits without writing.
132
+ - Re-running with `--action dismiss` on a finding at `acknowledged`: permitted — the `acknowledged → dismissed` transition is valid. The Triage Log entry uses the `<prev> → dismissed` shape.
133
+ - Re-running on a finding at `promoted`, `superseded`, or `archived`: halt with the status named. These are terminal from this Skill's perspective.
134
+
135
+ ## Stop conditions
136
+
137
+ Beyond the kernel's general stop conditions:
138
+
139
+ - The finding does not exist or is at a terminal-from-triage status (`dismissed`, `promoted`, `superseded`, `archived`). Halt with the status named.
140
+ - `--action` is missing or not in the enum. Halt with usage.
141
+ - `--action dismiss` without `--reason`, or with `--reason` <50 characters. Halt with the actual length count.
142
+ - `--action acknowledge` with `--reason` present. Halt — acknowledge does not take a reason.
143
+ - The engineer declines confirmation at step 7. Abort cleanly.
144
+ - Validator fails at step 8 — halt with the failing rule; revert the unstaged edit.
145
+
146
+ ## Failure modes
147
+
148
+ - **Edit fails (filesystem, validator, git).** The frontmatter flip and the Triage Log append must land together in a single auto-commit. If `Edit` succeeds but `git add` or `git commit` fails, the working tree carries the unstaged change — revert via `git checkout -- <finding-file>` and re-invoke.
149
+ - **Drive-by dismissal attempt.** The ≥50-character check at step 2 is the v1 defense. Short reasons fail before any write occurs.
150
+ - **Stale finding (post-scan supersession in flight).** If a concurrent `/hstack:kernel-fit-scan` has just superseded the finding the engineer is triaging, the post-edit validator would catch the inconsistent state (superseded finding cannot be re-triaged). Halt and let the engineer re-fetch the working tree.
151
+
152
+ ## Anti-patterns
153
+
154
+ - Never invoke a subagent for triage. The action's value is fully determined by `--action` and `--reason`; the kernel's Mechanical operations section requires direct Skill writes for cases like this (saving ~25k subagent-context tokens per call).
155
+ - Never accept a dismissal reason shorter than 50 characters. The audit signal depends on substantive rationales — drive-bys defeat the loop.
156
+ - Never edit a finding's `pattern`, `evidence-rows`, `confidence`, or `evidence-row-count` fields. Those are the analyst's domain; triage only flips `status`, sets `owner` / `updated`, and writes `dismissed-reason` (when dismissing).
157
+ - Never edit a `promoted` finding's `promoted-to` field. That is owned by `/hstack:kernel-fit-promote` and is reciprocally bound to the target ADR.
158
+ - Never re-open a `dismissed` or `superseded` finding. If the kernel-fit pattern recurs with new evidence, the next `/hstack:kernel-fit-scan` will produce a new finding with a new id — that's the right path.
159
+ - Never invoke `spec-author` for the triage action. Per ADR-0001, the Skill writes directly.
@@ -31,15 +31,16 @@ tools:
31
31
 
32
32
  `hstack-telemetry` produces a retrospective observability report — a markdown file under `hstack/telemetry/reports/` — by parsing on-disk artifacts, git history, and Claude Code transcripts. The Skill is mechanical: it shells out to a Python script, reports the output path, and returns. No subagent is invoked. The kernel's "no parallel tracker" rule is preserved because the report is derivative — re-runnable from source, never authoritative.
33
33
 
34
- The report covers five buckets:
34
+ The report covers six buckets:
35
35
 
36
36
  1. **Token economics** — TE-1 cost-score per Skill, TE-2 cache-hit ratio per Skill, TE-3 subagent entry-tax amortization.
37
37
  2. **Workflow shape** — WS-1 phase duration, WS-2 gate findings density, WS-4 scope-amendment rate, WS-6 halt reasons.
38
38
  3. **Quality outcomes** — QO-2 severity × resolution-type mix, QO-3 test-immutability audit, QO-4 verifier observed-vs-promised.
39
39
  4. **Overengineering** — OE-1 artifact tokens per diff line, OE-3 subagent invocations × host cost, OE-5 trivial-eligible changes that ran the full gauntlet.
40
40
  5. **Contract drift** — module-spec staleness × recent commit activity, ADR supersession lag, tech-debt half-life by exit path.
41
+ 6. **Kernel-fit candidates** — KF-P1 Category-A claim spans production paths (post-PR-#5 misclassification), KF-P2 halt-reason cluster, KF-P3 missed-gate recovery. Detection-only rollup; the canonical findings live at `hstack/kernel-fit/findings/` and are produced by `/hstack:kernel-fit-scan`. See ADR-0004.
41
42
 
42
- A watch-list at the report bottom surfaces anomalies (low cache-hit Skills, high-severity findings resolved as `justified-in-prose`, candidate test-immutability violations, scope-amendment rate above 30%, module drift).
43
+ A watch-list at the report bottom surfaces anomalies (low cache-hit Skills, high-severity findings resolved as `justified-in-prose`, candidate test-immutability violations, scope-amendment rate above 30%, module drift, fired kernel-fit patterns).
43
44
 
44
45
  ## When to invoke
45
46
 
@@ -321,6 +321,7 @@ Load-at-session-start rules by subagent:
321
321
  - `implementer`: change-spec, plan, test-plan, security-review, data-review and ui-brief and figma-handoff when present, tech-stack, infrastructure (when surfaces includes infra).
322
322
  - `verifier`: change-spec, plan, test-plan, ci-cd.
323
323
  - `adversarial-reviewer`: all change artifacts (including test-plan); explicitly no implementer transcripts.
324
+ - `kernel-fit-analyst`: hstack/CLAUDE.md (the artifact under analysis), the latest hstack/telemetry/reports/<date>.md, every prior finding at hstack/kernel-fit/findings/, all change-specs at status: shipped (full bodies), all ADRs, all tech-debt, all module-specs; explicitly no implementer transcripts and no scratchpads from in-flight authoring sessions.
324
325
  - `researcher`: query context plus relevant product-context docs as the query requires.
325
326
 
326
327
  A subagent that cannot reach a required context document halts and asks the human, rather than proceeding without it.
@@ -390,6 +391,24 @@ When this kernel is loaded in a session that is adding or removing a Skill or su
390
391
 
391
392
  ---
392
393
 
394
+ ## How hstack improves itself
395
+
396
+ hstack ships a closed-loop system for detecting when the kernel itself — this file, the templates, the validators, the Skill flows — is misaligned with how engineers and AI agents actually use it. The loop has five layers and one non-negotiable contract: **the human gates promotion to a kernel change.** Detection and synthesis can be automated; the decision to amend the kernel cannot.
397
+
398
+ - **Detection is post-hoc and derivative.** `hstack/scripts/telemetry/insights/kernel_fit.py` pattern-matches across shipped change-specs, ADRs, tech-debt, halt sentinels, and adversarial-review findings. Every detection is reconstructible from git + frontmatter; the no-parallel-tracker rule is preserved because the detector reads, never writes.
399
+
400
+ - **Synthesis is delegated to the `kernel-fit-analyst` subagent.** Model `opus`, loads the kernel and all shipped artifacts and every prior finding, explicitly *not* implementer transcripts (same session-isolation rule as `adversarial-reviewer`). The analyst produces one finding file per pattern at `hstack/kernel-fit/findings/KF-NNNN-<slug>.md`, with a mandatory two-bullet counter-explanation challenge prompt that defends against false-positives. Findings carry a `confidence` enum and a `status` lifecycle (`open → acknowledged → promoted` for actionable findings; `open → dismissed` for non-actionable; `open → superseded` for restated findings).
401
+
402
+ - **Three Skills drive the lifecycle.** `/hstack:kernel-fit-scan` runs detection + synthesis + Slack nudge. `/hstack:kernel-fit-triage <id> --action acknowledge|dismiss --reason <text>` is a mechanical status flip per ADR-0001. `/hstack:kernel-fit-promote <id> --slug <adr-slug>` routes to `/hstack:adr-new --from-kernel-fit <id>`, mirroring the `--from-research` pattern already in use by `/hstack:research --promote`. The ADR's Context section is seeded from the finding's Evidence + Kernel Surface + Proposed Direction; `spec-author` runs the normal Nygard interview — this is the human gate. The reciprocal `promoted-to` write on the finding lands atomically with the ADR commit per the kernel's reciprocal-pair atomicity rule.
403
+
404
+ - **Notification is best-effort via Slack MCP.** Threshold-gated (notify on `high` and `medium` confidence only; `low` lands silently on disk) and de-duplicated (no re-notification on an open pattern within a 14-day window). Graceful degradation: if the Slack MCP is unreachable or unwired, findings still land on disk; the Skill logs to stderr and exits 0. This is a deliberate carve-out from the kernel's general MCP-unreachable stop condition — the disk write is load-bearing, Slack is a side-channel pointer, not authoritative state.
405
+
406
+ - **The analyst never writes ADRs, change-specs, or edits existing findings** (one carve-out: it may set `status: superseded` on a prior finding when restating it more cleanly). Promotion is engineer-initiated and routes through the established authoring Skills. Auto-creation of ADRs is forbidden — the kernel's "AI writes, humans confirm" contract applies most forcefully at the kernel-modification layer, where the cost of a bad ADR cascades through every subsequent change.
407
+
408
+ The loop is the smallest expression of the kernel reasoning about itself without auto-modifying itself. v1 honesty: the analyst's output is an LLM-strategized judgment, not measured truth; the counter-explanation challenge is the false-positive mitigation. Same framing rule as `test-strategist` and `security-reviewer`. See ADR-0004 for the rationale; see `template/templates/kernel-fit-finding.md` for the artifact schema.
409
+
410
+ ---
411
+
393
412
  ## References
394
413
 
395
414
  - Architecture document (long-form companion): https://www.notion.so/360d6791656c813d955af822cb8814d1
@@ -0,0 +1,438 @@
1
+ """Kernel-fit insights: patterns suggesting the kernel itself needs revision.
2
+
3
+ This module is the detection layer of the kernel-fit closed-loop system. It
4
+ pattern-matches across shipped artifacts and emits evidence rows; an LLM
5
+ subagent (`kernel-fit-analyst`) then synthesizes findings from these rows.
6
+
7
+ See ADR-0004 for the full design rationale and `template/CLAUDE.md` § How
8
+ hstack improves itself for the loop contract.
9
+
10
+ Three starter patterns:
11
+
12
+ - KF-P1 — `category-a-claim-spans-production-paths`: changes flagged
13
+ `internal-tooling: true` (Category A — engineering-only) whose `in-scope`
14
+ touches production-code paths AND whose `enables` array is empty.
15
+ Under the post-PR-#5 schema (`enables` ↔ `enabled-by`, SP-13/SP-14),
16
+ this is the engineer mis-classifying what should be Category B
17
+ (foundational prerequisite) as Category A. The in-scope-overlap
18
+ heuristic surfaces candidate downstream consumers as evidence.
19
+ - KF-P2 — `halt-reason-cluster-uncovered-by-enum`: halt sentinels with
20
+ `reason=other` whose surrounding prose clusters above the Jaccard
21
+ threshold, suggesting the enum is missing a case.
22
+ - KF-P3 — `skill-precondition-violated-and-recoverable`: adversarial-review
23
+ spec-compliance findings whose resolution commit messages reveal a missed
24
+ upstream gate (the ADR-0002 pattern).
25
+
26
+ Detection is pure read — no writes. Output is a dict consumed by the
27
+ analyst subagent via the scan Skill orchestration.
28
+ """
29
+
30
+ from __future__ import annotations
31
+
32
+ import re
33
+ from collections import defaultdict
34
+ from pathlib import Path
35
+
36
+ from telemetry.parsers import frontmatter as fm_parser
37
+ from telemetry.parsers.bodies import parse_findings_section, split_sections
38
+
39
+
40
+ # File-path prefixes that count as "internal-only" for KF-P1 classification.
41
+ # A change whose in-scope is entirely under these prefixes is genuine
42
+ # Category A (true internal tooling). Anything outside is candidate Category B
43
+ # (foundational prerequisite — production code with deferred user value).
44
+ INTERNAL_ONLY_PREFIXES = (
45
+ "hstack/",
46
+ "scripts/",
47
+ ".github/",
48
+ "template/",
49
+ ".claude/",
50
+ "docs/",
51
+ "ci/",
52
+ )
53
+
54
+ # Kernel-rule keywords scanned in resolution commits for KF-P3. A
55
+ # `spec-compliance` adversarial finding whose resolving commit mentions any
56
+ # of these is a candidate "Skill precondition should have halted earlier"
57
+ # signal (the ADR-0002 missing-gate pattern).
58
+ KERNEL_GATE_KEYWORDS = re.compile(
59
+ r"\b(precondition|missed\s+gate|should\s+have\s+halted|upstream|"
60
+ r"ready-for-implementation|ready-for-review|ready-to-ship|status\s+gate)\b",
61
+ re.IGNORECASE,
62
+ )
63
+
64
+ # Jaccard threshold for KF-P2 cluster membership. Tunable; documented in the
65
+ # plan as a starting value. Lower → more clustering (more cluster merges,
66
+ # fewer clusters). Higher → tighter clusters (fewer merges, more clusters).
67
+ JACCARD_THRESHOLD = 0.6
68
+
69
+ # Minimum cluster size for KF-P2 to fire. Smaller than this is noise.
70
+ MIN_CLUSTER_SIZE = 3
71
+
72
+ # Minimum candidate-row count for KF-P1 to fire. A single Category-B mislabel
73
+ # is noise; recurrence is signal.
74
+ KF_P1_MIN_ROWS = 2
75
+
76
+
77
+ def compute(commits: list[dict], changes: dict, tech_debt: list[dict],
78
+ adrs: list[dict], module_specs: list[dict],
79
+ session_rows: list[dict], findings_dir: Path | None) -> dict:
80
+ """Run all kernel-fit detection patterns.
81
+
82
+ `findings_dir` may be `None` or non-existent on first run — the dedup
83
+ cross-reference returns an empty index in that case, and the analyst
84
+ treats every fired pattern as net-new.
85
+ """
86
+ existing = _load_existing_findings(findings_dir)
87
+ return {
88
+ "existing_open_findings_by_pattern": existing,
89
+ "kf_p1_category_a_claim_spans_production_paths": _kf_p1(changes, commits),
90
+ "kf_p2_halt_reason_cluster_uncovered_by_enum": _kf_p2(commits, session_rows),
91
+ "kf_p3_skill_precondition_violated_and_recoverable": _kf_p3(changes, commits),
92
+ }
93
+
94
+
95
+ # ---------------- existing-findings index ----------------
96
+
97
+ def _load_existing_findings(findings_dir: Path | None) -> dict:
98
+ """Read all KF-NNNN-*.md files in findings_dir and build an index of open
99
+ findings keyed by pattern. Used by the analyst for dedup / supersession
100
+ decisions. Tolerant of missing directory."""
101
+ out: dict[str, list[str]] = defaultdict(list)
102
+ if findings_dir is None or not findings_dir.is_dir():
103
+ return dict(out)
104
+ for path in sorted(findings_dir.glob("KF-*.md")):
105
+ parsed = fm_parser.read_artifact(path)
106
+ if parsed is None:
107
+ continue
108
+ fm, _body = parsed
109
+ status = fm.get("status")
110
+ pattern = fm.get("pattern")
111
+ kid = fm.get("id") or path.stem
112
+ if not pattern:
113
+ continue
114
+ # Only "open" and "acknowledged" findings count for dedup; promoted /
115
+ # dismissed / superseded / archived are terminal and do not suppress
116
+ # re-detection.
117
+ if status in ("open", "acknowledged"):
118
+ out[pattern].append(kid)
119
+ return dict(out)
120
+
121
+
122
+ # ---------------- KF-P1 ----------------
123
+
124
+ def _classify_inscope_paths(in_scope: list) -> tuple[list[str], list[str]]:
125
+ """Partition an in-scope list into (internal_only_paths, production_paths)."""
126
+ internal_only: list[str] = []
127
+ production: list[str] = []
128
+ for entry in in_scope or []:
129
+ if not isinstance(entry, str):
130
+ continue
131
+ path = entry.strip()
132
+ if not path:
133
+ continue
134
+ # Normalize leading "./" and any glob suffixes for prefix checking.
135
+ normalized = path[2:] if path.startswith("./") else path
136
+ if any(normalized.startswith(p) for p in INTERNAL_ONLY_PREFIXES):
137
+ internal_only.append(path)
138
+ else:
139
+ production.append(path)
140
+ return internal_only, production
141
+
142
+
143
+ def _forward_consumers(this_change_id: str, this_in_scope: list[str],
144
+ changes: dict, commits: list[dict]) -> list[str]:
145
+ """Return change-ids of later changes whose in-scope OR whose commit
146
+ file-lists overlap with this change's in-scope production paths."""
147
+ if not this_in_scope:
148
+ return []
149
+ # Build set of production-path-prefixes for cheap overlap checks. We treat
150
+ # each in-scope entry as a prefix; this is forgiving (catches edits inside
151
+ # subdirs) and matches what `internal-tooling: true` plumbing changes
152
+ # typically introduce (a dir of new types or a new module).
153
+ prefixes = {(p[2:] if p.startswith("./") else p).rstrip("/*") for p in this_in_scope}
154
+
155
+ consumers: set[str] = set()
156
+
157
+ # (1) Other change-specs whose in-scope overlaps.
158
+ for other_id, arts in changes.items():
159
+ if other_id == this_change_id:
160
+ continue
161
+ spec = arts.get("change-spec") or arts.get("spec")
162
+ if not spec:
163
+ continue
164
+ other_in_scope = spec["fm"].get("in-scope") or []
165
+ for entry in other_in_scope:
166
+ if not isinstance(entry, str):
167
+ continue
168
+ normalized = entry[2:] if entry.startswith("./") else entry
169
+ if any(normalized.startswith(p) for p in prefixes):
170
+ consumers.add(other_id)
171
+ break
172
+
173
+ # (2) Commits whose file-list touches our production paths AND whose
174
+ # artifact_id is a different change-spec (avoids self-attribution).
175
+ for c in commits:
176
+ cid = c.get("artifact_id")
177
+ if not cid or cid == this_change_id:
178
+ continue
179
+ for f in c.get("files", []):
180
+ normalized = f[2:] if f.startswith("./") else f
181
+ if any(normalized.startswith(p) for p in prefixes):
182
+ consumers.add(cid)
183
+ break
184
+
185
+ return sorted(consumers)
186
+
187
+
188
+ def _kf_p1(changes: dict, commits: list[dict]) -> dict:
189
+ """KF-P1 — Category A (`internal-tooling: true`) claims whose `in-scope`
190
+ spans production-code paths AND whose `enables` array is empty. Under
191
+ the post-PR-#5 schema, this is the engineer mis-classifying what should
192
+ be Category B (foundational prerequisite) as Category A. SP-13 makes
193
+ A and B mutually exclusive at the validator level; KF-P1 catches the
194
+ case the validator cannot — claiming A when the in-scope reveals B.
195
+ Fires on >= KF_P1_MIN_ROWS candidate rows.
196
+ """
197
+ rows: list[dict] = []
198
+ for cid, arts in changes.items():
199
+ spec = arts.get("change-spec") or arts.get("spec")
200
+ if not spec:
201
+ continue
202
+ fm = spec["fm"]
203
+ if fm.get("status") != "shipped":
204
+ continue
205
+ if not fm.get("internal-tooling"):
206
+ continue
207
+ in_scope = fm.get("in-scope") or []
208
+ enables = fm.get("enables") or []
209
+ internal_paths, production_paths = _classify_inscope_paths(in_scope)
210
+ # Classification (post-PR-#5 schema; SP-13 enforces mutual exclusivity):
211
+ # - no production paths → "true-category-a" (correctly classified)
212
+ # - has production paths AND enables empty → "category-b-misclassified" (bug)
213
+ # - has production paths AND enables non-empty → impossible under SP-13;
214
+ # if observed the validator failed and the analyst surfaces it separately
215
+ if not production_paths:
216
+ classification = "true-category-a"
217
+ elif not enables:
218
+ classification = "category-b-misclassified"
219
+ else:
220
+ # SP-13 violation should not reach here in a validated repo; flag
221
+ # explicitly so the analyst can route to a validator-bug finding.
222
+ classification = "sp-13-violation"
223
+ consumers: list[str] = []
224
+ if classification == "category-b-misclassified":
225
+ consumers = _forward_consumers(cid, production_paths or in_scope, changes, commits)
226
+ rows.append({
227
+ "change": cid,
228
+ "internal_only_paths_count": len(internal_paths),
229
+ "production_paths_count": len(production_paths),
230
+ "enables_count": len(enables),
231
+ "downstream_consumers": consumers,
232
+ "classification_candidate": classification,
233
+ })
234
+
235
+ candidate_rows = [r for r in rows
236
+ if r["classification_candidate"] == "category-b-misclassified"]
237
+ fired = len(candidate_rows) >= KF_P1_MIN_ROWS
238
+ return {
239
+ "pattern_id": "KF-P1",
240
+ "pattern_name": "category-a-claim-spans-production-paths",
241
+ "fired": fired,
242
+ "evidence_row_count": len(candidate_rows),
243
+ "min_rows_for_firing": KF_P1_MIN_ROWS,
244
+ "all_rows": rows,
245
+ "evidence_rows": candidate_rows,
246
+ "note": ("Changes flagged `internal-tooling: true` (Category A) whose in-scope "
247
+ "spans production-code paths AND whose `enables` array is empty. Under "
248
+ "the post-PR-#5 schema (Category A vs Category B with `enables`/`enabled-by`), "
249
+ "this is the engineer mis-classifying what should be Category B as Category A. "
250
+ "SP-13 catches the both-set case; KF-P1 catches the claim-A-while-looking-like-B case."),
251
+ }
252
+
253
+
254
+ # ---------------- KF-P2 ----------------
255
+
256
+ _TOKEN_RE = re.compile(r"[a-zA-Z][a-zA-Z0-9_-]{2,}")
257
+
258
+
259
+ def _tokens(text: str) -> set[str]:
260
+ """Lowercase token set from text; drops tokens <=3 chars and pure numbers."""
261
+ if not text:
262
+ return set()
263
+ return {t.lower() for t in _TOKEN_RE.findall(text)}
264
+
265
+
266
+ def _jaccard(a: set[str], b: set[str]) -> float:
267
+ if not a or not b:
268
+ return 0.0
269
+ inter = len(a & b)
270
+ union = len(a | b)
271
+ return inter / union if union else 0.0
272
+
273
+
274
+ def _kf_p2(commits: list[dict], session_rows: list[dict]) -> dict:
275
+ """KF-P2 — cluster halt sentinels with reason=other. Cluster size >=
276
+ MIN_CLUSTER_SIZE is evidence the enum is missing a case.
277
+
278
+ Sources: commit bodies (parser already extracted halt_reasons), and
279
+ session-row halt_reasons. For commits, we use the commit body as the
280
+ surrounding-prose context; for session rows we use the row's halt-context
281
+ if available, falling back to a label-only token set.
282
+ """
283
+ docs: list[dict] = []
284
+ for c in commits:
285
+ reasons = c.get("halt_reasons") or []
286
+ if not any(r.lower() == "other" for r in reasons):
287
+ continue
288
+ # Use commit body as the prose context — it is what the kernel
289
+ # contract says accompanies the sentinel.
290
+ context = c.get("body", "") or c.get("subject", "")
291
+ docs.append({
292
+ "source": "commit",
293
+ "ref": c.get("sha", "")[:8],
294
+ "context": context,
295
+ "tokens": _tokens(context),
296
+ })
297
+ for s in session_rows:
298
+ reasons = s.get("halt_reasons") or []
299
+ for r in reasons:
300
+ if not isinstance(r, str):
301
+ continue
302
+ if r.lower() != "other":
303
+ continue
304
+ docs.append({
305
+ "source": "session",
306
+ "ref": s.get("file", "") or s.get("skill", "") or "",
307
+ # Session parser does not capture surrounding prose in v1;
308
+ # use the session label as a degraded token source.
309
+ "context": s.get("skill", "") + " " + " ".join(reasons),
310
+ "tokens": _tokens(s.get("skill", "")),
311
+ })
312
+
313
+ # Greedy clustering: each new doc joins the first cluster whose
314
+ # representative has Jaccard >= threshold; else starts a new cluster.
315
+ clusters: list[list[dict]] = []
316
+ for d in docs:
317
+ placed = False
318
+ for cluster in clusters:
319
+ rep_tokens = cluster[0]["tokens"]
320
+ if _jaccard(d["tokens"], rep_tokens) >= JACCARD_THRESHOLD:
321
+ cluster.append(d)
322
+ placed = True
323
+ break
324
+ if not placed:
325
+ clusters.append([d])
326
+
327
+ evidence_clusters = [c for c in clusters if len(c) >= MIN_CLUSTER_SIZE]
328
+ rows = []
329
+ for i, cluster in enumerate(evidence_clusters):
330
+ rows.append({
331
+ "cluster_id": f"C-{i + 1}",
332
+ "size": len(cluster),
333
+ "representative_context": (cluster[0]["context"] or "")[:300],
334
+ "member_refs": [d["ref"] for d in cluster],
335
+ })
336
+
337
+ return {
338
+ "pattern_id": "KF-P2",
339
+ "pattern_name": "halt-reason-cluster-uncovered-by-enum",
340
+ "fired": len(evidence_clusters) > 0,
341
+ "evidence_row_count": len(evidence_clusters),
342
+ "jaccard_threshold": JACCARD_THRESHOLD,
343
+ "min_cluster_size": MIN_CLUSTER_SIZE,
344
+ "total_other_halts": len(docs),
345
+ "evidence_rows": rows,
346
+ "note": ("HSTACK-HALT sentinels with reason=other clustered by surrounding-prose "
347
+ f"token overlap (Jaccard >= {JACCARD_THRESHOLD}). A cluster of "
348
+ f">= {MIN_CLUSTER_SIZE} similar halts means the enum is missing a case."),
349
+ }
350
+
351
+
352
+ # ---------------- KF-P3 ----------------
353
+
354
+ def _commits_by_sha_prefix(commits: list[dict]) -> dict[str, dict]:
355
+ """Index commits by short sha (8 chars) for quick lookup. Falls back to
356
+ full sha if entries collide (rare with realistic repo sizes)."""
357
+ out: dict[str, dict] = {}
358
+ for c in commits:
359
+ sha = c.get("sha") or ""
360
+ if not sha:
361
+ continue
362
+ out[sha] = c
363
+ if len(sha) >= 8:
364
+ out[sha[:8]] = c
365
+ return out
366
+
367
+
368
+ def _kf_p3(changes: dict, commits: list[dict]) -> dict:
369
+ """KF-P3 — spec-compliance adversarial findings resolved via a commit
370
+ whose message reveals a kernel-gate keyword. This is the pattern that
371
+ produced ADR-0002 (the missed `ready-for-review` transition).
372
+ """
373
+ commit_index = _commits_by_sha_prefix(commits)
374
+ rows: list[dict] = []
375
+ for cid, arts in changes.items():
376
+ ar = arts.get("adversarial-review")
377
+ if not ar:
378
+ continue
379
+ ar_fm = ar["fm"]
380
+ if ar_fm.get("status") != "findings-resolved":
381
+ continue
382
+ # Findings array on frontmatter is authoritative; fall back to body
383
+ # parser when the array is missing or absent.
384
+ findings = ar_fm.get("findings") or []
385
+ if not findings:
386
+ sections = split_sections(ar["body"] or "")
387
+ findings_section = sections.get("Findings", "")
388
+ findings = parse_findings_section(findings_section)
389
+ for f in findings:
390
+ if not isinstance(f, dict):
391
+ continue
392
+ category = (f.get("category") or "").lower()
393
+ if category != "spec-compliance":
394
+ continue
395
+ resolution = (f.get("resolution") or "").strip()
396
+ if not resolution.startswith("commit:"):
397
+ continue
398
+ sha_token = resolution.split(":", 1)[1].strip().split()[0]
399
+ commit = commit_index.get(sha_token) or commit_index.get(sha_token[:8])
400
+ if not commit:
401
+ # Still record the candidate — the analyst can decide whether
402
+ # missing-commit-context is itself a signal.
403
+ if KERNEL_GATE_KEYWORDS.search(resolution):
404
+ rows.append({
405
+ "change": cid,
406
+ "finding_id": f.get("id"),
407
+ "category": category,
408
+ "resolution": resolution,
409
+ "commit_subject": None,
410
+ "matched_keywords": [],
411
+ "commit_resolved": False,
412
+ })
413
+ continue
414
+ haystack = (commit.get("subject") or "") + "\n" + (commit.get("body") or "")
415
+ matches = KERNEL_GATE_KEYWORDS.findall(haystack)
416
+ if not matches:
417
+ continue
418
+ rows.append({
419
+ "change": cid,
420
+ "finding_id": f.get("id"),
421
+ "category": category,
422
+ "resolution": resolution,
423
+ "commit_subject": commit.get("subject"),
424
+ "matched_keywords": [m.lower() for m in matches],
425
+ "commit_resolved": True,
426
+ })
427
+
428
+ return {
429
+ "pattern_id": "KF-P3",
430
+ "pattern_name": "skill-precondition-violated-and-recoverable",
431
+ "fired": len(rows) >= 1,
432
+ "evidence_row_count": len(rows),
433
+ "evidence_rows": rows,
434
+ "note": ("Adversarial spec-compliance findings whose resolving commit message "
435
+ "mentions a kernel gate or precondition. Each row is a candidate "
436
+ "'a Skill precondition should have halted earlier' signal — the "
437
+ "ADR-0002 pattern."),
438
+ }