@lifeaitools/rdc-skills 0.24.41 → 0.25.0

package/skills/rdc-extract-verifier-rules/SKILL.md

@@ -1,191 +1,191 @@
----
-name: rdc-extract-verifier-rules
-version: 0.1.0
-description: |
-  Read recent enhancement-log entries, cluster failures by pattern, generate candidate verifier rules, test them against the known-good corpus and the failure corpus, and propose pull requests adding the highest-confidence rules to forbidden-patterns.json. Use this skill on a nightly cadence (3 AM PT), or manually when the user says "extract verifier rules", "promote enhancement log", "what new rules should we add", or after a significant brochure run produced many failures.
-triggers:
-  - "rdc:extract-verifier-rules"
-  - "extract verifier rules"
-  - "promote enhancement log"
-  - "what new rules should we add"
-  - "verifier corpus update"
-  - nightly cron at 3:00 AM PT
----
-
-# rdc:extract-verifier-rules
-> **⚠️ OUTPUT CONTRACT (READ FIRST):** `guides/output-contract.md`
-> Return candidate rules, evidence, and PR status directly; do not dump raw tool logs.
-
-The self-learning loop. The verifier corpus is the moat (per `DECISIONS-LOG.md` D-009). This skill is how the corpus grows.
-
-## What it does
-
-1. Read `enhancement-log.jsonl` entries since last run timestamp
-2. Cluster entries by fingerprint similarity + reason text
-3. For each cluster with `N >= 3` occurrences:
-   - Generate a candidate regex or AST pattern
-   - Test the pattern against the **known-good corpus** (last 100 successful brochures) for false positives
-   - Test the pattern against the **failure cluster** for recall
-   - If false-positive rate = 0 AND recall ≥ 80%, mark as `propose`
-4. For each `propose` candidate:
-   - Open a PR on `regen-root` adding the rule to `verifiers/forbidden-patterns.json`
-   - Auto-merge if confidence ≥ 95% AND a maintainer's auto-merge flag is enabled
-   - Otherwise, await human review
-5. Write a markdown summary to `.rdc/reports/verifier-{YYYY-MM-DD}.md`
-
-## Cluster algorithm
-
-Two-stage clustering:
-
-**Stage 1 — Fingerprint clustering:**
-- Each enhancement-log entry has an `input_fingerprint` (sha256 of the offending JSX block)
-- Entries with identical fingerprints cluster trivially (same exact mistake)
-
-**Stage 2 — Semantic clustering:**
-- For unique fingerprints, embed the `reason` and `pattern` fields with `@xenova/transformers` (384-dim, local — no API)
-- Cosine similarity ≥ 0.85 → same cluster
-- Use HDBSCAN with min_samples=3 to identify real clusters vs. noise
-
-## Candidate rule generation
-
-For each cluster, generate one of:
-
-### Regex candidate
-- Take all `pattern` strings in the cluster
-- Find common substring or template
-- Generalize literal values to character classes
-- Validate the regex is well-formed and not catastrophic
-
-Example:
-```
-Cluster patterns:
-  <div className="flex flex-col gap-4">
-  <div className="flex flex-col gap-6">
-  <div className="flex flex-row items-center">
-
-Generated regex:
-  <div[^>]*className="[^"]*\\bflex\\b
-```
-
-### AST candidate
-- For cluster entries with rich AST context, build an AST query string
-- Test with @typescript-eslint/parser
-
-Example:
-```
-AST candidate:
-  JSXOpeningElement[name.name='div'] > JSXAttribute[name.name='className'] CallExpression[callee.name='clsx']
-```
-
-## False-positive check
-
-Run the candidate against the known-good corpus:
-- The 12 Phase 1 reference brochures
-- All design-partner brochures with `final_grade >= 85`
-- Any brochure tagged `corpus:reference` in Supabase
-
-If the candidate matches any known-good output → reject the candidate. False positives in the verifier corpus are unacceptable because they break working flows.
-
-## Recall check
-
-Run the candidate against the failure cluster:
-- Must match ≥ 80% of cluster entries
-- Below 80%, the candidate is too specific; widen and retry
-
-## PR generation
-
-For each promoted candidate, create a PR on `LIFEAI/regen-root`:
-
-```
-Title: [verifier-corpus] Add rule FP0{N} — {short description}
-
-Body:
-This rule was generated by rdc:extract-verifier-rules on {date}.
-
-**Cluster size:** {N} occurrences over {time span}
-**False positive rate:** {rate}
-**Recall:** {percent}
-
-**Sample failures:**
-{3-5 example enhancement-log entries}
-
-**Generated rule:**
-```json
-{rule JSON}
-```
-
-**Reviewer checklist:**
-- [ ] Verify the rule doesn't match any known-good brochure
-- [ ] Verify the suggested fix is reasonable
-- [ ] Confirm severity level
-- [ ] Confirm category
-
-Auto-merging: {yes/no based on confidence}
-```
-
-## Auto-merge policy
-
-Auto-merge if **all** are true:
-- False-positive rate = 0
-- Recall ≥ 95%
-- Cluster size ≥ 10
-- The rule pattern is a strict subset of an existing pattern (not a fundamentally new category)
-- A maintainer's auto-merge flag is enabled in `.rdc/config.json`
-
-Otherwise, await human review.
-
-## Output: nightly report
-
-`.rdc/reports/verifier-{YYYY-MM-DD}.md`:
-
-```markdown
-# Verifier Corpus Update — {date}
-
-## Summary
-
-- Enhancement log entries since last run: {N}
-- Clusters identified: {C}
-- Rules proposed: {R}
-- Rules auto-merged: {A}
-- Rules awaiting review: {W}
-
-## Top clusters
-
-| Cluster | Size | Pattern | Status |
-|---|---|---|---|
-| ...
-
-## Open PRs
-
-- #{N}: FP0{X} — {description}
-- ...
-
-## False-positive rejections
-
-Candidates that failed the known-good test:
-- {summary}
-
-## Corpus growth
-
-- Total rules in forbidden-patterns.json: {before} → {after}
-- Total component-allowlist entries: unchanged
-- Pagination rules: unchanged
-```
-
-## Manual invocation
-
-A maintainer can run this skill manually after a significant brochure run to immediately promote lessons learned:
-
-```
-rdc:extract-verifier-rules --since "2026-05-27" --auto-merge=false
-```
-
-The `--auto-merge=false` flag forces human review on all proposals regardless of confidence.
-
-## Why this matters
-
-The verifier corpus is the asset. Anyone can copy the kit. Anyone can copy the ESLint plugin. **Replicating 12 months of customer-tested failure patterns is the moat.** This skill is how that moat compounds. Every brochure run feeds it. Every nightly run grows it.
-
-After 12 months at moderate scale, the corpus will contain ~500-2,000 rules, calibrated against real customer documents, with each rule's hit count visible. New competitors entering this category will face a starting position 12 months behind.
-
-That is the point.
+---
+name: rdc-extract-verifier-rules
+version: 0.1.0
+description: |
+  Read recent enhancement-log entries, cluster failures by pattern, generate candidate verifier rules, test them against the known-good corpus and the failure corpus, and propose pull requests adding the highest-confidence rules to forbidden-patterns.json. Use this skill on a nightly cadence (3 AM PT), or manually when the user says "extract verifier rules", "promote enhancement log", "what new rules should we add", or after a significant brochure run produced many failures.
+triggers:
+  - "rdc:extract-verifier-rules"
+  - "extract verifier rules"
+  - "promote enhancement log"
+  - "what new rules should we add"
+  - "verifier corpus update"
+  - nightly cron at 3:00 AM PT
+---
+
+# rdc:extract-verifier-rules
+> **⚠️ OUTPUT CONTRACT (READ FIRST):** `guides/output-contract.md`
+> Return candidate rules, evidence, and PR status directly; do not dump raw tool logs.
+
+The self-learning loop. The verifier corpus is the moat (per `DECISIONS-LOG.md` D-009). This skill is how the corpus grows.
+
+## What it does
+
+1. Read `enhancement-log.jsonl` entries since last run timestamp
+2. Cluster entries by fingerprint similarity + reason text
+3. For each cluster with `N >= 3` occurrences:
+   - Generate a candidate regex or AST pattern
+   - Test the pattern against the **known-good corpus** (last 100 successful brochures) for false positives
+   - Test the pattern against the **failure cluster** for recall
+   - If false-positive rate = 0 AND recall ≥ 80%, mark as `propose`
+4. For each `propose` candidate:
+   - Open a PR on `regen-root` adding the rule to `verifiers/forbidden-patterns.json`
+   - Auto-merge if confidence ≥ 95% AND a maintainer's auto-merge flag is enabled
+   - Otherwise, await human review
+5. Write a markdown summary to `.rdc/reports/verifier-{YYYY-MM-DD}.md`
+
+## Cluster algorithm
+
+Two-stage clustering:
+
+**Stage 1 — Fingerprint clustering:**
+- Each enhancement-log entry has an `input_fingerprint` (sha256 of the offending JSX block)
+- Entries with identical fingerprints cluster trivially (same exact mistake)
+
+**Stage 2 — Semantic clustering:**
+- For unique fingerprints, embed the `reason` and `pattern` fields with `@xenova/transformers` (384-dim, local — no API)
+- Cosine similarity ≥ 0.85 → same cluster
+- Use HDBSCAN with min_samples=3 to identify real clusters vs. noise
+
+## Candidate rule generation
+
+For each cluster, generate one of:
+
+### Regex candidate
+- Take all `pattern` strings in the cluster
+- Find common substring or template
+- Generalize literal values to character classes
+- Validate the regex is well-formed and not catastrophic
+
+Example:
+```
+Cluster patterns:
+  <div className="flex flex-col gap-4">
+  <div className="flex flex-col gap-6">
+  <div className="flex flex-row items-center">
+
+Generated regex:
+  <div[^>]*className="[^"]*\\bflex\\b
+```
+
+### AST candidate
+- For cluster entries with rich AST context, build an AST query string
+- Test with @typescript-eslint/parser
+
+Example:
+```
+AST candidate:
+  JSXOpeningElement[name.name='div'] > JSXAttribute[name.name='className'] CallExpression[callee.name='clsx']
+```
+
+## False-positive check
+
+Run the candidate against the known-good corpus:
+- The 12 Phase 1 reference brochures
+- All design-partner brochures with `final_grade >= 85`
+- Any brochure tagged `corpus:reference` in Supabase
+
+If the candidate matches any known-good output → reject the candidate. False positives in the verifier corpus are unacceptable because they break working flows.
+
+## Recall check
+
+Run the candidate against the failure cluster:
+- Must match ≥ 80% of cluster entries
+- Below 80%, the candidate is too specific; widen and retry
+
+## PR generation
+
+For each promoted candidate, create a PR on `LIFEAI/regen-root`:
+
+```
+Title: [verifier-corpus] Add rule FP0{N} — {short description}
+
+Body:
+This rule was generated by rdc:extract-verifier-rules on {date}.
+
+**Cluster size:** {N} occurrences over {time span}
+**False positive rate:** {rate}
+**Recall:** {percent}
+
+**Sample failures:**
+{3-5 example enhancement-log entries}
+
+**Generated rule:**
+```json
+{rule JSON}
+```
+
+**Reviewer checklist:**
+- [ ] Verify the rule doesn't match any known-good brochure
+- [ ] Verify the suggested fix is reasonable
+- [ ] Confirm severity level
+- [ ] Confirm category
+
+Auto-merging: {yes/no based on confidence}
+```
+
+## Auto-merge policy
+
+Auto-merge if **all** are true:
+- False-positive rate = 0
+- Recall ≥ 95%
+- Cluster size ≥ 10
+- The rule pattern is a strict subset of an existing pattern (not a fundamentally new category)
+- A maintainer's auto-merge flag is enabled in `.rdc/config.json`
+
+Otherwise, await human review.
+
+## Output: nightly report
+
+`.rdc/reports/verifier-{YYYY-MM-DD}.md`:
+
+```markdown
+# Verifier Corpus Update — {date}
+
+## Summary
+
+- Enhancement log entries since last run: {N}
+- Clusters identified: {C}
+- Rules proposed: {R}
+- Rules auto-merged: {A}
+- Rules awaiting review: {W}
+
+## Top clusters
+
+| Cluster | Size | Pattern | Status |
+|---|---|---|---|
+| ...
+
+## Open PRs
+
+- #{N}: FP0{X} — {description}
+- ...
+
+## False-positive rejections
+
+Candidates that failed the known-good test:
+- {summary}
+
+## Corpus growth
+
+- Total rules in forbidden-patterns.json: {before} → {after}
+- Total component-allowlist entries: unchanged
+- Pagination rules: unchanged
+```
+
+## Manual invocation
+
+A maintainer can run this skill manually after a significant brochure run to immediately promote lessons learned:
+
+```
+rdc:extract-verifier-rules --since "2026-05-27" --auto-merge=false
+```
+
+The `--auto-merge=false` flag forces human review on all proposals regardless of confidence.
+
+## Why this matters
+
+The verifier corpus is the asset. Anyone can copy the kit. Anyone can copy the ESLint plugin. **Replicating 12 months of customer-tested failure patterns is the moat.** This skill is how that moat compounds. Every brochure run feeds it. Every nightly run grows it.
+
+After 12 months at moderate scale, the corpus will contain ~500-2,000 rules, calibrated against real customer documents, with each rule's hit count visible. New competitors entering this category will face a starting position 12 months behind.
+
+That is the point.