@harness-engineering/cli 2.6.1 → 2.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/agents/skills/claude-code/align-design-system/SKILL.md +174 -0
- package/dist/agents/skills/claude-code/align-design-system/skill.yaml +57 -0
- package/dist/agents/skills/claude-code/audit-brand-compliance/SKILL.md +189 -0
- package/dist/agents/skills/claude-code/audit-brand-compliance/skill.yaml +50 -0
- package/dist/agents/skills/claude-code/audit-component-anatomy/SKILL.md +181 -0
- package/dist/agents/skills/claude-code/audit-component-anatomy/skill.yaml +51 -0
- package/dist/agents/skills/claude-code/copy-craft/SKILL.md +192 -0
- package/dist/agents/skills/claude-code/copy-craft/skill.yaml +52 -0
- package/dist/agents/skills/claude-code/detect-design-drift/SKILL.md +139 -0
- package/dist/agents/skills/claude-code/detect-design-drift/skill.yaml +50 -0
- package/dist/agents/skills/claude-code/harness-accessibility/SKILL.md +10 -0
- package/dist/agents/skills/claude-code/harness-design-craft/SKILL.md +212 -0
- package/dist/agents/skills/claude-code/harness-design-craft/skill.yaml +57 -0
- package/dist/agents/skills/claude-code/harness-design-pipeline/SKILL.md +266 -0
- package/dist/agents/skills/claude-code/harness-design-pipeline/skill.yaml +70 -0
- package/dist/agents/skills/claude-code/knowledge-craft/SKILL.md +180 -0
- package/dist/agents/skills/claude-code/knowledge-craft/skill.yaml +49 -0
- package/dist/agents/skills/claude-code/naming-craft/SKILL.md +244 -0
- package/dist/agents/skills/claude-code/naming-craft/skill.yaml +51 -0
- package/dist/agents/skills/claude-code/security-craft/SKILL.md +205 -0
- package/dist/agents/skills/claude-code/security-craft/skill.yaml +58 -0
- package/dist/agents/skills/claude-code/spec-craft/SKILL.md +184 -0
- package/dist/agents/skills/claude-code/spec-craft/skill.yaml +55 -0
- package/dist/agents/skills/claude-code/test-craft/SKILL.md +212 -0
- package/dist/agents/skills/claude-code/test-craft/skill.yaml +58 -0
- package/dist/agents/skills/codex/align-design-system/SKILL.md +174 -0
- package/dist/agents/skills/codex/align-design-system/skill.yaml +57 -0
- package/dist/agents/skills/codex/audit-brand-compliance/SKILL.md +189 -0
- package/dist/agents/skills/codex/audit-brand-compliance/skill.yaml +50 -0
- package/dist/agents/skills/codex/audit-component-anatomy/SKILL.md +181 -0
- package/dist/agents/skills/codex/audit-component-anatomy/skill.yaml +51 -0
- package/dist/agents/skills/codex/copy-craft/SKILL.md +192 -0
- package/dist/agents/skills/codex/copy-craft/skill.yaml +52 -0
- package/dist/agents/skills/codex/detect-design-drift/SKILL.md +139 -0
- package/dist/agents/skills/codex/detect-design-drift/skill.yaml +50 -0
- package/dist/agents/skills/codex/harness-accessibility/SKILL.md +10 -0
- package/dist/agents/skills/codex/harness-design-craft/SKILL.md +212 -0
- package/dist/agents/skills/codex/harness-design-craft/skill.yaml +57 -0
- package/dist/agents/skills/codex/harness-design-pipeline/SKILL.md +266 -0
- package/dist/agents/skills/codex/harness-design-pipeline/skill.yaml +70 -0
- package/dist/agents/skills/codex/knowledge-craft/SKILL.md +180 -0
- package/dist/agents/skills/codex/knowledge-craft/skill.yaml +49 -0
- package/dist/agents/skills/codex/naming-craft/SKILL.md +244 -0
- package/dist/agents/skills/codex/naming-craft/skill.yaml +51 -0
- package/dist/agents/skills/codex/security-craft/SKILL.md +205 -0
- package/dist/agents/skills/codex/security-craft/skill.yaml +58 -0
- package/dist/agents/skills/codex/spec-craft/SKILL.md +184 -0
- package/dist/agents/skills/codex/spec-craft/skill.yaml +55 -0
- package/dist/agents/skills/codex/test-craft/SKILL.md +212 -0
- package/dist/agents/skills/codex/test-craft/skill.yaml +58 -0
- package/dist/agents/skills/cursor/align-design-system/SKILL.md +174 -0
- package/dist/agents/skills/cursor/align-design-system/skill.yaml +57 -0
- package/dist/agents/skills/cursor/audit-brand-compliance/SKILL.md +189 -0
- package/dist/agents/skills/cursor/audit-brand-compliance/skill.yaml +50 -0
- package/dist/agents/skills/cursor/audit-component-anatomy/SKILL.md +181 -0
- package/dist/agents/skills/cursor/audit-component-anatomy/skill.yaml +51 -0
- package/dist/agents/skills/cursor/copy-craft/SKILL.md +192 -0
- package/dist/agents/skills/cursor/copy-craft/skill.yaml +52 -0
- package/dist/agents/skills/cursor/detect-design-drift/SKILL.md +139 -0
- package/dist/agents/skills/cursor/detect-design-drift/skill.yaml +50 -0
- package/dist/agents/skills/cursor/harness-accessibility/SKILL.md +10 -0
- package/dist/agents/skills/cursor/harness-design-craft/SKILL.md +212 -0
- package/dist/agents/skills/cursor/harness-design-craft/skill.yaml +57 -0
- package/dist/agents/skills/cursor/harness-design-pipeline/SKILL.md +266 -0
- package/dist/agents/skills/cursor/harness-design-pipeline/skill.yaml +70 -0
- package/dist/agents/skills/cursor/knowledge-craft/SKILL.md +180 -0
- package/dist/agents/skills/cursor/knowledge-craft/skill.yaml +49 -0
- package/dist/agents/skills/cursor/naming-craft/SKILL.md +244 -0
- package/dist/agents/skills/cursor/naming-craft/skill.yaml +51 -0
- package/dist/agents/skills/cursor/security-craft/SKILL.md +205 -0
- package/dist/agents/skills/cursor/security-craft/skill.yaml +58 -0
- package/dist/agents/skills/cursor/spec-craft/SKILL.md +184 -0
- package/dist/agents/skills/cursor/spec-craft/skill.yaml +55 -0
- package/dist/agents/skills/cursor/test-craft/SKILL.md +212 -0
- package/dist/agents/skills/cursor/test-craft/skill.yaml +58 -0
- package/dist/agents/skills/gemini-cli/align-design-system/SKILL.md +174 -0
- package/dist/agents/skills/gemini-cli/align-design-system/skill.yaml +57 -0
- package/dist/agents/skills/gemini-cli/audit-brand-compliance/SKILL.md +189 -0
- package/dist/agents/skills/gemini-cli/audit-brand-compliance/skill.yaml +50 -0
- package/dist/agents/skills/gemini-cli/audit-component-anatomy/SKILL.md +181 -0
- package/dist/agents/skills/gemini-cli/audit-component-anatomy/skill.yaml +51 -0
- package/dist/agents/skills/gemini-cli/copy-craft/SKILL.md +192 -0
- package/dist/agents/skills/gemini-cli/copy-craft/skill.yaml +52 -0
- package/dist/agents/skills/gemini-cli/detect-design-drift/SKILL.md +139 -0
- package/dist/agents/skills/gemini-cli/detect-design-drift/skill.yaml +50 -0
- package/dist/agents/skills/gemini-cli/harness-accessibility/SKILL.md +10 -0
- package/dist/agents/skills/gemini-cli/harness-design-craft/SKILL.md +212 -0
- package/dist/agents/skills/gemini-cli/harness-design-craft/skill.yaml +57 -0
- package/dist/agents/skills/gemini-cli/harness-design-pipeline/SKILL.md +266 -0
- package/dist/agents/skills/gemini-cli/harness-design-pipeline/skill.yaml +70 -0
- package/dist/agents/skills/gemini-cli/knowledge-craft/SKILL.md +180 -0
- package/dist/agents/skills/gemini-cli/knowledge-craft/skill.yaml +49 -0
- package/dist/agents/skills/gemini-cli/naming-craft/SKILL.md +244 -0
- package/dist/agents/skills/gemini-cli/naming-craft/skill.yaml +51 -0
- package/dist/agents/skills/gemini-cli/security-craft/SKILL.md +205 -0
- package/dist/agents/skills/gemini-cli/security-craft/skill.yaml +58 -0
- package/dist/agents/skills/gemini-cli/spec-craft/SKILL.md +184 -0
- package/dist/agents/skills/gemini-cli/spec-craft/skill.yaml +55 -0
- package/dist/agents/skills/gemini-cli/test-craft/SKILL.md +212 -0
- package/dist/agents/skills/gemini-cli/test-craft/skill.yaml +58 -0
- package/dist/{agents-md-2JUQ4FE2.js → agents-md-FKMKH6ZF.js} +4 -4
- package/dist/{analyzer-VY6DJVOU-4AHIJLNS.js → analyzer-H3AHBFSL-KPOPJYRB.js} +8 -8
- package/dist/{architecture-CXAVNB5X.js → architecture-LUR2I67H.js} +6 -6
- package/dist/{assess-project-M626SSPN.js → assess-project-OJWECOP2.js} +2 -1
- package/dist/bin/harness-mcp.js +17 -17
- package/dist/bin/harness.js +27 -27
- package/dist/{check-phase-gate-XMU6LLUQ.js → check-phase-gate-HS5KYLRO.js} +5 -5
- package/dist/{chunk-5XLLIYYL.js → chunk-6UHDQP2N.js} +1 -1
- package/dist/{chunk-I6K43QQS.js → chunk-7AF6C5GE.js} +3 -3
- package/dist/{chunk-372MGNTI.js → chunk-7PLIWP7U.js} +6899 -133
- package/dist/{chunk-VUCGB2KH.js → chunk-B2VAYLYI.js} +1 -1
- package/dist/{chunk-HFOB2MPQ.js → chunk-CZPOPMMI.js} +3 -3
- package/dist/{chunk-43M3RLFQ.js → chunk-FC4JPM6W.js} +2 -2
- package/dist/{chunk-6AX6R3LM.js → chunk-FEKBXX2J.js} +9 -9
- package/dist/{chunk-VTTNIZJL.js → chunk-FXZI6NJ2.js} +131 -6
- package/dist/{chunk-5JNSFYZU.js → chunk-GGSNWH7P.js} +6 -6
- package/dist/{chunk-BCFDGOMA.js → chunk-H2ZJOJ7A.js} +81 -1
- package/dist/{chunk-PFZEADQM.js → chunk-INBTDBV2.js} +79 -2
- package/dist/{chunk-Q6ZR2L6Q.js → chunk-KN3EMDZ5.js} +2 -2
- package/dist/{chunk-JFZOWO7D.js → chunk-LECXPXS3.js} +1 -1
- package/dist/{chunk-CQOMVZDC.js → chunk-LGYXR2KZ.js} +2237 -604
- package/dist/{chunk-QOKMDDBJ.js → chunk-LYIPI2SJ.js} +6 -6
- package/dist/{chunk-L5BCZ5XS.js → chunk-N5NSSLMU.js} +1 -1
- package/dist/{chunk-GWE5LL3R.js → chunk-NIVWM7OW.js} +10 -10
- package/dist/{chunk-24KCOJJG.js → chunk-O2ASYKA6.js} +1 -1
- package/dist/{chunk-K246XQ2L.js → chunk-PTHX545Q.js} +1 -1
- package/dist/{chunk-EPUKTTJZ.js → chunk-RC3OI5NK.js} +5 -1
- package/dist/{chunk-YP7I25SL.js → chunk-RJ2WEES5.js} +1 -1
- package/dist/{chunk-5FDBXUFW.js → chunk-TIH53QXY.js} +1 -1
- package/dist/{chunk-OA6SWTU4.js → chunk-VI4IS55S.js} +11 -8
- package/dist/{chunk-GWF2OILZ.js → chunk-VSIRUZQP.js} +1 -1
- package/dist/{chunk-YYRQCQPZ.js → chunk-WZY5U6NS.js} +178 -109
- package/dist/{chunk-SZEFU6XC.js → chunk-Y4T6ENKQ.js} +9 -9
- package/dist/{chunk-HMW3VKUF.js → chunk-Z3TMCCZB.js} +10 -10
- package/dist/{ci-workflow-3JNDZQYD.js → ci-workflow-MDSTHJOY.js} +4 -4
- package/dist/{dist-OHDQBTSO.js → dist-72ID44UE.js} +3 -3
- package/dist/{dist-QPWXPCPL.js → dist-R3TOOPJ7.js} +1 -1
- package/dist/{docs-5RYMDHN3.js → docs-2RFOJ6XY.js} +6 -6
- package/dist/{engine-GKGT5ULT.js → engine-BSFKOGPN.js} +4 -4
- package/dist/{entropy-DQTAPSSE.js → entropy-3UX6644G.js} +5 -5
- package/dist/{feedback-WH6XPQJS.js → feedback-CVCFYND4.js} +2 -2
- package/dist/{generate-agent-definitions-SZA4QIHN.js → generate-agent-definitions-HZ3XENWO.js} +5 -5
- package/dist/{glob-helper-XE2UGEOV.js → glob-helper-EX4YZKZO.js} +1 -1
- package/dist/{graph-loader-EQBOIHFZ.js → graph-loader-EIFEOUVI.js} +1 -1
- package/dist/index.d.ts +1012 -12
- package/dist/index.js +27 -27
- package/dist/{loader-BNKGM23C.js → loader-6RHWU5FH.js} +4 -4
- package/dist/{mcp-YKKPWUMH.js → mcp-OYR26JDI.js} +17 -17
- package/dist/{performance-54TMJOEU.js → performance-JJMIRXCH.js} +6 -6
- package/dist/{review-pipeline-OTDDVPNY.js → review-pipeline-BHQZIXWZ.js} +4 -4
- package/dist/{runtime-W5THSNAL.js → runtime-DALTAVTN.js} +4 -4
- package/dist/{scan-4PPP6ZXT.js → scan-LYKLM4EQ.js} +1 -1
- package/dist/{security-27HW7CYT.js → security-RMLMYFPB.js} +1 -1
- package/dist/{validate-QRVCD4GP.js → validate-R7DZRDFK.js} +5 -5
- package/dist/{validate-cross-check-BUAGBAPO.js → validate-cross-check-AUUZFNVL.js} +4 -4
- package/package.json +8 -7
|
@@ -0,0 +1,189 @@
|
|
|
1
|
+
# Audit Brand Compliance
|
|
2
|
+
|
|
3
|
+
> Rule-based brand-semantics audit. Detects (a) tokens used in forbidden contexts per their `$extensions.harness.brand.forbidden_contexts` metadata (BRAND-T001) and (b) UI copy containing phrases listed in `DESIGN.md ## Brand Rules → voice.forbidden_phrases` (BRAND-V001). The 4th composed verifier in `harness check-design`, alongside audit-component-anatomy, design-craft critique, and detect-design-drift.
|
|
4
|
+
|
|
5
|
+
## When to Use
|
|
6
|
+
|
|
7
|
+
- After authoring or editing `DESIGN.md ## Brand Rules` — verify the new constraints are enforceable on the existing codebase
|
|
8
|
+
- After adding `$extensions.harness.brand` metadata to a token — discover existing call sites in forbidden contexts
|
|
9
|
+
- As part of `harness validate` (fast-mode hook, gated by `design.audit.brandCompliance.enabled`)
|
|
10
|
+
- As the 4th composed verifier in `harness check-design` (the unified design check)
|
|
11
|
+
- Before a PR with UI copy or token-usage changes lands
|
|
12
|
+
- NOT for tone-by-context rules (deferred to v1.x — requires component-state inference)
|
|
13
|
+
- NOT for reading-level or sentence-length rules (deferred to v1.x — ship with tone-context)
|
|
14
|
+
- NOT for asset-usage rules (deferred to v1.x — requires image-tag scanning)
|
|
15
|
+
- NOT for semantic-token-alias enforcement (overlaps with detect-design-drift T001 — design after both have shipped)
|
|
16
|
+
- NOT for brand-rule authoring (use `harness-design` skill to draft DESIGN.md sections)
|
|
17
|
+
|
|
18
|
+
## Process
|
|
19
|
+
|
|
20
|
+
### Phase 1: LOAD — Parse the two input sources
|
|
21
|
+
|
|
22
|
+
1. **Read project configuration.** Check `harness.config.json` for:
|
|
23
|
+
- `design.strictness` — `strict` / `standard` / `permissive` (default `standard`)
|
|
24
|
+
- `design.audit.brandCompliance.enabled` — gate (default `true`)
|
|
25
|
+
- `design.audit.brandCompliance.rules.{tokenMisuse,voice}` — per-rule toggles
|
|
26
|
+
|
|
27
|
+
2. **Load `design-system/DESIGN.md` `## Brand Rules`.** The parser extracts:
|
|
28
|
+
- `voice.forbiddenPhrases: string[]` — used by BRAND-V001 in v1
|
|
29
|
+
- `voice.constant`, `voice.readingLevel`, `voice.maxSentenceWords` — parsed but unused in v1 (forward-compat)
|
|
30
|
+
- `toneByContext`, `assets`, `semanticTokenAliases` — parsed but unused (v1.x)
|
|
31
|
+
- Returns `null` when DESIGN.md absent or `## Brand Rules` section missing → BRAND-V001 silently skips.
|
|
32
|
+
|
|
33
|
+
3. **Load `design-system/tokens.json` `$extensions.harness.brand`.** Walks the DTCG token tree capturing per-token `role`, `approved_contexts`, `forbidden_contexts`. Returns `null` when no token carries the extension → BRAND-T\* silently skips.
|
|
34
|
+
|
|
35
|
+
### Phase 2: SCAN — Apply the two rule families
|
|
36
|
+
|
|
37
|
+
1. **BRAND-T001 — token misuse (regex-based).** For each token whose `forbidden_contexts` is non-empty:
|
|
38
|
+
- Find every reference to the token's dotted path in source (recognizes three forms):
|
|
39
|
+
- `tokens.X.Y.Z` (JS accessor)
|
|
40
|
+
- `var(--X-Y-Z)` (CSS var, kebab-cased)
|
|
41
|
+
- `'X.Y.Z'` / `"X.Y.Z"` (string literal)
|
|
42
|
+
- Inspect surrounding context (same line + nearest non-blank previous and next line) for the v1 context-vocabulary keywords: `cta`, `selection`, `focus`, `data-visualization`, `decorative`, `background`, `text`, `border`, `error`, `success`, `warning`.
|
|
43
|
+
- If a forbidden context matches: emit BRAND-T001.
|
|
44
|
+
|
|
45
|
+
2. **BRAND-V001 — forbidden phrases (TS Compiler API).** For each `.tsx`/`.jsx` file:
|
|
46
|
+
- Walk the JSX tree.
|
|
47
|
+
- For each `JsxText` node: case-insensitive substring scan for any forbiddenPhrase.
|
|
48
|
+
- For each `JsxAttribute` whose initializer is a string literal: same scan.
|
|
49
|
+
- Deduplicate per `(file, line, phrase)`.
|
|
50
|
+
|
|
51
|
+
### Phase 3: REPORT — Aggregate and surface
|
|
52
|
+
|
|
53
|
+
1. **Severity from `design.strictness`** (uses `severityFor`):
|
|
54
|
+
- `strict` — all findings `error`
|
|
55
|
+
- `standard` — BRAND-T001 `error` (declared violation), BRAND-V001 `warn` (copy nuance)
|
|
56
|
+
- `permissive` — all findings `info`
|
|
57
|
+
|
|
58
|
+
2. **Aggregate `bySeverity` and `byCode`** into the standard Verifier shape: `{ findings, summary, catalog, meta }`.
|
|
59
|
+
|
|
60
|
+
3. **Persist findings to the graph (when composed by check-design).** check-design routes brand findings through `DesignConstraintAdapter.recordFindings()` alongside anatomy / craft / drift. v1 uses the shared `VIOLATES_design` edge; v1.x may add a brand-specific edge.
|
|
61
|
+
|
|
62
|
+
## Harness Integration
|
|
63
|
+
|
|
64
|
+
- **`harness validate`** — Fast-mode hook gated by `design.audit.brandCompliance.enabled`. Degrades gracefully on failure (single warning; other checks continue).
|
|
65
|
+
- **`harness check-design`** — Composes brand as the 4th verifier alongside audit-anatomy, design-craft critique, and detect-design-drift. This is the canonical invocation path.
|
|
66
|
+
- **`mcp__harness__audit_brand`** — MCP tool. Input: `{ path, mode, files?, designStrictness?, rules? }`. Output: `{ findings, summary, catalog, meta }`. Consumed by check-design and the (future) #5 design-pipeline orchestrator.
|
|
67
|
+
- **`DesignConstraintAdapter.recordFindings()`** — Generic graph persistence entry point shipped in PR #390. Brand findings reuse the adapter (no graph schema changes in v1).
|
|
68
|
+
- **`harness-design` skill** — Authors `DESIGN.md ## Brand Rules`. audit-brand-compliance is the matching enforcer.
|
|
69
|
+
- **`Verifier<F>` interface** — Extracted in this PR at the 4th-verifier threshold. Lives at `packages/cli/src/shared/verifier.ts`. Adding a 5th verifier requires only a type-alias declaration of conformance.
|
|
70
|
+
|
|
71
|
+
## Success Criteria
|
|
72
|
+
|
|
73
|
+
See `docs/changes/design-pipeline/audit-brand-compliance/proposal.md` for the full 34 success criteria. Highlights:
|
|
74
|
+
|
|
75
|
+
- DESIGN.md parser returns `null` when section absent (silent-skip pattern)
|
|
76
|
+
- Token-extensions walker returns `null` when no token carries `$extensions.harness.brand`
|
|
77
|
+
- BRAND-T001 fires on `tokens.X`, `var(--x)`, and `'X'` reference forms
|
|
78
|
+
- BRAND-T001 honors approved_contexts (no finding when context is allowed)
|
|
79
|
+
- BRAND-V001 fires on JSX text + string-typed JSX attributes (case-insensitive)
|
|
80
|
+
- BRAND-V001 deduplicates per `(file, line, phrase)`
|
|
81
|
+
- Verifier interface extraction: anatomy / drift / brand all declare structural conformance
|
|
82
|
+
- `harness check-design` test extended for 4-verifier composition (zero regressions)
|
|
83
|
+
- MCP tool count bumps 72 → 73
|
|
84
|
+
|
|
85
|
+
## Examples
|
|
86
|
+
|
|
87
|
+
### Example: Token used in forbidden context
|
|
88
|
+
|
|
89
|
+
**Input:**
|
|
90
|
+
|
|
91
|
+
`design-system/tokens.json`:
|
|
92
|
+
|
|
93
|
+
```json
|
|
94
|
+
{
|
|
95
|
+
"color": {
|
|
96
|
+
"brand": {
|
|
97
|
+
"500": {
|
|
98
|
+
"$type": "color",
|
|
99
|
+
"$value": "#3b82f6",
|
|
100
|
+
"$extensions": {
|
|
101
|
+
"harness": {
|
|
102
|
+
"brand": {
|
|
103
|
+
"role": "primary",
|
|
104
|
+
"approved_contexts": ["cta", "selection", "focus"],
|
|
105
|
+
"forbidden_contexts": ["data-visualization", "decorative"]
|
|
106
|
+
}
|
|
107
|
+
}
|
|
108
|
+
}
|
|
109
|
+
}
|
|
110
|
+
}
|
|
111
|
+
}
|
|
112
|
+
}
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
`src/Chart.tsx`:
|
|
116
|
+
|
|
117
|
+
```tsx
|
|
118
|
+
// data-visualization color palette
|
|
119
|
+
const palette = [tokens.color.brand.500, ...];
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
**Output:**
|
|
123
|
+
|
|
124
|
+
```
|
|
125
|
+
BRAND-T001 [error] src/Chart.tsx:2 — Token "color.brand.500" is used in forbidden context "data-visualization"
|
|
126
|
+
Fix: Token "color.brand.500" is not approved for the "data-visualization" context.
|
|
127
|
+
Use an approved token (allowed contexts: cta, selection, focus), or update
|
|
128
|
+
tokens.json $extensions.harness.brand if the policy is wrong.
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
### Example: Forbidden phrase in UI copy
|
|
132
|
+
|
|
133
|
+
**Input:**
|
|
134
|
+
|
|
135
|
+
`DESIGN.md`:
|
|
136
|
+
|
|
137
|
+
```markdown
|
|
138
|
+
## Brand Rules
|
|
139
|
+
|
|
140
|
+
### Voice
|
|
141
|
+
|
|
142
|
+
forbidden_phrases:
|
|
143
|
+
|
|
144
|
+
- "click here"
|
|
145
|
+
- "best-in-class"
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
`src/Cta.tsx`:
|
|
149
|
+
|
|
150
|
+
```tsx
|
|
151
|
+
export const Cta = () => <a href="/x">Click here</a>;
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
**Output:**
|
|
155
|
+
|
|
156
|
+
```
|
|
157
|
+
BRAND-V001 [warn] src/Cta.tsx:1 — UI copy contains forbidden phrase "click here" — declared at DESIGN.md ## Brand Rules → Voice → forbidden_phrases
|
|
158
|
+
Fix: Rewrite to avoid "click here". If the phrase is unavoidable for this context,
|
|
159
|
+
remove it from voice.forbidden_phrases (or scope the audit) — but the default
|
|
160
|
+
policy is that brand voice trumps convenience.
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
## Gates
|
|
164
|
+
|
|
165
|
+
- **No findings without parsed inputs.** DESIGN.md absent → BRAND-V001 skips silently. tokens.json `$extensions.harness.brand` absent on every token → BRAND-T001 skips silently. Either resolver returning null is NOT a verifier failure.
|
|
166
|
+
- **No `.ts`/`.js` file scans for BRAND-V001.** Only `.jsx`/`.tsx` (user-visible JSX). Doc copy in `.md` is a different audience.
|
|
167
|
+
- **No tone-by-context inference.** v1 only matches the explicit context-vocabulary keywords against surrounding source text. v1.x adds component-state inference.
|
|
168
|
+
- **No autofix.** audit-only. The matching `align-brand-compliance` fix-side skill is deferred until detect signals demand.
|
|
169
|
+
- **No graph schema changes.** v1 reuses `VIOLATES_design` via `recordFindings()`. v1.x may add `VIOLATES_brand` edge for queryability.
|
|
170
|
+
- **Strictness from config, not assumed.** Read `design.strictness` from `harness.config.json`; default `standard` if absent.
|
|
171
|
+
|
|
172
|
+
## Escalation
|
|
173
|
+
|
|
174
|
+
- **When BRAND-T001 false-positives on a far-context reference:** the v1 context inference is intentionally narrow (same line + adjacent non-blank). For a token used in a "background" context where the keyword appears 10 lines away, v1 misses it. v1.x adds richer context inference; for now, either widen the surrounding comment or accept the miss.
|
|
175
|
+
- **When BRAND-V001 false-positives on a substring (e.g., "as is" in "as issued"):** v1 uses substring match. Add word-boundary regex in v1.x. For now, rephrase the copy or remove the phrase from voice.forbidden_phrases.
|
|
176
|
+
- **When a project ships tokens with a different `$extensions` shape:** v1 reads only `harness.brand`. Document the actual shape your project uses and add it to the schema sketch in ADR 0028 — DTCG `$extensions` namespaces are vendor-prefixed and additions are forward-compatible.
|
|
177
|
+
- **When `harness validate` runtime exceeds 3 seconds:** Set `design.audit.brandCompliance.fastMode.maxFiles` to cap the scope. The MCP tool ignores the cap (`fast`/`full` equivalent in v1).
|
|
178
|
+
- **When the graph persistence fails:** Skip graph integration for that run; findings still appear in the report. The graph is a consumer, not a gate.
|
|
179
|
+
- **When you want tone-by-context rules today:** Manual audit until v1.x ships. Component-state inference (empty/error/success/loading) requires JSX-context analysis that's a separate brainstorm.
|
|
180
|
+
|
|
181
|
+
## Status
|
|
182
|
+
|
|
183
|
+
**v1 — in implementation.** See:
|
|
184
|
+
|
|
185
|
+
- Spec: `docs/changes/design-pipeline/audit-brand-compliance/proposal.md`
|
|
186
|
+
- ADR (input schema source): `docs/knowledge/decisions/0028-brand-guidelines-source-of-truth.md`
|
|
187
|
+
- Roadmap entry: `design-pipeline sub-project #3` in `docs/roadmap.md`
|
|
188
|
+
- Sibling rule-based audits: `audit-component-anatomy` (#2), `detect-design-drift` (#1)
|
|
189
|
+
- Cross-cutting: extracts `Verifier<F>` interface at `packages/cli/src/shared/verifier.ts` (deferred until 4th data point — this is it)
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
name: audit-brand-compliance
|
|
2
|
+
version: "0.1.0"
|
|
3
|
+
description: Rule-based brand-semantics audit. Detects token misuse (BRAND-T001 via $extensions.harness.brand.forbidden_contexts) and voice violations (BRAND-V001 via DESIGN.md voice.forbidden_phrases). 4th composed verifier in harness check-design. Triggers extraction of the formal verifier interface.
|
|
4
|
+
stability: draft
|
|
5
|
+
cognitive_mode: constructive-architect
|
|
6
|
+
triggers:
|
|
7
|
+
- manual
|
|
8
|
+
- on_pr
|
|
9
|
+
- on_new_feature
|
|
10
|
+
platforms:
|
|
11
|
+
- claude-code
|
|
12
|
+
tools:
|
|
13
|
+
- Bash
|
|
14
|
+
- Read
|
|
15
|
+
- Glob
|
|
16
|
+
- Grep
|
|
17
|
+
cli:
|
|
18
|
+
command: harness skill run audit-brand-compliance
|
|
19
|
+
args:
|
|
20
|
+
- name: path
|
|
21
|
+
description: Project root path
|
|
22
|
+
required: false
|
|
23
|
+
- name: mode
|
|
24
|
+
description: "fast or full (equivalent in v1)"
|
|
25
|
+
required: false
|
|
26
|
+
- name: files
|
|
27
|
+
description: Optional list of files to scope the scan
|
|
28
|
+
required: false
|
|
29
|
+
mcp:
|
|
30
|
+
tool: run_skill
|
|
31
|
+
input:
|
|
32
|
+
skill: audit-brand-compliance
|
|
33
|
+
path: string
|
|
34
|
+
type: rigid
|
|
35
|
+
tier: 2
|
|
36
|
+
phases:
|
|
37
|
+
- name: load
|
|
38
|
+
description: Parse DESIGN.md ## Brand Rules and tokens.json $extensions.harness.brand
|
|
39
|
+
required: true
|
|
40
|
+
- name: scan
|
|
41
|
+
description: Walk JSX text/attributes for forbidden phrases; scan source for token-misuse-in-context
|
|
42
|
+
required: true
|
|
43
|
+
- name: report
|
|
44
|
+
description: Aggregate findings; emit BRAND-T* / BRAND-V001 with severity per design.strictness
|
|
45
|
+
required: true
|
|
46
|
+
state:
|
|
47
|
+
persistent: false
|
|
48
|
+
depends_on:
|
|
49
|
+
- detect-design-drift
|
|
50
|
+
- harness-design
|
|
@@ -0,0 +1,181 @@
|
|
|
1
|
+
# Audit Component Anatomy
|
|
2
|
+
|
|
3
|
+
> Detect missing required anatomy parts in component definitions and missing-anatomy-component patterns in composition. First programmatic enforcer of component-anatomy rules — finds what design-component-anatomy reference content prescribes.
|
|
4
|
+
|
|
5
|
+
## When to Use
|
|
6
|
+
|
|
7
|
+
- Reviewing component-library code for completeness of slots, states, sizes
|
|
8
|
+
- Detecting compositional patterns where a needed anatomy component is missing (e.g., `.map()` over data with no empty branch; async actions with no loading boundary)
|
|
9
|
+
- After component-library changes to verify anatomy contracts still hold
|
|
10
|
+
- Before merging UI changes to catch anatomy regressions
|
|
11
|
+
- When `on_new_feature` triggers fire and the feature touches a component definition or usage
|
|
12
|
+
- NOT for token validation (use harness-design-system)
|
|
13
|
+
- NOT for accessibility compliance (use harness-accessibility — overlap is documented; this skill defers label-association findings to a11y)
|
|
14
|
+
- NOT for aesthetic critique or polish suggestions (use harness-design-craft)
|
|
15
|
+
- NOT for declared-anti-pattern enforcement (use harness-design)
|
|
16
|
+
|
|
17
|
+
## Process
|
|
18
|
+
|
|
19
|
+
### Phase 1: SCAN — Identify component types and run audits
|
|
20
|
+
|
|
21
|
+
1. **Read project configuration.** Check `harness.config.json` for:
|
|
22
|
+
- `design.strictness` — `strict`/`standard`/`permissive` (default `standard`)
|
|
23
|
+
- `design.audit.componentAnatomy.enabled` — gate for the entire audit AND the harness-accessibility deferral (default `true`)
|
|
24
|
+
- `design.audit.componentAnatomy.catalog` — `"default"` or path to project-supplied catalog
|
|
25
|
+
- `design.audit.componentAnatomy.patterns` — `"all"`, `"none"`, or explicit list of pattern codes
|
|
26
|
+
- `design.audit.componentAnatomy.fastMode.{patterns,maxFiles}` — fast-mode controls
|
|
27
|
+
|
|
28
|
+
2. **Load source-of-truth catalogs (resolution order, most specific wins):**
|
|
29
|
+
- Component author's JSDoc `@component-type X` + `@anatomy-slot ...` etc. — file-level self-declaration
|
|
30
|
+
- `design-system/DESIGN.md` `## Component Registry` + `## Component Anatomy Overrides` — project-level
|
|
31
|
+
- Built-in convention library (20 component types: Button, Input, Select, Modal/Dialog, EmptyState, Card, Tabs, Menu, Toast, Form, Accordion, Tooltip, Popover, Drawer, Slider, Switch, Checkbox, Radio, Avatar, Badge) — fallback default
|
|
32
|
+
- No match for a file's component-type → silent skip (no false positives)
|
|
33
|
+
|
|
34
|
+
3. **Identify component type per file** using the same resolution order:
|
|
35
|
+
- JSDoc `@component-type X` tag → file IS X
|
|
36
|
+
- DESIGN.md Component Registry mapping → use that
|
|
37
|
+
- Top-level export name matches a catalog entry → use that
|
|
38
|
+
- No match → skip silently
|
|
39
|
+
|
|
40
|
+
4. **Run convention audit (definition findings, `ANAT-D*` codes).** For each catalogued component file:
|
|
41
|
+
- Parse component definition via TypeScript Compiler API (AST)
|
|
42
|
+
- Extract exported component's prop type / interface
|
|
43
|
+
- Check that each required anatomy part (slot, state, variant, size) is exposed as a prop or sub-component
|
|
44
|
+
- Emit `ANAT-D{NNN}` finding for each missing required part
|
|
45
|
+
- Emit `ANAT-D000` info finding when JSDoc declaration diverges from catalog convention
|
|
46
|
+
|
|
47
|
+
5. **Run pattern audit (pattern-presence findings, `ANAT-P*` codes).** For each file matching pattern targets:
|
|
48
|
+
- Parse via tree-sitter (`tree-sitter-typescript` + `tree-sitter-tsx`)
|
|
49
|
+
- Run each enabled pattern's S-expression query
|
|
50
|
+
- Post-process captures to confirm pattern presence (e.g., `.map()` without an empty-branch guard)
|
|
51
|
+
- Emit `ANAT-P{NNN}` finding for each match
|
|
52
|
+
- Skip patterns NOT enabled in `design.audit.componentAnatomy.patterns`
|
|
53
|
+
|
|
54
|
+
6. **Check for harness-accessibility overlap deferral.** When `design.audit.componentAnatomy.enabled = true`, this skill OWNS label-slot definition findings; harness-accessibility defers A11Y-010 and A11Y-050 for catalogued components. Track deferrals in `meta.deferralsToHarnessDesign` count.
|
|
55
|
+
|
|
56
|
+
### Phase 2: EVALUATE — Apply severity model
|
|
57
|
+
|
|
58
|
+
1. **Map finding severity using `design.strictness`:**
|
|
59
|
+
- `strict` — all findings are `error` severity (CI blocks)
|
|
60
|
+
- `standard` — required-part missing → `error`; optional-part missing → `warn`; info findings → `info`
|
|
61
|
+
- `permissive` — all findings → `info` (nothing blocks)
|
|
62
|
+
|
|
63
|
+
2. **Cross-reference with graph constraints.** If a graph exists at `.harness/graph/`:
|
|
64
|
+
- Query existing `VIOLATES_CRAFT` edges via extended `DesignConstraintAdapter`
|
|
65
|
+
- Identify NEW violations (not in graph) and RESOLVED violations (in graph but not in current scan)
|
|
66
|
+
|
|
67
|
+
3. **De-duplicate via deferral.** When this skill produces a finding whose root-cause is also tracked by harness-accessibility, prefer this skill's finding (anatomy-level cause) and increment deferrals counter.
|
|
68
|
+
|
|
69
|
+
### Phase 3: REPORT — Format and persist findings
|
|
70
|
+
|
|
71
|
+
1. **Write VIOLATES_CRAFT edges to graph.** Each finding becomes:
|
|
72
|
+
- Source: component file (`code_file` node) or component (`component` node)
|
|
73
|
+
- Target: `design_rule` node keyed by finding code (`ANAT-D023`, `ANAT-P001`)
|
|
74
|
+
- Metadata: severity, line, message, evidence snippet, runId
|
|
75
|
+
- Idempotent: re-running the audit produces no duplicate edges
|
|
76
|
+
|
|
77
|
+
2. **Format the report.** Grouped by component file, with finding codes linked to `docs/changes/design-pipeline/audit-component-anatomy/finding-codes.md`. Each finding includes:
|
|
78
|
+
- Code (e.g., `ANAT-D023`)
|
|
79
|
+
- Severity per `design.strictness`
|
|
80
|
+
- File path + line number
|
|
81
|
+
- Component type (if identified)
|
|
82
|
+
- Message + evidence snippet
|
|
83
|
+
- Fix hint (concrete next-step text — not "fix this")
|
|
84
|
+
|
|
85
|
+
3. **Emit summary.** Total findings, breakdown by severity, count of `meta.deferralsToHarnessDesign`, mode (`fast`/`full`), `runId`.
|
|
86
|
+
|
|
87
|
+
## Harness Integration
|
|
88
|
+
|
|
89
|
+
- **`harness validate`** — Fast-mode audit hook (convention catalog only; patterns are opt-in via `fastMode.patterns: true`). Findings respect `design.strictness`.
|
|
90
|
+
- **`mcp__harness__audit_anatomy`** — Programmatic API (input: path, mode, files, designStrictness, catalog; output: findings, summary, catalog applied, deferrals count). Consumed by harness check-design verifier (sub-project #4) and design-pipeline orchestrator (sub-project #5).
|
|
91
|
+
- **`DesignConstraintAdapter`** — Extended to register `ANAT-*` rule code namespace and write VIOLATES_CRAFT edges. Mirrors how harness-accessibility uses it for A11Y-\* codes.
|
|
92
|
+
- **`harness-accessibility`** — Coordinates via i18n-style deferral pattern. When `design.audit.componentAnatomy.enabled = true`, a11y defers A11Y-010 and A11Y-050 for catalogued components.
|
|
93
|
+
- **`design-component-anatomy`** (knowledge skill) — Source of convention vocabulary (slot, variant, state, size, exclusivity, required). This skill's catalog operationalizes that knowledge.
|
|
94
|
+
|
|
95
|
+
## Success Criteria
|
|
96
|
+
|
|
97
|
+
See `docs/changes/design-pipeline/audit-component-anatomy/proposal.md` for the full 30 success criteria. Highlights:
|
|
98
|
+
|
|
99
|
+
- Convention findings produced for known component types only (silent skip for unknown — zero false positives)
|
|
100
|
+
- JSDoc self-declaration overrides convention; DESIGN.md overrides convention but not JSDoc
|
|
101
|
+
- Pattern false-positive rate ≤ 5% on the 50-fixture corpus
|
|
102
|
+
- harness-accessibility deferral works (one finding for one root cause, not two)
|
|
103
|
+
- Fast-mode runtime ≤ 3s on 500-file repo
|
|
104
|
+
|
|
105
|
+
## Examples
|
|
106
|
+
|
|
107
|
+
### Example: Button missing a required state
|
|
108
|
+
|
|
109
|
+
**Input:** `packages/ui/src/Button.tsx` exports a `Button` component whose prop type omits any `loading` prop.
|
|
110
|
+
|
|
111
|
+
**Output:**
|
|
112
|
+
|
|
113
|
+
```
|
|
114
|
+
ANAT-D002 [error] Button.tsx — Button convention requires state:loading (exclusive)
|
|
115
|
+
File: packages/ui/src/Button.tsx
|
|
116
|
+
Component: Button (identified via export-name match)
|
|
117
|
+
Convention: APG/button + Radix Primitives Button
|
|
118
|
+
Fix: Add a `loading?: boolean` prop, OR add `@anatomy-state loading exclusive`
|
|
119
|
+
JSDoc tag to the component if loading is intentionally omitted.
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
Severity is `error` under `design.strictness: standard` because the missing part is `required: true` in the Button convention.
|
|
123
|
+
|
|
124
|
+
### Example: Empty list with no fallback (pattern-presence finding)
|
|
125
|
+
|
|
126
|
+
**Input:** `src/pages/Dashboard.tsx` contains `{items.map(item => <Card item={item} />)}` with no empty-state branch.
|
|
127
|
+
|
|
128
|
+
**Output:**
|
|
129
|
+
|
|
130
|
+
```
|
|
131
|
+
ANAT-P001 [warn] Dashboard.tsx:42 — map() over data with no empty-state branch
|
|
132
|
+
File: src/pages/Dashboard.tsx:42
|
|
133
|
+
Pattern: ANAT-P001 map-without-empty
|
|
134
|
+
Fix: Wrap the map in a conditional that renders <EmptyState> when items.length === 0:
|
|
135
|
+
items.length === 0 ? <EmptyState ... /> : items.map(...)
|
|
136
|
+
Or extract a guard component that handles both branches.
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
Severity is `warn` for pattern-presence findings under `design.strictness: standard`; `error` under `strict`.
|
|
140
|
+
|
|
141
|
+
### Example: Catalogued component with JSDoc divergence
|
|
142
|
+
|
|
143
|
+
**Input:** A Tabs.tsx file has `@anatomy-slot trigger` JSDoc but the convention library says Tabs requires both `trigger` AND `panel`.
|
|
144
|
+
|
|
145
|
+
**Output:**
|
|
146
|
+
|
|
147
|
+
```
|
|
148
|
+
ANAT-D000 [info] Tabs.tsx — JSDoc declaration omits 2 conventional anatomy parts
|
|
149
|
+
File: packages/ui/src/Tabs.tsx
|
|
150
|
+
Component: Tabs (identified via JSDoc @component-type)
|
|
151
|
+
Divergence: declared slots [trigger]; convention also expects [panel, list]
|
|
152
|
+
Note: JSDoc wins (resolution order #1). This is an info finding to surface
|
|
153
|
+
the divergence — no action required if intentional.
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
The `ANAT-D000` info code surfaces JSDoc-vs-convention divergence so authors can confirm the divergence is intentional.
|
|
157
|
+
|
|
158
|
+
## Gates
|
|
159
|
+
|
|
160
|
+
- **No findings without a parsed catalog.** If the convention library or DESIGN.md catalog cannot be loaded, stop with an explanatory error — do not emit speculative findings.
|
|
161
|
+
- **No usage-side findings in v1.** Only `ANAT-D*` (definition) and `ANAT-P*` (pattern) codes. Usage findings (`ANAT-U*`) ship in v2.
|
|
162
|
+
- **No autofix.** Findings include fix-hint text only; source files are never modified by this skill.
|
|
163
|
+
- **Strictness from config, not assumed.** Read `design.strictness` from `harness.config.json`; default to `standard` if absent.
|
|
164
|
+
|
|
165
|
+
## Escalation
|
|
166
|
+
|
|
167
|
+
- **When a project uses a non-standard component name (e.g., `PrimaryButton` instead of `Button`).** The export-name resolver won't match. Two paths: (1) the component author adds `@component-type Button` JSDoc, or (2) the project adds the file to DESIGN.md `## Component Registry`. Both unblock the audit without touching the catalog.
|
|
168
|
+
- **When the convention catalog is wrong for an opinionated project.** Add a per-component override block to DESIGN.md `## Component Anatomy Overrides`. The override wins for that component type; other catalogued types behave normally.
|
|
169
|
+
- **When pattern-presence findings have a false-positive rate >5% in a particular project.** Disable the specific pattern via `design.audit.componentAnatomy.patterns: ["-ANAT-P003"]` (negated form). Report the false positive — pattern catalog evolves based on real-world signal.
|
|
170
|
+
- **When harness-accessibility produces a finding this skill should own.** Verify `design.audit.componentAnatomy.enabled = true` and the component IS in the catalog. If both, the a11y deferral is misfiring — file an issue; likely a `getCatalogTypes()` cache staleness.
|
|
171
|
+
- **When `harness validate` runtime exceeds 3 seconds.** Set `design.audit.componentAnatomy.fastMode.maxFiles` to cap the scope, OR set `design.audit.componentAnatomy.fastMode.patterns: false` to skip pattern runs in fast mode (patterns are full-mode only by default).
|
|
172
|
+
- **When a graph operation fails.** Skip graph integration for that run; emit findings to the report only. Log a warning that `VIOLATES_CRAFT` edges were not persisted. The audit's findings are still actionable; the graph is a consumer, not a gate.
|
|
173
|
+
|
|
174
|
+
## Status
|
|
175
|
+
|
|
176
|
+
**v1 — in implementation.** See:
|
|
177
|
+
|
|
178
|
+
- Spec: `docs/changes/design-pipeline/audit-component-anatomy/proposal.md`
|
|
179
|
+
- Plan: `docs/changes/design-pipeline/audit-component-anatomy/plans/2026-05-23-audit-component-anatomy-plan.md`
|
|
180
|
+
- Finding codes: `docs/changes/design-pipeline/audit-component-anatomy/finding-codes.md`
|
|
181
|
+
- Roadmap entry: `design-pipeline sub-project #2` in `docs/roadmap.md`
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
name: audit-component-anatomy
|
|
2
|
+
version: "0.1.0"
|
|
3
|
+
description: Audit component definitions for missing required anatomy parts (slots, states, sizes) and detect missing-anatomy-component patterns (data without empty states, async without loading boundaries). First programmatic enforcer of component-anatomy rules.
|
|
4
|
+
stability: draft
|
|
5
|
+
cognitive_mode: constructive-architect
|
|
6
|
+
triggers:
|
|
7
|
+
- manual
|
|
8
|
+
- on_new_feature
|
|
9
|
+
platforms:
|
|
10
|
+
- claude-code
|
|
11
|
+
tools:
|
|
12
|
+
- Bash
|
|
13
|
+
- Read
|
|
14
|
+
- Write
|
|
15
|
+
- Edit
|
|
16
|
+
- Glob
|
|
17
|
+
- Grep
|
|
18
|
+
cli:
|
|
19
|
+
command: harness skill run audit-component-anatomy
|
|
20
|
+
args:
|
|
21
|
+
- name: path
|
|
22
|
+
description: Project root path
|
|
23
|
+
required: false
|
|
24
|
+
- name: mode
|
|
25
|
+
description: "fast (definition findings only) or full (definitions + patterns)"
|
|
26
|
+
required: false
|
|
27
|
+
- name: files
|
|
28
|
+
description: Optional glob/path scoping
|
|
29
|
+
required: false
|
|
30
|
+
mcp:
|
|
31
|
+
tool: run_skill
|
|
32
|
+
input:
|
|
33
|
+
skill: audit-component-anatomy
|
|
34
|
+
path: string
|
|
35
|
+
type: rigid
|
|
36
|
+
tier: 2
|
|
37
|
+
phases:
|
|
38
|
+
- name: scan
|
|
39
|
+
description: Identify component types and run convention/pattern audits
|
|
40
|
+
required: true
|
|
41
|
+
- name: evaluate
|
|
42
|
+
description: Apply severity model based on design.strictness
|
|
43
|
+
required: true
|
|
44
|
+
- name: report
|
|
45
|
+
description: Format findings as report + graph edges
|
|
46
|
+
required: true
|
|
47
|
+
state:
|
|
48
|
+
persistent: false
|
|
49
|
+
depends_on:
|
|
50
|
+
- design-component-anatomy
|
|
51
|
+
- harness-accessibility
|
|
@@ -0,0 +1,192 @@
|
|
|
1
|
+
# Copy Craft
|
|
2
|
+
|
|
3
|
+
> LLM-judgment critique of prose-in-code across six surfaces: error messages, log lines, CLI output strings, commit subjects, PR descriptions, and code comments. Primary domain is error messages (universally bad in most codebases). Third member of the craft-pipeline initiative. NO rule-based floor exists — pure ceiling. Emits 3-axis findings (tier × impact × confidence per ADR 0019).
|
|
4
|
+
|
|
5
|
+
## When to Use
|
|
6
|
+
|
|
7
|
+
- During PR review on code that adds or changes error messages, log lines, or CLI output
|
|
8
|
+
- After a feature ships, to audit error-message quality across the changed surfaces
|
|
9
|
+
- Periodically (per-release) to catch accumulated noise in log lines + comment rot
|
|
10
|
+
- As the user-facing-copy critic alongside design-craft (which owns UI copy)
|
|
11
|
+
- NOT for UI copy in components (use design-craft)
|
|
12
|
+
- NOT for prose documentation in `docs/` (use docs-craft #2 when it ships)
|
|
13
|
+
- NOT for autofix / rewriting (this is judgment-only; v2 may ship `align-copy`)
|
|
14
|
+
- NOT for JSDoc / TSDoc structured API docs (docs-craft territory)
|
|
15
|
+
- NOT for non-TS/JS languages in v1 (v1.x)
|
|
16
|
+
|
|
17
|
+
## Process
|
|
18
|
+
|
|
19
|
+
### Phase 1: EXTRACT — Six surfaces, three infrastructures
|
|
20
|
+
|
|
21
|
+
1. **Read project configuration.** Check `harness.config.json` for:
|
|
22
|
+
- `craft.copy.enabled` — gate (default `true`)
|
|
23
|
+
- `craft.copy.surfaces` — restrict to specific surfaces (default: all 6)
|
|
24
|
+
- `craft.copy.maxFiles` (default 100), `craft.copy.maxItemsPerFile` (default 20)
|
|
25
|
+
- `craft.copy.commitsSince` (default `'1 month ago'`), `craft.copy.prLimit` (default 20)
|
|
26
|
+
|
|
27
|
+
2. **Source-side surfaces** (errors / logs / CLI output / comments): single TS Compiler API walk per source file. Amortizes parse cost across surfaces.
|
|
28
|
+
- **errors:** `throw new <X>Error("...")` where the constructor name ends in `Error`; also `Err({ message: "..." })` for Result-style returns
|
|
29
|
+
- **logs:** `console.log/info/warn/error/debug` and `logger.X` / `log.X` / `pino.X` / `winston.X` where X is a known level
|
|
30
|
+
- **cli-output:** strings inside files under `packages/*/src/commands/` (configurable via `cliOutputPaths`); takes precedence over `log` for files matching the glob
|
|
31
|
+
- **comments:** `ts.getLeadingCommentRanges()` + `getTrailingCommentRanges()`; excludes JSDoc and license banners
|
|
32
|
+
|
|
33
|
+
3. **Git surface** (commits): shell-out to `git log --pretty=format:'%H%x09%s' --since=...`. Skip silently when not in a git repo.
|
|
34
|
+
|
|
35
|
+
4. **GitHub surface** (PR descriptions): shell-out to `gh pr list --json number,title,body`. Skip silently when `gh` binary missing or `gh auth status` fails.
|
|
36
|
+
|
|
37
|
+
5. **Skipped surfaces** recorded in `summary.skippedSurfaces` with the reason — visible in the report, not a failure.
|
|
38
|
+
|
|
39
|
+
### Phase 2: CRITIQUE — Per (item, rubric) loop, surface-filtered
|
|
40
|
+
|
|
41
|
+
8 seed rubrics, each declares which surfaces it applies to:
|
|
42
|
+
|
|
43
|
+
| Rubric | Surfaces |
|
|
44
|
+
| ------------------------------------- | ------------------------------- |
|
|
45
|
+
| `COPY-R001` WHAT/WHY/HOW-TO-FIX | error |
|
|
46
|
+
| `COPY-R002` calm-not-panicky | error, log |
|
|
47
|
+
| `COPY-R003` specific-not-generic | error, log, cli-output |
|
|
48
|
+
| `COPY-R004` signal-not-noise | log |
|
|
49
|
+
| `COPY-R005` grep-survives | log, cli-output |
|
|
50
|
+
| `COPY-R006` describes-change-not-work | commit, pr-description |
|
|
51
|
+
| `COPY-R007` stranger-in-6-months | commit, pr-description, comment |
|
|
52
|
+
| `COPY-R008` WHY-not-WHAT | comment |
|
|
53
|
+
|
|
54
|
+
For each (item, rubric) where the rubric applies to the item's surface:
|
|
55
|
+
|
|
56
|
+
1. Build prompt with rubric description + surface + context (errorType / logLevel / ref) + snippet (truncated to 1500 chars).
|
|
57
|
+
2. LLM returns fenced JSON: `null` (rubric doesn't apply / copy is fine) OR `{ tier, impact, confidence, message }`.
|
|
58
|
+
3. On non-null: emit `CopyFinding` with `cite.rubricId` for ADR 0020 traceability.
|
|
59
|
+
|
|
60
|
+
### Phase 3: REPORT — Aggregate + cost telemetry
|
|
61
|
+
|
|
62
|
+
Emit `CopyCraftOutput`:
|
|
63
|
+
|
|
64
|
+
```ts
|
|
65
|
+
{
|
|
66
|
+
findings: CopyFinding[];
|
|
67
|
+
summary: {
|
|
68
|
+
phaseRun: ['critique'];
|
|
69
|
+
durationMs: number;
|
|
70
|
+
llmCalls: { provider, model, count, costUsd };
|
|
71
|
+
catalog: { rubricsApplied: string[]; surfacesScanned: CopySurface[] };
|
|
72
|
+
counts: Record<CopySurface, number>;
|
|
73
|
+
skippedSurfaces: Array<{ surface, reason }>;
|
|
74
|
+
runId: string;
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## Harness Integration
|
|
80
|
+
|
|
81
|
+
- **`harness copy-craft`** — CLI entry. `--files` / `--surfaces` / `--max-files` / `--max-items-per-file` / `--commits-since` / `--pr-limit` / `--json` / `--verbose`.
|
|
82
|
+
- **`mcp__harness__copy_craft`** — MCP tool. Same input/output. Consumed by agents.
|
|
83
|
+
- **Cross-cutting API:** `critiqueCopyInFile(file, opts)` exported. Source-side surfaces only; git surfaces are project-scoped.
|
|
84
|
+
- **Shared craft infrastructure:** imports `LlmProvider` + 3-axis types + `derivePriority` from `packages/cli/src/shared/craft/` (extracted by spec-craft).
|
|
85
|
+
|
|
86
|
+
## Success Criteria
|
|
87
|
+
|
|
88
|
+
See `docs/changes/craft-pipeline/copy-craft/proposal.md` for the full 39 success criteria. Highlights:
|
|
89
|
+
|
|
90
|
+
- 8 seed rubrics ship in `catalog/rubrics/<id>.ts` (file-per-rubric, matches naming/spec-craft)
|
|
91
|
+
- 3-axis output preserved (tier × impact × confidence, never collapsed)
|
|
92
|
+
- `cite.rubricId` populated on every finding (ADR 0020)
|
|
93
|
+
- Single TS AST walk amortizes parse cost across 4 source surfaces
|
|
94
|
+
- Graceful degradation: `summary.skippedSurfaces` records when git/gh surfaces couldn't run
|
|
95
|
+
- Cross-cutting `critiqueCopyInFile` exported (source surfaces only)
|
|
96
|
+
|
|
97
|
+
## Examples
|
|
98
|
+
|
|
99
|
+
### Example: Generic error message
|
|
100
|
+
|
|
101
|
+
**Input:** `src/parse.ts`:
|
|
102
|
+
|
|
103
|
+
```ts
|
|
104
|
+
throw new Error('parse error');
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
**Output (mock LLM):**
|
|
108
|
+
|
|
109
|
+
```
|
|
110
|
+
[error]
|
|
111
|
+
COPY-R001 [foundational/large/medium] src/parse.ts:14 error
|
|
112
|
+
"parse error"
|
|
113
|
+
Doesn't tell WHAT was being parsed, WHY it failed, or HOW the user can
|
|
114
|
+
recover. Try: "Failed to parse design-system/tokens.json at line 12: ..."
|
|
115
|
+
COPY-R003 [polish/medium/high] src/parse.ts:14 error
|
|
116
|
+
"parse error"
|
|
117
|
+
Generic — no operation, no artifact. Name the file being parsed and the
|
|
118
|
+
specific failure mode.
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
### Example: Noisy log line
|
|
122
|
+
|
|
123
|
+
**Input:**
|
|
124
|
+
|
|
125
|
+
```ts
|
|
126
|
+
console.log('entered function');
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
**Output:**
|
|
130
|
+
|
|
131
|
+
```
|
|
132
|
+
[log]
|
|
133
|
+
COPY-R004 [foundational/medium/high] src/handler.ts:23 log
|
|
134
|
+
"entered function"
|
|
135
|
+
Pure noise — fires on every invocation; carries no state or decision.
|
|
136
|
+
Either remove or replace with a state-transition log at the relevant
|
|
137
|
+
boundary.
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
### Example: Work-not-change commit subject
|
|
141
|
+
|
|
142
|
+
**Input:** A commit with subject `"update tests"`.
|
|
143
|
+
|
|
144
|
+
**Output:**
|
|
145
|
+
|
|
146
|
+
```
|
|
147
|
+
[commit]
|
|
148
|
+
COPY-R006 [polish/medium/medium] git:abc1234 commit
|
|
149
|
+
"update tests"
|
|
150
|
+
Describes the work, not the change. A reader six months from now needs
|
|
151
|
+
to know what behaviour changed. Try: "ratchet drift threshold to 0.5%
|
|
152
|
+
after Hermes Phase 4 baseline reset" or similar.
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
### Example: Missing prerequisites — graceful skip
|
|
156
|
+
|
|
157
|
+
When not in a git repo OR `gh` is missing/unauthenticated, those surfaces silently skip and the report shows:
|
|
158
|
+
|
|
159
|
+
```
|
|
160
|
+
Skipped surfaces:
|
|
161
|
+
- commit: not a git repo
|
|
162
|
+
- pr-description: gh binary not found
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
## Gates
|
|
166
|
+
|
|
167
|
+
- **No autofix.** v2's `align-copy` may add safe rewrites.
|
|
168
|
+
- **No JSDoc / TSDoc.** docs-craft #2 territory.
|
|
169
|
+
- **No PR / review comments.** v1.x.
|
|
170
|
+
- **No commit BODY critique.** Subjects only in v1.
|
|
171
|
+
- **No B' bootstrap.** Same posture as naming/spec-craft.
|
|
172
|
+
- **No graph persistence.** Phase 1 MVP.
|
|
173
|
+
- **No non-TS/JS language support.** v1.x.
|
|
174
|
+
- **No author-attributed signals.** v1.x telemetry.
|
|
175
|
+
|
|
176
|
+
## Escalation
|
|
177
|
+
|
|
178
|
+
- **When LLM cost is too high:** drop `maxItemsPerFile` (default 20) or scope to specific surfaces with `--surfaces error`. Cost ≈ items × applicable rubrics × per-call.
|
|
179
|
+
- **When intentionally-bad test fixtures get flagged:** scope via `--files` to exclude fixtures. v1.x adds `<!-- copy-craft:skip -->` annotation + JSDoc tag.
|
|
180
|
+
- **When git surface skips though you ARE in a repo:** the walk goes up 10 levels looking for `.git`. If you're deeper than that, run from a closer cwd.
|
|
181
|
+
- **When PR surface skips with `gh auth status` failing:** run `gh auth login` first. Or scope away with `--surfaces error,log,comment`.
|
|
182
|
+
- **When `console.log` in a CLI file gets surface='log' instead of 'cli-output':** verify the file path contains `packages/cli/src/commands/` (or a similar default substring). Override with `craft.copy.cliOutputGlobs` config in v1.x.
|
|
183
|
+
|
|
184
|
+
## Status
|
|
185
|
+
|
|
186
|
+
**v1 — in implementation.** See:
|
|
187
|
+
|
|
188
|
+
- Spec: `docs/changes/craft-pipeline/copy-craft/proposal.md`
|
|
189
|
+
- Roadmap entry: `craft-pipeline sub-project #5`
|
|
190
|
+
- Sibling craft skills: `naming-craft` (#1), `spec-craft` (#6)
|
|
191
|
+
- Shared infrastructure: `packages/cli/src/shared/craft/` (extracted by spec-craft)
|
|
192
|
+
- Future: `align-copy` (FIX side, v2), docs-craft (#2, prose docs), test-craft (#3), code-craft (#4)
|