@cleocode/skills 2026.5.83 → 2026.5.86

package/skills/ct-validator/references/anti-patterns.md

@@ -0,0 +1,194 @@
+# Anti-Patterns
+
+Common failure modes when running validation tasks. These degrade the
+report's usefulness, break downstream consumers, or mask defects. Avoid
+them by following the detection cue and the remediation.
+
+## 1. The Short-Circuit Validator
+
+**Symptom.** Validator stops on the first failure and reports only that
+one. The report claims "FAIL" but lists only one issue when there are
+several.
+
+**Detection cue.** Issue count is exactly 1 but the underlying tool
+(biome, tsc, jsonschema) has multi-issue output.
+
+**Root cause.** Engine was run without `--all-errors` (AJV),
+`--exhaustive`, or equivalent. Or a try/catch wrapped the validation
+and bailed on first throw.
+
+**Fix.** Always pass the all-errors flag. For Zod, use `safeParse` and
+collect every issue from `result.error.issues`. For TypeScript, use
+`tsc -b --pretty false` and capture every line.
+
+## 2. The Soft Pass
+
+**Symptom.** Report status is `PASS` but the target has known
+unaddressed warnings.
+
+**Detection cue.** Findings table contains entries; status is PASS.
+
+**Root cause.** Status calculus violation — PASS requires zero
+critical AND ≤2 warnings (per `compliance-reports.md`). Warnings
+above threshold demote to PARTIAL.
+
+**Fix.** Apply the status calculus mechanically. PASS only when the
+table is empty (or warnings ≤2). When in doubt, demote.
+
+## 3. The Unreproducible Report
+
+**Symptom.** Six weeks later, someone tries to re-validate the same
+target. The report's claims cannot be verified.
+
+**Detection cue.** Report has no `## Trace` section or the Trace lists
+no reproducer command.
+
+**Root cause.** Reporter forgot to record the command sequence and
+tool versions used.
+
+**Fix.** Always include the reproducer. The Trace section is mandatory
+per the canonical scaffold.
+
+```markdown
+## Trace
+
+- Target: packages/cleo/src/release.ts (at 5608f75cd)
+- Tooling: biome@2.4.11, tsc@5.6.0, pnpm@9.x
+- Reproducer:
+  pnpm biome check packages/cleo/src/release.ts && \
+  pnpm exec tsc -b packages/cleo --pretty false
+```
+
+## 4. The Mode Confusion
+
+**Symptom.** A Document-mode report applies Code-mode rules, or
+vice-versa.
+
+**Detection cue.** Findings about TypeScript strictness in a spec
+review; findings about RFC 2119 in a code lint.
+
+**Root cause.** Mode was not made explicit at the start; the validator
+defaulted to its strongest familiarity.
+
+**Fix.** Declare mode in the report header. The spawn prompt or task
+body MUST set it; if absent, ask the orchestrator to clarify before
+proceeding.
+
+## 5. The Missing Severity
+
+**Symptom.** Findings listed but consumer cannot prioritize.
+
+**Detection cue.** Findings without `Critical | Warning | Suggestion`
+classification.
+
+**Root cause.** Reporter treated all findings as equal.
+
+**Fix.** Every finding MUST carry severity. When unclear, apply the
+"blocks ship?" test: yes → Critical; no → Warning; "nice to have" →
+Suggestion.
+
+## 6. The Vague Remediation
+
+**Symptom.** Issue says "fix this". Reader does not know how.
+
+**Detection cue.** `Fix:` line is missing or contains only "see above"
+/ "obvious from context" / "fix the issue".
+
+**Root cause.** Reporter ran out of energy by the time they got to
+the fix column.
+
+**Fix.** Every finding has a concrete fix step. If the fix requires
+discussion (architectural change), state that — but specifically:
+"file a task to redesign the credential rotation flow; reference
+ADR-XX-NEW once it lands".
+
+## 7. The Phantom Test
+
+**Symptom.** Protocol mode report claims REQ-NNN is verified by
+`test/foo.test.ts::case-name`. The test does not exist.
+
+**Detection cue.** Reproducer command fails with "no such test".
+
+**Root cause.** Reporter copied the spec's traceability matrix without
+re-checking that the named tests still exist (or ever existed).
+
+**Fix.** Re-run each verification command from the traceability
+matrix during validation. Flag REQs whose verification fails to
+resolve as Critical findings — the spec drifted from the implementation.
+
+## 8. The Single-File Tunnel
+
+**Symptom.** Code-mode validation runs only on the file that was
+explicitly named. Misses related files that share the violation.
+
+**Detection cue.** Report has findings in `release.ts` only; the
+same issue exists in 5 sibling files that were not checked.
+
+**Root cause.** Reporter took the task literally instead of the
+useful scope.
+
+**Fix.** Apply the validation to all related files when feasible —
+"the diff" usually means "all files touched by the task branch", not
+"only the one file the orchestrator mentioned".
+
+## 9. The Stale Schema
+
+**Symptom.** Schema-mode validation passes but downstream consumers
+still reject the data.
+
+**Detection cue.** Schema version pinned in the report does not match
+the producer's contract.
+
+**Root cause.** Schema was updated upstream; validator used a cached
+or pinned older version.
+
+**Fix.** Always resolve the schema fresh from the contract source
+(`@cleocode/contracts`). Pin the consumer/producer relationship in
+the schema's `$id` and `version` fields and check both.
+
+## 10. The Disposable Sidecar
+
+**Symptom.** JSON sidecar emitted, but it has different findings than
+the markdown report.
+
+**Detection cue.** `diff <(md-extract-findings report.md)
+<(jq '.findings' report.json)` shows mismatched data.
+
+**Root cause.** Reporter wrote the markdown manually and the JSON
+separately; they drifted.
+
+**Fix.** Generate one from the other. Write the structured data first
+(JSON), then render the markdown from it. A `--render-md` flag on the
+report generator enforces this.
+
+## 11. The Unsourced Rule
+
+**Symptom.** A finding cites a rule that does not exist in any visible
+standard.
+
+**Detection cue.** Rule reference is prose ("naming should be
+consistent") rather than a stable identifier ("AGENTS.md §Package
+Boundary").
+
+**Root cause.** Reporter applied a personal preference and labeled
+it as a rule.
+
+**Fix.** Every rule citation MUST link to a stable source:
+`AGENTS.md §X`, `ADR-NNN`, `biome.json#rules.style.Y`. If the
+preference is real but unwritten, the report SHOULD propose adding
+it to the standard (file a task) rather than enforcing it silently.
+
+## 12. The Pre-Empty Report
+
+**Symptom.** Validator runs against an empty or absent target, reports
+`PASS`.
+
+**Detection cue.** Target file is empty or does not exist; report says
+"100% compliance".
+
+**Root cause.** Engine returned zero violations because there was
+nothing to violate.
+
+**Fix.** Always sanity-check the target exists and has content before
+running validation. Empty/missing target is itself a Critical finding —
+not a clean pass.

package/skills/ct-validator/references/compliance-reports.md

@@ -0,0 +1,199 @@
+# Compliance Reports
+
+The shape, severity calculus, and downstream consumption of the
+validation report. The report is the validator's product — its format
+is rigid because `ct-ivt-looper`, `ct-release-orchestrator`, and HITL
+gates all parse it.
+
+## Canonical Report Scaffold
+
+Every report MUST use this structure. Sections are mandatory; their
+content varies by mode.
+
+```markdown
+# Validation Report: {{VALIDATION_TARGET}}
+
+**Mode**: schema | code | document | protocol
+**Date**: {{DATE}}
+**Validator**: ct-validator v2.0.0
+
+---
+
+## Summary
+
+- **Status**: PASS | PARTIAL | FAIL
+- **Compliance**: {X}%
+- **Critical Issues**: {N}
+- **Warnings**: {N}
+- **Suggestions**: {N}
+
+## Checklist Results
+
+| Check | Status | Details |
+|-------|--------|---------|
+| {CHECK_1} | PASS/FAIL | {Details} |
+| {CHECK_2} | PASS/FAIL | {Details} |
+
+## Issues Found
+
+### Critical
+{List or "None"}
+
+### Warnings
+{List or "None"}
+
+### Suggestions
+{List or "None"}
+
+## Remediation
+
+{Required fixes if FAIL/PARTIAL, or "No remediation required" if PASS}
+
+## Trace
+
+- Target: {file/spec/diff path}
+- Tooling: {versions of tools run}
+- Reproducer: {one command that re-runs this report}
+```
+
+The Trace section is essential — without a reproducer, the report
+cannot be re-run later to confirm a remediation worked.
+
+## Status Calculus
+
+| Mode | PASS | PARTIAL | FAIL |
+|------|------|---------|------|
+| Schema | 0 violations | (n/a) | ≥1 violation |
+| Code | 0 critical, ≤2 warnings | 0 critical, 3+ warnings | ≥1 critical |
+| Document | All required sections present, 0 broken links | 1-2 missing sections | 3+ missing sections OR broken refs |
+| Protocol | 100% REQ compliance | 70-99% compliance | <70% compliance |
+
+The status is not a vote — it is a computed function of the findings.
+Two validators running on the same target MUST produce the same
+status.
+
+PARTIAL is reserved for cases where the target is shippable WITH
+recorded remediation; FAIL means the target cannot ship as-is.
+
+## Severity Definitions
+
+| Severity | Meaning | Examples |
+|----------|---------|----------|
+| Critical | Blocks ship; data corruption, security, AGENTS.md rejection | `any` type, missing required field, broken link in spec |
+| Warning | Should fix soon; degrades quality but does not block | Long body, unused import, missing optional section |
+| Suggestion | Improvement opportunity; no obligation | Extract helper, rename for clarity |
+
+Each finding MUST carry a severity. Findings without severity are
+unactionable — the consumer cannot prioritize.
+
+## Per-Finding Format
+
+```markdown
+### Critical
+
+**C-001**: `packages/cleo/src/dispatch.ts:42` uses `any` type.
+- Rule: AGENTS.md Type Safety §1 — "NEVER use `any` type"
+- Fix: Import the relevant type from `@cleocode/contracts` or
+  define a narrower union.
+- Verification: `pnpm biome check packages/cleo/src/dispatch.ts`
+
+**C-002**: `packages/cleo/src/release.ts:107` catches `err: unknown`.
+- Rule: AGENTS.md Type Safety §5 — "NEVER use `catch (err: unknown)`"
+- Fix: Throw and catch by class from `@cleocode/contracts/errors`.
+- Verification: Replace with `catch (err)` and pattern-match by
+  `instanceof`.
+
+### Warnings
+
+**W-001**: `packages/cleo/src/dispatch.ts` is 487 lines (warn at 400).
+- Rule: ct-skill-validator audit_body §body-length
+- Fix: Extract sub-handlers into sibling files.
+- Verification: re-run audit_body.py.
+
+### Suggestions
+
+**S-001**: `packages/cleo/src/dispatch.ts:120` has a TODO with no task ID.
+- Rule: Internal — TODOs should reference task IDs.
+- Fix: File a task, attach the ID: `// TODO(T9999): ...`.
+```
+
+Each finding gets a stable ID (`C-NNN`, `W-NNN`, `S-NNN`) so subsequent
+reports can reference whether the same issue persists.
+
+## Compliance Percentage
+
+For modes that report a percentage:
+
+```text
+compliance = (passed_checks / total_checks) × 100
+```
+
+Round to one decimal. For Protocol mode (REQ compliance), checks are
+the REQs in the spec's traceability matrix. For Schema mode, "checks"
+is the count of constraints evaluated.
+
+When the denominator is zero (e.g., a spec with no REQs), the
+percentage is undefined — emit `N/A` and report a Critical finding for
+the empty spec.
+
+## Integration with ct-ivt-looper
+
+The IVT loop reads the validator's `status` and `compliance` fields
+to decide whether to continue iterating. The contract:
+
+- `status: PASS` → IVT loop converges; release-orchestrator may proceed.
+- `status: PARTIAL` → IVT loop loops once more with the remediation as
+  input; if still PARTIAL on iteration 3, escalates to HITL.
+- `status: FAIL` → IVT loop blocks; release-orchestrator MUST not
+  proceed; HITL escalation immediate.
+
+This contract is the reason status calculus is rigid — fuzzy statuses
+break the loop's termination conditions.
+
+## Integration with Release
+
+`ct-release-orchestrator` reads the most recent validation report
+for the epic before allowing ship. The release pipeline (ADR-065)
+contains a gate that asserts `status: PASS` on the implementation
+against the spec. PARTIAL reports trigger a recorded remediation
+plan; FAIL reports block.
+
+## Machine-Readable Sidecar
+
+Always emit a JSON sidecar next to the markdown report:
+
+```json
+{
+  "target": "packages/cleo/src/release.ts",
+  "mode": "code",
+  "status": "FAIL",
+  "compliance": 88.5,
+  "findings": {
+    "critical": [
+      {"id": "C-001", "file": "...", "line": 42, "rule": "...", "fix": "..."},
+      {"id": "C-002", "file": "...", "line": 107, "rule": "...", "fix": "..."}
+    ],
+    "warnings": [...],
+    "suggestions": [...]
+  },
+  "trace": {
+    "tooling": ["biome@2.4.11", "tsc@5.6.0"],
+    "reproducer": "pnpm biome check . && pnpm exec tsc -b",
+    "timestamp": "2026-05-19T19:35:00Z"
+  }
+}
+```
+
+The orchestrator parses this when programmatic decisions are needed;
+the human report (.md) is for review.
+
+## Anti-Patterns
+
+| Anti-pattern | Why it fails |
+|--------------|--------------|
+| Vague finding ("file looks weird") | Cannot remediate |
+| Missing severity | Consumer cannot prioritize |
+| No reproducer in trace | Cannot re-validate after fix |
+| PARTIAL with no remediation plan | Status calculus violation |
+| Findings without rule reference | Disputable, not actionable |
+| Markdown only, no JSON sidecar | Breaks orchestrator integration |

package/skills/ct-validator/references/schema-checking.md

@@ -0,0 +1,191 @@
+# Schema Checking
+
+Deeper guidance for Schema Validation mode (Mode 1 from
+`validation-modes.md`). Schema checking is the most mechanical of the
+four modes — but only if the inputs are correctly configured. This
+reference covers the configuration pitfalls and tool-selection rules.
+
+## Pick the Right Engine
+
+| Engine | Strength | When to use |
+|--------|----------|-------------|
+| AJV (`ajv-cli`) | Fast; draft-07 / 2019-09 / 2020-12 | Generic JSON Schema validation |
+| Zod | Type-safe; integrates with TypeScript | Project-internal data validation |
+| drizzle-orm/zod | Schema → Zod auto-generation | Drizzle ORM consumers |
+| jsonschema (Python) | Cross-language; same draft support | Python-side validation |
+| `cargo schema` / serde | Type-safe; Rust-side | Rust crates |
+
+The CLEO repo's primary engine is Zod (via `@cleocode/contracts`) — most
+internal schemas are exported as Zod schemas with optional drizzle
+codegen. Use Zod's `safeParse` to get structured errors.
+
+For external contracts (LAFS envelope spec, JSON-RPC requests from
+non-CLEO clients), use AJV or jsonschema for cross-language portability.
+
+## Draft Selection
+
+Always state the draft explicitly. Draft-07 and draft-2020-12 have
+incompatible semantics around `$ref`, `if/then/else`, and tuple arrays.
+
+| Draft | Best for | Caveats |
+|-------|----------|---------|
+| draft-04 | Legacy | `id` not `$id`; many engines deprecated this |
+| draft-07 | Industry standard | Most engines support fully |
+| draft-2019-09 | New work, conservative | `$defs` replaces `definitions` |
+| draft-2020-12 | New work, full | Some engines lag |
+
+CLEO contracts target draft-07 unless the schema needs 2020-12-only
+features (typically `prefixItems` for tuple arrays). Add the explicit
+`"$schema"` field on every published schema:
+
+```json
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "title": "TaskEnvelope",
+  "type": "object",
+  "required": ["success", "meta"],
+  ...
+}
+```
+
+## The Standard Validation Loop
+
+```bash
+# 1. Choose engine; choose draft
+ENGINE=ajv
+DRAFT=draft-07
+
+# 2. Resolve schema file (must include $schema or pass --spec)
+SCHEMA=packages/contracts/schemas/envelope.json
+
+# 3. Iterate over data files
+for DATA in path/to/*.json; do
+  echo "=== $DATA ==="
+  npx ajv validate -s "$SCHEMA" -d "$DATA" --spec=$DRAFT --all-errors
+done
+
+# 4. Aggregate by file in the report
+```
+
+The `--all-errors` flag (AJV) or `--exhaustive` (Zod's `safeParse` does
+this by default) is essential — the spec ground rule is "report every
+violation", not "stop at the first one".
+
+## Pitfalls
+
+### 1. $ref Resolution Failure
+
+**Symptom.** Validator complains "could not resolve $ref" or "unknown
+keyword $ref".
+
+**Cause.** Schema references another file by relative path; the engine
+does not know the base directory.
+
+**Fix.** Pass `--ref` (AJV) pointing at the referenced files, or inline
+the references with `$defs`. For multi-file schemas, use AJV's
+`addSchema()` API or a bundler step that flattens before validation.
+
+### 2. Implicit Type Coercion
+
+**Symptom.** A string `"123"` validates against `{"type": "number"}`.
+
+**Cause.** Engine has coercion enabled (`coerceTypes: true` in AJV).
+
+**Fix.** Disable coercion for strict validation:
+
+```javascript
+new Ajv({ coerceTypes: false, strict: true, allErrors: true })
+```
+
+CLEO contracts are strict by default — incoming data must already be
+typed correctly.
+
+### 3. Missing required not flagged
+
+**Symptom.** A required field is missing; schema says it's required;
+validator passes.
+
+**Cause.** The schema's `required` is at the wrong level (nested vs
+top), or the validator is in non-strict mode.
+
+**Fix.** Re-check the schema structure. `required` must be a peer of
+`properties` at the same nesting level.
+
+### 4. Additional Properties Silently Allowed
+
+**Symptom.** Data has extra fields not in the schema; validator passes.
+
+**Cause.** Schema does not specify `additionalProperties: false`.
+
+**Fix.** CLEO contracts default to `additionalProperties: false` — extra
+fields indicate either drift in the producer or a malicious payload.
+Override only with an inline justification.
+
+## Reporting Schema Violations
+
+Each violation goes in the report as:
+
+```markdown
+### Violations
+
+| Path | Keyword | Actual | Expected | File |
+|------|---------|--------|----------|------|
+| `/data/items/3/id` | required | (absent) | string | manifest-instance.json |
+| `/data/version` | type | "1.2.3" (string) | number | manifest-instance.json |
+| `/meta/timestamp` | format | "today" | date-time | manifest-instance.json |
+| `/error.code` | enum | "E_WUT" | one of [E_NOT_FOUND, E_VALIDATION, ...] | response.json |
+```
+
+The Path column uses JSON Pointer (RFC 6901). The Keyword column names
+the schema constraint violated. Actual + Expected gives the diff
+needed to fix.
+
+## Bulk Schema Validation
+
+When validating many instances (e.g., every manifest file in
+`.cleo/agent-outputs/`), produce a summary table first, then drill
+into failures.
+
+```markdown
+## Summary
+
+- **Files validated**: 47
+- **Pass**: 42
+- **Fail**: 5
+- **Compliance**: 89.4%
+
+## Failed Files
+
+| File | Violations |
+|------|-----------|
+| `.cleo/agent-outputs/2026-05-19_old.md` | 3 |
+| `.cleo/agent-outputs/2026-05-19_draft.md` | 1 |
+| ... | ... |
+
+## Detail (per-file)
+
+### `.cleo/agent-outputs/2026-05-19_old.md`
+
+| Path | Keyword | Actual | Expected |
+|------|---------|--------|----------|
+| ... | ... | ... | ... |
+```
+
+Bulk reports SHOULD be machine-readable. Emit a sibling JSON report
+under the same name when the consumer is automation.
+
+## Self-Test the Schema
+
+Before trusting any schema for validation, verify it accepts known-good
+instances and rejects known-bad instances. Maintain a `fixtures/`
+directory next to the schema:
+
+```text
+packages/contracts/schemas/envelope.json
+packages/contracts/schemas/fixtures/envelope-valid.json
+packages/contracts/schemas/fixtures/envelope-missing-success.json
+packages/contracts/schemas/fixtures/envelope-wrong-type.json
+```
+
+A test that exercises all three confirms the schema does what it
+claims. Without fixtures, schema drift goes undetected.