qualia-framework 6.3.0 → 6.5.0

package/docs/reviews/matt-pocock-skills-analysis.md

@@ -1,300 +0,0 @@
-# Matt Pocock's Skills — Deep Analysis & Qualia 5.0 Proposal
-
-**Source material**
-- Repo: https://github.com/mattpocock/skills (46k stars, #1 on GitHub Trending the day it dropped)
-- Course: https://www.aihero.dev/cohorts/claude-code-for-real-engineers-2026-04
-- Talk: https://www.youtube.com/watch?v=kZ-zzHVUrO4
-
-**Author of this analysis:** Synthesized 2026-05-03 from a full read of Matt's CLAUDE.md, CONTEXT.md, and every SKILL.md in `engineering/`, `productivity/`, and `misc/`.
-
----
-
-## Part 1 — Matt's worldview, distilled
-
-Matt is teaching one thing: **the real bottleneck on AI-coding quality is not the model, it's the alignment between human, agent, code, and domain.** Every skill exists to fix one alignment failure.
-
-### The four pillars (from his CLAUDE.md)
-
-| Pillar | Source | What it fixes |
-|---|---|---|
-| **Alignment Before Coding** | Pragmatic Programmer — "no-one knows exactly what they want" | Misalignment is the #1 failure mode. Cure: relentless one-question grilling until every branch of the decision tree is resolved. |
-| **Domain Language Matters** | DDD ubiquitous language | Shared vocabulary reduces tokens, improves agent navigation, prevents "what do you mean by user?" rework. |
-| **Feedback Loops Drive Quality** | Pragmatic Programmer — "the rate of feedback is your speed limit" | Without a fast deterministic pass/fail signal, agents fly blind. Cure: build the loop FIRST, then debug. |
-| **Design Discipline** | Ousterhout — "the best modules are deep" | Daily attention to architecture or entropy wins. Cure: deepening refactors framed in domain language. |
-
-### The structural innovations
-
-1. **CONTEXT.md as a domain glossary** — not a project README, not a spec. A glossary of terms with **`Avoid`** lists. The agent literally reads "use *milestone*, not *sprint*" and conforms. Reduces token waste from disambiguation rounds.
-2. **ADRs (Architecture Decision Records)** in `docs/adr/` — created sparingly, only for hard-to-reverse, surprising-without-context decisions with real trade-offs. Becomes the load-bearing memory.
-3. **Vertical slices everywhere** — both for issues (each issue = a complete path through schema → API → UI → tests) and for TDD (one test → one impl → repeat). Anti-horizontal as a **rule**.
-4. **Skill descriptions are the API surface** — "the only thing your agent sees." Discriminative descriptions beat clever prose. <1024 chars. Third-person. Always include "Use when {context}".
-5. **Skills under 100 lines** — overflow goes to `REFERENCE.md` / `EXAMPLES.md` / `scripts/` (progressive disclosure).
-6. **`disable-model-invocation: true`** — some skills (e.g. `zoom-out`) are user-fired-only. Prevents auto-firing on weak signals.
-7. **scripts/ co-located with skills** — for deterministic operations. "Trade tokens for reliability" — the script can't hallucinate.
-
-### The 21 skills, mapped
-
-| Bucket | Skill | One-liner | Maps to (Qualia equivalent) |
-|---|---|---|---|
-| eng | **diagnose** | Build feedback loop FIRST, then reproduce → hypothesize → instrument → fix → regression-test | `qualia-debug` (weaker — doesn't enforce loop-first) |
-| eng | **grill-with-docs** | Interview the user, update CONTEXT.md/ADRs inline as decisions crystallize | `qualia-discuss` (much weaker — no glossary update) |
-| eng | **improve-codebase-architecture** | Find shallow modules, propose deepening refactors, use domain language | `qualia-optimize` (broader but lacks Ousterhout language) |
-| eng | **tdd** | Vertical-slice tracer-bullet loop. ANTI horizontal slicing (writing all tests then all code) | `qualia-test` (lacks the methodology rule) |
-| eng | **to-prd** | Conversation → PRD → GH Issue with `needs-triage` label | `qualia-new` (stays internal; never touches GH issues) |
-| eng | **to-issues** | PRD → independent vertical-slice GH Issues, dependency-ordered | **NO EQUIVALENT** |
-| eng | **triage** | State machine: needs-triage / needs-info / ready-for-agent / ready-for-human / wontfix | **NO EQUIVALENT** (we use tracking.json status fields, not GH labels) |
-| eng | **zoom-out** | "Map this area using the project glossary" — 5-line skill, user-only | **NO EQUIVALENT** |
-| eng | **setup-matt-pocock-skills** | Detect repo state, write `docs/agents/{tracker,labels,domain}.md`, append to CLAUDE.md | `qualia-map` (broader, but doesn't externalize agent config) |
-| prod | **caveman** | 75% token reduction via dropped articles/fillers, fragment sentences, abbreviations, arrows | **NO EQUIVALENT** |
-| prod | **grill-me** | Aggressive one-question-at-a-time interview until decision tree resolves | partial overlap with `qualia-discuss` |
-| prod | **write-a-skill** | Meta-skill for authoring skills — frontmatter rules, file split rules | `qualia-skill-new` (similar) |
-| misc | **git-guardrails** | Block dangerous git commands | partial — we have hooks |
-| misc | **migrate-to-shoehorn** | Codemod-y tool | n/a |
-| misc | **scaffold-exercises** | Course-specific | n/a |
-| misc | **setup-pre-commit** | Husky setup | n/a |
-
----
-
-## Part 2 — Where Qualia is already strong
-
-We should NOT abandon what's working:
-
-1. **Phase/wave/task hierarchy** — Matt has nothing equivalent to `JOURNEY.md → ROADMAP.md → phase plans → tasks with verification contracts`. This is a Qualia advantage.
-2. **Fresh-context subagent spawning** — task-level context isolation prevents rot. Matt mentions subagents conceptually; we do it systematically.
-3. **The verifier loop with goal-backward checks** — `qualia-verify` is sharper than anything Matt ships.
-4. **Design substrate (v4.5.0)** — DESIGN.md + design-laws.md + slop-detect + 8-dimension scoring. Matt has nothing on visual design.
-5. **Smart router (`qualia` skill)** — state-driven next-action recommendation. Matt's flow is more linear.
-6. **The 4 mandatory Handoff deliverables** — client-delivery rigor that doesn't exist in Matt's repo (he targets engineers building in their own repos, not agencies shipping to clients).
-
-The combination of Matt's alignment discipline + Qualia's execution machinery is the play.
-
----
-
-## Part 3 — Qualia 5.0 proposal: 15 changes ranked by ROI
-
-### Tier 1 — Adopt these immediately (high ROI, low effort)
-
-#### 1. **CONTEXT.md substrate (domain glossary)** — `.planning/CONTEXT.md`
-
-The single biggest miss. Every Qualia agent currently re-derives domain terms from PROJECT.md prose. Matt's pattern: a structured glossary with `Avoid` lists.
-
-**Format:**
-```markdown
-## Language
-
-### Milestone
-A bounded slice of the journey. Always one of: Foundation, MVP, Polish, Handoff.
-**Avoid:** sprint, iteration, release.
-
-### Phase
-A unit of work inside a milestone, 2–5 tasks, ends in a verification gate.
-**Avoid:** epic, story, ticket.
-
-### Task
-A single commit-sized unit with one verification contract.
-**Avoid:** subtask, chore.
-
-## Relationships
-- Project holds many Milestones
-- Milestone holds many Phases
-- Phase holds many Tasks
-- Task carries one Verification Contract
-
-## Flagged ambiguities
-- "User" was previously used for both `auth.users` row and customer profile. Now: **AuthUser** (auth row), **Customer** (profile).
-```
-
-- Generated by `/qualia-new` from the discovery questioning
-- Loaded **first** by every subagent (before PROJECT.md, before DESIGN.md)
-- Updated inline by `/qualia-discuss` and the new `/qualia-grill`
-- **Expected impact:** ~30% reduction in disambiguation back-and-forth, fewer wrong-assumption rework cycles
-
-#### 2. **ADR pattern** — `.planning/decisions/ADR-{N}-{slug}.md`
-
-Replace the ad-hoc `phase-{N}-context.md` with structured ADRs. Created **sparingly**: only when the decision is hard-to-reverse, surprising-without-context, AND involves real trade-offs.
-
-**Format:**
-```markdown
-# ADR-007 — Use Supabase RLS instead of API-layer auth checks
-Date: 2026-05-03
-Phase: 2
-Status: Accepted
-
-## Context
-{why this decision is being made now}
-
-## Decision
-{what we are doing}
-
-## Consequences
-{what becomes easier; what becomes harder}
-
-## Alternatives considered
-{what we rejected and why}
-```
-
-- Loaded by relevant phases via `@.planning/decisions/ADR-007-*.md`
-- Listed in CONTEXT.md "Flagged ambiguities" when they resolve a term
-
-#### 3. **`/qualia-zoom`** — 5-line user-only skill
-
-```yaml
----
-name: qualia-zoom
-description: Zoom out and map the surrounding modules using project glossary terms. Use when you don't know an area of code well or need to orient before editing.
-disable-model-invocation: true
----
-
-I don't know this area well. Go up a layer of abstraction. Give me a map of all relevant modules and callers using the vocabulary in `.planning/CONTEXT.md`.
-```
-
-Tiny. Used 50× a day in mature projects. Replaces ad-hoc "explain this code" requests with a glossary-anchored response.
-
-#### 4. **Caveman mode for subagent spawning** (cost reduction)
-
-Add to the prompt-cache template in `rules/grounding.md`:
-
-> **Subagent compression**: All in-flight spawned-subagent system prompts use compressed form — drop articles, fillers, pleasantries. Use abbreviations (DB, auth, fn, impl, req, res). Fragment sentences over full clauses. Arrow notation for causality. Suspend compression for security warnings, irreversible action confirmations, and multi-step sequences where brevity risks misinterpretation.
-
-The verifier and builder spawn 50+ times per project. ~25% token reduction per spawn. Real money at scale.
-
-#### 5. **Tighten skill descriptions** (the API surface)
-
-Audit all 74 skills. Each description must:
-- Be ≤ 1024 chars
-- Be in third person
-- Include "Use when {specific context}" with **discriminative** triggers
-- Have **zero overlap** with sibling skills
-
-Currently `/qualia`, `/qualia-idk`, `/qualia-resume` all overlap on "lost? what next?" triggers. Wrong-skill-firing happens. Fix it.
-
-### Tier 2 — Adopt these next (high ROI, medium effort)
-
-#### 6. **`/qualia-grill`** — replace or augment `/qualia-discuss`
-
-Aggressive one-question-at-a-time grilling. Walks every branch of the decision tree. Updates CONTEXT.md inline.
-
-**Critical rule from Matt's grill-me:** *"For each question, provide your recommended answer."* — the agent isn't just asking, it's proposing. The user accepts, edits, or rejects. Way faster than open-ended interview.
-
-Used **before** `/qualia-plan` for high-stakes phases (auth, payments, multi-tenant, anything with regulatory weight).
-
-#### 7. **Restructure `/qualia-debug` around feedback-loop-first**
-
-Current `qualia-debug` jumps to symptom analysis. Matt's diagnose enforces **build the loop first, then everything else is mechanical**. New phase structure:
-
-1. Build a fast deterministic agent-runnable pass/fail signal (failing test > curl > CLI invocation > headless browser > trace replay > minimal harness > property test > bisection > differential > human-in-loop bash). Pick the highest-tier feasible.
-2. Reproduce against the loop
-3. Generate 3–5 ranked falsifiable hypotheses BEFORE testing
-4. Instrument with `[DEBUG-{tag}]` prefixes (cleanup later)
-5. Change one variable at a time
-6. Fix at the correct seam, write regression test there
-7. Cleanup `[DEBUG-*]` markers, verify scenario, document architectural finding answering "what would have prevented this?"
-
-#### 8. **`/qualia-deepen`** — split out from `/qualia-optimize`
-
-`/qualia-optimize` is currently a kitchen sink (perf + UI + backend + alignment). Pull out architecture-deepening as its own skill that uses Ousterhout's vocabulary explicitly: **depth, locality, leverage, seam, deletion test**.
-
-Process:
-1. Read `.planning/CONTEXT.md` glossary first
-2. Walk codebase noting friction points: shallow modules where interface ≈ implementation, modules requiring bouncing across many files for one concept, pure functions extracted only for testability (no locality), tightly-coupled cross-seam leakage
-3. Present candidates with **Files / Problem / Solution / Benefits**
-4. Grilling loop: walk the design tree with the user, update CONTEXT.md/ADRs as decisions crystallize
-
-#### 9. **`/qualia-tdd`** — vertical-slice rule enforcement
-
-```
-WRONG (horizontal):  RED test1-5 → GREEN impl1-5
-RIGHT (vertical):    RED→GREEN test1→impl1, test2→impl2, ...
-```
-
-One test at a time. Only enough code to pass. Tests describe **behavior through public interface**, never implementation. Never refactor while red.
-
-This is the #1 missing engineering discipline in the framework. We have `/qualia-test` (generate tests) but no test-first methodology.
-
-#### 10. **Skill file structure cleanup** (progressive disclosure)
-
-Audit pass: every SKILL.md must be **<100 lines**. Overflow goes to:
-- `REFERENCE.md` (advanced/rare features)
-- `EXAMPLES.md` (worked examples)
-- `scripts/` (deterministic operations co-located with the skill)
-
-Move `bin/state.js` and `bin/qualia-ui.js` from `~/.claude/bin/` into `skills/qualia/scripts/` (or symlink). The script being co-located with the skill means it's discoverable when reading the skill.
-
-### Tier 3 — Strategic bets (very high ROI, larger effort)
-
-#### 11. **`/qualia-issues`** — externalize work to GitHub
-
-Convert the current phase plan into independent vertical-slice GH issues. Each issue:
-- Title describing the slice
-- End-to-end behavior description
-- Acceptance criteria checklist
-- Blocking-dependencies field
-- Label: `needs-triage`
-
-This **unlocks the autonomous loop**: other agents (or other Claude sessions, or Codex, or human contributors) can pull issues from the queue. Currently Qualia only runs in the originating session.
-
-#### 12. **`/qualia-triage`** — state machine over the issue queue
-
-Pulls open issues. Applies labels: `needs-triage` / `needs-info` / `ready-for-agent` / `ready-for-human` / `wontfix`. Routes:
-- `ready-for-agent` → autonomous `/qualia-build` or `/qualia-quick`
-- `ready-for-human` → notification to Fawzi
-- `needs-info` → questions back to reporter
-
-Combined with `/qualia-issues`, this is the **Ralph Wiggum loop** Matt teaches: agent pulls from backlog, builds, verifies, ships, picks next. Human-in-loop is optional, not required.
-
-#### 13. **AGENTS.md emission** alongside CLAUDE.md
-
-`/qualia-new` should write **both** `CLAUDE.md` (Anthropic) and `AGENTS.md` (the cross-vendor open standard now adopted by Codex, Cursor, Continue, Devin, Aider, etc.). Same content, different file. One project, multi-agent compatible. Costs nothing, expands the ecosystem.
-
-#### 14. **`/qualia-onboard`** — adapt-to-existing-repo skill
-
-Matt's `setup-matt-pocock-skills` is the model. Companion to `/qualia-map`. Detects:
-- Existing issue tracker (GH, GL, Jira, local `.scratch/`)
-- Existing labels — maps them to canonical roles
-- Existing CONTEXT.md / docs/adr / glossary structure
-
-Writes `.planning/agents/{tracker.md, labels.md, domain.md}` and appends an `## Agent skills` block to existing CLAUDE.md/AGENTS.md. **Adapts Qualia to the repo's conventions** instead of forcing Qualia conventions on the repo.
-
-This is what makes the framework usable on brownfield client projects without ripping their existing process out.
-
-#### 15. **`disable-model-invocation` flag adoption**
-
-Add to skills that should never auto-fire on a weak trigger: `qualia-zoom`, `qualia-caveman`, anything destructive (`qualia-ship`, `qualia-handoff` arguably). Reduces accidental invocations.
-
----
-
-## Part 4 — Sequencing recommendation
-
-### Wave 1 (immediate, ~1 day)
-- 1, 2, 3, 4, 5 (CONTEXT.md, ADRs, qualia-zoom, caveman mode, description audit)
-
-### Wave 2 (short, ~3 days)
-- 6, 7, 8, 9, 10 (qualia-grill, debug restructure, qualia-deepen, qualia-tdd, file structure cleanup)
-
-### Wave 3 (strategic, ~1 week)
-- 11, 12 (qualia-issues, qualia-triage) — these together unlock autonomous operation
-
-### Wave 4 (polish, ~2 days)
-- 13, 14, 15 (AGENTS.md, qualia-onboard, disable-model-invocation)
-
-### Ship as v5.0
-- New release: "Qualia 5.0 — alignment discipline" — leans into Matt's framing
-
----
-
-## Part 5 — What we should NOT copy
-
-- **Matt's flat skill list (no router)**. Our `/qualia` smart router and state machine are better for non-trivial projects.
-- **Matt's lack of design discipline**. Our v4.5.0 design substrate is genuinely ahead. Don't dilute it.
-- **Matt's per-project `.scratch/` dir as default tracker**. We need real GH issues for client work and ERP integration.
-- **Matt's "scaffold-exercises"** etc. — course-specific, not relevant.
-
----
-
-## Part 6 — The honest assessment
-
-Matt is teaching **alignment as the primary engineering discipline**. The framework currently treats alignment as a one-shot at `/qualia-new` and assumes it holds for the rest of the project. It doesn't. Decisions drift. Terms overload. Hypotheses go untested. Feedback loops degrade.
-
-The 15 changes above are not features — they're **rituals** that keep alignment from rotting between phases. That's the x20 unlock. Not more skills. Better-disciplined skills, anchored to a domain glossary, with feedback loops built first.
-
-The single highest-ROI change is **#1 (CONTEXT.md)**. If you do nothing else from this list, do that one. Every other change compounds off it.