@kiwidata/grimoire 0.1.3 → 0.1.5

package/skills/references/pattern-guard.md

@@ -0,0 +1,180 @@
+# Pattern Guard Reference
+
+Loaded by `grimoire-apply` and `grimoire-bug`. Run **before writing the test** for each task — not after, not as a review pass. The goal is to write code that matches the codebase's established conventions the first time, rather than writing to generic patterns and fixing divergence later.
+
+This is not a quality checklist. It is a reconnaissance step: find out how this codebase already solves this class of problem, then write to that pattern.
+
+Requires `codebase-memory-mcp` indexed. If the graph is not available, skip this reference entirely and rely on `code-quality.md` alone.
+
+---
+
+## Run Before Each Task
+
+### Step 1 — Classify the code being written
+
+From the task description and feature file, identify what category of code this task produces:
+
+| Code type | Examples |
+|---|---|
+| `api_handler` | Route handler, view, controller, endpoint function |
+| `service` | Use case, interactor, domain service, business logic function |
+| `repository` | Data access, ORM query, store method, DAO |
+| `model` | ORM model, dataclass, schema, type definition |
+| `utility` | Helper, formatter, validator, parser |
+| `test` | Step definition, unit test, fixture, factory |
+| `middleware` | Auth, logging, rate-limiting, request transform |
+| `integration` | External API client, webhook handler, adapter |
+
+A task may touch multiple types — classify the primary one.
+
+### Step 1b — Reuse discovery
+
+Before finding peer patterns, ask: **does what I'm about to write already exist?**
+
+These are two different questions. Step 2 finds code to *pattern-match against*. This step finds code to *call instead of writing*.
+
+For each function, helper, or class the task requires, run both searches:
+
+**Semantic search** — find it by concept, not by name:
+```
+search_graph(semantic_query=["<primary_concept>", "<action_verb>", "<domain_noun>"])
+```
+Example: about to write something that formats a currency amount →
+```
+search_graph(semantic_query=["format", "currency", "amount"])
+```
+This finds `render_price`, `display_amount`, `format_currency` — whatever name the codebase already uses.
+
+**Name-pattern search** — find it by likely prefix or suffix:
+```
+search_graph(name_pattern="(format_|_format|currency|amount|price)")
+```
+
+**Decision rules:**
+- Result does the job → **call it**. Do not re-implement.
+- Result almost fits → **use it directly**. Do not generalize it for a second case that doesn't exist yet.
+- Both searches return nothing usable → write new code and proceed to Step 2.
+
+**Log the outcome in the pattern brief** (Step 4): note which searches ran and what they found. If calling an existing function instead of writing new code, note it explicitly: `Reused format_currency from billing/utils.py — no new function needed.`
+
+Do not skip this step. Writing new code without a reuse search is the primary source of duplication in LLM-generated codebases. The semantic_query mode bridges vocabulary gaps — it finds "publish" when you search "send".
+
+### Step 2 — Find peer examples
+
+Use `search_graph` to find 3–5 existing functions/classes of the same type. Prefer the most established (oldest, least recently changed) — these are the modal pattern, not the recent drift.
+
+**Queries by code type:**
+
+```
+api_handler:   search_graph(label="Function", name_pattern="(handle|view|endpoint|get_|post_|put_|delete_|patch_)")
+service:       search_graph(label="Function", name_pattern="(service|use_case|create_|update_|delete_|process_)")
+repository:    search_graph(label="Function", name_pattern="(repo|get_by|find_by|list_|save_|delete_)")
+model:         search_graph(label="Class", name_pattern="(Model|Schema|DTO|Type|Entity)")
+utility:       search_graph(label="Function", name_pattern="(parse_|format_|validate_|convert_|build_)")
+test:          search_graph(label="Module", name_pattern="(test_|_test|spec|_spec|conftest|fixture)")
+middleware:    search_graph(label="Function", name_pattern="(middleware|guard|interceptor|filter|auth)")
+integration:   search_graph(label="Class", name_pattern="(Client|Adapter|Gateway|Connector|Webhook)")
+```
+
+If the query returns > 10 results, filter to the same area as the task's target file (check area docs or directory).
+
+Exclude files changed in the last 60 days from your sample — those may already be drifted. Use `git log --since="60 days ago" --name-only --format=` to get the recent list.
+
+If < 3 peers exist in the graph, skip the pattern brief — there's no established pattern yet. Write to `code-quality.md` rules and the feature spec only.
+
+### Step 3 — Extract the modal pattern
+
+`get_code_snippet(qualified_name)` for each peer. Read across all samples and identify:
+
+**Four critical seams** (these are the ones that cause architectural drift if broken):
+
+1. **Error handling** — Does this codebase raise exceptions or return result/error values at this layer? Do handlers catch specific exception types? Is there a central error handler or per-function handling?
+
+2. **Dependency access** — Are dependencies injected (constructor, function arg) or imported directly? Is there an established pattern (DI container, FastAPI `Depends`, Django `self.repository`, etc.)?
+
+3. **Abstraction depth** — Does this code type contain business logic, or does it delegate? (e.g., handlers should be thin, services should be thick — but check what *this* codebase actually does)
+
+4. **Return shape** — Dict? Typed dataclass/schema? Model instance? Tuple `(result, error)`? Pydantic model? Match exactly.
+
+**Three secondary seams** (style drift, not architecture):
+
+5. **Naming** — snake_case vs camelCase beyond language default, verb-first vs noun-first for functions, consistent abbreviation patterns
+
+6. **Test structure** — `pytest` fixtures vs factories vs inline setup? `unittest.mock` vs `pytest-mock`? Arrange/Act/Assert comments or no?
+
+7. **Import order / grouping** — stdlib → third-party → local? Relative vs absolute imports?
+
+### Step 4 — Write the pattern brief
+
+Produce a short, concrete brief of 5–8 rules derived from the samples. Not generic rules — rules for *this task in this codebase*. Example:
+
+```
+Pattern brief for: POST /invoices handler (api_handler)
+
+From 4 peers (billing/views.py, orders/views.py, customers/views.py, auth/views.py):
+
+1. Error handling: raise ValidationError / NotFound — do NOT return {"error": ...}. 
+   Central handler in middleware/errors.py converts exceptions to HTTP responses.
+2. Dependency: inject service via constructor arg — `def __init__(self, invoice_service: InvoiceService)`
+   Do NOT call InvoiceService() inline.
+3. Abstraction: handler validates request, calls one service method, serializes response.
+   No business logic in the handler.
+4. Return shape: return InvoiceSerializer(result).data with DRF Response — not a raw dict.
+5. Naming: method names are HTTP verb — `def post(self, request)` not `def create_invoice`.
+```
+
+This brief is your constraint set for this task. Apply it while writing — not as a review after.
+
+### Step 5 — Write to the brief
+
+When writing the test and production code for this task:
+
+- Apply the brief's rules as hard constraints, not suggestions
+- If the task spec conflicts with the brief (e.g., feature file implies a return shape the codebase doesn't use), flag it to the user before writing — don't silently choose one
+- If you must deviate from the brief (e.g., the brief's pattern won't work for this specific case), note the deviation inline with a comment explaining why, and add it to the handoff note
+
+### Step 6 — Verify called functions exist
+
+After writing production code, before running tests:
+
+Extract every external function/method call your new code makes (exclude stdlib and known third-party packages). For each:
+
+```
+search_graph(name_pattern="<function_name>")
+```
+
+If a called function is not found in the graph:
+- Check the import — is it an alias or renamed import?
+- Check for typos against similar names in the graph
+- If genuinely missing: **stop**. Do not call a function that doesn't exist. Either find the correct function via `search_graph` or flag to the user.
+
+This catches hallucinated API calls before they become broken tests.
+
+---
+
+## Pattern Brief Template
+
+```
+Pattern brief for: <task title> (<code_type>)
+
+From <N> peers (<file1>, <file2>, ...):
+
+1. Error handling: <what the peers do>
+2. Dependency: <injection pattern used>
+3. Abstraction: <what this layer does vs delegates>
+4. Return shape: <concrete type/shape>
+5. Naming: <any non-obvious conventions>
+[6. Test structure: <if this is a test task>]
+[7. Deviation noted: <if you must deviate, why>]
+```
+
+Write the brief into the task's handoff note in `tasks.md` so future sessions have it.
+
+---
+
+## When to Skip
+
+- Graph not indexed → skip entirely, use `code-quality.md` only
+- < 3 peers found → skip the brief, note "no established pattern yet"
+- Task is adding a new code type with no prior examples → skip the brief, note "first of this type"
+- Hotfix / bug task in `grimoire-bug` → run only Step 6 (hallucination check); skip the full brief to avoid over-constraining the fix

package/skills/references/principles.md

@@ -0,0 +1,82 @@
+# Grimoire Design Principles
+
+Four principles govern every grimoire artifact and every change. `grimoire-draft`,
+`grimoire-plan`, and `grimoire-review` each enforce them at their own stage. They
+are not style preferences — they are gates. A draft, plan, or design that violates
+one without a stated reason is rejected, not merged.
+
+This file is the single home for the principles (it practices what it preaches —
+the skills cite it rather than restating it).
+
+---
+
+## 1. One right way to do a thing
+
+There is exactly **one** sanctioned way to do each thing in the codebase, and one
+authoritative home for each fact. Two ways to do the same thing is a defect, even
+if both work.
+
+- One capability → one feature spec. One decision → one MADR. One constraint → one
+  register entry. One fact → one home. No capability described in three places.
+- When a second mechanism appears for an existing job, the right move is to delete
+  one and converge — never to keep both "for flexibility."
+- **Tell:** "we could do it this way *or* that way" in a spec/plan. Pick one. Record
+  why in a MADR if the choice is non-obvious; don't leave both paths in the code.
+
+## 2. DRY — don't repeat yourself
+
+Every piece of knowledge has a single, unambiguous representation.
+
+- Don't store what's derivable. Code structure comes from codebase-memory-mcp on
+  demand — never freeze it into a doc that drifts. Generated overviews regenerate;
+  they are not hand-edited.
+- Reuse before write: search the graph for an existing function/utility before
+  writing a new one. Three near-identical copies is the trigger to converge — but
+  do not abstract before the third (see KISS).
+- Duplication of *content* (the same rule in three skill files, the same constant in
+  three modules, the same scenario in feature + MADR) is the target. Eliminate it.
+
+## 3. Don't reinvent the wheel — use existing tools
+
+If an established tool already does a job well, use it. Do not build a parallel
+grimoire mechanism that duplicates it.
+
+- **git** is the wheel for change processes: branches = isolation, `git diff` =
+  staging, `git log` + PR + commit trailers = history and change identity. Do not
+  build change-folder copies, promote/sync steps, or bespoke archive/changelog trees.
+- For auth, crypto, parsing, HTTP, queues, etc. — adopt the battle-tested library.
+  Never roll custom crypto, custom session management, custom auth tokens.
+- Before building any tracking/versioning/state/diff mechanism, ask: does a standard
+  tool already in the stack do this? If yes, wire to it.
+- **Exception that proves the rule:** when no single standard tool exists (e.g. issue
+  tracking is a fractured landscape), don't force-adopt one *and* don't build a
+  general-purpose clone. Keep any local mechanism narrow and purpose-scoped.
+
+## 4. Keep it simple (KISS)
+
+The simplest thing that fully solves the *stated* problem wins.
+
+- Least code, fewest new files, smallest surface area. A few lines in an existing
+  file beats a new module. A standard-library call beats a new dependency. Inline
+  beats a one-line wrapper.
+- No premature abstraction. No `BaseX`/factory/strategy/config-object for a single
+  caller. No speculative generality "for a future second caller" that doesn't exist.
+- Solve the problem in front of you, not the imagined one. Non-goals are real scope
+  boundaries — do not plan or build past them.
+- **Tell:** an abstraction, indirection, or dependency whose only justification is a
+  hypothetical. Cut it.
+
+---
+
+## How the stages apply these
+
+- **draft** — admission-test every artifact: does this fact already have a home
+  (one-right-way/DRY)? Is it behavior (→ feature) or a constraint/decision/structure
+  (→ its own home, not a feature)? Is there an existing tool/library for it
+  (don't-reinvent)? Is the scope the stated problem only (KISS)?
+- **plan** — every task names the single approach (one-right-way), reuses before
+  writing (DRY), follows a proven pattern / existing tool rather than a bespoke one
+  (don't-reinvent), and chooses the least-code option within non-goals (KISS). Flag
+  any task that adds an abstraction, dependency, or second mechanism.
+- **review** — a dedicated principles pass: hunt for duplicate homes, derivable-but-
+  stored facts, reinvented wheels, and speculative complexity. Each is a finding.

package/skills/references/refactor-scan-categories.md

@@ -28,6 +28,21 @@ Files that change frequently AND are hard to change. Highest-ROI refactoring tar
 
 **Severity:** high = 2x+ threshold, medium = 1-2x, low = marginally over
 
+**Graph-powered LLM bloat checks** (requires `codebase-memory-mcp`; skip if not indexed):
+
+These target patterns that static size checks miss — structurally valid code that adds indirection without value. Primary signal of LLM-generated over-engineering.
+
+| Pattern | Query | Flag when |
+|---|---|---|
+| Single-subclass base class | `query_graph("MATCH (sub)-[:INHERITS]->(base:Class) WITH base, collect(sub) AS subs WHERE size(subs) = 1 RETURN base.qualified_name, base.file, subs[0].qualified_name AS only_subclass")` | Any result — a base with one child is premature abstraction |
+| Single-caller wrapper | Step 1: `query_graph("MATCH (caller)-[:CALLS]->(fn) WITH fn, collect(caller) AS callers WHERE size(callers) = 1 RETURN fn.qualified_name, fn.file, callers[0].qualified_name AS only_caller")`. Step 2: for each result, `get_code_snippet(qualified_name)` and count body lines. | Wrapper with 1 caller and ≤7 body lines — inline candidate |
+| Zero-caller export | `query_graph("MATCH (f:Function) WHERE f.exported = true AND NOT ()-[:CALLS]->(f) RETURN f.qualified_name, f.file")` — then filter out entry points manually: skip files named `index.ts`, `__init__.py`, `main.py`, `cli.py`, `app.py`, or in a `public/` directory | Exported, unreachable within repo, not an entry point — dead export |
+| Single-implementation interface | `query_graph("MATCH (impl)-[:IMPLEMENTS]->(iface:Interface) WITH iface, collect(impl) AS impls WHERE size(impls) = 1 RETURN iface.qualified_name, iface.file, impls[0].qualified_name AS only_impl")` | Any result — interface with one implementor adds no polymorphism |
+
+Note: the exact Cypher depends on the graph schema. If a query returns an error, adjust field names using `get_graph_schema()` to inspect available properties.
+
+**Severity for graph findings:** high = single-implementation interface or zero-caller export, medium = single-subclass base or single-caller wrapper
+
 ## 2c. Data Structure Complexity
 
 | Signal | Meaning |
@@ -76,12 +91,28 @@ TODO/FIXME/HACK/XXX comments that have aged.
 ## 2g. Duplication
 
 **How to scan:**
-- Read `.grimoire/docs/.snapshot.json` `duplicates` section if present
-- Or run `config.tools.duplicates` if configured (e.g., jscpd)
+- Run `config.tools.duplicates` if configured (e.g., jscpd), or `grimoire health` (config-driven `duplicates` metric)
+- Plus semantic clones via `search_graph(semantic_query=[...])` (requires codebase-memory-mcp) to catch re-implementations under different names
 - Group by area — within-area dupes are easy to consolidate
 
 **Severity:** high = >30 lines or >3 copies, medium = 10-30 lines or 2 copies, low = <10 lines
 
+**Concept-based duplicate detection** (requires `codebase-memory-mcp`; supplements jscpd which only finds textual clones):
+
+LLM-generated code frequently re-implements existing utilities under a different name. jscpd won't catch these — the code is structurally different even though it does the same thing.
+
+**How to scan:**
+1. Find utility/helper functions: `search_graph(label="Function", name_pattern="(parse_|format_|validate_|convert_|build_|get_|find_|create_|check_|is_|has_)")`
+2. For each result, extract 2–3 concept words from the function name (e.g., `format_invoice_date` → `["format", "date", "invoice"]`)
+3. Run: `search_graph(semantic_query=["<concept1>", "<concept2>", "<concept3>"])` — if `semantic_query` is unsupported, fall back to `search_graph(name_pattern="(<concept1>|<concept2>)")`
+4. Compare: if the search returns a different function, read both with `get_code_snippet` and assess whether they do the same job
+
+**Flag when:** two functions accept similar inputs, produce similar outputs, and operate on the same domain concept. Assessment is qualitative — the tool returns ranked results, not similarity scores.
+
+**Focus on:** utility directories (`utils/`, `helpers/`, `lib/`, `common/`), validators, formatters, parsers. These are where re-implementations accumulate.
+
+**Severity:** high = identical behavior under different names, medium = near-duplicate with minor variations that could be unified with a parameter, low = similar but distinct enough to keep
+
 ## 2h. Dead Code
 
 **How to scan:**
@@ -100,3 +131,124 @@ TODO/FIXME/HACK/XXX comments that have aged.
 - Check for over-mocked tests (testing mocks, not behavior)
 
 **Severity:** high = complex code (top quartile) with <30% coverage, medium = moderate complexity with <50%, low = simple code with low coverage
+
+## 2j. Pattern Divergence
+
+Code that solves a problem in a way that contradicts how the codebase already solves the same class of problem. The primary AI slop signal — structurally valid code that ignores established conventions and accumulates architectural drift.
+
+**Requires:** `codebase-memory-mcp` indexed. Skip this category if graph is not available.
+
+**How to scan:**
+
+**Step 1 — Identify peer groups**
+
+A peer group is a set of nodes in the graph that share the same role. Use `search_graph` to find them:
+
+| Peer group | Query |
+|---|---|
+| API/route handlers | `search_graph(label="Function", name_pattern="(handle|view|endpoint|route|controller)")` |
+| Service methods | `search_graph(label="Function", name_pattern="(service|use_case|interactor)")` |
+| Repository/data access | `search_graph(label="Function", name_pattern="(repo|repository|store|dao|query)")` |
+| Test files | `search_graph(label="Module", name_pattern="(test_|_test|spec)")` |
+| Error handlers | `search_graph(label="Function", name_pattern="(error|exception|fail|catch)")` |
+
+Supplement with area docs if available — each area doc lists files by role.
+
+**Step 2 — Extract modal pattern per peer group**
+
+For each peer group with ≥3 members, sample 3-5 established members (oldest by `git log`, not recently changed):
+- `get_code_snippet(qualified_name)` for each sample
+- Identify the modal pattern across: error handling style, dependency access (injected vs imported), abstraction depth (business logic in handler vs delegated to service), naming convention, return type shape
+
+This is the **baseline** — what the codebase already does.
+
+**Step 3 — Compare recent code against baseline**
+
+Scope: files changed in the last 60 days (`git log --since="60 days ago" --name-only --format=`). Cross-reference with the peer groups from step 1.
+
+For each recently changed file that belongs to a peer group:
+1. `get_code_snippet` for the changed function/class
+2. Compare against the modal pattern from step 2
+3. Flag if it diverges on any of the four critical seams (see below)
+
+**Step 4 — Flag divergences**
+
+Only flag divergence on seams that matter architecturally. Cosmetic drift (whitespace, docstring style) is not a debt item.
+
+| Seam | Divergence signal | Example |
+|---|---|---|
+| **Error handling** | Mix of exception-raise vs return-value-error in same layer | Most handlers raise `ValueError`; new one returns `{"error": ...}` |
+| **Data access** | Bypass of established access layer | Most services call `repo.get()`; new one imports ORM model directly |
+| **Abstraction depth** | Business logic at wrong layer | All handlers delegate; new handler contains domain logic inline |
+| **Dependency wiring** | Injected vs hardcoded import for same dependency | All services receive `db` via constructor; new one calls `get_db()` directly |
+| **Test structure** | Different test strategy in same area | All tests in area use factory fixtures; new tests use heavy mocks |
+
+**Step 5 — Check for hallucinated or non-existent references**
+
+Use `search_graph` to verify function calls in recently changed files:
+- Extract all function calls in the diff using `search_code(pattern)` or `get_code_snippet`
+- For each called function/method: `search_graph(name_pattern=<name>)` — does it exist?
+- Missing = hallucinated API, deprecated method, or invented config option
+
+Flag as `pattern_divergence` with detail: "Called `foo.bar()` — no matching node in graph."
+
+**Severity:**
+- high = divergence at a core architectural seam (data access, error handling, auth) OR hallucinated reference
+- medium = wrong abstraction layer or dependency wiring inconsistency
+- low = test strategy divergence or naming/convention drift
+
+**Suggested action (per seam):**
+- Error handling: align to codebase's exception or result pattern
+- Data access: route through established repository/service layer
+- Abstraction: extract domain logic to service, slim the handler
+- Dependency: adopt constructor injection or established DI pattern
+- Hallucinated ref: replace with actual existing function (use `search_graph` to find it)
+
+## 2k. Comment Noise
+
+Comments that restate the code, reference stale context, or pad function bodies without conveying non-obvious intent. A secondary LLM bloat signal — LLMs are trained to produce documentation and carry that habit into code generation.
+
+**How to scan:**
+
+**Step 1 — High comment density files**
+```bash
+grep -rcE "^\s*#|^\s*//" --include="*.py" --include="*.ts" --include="*.js" <src_dirs> | \
+  grep -v ":0$" | sort -t: -k2 -rn | head -20
+```
+Flag files with >30 comment lines. Raw count, not ratio — a 30-comment file is a candidate regardless of size.
+
+**Step 2 — Restatement pattern grep**
+```bash
+grep -rni \
+  -e "# loop over" -e "# iterate over" -e "# return the" -e "# return result" \
+  -e "# loop through" -e "# now call" -e "# call the" -e "# increment" -e "# decrement" \
+  -e "// loop over" -e "// iterate over" -e "// return the" -e "// return result" \
+  -e "// loop through" -e "// now call" -e "// call the" -e "// increment" -e "// decrement" \
+  --include="*.py" --include="*.ts" --include="*.js" <src_dirs>
+```
+Treat results as candidates — quick human scan to confirm before deleting.
+
+**Step 3 — Task/PR reference comments**
+```bash
+grep -rn \
+  -e "# added for" -e "# used by" -e "# see issue" -e "# handles the case" -e "# added in" \
+  -e "// added for" -e "// used by" -e "// see issue" -e "// handles the case" -e "// added in" \
+  --include="*.py" --include="*.ts" --include="*.js" <src_dirs>
+```
+These belong in commit messages, not source. Treat results as candidates — review before flagging, as patterns like `# see issue` can appear in legitimate context.
+
+**Step 4 — Docstrings on private/internal functions**
+```bash
+# Python: single-underscore private functions (excludes dunders)
+grep -rn "def _[^_]" --include="*.py" <src_dirs>
+# TS/JS: JSDoc blocks
+grep -rn "/\*\*" --include="*.ts" --include="*.js" <src_dirs>
+```
+Manual triage: open each hit and check whether a multi-line docstring follows. Python `def _name` functions and TS/JS non-exported functions don't need docstrings. Delete multi-line blocks; a single-line doc is acceptable if `comment_style` requires it.
+
+**Severity:**
+- high = >20 restatement comments in a single file, or task/PR references in core business logic
+- medium = 5–20 restatement comments, or any task/PR references found
+- low = multi-line docstrings on private functions
+
+**Suggested action:** Delete restatement comments. Move task/PR references to commit history. Trim private function docstrings to one line or remove entirely.