scip-query 0.9.0 → 0.10.1

package/docs/AGENT_GUIDE.md

@@ -0,0 +1,353 @@
+# scip-query Agent Guide
+
+Goal-oriented workflows for AI agents and developers. Each section starts with a goal and walks through the exact commands to run, what to expect back, and how to use the results.
+
+For command syntax and options reference, see [Command Reference](COMMAND_REFERENCE.md).
+
+---
+
+## Workflow 1: Understand a system before making changes
+
+**Goal:** Build a complete mental model of a module or feature area so you can write a precise implementation plan with no ambiguity about what code exists, what it does, and what depends on it.
+
+### Steps
+
+1. **Map the module**
+   ```bash
+   scip-query system <module-path>
+   ```
+   Returns: all files in the module, all exported symbols with line ranges, all inbound and outbound dependencies. This is your starting map.
+
+2. **Understand the public contract**
+   ```bash
+   scip-query surface <module-path>
+   ```
+   Returns: which symbols external consumers actually reference. This is the true public API — not what's exported, but what's used. Any change to these symbols is a breaking change.
+
+3. **Trace specific symbols**
+   ```bash
+   scip-query trace <symbol-name>
+   ```
+   Returns: where the symbol is defined (file + line range + signature) and every file that references it. Use this for any symbol you need to understand deeply.
+
+4. **Map the call graph**
+   ```bash
+   scip-query call-graph <function-name>
+   ```
+   Returns: what calls this function (incoming) and what this function calls (outgoing). Gives you the function's role in the execution flow.
+
+5. **Check blast radius**
+   ```bash
+   scip-query affected <symbol-name>
+   ```
+   Returns: the full transitive closure of symbols that could break if this symbol changes. Depth 1 = direct consumers. Depth 2 = consumers of consumers. Shows the complete ripple effect.
+
+6. **Pre-change briefing**
+   ```bash
+   scip-query change-surface <file>
+   ```
+   Returns: every symbol in the file, how many external consumers each has, and a risk level (high/medium/low). Run this before modifying any file.
+
+### What you should know after this workflow
+
+- Every file in the module and what it contains
+- The true public API (what consumers actually use)
+- The full dependency graph (what the module depends on and what depends on it)
+- The blast radius of any specific symbol change
+- Which symbols are high-risk (many consumers, wide blast radius)
+
+---
+
+## Workflow 2: Write a concrete implementation plan
+
+**Goal:** Produce an implementation plan where every file to create/modify is named, every symbol to change is identified with line numbers, every dependency is mapped, and every risk is called out.
+
+### Steps
+
+1. **Map the target area**
+   ```bash
+   scip-query system <module-path>
+   scip-query outline <each-file-you-will-modify>
+   ```
+   Get the structural outline with line ranges for every file in scope. Add `--signatures` only when type details are useful.
+
+2. **Identify the public contract you must preserve**
+   ```bash
+   scip-query surface <module-path>
+   ```
+   Any symbol that appears here must maintain backward compatibility or all consumers must be updated.
+
+3. **Map every symbol you plan to change**
+   ```bash
+   scip-query refs <symbol>        # who uses it
+   scip-query affected <symbol>    # transitive blast radius
+   scip-query fan-in <symbol>      # quantified consumer count
+   ```
+   For each symbol you'll modify: know exactly who consumes it and how many layers deep the impact goes.
+
+4. **Check blast radius before editing**
+   ```bash
+   scip-query change-surface <file>
+   scip-query diff-impact
+   ```
+   Identify which symbols in your change set have many external consumers and which downstream files will be affected.
+
+5. **Find reusable code**
+   ```bash
+   scip-query similar <symbol-you-plan-to-write>
+   scip-query deps <file>
+   ```
+   Before writing new code, check if something similar already exists. `similar` finds functions with overlapping callee patterns. `deps` shows what the file already imports that you can reuse.
+
+6. **After making changes, verify impact**
+   ```bash
+   scip-query diff-impact
+   scip-query drift
+   ```
+   Shows every symbol affected by your git diff, every consumer file impacted, and whether the change introduced new structural drift.
+
+### Plan template
+
+```
+## Change: [description]
+
+### Files to modify
+- `path/to/file.ts` — [what changes, which symbols]
+  - `symbolName` (lines X-Y) — [change description]
+  - Fan-in: N, External consumers: N, Risk: low/medium/high
+
+### Files to create
+- `path/to/new-file.ts` — [purpose]
+  - Similar to: `existing-file.ts` (N% callee overlap via `similar`)
+
+### Public contract impact
+- `surface` shows N symbols consumed externally
+- [List any breaking changes]
+
+### Blast radius
+- `affected` shows N symbols across M files at depth 1-2
+- [List high-risk symbols]
+
+### Impact checks
+- `change-surface` shows N externally consumed symbols
+- `diff-impact` shows N downstream consumer files
+```
+
+---
+
+## Workflow 3: Clean up and de-bloat a codebase
+
+**Goal:** Systematically reduce unnecessary code, eliminate duplication, and improve structural health.
+
+### Steps
+
+1. **Get the full health report**
+   ```bash
+   scip-query health
+   ```
+   This runs every analysis and produces a prioritized action list. Start here. The actions are sorted by impact/effort ratio — do the top ones first.
+
+2. **Delete dead code (safest, highest impact)**
+   ```bash
+   scip-query dead --min-loc 10 --skip-barrels
+   ```
+   These symbols have zero cross-file references. They can be safely deleted. `--skip-barrels` ignores references from inactive barrel files, which helps surface exports kept alive only by unused re-export layers without hiding live package entry surfaces.
+
+3. **Delete isolated symbols**
+   ```bash
+   scip-query isolated --min-loc 5
+   ```
+   Stricter than `dead` — these symbols have zero references anywhere, including in their own file. Completely disconnected from the codebase.
+
+4. **Break circular dependencies**
+   ```bash
+   scip-query cycles
+   ```
+   If any exist, they need structural fixes: dependency inversion, module splitting, or interface extraction.
+
+5. **Consolidate similar functions**
+   ```bash
+   scip-query similar --min-similarity 0.5
+   ```
+   Pairs of functions with overlapping callee sets. For each pair:
+   ```bash
+   scip-query convergence <symbol1> <symbol2>
+   ```
+   Shows what the consolidated version would look like: shared callees = common body, unique callees = parameterization points.
+
+6. **Extract large functions**
+   ```bash
+   scip-query extract-candidates --min-loc 20
+   ```
+   Functions with isolated callee clusters — natural "Extract Method" seams.
+
+7. **Remove unnecessary indirection**
+   ```bash
+   scip-query wrapper-candidates
+   scip-query passthrough-candidates
+   ```
+   Wrappers: single-consumer symbols that can be inlined. Passthroughs: functions that just forward to one callee.
+
+8. **Prune premature abstractions**
+   ```bash
+   scip-query stale-abstractions
+   ```
+   Types and interfaces with 0-1 consumers. An interface with one implementation isn't an abstraction.
+
+9. **Fix pattern drift**
+   ```bash
+   scip-query drift
+   ```
+   Files that deviate from their directory's typical dependency pattern. Bring them into line with their neighbors.
+
+10. **Remove redundant re-exports**
+    ```bash
+    scip-query redundant-reexports
+    ```
+    Barrel file entries that nobody imports through. Clean up the barrel.
+
+11. **Find same-shape functions**
+    ```bash
+    scip-query similar-signatures --min-loc 5
+    ```
+    Functions with identical parameter/return types. Different signal from callee similarity — catches "same interface, different implementation."
+
+### Priority order
+
+| Priority | What | Why |
+|---|---|---|
+| 1 | Dead code | Zero risk, immediate LOC reduction |
+| 2 | Isolated symbols | Zero risk, zero consumers |
+| 3 | Circular deps | Structural fix, prevents future problems |
+| 4 | Similar functions | Reduces duplication, use `convergence` for prescription |
+| 5 | Extraction candidates | Reduces function complexity |
+| 6 | Wrappers / passthroughs | Removes unnecessary indirection |
+| 7 | Stale abstractions | Removes premature over-engineering |
+| 8 | Pattern drift | Consistency improvement |
+
+---
+
+## Workflow 4: Assess code quality and risk
+
+**Goal:** Produce a quality assessment of a codebase or module with quantified metrics.
+
+### Steps
+
+1. **Overall health**
+   ```bash
+   scip-query health
+   scip-query health --json    # for programmatic use
+   ```
+
+2. **Complexity risks**
+   ```bash
+   scip-query complexity-hotspots -n 20
+   ```
+   Symbols with the highest composite score (LOC x fan-in x fan-out). These are the most likely to contain bugs and the hardest to modify.
+
+3. **Coupling risks**
+   ```bash
+   scip-query bottlenecks -n 20
+   ```
+   Symbols with both high fan-in (many consumers) AND high fan-out (many dependencies). Changes to these are risky in both directions.
+
+4. **Architecture depth**
+   ```bash
+   scip-query deep-chains --min-depth 5
+   ```
+   Long transitive dependency chains. If chains are deeper than 6-7, the architecture may need flattening.
+
+5. **Structural drift**
+   ```bash
+   scip-query drift
+   ```
+   Files with unused imports, layer violations, or dependency profiles that deviate from their neighbors.
+
+### Quality report template
+
+```
+## Quality Assessment: [project/module]
+
+### Overview
+- Files: N | Symbols: N | Index size: N
+- Health score: N/100
+
+### Risk Areas
+- Complexity hotspots: [top 5 from complexity-hotspots]
+- Coupling bottlenecks: [top 5 from bottlenecks]
+- Deepest dependency chain: N layers
+- Circular dependencies: N
+
+### Structural quality
+- Pattern drift: N files
+
+### Cleanup Opportunities
+- Dead code: N symbols (N LOC recoverable)
+- Similar function pairs: N
+- Stale abstractions: N
+```
+
+---
+
+## Workflow 5: Understand impact after making changes
+
+**Goal:** After modifying code, verify what was affected and identify gaps.
+
+### Steps
+
+1. **Compute diff impact**
+   ```bash
+   scip-query diff-impact
+   ```
+   Shows: changed files, changed symbols with fan-in counts, and affected consumer files.
+
+2. **Check transitive impact for critical symbols**
+   ```bash
+   scip-query affected <changed-symbol>
+   ```
+   For any high fan-in symbol that changed, check the full transitive blast wave.
+
+3. **Re-check structural drift around the changed area**
+   ```bash
+   scip-query drift
+   scip-query change-surface <changed-file>
+   ```
+   Verify the change did not introduce new dependency-pattern outliers and understand the remaining blast radius.
+
+---
+
+## Quick Reference
+
+| I want to... | Run |
+|---|---|
+| Understand a module | `system <module>` |
+| See what consumers actually use | `surface <module>` |
+| Find all references to a symbol | `refs <symbol>` or `trace <symbol>` |
+| See what a function calls and who calls it | `call-graph <symbol>` |
+| Check blast radius of a change | `affected <symbol>` |
+| Get a pre-change briefing | `change-surface <file>` |
+| See impact of my git changes | `diff-impact` |
+| Find dead code to delete | `dead --min-loc 10 --skip-barrels` |
+| Find duplicate functions | `similar --min-similarity 0.5` |
+| Find same-shape functions | `similar-signatures --min-loc 5` |
+| Get a refactoring prescription | `convergence <sym1> <sym2>` |
+| Find redundant barrel re-exports | `redundant-reexports` |
+| Find extraction opportunities | `extract-candidates --min-loc 20` |
+| Find unnecessary wrappers | `wrapper-candidates` |
+| Find single-implementation types | `stale-abstractions` |
+| Find pattern outliers | `drift` |
+| Get overall codebase health | `health` |
+| Find riskiest symbols | `complexity-hotspots` |
+| Find coupling pressure points | `bottlenecks` |
+| Find circular dependencies | `cycles` |
+---
+
+## Tips for AI Agents
+
+- **Always reindex before analysis** if the codebase has changed significantly: `scip-query reindex`
+- **Use `--json` on `health`** for programmatic consumption — parse the JSON to make decisions
+- **Run `change-surface` before every file modification** — it takes <1 second and prevents surprises
+- **Run `diff-impact` before committing** — catches unexpected blast radius across downstream consumers
+- **Use `convergence` after `similar`** — `similar` finds the problem, `convergence` gives the solution
+- **Start cleanup with `health`** — it prioritizes for you so you don't have to decide what to fix first
+- **Scope commands with `-s`** — most commands accept `--scope <path>` to limit analysis to a specific module. Use this on large codebases to keep results focused.

package/docs/AI_FAILURE_MODES.md

@@ -0,0 +1,278 @@
+# The Ways AI Coding Rots a Codebase — and the Detector Built for Each
+
+AI-assisted development fails in *specific, repeatable* ways. None of them are
+visible in a single file, which is why linters and code review miss them: every
+one lives in the relationships between files — the reference graph, the git
+change graph, or the gap between docs and code. Each failure mode below names
+the behavior, the detector built for it, exactly how to run it, and how to wire
+it in so it gets caught automatically.
+
+Every detector is evidence-ranked and honest about confidence: graph facts vs
+heuristic candidates are labeled, and heuristic output always says so.
+
+---
+
+## 1. Re-implementing code that already exists
+
+**What the agent does:** it can't see the whole repo, so it writes a helper,
+hook, composable, or frontend component that already exists - a date formatter,
+a retry wrapper, a validation guard, a table toolbar. Now there are two implementations that drift independently until they
+contradict each other.
+
+**The detector:** `recent-duplicates` makes similarity *directional* using git
+file ages - which side is the established original, which is the freshly-added
+echo:
+
+```
+91%  ECHO  react-component  src/components/ProjectCardVisual.tsx  ProjectCardVisual  (added 62 commits ago)
+     duplicates established  src/pages/HomePage.tsx  RecentProjectRow()
+     basis: jsx-structure
+```
+
+**Use it:**
+
+```bash
+scip-query recent-duplicates              # after any AI session
+scip-query similar <closest-existing-fn>  # BEFORE writing a new helper
+```
+
+**Caught automatically by:** the `echo` check in `diff-gate` — flags any
+changed symbol that is ≥80% similar to established code elsewhere.
+
+## 2. Duplicating itself within one session
+
+**What the agent does:** generates the same function in two places during one
+session — neither copy is "established," both are new, and they diverge from
+day one.
+
+**The detector:** `recent-duplicates` reports these as **TWIN** (both sides
+inside the recency window) and tells you to pick one before they drift:
+
+```
+100% TWIN  src/workflows/a.ts ensureAccessible() / src/workflows/b.ts ensureAccessible()
+```
+
+**Use it:** `scip-query recent-duplicates` at the end of every agent session.
+
+## 3. Extracting a helper but only migrating some call sites
+
+**What the agent does:** creates an abstraction, rewires one or two call
+sites into it, and abandons the rest — the extracted logic survives inline at
+every site it missed. The codebase ends up with the worst of both worlds: an
+abstraction *and* the duplication it was meant to remove.
+
+**The detector:** `incomplete-migration` finds symbols that are new at the
+base ref, confirms they were wired into at least one site, then reports
+established untouched functions whose callee sets *contain* the helper's
+(containment scoring, because a missed site holds the helper's logic plus its
+own — symmetric similarity under-scores exactly these):
+
+```
+src:demo:summarize:recordScore()  (src/demo/summarize.ts)
+  wired into: src/demo/report-a.ts
+  un-migrated: 100%  buildReportB()  (src/demo/report-b.ts)
+  un-migrated: 100%  buildReportC()  (src/demo/report-c.ts)
+```
+
+**Use it:**
+
+```bash
+scip-query incomplete-migration                       # after any extraction
+scip-query incomplete-migration --base origin/main    # gate a whole branch
+```
+
+**Caught automatically by:** the `incomplete-migration` check in `diff-gate`.
+
+## 4. Writing code that never gets wired up
+
+**What the agent does:** builds the function, the type, the handler — and
+never connects it. Or it *was* connected, then a later session rewired the
+flow and left the original dangling. Dead code that looks intentional.
+
+**The detectors:** `dead` (evidence-ranked, entrypoint-aware), and the
+`new-dead` check in `diff-gate` that catches it *at the moment of creation*:
+
+```
+[new-dead] resolveTheme (src/theme.ts) was changed but has zero indexed consumers
+  -> Wire it up, or remove it before it becomes permanent dead code.
+```
+
+**Use it:**
+
+```bash
+scip-query dead --min-loc 10 --skip-barrels
+scip-query isolated                # whole files nothing imports
+```
+
+## 5. Speculative generality — options and parameters "for later"
+
+**What the agent does:** adds trailing parameters, option bags, and config
+flags for futures that never arrive. Every one is permanent API surface that
+every future reader has to understand.
+
+**The detectors:** `unused-params` (trailing parameters no body uses, scoped
+to removals that are type-safe by construction), plus the abstraction-level
+versions: `wrapper-candidates` (functions that only forward), 
+`passthrough-candidates` (layers that add nothing), and `stale-abstractions`
+(interfaces/bases with a single implementation).
+
+**Use it:**
+
+```bash
+scip-query unused-params
+scip-query wrapper-candidates
+scip-query stale-abstractions
+```
+
+**Caught automatically by:** the `unused-params` check in `diff-gate`.
+
+## 6. Letting the standards docs lie
+
+**What the agent does:** nothing — that's the problem. You write in-repo
+standards *for* agents; the code moves on; the doc doesn't. The next agent
+reads the doc and faithfully implements against a dead spec. A stale standard
+is worse than none.
+
+**The detector:** `doc-drift` reads every doc's file citations *and* its
+co-change history, and flags docs whose referenced code kept changing after
+the doc stopped — including broken references to files that no longer exist:
+
+```
+staleness 94  product/domain-model.md
+  BROKEN REFERENCE: cites src/api/servicePlans.ts — that file no longer exists
+  22 change(s) since doc update  src/workflows/serviceTasks.ts
+```
+
+**Use it:** `scip-query doc-drift`, then run the `scip-doc-reconcile` skill to
+drive staleness to zero (it updates descriptive claims and *escalates*
+normative violations instead of silently blessing them).
+
+**Caught automatically by:** the `doc-reference` check in `diff-gate` — a doc
+cites a file you changed and wasn't updated in the same diff.
+
+## 7. Editing one side of an invisible contract
+
+**What the agent does:** changes the schema but not the generated inventory;
+the `.env.example` but not its parser; the API contract but not the frontend
+store. The reference graph can't see these pairs — no import connects them —
+but git history can: they've changed together in 12 of the last 14 commits.
+
+**The detector:** `co-change` finds file pairs that repeatedly change in the
+same commits with *no* dependency edge.
+
+**Use it:**
+
+```bash
+scip-query co-change                       # repo-wide hidden coupling
+scip-query co-change src/db/schema.prisma  # partners of one file
+```
+
+**Caught automatically by:** the `co-change-partner` check in `diff-gate`:
+
+```
+[co-change-partner] schema.prisma changed, but scripts/scope-inventory.mjs did not — they change together 12x (86%)
+```
+
+## 8. Deleting things that are still used — or refusing to delete at all
+
+**What the agent does:** both. It deletes a "dead" function that a dynamic
+path still references, or it hoards code because it can't prove anything is
+safe to remove.
+
+**The detector:** `cleanup-plan` runs dead-code analysis to a *fixpoint* —
+deleting batch 0 makes batch 1 dead, and the plan shows the cascade. Then
+`--verify` applies each batch in a throwaway git worktree and runs **your own
+compiler** (tsc, cargo, go, python oracles — differentially, so pre-existing
+errors don't drown the signal):
+
+```
+── Batch 0: deletable now (graph-fact, 67 LOC) ──
+Batch 0: COMPILER-VERIFIED
+```
+
+When verification fails, the errors name the exact references the static
+evidence missed. Delete with a proof in hand, not vibes.
+
+**Use it:**
+
+```bash
+scip-query cleanup-plan --verify
+```
+
+## 9. Making every change blind to its blast radius
+
+**What the agent does:** edits a symbol without knowing who consumes it, what
+breaks transitively, or which files historically move together with it — then
+"finishes" with three consumers silently broken.
+
+**The detectors:** `plan-context` (one command bundling definitions,
+references, call graph, blast radius, churn, and co-change partners — the
+pre-edit briefing), `change-surface`, `affected`, `diff-impact`.
+
+**Use it:**
+
+```bash
+scip-query plan-context <symbol-or-file>   # before the edit
+scip-query diff-impact                     # after the edit
+```
+
+The `concrete-plan` skill enforces this end-to-end: every step in a plan must
+cite the scip-query command that verified it.
+
+## 10. Slow quality decay nobody notices
+
+**What the agent does:** each session adds a little rot. No single diff is
+alarming; six weeks later the repo is unrecognizable.
+
+**The detector:** the ratchet. `health --write-baseline` snapshots finding
+identities into a committable file; `health --baseline` exits 1 on any *new*
+finding. "Don't get worse" becomes an objective gate no score arithmetic can
+game.
+
+**Use it:**
+
+```bash
+scip-query health --write-baseline   # once, committed
+scip-query health --baseline         # in CI
+```
+
+**Caught automatically by:** the `baseline` check in `diff-gate`.
+
+---
+
+## Wiring it all up so nobody has to remember any of this
+
+The detectors only help if they run. Three layers, in increasing strength:
+
+**1. Skills (routing).** Installing scip-query symlinks nine skills into
+`~/.agents/skills/`, `~/.claude/skills/`, and `~/.codex/skills/` — they update
+automatically with the package. The `scip-query` router skill triggers on any
+codebase work and dispatches to the right specialist (explore → plan →
+implement → verify → clean up), carrying the non-negotiables: similarity check
+before new helpers, `incomplete-migration` after extractions, `diff-gate`
+before done.
+
+**2. Project guidance (instructions).** Run once per project:
+
+```bash
+scip-query setup-agent
+```
+
+Seeds a managed block in `AGENTS.md` (the cross-tool standard Codex, Cursor,
+Gemini, and others read) pointing at the router skill and the gate — plus an
+`@AGENTS.md` import shim in `CLAUDE.md`, because Claude Code doesn't read
+AGENTS.md natively. Only the marked block is ever managed; your content is
+never touched, and an existing `@AGENTS.md` bridge is left alone.
+
+**3. The gate (enforcement).**
+
+```bash
+scip-query diff-gate                 # one command, every check above, scoped to the diff, exit 1 on findings
+scip-query setup-agent --git-hook    # pre-commit backstop: fires whoever wrote the diff
+```
+
+Every finding ships with a remediation an agent can act on without human
+triage. For in-session enforcement, `diff-gate --hook` speaks the turn-end
+hook contract shared by Claude Code, Codex, and Gemini CLI (blocks the agent's
+"done" and feeds the findings back as its next prompt) — wire it into your
+tool's hook config if you want the gate to be unskippable.

package/docs/API.md

@@ -0,0 +1,40 @@
+# Programmatic API
+
+Every CLI command is also available as a TypeScript function. The `queries` namespace exports cover the public commands, including the `top*` variants of `fan-in`, `fan-out`, and `coupling`, plus `similarAll` for the cross-codebase mode of `similar`.
+
+```typescript
+import { ScipDatabase, createGitignoreFilter } from 'scip-query';
+import {
+  health,
+  affected,
+  changeSurface,
+  diffImpact,
+  hotspots,
+  similar,
+  dead,
+  convergence,
+} from 'scip-query/queries';
+
+const filter = createGitignoreFilter('/path/to/project');
+const db = new ScipDatabase(
+  {
+    dbPath: '/path/to/index.db',
+    indexPath: '/path/to/index.scip',
+    projectRoot: '/path/to/project',
+  },
+  filter,
+);
+
+const report = health(db);
+console.log(`Score: ${report.score}/100`);
+console.log(`Actions: ${report.actions.length}`);
+
+const blast = affected(db, 'login', { maxDepth: 3 });
+const brief = changeSurface(db, 'auth.service.ts');
+const impact = diffImpact(db, { base: 'main' });
+
+const pairs = similar(db, 'myFunction', { minSimilarity: 0.5 });
+const recipe = convergence(db, 'funcA', 'funcB');
+
+db.close();
+```