pi-lens 3.3.1 → 3.6.1

package/CHANGELOG.md

@@ -2,6 +2,89 @@
 
 All notable changes to pi-lens will be documented in this file.
 
+## [3.6.0] - 2026-04-02
+
+### Added
+- **LSP Call Hierarchy Support** — Added 3 new operations to `lsp_navigation` tool:
+  - `prepareCallHierarchy` — Get callable item at position
+  - `incomingCalls` — Find all functions/methods that CALL this function
+  - `outgoingCalls` — Find all functions/methods CALLED by this function
+  - Use case: "Who calls this function?" and "What does this function depend on?"
+- **LSP Navigation Skill** — New built-in skill (`skills/lsp-navigation/SKILL.md`) that guides LLM on when to use LSP for code intelligence vs other tools
+- **AST-Grep Skill Improvements** — Enhanced `skills/ast-grep/SKILL.md` with:
+  - Testing Tips section (Search → Dry-run → Apply workflow)
+  - Metavariable selection guide ($ vs $$$)
+  - Specific guidance for "Multiple AST nodes" error
+- **Skills Registration** — Extension now registers `skills/` directory via `resources_discover` event, exposing both `ast-grep` and `lsp-navigation` skills to pi
+- **Enhanced TDI (Technical Debt Index) with 5-factor formula** — Now captures "worst offender" functions and code unpredictability:
+  - **Max Cyclomatic (10%)**: Catches worst function complexity (avg hides bad apples)
+  - **Entropy (5%)**: Measures code unpredictability/vocabulary richness in bits
+  - Rebalanced weights: MI (45%), Cognitive (30%), Nesting (10%), MaxCyc (10%), Entropy (5%)
+  - New thresholds: MaxCyc >10 bad, >30 critical; Entropy >4.0 bits risky, >7.0 critical
+
+### Removed
+- **TDR (Technical Debt Ratio)** — Removed orphaned metric tracking system:
+  - Deleted `TDREntry`, `TDRCategory` types, `tdrFindings` Map, `updateTDR()` method
+  - Removed `convertDiagnosticsToTDREntries()` helper and all `tdrCategory` assignments
+  - Deleted TDR test file
+  - TDI is sufficient for code health tracking; inline diagnostics provide immediate feedback
+
+### Changed
+- **Updated `/lens-tdi` display** — Shows 5 category breakdown with descriptions:
+  ```
+  Debt breakdown:
+    Maintainability: 45% (MI-based)
+    Cognitive: 30%
+    Nesting: 10%
+    Max Cyclomatic: 10% (worst function)
+    Entropy: 5% (code unpredictability)
+  ```
+- **Extended MetricSnapshot** — Added `maxCyclomatic` and `entropy` fields for historical tracking
+
+---
+
+## [3.5.0] - 2026-04-02
+
+### Added
+- **Tree-sitter query compilation cache** — 10× performance improvement for structural analysis. Query files (`.yml`) are compiled to binary `.wasm-cache` format once and cached to disk. Subsequent loads use the compiled cache directly, reducing tree-sitter startup from ~50ms to ~5ms per query. Cache uses mtime-based invalidation — automatically recompiles when source `.yml` changes.
+- **Rule cache infrastructure** (`clients/cache/`) — New disk-backed cache system with:
+  - `RuleCache` class for storing compiled artifacts
+  - mtime-based invalidation (auto-refresh when source files change)
+  - JSON metadata tracking for cache entries
+  - TTL and integrity validation
+
+### Fixed
+- **YAML parser colon truncation** — Fixed regex-based parser that incorrectly truncated values containing colons. Changed from `split(':', 2)` to `indexOf(':')` for proper value extraction.
+- **Tree-sitter rules directory resolution** — Fixed path resolution to use `ctx.cwd` instead of hardcoded `.pi-lens/rules/` path. Rules now load correctly from the actual project root regardless of where pi is invoked.
+- **Tree-sitter post_filter support** — Implemented missing `post_filter` functionality for tree-sitter queries. Rules with post-filters (e.g., semantic validation for `bare-except` vs specific exception handlers) now work correctly instead of being silently skipped.
+- **Event handler silent crashes** — Wrapped all event handlers in try/catch to prevent unhandled exceptions from crashing the extension silently. Errors are now logged to stderr instead of terminating the process.
+- **Latency logging restored** — Fixed missing latency logging in `tool_result` handler. Runner timing data now correctly flows to `~/.pi-lens/latency.log` again.
+
+### Removed
+- **Broken ast-grep rules** — Removed overlapping rules that were causing false positives or conflicts with tree-sitter coverage.
+
+---
+
+## [3.4.0] - 2026-04-02
+
+### Fixed
+- **Delta mode was broken** — `dispatchLint()` created a fresh empty baseline store on every call, making delta filtering a complete no-op. Every issue looked "new" every time. Now uses a persistent session-level baseline store. First write captures baseline, subsequent writes only show NEW issues.
+- **Duplicate type-checking with `--lens-lsp`** — Both the `lsp` runner (priority 4) and `ts-lsp` runner (priority 5) were calling the same LSP service for TypeScript files. `ts-lsp` now skips when `--lens-lsp` is active.
+
+### Added
+- **Inline security rules via ast-grep-napi** — Re-enabled the ast-grep-napi runner for real-time blocking on security violations (`no-eval`, `jwt-no-verify`, `no-hardcoded-secrets`, `weak-rsa-key`, `no-open-redirect`, etc.). Only error-severity rules fire inline; warnings remain in `/lens-booboo`. Skips 5 rules already covered by tree-sitter to avoid duplicates. ~9ms execution time.
+- **Pre-write duplicate detection (two layers):**
+  - **Exact name match** — Checks exported names in new content against the session’s cached export index. If a function/class/type already exists in another file, blocks the write: `🔴 STOP — function X already exists in utils.ts. Import instead.`
+  - **Structural similarity** — Parses new functions, builds AST state matrices, compares against the project index (built at session start). Functions with ≥80% structural similarity trigger a warning with the match location. Non-blocking.
+- **Project similarity index at session start** — Builds 57×72 state matrices for all TS functions at session start (cached to `.pi-lens/index.json`). Makes pre-write similarity checks ~50ms instead of seconds.
+
+### Changed
+- **Extracted post-write pipeline** — Moved the entire post-write pipeline (secrets, format, autofix, dispatch, tests, cascade diagnostics) from `index.ts` into `clients/pipeline.ts`. `index.ts` reduced from 1764 to 1439 lines.
+- **Removed inline complexity warnings** — `⚠️ Complexity increased: +4 cognitive` no longer shown on every write. No agent acts on this mid-task. Complexity data still captured for `/lens-booboo` and `/lens-tdi`.
+- **Simplified pre-write handler** — Removed pre-write TypeScript and LSP diagnostics checks (checked old content before write landed — post-write catches everything). Kept only complexity baseline capture and duplicate detection.
+
+---
+
 ## [3.3.1] - 2026-04-02
 
 ### Fixed

package/README.md

@@ -10,7 +10,7 @@
 3. **Scans for secrets** — Blocks on hardcoded API keys, tokens, passwords
 4. **Runs linters** — Biome (TS/JS), Ruff (Python), plus structural analysis
 5. **Tree-sitter analysis** — Deep structural patterns (empty catch, eval, deep nesting, mixed async styles)
-6. **Auto-installs** — TypeScript, Python, Biome, Ruff tools install automatically on first use
+6. **Auto-installs** — TypeScript, Python, Biome, Ruff, and analysis tools auto-install on first use
 7. **Only shows NEW issues** — Delta-mode tracks baselines and filters pre-existing problems
 
 **🔴 Blockers** (type errors, secrets, empty catches) appear inline and stop the agent until fixed.  
@@ -28,10 +28,7 @@ pi
 # Disable auto-formatting if needed
 pi --no-autoformat
 
-# Full LSP mode (31 language servers)
-pi --lens-lsp
-
-# LSP mode (recommended for large projects)
+# Full LSP mode (31 language servers) — recommended for large/multi-language projects
 pi --lens-lsp
 ```
 
@@ -131,7 +128,7 @@ Enable full Language Server Protocol support with `--lens-lsp`:
 | **Config** | YAML, JSON, Prisma |
 | **Web** | Vue, Svelte, CSS/SCSS/Sass/Less |
 
-**Auto-installation (4 core tools):** TypeScript, Python, and formatting tools auto-install on first use to `.pi-lens/tools/`. Other LSP servers are launched via `npx` when available or require manual installation.
+**Auto-installation (8 tools):** TypeScript, Python, Biome, Ruff, and analysis tools (Madge, jscpd, ast-grep, Knip) auto-install on first use to `.pi-lens/tools/`. Other LSP servers are launched via `npx` when available or require manual installation.
 
 **Usage:**
 ```bash
@@ -156,14 +153,6 @@ See [docs/LSP_CONFIG.md](docs/LSP_CONFIG.md) for configuration options.
 
 ---
 
-### Execution Modes
-
-| Mode | Flag | Description |
-|------|------|-------------|
-| **Sequential** | (default) | Runners execute one at a time |
-
----
-
 ### On every write / edit
 
 Every file write/edit triggers multiple analysis phases:
@@ -202,12 +191,12 @@ pi-lens uses a **dispatcher-runner architecture** for extensible multi-language
 | **ruff** | Python | 10 | Warning | Python linting (delta-tracked) |
 | **oxlint** | TS/JS | 12 | Warning | Fast Rust-based JS/TS linter |
 | **tree-sitter** | TS/JS, Python | 14 | Mixed | AST-based structural analysis (21 patterns) — **singleton WASM client** |
-| **ast-grep-napi** | TS/JS | 15 | — | **Disabled by default** — heavy; use `/lens-booboo` for full analysis |
+| **ast-grep-napi** | TS/JS | 15 | Blocking | Security rules inline (no-eval, jwt-no-verify, no-hardcoded-secrets, etc.) |
 | **type-safety** | TS | 20 | Mixed | Switch exhaustiveness (blocking), other (warning) |
 | **shellcheck** | Shell | 20 | Warning | Bash/sh/zsh/fish linting |
 | **python-slop** | Python | 25 | Warning | AI slop detection (~40 patterns) |
 | **spellcheck** | Markdown | 30 | Warning | Typo detection in docs |
-| **similarity** | TS | 35 | Silent | Semantic duplicate detection (metrics only) |
+| **similarity** | TS | 35 | Warning | Semantic duplicate detection (structural similarity) |
 | **architect** | All | 40 | Warning | Architectural rule violations |
 | **go-vet** | Go | 50 | Warning | Go static analysis |
 | **rust-clippy** | Rust | 50 | Warning | Rust linting |
@@ -224,7 +213,7 @@ pi-lens uses a **dispatcher-runner architecture** for extensible multi-language
 - **Warning** — Shown in `/lens-booboo`, not inline (noise reduction)
 - **Silent** — Tracked in metrics only, never shown
 
-**Consolidated runners:** `ts-slop` merged into `ast-grep-napi` (disabled by default) — CLI ast-grep used for full linter only
+**Consolidated runners:** `ts-slop` merged into `ast-grep-napi` — CLI ast-grep used for full linter via `/lens-booboo`
 
 **Tree-sitter runner patterns** (priority 14, AST-based structural analysis):
 
@@ -244,7 +233,7 @@ Python (6 patterns):
 
 **AI Slop Detection:** 
 - `python-slop` runner (priority 25): ~40 patterns for Python code quality
-- `ast-grep-napi` runner (priority 15): 33 slop patterns + 68 security/architecture rules for TypeScript/JavaScript (disabled by default — use `/lens-booboo` for full ast-grep analysis via CLI)
+- `ast-grep-napi` runner (priority 15): Security rules fire inline (blocking); slop/architecture warnings via `/lens-booboo` only. Skips 5 rules already covered by tree-sitter.
 
 ---
 
@@ -294,7 +283,7 @@ message: "Remove console statements before production"
 severity: warning
 ```
 
-See [docs/ast-grep-rules.md](docs/ast-grep-rules.md) for full guide.
+See [AST_GREP_RULES.md](AST_GREP_RULES.md) for full guide.
 
 ---
 
@@ -393,26 +382,20 @@ Running the full suite on every edit would be too slow. Targeted testing gives i
 
 ### Complexity Metrics
 
-pi-lens calculates comprehensive code quality metrics for every source file:
+pi-lens tracks code quality metrics for every file:
 
-| Metric | Range | Description | Thresholds |
-|--------|-------|-------------|------------|
-| **Maintainability Index (MI)** | 0-100 | Composite score combining complexity, size, and structure | <20: 🔴 Unmaintainable, 20-40: 🟡 Poor, >60: ✅ Good |
-| **Cognitive Complexity** | 0+ | Human mental effort to understand code (nesting penalties) | >20: 🟡 Hard to understand, >50: 🔴 Very complex |
-| **Cyclomatic Complexity** | 1+ | Independent code paths (branch points + 1) | >10: 🟡 Complex function, >20: 🔴 Highly complex |
-| **Max Cyclomatic** | 1+ | Worst function in file | >10 flagged |
-| **Nesting Depth** | 0+ | Maximum block nesting level | >4: 🟡 Deep nesting, >6: 🔴 Excessive |
-| **Code Entropy** | 0-8+ bits | Shannon entropy — unpredictability of code patterns | >3.5: 🟡 Risky AI-induced complexity |
-| **Halstead Volume** | 0+ | Vocabulary × length — unique ops/operands | High = many different operations |
+| Metric | Description | Threshold |
+|--------|-------------|-----------|
+| **Maintainability Index** | 0-100 composite score | >60 ✅ <20 🔴 |
+| **Cognitive Complexity** | Mental effort to understand | >20 🟡 >50 🔴 |
+| **Cyclomatic Complexity** | Independent code paths | >10 🟡 >20 🔴 |
+| **Code Entropy** | Shannon entropy in bits | >4.0 🟡 >7.0 🔴 |
 
-**AI Slop Indicators:**
-- Low MI + high cognitive complexity + high entropy = potential AI-generated spaghetti code
-- Excessive comments (>40%) + low MI = hand-holding anti-patterns
-- Single-use helpers with high entropy = over-abstraction
+**Commands:**
+- `/lens-tdi` — Technical Debt Index (0-100) with grades A-F
+- `/lens-booboo` — Full complexity table for all files
 
-**Usage:**
-- `/lens-booboo` — Shows complexity table for all files
-- `tool_result` — Complexity tracked per file, AI slop warnings inline
+See [docs/COMPLEXITY_METRICS.md](docs/COMPLEXITY_METRICS.md) for formulas and detailed calculations.
 
 ---
 
@@ -429,7 +412,7 @@ pi-lens works out of the box for TypeScript/JavaScript. For full language suppor
 | `knip` | `npm i -D knip` | Dead code / unused exports |
 | `jscpd` | `npm i -D jscpd` | Copy-paste detection |
 | `type-coverage` | `npm i -D type-coverage` | TypeScript `any` coverage % |
-| `@ast-grep/napi` | `npm i -D @ast-grep/napi` | Fast structural analysis (TS/JS) — currently disabled in realtime |
+| `@ast-grep/napi` | `npm i -D @ast-grep/napi` | Fast structural analysis (TS/JS) — security rules inline, slop in booboo |
 | `@ast-grep/cli` | `npm i -D @ast-grep/cli` | Structural pattern matching (all languages) |
 | `typos-cli` | `cargo install typos-cli` | Spellcheck for Markdown |
 
@@ -465,7 +448,6 @@ pi-lens works out of the box for TypeScript/JavaScript. For full language suppor
 | Command | Description |
 |---------|-------------|
 | `/lens-booboo` | Full codebase review (10 analysis runners) |
-| `/lens-format` | Apply Biome formatting |
 | `/lens-tdi` | Technical Debt Index and trends |
 
 ---
@@ -476,7 +458,6 @@ pi-lens works out of the box for TypeScript/JavaScript. For full language suppor
 |------|---------|--------------|
 | **Standard** (default) | `pi` | Auto-formatting, TS/Python type-checking, sequential execution |
 | **Full LSP** | `pi --lens-lsp` | Real LSP servers (31 languages), sequential execution |
-| **Fastest** | `pi --lens-lsp` | Real LSP + full runner suite |
 
 
 ### Flag Reference
@@ -502,193 +483,90 @@ pi-lens works out of the box for TypeScript/JavaScript. For full language suppor
 ```bash
 pi                               # Default: auto-format, auto-fix, built-in type-checking
 pi --lens-lsp                    # LSP type-checking (31 languages)
-pi --lens-lsp                    # LSP mode (recommended)
 ```
 
 ---
 
 ## TypeScript LSP — tsconfig detection
 
-The LSP walks up from the edited file's directory until it finds a `tsconfig.json`. If found, it uses that project's exact `compilerOptions` (paths, strict settings, lib, etc.). If not found, it falls back to sensible defaults:
-
-- `target: ES2020`
-- `lib: ["es2020", "dom", "dom.iterable"]`
-- `moduleResolution: bundler`
-- `strict: true`
-
-The compiler options are refreshed automatically when you switch between projects within a session.
+The LSP walks up from edited files to find `tsconfig.json`, using its `compilerOptions` (paths, strict settings, etc.). Falls back to sensible defaults if not found.
 
 ---
 
-## Exclusion Criteria
-
-pi-lens automatically excludes certain files from analysis to reduce noise and focus on production code.
-
-### Test Files
-
-All runners respect test file exclusions — both in the dispatch system (`skipTestFiles: true`) and the `/lens-booboo` command.
-
-**Excluded patterns:**
-```
-**/*.test.ts      **/*.test.tsx      **/*.test.js      **/*.test.jsx
-**/*.spec.ts      **/*.spec.tsx      **/*.spec.js      **/*.spec.jsx
-**/*.poc.test.ts  **/*.poc.test.tsx
-**/test-utils.ts  **/test-*.ts
-**/__tests__/**  **/tests/**  **/test/**
-```
-
-**Why:** Test files intentionally duplicate patterns (test fixtures, mock setups) and have different complexity standards. Including them creates false positives.
-
-### Build Artifacts (TypeScript Projects)
-
-In TypeScript projects (detected by `tsconfig.json` presence), compiled `.js` files are excluded:
+## Project Structure
 
 ```
-**/*.js   **/*.jsx   (when corresponding .ts/.tsx exists)
+pi-lens/
+├── clients/          # Lint tools, LSP clients, formatters
+├── commands/         # /lens-booboo, /lens-format commands
+├── docs/             # Documentation
+├── rules/            # AST-grep rules
+├── skills/           # Built-in pi skills
+├── index.ts          # Main extension entry point
+└── package.json
 ```
 
-**Why:** In TS projects, `.js` files are build artifacts. Analyzing them duplicates every issue (once in source `.ts`, once in compiled `.js`).
-
-**Note:** In pure JavaScript projects (no `tsconfig.json`), `.js` files are **included** as they are the source files.
-
-### Excluded Directories
-
-| Directory | Reason |
-|-----------|--------|
-| `node_modules/` | Third-party dependencies |
-| `.git/` | Version control metadata |
-| `dist/`, `build/` | Build outputs |
-| `.pi-lens/`, `.pi/` | pi agent internal files |
-| `.next/`, `.ruff_cache/` | Framework/build caches |
-| `coverage/` | Test coverage reports |
-
-### Per-Runner Exclusion Summary
-
-| Runner | Test Files | Build Artifacts | Directories |
-|--------|-----------|-----------------|-------------|
-| **dispatch runners** | ✅ `skipTestFiles` | ✅ `.js` excluded in TS | ✅ `EXCLUDED_DIRS` |
-| **booboo /lens-booboo** | ✅ `shouldIncludeFile()` | ✅ `isTsProject` check | ✅ `EXCLUDED_DIRS` |
-| **Secrets scan** | ❌ No exclusion (security) | ❌ No exclusion | ✅ Dirs excluded |
+See source for detailed structure.
 
 ---
 
-## Caching Architecture
+## Skills
 
-pi-lens uses a multi-layer caching strategy to avoid redundant work:
+pi-lens includes two built-in skills that guide the LLM on when to use specific tools:
 
-### 1. Tool Availability Cache
+### ast-grep
 
-**Location:** `clients/tool-availability.ts`
+**Purpose:** Guide AST-aware pattern matching for semantic code search/replace.
 
-```
-┌─────────────────────────────────────────┐
-│         TOOL AVAILABILITY CACHE          │
-│  Map<toolName, {available, version}>     │
-│  • Persisted for session lifetime         │
-│  • Refreshed on extension restart        │
-└─────────────────────────────────────────┘
-```
-
-Avoids repeated `which`/`where` calls to check if `biome`, `ruff`, `pyright`, etc. are installed.
+**When to load:** Use `/skill:ast-grep` when performing structural code searches (finding function calls, class methods, imports) or replacements across files.
 
-### 2. Dispatch Baselines (Delta Mode)
+**Key guidance:**
+- Use `$VAR` for single nodes, `$$$` for multiple
+- Patterns must be **complete valid code** (not fragments)
+- **Workflow:** Search → Dry-run (`apply: false`) → Apply (`apply: true`)
+- **Error "Multiple AST nodes":** Use metavariables like `it($TEST)` not raw text like `it"test"`
 
-**Location:** `clients/dispatch/dispatcher.ts`
+```typescript
+// ✅ GOOD: Complete code with metavariables
+ast_grep_search
+  pattern: "console.log($MSG)"
+  lang: typescript
+  paths: ["src/"]
 
-```
-┌─────────────────────────────────────────┐
-│         DISPATCH BASELINES              │
-│  Map<filePath, Diagnostic[]>            │
-│  • Cleared at turn start                 │
-│  • Updated after each runner execution   │
-│  • Filters: only NEW issues shown        │
-└─────────────────────────────────────────┘
+// ❌ BAD: Incomplete pattern
+pattern: "console.log("  // Missing args/body
 ```
 
-Delta mode tracking: first edit shows all issues, subsequent edits only show issues that weren't there before.
+### lsp-navigation
 
-### 3. Client-Level Caches
+**Purpose:** Guide code intelligence via Language Server Protocol.
 
-| Client | Cache | TTL | Purpose |
-|--------|-------|-----|---------|
-| **Knip** | `clients/cache-manager.ts` | 5 min | Dead code analysis (slow) |
-| **jscpd** | `clients/cache-manager.ts` | 5 min | Duplicate detection (slow) |
-| **Type Coverage** | In-memory | Session | `any` type percentage |
-| **Complexity** | In-memory | File-level | MI, cognitive complexity per file |
+**When to load:** Use `/skill:lsp-navigation` for understanding code structure — definitions, references, types, call hierarchy.
 
-### 4. Session Turn State
+**Key guidance:**
+- **LSP is PRIMARY** for code intelligence — NOT grep/glob/ast-grep
+- Requires `--lens-lsp` flag
+- Call hierarchy: `prepareCallHierarchy` → `incomingCalls`/`outgoingCalls`
 
-**Location:** `clients/cache-manager.ts`
+| Task | Use LSP | Use Other |
+|------|---------|-----------|
+| "Where is this defined?" | `definition` | — |
+| "Find all usages" | `references` | — |
+| "What type is this?" | `hover` | — |
+| "Who calls this function?" | `prepareCallHierarchy` → `incomingCalls` | — |
+| Find patterns (console.log) | — | `ast_grep_search` |
+| Find TODO comments | — | `grep` |
 
-```
-┌─────────────────────────────────────────┐
-│         TURN STATE TRACKING               │
-│  • Modified files this turn              │
-│  • Modified line ranges per file         │
-│  • Import changes detected               │
-│  • Turn cycle counter (max 10)           │
-└─────────────────────────────────────────┘
-```
+```typescript
+// ✅ Code intelligence → LSP
+lsp_navigation
+  operation: "references"
+  filePath: "src/utils.ts"
+  line: 42
+  character: 10
 
-Tracks which files were edited in the current agent turn for:
-- jscpd: Only re-scan modified files
-- Madge: Only check deps if imports changed
-- Cycle detection: Prevents infinite fix loops
-
-### 5. Runner Internal Caches
-
-| Runner | Cache | Notes |
-|--------|-------|-------|
-| `ast-grep-napi` | Rule descriptions | Loaded once per session (disabled by default) |
-| `biome` | Tool availability | Checked once, cached |
-| `pyright` | Command path | Venv lookup cached |
-| `ruff` | Command path | Venv lookup cached |
-
----
-
-## Project Structure
-
-```
-pi-lens/
-├── clients/              # Lint tool wrappers and utilities
-│   ├── bus/              # Event bus system (Phase 1)
-│   │   ├── bus.ts
-│   │   ├── events.ts
-│   │   └── integration.ts
-│   ├── dispatch/         # Dispatcher and runners
-│   │   ├── dispatcher.ts
-│   │   └── runners/      # Individual runners
-│   │       ├── ast-grep-napi.ts      # Fast TS/JS runner (disabled by default)
-│   │       ├── python-slop.ts        # Python slop detection
-│   │       ├── ts-lsp.ts             # TS type checking
-│   │       ├── biome.ts
-│   │       ├── ruff.ts
-│   │       ├── pyright.ts
-│   │       ├── go-vet.ts
-│   │       └── rust-clippy.ts
-│   ├── lsp/              # LSP client system (Phase 3)
-│   │   ├── client.ts
-│   │   ├── server.ts     # 31 LSP server definitions
-│   │   ├── language.ts
-│   │   ├── launch.ts
-│   │   └── config.ts     # Custom LSP configuration
-│   ├── installer/          # Auto-installation (Phase 4)
-│   │   └── index.ts
-│   ├── services/           # Effect-TS services (Phase 2)
-│   │   ├── runner-service.ts
-│   │   └── effect-integration.ts
-│   ├── complexity-client.ts
-│   ├── type-safety-client.ts
-│   └── secrets-scanner.ts
-├── commands/             # pi commands
-│   ├── booboo.ts
-│   └── fix-simplified.ts
-├── docs/                 # Documentation
-│   └── LSP_CONFIG.md     # LSP configuration guide
-├── rules/                # AST-grep rules
-│   └── ast-grep-rules/   # General structural rules
-├── index.ts              # Main entry point
-└── package.json
+// ❌ Don't use LSP for text patterns
+pattern: "TODO"  // Use grep instead
 ```
 
 ---
@@ -699,8 +577,9 @@ See [CHANGELOG.md](CHANGELOG.md) for full history.
 
 ### Latest Highlights
 
+- **Tree-sitter Query Cache:** Compiled query cache with mtime-based invalidation — 10× faster structural analysis startup
 - **LSP Support:** 31 Language Server Protocol clients (4 core auto-installed, others via npx or manual)
-- **NAPI Runner:** 100x faster TypeScript/JavaScript structural analysis (~9ms vs ~1200ms) — currently disabled in realtime due to stability
+- **NAPI Runner:** 100x faster TypeScript/JavaScript structural analysis (~9ms vs ~1200ms) — security rules fire inline
 - **Slop Detection:** 33+ TypeScript and 40+ Python patterns for AI-generated code quality issues
 
 ---

package/clients/biome-client.js

@@ -73,6 +73,34 @@ export class BiomeClient {
         }
         return this.biomeAvailable;
     }
+    /**
+     * Ensure Biome is available, auto-installing if necessary.
+     * Prefer this over isAvailable() for auto-install behavior.
+     */
+    async ensureAvailable() {
+        if (this.biomeAvailable !== null)
+            return this.biomeAvailable;
+        // Check if already available
+        const result = this.spawnBiome(["--version"], 10000);
+        if (!result.error && result.status === 0) {
+            this.biomeAvailable = true;
+            return true;
+        }
+        // Auto-install via pi-lens installer
+        this.log("Biome not found, attempting auto-install...");
+        const { ensureTool } = await import("./installer/index.js");
+        const installedPath = await ensureTool("biome");
+        if (installedPath) {
+            this.log(`Biome auto-installed: ${installedPath}`);
+            // Set the installed path as local binary to avoid npx overhead
+            this.localBinaryPath = installedPath;
+            this.biomeAvailable = true;
+            return true;
+        }
+        this.log("Biome auto-install failed");
+        this.biomeAvailable = false;
+        return false;
+    }
     /**
      * Check if a file is supported by Biome
      */

package/clients/biome-client.ts

@@ -110,6 +110,38 @@ export class BiomeClient {
 		return this.biomeAvailable;
 	}
 
+	/**
+	 * Ensure Biome is available, auto-installing if necessary.
+	 * Prefer this over isAvailable() for auto-install behavior.
+	 */
+	async ensureAvailable(): Promise<boolean> {
+		if (this.biomeAvailable !== null) return this.biomeAvailable;
+
+		// Check if already available
+		const result = this.spawnBiome(["--version"], 10000);
+		if (!result.error && result.status === 0) {
+			this.biomeAvailable = true;
+			return true;
+		}
+
+		// Auto-install via pi-lens installer
+		this.log("Biome not found, attempting auto-install...");
+		const { ensureTool } = await import("./installer/index.js");
+		const installedPath = await ensureTool("biome");
+
+		if (installedPath) {
+			this.log(`Biome auto-installed: ${installedPath}`);
+			// Set the installed path as local binary to avoid npx overhead
+			this.localBinaryPath = installedPath;
+			this.biomeAvailable = true;
+			return true;
+		}
+
+		this.log("Biome auto-install failed");
+		this.biomeAvailable = false;
+		return false;
+	}
+
 	/**
 	 * Check if a file is supported by Biome
 	 */

package/clients/cache/rule-cache.js

@@ -0,0 +1,72 @@
+/**
+ * Rule Cache for pi-lens
+ *
+ * Provides disk-based caching for parsed tree-sitter rules with
+ * automatic invalidation based on rule file modification times.
+ */
+import * as crypto from "node:crypto";
+import * as fs from "node:fs";
+import * as path from "node:path";
+const CACHE_DIR = path.join(process.cwd(), ".pi-lens", "cache");
+const CACHE_VERSION = "v1";
+export class RuleCache {
+    constructor(language) {
+        this.cacheFile = path.join(CACHE_DIR, `${language}-rules-${CACHE_VERSION}.json`);
+    }
+    ensureCacheDir() {
+        if (!fs.existsSync(CACHE_DIR)) {
+            fs.mkdirSync(CACHE_DIR, { recursive: true });
+        }
+    }
+    computeRuleHash(ruleFiles) {
+        const hash = crypto.createHash("sha256");
+        for (const file of ruleFiles.sort()) {
+            if (fs.existsSync(file)) {
+                const stat = fs.statSync(file);
+                hash.update(`${file}:${stat.mtimeMs}:${stat.size}`);
+            }
+        }
+        return hash.digest("hex").slice(0, 16);
+    }
+    get(ruleFiles) {
+        try {
+            this.ensureCacheDir();
+            if (!fs.existsSync(this.cacheFile))
+                return null;
+            const cached = JSON.parse(fs.readFileSync(this.cacheFile, "utf-8"));
+            const currentHash = this.computeRuleHash(ruleFiles);
+            if (cached.version !== CACHE_VERSION || cached.ruleHash !== currentHash) {
+                return null; // Cache invalid
+            }
+            return cached;
+        }
+        catch {
+            return null;
+        }
+    }
+    set(ruleFiles, queries) {
+        try {
+            this.ensureCacheDir();
+            const entry = {
+                version: CACHE_VERSION,
+                timestamp: Date.now(),
+                ruleHash: this.computeRuleHash(ruleFiles),
+                queries,
+            };
+            fs.writeFileSync(this.cacheFile, JSON.stringify(entry, null, 2));
+        }
+        catch {
+            // Cache write failure is non-fatal
+        }
+    }
+    clear() {
+        try {
+            if (fs.existsSync(this.cacheFile)) {
+                fs.unlinkSync(this.cacheFile);
+            }
+        }
+        catch {
+            // Ignore
+        }
+    }
+}