pi-lens 3.0.0 → 3.1.0

package/CHANGELOG.md

@@ -2,6 +2,99 @@
 
 All notable changes to pi-lens will be documented in this file.
 
+## [3.1.0] - 2026-04-01
+
+### Changed
+- **Consolidated ast-grep runners** — Unified CLI and NAPI runners with shared rule set
+  - NAPI runner now primary for dispatch (100x faster than CLI spawn)
+  - Merged ts-slop-rules (21 files) into ast-grep-rules/slop-patterns.yml (33 patterns)
+  - Removed 20 duplicate rule files with conflicting IDs (e.g., `ts-jwt-no-verify` vs `jwt-no-verify`)
+  - Total: 104 unified rules (71 security/architecture + 33 slop patterns)
+  - CLI ast-grep kept only for `ast_grep_search` / `ast_grep_replace` tools
+
+### Fixed
+- **ast-grep-napi stability** — Fixed stack overflow crashes in AST traversal
+  - Added `_MAX_AST_DEPTH = 50` depth limit to `findByKind()` and `getAllNodes()`
+  - Added `_MAX_RULE_DEPTH = 5` recursion limit for structured rules
+  - Added `MAX_MATCHES_PER_RULE = 10` to prevent false positive explosions
+  - Added `MAX_TOTAL_DIAGNOSTICS = 50` to prevent output spam
+  - NAPI runner now safely handles deeply nested TypeScript files
+
+---
+
+## [3.0.1] - 2026-03-31
+
+### Changed
+- **Documentation refresh**: Updated npm and README descriptions for v3.0.0 features
+  - New tagline: "pi extension for real-time code quality"
+  - Highlights 31 LSP servers, tree-sitter analysis, auto-install capability
+  - Clarified blockers vs warnings split (inline vs `/lens-booboo`)
+
+### Fixed
+- **Entropy threshold**: Increased from 3.5 → 5.5 bits to reduce false positives
+  - Previous threshold was too sensitive for tooling codebases
+  - Eliminates ~70-80% of "High entropy" warnings on legitimate complex code
+
+---
+
+## [3.0.0] - 2026-03-31
+
+### Breaking Changes
+
+#### Removed - Deprecated Commands
+The following deprecated commands have been removed:
+- `/lens-booboo-fix` → Use `/lens-booboo` with autofix capability
+- `/lens-booboo-delta` → Delta mode now automatic
+- `/lens-booboo-refactor` → Use `/lens-booboo` findings
+- `/lens-metrics` → Metrics now in `/lens-booboo` report
+- `/lens-rate` → Use `/lens-booboo` quality scoring
+
+#### Changed - Blockers vs Warnings Architecture
+- **🔴 Blockers** (type errors, secrets, empty catch blocks) → Appear **inline** and stop the agent
+- **🟡 Warnings** (complexity, code smells) → Go to **`/lens-booboo`** only (not inline)
+- Tree-sitter rules with `severity: error` now properly block inline
+- Dispatcher checks individual diagnostic semantic, not just group default
+
+### Added - Tree-Sitter Runner
+New structural analysis runner at priority 14:
+- **18 YAML query files** for TypeScript and Python patterns
+- TypeScript: empty-catch, eval, debugger, console-statement, hardcoded-secrets, deep-nesting, deep-promise-chain, mixed-async-styles, nested-ternary, long-parameter-list, await-in-loop, dangerously-set-inner-html
+- Python: bare-except, eval-exec, wildcard-import, is-vs-equals, mutable-default-arg, unreachable-except
+- Blockers appear inline (severity: error), warnings go to `/lens-booboo` (severity: warning)
+
+### Added - Auto-Install for Core Tools
+Four tools now auto-install on first use (no manual setup required):
+1. **TypeScript Language Server** (`typescript-language-server`) — TS/JS type checking
+2. **Pyright** — Python type checking (`pip install pyright`)
+3. **Ruff** — Python linting (`pip install ruff`)
+4. **Biome** — JS/TS/JSON linting and formatting
+
+Installs to `.pi-lens/tools/` with verification step (`--version` check).
+
+### Added - NAPI Security Rules
+Migrated 20 critical security rules to NAPI (fast native execution):
+- Rules with `weight >= 4` are **blocking** (stop the agent)
+- Includes: no-eval, no-hardcoded-secrets, no-implied-eval, no-inner-html, no-dangerously-set-inner-html, no-debugger, no-javascript-url, no-open-redirect, no-mutable-default, weak-rsa-key, jwt-no-verify, and more
+- NAPI runs at priority 15 (after tree-sitter, before slop rules)
+
+### Fixed
+- **Tree-sitter query loading**: Added missing `loadQueries()` call before `getAllQueries()`
+- **Windows path handling**: Changed from `lastIndexOf("/")` to `path.dirname()` for cross-platform compatibility
+- **Dispatcher blocker detection**: Now checks if any individual diagnostic has `semantic === "blocking"`
+- **Biome runner npx fallback**: Uses `npx biome` when `biome` not in PATH directly
+- **LSP ENOENT crashes**: Added `_attachErrorHandler()` to all 23 manual-install LSP servers
+- **LSP initialization timeout**: Increased to 120s (was 45s)
+- **ESLint scope reduction**: Removed `.ts/.tsx` from ESLint LSP (now JS/framework files only)
+- **Biome/Prettier race**: Biome is now default (priority 10), Prettier is fallback only
+
+### Changed
+- **README reorganization**: Removed redundant sections (Architecture, Language Support, Rules, Delta-mode, Slop Detection)
+- **Consolidated Additional Safeguards** into Features section with Runners table
+- **Updated .gitignore**: Local tracking files stay out of repo
+- **Tuned thresholds**: 70-80% false positive reduction in booboo reports
+
+---
+
 ## [2.7.0] - 2026-03-31
 
 ### Added - New Lint Runners

package/README.md

@@ -1,18 +1,20 @@
 # pi-lens
 
-Real-time code feedback for [pi](https://github.com/mariozechner/pi-coding-agent) — LSP, linters, formatters, type-checking, structural analysis (ast-grep), TODO scanner, dead code detection, duplicate detection, type coverage, complexity metrics, and AI slop detection.
+**pi extension for real-time code quality.** 31 LSP servers, tree-sitter structural analysis, AST pattern matching, auto-install for TypeScript/Python tooling, duplicate detection, complexity metrics, and inline blockers with comprehensive `/lens-booboo` reports.
 
 ## What pi-lens Does
 
 **For every file you edit:**
 1. **Auto-formats** — Detects and runs formatters (Biome, Prettier, Ruff, gofmt, rustfmt, etc.)
-2. **Type-checks** TypeScript, Python, Go, Rust (and 27 more languages with `--lens-lsp`)
-3. **Scans for secrets** — blocks on hardcoded API keys, tokens, passwords
+2. **Type-checks** — TypeScript, Python, Go, Rust (31 languages with `--lens-lsp`)
+3. **Scans for secrets** — Blocks on hardcoded API keys, tokens, passwords
 4. **Runs linters** — Biome (TS/JS), Ruff (Python), plus structural analysis
-5. **Detects code smells** — empty catch blocks, debuggers, nested ternaries, etc.
-6. **Only shows NEW issues** — delta-mode tracks baselines and filters pre-existing problems (reduces noise)
+5. **Tree-sitter analysis** — Deep structural patterns (empty catch, eval, deep nesting, mixed async styles)
+6. **Auto-installs** — TypeScript, Python, Biome, Ruff tools install automatically on first use
+7. **Only shows NEW issues** — Delta-mode tracks baselines and filters pre-existing problems
 
-**Blocking issues** (type errors, secrets) appear inline and stop the agent until fixed. **Warnings** are tracked but hidden inline — run `/lens-booboo` to see them all.
+**🔴 Blockers** (type errors, secrets, empty catches) appear inline and stop the agent until fixed.  
+**🟡 Warnings** (complexity, code smells) go to `/lens-booboo` — run it to see them all.
 
 ## Quick Start
 
@@ -204,12 +206,11 @@ pi-lens uses a **dispatcher-runner architecture** for extensible multi-language
 | **ruff** | Python | 10 | Warning | Python linting (delta-tracked) |
 | **oxlint** | TS/JS | 12 | Warning | Fast Rust-based JS/TS linter |
 | **tree-sitter** | TS/JS, Python | 14 | Mixed | AST-based structural analysis (17 patterns) |
-| **ast-grep-napi** | TS/JS | 15 | Warning | **100x faster** structural analysis |
+| **ast-grep-napi** | TS/JS | 15 | Warning | **Unified structural analysis** (104 rules) |
 | **type-safety** | TS | 20 | Mixed | Switch exhaustiveness (blocking), other (warning) |
 | **shellcheck** | Shell | 20 | Warning | Bash/sh/zsh/fish linting |
 | **python-slop** | Python | 25 | Warning | AI slop detection (~40 patterns) |
 | **spellcheck** | Markdown | 30 | Warning | Typo detection in docs |
-| **ast-grep** | Go, Rust, Python, etc. | 30 | Warning | Structural analysis via CLI (fallback for non-TS/JS) |
 | **similarity** | TS | 35 | Silent | Semantic duplicate detection (metrics only) |
 | **architect** | All | 40 | Warning | Architectural rule violations |
 | **go-vet** | Go | 50 | Warning | Go static analysis |
@@ -227,7 +228,7 @@ pi-lens uses a **dispatcher-runner architecture** for extensible multi-language
 - **Warning** — Shown in `/lens-booboo`, not inline (noise reduction)
 - **Silent** — Tracked in metrics only, never shown
 
-**Disabled runners:** `ts-slop` (merged into `ast-grep-napi`)
+**Consolidated runners:** `ast-grep` (CLI) and `ts-slop` merged into `ast-grep-napi` — unified 104-rule set
 
 **Tree-sitter runner patterns** (priority 14, AST-based structural analysis):
 
@@ -241,7 +242,9 @@ Python (6 patterns):
 
 **Custom tree-sitter queries:** Add `.yml` files to `.pi-lens/rules/tree-sitter-queries/{typescript,python}/`
 
-**AI Slop Detection:** The `python-slop` runner (priority 25) detects low-quality patterns in Python code (~40 patterns). TypeScript/JavaScript slop detection is integrated into `ast-grep-napi` runner.
+**AI Slop Detection:** 
+- `python-slop` runner (priority 25): ~40 patterns for Python code quality
+- `ast-grep-napi` runner (priority 15): 33 slop patterns + 71 security/architecture rules for TypeScript/JavaScript
 
 ---
 

package/clients/bus/integration.js

@@ -7,8 +7,8 @@
  * - Real-time progress tracking
  * - Hook integration for tool_result handler
  */
-import { RunnerStarted, RunnerCompleted, ReportReady, FileModified, DiagnosticAggregator, } from "./events.js";
 import { enableDebug } from "./bus.js";
+import { DiagnosticAggregator, FileModified, ReportReady, RunnerCompleted, RunnerStarted, } from "./events.js";
 const state = {
     aggregator: new DiagnosticAggregator(),
     runnerInProgress: new Set(),
@@ -21,7 +21,7 @@ let unsubscribers = [];
  * Initialize the bus integration
  * Call this from session_start handler
  */
-export function initBusIntegration(pi, options) {
+export function initBusIntegration(_pi, options) {
     if (state.isEnabled)
         return; // Already initialized
     if (options?.debug) {
@@ -37,13 +37,13 @@ export function initBusIntegration(pi, options) {
     const unsubRunnerCompleted = RunnerCompleted.subscribe((event) => {
         const { runnerId, filePath, durationMs, diagnosticCount } = event.properties;
         state.runnerInProgress.delete(`${runnerId}:${filePath}`);
-        // Log slow runners in debug mode
+        // Log slow runners in debug mode only
         if (options?.debug && durationMs > 5000) {
             console.error(`[bus] Slow runner: ${runnerId} took ${durationMs}ms for ${filePath}`);
         }
-        // Log runners that found issues
-        if (diagnosticCount > 0) {
-            console.error(`[bus] ${runnerId} found ${diagnosticCount} issues in ${filePath}`);
+        // Log runners that found excessive issues (indicates rule misconfiguration)
+        if (diagnosticCount > 100) {
+            console.error(`[bus] ${runnerId} found ${diagnosticCount} issues in ${filePath} (rules may be too broad)`);
         }
     });
     // Cache reports for quick retrieval

package/clients/bus/integration.ts

@@ -1,6 +1,6 @@
 /**
  * Bus Integration for pi-lens
- * 
+ *
  * Connects the event bus system to the existing pi-lens architecture.
  * This provides:
  * - Event aggregation for diagnostic collection
@@ -8,19 +8,16 @@
  * - Hook integration for tool_result handler
  */
 
+import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";
+import { enableDebug } from "./bus.js";
 import {
-	DiagnosticFound,
-	RunnerStarted,
-	RunnerCompleted,
-	ReportReady,
-	FileModified,
-	SessionStarted,
-	TurnEnded,
-	DiagnosticAggregator,
 	type Diagnostic,
+	DiagnosticAggregator,
+	FileModified,
+	ReportReady,
+	RunnerCompleted,
+	RunnerStarted,
 } from "./events.js";
-import { subscribe, enableDebug } from "./bus.js";
-import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";
 
 // --- Integration State ---
 
@@ -46,7 +43,10 @@ let unsubscribers: Array<() => void> = [];
  * Initialize the bus integration
  * Call this from session_start handler
  */
-export function initBusIntegration(pi: ExtensionAPI, options?: { debug?: boolean }): void {
+export function initBusIntegration(
+	_pi: ExtensionAPI,
+	options?: { debug?: boolean },
+): void {
 	if (state.isEnabled) return; // Already initialized
 
 	if (options?.debug) {
@@ -63,24 +63,29 @@ export function initBusIntegration(pi: ExtensionAPI, options?: { debug?: boolean
 	});
 
 	const unsubRunnerCompleted = RunnerCompleted.subscribe((event) => {
-		const { runnerId, filePath, durationMs, diagnosticCount } = event.properties;
+		const { runnerId, filePath, durationMs, diagnosticCount } =
+			event.properties;
 		state.runnerInProgress.delete(`${runnerId}:${filePath}`);
 
-		// Log slow runners in debug mode
+		// Log slow runners in debug mode only
 		if (options?.debug && durationMs > 5000) {
-			console.error(`[bus] Slow runner: ${runnerId} took ${durationMs}ms for ${filePath}`);
+			console.error(
+				`[bus] Slow runner: ${runnerId} took ${durationMs}ms for ${filePath}`,
+			);
 		}
 
-		// Log runners that found issues
-		if (diagnosticCount > 0) {
-			console.error(`[bus] ${runnerId} found ${diagnosticCount} issues in ${filePath}`);
+		// Log runners that found excessive issues (indicates rule misconfiguration)
+		if (diagnosticCount > 100) {
+			console.error(
+				`[bus] ${runnerId} found ${diagnosticCount} issues in ${filePath} (rules may be too broad)`,
+			);
 		}
 	});
 
 	// Cache reports for quick retrieval
 	const unsubReportReady = ReportReady.subscribe((event) => {
 		const { filePath, report, durationMs } = event.properties;
-		
+
 		// Store the report
 		state.lastReport.set(filePath, {
 			output: formatReport(report, durationMs),
@@ -91,10 +96,10 @@ export function initBusIntegration(pi: ExtensionAPI, options?: { debug?: boolean
 	// Track file modifications to clear stale data
 	const unsubFileModified = FileModified.subscribe((event) => {
 		const { filePath } = event.properties;
-		
+
 		// Clear cached report for modified file
 		state.lastReport.delete(filePath);
-		
+
 		// Clear diagnostics aggregator for this file (will be repopulated)
 		state.aggregator.clear(filePath);
 	});
@@ -139,7 +144,12 @@ export function shutdownBusIntegration(): void {
 // --- Helper Functions ---
 
 function formatReport(
-	report: { blockers: Diagnostic[]; warnings: Diagnostic[]; fixed: Diagnostic[]; silent: Diagnostic[] },
+	report: {
+		blockers: Diagnostic[];
+		warnings: Diagnostic[];
+		fixed: Diagnostic[];
+		silent: Diagnostic[];
+	},
 	durationMs: number,
 ): string {
 	const lines: string[] = [];
@@ -198,7 +208,10 @@ export function hasRunnersInProgress(filePath?: string): boolean {
 /**
  * Get list of runners in progress
  */
-export function getRunnersInProgress(): Array<{ runnerId: string; filePath: string }> {
+export function getRunnersInProgress(): Array<{
+	runnerId: string;
+	filePath: string;
+}> {
 	return Array.from(state.runnerInProgress).map((key) => {
 		const [runnerId, filePath] = key.split(":");
 		return { runnerId, filePath };

package/clients/complexity-client.js

@@ -234,9 +234,11 @@ export class ComplexityClient {
         if (metrics.maxNestingDepth > 4) {
             parts.push(`  Max nesting: ${metrics.maxNestingDepth} levels (consider extracting)`);
         }
-        // Code entropy (in bits, >3.5 = risky AI-induced complexity)
-        if (metrics.codeEntropy > 3.5) {
-            parts.push(`  Entropy: ${metrics.codeEntropy.toFixed(1)} bits (>3.5 — risky AI-induced complexity)`);
+        // Code entropy (in bits, >5.5 = risky AI-induced complexity)
+        // Threshold increased from 3.5 to 5.5 to reduce false positives in tooling codebases
+        // where diverse method/variable names are naturally expected
+        if (metrics.codeEntropy > 5.5) {
+            parts.push(`  Entropy: ${metrics.codeEntropy.toFixed(1)} bits (>5.5 — risky AI-induced complexity)`);
         }
         // Function length
         if (metrics.maxFunctionLength > 50) {
@@ -336,8 +338,8 @@ export class ComplexityClient {
         if (metrics.maxNestingDepth > 6) {
             warnings.push(`Deep nesting (${metrics.maxNestingDepth} levels) — extract nested logic into separate functions`);
         }
-        // Entropy > 5.0 is high (was > 3.5, too sensitive)
-        if (metrics.codeEntropy > 5.0) {
+        // Entropy > 5.5 is high (was > 3.5 → 5.0, still too sensitive for tooling codebases)
+        if (metrics.codeEntropy > 5.5) {
             warnings.push(`High entropy (${metrics.codeEntropy.toFixed(1)} bits) — follow project conventions`);
         }
         // Comments ratio (>60% = excessive, was > 40%)
@@ -576,7 +578,8 @@ export class ComplexityClient {
     /**
      * Calculate Shannon entropy of code tokens (in bits)
      * Uses log2 for entropy measured in bits
-     * Threshold: >3.5 bits indicates risky AI-induced complexity
+     * Threshold: >5.5 bits indicates risky AI-induced complexity
+     * (Increased from 3.5 to reduce false positives in tooling codebases)
      */
     calculateCodeEntropy(sourceText) {
         // Tokenize by splitting on whitespace and common delimiters

package/clients/complexity-client.ts

@@ -331,10 +331,12 @@ export class ComplexityClient {
 			);
 		}
 
-		// Code entropy (in bits, >3.5 = risky AI-induced complexity)
-		if (metrics.codeEntropy > 3.5) {
+		// Code entropy (in bits, >5.5 = risky AI-induced complexity)
+		// Threshold increased from 3.5 to 5.5 to reduce false positives in tooling codebases
+		// where diverse method/variable names are naturally expected
+		if (metrics.codeEntropy > 5.5) {
 			parts.push(
-				`  Entropy: ${metrics.codeEntropy.toFixed(1)} bits (>3.5 — risky AI-induced complexity)`,
+				`  Entropy: ${metrics.codeEntropy.toFixed(1)} bits (>5.5 — risky AI-induced complexity)`,
 			);
 		}
 
@@ -480,8 +482,8 @@ export class ComplexityClient {
 			);
 		}
 
-		// Entropy > 5.0 is high (was > 3.5, too sensitive)
-		if (metrics.codeEntropy > 5.0) {
+		// Entropy > 5.5 is high (was > 3.5 → 5.0, still too sensitive for tooling codebases)
+		if (metrics.codeEntropy > 5.5) {
 			warnings.push(
 				`High entropy (${metrics.codeEntropy.toFixed(1)} bits) — follow project conventions`,
 			);
@@ -807,7 +809,8 @@ export class ComplexityClient {
 	/**
 	 * Calculate Shannon entropy of code tokens (in bits)
 	 * Uses log2 for entropy measured in bits
-	 * Threshold: >3.5 bits indicates risky AI-induced complexity
+	 * Threshold: >5.5 bits indicates risky AI-induced complexity
+	 * (Increased from 3.5 to reduce false positives in tooling codebases)
 	 */
 	private calculateCodeEntropy(sourceText: string): number {
 		// Tokenize by splitting on whitespace and common delimiters

package/clients/dispatch/plan.js

@@ -23,12 +23,11 @@ export const TOOL_PLANS = {
             { mode: "all", runnerIds: ["ts-lsp"], filterKinds: ["jsts"] },
             // Then biome or oxlint for fast linting (user preference)
             { mode: "fallback", runnerIds: ["biome-lint", "oxlint"] },
-            // Fast structural analysis via NAPI (weight >= 4 is blocking)
+            // Fast structural analysis via NAPI (replaces CLI - 100x faster)
             { mode: "all", runnerIds: ["ast-grep-napi"] },
             // Type safety checks
             { mode: "fallback", runnerIds: ["type-safety"] },
-            // Comprehensive structural analysis via CLI (severity: error is blocking)
-            { mode: "fallback", runnerIds: ["ast-grep"] },
+            // Note: ast-grep CLI kept for ast_grep_search/ast_grep_replace tools only
             // Architectural rules
             { mode: "fallback", runnerIds: ["architect"] },
         ],

package/clients/dispatch/plan.ts

@@ -27,12 +27,11 @@ export const TOOL_PLANS: Record<string, ToolPlan> = {
 			{ mode: "all", runnerIds: ["ts-lsp"], filterKinds: ["jsts"] },
 			// Then biome or oxlint for fast linting (user preference)
 			{ mode: "fallback", runnerIds: ["biome-lint", "oxlint"] },
-			// Fast structural analysis via NAPI (weight >= 4 is blocking)
+			// Fast structural analysis via NAPI (replaces CLI - 100x faster)
 			{ mode: "all", runnerIds: ["ast-grep-napi"] },
 			// Type safety checks
 			{ mode: "fallback", runnerIds: ["type-safety"] },
-			// Comprehensive structural analysis via CLI (severity: error is blocking)
-			{ mode: "fallback", runnerIds: ["ast-grep"] },
+			// Note: ast-grep CLI kept for ast_grep_search/ast_grep_replace tools only
 			// Architectural rules
 			{ mode: "fallback", runnerIds: ["architect"] },
 		],