pi-lens 3.8.64 → 3.8.66

package/CHANGELOG.md

@@ -10,6 +10,50 @@ All notable changes to pi-lens will be documented in this file.
 
 ### Fixed
 
+## [3.8.66] - 2026-07-07
+
+### Added
+
+- **`lens_diagnostics mode=full` now surfaces the heavyweight project analyzers via an extractor registry** — previously only knip crossed from the heavyweight analyzers into the diagnostic surface; the rest reached the agent only via next-turn context injection. New `project-diagnostics/extractors.ts` registry maps each analyzer's **cached** result to per-file `ProjectDiagnostic`s through pure `runner-adapters/*` functions (mirroring `knip.ts`): **jscpd** copy-paste (a clone → a diagnostic on **both** ends, each naming the other span), **madge** circular deps (a cycle → one on **each** participating file), **gitleaks** secrets (→ **blocking**), **govulncheck** reachable Go CVEs (anchored at the first traced source frame), **trivy** dependency CVEs (anchored at the manifest), and **dead-code** (vulture/Python — unused symbols; unlisted deps → **blocking**). **Cache-only, never re-launched:** `mode=full` reads each analyzer's session-start cache and folds the results in — it never spawns a scan, so it can't relaunch or contend with the background runs (which share a global abort signal). Adding a new analyzer is now one adapter + one registry row. Included when `refreshRunners` is `cached`/`cheap`/`all`.
+- **madge now runs whole-project at session-start and caches its result** — bringing it in line with knip/jscpd/gitleaks/govulncheck (it was the lone analyzer running per-edited-file at turn-end and discarding its output). `lens_diagnostics mode=full` reads the new `madge` cache via the extractor registry, giving whole-project circular-dependency coverage.
+
+### Changed
+
+- **Dropped the `typescript` compiler from runtime dependencies** (#402) — it's now a **devDependency** (the `tsc` build/type-check tool), so it no longer ships to users (`npm install --omit=dev`), saving ~8.4 MB bundle / ~23 MB `node_modules`. The last runtime consumers were the dispatch fact-rules: the low-signal style smells were **dropped** (commented-out-code, duplicate-string-literal, max-switch-cases, no-magic-numbers, no-boolean-params, no-complex-conditionals), two rules were already **dead** (no-magic-numbers, high-entropy-string — never registered; secret detection is covered by the gitleaks extractor), dynamic-regexp/ReDoS was dropped in favor of the existing pattern-based ast-grep `redos-nested-quantifier` + tree-sitter `unsafe-regex` rules, and function-in-loop moves to a follow-up ast-grep rule (#428). The three survivors — **cors-wildcard**, **no-commented-credentials** (regex, multi-language), and **high-import-coupling** (fact-based) — are now individual files under `dispatch/rules/` (matching the one-rule-per-file convention) and register **eagerly** in `integration.ts` like every other rule. That unifies fact-rule registration (#421) and removes the now-obsolete lazy `ensureTypeScriptDispatchUnits` degrade indirection from `fact-runner`: with no `typescript` in the graph and `web-tree-sitter` loaded via a dynamic import, the providers are safe to register eagerly. No user-facing behavior change beyond the dropped rules.
+- **Complexity metrics are now language-agnostic and tree-sitter-based** (#402) — `ComplexityClient` no longer uses the `typescript` compiler; it computes cyclomatic + cognitive complexity, nesting depth, function metrics, LOC/comments, code entropy, and AI-slop indicators over the shared tree-sitter client via a per-language node table. Beyond JS/TS it now also analyzes **Python, Go, and Rust** (adding a language is one table entry). Halstead volume is dropped (its maintainability-index term is replaced with the Halstead-free variant `171 − 0.23·CC − 16.2·ln(LOC)` + comment bonus); the dead `formatMetrics`/`checkThresholds` methods are removed. These are silent session-summary baselines, so the metric surface is unchanged aside from the dropped `halsteadVolume` field. `analyzeFile` is now async (grammar parse). Only the sonar/quality **rules** still import `typescript` — the last #402 Phase-2 step before the dependency is dropped.
+- **The comment / try-catch / function fact extractors now parse via tree-sitter instead of the TypeScript compiler** (#402, Phase 2) — `commentFactProvider`, `tryCatchFactProvider`, and `functionFactProvider` are ported off `ts.createSourceFile` onto the shared tree-sitter client via a new shared `facts/tree-sitter-facts.ts` (parse boilerplate + node-walk helpers, which `import-facts` also now uses). Output is unchanged — the fact-consuming rules (`high-complexity`, `high-fan-out`, `async-noise`, `error-swallowing`, `pass-through-wrappers`, `placeholder-comments`, …) and the review graph read the same `file.comments`/`file.tryCatchSummaries`/`file.functionSummaries`, and these rules still surface as the same code-quality warnings. Parity is locked by the existing try-catch suite + new comment/function-fact tests (names, async/await, pass-through/boundary wrappers, cyclomatic complexity, nesting depth, outgoing calls). Providers are now async; the review-graph caller awaits `functionFactProvider` too. Only `complexity-client` + the sonar/quality rules still use `typescript` (#402 Phase 2 remainder).
+- **`import-facts` now extracts imports via tree-sitter instead of the TypeScript compiler** (#402, Phase 2 pilot) — the `importFactProvider` (static/default/namespace/side-effect imports, dynamic `import()`/`require()`, named + star re-exports, esm/cjs/unknown module-type) is ported off `ts.createSourceFile` onto the shared, cached tree-sitter client (#416). Output (`file.imports`/`file.reexports`, consumed by the `high-import-coupling` rule and the review graph) is unchanged — parity is locked by the existing suite plus new cases (aliased/combined imports, `.tsx`-grammar smoke). The provider's `run()` is now async (grammar parse); the one non-awaited caller (`review-graph/builder.ts`) is fixed to await it. First of the syntactic TS-AST consumers to move off the `typescript` dependency.
+- **Unified tree-sitter parsing on a single shared client** (refs #402) — the dispatch tree-sitter runner, project scanner, `module-report`, and review-graph each held their **own** `TreeSitterClient`, so a file written on the hot path was parsed multiple times with no shared tree cache, and each subsystem re-loaded grammars independently. They now share one process-wide client via `clients/tree-sitter-shared.ts` (`getSharedTreeSitterClient` + a single `resolveTreeSitterLanguage` ext→grammar map), so a file parsed by one subsystem is served from the shared tree cache for the others (one parse per write). This also fixes a latent gap: web-tree-sitter's WASM runtime is module-level (one per process), so an Emscripten `abort()` corrupts it for **everyone** — previously only the runner tracked the poison flag while the scanner/module-report/review-graph kept calling the dead runtime; `markTreeSitterWasmAborted()` now makes every consumer skip. Foundation for porting the syntactic TS-AST consumers (fact providers, complexity, rules) off the `typescript` dependency onto tree-sitter.
+- **CLI tool resolution now finds binaries installed by any package manager, not just npm/PATH** (#375) — the shared `resolveLocalFirstAsync` helper (the runner fleet's "local `.bin` → global → `npx --no`" resolver) plus the per-tool resolvers for **biome** (`biome-client.ts`, `dispatch/runners/biome.ts`, `formatters.ts`), **prettier** (`formatters.ts`), **type-coverage**, **jscpd**, **madge** (`dependency-checker.ts`), **ast-grep** (`sg-runner.ts` + the shared `isSgAvailableAsync`), and the **test runners** (`test-runner-client.ts`) now check every installed manager's global bin dir (npm/pnpm/yarn/bun) by direct file lookup before falling back to `npx`. This finds tools installed via `pnpm add -g` / `bun add -g` (whose bin dirs are often off PATH) and survives PATH-cache staleness right after an `install -g`. Each site's existing `npx` fallback is unchanged — the lookup is purely additive, so no user ever gets a surprise `dlx` download. New `findGlobalBinary` / `findNodeToolBinary` helpers in `package-manager.ts`, which also de-duplicate the identical private lookup in `lsp/launch.ts`.
+- **`lens_diagnostics mode=full` wall-clock ceiling raised from 3 min to 5 min** — large monorepos with many cold language servers were hitting the 180s cap and returning partial results before the sweep finished. The default is now 300s (still env-tunable via `PI_LENS_LENS_DIAGNOSTICS_FULL_TIMEOUT_MS`).
+- **In-flight LSP nav requests now cancel when the turn is abandoned** (#238 Item 1) — `navRequest`/`safeSendRequest` (`clients/lsp/client.ts`) thread the ambient abort signal into a vscode-jsonrpc `CancellationToken`, so aborting a turn (Escape) sends an LSP `$/cancelRequest` and the server stops computing a definition/references/hover/etc. result the agent has already walked away from — reclaiming clangd/pyright/tsserver CPU on the hot path instead of running the request to completion and discarding it. An already-aborted signal skips the send entirely; the resulting cancellation rejection (`RequestCancelled`/`ServerCancelled`) is treated as "no result." Defaults to the ambient signal, so all ~12 nav call sites get it with no signature change.
+
+### Removed
+
+- **Removed the deprecated built-in TypeScript type-checker fallback** (#402, Phase 1) — the `ts-lsp` dispatch runner and its `TypeScriptClient` (`typescript-client.ts`) are deleted, along with the dead `TypeScriptService` (`ts-service.ts`, which had no consumers). TS type-checking is now **LSP-only** (tsserver via the unified `lsp` runner, which is default-on and was already the primary path — `ts-lsp` merely deferred to it). **Behaviour change:** running with `--no-lsp` no longer provides TS *type* diagnostics; the write-path linters (eslint/oxlint/biome) and structural analysis (tree-sitter, ast-grep, fact-rules) are unaffected. This removes the only `ts.createProgram`/`createLanguageService`/`TypeChecker` usage in the codebase — the first step toward dropping the heavyweight `typescript` dependency entirely (remaining usage is purely syntactic AST parsing, portable to tree-sitter).
+- **The `/lens-booboo` command is gone** — its full-codebase review (design smells, complexity, dead code, duplicates, circular deps, secrets, vulns) is now available through the normal diagnostic surface: **`lens_diagnostics mode=full refreshRunners=all`**, which folds in the same heavyweight analyzers via the extractor registry. Also removed the dormant `TypeCoverageClient` (its only caller was `/lens-booboo`; it was never run on the normal path and is TS-only, redundant with LSP strict-mode + biome). The `--lens-guard` commit-block message and `/lens-tdi` now point to `lens_diagnostics` instead. (`FULL_LINT_PLANS`/`fullOnlyGroups` in `dispatch/plan.ts` were orphaned by this removal and are deleted in the follow-up below.)
+- **Removed the orphaned full-project lint plan machinery** (#399, refs #398) — `FULL_LINT_PLANS` + `toFullPlan()` and the `fullOnlyGroups` field/entries in `dispatch/plan.ts` are gone now that `/lens-booboo` (their only consumer) is removed; `TOOL_PLANS` (the per-write plans) is the sole plan surface. The two full-plan-only runners this orphaned — **`biome-lint`** (`dispatch/runners/biome.ts`) and **`python-slop`** (`dispatch/runners/python-slop.ts`, `PRIORITY.PYTHON_SLOP`) — are deregistered and deleted. The `python-slop` **ast-grep rules** (`rules/python-slop-rules/`, ~45 warning-severity Python "slop" patterns) are **kept in the tree** pending a decision to migrate them into the shipped `ast-grep-rules` corpus or delete them (tracked in #400); they were never loaded by the ast-grep LSP or the `ast-grep-napi` runner.
+
+### Fixed
+
+- **`pi install git:…` (GitHub install) no longer fails on a clean machine** (#437) — pi builds the GitHub install by cloning master HEAD and running `npm install --omit=dev` in-place, which triggers `prepare` → `build:dist` → `tsc`. Since #402 made `typescript` a **devDependency**, `--omit=dev` omits it, so on a machine without a cached/global `tsc` the from-source build had no compiler and the install failed (`npm run build:dist … exit 1`). The npm-registry install path was unaffected (it ships prebuilt `dist/`; `prepare` never runs). `build:dist` now resolves the compiler on demand via `npx --yes -p typescript@6 tsc …` — it uses the local devDep when present (dev/publish) and fetches it transiently only for a clean `--omit=dev` source build. `typescript` stays a devDependency (nothing extra ships to registry users, no build output committed). The `prod-install-build` CI job is hardened to remove the globally-installed `typescript` first (so the from-source build can no longer pass by leaking `tsc` off the runner's PATH — which is how the regression slipped through), and now also runs a faithful `pi install git:…` simulation: a clean clone + `npm install --omit=dev` that exercises the whole `prepare` (npx build + grammar download) from a pristine tree.
+- **Lua symbols + imports no longer silently break in multi-language repos** (#255) — the aggregator's `tree-sitter-lua.wasm` (`tree-sitter-wasms@0.1.13`) parsed lua correctly only as the **sole** grammar: the moment any second grammar loaded into web-tree-sitter's process-global WASM `Module`, every subsequent lua parse became an `ERROR` tree, so `SYMBOL_QUERIES.lua`/`IMPORT_QUERIES.lua` extracted nothing — lua symbol search, symbol-level impact, and `module_report` outlines were silently empty in essentially every real repo (this is why the #249 lua import query couldn't ship). Root cause is that specific stale wasm, not the runtime (bash/ruby/python/go/js are all fine after a 2nd grammar). Fixed with a new **per-grammar source override** (`GRAMMAR_SOURCE_OVERRIDES`): lua now downloads from the maintained **`@tree-sitter-grammars/tree-sitter-lua@0.4.1`** prebuilt wasm instead of the frozen aggregator, which parses cleanly in a multi-grammar process. The lua defs/refs/import queries are rewritten for that grammar's node types (`function_declaration`/`function_call`/`dot_index_expression`), and lua is now covered by the symbol + import smokes plus a dedicated shared-Module regression test.
+- **The bundled YAML grammar now actually loads** (#427) — the aggregator's `tree-sitter-yaml.wasm` (`tree-sitter-wasms@0.1.13`) is ABI-incompatible with the pinned `web-tree-sitter@0.25` and fails `Language.load` outright, so YAML parsing silently returned nothing despite the grammar shipping in the bundled CORE set (dead weight in the tarball, and the lone grammar the grammar-health sweep reported as "unavailable"). Uses the same new `GRAMMAR_SOURCE_OVERRIDES` mechanism to pull the maintained **`@tree-sitter-grammars/tree-sitter-yaml@0.7.1`** prebuilt wasm, which loads + parses cleanly. Covered by a load regression test.
+- **Swift files no longer crash `pi` on Node 24** (#423, #432) — the prebuilt `tree-sitter-swift.wasm` (from `tree-sitter-wasms@0.1.13`) triggers a **fatal, uncatchable V8 crash** (`Fatal process out of memory: Zone`, in the background Turboshaft-WASM optimizer) the first time a `.swift` file is analyzed on **Node 24, every OS** — taking down the whole agent. The crash is a process **abort**, so it can't be caught or degraded in-process, and rebuilding the grammar from source does **not** reliably dodge it (proven by the grammar-health nightly: the from-source wasm crashes on Node 24.18 identically to the prebuilt). pi-lens now **refuses to load the grammar at the point of use on the affected runtime** (`BLOCKED_GRAMMARS` / `grammarBlockReason`, gated on V8 + Node major ≥ 24): a `.swift` file simply gets no tree-sitter structural symbols (graceful degrade) instead of crashing the session. **bun (JavaScriptCore) and Node ≤ 22 are unaffected** and keep full Swift support via the normal CDN grammar download. Membership of the blocklist is **guard-driven**: the **`npm run check:grammar-load`** guard (loads each grammar in an isolated child process, skipping blocked ones — a hard gate for any *new* crasher) plus the **nightly cross-OS grammar-health workflow** watch (via a force-load probe) for when a future Node/V8 makes it safe to lift. Supersedes the earlier from-source **vendoring** approach (#426), which added a committed-wasm + provenance mechanism without actually dodging the crash — now removed.
+- **Tree-sitter no longer leaks WASM heap memory across a session** (#417) — web-tree-sitter `Tree` objects live in the WASM heap, which JS GC does **not** reclaim (0.25 has no auto-free); the tree cache dropped evicted/invalidated/overwritten trees with `Map.delete()` and never called `tree.delete()`, so every removed tree leaked. The cache bounded entry count (50) but not the heap, so it grew unbounded over a long editing session. `TreeCache` now frees the WASM tree on every removal path — eviction, same-file re-parse (same-key overwrite), on-disk change/deletion invalidations, `invalidate()`, and `clear()` — via a guarded `freeTree()` (best-effort; tolerates a dead/aborted runtime). The retained-for-incremental path (content changed, tree kept) is deliberately not freed. Safe because every consumer uses a parsed tree transiently (parse → extract → discard) and eviction only ever targets the oldest entry, never a just-parsed tree still in use.
+- **`lsp_diagnostics` now stops opening files in the language server once the turn is abandoned** (#343) — the batch and directory scans thread the tool-call + turn (`ctx.signal`) abort signal into their concurrency fan-out, so an Escape/abort mid-scan stops scheduling new files (each in-flight file stays bounded by `waitMs`) and returns partial results, instead of grinding the whole capped batch into the server after the agent has moved on.
+- **LSP nav requests retry once on `ContentModified` instead of returning empty** (#238 Item 2) — when a file changes under an in-flight `definition`/`references`/`hover`/etc. request the server rejects with `ContentModified` (-32801); `safeSendRequest` now does a single safe retry against the fresh state (correctness-under-edit is the hot path), returning empty only if it still can't answer. `RequestCancelled`/`ServerCancelled` are surfaced as "no result" (no retry) and `RequestFailed` (-32803) is treated as permanent — the JSON-RPC error code is now discriminated rather than blanket-rethrown.
+
+## [3.8.65] - 2026-07-04
+
+### Added
+
+### Changed
+
+### Fixed
+
+- **Full-scan progress bar now actually renders** — the progress bar added in 3.8.64 was computed and streamed to the tool's `onUpdate`, but never displayed: `lens_diagnostics`/`lsp_diagnostics` define a custom `renderResult` (`compactRenderResult`), and the pi host renders a partial update through that renderer, which drove its summary off structured `details` and ignored the progress text in `content`. The summarizers now detect a streaming progress partial (`details.phase === "scanning"`, via the new `scanningSummaryLine` helper) and show the bar (`Scanning… [████░░░░░░] 62/123 (50%)`) during the scan, falling back to the normal diagnostic summary on completion. Also ran `npm pkg fix` to drop the `./` prefix from the `bin` paths, silencing an npm publish auto-correct warning (the published bins were already correct).
+
 ## [3.8.64] - 2026-07-04
 
 ### Added

package/dist/clients/biome-client.js

@@ -11,6 +11,7 @@ import * as fs from "node:fs";
 import * as os from "node:os";
 import * as path from "node:path";
 import { isFileKind } from "./file-kinds.js";
+import { findGlobalBinary } from "./package-manager.js";
 import { safeSpawnAsync } from "./safe-spawn.js";
 // --- Client ---
 export class BiomeClient {
@@ -41,7 +42,7 @@ export class BiomeClient {
      * falls back to `process.cwd()`, which is wrong when pi is invoked from
      * a different directory than the file being edited.
      */
-    getBiomeBinary(cwd) {
+    async getBiomeBinary(cwd) {
         const resolveCwd = cwd ?? process.cwd();
         const cached = this.localBinaryByCwd.get(resolveCwd);
         if (cached)
@@ -74,11 +75,18 @@ export class BiomeClient {
                 return { cmd: p, args: [] };
             }
         }
+        // Any package manager's global bin dir (npm/pnpm/yarn/bun) before npx —
+        // catches `pnpm add -g @biomejs/biome` installs that PATH misses (#375).
+        const global = await findGlobalBinary("biome");
+        if (global) {
+            this.localBinaryByCwd.set(resolveCwd, global);
+            return { cmd: global, args: [] };
+        }
         // Fallback: npx (slower but works anywhere)
         return { cmd: "npx", args: ["@biomejs/biome"] };
     }
     async spawnBiomeAsync(args, timeout = 15000, cwd) {
-        const { cmd, args: prefix } = this.getBiomeBinary(cwd);
+        const { cmd, args: prefix } = await this.getBiomeBinary(cwd);
         return safeSpawnAsync(cmd, [...prefix, ...args], { timeout });
     }
     /**

package/dist/clients/bootstrap.js

@@ -63,13 +63,12 @@ export function loadBootstrapClients() {
                 return [];
             }
         }
-        const [ruffClient, biomeClient, knipClient, todoScanner, jscpdClient, typeCoverageClient, depChecker, testRunnerClient, metricsClient, complexityClient, goClient, govulncheckClient, gitleaksClient, trivyClient, rustClient, agentBehaviorClient, deadCodeClients,] = await Promise.all([
+        const [ruffClient, biomeClient, knipClient, todoScanner, jscpdClient, depChecker, testRunnerClient, metricsClient, complexityClient, goClient, govulncheckClient, gitleaksClient, trivyClient, rustClient, agentBehaviorClient, deadCodeClients,] = await Promise.all([
             load("ruff", async () => new (await import("./ruff-client.js")).RuffClient()),
             load("biome", async () => new (await import("./biome-client.js")).BiomeClient()),
             load("knip", async () => new (await import("./knip-client.js")).KnipClient()),
             load("todo", async () => new (await import("./todo-scanner.js")).TodoScanner()),
             load("jscpd", async () => new (await import("./jscpd-client.js")).JscpdClient()),
-            load("type-coverage", async () => new (await import("./type-coverage-client.js")).TypeCoverageClient()),
             load("dependency-checker", async () => new (await import("./dependency-checker.js")).DependencyChecker()),
             load("test-runner", async () => new (await import("./test-runner-client.js")).TestRunnerClient()),
             load("metrics", async () => new (await import("./metrics-client.js")).MetricsClient()),
@@ -90,7 +89,6 @@ export function loadBootstrapClients() {
             knipClient,
             todoScanner,
             jscpdClient,
-            typeCoverageClient,
             depChecker,
             testRunnerClient,
             metricsClient,