pi-lens 4.3.0 → 4.4.1

package/docs/features.md

@@ -2,7 +2,7 @@
 
 ### LSP Support
 
-pi-lens includes **46 language server definitions** (including four cross-cutting *auxiliary* scanners that attach alongside the file's language server — Opengrep, ast-grep, zizmor, and typos — see below). LSP is **enabled by default**; use `--no-lsp` to disable it for a session. Servers are auto-discovered from PATH, project `node_modules`, and managed installs. When a server is not installed, pi-lens offers an interactive install prompt.
+pi-lens includes **48 language server definitions** (including four cross-cutting *auxiliary* scanners that attach alongside the file's language server — Opengrep, ast-grep, zizmor, and typos — see below). LSP is **enabled by default**; use `--no-lsp` to disable it for a session. Servers are auto-discovered from PATH, project `node_modules`, and managed installs. When a server is not installed, pi-lens offers an interactive install prompt.
 
 **LSP Idle Management:** LSP servers shut down after 240 seconds of inactivity (no files modified) to free resources. The timer resets when you resume editing, preventing cold-start penalties during active development.
 
@@ -19,7 +19,7 @@ pi-lens includes **46 language server definitions** (including four cross-cuttin
 - **`capabilities`** — shows which operations are supported by the active LSP server(s) for a file, read directly from the cached `initialize` response (no round-trip).
 - **Symbol column resolution** — passing `symbol: "myFunc"` instead of an exact `character` position resolves the correct column automatically. Use `symbol: "foo#2"` for the second occurrence of `foo` on the line.
 
-LSP servers for: TypeScript, Deno, Python (pyright/basedpyright + jedi), Go, Rust, Ruby (ruby-lsp + solargraph), PHP, PowerShell, C# (omnisharp), F#, Java (JDT LS, with Lombok javaagent support when a Lombok jar is available), Kotlin, Swift, Dart, Lua, C/C++, Zig, Haskell, Elixir, Gleam, Typst (tinymist), Markdown (marksman), OCaml, Clojure, CUE (syntax and parse diagnostics; evaluation errors via the cue-vet auxiliary runner), Terraform, Nix, Bash, Fish, CMake, Docker, YAML, JSON, HTML, TOML, Prisma, Vue, Svelte, CSS.
+LSP servers for: TypeScript, Deno, Python (pyright/basedpyright + jedi), Go, Rust, Ruby (ruby-lsp + solargraph), PHP, PowerShell, C# (omnisharp), F#, Java (JDT LS, with Lombok javaagent support when a Lombok jar is available), Kotlin, Swift, Dart, Lua, C/C++, Zig, Haskell, Elixir, Gleam, Typst (tinymist), Markdown (marksman), OCaml, Clojure, CUE (syntax and parse diagnostics; evaluation errors via the cue-vet auxiliary runner), Terraform, Nix, Bash, Shuck (zsh-aware shell language server), Fish, CMake, Docker, YAML, JSON, HTML, TOML, Prisma, Vue, Svelte, CSS.
 
 ### Formatters
 
@@ -109,6 +109,10 @@ the full behavior and honest limits.
 
 pi-lens builds a review graph (`file → symbol → dependency`) during session and uses it at turn end to render an impact cascade: which files were affected by a change and how diagnostics propagated through the dependency graph. Nodes track kind, language, and export status; edges track contains/imports/calls/references.
 
+### Knip at turn end and linked worktrees
+
+knip follows git ignore rules (`.gitignore` and `.git/info/exclude`) and nothing else, so linked worktrees that live inside the project (for example `.worktrees/*`) are walked as project files: on a pi-lens clone with six of them, 83 percent of the project paths knip saw were worktree files, and a scan took 10.9 s cold and 3.1 s warm instead of 2.6 s and 0.6 s. pi-lens drops their issues from the verdict, runs knip in the worktree that owns an edit, and stops waiting for a root whose last two scans both outlasted the turn_end budget (the knip row then reads `reason: "scan-exceeds-budget"` and the ledger counts `turn-end-knip-nested-worktrees`). It does not edit your repository: to remove the cost, add `/.worktrees/` (or your worktree directory) to the project's `.gitignore`.
+
 ### Read-Before-Edit Guard
 
 pi-lens enforces a **read-before-edit** policy on all file writes and edits. Before allowing a `write` or `edit` tool call on an existing file, it verifies that the agent has previously read sufficient context:
@@ -118,7 +122,7 @@ pi-lens enforces a **read-before-edit** policy on all file writes and edits. Bef
 - **Out-of-range block** — blocks if the edit target lines fall outside the ranges previously read, ensuring the agent cannot modify code it hasn't seen
 - **Snapshot validation** — covered edit ranges are hash-checked against the lines the agent actually saw at read time; stale-range edits are rejected even when range coverage exists. Hash capture covers reads up to 3 000 lines
 
-Coverage is tracked across multiple reads: two reads of lines 1–100 and 101–200 together satisfy a full-file write. Symbol-expanded reads (small reads silently widened to the enclosing symbol via tree-sitter) count toward coverage at the symbol level. Markdown files generate a warning instead of blocking (edits outside the section-expanded read range are warned, not silently passed). Plain-text (`.txt`) and log (`.log`) files remain fully exempt.
+Coverage is tracked across multiple reads: two reads of lines 1–100 and 101–200 together satisfy a full-file write. Symbol-expanded reads (small reads widened to the enclosing symbol via tree-sitter, labelled with a leading note) count toward coverage at the symbol level. Markdown files generate a warning instead of blocking (edits outside the section-expanded read range are warned, not silently passed). Plain-text (`.txt`) and log (`.log`) files remain fully exempt.
 
 Override for a single edit: `/lens-allow-edit <path>`
 
@@ -179,7 +183,7 @@ pi-lens MCP server expose the same shape to Claude Code / any MCP client.
 
 At `turn_end`, pi-lens writes `<project-data-dir>/cache/actionable-warnings.json` summarizing fixable warnings introduced by the current turn. This powers the optional conservative autofix at `agent_end`.
 
-`<project-data-dir>` is whatever `getProjectDataDir(cwd)` resolves to: `<project>/.pi-lens` only when that legacy directory already exists, otherwise `~/.pi-lens/projects/<project-slug>` (or a `PILENS_DATA_DIR` location). The turn-end advisory points at `lens_diagnostics mode=delta` first and names the resolved file second, so you never have to work the layout out by hand (#2521).
+`<project-data-dir>` is whatever `getProjectDataDir(cwd)` resolves to: `<project>/.pi-lens` only when that legacy directory already exists when the process first resolves the root (a `.pi-lens` created mid-process takes effect at the next process start), otherwise `~/.pi-lens/projects/<project-slug>` (or a `PILENS_DATA_DIR` location). The turn-end advisory points at `lens_diagnostics mode=delta` first and names the resolved file second, so you never have to work the layout out by hand (#2521).
 
 **Report contents:**
 
@@ -230,6 +234,42 @@ One event per logical write batch (not per file) — e.g. a single eslint `--fix
 
 **Kill switch:** `PI_LENS_BUS_PUBLISH=0` disables publishing entirely (see `docs/environment-variables.md`). Publishing is fire-and-forget — a disabled/unavailable/throwing bus never affects the write path's own success or latency.
 
+### Bus Events — `pilens:format:*` (#673)
+
+Deferred formatting has a separate lifecycle on the same `pi.events` bus:
+
+```
+pilens:format:queued  { v: 1, source: "pi-lens", filePath, cwd, tool, kinds,
+                        ownerSessionId?, turnIndex?, batchId? }
+pilens:format:start   { v: 1, source: "pi-lens", cwd, paths, fileCount, kinds,
+                        ownerSessionId?, turnIndex?, batchId? }
+pilens:format:done    { v: 1, source: "pi-lens", cwd, paths, fileCount, kinds,
+                        ownerSessionId?, turnIndex?, batchId?, settled? }
+```
+
+Paths and `cwd` are absolute, normalized strings. `queued` is emitted when a
+file first enters the deferred queue; `start` is emitted when a non-empty
+claimed format batch begins; and `done` is emitted after all formatters in that
+batch and any late-write resyncs have settled. `paths: []` on `done` means the
+batch changed no bytes. `batchId` is process-unique and monotonic across
+coordinator generations, so
+two same-millisecond reloads cannot share a lifecycle identity. A normal
+`pilens:format:done` has `settled: true` (or omits the additive field for old
+producers). If the formatter remains unsettled after the bounded hook/format
+drain, pi-lens emits one terminal `pilens:format:done` with `settled: false`
+and the still-pending paths, and records one `deferred-format-unsettled`
+degradation for that batch. A later formatter write is resynced by the late
+continuation and does not emit a second done event.
+The session and turn fields let a listener match the lifecycle to its own
+session and turn; consumers must not infer ownership from paths or timing.
+
+These are visibility events, not a flush API. They are fire-and-forget and can
+be absent when the bus is disabled, unwired, stale, or throws. Replaced-session
+and aborted drains publish only for work they actually claim and finish; work
+requeued for its owner is not
+announced as done by the replacing session. The events are additive v1 schemas
+and may gain optional fields without a version bump.
+
 **Fix provenance (#502):** `FilesTouchedPayload` gained an additive, optional `fixes` field so a diff/review consumer can distinguish a pi-lens-mechanical hunk from an agent edit:
 
 ```
@@ -294,7 +334,7 @@ Late-joiners are a non-problem in-process — extensions activate at `session_st
 
 ### Opportunistic Read Expansion
 
-When the agent reads a small slice of a file (≤ 60 lines), pi-lens transparently expands the read to the full enclosing symbol (function, method, or class) using the tree-sitter AST. The agent receives the full symbol as context, and the read guard records symbol-level coverage so edits anywhere within that symbol pass without requiring the agent to have read every line individually. Expansion runs within a 200 ms budget and falls back silently on unsupported file types or parse failures.
+When the agent reads a small slice of a file (≤ 100 lines), pi-lens expands the read to the full enclosing symbol (function, method, or class) using the tree-sitter AST, or to the enclosing heading section of a Markdown file. The result starts with a `[pi-lens: read widened ...]` note naming the requested and the returned lines; a `limit` above 100 reads the exact range. Expansion is off with the read guard (`--no-read-guard` or `readGuard.enabled=false`). The agent receives the full symbol as context, and the read guard records symbol-level coverage so edits anywhere within that symbol pass without requiring the agent to have read every line individually. Expansion runs within a 200 ms budget and falls back silently on unsupported file types or parse failures.
 
 Supported: TypeScript, TSX, JavaScript, JSX, Python, Go, Rust, Ruby, Java, Kotlin, Dart, Elixir, C, C++, C#, PHP, Swift, Lua, OCaml, Zig, Bash.
 

package/docs/globalconfig.md

@@ -33,6 +33,7 @@ Each runtime toggle is settable from the CLI *and* from `config.json`. The two a
 | `--lens-compact-tool-line` | `ui.compactToolLine` | `false` |
 | `--lens-compact-lsp-status` | `ui.compactLspStatus` | `false` |
 | `--lens-hide-lsp-status` | `ui.hideLspStatus` | `false` |
+| `--lens-compact-widget` | `ui.compactWidget` | `false` |
 | `--no-lazy-tools` | `tools.lazy` | `true` |
 | `--lens-turn-end-madge` | `turnEnd.madge.enabled` | `false` |
 | `--no-knip` | `knip.enabled` | `true` |
@@ -262,7 +263,7 @@ not corrupt delta baselines.
 
 ### `maxProjectFiles`
 
-Single scale knob (default `2000`) that a large-but-healthy repo can raise to scale five independent size budgets together instead of tripping each one separately: the project-diagnostics scanner (0.25×, default 500 files), the review graph (0.5× up to a point, default 1,000 files — see below), the startup scan (1×, default 2,000 source files), jscpd (3×, default 6,000 directory entries), and the word index (3×, default 6,000 files). Raising it (e.g. to `5000`) scales all five proportionally. Each subsystem's own environment-variable override (e.g. `PI_LENS_REVIEW_GRAPH_MAX_FILES`, `PI_LENS_STARTUP_SCAN_MAX_ENTRIES`) still takes precedence over this knob when set, and a separate `PI_LENS_MAX_PROJECT_FILES` environment variable sits below `maxProjectFiles` but above the built-in default. See `clients/project-scale.ts` for the full ratio table and precedence order.
+Single scale knob (default `2000`) that a large-but-healthy repo can raise to scale five independent size budgets together instead of tripping each one separately: the project-diagnostics scanner (0.25×, default 500 files), the review graph (0.5× up to a point, default 1,000 files — see below), the startup scan (1.25×, default 2,500 source files), jscpd (3×, default 6,000 directory entries), and the word index (3×, default 6,000 files). Raising it (e.g. to `5000`) scales all five proportionally. Each subsystem's own environment-variable override (e.g. `PI_LENS_REVIEW_GRAPH_MAX_FILES`, `PI_LENS_STARTUP_SCAN_MAX_ENTRIES`) still takes precedence over this knob when set, and a separate `PI_LENS_MAX_PROJECT_FILES` environment variable sits below `maxProjectFiles` but above the built-in default. See `clients/project-scale.ts` for the full ratio table and precedence order.
 
 Unlike the other four, the review graph's budget stops scaling flatly at 0.5× once `maxProjectFiles` climbs past 4,000 — it tapers toward a 5,000-file ceiling instead of growing unbounded (#775). Its higher per-file cold-build cost bounds this default file budget independently of the graph's disk-persist circuit-breaker, whose measured default is now 500,000 elements (#936; override with `PI_LENS_GRAPH_PERSIST_MAX_ELEMENTS`). An over-cap graph persists a centrality-ranked partial snapshot with explicit total-versus-persisted coverage; read-only orientation may use it, but incremental builds never reuse it as a complete base. Below the 4,000 threshold (which covers every typical/default-sized project) it's the same flat 0.5× ratio as before. See `reviewGraph.maxFiles` below for an explicit opt-in past the taper.
 

package/docs/io-bridge-v2-migration.md

@@ -0,0 +1,94 @@
+# Migration Guide: File I/O Lifecycle Bridge v1 → v2
+
+**Tracking RFC:** [RFC 3654](rfcs/3654-unified-io-bridge-v2.md)  
+**Tracking Issue:** [apmantza/pi-lens#3654](https://github.com/apmantza/pi-lens/issues/3654)  
+**Applicability:** Producers interacting with pi-lens via `Symbol.for("pi-lens:read-bridge")` or `Symbol.for("pi-lens:mutation-bridge")`.
+
+---
+
+## 1. Who is Affected
+
+This guide applies to authors and maintainers of pi coding-agent extensions, tools, and integrations that report file read or mutation events to `pi-lens` out-of-band:
+- Multi-file editing extensions and hash-anchor tools.
+- Custom LSP or code-generation harnesses operating outside native pi tool hooks.
+- Internal callers transitioning from dual legacy bridge invocations to the unified v2 bridge.
+
+---
+
+## 2. What Remains Unchanged (Frozen v1 Compatibility)
+
+Under **Decision D14** (Backward-Compatible Refinement):
+- **Symbol Availability**: `Symbol.for("pi-lens:read-bridge")` and `Symbol.for("pi-lens:mutation-bridge")` continue to be mounted alongside v2 in the same first-wins pass.
+- **Return Conventions**:
+  - `readBridge.recordRead(...)` returns `void`.
+  - `mutationBridge.recordMutation(...)` returns `boolean` (`true` if recorded, `false` if rejected).
+- **Semantics & Keying**:
+  - Path arguments are accepted as strings without clobbering caller display paths.
+  - Map keys in `ReadGuard` and `CacheManager` are derived identically.
+  - No v1-valid call is re-keyed by upgrading `pi-lens`, and every v1 call keeps its return value.
+- **Two recorded differences** (the v1 read shim now runs v2's `disk`-evidence read):
+  - A read of a file that is absent when the call lands records nothing; v1 recorded coverage for it. The drop is an `io-bridge-read-dropped` ledger row.
+  - A zero-line read (`requestedLimit: 0`) of a non-empty or unreadable file is still dropped, but its ledger row is `io-bridge-read-dropped` (subject `<consumer>:bookkeeping-error`) instead of `read-bridge-zero-line-dropped`.
+
+---
+
+## 3. Call-to-Call Mapping
+
+| Operation | v1 Legacy Call | v2 Unified Call |
+|---|---|---|
+| **Read File Window** | `readBridge.recordRead({ filePath, requestedOffset: 10, requestedLimit: 20, consumer: "my-tool" })` | `ioBridge.record({ filePath, consumer: "my-tool", read: { ranges: [[10, 29]], evidence: "disk" } })` |
+| **Read Entire File** | `readBridge.recordRead({ filePath, requestedOffset: 1, consumer: "my-tool" })` | `ioBridge.record({ filePath, consumer: "my-tool", read: { ranges: [[1, lineCount]], content: inMemoryText } })` |
+| **Partial Edit** | `mutationBridge.recordMutation({ filePath, editRanges: [[12, 15]], kind: "edit", importsChanged: false })` | `ioBridge.record({ filePath, mutate: { kind: "edit", ranges: [[12, 15]], importsChanged: false } })` |
+| **Whole-File Write** | `mutationBridge.recordMutation({ filePath, kind: "write" })` | `ioBridge.record({ filePath, mutate: { kind: "write" } })` |
+| **Atomic Edit + Read** | *Requires 2 separate calls to two different global symbols* | `ioBridge.record({ filePath, mutate: { kind: "edit", ranges: [[10, 15]] }, read: { ranges: [[5, 25]], content: previewText } })` |
+| **File Deletion** | *Not supported in v1* | `ioBridge.record({ filePath, mutate: { kind: "delete" } })` |
+
+---
+
+## 4. Refinements in v2
+
+| Feature | Category | Description |
+|---|---|---|
+| **Atomic Mutate-Before-Read** | `COMPATIBLE` | Compound `{ mutate, read }` calls guarantee `mutate` runs before `read`, so the preview read stays outstanding: a write marks the file's earlier reads consumed (evictable on idle), and the read that follows it is not. |
+| **In-Memory Hash Evidence** | `OPT-IN` | Passing `content` (or `lineHashes`) hashes in memory, eliminating two synchronous `fs.readFileSync` calls per read (#3651). |
+| **Coverage-Only Reads** | `OPT-IN` | Default `evidence: "caller"` records line range coverage in `ReadGuard` without reading or hashing disk. |
+| **Explicit Zero-Line Reads** | `COMPATIBLE` | `ranges: []` records an empty/0-byte read without tripping staleness checks (#3652). |
+| **Lifecycle Deletion** | `OPT-IN` | `mutate: { kind: "delete" }` verifies the file is absent from disk, evicts `readGuard` records, and notifies LSP DidClose / FileChangeType 3 without corrupting session state. |
+| **Deferred Format Event Bus Parity** | `COMPATIBLE` | When `deferAutofix !== false`, queuing deferred formatting reliably emits `pilens:format:queued` on `pi.events`, warning in-process listeners before `agent_settled` rewrites. |
+
+---
+
+## 5. Recommended Migration Steps
+
+1. **Feature-Detect v2**:
+   ```typescript
+   import { getIOBridge, IO_BRIDGE_SYMBOL } from "pi-lens/clients/io-bridge"; // or via Symbol.for("pi-lens:io-bridge")
+
+   const ioBridge = (globalThis as any)[Symbol.for("pi-lens:io-bridge")];
+   if (ioBridge && ioBridge.version === 2) {
+     ioBridge.record({
+       filePath,
+       mutate: { kind: "edit", ranges: [[10, 20]] },
+       read: { ranges: [[1, 50]], content: fileBuffer },
+     });
+   } else {
+     // Graceful fallback to legacy v1 symbols
+     const mutBridge = (globalThis as any)[Symbol.for("pi-lens:mutation-bridge")];
+     mutBridge?.recordMutation({ filePath, editRanges: [[10, 20]], kind: "edit" });
+     const readBridge = (globalThis as any)[Symbol.for("pi-lens:read-bridge")];
+     readBridge?.recordRead({ filePath, requestedOffset: 1, requestedLimit: 50 });
+   }
+   ```
+
+2. **Leverage In-Memory Content**:
+   When reading or previewing diffs, supply `content: previewBuffer` in the `read` facet. This avoids redundant disk reads and makes preview verification resilient to external file timestamp drift.
+
+3. **Handle Deletions Explicitly**:
+   When removing a file, delete the file from the filesystem first, then record `mutate: { kind: "delete" }` to prevent zombie records in `ReadGuard`.
+
+---
+
+## 6. Drift Policy & Stability Guarantees
+
+- **Shim Immutability**: The v1 shims mounted at `Symbol.for("pi-lens:read-bridge")` and `Symbol.for("pi-lens:mutation-bridge")` are strictly frozen compatibility layers. They will never adopt breaking changes or alter return types.
+- **Process Singleton Guarantee**: The bridge is registered via `registerProcessBridge` (`clients/process-bridge.ts`), enforcing first-wins registration and `Object.freeze`. A running process cannot have its bridge mutated or replaced midway through a session.

package/docs/lsp-capability-matrix.md

@@ -77,7 +77,7 @@ refresh cannot merge until the marker is updated with it.
 |---|---|---|---|
 | **1 — pull** | `textDocument/diagnostic` returns an authoritative report (empty = clean) | YES, deterministic | rust-analyzer |
 | **2 — push, publishes-versioned** | `publishDiagnostics([])` **with version** on every scan, incl. clean→clean | YES, currency-proven via version | ast-grep |
-| **2\* — push, publishes-unversioned** | re-publishes on a clean scan but **version-less** — the wait still early-returns (the client accepts a version-less publish as fresh: it can't be proven stale), but currency is only *temporally correlated*, not proven | YES at runtime, with a staleness-risk caveat (not a latency cost) | opengrep |
+| **2\* — push, publishes-unversioned** | re-publishes on a clean scan but **version-less** — the wait still early-returns (the client accepts a version-less publish as fresh: it can't be proven stale), but currency is only *temporally correlated*, not proven. Since #3484 the client fences a server measured to answer a `documentSymbol` fence before it publishes (`diagnosticsFence: "reply-first"`: yaml, intelephense): a pre-edit publish is dropped until the fence's reply. Every other 2\* server (docker publishes before answering; prisma, taplo, zls, dart, gleam, clojure unmarked) keeps this behaviour | YES at runtime, with a staleness-risk caveat (not a latency cost) | opengrep |
 | **3 — push, silent on clean** | server publishes nothing when nothing changed | **NO** — budget-wait floor (safe; a timeout is *not* a false clean). **This tier is #458's learned-deadline target set.** | typescript-language-server |
 | **Navigation-only — custom, no evidence** | custom `lsp.servers.*` entry has no pull provider and has not published in this session | **NO** — diagnostics are unsupported; skip the wait and report navigation-only | Dexter |
 
@@ -107,6 +107,28 @@ nightly steps, **dev** = the dev box (a row measured on both reads `dev+ci`).
 Merges never blank a prior good value, so a CI non-result leaves the dev
 classification standing.
 
+Two bounded guards keep that preservation from hiding a dead instrument
+(#3401). A `direct` `first-publish` cell whose axis the nightly probe does not
+re-observe is stamped once with the date of its first miss and degrades to
+`unknown` once five calendar days have elapsed since it (skipped nights do not
+stall it, and a re-observation clears the stamp). `empty-first` cells are never
+expired: they back the live `emptyFirstPublish` markers (php, terraform) and
+expiring one would erase the measurement behind a marker. A `clean-behavior`/
+`tier` change is written only after two consecutive nightly runs observe the
+same new value, so one flapping nightly (ast-grep went 2 → 2* → 3 → 2* across
+four runs) cannot rewrite a cell; a night that measured nothing for the lang
+resets the hold. A subset probe (`probe-clean-signal.mjs <langs>`) leaves the
+bookkeeping of the langs it did not probe untouched.
+
+The bookkeeping lives in the generated `## Capability matrix refresh state`
+section at the end of this doc, the only state the refresh persists. The clock
+is the nightly run, not the bot PR's merge: each nightly starts from the last
+`bot/lsp-docs-refresh` doc when that branch is ahead of master and was built on
+master's current doc (`scripts/seed-matrix-from-bot-branch.mjs`), and from
+master's doc otherwise (branch absent, squash-merged, or master edited the doc
+since). Closing the bot PR unmerged does not reset the bookkeeping: the branch
+is kept and keeps seeding, so only deleting `bot/lsp-docs-refresh` resets it.
+
 `vue`'s `clean-behavior` was hand-reset to `unknown` (#3390): its
 `publishes-unversioned` cell came from 58/45 publishes that the shared
 `extension.log` window had attributed to vue but that belonged to `tinymist`.
@@ -121,7 +143,7 @@ carries the named admission until then.
 | json | vscode-json-language-server | pull | — | n/a (pull) | 1 | dev+ci |
 | css | vscode-css-language-server | pull | — | n/a (pull) | 1 | dev+ci |
 | html | vscode-html-language-server | pull | — | n/a (pull) | 1 | dev+ci |
-| rust | rust-analyzer | pull | — | n/a (pull) | 1 | dev |
+| rust | rust-analyzer | pull | — | n/a (pull) | 1 | dev+ci |
 | svelte | svelte-language-server | pull | — | n/a (pull) | 1 | dev+ci |
 | deno | deno (alt of typescript) | pull | — | n/a (pull) | 1 | dev+ci |
 | ruby | ruby-lsp | pull | — | n/a (pull) | 1 | ci |
@@ -134,12 +156,13 @@ carries the named admission until then.
 | yaml | yaml-language-server | push-only | publishes-unversioned | direct | 2* | dev+ci |
 | shell | bash-language-server | push-only | publishes-versioned | direct | 2 | dev+ci |
 | dockerfile | docker-langserver | push-only | publishes-unversioned | direct | 2* | dev+ci |
+| dockerfile-official | docker-language-server (official) | push-only | unknown | empty-only | ? | dev+ci |
 | toml | taplo | push-only | publishes-unversioned | direct | 2* | dev+ci |
-| terraform | terraform-ls | push-only | TBD | TBD | 2/3? | dev+ci |
+| terraform | terraform-ls | push-only | publishes-unversioned | empty-first | 2* | dev+ci |
 | prisma | @prisma/language-server | push-only | publishes-unversioned | direct | 2* | dev+ci |
 | php | intelephense | push-only | publishes-unversioned | empty-first | 2* | dev+ci |
 | zig | zls | push-only | publishes-unversioned | direct | 2* | dev+ci |
-| vue | @vue/language-server | push-only | unknown | direct | 2/3? | dev+ci |
+| vue | @vue/language-server | push-only | unknown | unknown | 2/3? | dev+ci |
 | dart | dart language-server | push-only | publishes-unversioned | direct | 2* | ci |
 | gleam | gleam lsp | push-only | publishes-unversioned | direct | 2* | ci |
 | clojure | clojure-lsp | push-only | publishes-unversioned | direct | 2* | ci |
@@ -230,3 +253,13 @@ marker; a mismatch means the marker may need a human update (#529). `unknown`
 observations are never compared (a slow/absent server is not evidence either way).
 
 _None observed as of the last probe run._
+
+## Capability matrix refresh state (nightly-generated)
+
+Bookkeeping for the date-based `direct` `first-publish` expiry (#3401), the
+two-run `clean-behavior` hysteresis and the consecutive-night `idle-eviction`
+counts (#3989). Regenerated every run; never a measurement.
+
+```json
+{"clean-behavior":{"dockerfile-official":{"pendingBehavior":"publishes-versioned","pendingTier":"2","runs":1}},"idle-eviction":{"docker":{"nights":[{"day":"2026-10-07","rssMb":64,"coldMs":566}]},"json":{"nights":[{"day":"2026-10-07","rssMb":67,"coldMs":1116}]},"powershell":{"nights":[{"day":"2026-10-07","rssMb":155,"coldMs":2601}]},"python-jedi":{"nights":[{"day":"2026-10-07","rssMb":51,"coldMs":1796}]},"zizmor":{"nights":[{"day":"2026-10-07","rssMb":58,"coldMs":608}]}}}
+```

package/docs/lsp-idle-eviction.md

@@ -0,0 +1,124 @@
+# LSP idle-eviction measurement
+
+Per-server cost and safety of releasing an idle language-server client, measured
+by the nightly so each server's `idleEviction` declaration in
+`clients/lsp/server.ts` can be made from evidence (#3645). Generated by
+`node scripts/measure-lsp-idle-eviction.mjs [--install]` (requires
+`npm run build:dist`); it changes no server's policy.
+
+For each registry server the probe spawns the server on its smoke fixture,
+records the first diagnostics, then makes the server evictable in the probe
+process only and lets the real idle-eviction timer release it. The next request
+respawns it through the ordinary path. A server is `eligible` only when the
+respawned server reports every finding it reported before eviction; the
+`coverage` column is that comparison, and for an auxiliary scanner it is the
+content-bound coverage check the auxiliary freeze depends on. A server that
+cannot demonstrate it is never proposed for eviction on perf grounds alone.
+
+_Last generated: 2026-10-07 on linux; 48 registry servers: 32 eligible, 0 vetoed, 2 inconclusive, 14 unavailable (0 not reached: budget)._
+
+## Per-server rows
+
+Only stable columns are committed, so the nightly refresh changes this file
+only when a server's outcome changes. The measured init duration, resident
+memory of the server's process tree and post-eviction cold start are in the
+nightly run's step log and job summary, one row per server. `n/a` means not
+measured, never zero: **respawn** is whether the next request after eviction
+brought the server back, **coverage** is whether the respawned server kept
+every baseline finding.
+
+A server that reports extra findings after the respawn (`widened`) still reads
+`preserved`, because nothing was lost, but a first scan that grows on respawn
+is a sign the baseline was partial. Any policy flip needs `preserved` on
+several nights, not one; a `widened` disclosure is per run and is shown in the
+step log only, because it flaps.
+
+| server | role | declared | result | reason | respawn | coverage |
+|---|---|---|---|---|---|---|
+| ast-grep | auxiliary | unmeasured | eligible | · | ok | preserved |
+| bash | primary | transparent | eligible | · | ok | preserved |
+| clojure | primary | transparent | eligible | · | ok | preserved |
+| cmake | primary | unmeasured | inconclusive | no-baseline | n/a | unproven |
+| cpp | primary | transparent | eligible | · | ok | preserved |
+| csharp | primary | unmeasured | unavailable | tool-unavailable | n/a | n/a |
+| css | primary | transparent | eligible | · | ok | preserved |
+| cue | primary | unmeasured | eligible | · | ok | preserved |
+| dart | primary | unmeasured | unavailable | tool-unavailable | n/a | n/a |
+| deno | primary | transparent | eligible | · | ok | preserved |
+| docker | primary | unmeasured | eligible | · | ok | preserved |
+| docker-official | primary | unmeasured | eligible | · | ok | preserved |
+| elixir | primary | unmeasured | unavailable | tool-unavailable | n/a | n/a |
+| expert | primary | unmeasured | eligible | · | ok | preserved |
+| fish | primary | transparent | eligible | · | ok | preserved |
+| fsharp | primary | unmeasured | unavailable | tool-unavailable | n/a | n/a |
+| gleam | primary | unmeasured | eligible | · | ok | preserved |
+| go | primary | unmeasured | unavailable | tool-unavailable | n/a | n/a |
+| haskell | primary | unmeasured | unavailable | tool-unavailable | n/a | n/a |
+| html | primary | transparent | eligible | · | ok | preserved |
+| java | primary | unmeasured | unavailable | tool-unavailable | n/a | n/a |
+| json | primary | unmeasured | eligible | · | ok | preserved |
+| kotlin | primary | unmeasured | eligible | · | ok | preserved |
+| lua | primary | unmeasured | eligible | · | ok | preserved |
+| marksman | primary | transparent | eligible | · | ok | preserved |
+| nix | primary | unmeasured | unavailable | tool-unavailable | n/a | n/a |
+| ocaml | primary | unmeasured | unavailable | tool-unavailable | n/a | n/a |
+| omnisharp | primary | unmeasured | unavailable | no-fixture | n/a | n/a |
+| opengrep | auxiliary | transparent | eligible | · | ok | preserved |
+| php | primary | transparent | eligible | · | ok | preserved |
+| powershell | primary | unmeasured | eligible | · | ok | preserved |
+| prisma | primary | transparent | eligible | · | ok | preserved |
+| python | primary | transparent | eligible | · | ok | preserved |
+| python-jedi | primary | unmeasured | eligible | · | ok | preserved |
+| ruby | primary | unmeasured | unavailable | tool-unavailable | n/a | n/a |
+| rust | primary | unmeasured | eligible | · | ok | preserved |
+| shuck | primary | unmeasured | unavailable | server-not-started | n/a | n/a |
+| svelte | primary | unmeasured | eligible | · | ok | preserved |
+| swift | primary | unmeasured | unavailable | tool-unavailable | n/a | n/a |
+| terraform | primary | unmeasured | eligible | · | ok | preserved |
+| tinymist | primary | unmeasured | eligible | · | ok | preserved |
+| toml | primary | unmeasured | eligible | · | ok | preserved |
+| typescript | primary | transparent | eligible | · | ok | preserved |
+| typos | auxiliary | unmeasured | eligible | · | ok | preserved |
+| vue | primary | unmeasured | inconclusive | no-baseline | n/a | unproven |
+| yaml | primary | transparent | eligible | · | ok | preserved |
+| zig | primary | unmeasured | eligible | · | ok | preserved |
+| zizmor | auxiliary | unmeasured | eligible | · | ok | preserved |
+
+## Declared versus measured
+
+`drift` means the registry evicts a server the measurement vetoes. A `proposal`
+or `info` line is for a maintainer to act on in a follow-up; this run changed
+nothing.
+
+- **ast-grep** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **cue** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **docker** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **expert** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **gleam** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **json** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **kotlin** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **lua** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **powershell** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **python-jedi** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **rust** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **svelte** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **terraform** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **tinymist** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **toml** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **typos** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **zig** [proposal] declared unmeasured but eviction and respawn preserved its findings
+- **zizmor** [proposal] declared unmeasured but eviction and respawn preserved its findings
+
+## Reasons
+
+- `no-fixture`: No smoke fixture routes to this server, so there is nothing to spawn.
+- `tool-unavailable`: A tool this server's fixture needs could not be installed or found on the generating host.
+- `setup-failed`: The fixture's workspace setup step failed.
+- `server-not-started`: The fixture ran but this server never became a live client.
+- `budget-exhausted`: The run's wall-clock budget ended before this server was reached.
+- `probe-error`: The probe itself threw before it had a baseline.
+- `no-baseline`: The server produced no finding on the fixture, so nothing can show its coverage survives eviction.
+- `not-evicted`: The idle-eviction timer did not release the client within the bound.
+- `client-died`: The client went away without an idle-eviction record, so the release was a crash, not the eviction path.
+- `respawn-failed`: After eviction the next request could not bring the server back.
+- `findings-narrowed`: After the respawn the server reported fewer findings than before eviction.

package/docs/mcp.md

@@ -112,7 +112,7 @@ New files:
 
 | tool | maps to | purpose |
 |------|---------|---------|
-| `pilens_analyze` | `analyzeFile` (warm) / worker (fresh) | run the per-edit pipeline on a file; returns diagnostics + timing. `mode: warm\|fresh`. **Correctness + mechanism probe.** |
+| `pilens_analyze` | `analyzeFile` (warm) / worker (fresh) | run the per-edit pipeline on a file; returns diagnostics + timing. `mode: warm\|fresh`. **Correctness + mechanism probe.** Each `latency.runners[]` row carries `failureKind` when the runner set one: `failed` + `blocking_diagnostics` is a run whose findings failed the check; `failed` with any other kind, or none, is a runner that produced no usable result. |
 | `pilens_ast_grep_search` | `ast_grep_search` | AST-aware structural search. |
 | `pilens_ast_grep_replace` | `ast_grep_replace` | AST-aware structural replace. |
 | `pilens_diagnostics` | `lens_diagnostics` | session or LSP diagnostics, selected by `source`, `scope`, and `severity`. |
@@ -131,6 +131,17 @@ New files:
 | `pilens_symbol_search` | `symbol_search` | search the persisted symbol index. |
 | `pilens_turn_end` | turn lifecycle | settle turn-end work and return findings. |
 
+Every tool's `inputSchema` (from `tools/list`) is also the set of argument keys
+the dispatcher accepts without comment: an undeclared key is reported in the
+result (a leading `Ignored unknown argument(s) ...` line, plus
+`structuredContent.ignoredArguments`). An ignored key that leaves a required
+input missing, or that is a declared parameter the call did not send written
+another way (`pilens_diagnostics {"filePath": ...}` for `path`; predicate in
+[public-api-stability.md](public-api-stability.md)), turns the call into an error
+instead of a run on defaults; a looser "did you mean" stays a warning (#3749;
+details in [agent-tools.md](agent-tools.md)). The check is the one seam in
+`mcp/server.ts` `tools/call`, through `mcp/tool-arguments.ts`.
+
 ## Packaging / wiring
 
 - `package.json`: add `"bin": { "pi-lens-mcp": "./dist/mcp/server.js" }`; add
@@ -383,9 +394,14 @@ be exercised + debugged directly through Claude Code without running pi.
   hook (reads the tool payload from stdin → `tool_input.path`/`file_path` + cwd)
   AND as a plain CLI (`--file=`). Defaults to `no-lsp` (FAST: ~1-2s, the cold
   LSP would cost ~5s/edit and under-report anyway — pull `pilens_analyze` on the
-  warm server for type errors). Silent on clean files; advisory (always exit 0).
-  `--hook` emits a PostToolUse `additionalContext` envelope; plain mode prints a
-  report. 4 bin tests (CLI, --hook envelope, clean-file silence, stdin payload).
+  warm server for type errors). Silent on clean files. Clean scans and findings
+  exit 0; an invocation that cannot run reports its failure on stdout and stderr.
+  Plain CLI failures exit 2. `--hook`, stdin hook payloads, and `--turn-end` stay
+  advisory and exit 0, including on failure. `--hook` emits a PostToolUse
+  `additionalContext` envelope; plain mode and Stop hooks print text.
+  `pilens_health` reports failed invocation counts and the last operation,
+  timestamp, and bounded, redacted reason from the workspace status record.
+  These counters are best-effort history, not a current readiness verdict.
 
   Wire it in Claude Code `settings.json`:
 
@@ -443,7 +459,7 @@ be exercised + debugged directly through Claude Code without running pi.
   in the server, and concurrent turn-ends cannot race the turn-state clear.
   **Warm-only, no cold fallback**: only the server process owns the session state
   and pending turn work, so a local pass would report a false clean — no warm
-  server means one stderr line, silent stdout, exit 0.
+  server means a skip reason on stdout and stderr, exit 0.
   The client waits 55s so it gives up inside Claude Code's 60s hook timeout.
   `SubagentStop` is deliberately NOT registered: subagent edits already fire
   PostToolUse into the shared turn-state, the consume bridges are one-shot (a

package/docs/merge-queue-rollout.md

@@ -0,0 +1,77 @@
+# Merge queue rollout
+
+Adopting GitHub's merge queue for `master` (#3754). The repository side is in
+the same change as this page; the settings below are the maintainer's step and
+are deliberately not applied by any script or workflow.
+
+## Why
+
+Every master move forced `gh pr update-branch` and a full CI re-run on every
+open PR. A queue tests each PR merged onto the latest master, in order, so
+neither is needed. Sharded Unit tests (#3753) keep one queue run short.
+
+## Repository side (already in this change)
+
+- `ci.yml` and `lint.yml` run on `merge_group` (`types: [checks_requested]`).
+  Together they produce every required check: `Lint & type-check`,
+  `Unit tests`, `Install test (ubuntu-latest)`, `Install test (windows-latest)`,
+  `Install test (macos-latest)`, `knip`, `oxfmt format check`, `TLA+ models`.
+- PR-only jobs (PR title, PR body, changelog fast-fail, targeted advisory) are
+  guarded by `github.event_name == 'pull_request'` and are skipped on
+  `merge_group`; none of them is required.
+- `tests/config/merge-queue-workflows.test.ts` pins the trigger, the required
+  job names, and that no required job (or a job it `needs:`) is gated off on
+  `merge_group`.
+- `scripts/ci-verdict.mjs` reports a queued PR as kind `in-queue` (exit 3) and
+  a failed queue run as a FAIL event.
+
+## Maintainer settings (Settings, Rules, Rulesets, New branch ruleset)
+
+| Setting | Value | Why |
+| --- | --- | --- |
+| Ruleset name | `master merge queue` | |
+| Enforcement status | Active | |
+| Target branches | Include default branch (`master`) | |
+| Require status checks to pass | the eight names above, source GitHub Actions | unchanged from branch protection today |
+| Require branches to be up to date | off | the queue replaces it |
+| Require merge queue | on | |
+| Merge method | Merge commit | repo convention (`gh pr merge --merge`) |
+| Build concurrency (maximum pull requests to build) | 3 | queue builds run three required-check sets at once on hosted runners |
+| Minimum pull requests to merge | 1 | never hold a green PR waiting for company |
+| Maximum pull requests to merge | 3 | a failing group costs at most three re-tests |
+| Wait time to meet minimum group size | 1 minute | |
+| Only merge non-failing pull requests | All commits must pass (`ALLGREEN`) | a red group member ejects only itself |
+| Status check timeout | 30 minutes | the required set took about 10 minutes at the median and about 12 at the maximum over 30 runs before sharding; 30 minutes is 2.5x that |
+
+Branch protection on `master` today is classic (no ruleset exists).
+`scripts/ci-verdict.mjs` reads the live required-check names from classic
+branch protection; if you move the required checks into the ruleset it falls
+back to the static list (`Unit tests`, `Lint & type-check`) and still gates on
+every non-advisory check-run from a PR/push run it discovers. Either keep the classic rule
+(enable "Require merge queue" there) or accept the narrower absent-check
+detection.
+
+## Rollout steps
+
+1. Merge #3753 (sharded Unit tests) first, then this change.
+2. Create the ruleset above.
+3. Enqueue one low-risk PR: `gh pr merge <N> --merge`.
+4. Confirm in `gh run list --event merge_group` that `CI` and `Lint` ran on the
+   `gh-readonly-queue/master/pr-<N>-<sha>` ref and that all seven required
+   checks reported there. A required check that never reports stalls the queue
+   until the 30-minute timeout.
+5. Stop using `gh pr update-branch` and "re-arm" pushes; both are moot and a
+   push to a queued PR ejects it.
+
+## Rollback
+
+Disable the ruleset's "Require merge queue" rule. PRs merge directly again;
+nothing in the repository needs reverting, because `merge_group` triggers and
+the ci-verdict queue reads are inert without a queue.
+
+## Known remainder
+
+The `train:approved` lane that merged through the REST merge API was retired
+(#3837), so no competing lane holds a merge the queue would refuse. Every
+merge goes through `gh pr merge` (auto-merge or direct), which the queue
+intercepts when the base branch requires one.