sverklo 0.18.1 → 0.19.0

package/README.md

@@ -2,16 +2,30 @@
   <img src="./docs/logo.svg" alt="sverklo" width="280" height="79"/>
 </p>
 
-> **The only code-intel MCP with a published benchmark and reproducible eval harness.**
-> Local-first MCP server that gives Claude Code, Cursor, Windsurf, and Zed a real symbol graph, a blast-radius lens, and a git-pinned memory — so the agent stops guessing. MIT. Zero config. Your code never leaves the machine.
+<p align="left">
+  🇬🇧 <b>English</b> · 🇨🇳 <a href="./README-zh-CN.md">中文</a>
+</p>
+
+> *"The map is not the territory."* — Alfred Korzybski
 >
-> [Paper (Zenodo, CC BY 4.0)](https://doi.org/10.5281/zenodo.19802051) · [bench:primitives](./benchmark/) — sverklo cuts agent context by **65 % vs grep** (255 vs 731 tokens per task, n=60) · [bench:swe](https://sverklo.com/blog/bench-swe-first-results/) — 38/65 perfect recall on 5 OSS repos, including the runs we lose.
+> Training data is the map. Your codebase is the territory. **Sverklo gives the agent the territory.**
+
+**The only code-intel MCP with a published benchmark and reproducible eval harness.** Local-first MCP server that gives Claude Code, Cursor, Windsurf, and Zed a real symbol graph, a blast-radius lens, and a git-pinned memory — so the agent stops guessing. MIT. Zero config. Your code never leaves the machine.
+
+[Paper (Zenodo, CC BY 4.0)](https://doi.org/10.5281/zenodo.19802051) · [bench:primitives](https://sverklo.com/bench/) — **62× fewer tokens than naive grep**, 2.9× fewer than tuned grep, single tool call vs grep's 7-12 (n=60) · [bench:swe](https://sverklo.com/blog/bench-swe-first-results/) — 38/65 perfect recall on 5 OSS repos, including the runs we lose.
+
+### One-click install
+
+[![Install in Claude Code](https://img.shields.io/badge/Claude_Code-Install_Plugin-CC785C?style=for-the-badge&logoColor=white)](#claude-code) [![Install in Cursor](https://img.shields.io/badge/Cursor-Install_MCP-F14C28?style=for-the-badge&logo=cursor&logoColor=white)](cursor://anysphere.cursor-deeplink/mcp/install?name=sverklo&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsInN2ZXJrbG8iXX0=) [![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_MCP-0098FF?style=for-the-badge&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=sverklo&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22sverklo%22%5D%7D) [![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install_MCP-24bfa5?style=for-the-badge&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=sverklo&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22sverklo%22%5D%7D&quality=insiders) [![Install in Windsurf](https://img.shields.io/badge/Windsurf-sverklo_init-09B6A2?style=for-the-badge&logoColor=white)](#windsurf--zed--vs-code--jetbrains)
 
 [![npm version](https://img.shields.io/npm/v/sverklo.svg?color=E85A2A)](https://www.npmjs.com/package/sverklo)
 [![npm downloads](https://img.shields.io/npm/dw/sverklo.svg?color=E85A2A)](https://www.npmjs.com/package/sverklo)
 [![License: MIT](https://img.shields.io/badge/license-MIT-E85A2A.svg)](LICENSE)
 [![Audited repos](https://img.shields.io/badge/audited_repos-47-E85A2A)](https://sverklo.com/report)
 [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.19802051.svg)](https://doi.org/10.5281/zenodo.19802051)
+[![GitHub stars](https://img.shields.io/github/stars/sverklo/sverklo?style=social)](https://github.com/sverklo/sverklo/stargazers)
+
+> ⭐ **If sverklo saved your AI from hallucinating, please star this repo** — it's the single most useful thing you can do to help others find it. Then [share with one teammate](https://twitter.com/intent/tweet?text=Local-first%20MCP%20code%20intelligence%20for%20AI%20coding%20agents%20%E2%80%94%20MIT%2C%20zero-deps%2C%20honest%20benchmark%20%40%20https%3A%2F%2Fsverklo.com%2Fbench%2F) who's tired of `getUserByEmail()` not existing in their codebase.
 
 ![Sverklo cuts agent context by 65 % vs grep — bench:primitives, 60 retrieval tasks, peer-reviewable](./docs/hero-token-comparison.png)
 
@@ -26,7 +40,7 @@ Sverklo drills into your repo before the agent does — symbol graph, blast radi
 <table>
 <tr>
 <td align="center"><b>37</b><br/>MCP tools your agent uses</td>
-<td align="center"><b>&lt; 2 s</b><br/>to index a 1,700-file monorepo</td>
+<td align="center"><b>&lt; 1 s</b><br/>incremental refresh after each edit</td>
 <td align="center"><b>0 bytes</b><br/>of your code leave the machine</td>
 </tr>
 </table>
@@ -44,6 +58,30 @@ That's it. `sverklo init` auto-detects your installed AI coding agent (Claude Co
 
 ---
 
+## "But isn't this just…?"
+
+Likely you've seen tools that look adjacent. The honest one-paragraph answers, with detailed comparisons linked.
+
+**…just grep with extra steps?** No, but tuned grep is genuinely competitive on F1. On the [60-task bench](https://sverklo.com/bench/) a tuned grep beats sverklo on F1 by 9 points. Sverklo wins by 62× on input tokens and 7-12× on tool-call count. For an AI agent inside a 200K context window, that's the load-bearing axis. For a human at a terminal, smart grep is fine.
+
+**…just Sourcegraph Cody?** Same retrieval surface (hybrid BM25 + vector + graph), different deployment model and license. Cody is source-available with enterprise per-developer pricing ($9-19/dev/mo); sverklo is MIT and runs on a laptop with no signup. [Full comparison →](https://sverklo.com/vs/sourcegraph-cody/)
+
+**…just Greptile?** Greptile is a hosted PR-review bot ($30/dev/mo). Sverklo is local-first MCP. Same risk-scoring goal, opposite deployment model. If your code can't leave the machine for compliance reasons, Greptile isn't an option. [Full comparison →](https://sverklo.com/vs/greptile/)
+
+**…just Cursor's @codebase?** Cursor's indexing is cloud-based and editor-bound. Sverklo runs alongside Cursor as an MCP server, adding the symbol graph, blast-radius, and bi-temporal memory that Cursor's @codebase doesn't expose. [Full comparison →](https://sverklo.com/vs/cursor-codebase/)
+
+**…just Claude Context (Zilliz)?** Claude Context requires a Milvus database. Sverklo runs entirely on embedded SQLite — no extra services to manage. [Full comparison →](https://sverklo.com/vs/claude-context/)
+
+**…just Aider's repo-map?** Aider's repo-map is a static signal in the system prompt — fine for small repos, doesn't scale past ~100 files. Sverklo is the queryable retrieval layer Aider can call via MCP for larger codebases. They're complementary, not competing. [Full comparison →](https://sverklo.com/vs/aider/)
+
+**…a niche memory MCP?** Most memory MCPs are wrappers around an external vector DB. Sverklo's memory is bi-temporal (`valid_from_sha`, `valid_until_sha`, `superseded_by`) and pinned to git SHAs, so you can ask "what did this team believe about auth at commit `abc123`?" and get the answer that was true *then*. [Full comparison vs codebase-memory-mcp →](https://sverklo.com/vs/codebase-memory-mcp/)
+
+**…what about Aider, Continue, Codex CLI, Claude Code?** Those are *agents* — they generate and apply edits. Sverklo is the *retrieval layer* the agent calls before writing code. Use both. [`sverklo init` auto-detects which agents you have →](#works-with-every-mcp-editor)
+
+If something is missing here that you'd ask about, [open an issue](https://github.com/sverklo/sverklo/issues) — I'll add it.
+
+---
+
 ## What's new in 0.18
 
 - **Vue.js (.vue) support.** Single-file components are now first-class: the `<script>` block parses through the existing TS/JS pipeline (with line remapping back to the SFC), Composition API helpers (`ref`, `computed`, `reactive`, `defineProps`, …) are indexed as symbols, and PascalCase template tags emit relative imports so PageRank sees component graphs. Also fixes a preexisting TS bug where `import type { X } from 'y'` was missed.
@@ -108,10 +146,16 @@ If the answer to your question is "exact string X exists somewhere," grep wins.
 | Tool | What |
 |------|------|
 | `sverklo_search` | Hybrid BM25 + ONNX vector + PageRank, fused with Reciprocal Rank Fusion |
+| `sverklo_search_iterative` | Wider candidate pool with refinement hints between rounds |
+| `sverklo_investigate` | Parallel multi-channel fan-out (FTS / vector / path / symbol) with per-channel RRF |
+| `sverklo_ask` | Natural-language router — concepts + investigate + refs in one call |
 | `sverklo_overview` | Structural codebase map ranked by PageRank importance |
 | `sverklo_lookup` | Find any function, class, or type by name (typo-tolerant) |
 | `sverklo_context` | One-call onboarding — combines overview, code, and saved memories |
 | `sverklo_ast_grep` | Structural pattern matching across the AST, not just text |
+| `sverklo_concepts` | Browse the LLM-derived concept index (themes across the codebase) |
+| `sverklo_clusters` | Semantic clusters of related symbols, computed offline |
+| `sverklo_patterns` | Query symbols tagged with a design pattern (observer, repository, validator, ...) |
 
 ### Impact — refactor without the regression
 | Tool | What |
@@ -119,14 +163,16 @@ If the answer to your question is "exact string X exists somewhere," grep wins.
 | `sverklo_impact` | Walk the symbol graph, return ranked transitive callers (the real blast radius) |
 | `sverklo_refs` | Find all references to a symbol, with caller context |
 | `sverklo_deps` | File dependency graph — both directions, importers and imports |
-| `sverklo_audit` | Surface god nodes, hub files, dead code candidates in one call |
+| `sverklo_audit` | **Lint your codebase for AI-readiness.** God nodes, hub files, dead code, circular deps, security smells, A-F health grade — all in one call |
 
 ### Review — diff-aware MR review with risk scoring
 | Tool | What |
 |------|------|
 | `sverklo_review_diff` | Risk-scored review of `git diff` — touched-symbol importance x coverage x churn |
+| `sverklo_critique` | Second-pass critique of a review — what did the first read miss |
 | `sverklo_test_map` | Which tests cover which changed symbols; flag untested production changes |
 | `sverklo_diff_search` | Semantic search restricted to the changed surface of a diff |
+| `sverklo_verify` | Verify a quoted code span is still present at the cited SHA — citation gate |
 
 ### Memory — bi-temporal, git-aware, never stale
 | Tool | What |
@@ -135,13 +181,24 @@ If the answer to your question is "exact string X exists somewhere," grep wins.
 | `sverklo_recall` | Semantic search over saved memories with staleness detection |
 | `sverklo_memories` | List all memories with health metrics (still valid / stale / orphaned) |
 | `sverklo_forget` | Delete a memory |
-| `sverklo_promote` / `sverklo_demote` | Move memories between tiers (project / global / archived) |
+| `sverklo_promote` / `sverklo_demote` | Move memories between tiers (core / archive) |
+| `sverklo_pin` / `sverklo_unpin` | Pin a memory to a file path or symbol so recall surfaces it without semantic search |
+
+### Post-filter primitives — refine the last response without re-querying
+| Tool | What |
+|------|------|
+| `sverklo_grep_results` | Grep inside the previous result block instead of re-running the search |
+| `sverklo_head_results` | Take the first N hits from the previous response |
+| `sverklo_ctx_peek` | Peek at a referenced span by its handle without expanding it fully |
+| `sverklo_ctx_slice` | Slice a stored response by line range |
+| `sverklo_ctx_grep` | Grep within a stored context window |
+| `sverklo_ctx_stats` | Token-budget stats for stored response handles |
 
 ### Index health
 | Tool | What |
 |------|------|
 | `sverklo_status` | Index health check, file counts, last update |
-| `sverklo_wakeup` | Warm the index after a long pause; incremental refresh |
+| `sverklo_wakeup` | 500-token codebase summary for system prompts on agents that can't run MCP |
 
 </details>
 
@@ -211,28 +268,57 @@ Every memory carries `valid_from_sha` and `valid_until_sha`. Updating a memory d
 
 ## How It Works
 
-```mermaid
-graph LR
-    A[Your Code] --> B[Parse<br/>11 languages]
-    B --> C[Embed<br/>ONNX/Ollama]
-    B --> D[Build Graph<br/>imports/exports]
-    D --> E[PageRank<br/>importance]
-    
-    F[Agent Query] --> G[BM25]
-    F --> H[Vector Search]
-    E --> I[PageRank Boost]
-    G --> J[RRF Fusion]
-    H --> J
-    I --> J
-    J --> K[Token-Budgeted<br/>Response]
+```
+Your codebase                                              Agent query
+     │                                                          │
+     ▼                                                          ▼
+┌─────────────┐                                          ┌─────────────┐
+│   Parse     │  tree-sitter (12 langs) or               │  Tool call  │
+│   chunks    │  regex fallback                          │  (1 of 37)  │
+└──────┬──────┘                                          └──────┬──────┘
+       │                                                         │
+       ├─────────────┐         Index time                        │
+       │             │                                           │
+       ▼             ▼                                           │
+  ┌────────┐    ┌─────────┐                                      │
+  │ Embed  │    │ Import  │                                      │
+  │ ONNX   │    │ graph   │                                      │
+  │ MiniLM │    │ + PageRank                                     │
+  └───┬────┘    └────┬────┘                                      │
+      │              │                                           │
+      ▼              ▼                                           │
+  ┌──────────────────────────┐                                   │
+  │ SQLite + sqlite-vec      │ ← single-file index, ~/.sverklo/  │
+  │ chunks · embeddings ·    │                                   │
+  │ symbols · refs · imports │                                   │
+  │ memories (bi-temporal)   │                                   │
+  └────────────┬─────────────┘                                   │
+               │                                                 │
+               │            Query time                           │
+               ▼                                                 ▼
+        ┌──────────────────────────────────────────────────────────┐
+        │             Channelized RRF retrieval                    │
+        │                                                          │
+        │   FTS · Vector · Doc-section · Path · Symbol-name        │
+        │      └─ each ranked independently ─┘                     │
+        │                                                          │
+        │   Fused with channel weights (path 1.5×, doc 0.7×, …)   │
+        └─────────────────────────┬────────────────────────────────┘
+                                  │
+                                  ▼
+                       ┌────────────────────────┐
+                       │ Token-budgeted answer  │ ← the agent gets
+                       │ file:line + chunk      │   ranked code, not
+                       │ + provenance           │   a wall of text
+                       └────────────────────────┘
 ```
 
-1. **Parses** your codebase into functions, classes, types (TS, JS, Vue, Python, Go, Rust, Java, C, C++, Ruby, PHP)
-2. **Embeds** code using all-MiniLM-L6-v2 ONNX model (384d, fully local) — or any Ollama model via config
-3. **Builds** a dependency graph and computes PageRank (structurally important files rank higher)
-4. **Searches** using hybrid BM25 + vector similarity + PageRank, fused via Reciprocal Rank Fusion
-5. **Remembers** decisions and patterns across sessions, linked to git state
-6. **Watches** for file changes and updates incrementally
+1. **Parse** your codebase into functions, classes, types (TS, JS, Vue, Python, Go, Rust, Java, C, C++, Ruby, PHP, C#)
+2. **Embed** code using all-MiniLM-L6-v2 ONNX (384d, fully local) — or any Ollama model via config
+3. **Graph** dependencies and compute PageRank (structurally important files rank higher)
+4. **Retrieve** via channelized RRF — per-channel rank fusion with channel-specific weights, the architectural choice that closes the private-helper-function recall gap
+5. **Remember** decisions across sessions, pinned to git SHAs (bi-temporal memory)
+6. **Watch** for file changes and re-index incrementally (~1 s per edit)
 
 ---
 
@@ -250,6 +336,18 @@ Real measurements on real codebases. Reproducible via `npm run bench` ([methodol
 - **Impact analysis is sub-millisecond** — indexed SQL join, not a string scan
 - **11 languages:** TS, JS, Vue, Python, Go, Rust, Java, C, C++, Ruby, PHP
 
+### Retrieval benchmark — bench:primitives
+
+Hybrid retrieval F1 vs grep baselines on a 60-task hand-verified evaluation across two OSS codebases. Public report at **[sverklo.com/bench/](https://sverklo.com/bench/)** — including the slice where sverklo *loses* (P5 dead-code detection, F1 = 0.02).
+
+| baseline | F1 | input tokens | tool calls |
+|---|---:|---:|---:|
+| naive-grep | 0.35 | 15,814 | 7.6 |
+| smart-grep (tuned) | 0.67 | 731 | 11.8 |
+| **sverklo** | 0.58 | **255** | **1.0** |
+
+A tuned grep beats sverklo on F1. Sverklo wins decisively on token economy (62× fewer than naive grep, 2.9× fewer than tuned grep) and tool-call count, which is the load-bearing axis for AI agents with bounded context windows. Reproduce with `npm run bench:primitives`.
+
 ---
 
 ## Quick Start
@@ -308,9 +406,19 @@ npx sverklo /path/to/your/project
 
 ---
 
-## Audit formats
+## Lint for AI-readiness — `sverklo audit`
+
+Most lints check syntax. `sverklo audit` lints whether your codebase is **legible to an AI agent**: high blast-radius "god nodes" the agent will trip on, hub files that cascade widely on every change, orphan symbols that might be dead code or might be public API, circular dependencies that confuse the symbol graph, and a security smell scan. Outputs an A-F health grade you can pin as a README badge.
 
-`sverklo audit` generates codebase health reports in six formats: `markdown`, `html`, `json`, `sarif`, `csv`, and `badges`. Run `sverklo audit --format html --open` for a self-contained report with god nodes, hub files, orphan detection, coupling analysis, and language distribution. Use `sverklo audit --badge` to add an A-F health grade shield to your README.
+```bash
+sverklo audit                     # markdown report in the terminal
+sverklo audit --format html --open  # self-contained HTML you can share
+sverklo audit --badge             # A-F shield markdown for your README
+sverklo audit --format sarif      # GitHub code-scanning alerts
+sverklo audit --format json       # machine-readable for CI gates
+```
+
+Six formats: `markdown`, `html`, `json`, `sarif`, `csv`, `badges`. Pair with `sverklo_impact` (the MCP tool) when you want to see the per-symbol blast radius before refactoring.
 
 ---
 
@@ -390,6 +498,18 @@ BibTeX:
 }
 ```
 
+## Star history
+
+<a href="https://www.star-history.com/#sverklo/sverklo&type=date&legend=top-left">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=sverklo/sverklo&type=date&theme=dark&legend=top-left" />
+    <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=sverklo/sverklo&type=date&legend=top-left" />
+    <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=sverklo/sverklo&type=date&legend=top-left" />
+  </picture>
+</a>
+
+If sverklo saved your AI from inventing function names that don't exist in your codebase, the most useful thing you can do is **⭐ star this repo** and share with one teammate.
+
 ## License
 
 MIT

package/dist/bin/sverklo.js

@@ -1883,6 +1883,50 @@ if (modeResolution.mode !== "embedded") {
 const positionalArgs = args.filter((a) => !a.startsWith("--mode=") && !a.startsWith("--global"));
 const hasExplicitPath = positionalArgs.length > 0 && positionalArgs[0] !== undefined;
 const isGlobalFlag = args.includes("--global");
+// Bug-bash 2 finding: `sverklo search "query"` silently fell through
+// to MCP-server-start with "search" treated as a project path. The
+// server then hung on stdin waiting for MCP protocol traffic. First-
+// time users would conclude sverklo was broken. Detect that case
+// up-front and emit a friendly error.
+const MCP_TOOL_VERBS = new Set([
+    "search", "lookup", "refs", "find-references", "impact", "deps", "dependencies",
+    "overview", "context", "ask", "recall", "remember", "forget", "promote", "demote",
+    "memories", "ast-grep", "ast_grep", "review-diff", "review_diff", "test-map", "test_map",
+    "diff-search", "diff_search", "status", "pin", "unpin", "clusters", "concepts",
+    "patterns", "critique", "verify", "investigate", "search-iterative", "search_iterative",
+    "grep-results", "grep_results", "head-results", "head_results",
+    "ctx-peek", "ctx_peek", "ctx-slice", "ctx_slice", "ctx-grep", "ctx_grep",
+    "ctx-stats", "ctx_stats",
+]);
+if (hasExplicitPath) {
+    const candidate = positionalArgs[0];
+    if (MCP_TOOL_VERBS.has(candidate.toLowerCase())) {
+        process.stderr.write(`Error: \`sverklo ${candidate}\` is an MCP tool, not a CLI command.\n\n`);
+        process.stderr.write(`MCP tools are called by AI agents (Claude Code, Cursor, Windsurf, Zed) —\n`);
+        process.stderr.write(`not from the command line. To use them:\n\n`);
+        process.stderr.write(`  1. cd to your project\n`);
+        process.stderr.write(`  2. sverklo init                  # set up MCP integration\n`);
+        process.stderr.write(`  3. claude                        # ask in natural language\n\n`);
+        process.stderr.write(`If you wanted to start the MCP server directly (e.g. for a custom\n`);
+        process.stderr.write(`client), run \`sverklo\` with no args from the project root.\n\n`);
+        process.stderr.write(`For CLI commands (init, audit, doctor, ui, …) run \`sverklo --help\`.\n`);
+        process.exit(2);
+    }
+    // Validate the path actually exists, otherwise the user typo'd a
+    // command verb and starting an MCP server on a non-existent dir
+    // would silently hang on stdin like the case above.
+    const { statSync: _stat } = await import("node:fs");
+    try {
+        if (!_stat(resolve(candidate)).isDirectory())
+            throw new Error("not a directory");
+    }
+    catch {
+        process.stderr.write(`Error: \`${candidate}\` is not a valid project path or known command.\n\n`);
+        process.stderr.write(`Run \`sverklo --help\` to see available commands, or \`sverklo init\`\n`);
+        process.stderr.write(`from a project directory to set up MCP integration.\n`);
+        process.exit(2);
+    }
+}
 // Auto-download model if missing (no separate setup step needed)
 const { existsSync } = await import("node:fs");
 const { join } = await import("node:path");