carto-md 2.0.8 → 2.1.0

package/docs/guides/pre-merge-review.md

@@ -0,0 +1,139 @@
+# Guide: Pre-merge review
+
+> Architecture review at PR time, automatically. The reviewer sees structural impact alongside the diff.
+
+## What the GitHub Action does
+
+On every pull request:
+
+1. Checks out the PR head
+2. Restores `.carto/` from `actions/cache` (or runs `carto init` cold)
+3. Runs `carto pr-impact --base $BASE --head $HEAD --format markdown`
+4. Posts the result as a sticky comment on the PR (one comment per PR, updated in place on every push)
+
+The whole flow runs in 30–120 seconds on a warm cache, 1–3 minutes cold.
+
+## Setup
+
+In `.github/workflows/carto.yml`:
+
+```yaml
+name: Carto Impact Report
+on:
+  pull_request:
+    branches: [main]
+permissions:
+  contents: read
+  pull-requests: write
+jobs:
+  carto:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with: { fetch-depth: 0 }       # full history — pr-impact needs the base ref
+      - uses: theanshsonkar/carto@v2.0.9
+```
+
+That's the entire config. The action handles npm install, cache restore, sync, pr-impact, and the comment.
+
+## What the comment looks like
+
+```markdown
+## 🗺️ Carto Impact Report
+
+This PR touches **AUTH** and **DATABASE** domains.
+
+| Metric | Value |
+|--------|-------|
+| Risk | 🔴 HIGH |
+| Blast radius (union) | 47 files |
+| Files changed | 6 |
+| Cross-domain violations introduced | 2 |
+| Files without tests in blast radius | 5 of 31 |
+| High-impact file changed | src/auth/session.ts (8 direct dependents) |
+
+<details>
+<summary>Affected routes (4)</summary>
+
+- POST /auth/login — risk: HIGH
+- GET /auth/me — risk: HIGH
+- POST /auth/register — risk: MEDIUM
+- POST /api/users — risk: LOW
+
+</details>
+
+<details>
+<summary>Cross-domain violations (2)</summary>
+
+- auth/login.ts now imports from payments/billing.ts (AUTH→PAYMENTS)
+- database/user-repo.ts now imports from auth/jwt.ts (DATABASE→AUTH)
+
+</details>
+
+<details>
+<summary>Files without tests in blast radius (5)</summary>
+
+- src/auth/middleware.ts
+- src/auth/jwt-helpers.ts
+- …
+</details>
+```
+
+## What a reviewer does with this
+
+Look at the four signals in order:
+
+1. **Risk badge.** HIGH / MEDIUM / LOW / SAFE. If it's SAFE or LOW, you can mostly skip the structural review and focus on the code itself. If it's HIGH, look more carefully.
+
+2. **Cross-domain violations.** Each one is a new dependency between domains. Sometimes correct (a refactor moving shared logic), sometimes a smell (auth pulling from payments). Reviewer applies judgment.
+
+3. **Files without tests.** Files in the blast radius that don't have a sibling test file. Lower confidence in the change because there's no automated check. Suggest the author add tests, or document why not.
+
+4. **High-impact file changed.** Anything modifying a >20-dependent file deserves an extra read-through. Subtle changes to a widely-used utility have outsized blast.
+
+## Failing the build on risk
+
+For repos that want a harder gate, set `fail-on` in the workflow:
+
+```yaml
+- uses: theanshsonkar/carto@v2.0.9
+  with:
+    fail-on: HIGH
+```
+
+Now the build fails when the risk is HIGH. PRs need to lower the risk (split, add tests, refactor) before they can merge.
+
+Use sparingly. HIGH-risk changes are sometimes correct and *necessary* — the right reviewer call is "look more carefully", not "block automatically".
+
+## Inputs reference
+
+| Input         | Default     | What it does |
+|---------------|-------------|--------------|
+| `carto-version` | `latest`  | Pin in production for reproducibility. |
+| `base`        | auto        | Git ref the PR branched from. Auto-detected from `origin/$GITHUB_BASE_REF`. |
+| `head`        | auto        | Git ref of the PR head. Auto-detected from `$GITHUB_SHA`. |
+| `fail-on`     | _(empty)_   | Fail when risk ≥ this severity. `HIGH`, `MEDIUM`, or `LOW`. |
+| `comment-mode`| `sticky`    | `sticky` updates existing comment, `new` posts a new one every push, `none` skips the comment (renders to stdout). |
+| `node-version`| `20`        | Node version on the runner. |
+
+## Outputs
+
+| Output       | Description |
+|--------------|-------------|
+| `risk`       | `SAFE` / `LOW` / `MEDIUM` / `HIGH` — gate downstream steps on it. |
+| `comment-url`| URL of the posted comment. |
+
+## Running it locally (mirror the CI)
+
+```bash
+carto pr-impact --base origin/main --head HEAD
+carto pr-impact --base origin/main --head HEAD --format json
+carto pr-impact --base origin/main --head HEAD --fail-on HIGH
+```
+
+Same engine, same output. Useful for testing changes before pushing.
+
+## Related
+
+- [`docs/guides/ci-integration.md`](./ci-integration.md) — broader CI patterns
+- [`docs/concepts/blast-radius.md`](../concepts/blast-radius.md) — the math behind the metric

package/docs/migration/v1-to-v2.md

@@ -0,0 +1,110 @@
+# Migration: V1 → V2
+
+> Notes for anyone who installed `carto-md` pre-2.0.6.
+
+## What V2 changed
+
+V1 stored the index as JSON files at `.carto/cache/graph-cache.json` and `.carto/cache/hashes.json`. V2 stores it as SQLite at `.carto/carto.db` with a derived bitmap sidecar at `.carto/bitmap.bin`.
+
+Why: SQLite handles 100K-file repos cleanly where the JSON approach started getting slow at ~10K files. The bitmap layer makes graph queries 100–10,000× faster than SQL at scale (see [`docs/scale.md`](../scale.md)).
+
+V1 has been fully deleted from the tree as of carto-md@2.0.6.
+
+## What you need to do
+
+In most cases — nothing. `carto init` (or any `carto` command that touches the index) detects the old V1 cache files and migrates them transparently on first run.
+
+Specifically, the migration:
+
+1. Reads `.carto/cache/graph-cache.json` if present (V1 metadata)
+2. Notices it's incompatible with the V2 schema
+3. Runs a fresh `carto sync` to repopulate `.carto/carto.db` from source
+4. Leaves the V1 files in place (they're harmless dead bytes)
+
+The Spec 7 test suite explicitly covers this case: *"carto init migrates leftover V1 graph-cache.json cleanly (no errors)"*.
+
+## Manual cleanup (optional)
+
+After the first V2 sync succeeds, the V1 files are no longer read. Reclaim a bit of disk:
+
+```bash
+rm -f .carto/cache/graph-cache.json
+rm -f .carto/cache/hashes.json
+rmdir .carto/cache  # if empty
+```
+
+Or just `carto remove && carto init` for a clean slate.
+
+## MCP wiring changes
+
+V1 exposed 12 MCP tools. V2 ships ~75, organized into 8 categories (core graph, episodic memory, temporal, brain, predictive, AI-native retrieval, adjacent, org-wide).
+
+The highest-impact additions from V1:
+
+- `get_architecture()` — the "use this first" tool
+- `get_file_summary(file)` — surfaces what `carto why` shows
+- `get_change_plan(intent)` — real graph traversal (V1 was keyword grep — Spec 2 rewrite)
+- `get_similar_patterns(file)` — Jaccard-similarity over import sets
+- `validate_diff(diff)` — sub-15ms pre-write governance, writes every call into the episodic log
+- `did_we_discuss_this(topic)` — six-week recall over the decision log
+- `get_predictive_risk(file?)` — P(file causes the next incident) score
+- `get_minimal_context_for_intent(intent, budget)` — token-budgeted hybrid retrieval (structural + lexical + semantic with RRF fusion)
+- `simulate_change_impact(files)` — bitmap-backed multi-file blast radius
+
+Plus the temporal trio (`get_architectural_drift`, `get_domain_evolution`, `get_hotspot_files`), the brain stack (`get_invariants`, `get_conventions`, `get_action_patterns`, `scaffold_for_intent`), and the org/cross-repo set (`get_org_architecture`, `get_service_dependency_graph`, `find_consumers_of_api`).
+
+Full reference: [`docs/api/`](../api/).
+
+If your AI tool was configured against V1 and you're seeing tool-not-found errors, restart the tool to pick up the V2 server's new `tools/list` response. The wire protocol is the same.
+
+## ACP agent
+
+V1's ACP agent (in Zed / JetBrains / VS Code via `carto agent`) used its own in-memory index. V2's ACP agent shares the SQLite index with the MCP server (Spec 5). Re-running `carto init` in your project ensures both ACP and MCP see the same data.
+
+## `.cartoignore` defaults
+
+V2 expanded the default exclusion list from 12 patterns to 39 (Spec 8: SSH keys, AWS/GCP creds, `.netrc`, kubeconfig, etc.). If your project had a custom `.cartoignore`, it's preserved — Carto only writes a default when none exists. To pick up the new defaults, delete your `.cartoignore` and run `carto init` to regenerate.
+
+## Git hooks
+
+V1 didn't install git hooks. V2 (Spec 9) installs four:
+
+- `pre-commit`
+- `post-checkout`
+- `post-merge`
+- `post-rewrite`
+
+Each runs `carto sync >/dev/null 2>&1 || true` — never blocks git, never errors loudly. If you already have hooks for other purposes, Carto appends to them non-destructively. Run `carto doctor` to confirm the hooks installed.
+
+## Behavior changes worth flagging
+
+| Behavior | V1 | V2 |
+|----------|----|----|
+| File-discovery cap | hard cap at ~50K files | uncapped |
+| Index format | JSON | SQLite + bitmap |
+| Re-parse on edit | only via `carto watch` | lazy at MCP query time + git hooks |
+| `get_change_plan` | keyword grep | real graph traversal |
+| Cross-domain detection | path-based | graph-clustered |
+| ACP / MCP share index | no | yes |
+| Read-only MCP DB | no | yes |
+| Default `.cartoignore` | 12 patterns | 39 patterns |
+
+If you depended on a V1-specific behavior that's no longer accurate, file an issue — we may have broken something we shouldn't have.
+
+## Pinned-version recommendation
+
+If you're upgrading in CI, pin a known-good Carto version:
+
+```yaml
+# .github/workflows/carto.yml
+- uses: theanshsonkar/carto@v2.0.9
+  with:
+    carto-version: '2.0.9'    # pin instead of `latest`
+```
+
+This way the action's output (and the PR-comment shape) stays stable across CI runs even when newer Carto versions ship.
+
+## Related
+
+- [`docs/troubleshooting.md`](../troubleshooting.md) — what to do when migration goes sideways
+- [Spec 6](../../Progress/working.md) in `Progress/working.md` — the deletion that removed V1

package/docs/quickstart.md

@@ -0,0 +1,95 @@
+# Quickstart — 10 minutes from install to first MCP query
+
+Get carto wired into your AI tool and answering questions about your codebase in under ten minutes.
+
+## 1. Install (30 seconds)
+
+```bash
+npm install -g carto-md
+```
+
+Carto ships prebuilt native binaries for macOS arm64/x64, Linux x64 (glibc + musl), and Windows x64. If you're on a platform without prebuilds you'll fall back to compiling from source — see [troubleshooting.md](./troubleshooting.md#native-build-fails) if that breaks.
+
+Verify the install:
+
+```bash
+carto --version
+```
+
+## 2. Index your project (1 minute)
+
+```bash
+cd ~/your-project
+carto init
+```
+
+You'll see something like:
+
+```
+[CARTO] Detecting project...
+[CARTO] Detected: nextjs (typescript)
+[CARTO] Found 847 JS/TS files (847 total)
+[CARTO] Discovered 847 source files
+[CARTO] Computing reverse dependencies...
+[CARTO] Indexed 847 files in 1.8s
+[CARTO] MCP auto-wired into: Cursor, Claude Code, Kiro
+[CARTO] AGENTS.md generated.
+
+┌─ Carto · indexed ─────────────────────────────────────────
+│  847 files · 5 domains · 23 routes · 1,204 import edges
+│
+│  Top domains:
+│    CORE          (677 files)
+│    AUTH          (42 files)
+│    PAYMENTS      (38 files)
+│    DATABASE      (67 files)
+│
+│  💡 Highest-risk file: src/lib/db.ts
+│     (34 files depend on it — try `carto why src/lib/db.ts`)
+└───────────────────────────────────────────────────────────
+```
+
+What just happened:
+
+- Carto walked the project, parsed every source file with tree-sitter
+- Built an import graph, clustered files into domains (Leiden+CPM), extracted routes + models
+- Wrote `.carto/` containing the SQLite index, the bitmap sidecar, the per-domain context files
+- Wrote `AGENTS.md` at the project root
+- Installed 4 git hooks so the index re-syncs on commit/checkout/merge/rebase
+- Detected your installed AI tools and wrote MCP config files for each
+
+## 3. Restart your AI tool
+
+The MCP wiring lands when the AI tool starts fresh. Kill it and reopen, then verify carto loaded — most tools log MCP server connections.
+
+## 4. Ask your first question
+
+Open any prompt your AI tool exposes and try one of:
+
+> *"What's the blast radius of `src/lib/db.ts`?"*
+
+> *"Show me the architecture overview."*
+
+> *"What files would I need to change to add rate limiting to /api/users?"*
+
+The AI calls Carto's MCP tools (`get_blast_radius`, `get_architecture`, `get_change_plan`) and answers with structural facts instead of grepping.
+
+## 5. Try the CLI too
+
+While you're here:
+
+```bash
+carto status                              # one-screen health view
+carto why src/lib/db.ts                   # 3-line summary of a file
+carto explain "add rate limiting"         # natural-language → plan
+carto check                               # cross-domain violations
+carto doctor                              # diagnose any setup issues
+```
+
+## What's next
+
+- [`docs/concepts/blast-radius.md`](./concepts/blast-radius.md) — what "blast radius" means and how it's computed
+- [`docs/concepts/domains.md`](./concepts/domains.md) — how the Leiden clusterer infers AUTH/PAYMENTS/etc. without config
+- [`docs/concepts/mcp-integration.md`](./concepts/mcp-integration.md) — every MCP tool with an example call
+- [`docs/guides/adding-feature-safely.md`](./guides/adding-feature-safely.md) — the workflow this whole thing enables
+- [`docs/guides/ci-integration.md`](./guides/ci-integration.md) — drop the GitHub Action onto every PR

package/docs/scale.md

@@ -0,0 +1,129 @@
+# Carto at scale
+
+> Empirical scale data for the bitmap engine — synthetic stress sweep + every real-world repo in the Carto corpus. Reproducer commands at the bottom.
+
+The bitmap-backed graph engine is designed to handle 100K-1M files cleanly. This page is the empirical receipt. It captures both the **dense-synth stress sweep** (deterministic worst-case fan-out distribution) and the **real-world corpus** (4 of the 7 larger repos, the largest open-source codebases under MIT/permissive licenses we routinely test against).
+
+## What's measured
+
+For each run we capture:
+
+- **Init time** — full `runSync` from a wiped `.carto/` directory.
+- **Sync time** — second `runSync` immediately after, no files changed (mtime+hash skip path).
+- **DB size** — `.carto/carto.db` bytes on disk.
+- **bitmap.bin size** — the sidecar bytes on disk.
+- **Sidecar (RAM)** — sum of all bitmap word-array byte lengths held in memory.
+- **Peak RSS** — process resident memory after init and during query workloads.
+- **MCP query p50 / p99** — 1000 calls each (50-call warmup) for the 5 production bitmap tools (`blastRadius`, `crossDomain`, `highImpactFiles`, `similarPatterns`) plus `simulate_change_impact`. Random query inputs are seeded so identical `--seed` produces identical workloads.
+
+The query stage opens SQLite **read-only** — invariant honored. The bitmap-as-derived-disposable invariant is also honored: the runner only reads from `bitmap.bin`, never writes.
+
+All numbers below were captured on the maintainer dev box: Apple silicon, Node 20.20, macOS, NVMe SSD. Recapture via the commands at the bottom.
+
+## Synthetic stress sweep (dense fan-out)
+
+Source: `bench/scale-test/`. Generates a deterministic-for-seed TypeScript repo with a worst-case-shaped import graph: 5 domain-like top-level dirs (`auth`/`payments`/`database`/`events`/`core`), Pareto(α=2) fan-out clipped to [1,8], 75% same-domain / 25% cross-domain, hotspot-biased targets so the first ~10% of file ids accumulate most in-edges. Imports always reference an earlier file id (acyclic).
+
+Edge density ≈ 1.5 per file. **This is denser than every real-world repo in the corpus** — real codebases have many leaf files with zero imports, so the synth deliberately exercises the worst case for the bitmap layer.
+
+| Files | Edges | Init | Sync | DB | bitmap.bin | Sidecar (RAM) | Peak RSS |
+|------:|------:|-----:|-----:|---:|-----------:|--------------:|---------:|
+| 1,000 | 1,392 | 533ms | 70ms | 756 KB | 222 KB | 186 KB | 149 MB |
+| 10,000 | 14,876 | 3.28s | 627ms | 6.5 MB | 17.0 MB | 18.1 MB | 350 MB |
+| 50,000 | 76,145 | 1.1m | 5.14s | 36.9 MB | 414.9 MB | 488.4 MB | 1.35 GB |
+
+| Files | blastRadius p50/p99 | crossDomain p50/p99 | highImpactFiles p50/p99 | similarPatterns p50/p99 | simulate_change_impact p50/p99 |
+|------:|---------------------|---------------------|------------------------|------------------------|--------------------------------|
+| 1,000 | 1.6µs / 24.4µs | 19.5µs / 25.0µs | 709ns / 4.3µs | 276µs / 484µs | 7.0µs / 78.8µs |
+| 10,000 | 6.6µs / 67.7µs | 705µs / 914µs | 750ns / 1.4µs | 17.0ms / 19.3ms | 11.6µs / 113µs |
+| 50,000 | 22.4µs / 472µs | 28.1ms / 28.9ms | 750ns / 1.3µs | 462ms / 798ms | 49.9µs / 455µs |
+
+### What 50K reveals
+
+Three distinct failure modes show up at the dense-synth 50K row, **all of them addressable by a Roaring-bitmap upgrade on the roadmap**:
+
+1. **Storage scales linearly with N** — every file gets its own forward + reverse bitmap of size ~`N/8` bytes. At N=50K bitmap.bin is 415 MB; at N=100K linear extrapolation puts it at ~1.7 GB. The current pure-Node `Uint32Array` bitset (`src/bitmap/bitset.js`) trades storage efficiency for raw query speed and zero native deps. Roaring would cut this 5-50× depending on density.
+2. **`crossDomain` p50 hits 28 ms** — linear sweep over the `crossForward` Map. Fixable with a precomputed cross-domain edge bitmap that's OR-aggregated once at build time, not iterated per query.
+3. **`similarPatterns` p50 hits 462 ms** — Jaccard over import sets at scale costs O(candidates × set-intersection). The MCP layer calls this with `top_k=5` so user-facing latency is well under the raw bench number, but the absolute cost is real.
+
+Three things stay healthy at 50K: `blastRadius` p50 22µs, `highImpactFiles` p50 750ns (popcount index is O(1) array slice), `simulate_change_impact` p50 50µs.
+
+### Pending (not yet captured on this box)
+
+| Files | Notes |
+|------:|-------|
+| 100,000 | Stress-test for the dense-bitset implementation. ~3-5 min walltime expected. Worth running before any Roaring work to lock in the regression baseline. |
+| 500,000 | Practical ceiling for the dense-bitset; expect bitmap.bin > 4 GB, possibly hitting OOM on 16 GB-RAM dev boxes. |
+| 1,000,000 | Roaring-bitmap upgrade target. Expect dense-bitset OOM. |
+
+Reproducible via `npm run bench:scale -- --size 100000 --keep --out /tmp/carto-100k`.
+
+## Real-world corpus
+
+These are the 4 largest repos in `~/carto-test-repos`, all already indexed by `~/carto-test-repos/run-bench.sh` (the bench harness baseline). The query latency below was measured against those existing indexes via `--queries-only` so this measurement does not perturb the indexes used by `node test/accuracy-corpus.js`.
+
+| Repo | Indexed files | Edges | DB | bitmap.bin | Sidecar (RAM) |
+|------|--------------:|------:|---:|-----------:|--------------:|
+| [cal.com](https://github.com/calcom/cal.com) | 4,351 | 3,478 | 3.1 MB | 1.9 MB | 2.0 MB |
+| [nextjs](https://github.com/vercel/next.js) | 6,193 | 7,930 | 15.0 MB | 4.4 MB | 4.3 MB |
+| [supabase](https://github.com/supabase/supabase) | 6,330 | 5,189 | 4.8 MB | 4.4 MB | 4.4 MB |
+| [vscode](https://github.com/microsoft/vscode) | 7,567 | 13,335 | 14.3 MB | 4.1 MB | 4.5 MB |
+
+| Repo | blastRadius p50/p99 | crossDomain p50/p99 | highImpactFiles p50/p99 | similarPatterns p50/p99 | simulate_change_impact p50/p99 |
+|------|---------------------|---------------------|------------------------|------------------------|--------------------------------|
+| cal.com | 4.3µs / 51µs | 148µs / 279µs | 667ns / 1.0µs | 1.0µs / 1.99ms | 10.3µs / 71.0µs |
+| nextjs | 5.0µs / 155µs | 196µs / 284µs | 750ns / 1.3µs | 1.2µs / 3.01ms | 28.9µs / 211µs |
+| supabase | 5.5µs / 47.9µs | 201µs / 260µs | 708ns / 1.0µs | 1.5µs / 3.19ms | 10.8µs / 65.9µs |
+| vscode | 2.7µs / 428µs | 1.23ms / 1.47ms | 750ns / 1.7µs | 834ns / 4.03ms | 19.3µs / 637µs |
+
+### Real-world vs dense-synth
+
+The 7,567-file vscode row vs the 10,000-file synth row tells the most important story:
+
+- vscode `similarPatterns` p50 is **834ns**; synth-10K is **17ms**. Real codebases have far sparser per-file import sets — most files import 0-3 things, a handful import many. Synth is a deliberate worst case (Pareto fan-out always ≥1, often 4-8). Real Jaccard-over-import-sets fits in cache; synth doesn't.
+- vscode `crossDomain` p50 is **1.23ms**; synth-10K is **704µs**. Cross-domain edges scale with edge density; synth is denser. Both are sub-2ms at any sensible scale.
+- vscode `bitmap.bin` is **4.1 MB** at 7,567 files; synth-10K is **17 MB** at 10,000 files. ~4× density difference, reflected directly in storage.
+
+The takeaway: **dense-synth identifies the failure modes; real-world data sets the actual user expectations.** Both are needed.
+
+## Linux kernel and Chromium
+
+Block 1.B in PEAK §10 explicitly calls out the Linux kernel and Chromium as scale targets. Neither is in `~/carto-test-repos` today (multi-GB clones, gated on the maintainer's disk budget). The `bench/scale-test/real-world.js` driver lands here so capturing those rows is one command once a clone exists:
+
+```bash
+git clone --depth=1 https://github.com/torvalds/linux ~/clones/linux
+node bench/scale-test/real-world.js --repo ~/clones/linux --name linux-kernel
+```
+
+```bash
+# Chromium follows the official `fetch` flow; expect a 30+ GB clone.
+node bench/scale-test/real-world.js --repo ~/clones/chromium/src --name chromium
+```
+
+These rows are intentionally not yet captured. The 50K dense-synth + 7,567-file vscode + 6,330-file supabase rows above already establish the scaling profile of the current implementation; whether kernel/Chromium clear or fail is *known* in the bitmap-storage sense (linear in N, hits the GB scale at 100K+ files in the worst case). Capturing them on the maintainer box is a confirm-the-known activity, not a discover-the-unknown one.
+
+## Reproducing this page
+
+Every number above is mechanically reproducible. The exact commands:
+
+```bash
+# Synth sweep (deterministic, dense-fan-out worst case)
+npm run bench:scale -- --size 1000
+npm run bench:scale -- --size 10000
+npm run bench:scale -- --size 50000
+
+# Real-world corpus (uses existing indexes; doesn't re-index)
+node bench/scale-test/real-world.js --repo ~/carto-test-repos/tier-b/vscode --name vscode --queries-only
+node bench/scale-test/real-world.js --repo ~/carto-test-repos/tier-b/supabase --name supabase --queries-only
+node bench/scale-test/real-world.js --repo ~/carto-test-repos/tier-b/nextjs --name nextjs --queries-only
+node bench/scale-test/real-world.js --repo ~/carto-test-repos/tier-b/cal.com --name cal.com --queries-only
+
+# Larger (maintainer-machine work)
+npm run bench:scale -- --size 100000 --keep --out /tmp/carto-100k
+```
+
+Each run writes `bench/scale-test/results/<tag>-<ISO-timestamp>.json` and refreshes `bench/scale-test/REPORT.md`. The aggregator keeps the latest run per (kind, label) so re-running a row replaces the old number.
+
+---
+
+_See `bench/scale-test/README.md` for full harness documentation._

package/docs/troubleshooting.md

@@ -0,0 +1,180 @@
+# Troubleshooting
+
+> Real failure modes, with fixes.
+
+## Install / native modules
+
+### `node-gyp rebuild` fails on install
+
+Symptom: `npm install -g carto-md` exits with C++ errors mentioning `tree-sitter` or `better-sqlite3`.
+
+Why: no prebuilt binary for your platform → falling back to compile-from-source → no C++ toolchain present.
+
+Fix: install build tools, then retry.
+
+| OS | Command |
+|---|---|
+| macOS | `xcode-select --install` |
+| Ubuntu / Debian | `sudo apt-get install build-essential python3` |
+| RHEL / Fedora | `sudo dnf groupinstall "Development Tools"` |
+| Windows | `npm install --global windows-build-tools` (run from elevated PowerShell) |
+| Alpine | `apk add --no-cache python3 make g++` |
+
+If you're on a supported platform (macOS arm64/x64, Linux x64 glibc + musl, Windows x64), you *shouldn't* be hitting this. Run `carto doctor` — the output will tell you which native module is the problem.
+
+### Node version refused at install time
+
+```
+[CARTO] Node v16.x.x is too old. carto-md requires Node ≥ v18.
+```
+
+Upgrade Node. https://nodejs.org/ — even-numbered LTS releases (18, 20, 22) are best.
+
+### Missing optional grammars
+
+```
+⚠ Tree-sitter grammars (2 of 8 missing: tree-sitter-rust, tree-sitter-java)
+```
+
+The optional language grammars failed to prebuild on your platform. Files in those languages still get indexed — Carto falls back to regex extraction (less accurate but functional).
+
+To force a grammar to install:
+
+```bash
+npm install -g tree-sitter-rust
+```
+
+If that fails too, the platform doesn't have a prebuild and won't compile. That's fine — regex fallback works.
+
+## Index / sync
+
+### `carto sync` says "0 files re-parsed" but I changed files
+
+Cache hit. Carto stats each file's mtime+size and skips ones that haven't changed. If you genuinely changed a file but mtime didn't move (rare — usually copying a same-content file), force a re-parse:
+
+```bash
+touch src/the/file.ts
+carto sync
+```
+
+For a full rebuild from scratch:
+
+```bash
+carto remove
+carto init
+```
+
+### `carto check` claims unstable clustering
+
+```
+[CARTO] Warning: 8.2% of files changed domain since last sync — clustering unstable.
+```
+
+The Leiden+CPM clusterer can be sensitive when the codebase is borderline between two stable partitions. If this happens repeatedly:
+
+1. Pin the important files with `anchor` in `carto.config.json`. See [`docs/concepts/domains.md`](./concepts/domains.md).
+2. If a domain is genuinely splitting (a refactor in progress), this is informational — accept the drift.
+3. If you don't care about exact domain names but want stability, set keyword hints in `carto.config.json` so the names are forced.
+
+### Index is huge
+
+A 100K-file repo produces a 160 MB `carto.db` and 20 MB `bitmap.bin`. Larger than expected? Most likely you're indexing `node_modules/`, a `vendor/` dir, or a generated `dist/`. Run:
+
+```bash
+carto inspect | head -20
+```
+
+That shows the file count. If it's way bigger than your source tree, edit `.cartoignore` to exclude the offender:
+
+```
+node_modules/
+vendor/
+**/__generated__/
+build/
+dist/
+out/
+```
+
+## MCP / AI tool integration
+
+### AI tool doesn't see Carto's tools
+
+1. Restart the AI tool (it loads MCP config at startup).
+2. Run `carto doctor` — it surfaces whether config files exist for the tools you have installed.
+3. If `doctor` says "MCP configuration: no MCP config files found", re-run `carto init` (it auto-wires).
+4. If `init` doesn't detect your tool, see the README's manual config snippets in the [Manual MCP wiring](../README.md) section (collapsed `<details>` block right after the install snippet).
+
+### MCP server crashes
+
+Pre-2.0.7 the MCP server could crash on tool errors and the AI tool would see `-32000 Failed to reconnect`. Fixed in 2.0.7. If you're seeing it on a newer version, set `CARTO_DEBUG=1` before launching the AI tool to capture stderr.
+
+### "Lazy re-parse" doesn't fire on stale files
+
+The MCP server stats the file on every file-aware query and re-parses if mtime is newer than the DB row. If a query returns stale data:
+
+1. Check the file's mtime: `stat src/the/file.ts`
+2. Compare against the DB row: `carto inspect | grep "last full sync"`
+3. If mtime is older, sync was after edit and the data isn't stale.
+4. If mtime is newer but the data is stale, file a bug — that's the regression test target.
+
+## Validation API / PR Action
+
+### `carto pr-impact` says "No carto index"
+
+The action's cache restored an empty `.carto/`. Either:
+
+- Cache key isn't invalidating. Check `hashFiles` glob in the workflow.
+- First run on this branch — let it run `carto init` cold once.
+- Index was removed; run `carto init` manually inside the workflow before pr-impact.
+
+### Sticky comment isn't sticky
+
+The action looks for the `<!-- carto-impact-report -->` marker in existing PR comments and updates the first match. If multiple comments have the marker (somehow), the action updates the first. If you see duplicate comments, check that the workflow isn't running twice (e.g. matrix builds, multiple triggers).
+
+### `--fail-on HIGH` trips too often
+
+Lower the threshold to `MEDIUM` for a softer gate, or use `--fail-on` only on `main`-bound PRs and not feature-branch PRs. Or accept that HIGH-risk PRs deserve careful review and let the gate work as intended.
+
+## SWE-bench harness
+
+### `bench/swe-bench/run.sh --task-set verified` says "dataset not found"
+
+Download the SWE-bench-Verified dataset from Hugging Face:
+
+```bash
+pip install huggingface_hub
+huggingface-cli download princeton-nlp/SWE-bench_Verified \
+  --repo-type dataset --local-dir ~/swe-bench-verified
+```
+
+Or set `CARTO_SWE_VERIFIED_PATH=/path/to/test.jsonl`.
+
+### AnthropicAgent throws on `--arm carto`
+
+You ran `--task-set verified` without `ANTHROPIC_API_KEY`. Set the key, or stick to `--task-set sample` which uses the deterministic StubAgent.
+
+### Cost runaway during a real-API run
+
+Set an Anthropic spend limit on your account dashboard *before* a long run. The harness emits per-task token counts to the JSONL output — tail it during the run to spot anomalies early:
+
+```bash
+tail -f bench/swe-bench/results/<run-id>.jsonl | jq .tokensInput
+```
+
+## General
+
+### `carto doctor` is the right starting point
+
+For any "something is broken" report, run `carto doctor` first. It checks Node, native modules, the index, git hooks, MCP config, .cartoignore. The output points at the broken component with a `Fix:` line.
+
+### Found a real bug
+
+Open an issue with:
+
+- `node --version`
+- `carto --version`
+- Output of `carto doctor`
+- Output of `carto inspect | head -30`
+- Steps to reproduce
+
+The maintainer triage is fast on real bugs with reproducible info. The triage is slow on vague "carto doesn't work" reports.