carto-md 2.0.8 → 2.1.0

package/docs/api/get_working_memory.md

@@ -0,0 +1,25 @@
+# `get_working_memory`
+
+Live state snapshot: branch, HEAD, uncommitted files, recent decision count, open HIGH-severity warnings, recent drift. Read this at the start of every AI session.
+
+## Input schema
+
+```json
+{
+  "type": "object",
+  "properties": {},
+  "required": []
+}
+```
+
+## Required arguments
+
+_None._
+
+## Properties
+
+_None._
+
+## See also
+
+- [Index of all MCP tools](./README.md)

package/docs/api/ingest_otlp_traces.md

@@ -0,0 +1,34 @@
+# `ingest_otlp_traces`
+
+Parse an OpenTelemetry OTLP JSON/JSONL trace file and aggregate per-route hit counts. Use the resulting counts with get_risk_weighted_blast_radius.
+
+## Input schema
+
+```json
+{
+  "type": "object",
+  "properties": {
+    "path": {
+      "type": "string",
+      "description": "Path to OTLP file"
+    }
+  },
+  "required": [
+    "path"
+  ]
+}
+```
+
+## Required arguments
+
+- `path`
+
+## Properties
+
+| Name | Type | Description |
+|------|------|-------------|
+| `path` | string | Path to OTLP file |
+
+## See also
+
+- [Index of all MCP tools](./README.md)

package/docs/api/scaffold_for_intent.md

@@ -0,0 +1,34 @@
+# `scaffold_for_intent`
+
+For a natural-language intent ("add a payment route"), returns: anchor file + co-changed files + canonical pattern + conventions to follow. Combines invariants, conventions, and procedural memory.
+
+## Input schema
+
+```json
+{
+  "type": "object",
+  "properties": {
+    "intent": {
+      "type": "string",
+      "description": "Natural-language description of the change."
+    }
+  },
+  "required": [
+    "intent"
+  ]
+}
+```
+
+## Required arguments
+
+- `intent`
+
+## Properties
+
+| Name | Type | Description |
+|------|------|-------------|
+| `intent` | string | Natural-language description of the change. |
+
+## See also
+
+- [Index of all MCP tools](./README.md)

package/docs/api/search_routes.md

@@ -0,0 +1,34 @@
+# `search_routes`
+
+Search API routes by path or method. Case-insensitive.
+
+## Input schema
+
+```json
+{
+  "type": "object",
+  "properties": {
+    "query": {
+      "type": "string",
+      "description": "Search query e.g. \"auth\", \"POST\", \"/api/users\""
+    }
+  },
+  "required": [
+    "query"
+  ]
+}
+```
+
+## Required arguments
+
+- `query`
+
+## Properties
+
+| Name | Type | Description |
+|------|------|-------------|
+| `query` | string | Search query e.g. "auth", "POST", "/api/users" |
+
+## See also
+
+- [Index of all MCP tools](./README.md)

package/docs/api/simulate_change_impact.md

@@ -0,0 +1,37 @@
+# `simulate_change_impact`
+
+Given a list of files, returns all files transitively affected by changing them simultaneously, with hop distance. Powered by the bitmap engine — only feasible at this speed (sub-millisecond) with bitmap OR-aggregation. Use when planning a refactor that touches multiple files.
+
+## Input schema
+
+```json
+{
+  "type": "object",
+  "properties": {
+    "files": {
+      "type": "array",
+      "items": {
+        "type": "string"
+      },
+      "description": "Array of relative file paths from project root"
+    }
+  },
+  "required": [
+    "files"
+  ]
+}
+```
+
+## Required arguments
+
+- `files`
+
+## Properties
+
+| Name | Type | Description |
+|------|------|-------------|
+| `files` | array | Array of relative file paths from project root |
+
+## See also
+
+- [Index of all MCP tools](./README.md)

package/docs/api/validate_change.md

@@ -0,0 +1,39 @@
+# `validate_change`
+
+Pre-write governance: given a file + proposed full content, synthesizes a diff vs disk and runs validate_diff. Use in IDE onWillSaveTextDocument hooks.
+
+## Input schema
+
+```json
+{
+  "type": "object",
+  "properties": {
+    "file": {
+      "type": "string"
+    },
+    "content": {
+      "type": "string"
+    }
+  },
+  "required": [
+    "file",
+    "content"
+  ]
+}
+```
+
+## Required arguments
+
+- `file`
+- `content`
+
+## Properties
+
+| Name | Type | Description |
+|------|------|-------------|
+| `file` | string |  |
+| `content` | string |  |
+
+## See also
+
+- [Index of all MCP tools](./README.md)

package/docs/api/validate_diff.md

@@ -0,0 +1,39 @@
+# `validate_diff`
+
+Given a unified diff, returns: violations (cross-domain imports, high-blast files), blast radius per file, risk level (SAFE/LOW/MEDIUM/HIGH), and suggestions. Sub-15ms p99 on a 7K-file repo. Each call is recorded in the episodic memory log so other tools can ask "did we discuss this?".
+
+## Input schema
+
+```json
+{
+  "type": "object",
+  "properties": {
+    "diff": {
+      "type": "string",
+      "description": "Unified diff text (output of `git diff` / GitHub PR patch)."
+    },
+    "session_id": {
+      "type": "number",
+      "description": "Optional session id. Defaults to the most recent active session, or a fresh one."
+    }
+  },
+  "required": [
+    "diff"
+  ]
+}
+```
+
+## Required arguments
+
+- `diff`
+
+## Properties
+
+| Name | Type | Description |
+|------|------|-------------|
+| `diff` | string | Unified diff text (output of `git diff` / GitHub PR patch). |
+| `session_id` | number | Optional session id. Defaults to the most recent active session, or a fresh one. |
+
+## See also
+
+- [Index of all MCP tools](./README.md)

package/docs/concepts/anci.md

@@ -0,0 +1,87 @@
+# Concept: ANCI
+
+> **Architecturally Normalized Code Index** — the open file format for any codebase to describe its architecture to AI tools.
+
+## The problem
+
+Every AI coding tool today re-discovers a codebase's architecture from scratch on every session. Cursor builds its own embedding index. Cline builds its own. Continue builds its own. Same parsing, every tool, every session, every time. Wasted CPU; wasted disk; inconsistent results.
+
+What's missing is a *format*. OpenAPI did this for REST APIs — once any API has an `openapi.yaml`, every tool consumes it the same way. ANCI is the equivalent for codebases.
+
+## What it is
+
+Two files at `.carto/anci.{yaml,bin}`. Together they describe:
+
+- the domain partition (which files belong to which cluster)
+- the import graph (forward + reverse adjacency)
+- per-file metadata (path, language, exports)
+- popcount index (files ranked by transitive dependents)
+
+The split is hybrid by design:
+
+- **`anci.yaml`** — strict-subset YAML 1.2, human-readable, grep-able. Holds the schema version, generator info, domain names, and counts. ~1–4 KB.
+- **`anci.bin`** — binary body. Magic `0x49434E41` ("ANCI"), version byte, six sections (forward graph, reverse graph, popcount, paths, file→domain, domain names) packed via Roaring-bitmap-like compression. ~200 KB to ~5 MB depending on repo size.
+
+Why split: a 100K-file repo as pure YAML is 50–500 MB — too large for an AI tool to consume per-session. The binary section solves the size problem without sacrificing the human-readable header.
+
+## Carto is the reference implementation
+
+`carto sync` writes both files automatically on every full sync. The same data the MCP server hands the AI is what ANCI captures on disk. Any tool can consume it without speaking MCP:
+
+```js
+const { loadAnci } = require('carto-md/src/anci/consumer');
+
+const reader = loadAnci('./.carto');
+console.log(reader.domains);                                    // [{name, file_count}, ...]
+console.log(reader.getHighImpactFiles(5));                      // top 5 by transitive dependents
+console.log(reader.blastRadius('src/auth/session.ts'));         // { count, hops, files }
+console.log(reader.simulateChangeImpact([                       // multi-file change blast radius
+  'src/auth/session.ts',
+  'src/db/connection.ts',
+]));
+```
+
+`loadAnci()` has zero dependencies. Anyone shipping an AI tool can drop it in and consume Carto's index without taking on Carto's whole tarball.
+
+## The CLI surface
+
+```bash
+carto anci publish               # re-emit anci.{yaml,bin} from the current index
+carto anci show                  # human-readable summary
+carto anci validate ./.carto     # validate a published pair
+```
+
+Publish runs implicitly on `carto sync`, so you only need to invoke these manually if you're debugging.
+
+## Versioning
+
+The spec is at `docs/anci/v0.1-DRAFT.md`. It's a draft; the wire format may break between v0.1.x patches. v1.0+ will follow semver and consumers refusing unsupported major versions is the contract.
+
+The header version is what consumers look at first:
+
+```yaml
+anci:
+  version: "0.1.0-DRAFT"
+  generator: "carto-md@2.0.9"
+  generated_at: "2026-06-26T12:34:56Z"
+```
+
+A consumer should reject anything outside its supported major.minor prefix.
+
+## Why publish a spec at all
+
+If only Carto consumes Carto's binary format, it's just an internal optimization. If three AI tools agree on a format, it becomes the standard. The OpenAPI precedent: Swagger shipped as a proprietary format in 2011, became de-facto by 2014, formalized as the OpenAPI Initiative by 2016.
+
+Carto's bet is the same: publish the binary spec early, support it as the reference implementation, and let the ecosystem coalesce. Once 3+ AI tools read ANCI, every AI tool has to.
+
+## Status
+
+- Spec: v0.1 DRAFT, published in [`docs/anci/v0.1-DRAFT.md`](../anci/v0.1-DRAFT.md)
+- Carto generates the files on every `carto sync`
+- Consumer library shipped: `src/anci/consumer.js`
+- Partner integration: TBD — first one is the unblocker
+
+## Related
+
+- [`docs/anci/v0.1-DRAFT.md`](../anci/v0.1-DRAFT.md) — the spec itself
+- [`mcp-integration.md`](./mcp-integration.md) — the alternative consumption path (live, slightly more flexible, but every consumer pays the MCP-protocol surface tax)

package/docs/concepts/blast-radius.md

@@ -0,0 +1,66 @@
+# Concept: Blast Radius
+
+> The set of files affected by changing a given file, computed transitively over the import graph.
+
+## The intuition
+
+Open `src/lib/db.ts`. Other files `import` from it. Those files have other files that import *them*. And so on. Anything reachable by following reverse-import edges is in the **blast radius** of `db.ts`.
+
+```
+src/lib/db.ts
+   ↑
+src/store/user.ts ← src/api/users.ts
+                  ← src/api/admin.ts
+   ↑
+src/store/order.ts ← src/api/orders.ts
+                   ← src/jobs/billing.ts
+```
+
+If you change `db.ts`, six files might break. That's the blast radius.
+
+## How carto computes it
+
+Two layers:
+
+**1. Direct dependents (1 hop).** The import graph stores every `from_file → to_file` edge. The reverse — *who imports me* — is one SQL query: `SELECT from_file_id FROM imports WHERE to_file_id = ?`.
+
+**2. Transitive dependents (≤5 hops).** From the seed file, BFS over the reverse-import graph. Carto caps at 5 hops because diminishing returns: 99% of real change impact lives in the first 3 hops, hops 4–5 catch the outliers, and beyond that you're paying memory for noise.
+
+The bitmap engine makes this fast. A 1M-edge graph fits in a few MB of Roaring Bitmaps; one transitive query is a chain of `bitmap_OR` operations executed in SIMD at ~1 cycle per 512 bits. On the vscode repo (7,567 files, 13,335 edges) a 5-hop blast-radius query returns in ~3 µs.
+
+## What you do with it
+
+Three places it lands:
+
+- **`carto impact <file>`** prints the radius for a single file
+- **`get_blast_radius(file)`** MCP tool gives the AI the same data
+- **`validate_diff(patch)`** uses blast radius to assign a risk grade to a PR
+
+For the last one, the rule of thumb is:
+
+| Direct dependents | Risk (default thresholds) |
+|------------------:|---------------------------|
+| 0–2               | SAFE                      |
+| 3–20              | LOW                       |
+| 21–50             | MEDIUM                    |
+| 51+               | HIGH                      |
+
+These thresholds are tunable per project via `carto.config.json`. Cross-domain edges (e.g. AUTH importing PAYMENTS) escalate independently — see [`domains.md`](./domains.md).
+
+## What it isn't
+
+- **Not a behavioral analysis.** Blast radius doesn't tell you which dependents will actually break, just which ones *might*. The AI uses this as a pre-filter, not as a verdict.
+- **Not call-graph reachability.** It's import-graph reachability. If A imports B but never calls anything from it (dead import), B is still in A's blast radius. We don't try to model dynamic dispatch or runtime imports.
+- **Not a refactoring oracle.** A 50-dependent file isn't automatically bad. Some files (utility modules, type definitions) *should* be widely depended on. Blast radius is a signal, not a sentence.
+
+## Why the bitmap version matters
+
+The original SQLite-only implementation worked fine to ~10K files but slowed down rapidly past that — each transitive query is `O(hops × dependents × indexed-lookup)` which adds up. Bitmap queries are `O(hops × bitmap-width)` regardless of how many dependents each file has. On 1M-file synthetic graphs the bitmap path returns in microseconds where SQLite takes seconds.
+
+See [`docs/scale.md`](../scale.md) for the per-tool latency table at synthetic scale and on real-world repos.
+
+## Related
+
+- [`get_blast_radius`](./mcp-integration.md#get_blast_radius) MCP tool reference
+- [`simulate_change_impact`](./mcp-integration.md#simulate_change_impact) — union of the blast radius for multiple files
+- [`validate_diff`](./mcp-integration.md#validate_diff) — diff-shaped risk grade built on blast radius

package/docs/concepts/domains.md

@@ -0,0 +1,91 @@
+# Concept: Domains
+
+> Auto-detected clusters of tightly-coupled files. Carto's "AUTH", "PAYMENTS", "DATABASE" — without you writing a config.
+
+## The intuition
+
+Files that import each other heavily belong together. Files that don't, don't. Cluster the import graph and you get a set of communities — those communities are the codebase's *domains*.
+
+Carto runs **Leiden+CPM** (Constant Potts Model) on the import graph. It's the same family of community-detection algorithms used in social-network analysis. The output: each file gets a domain id; the bitmap engine indexes by it; everything downstream (cross-domain checks, the impact report, the validation API) reads off this.
+
+## Why not just regex on paths?
+
+Path-based detection works for greenfield projects with a clean `src/auth/`, `src/payments/` layout. It collapses on:
+
+- Monorepos where every package has its own `auth/` subdir
+- Legacy projects where the directory structure doesn't match the actual coupling (auth code mysteriously lives in `lib/middleware/`)
+- Frameworks that hide the structure (Next.js routes spread across `pages/`, `app/`, `api/`)
+
+Graph clustering is path-agnostic. It looks at who imports whom and produces named communities whose names are derived from the path tokens *most common in that cluster*. That's the magic — the cluster is found by structure, the *name* is found by frequency on the actual file paths in it.
+
+## Naming heuristics
+
+After clustering, Carto picks a name for each cluster:
+
+1. Tokenize each file path: `src/auth/login.ts` → `[src, auth, login]`
+2. Drop stopwords (`src`, `lib`, `app`, `index`, `utils`, `helpers`)
+3. Cross-reference with a small built-in dictionary of well-known domain hints: `AUTH`, `PAYMENTS`, `DATABASE`, `NOTIFICATIONS`, `EVENTS`, `TRPC`, `CORE`
+4. Most-frequent matching token wins. Ties broken by dictionary priority (security-relevant domains first).
+5. If no token matches the dictionary, the most-common path token becomes the domain name (uppercased).
+
+The result: real repos light up with sensible labels — vscode gets `EXTENSIONS`, `EVENTS`, `DATABASE`, `CLI`, `CORE`; zed (Rust) gets `DATABASE`, `AUTH`, `EVENTS`, `CORE`. A game engine would get `RENDERER`, `PHYSICS`, `AUDIO` automatically because those tokens dominate the path namespace.
+
+## Adaptive gamma
+
+The Leiden+CPM algorithm has a resolution parameter γ (gamma). High γ → many small clusters. Low γ → few big clusters. Carto picks γ based on repo size:
+
+- **<100 files:** keyword-only fallback (graph clustering over-fragments tiny repos)
+- **100–1,000 files:** γ = 0.04
+- **1,000–10,000 files:** γ scales linearly to 0.02
+- **>10,000 files:** γ = 0.015 (large repos have dense graphs; tighter resolution prevents one giant component)
+
+These constants come from validation on 12 real repos. See `src/store/sync.js` for the exact formula.
+
+## Custom domains via `carto.config.json`
+
+Override the auto-detection if your repo needs it. Two shapes:
+
+**Simple — keyword hints:**
+
+```json
+{
+  "domains": {
+    "EDITOR": ["editor", "monaco", "text"],
+    "WORKBENCH": ["workbench", "layout", "panel"]
+  }
+}
+```
+
+Files whose path tokens match the hints get pulled into the named domain even if the graph clusterer would have put them elsewhere.
+
+**Full — keywords + anchors:**
+
+```json
+{
+  "domains": {
+    "AUTH": {
+      "keywords": ["auth", "login", "session"],
+      "anchor": ["src/auth/session.ts", "src/auth/middleware.ts"]
+    }
+  }
+}
+```
+
+Anchor files are *pinned*. They always end up in their named domain regardless of clustering. Useful when you've got a deliberately central auth file that the graph clusterer would otherwise lump into CORE.
+
+## Stability across syncs
+
+Re-clustering after a code change can shuffle domain assignments. Carto tracks the previous mapping and warns when >5% of files change domain between syncs (`carto check` surfaces it as "unstable clustering"). Anchor files give you a way to force stability where it matters.
+
+## Where domains show up
+
+- **`AGENTS.md`** lists them with file counts
+- **`.carto/context/<DOMAIN>.md`** has per-domain context the AI reads when working in that area
+- **`get_domain(name)`** MCP tool returns the full list of files, routes, models in a domain
+- **`validate_diff`** uses cross-domain edges as a violation signal — adding `auth/login.ts → payments/billing.ts` is HIGH-risk because it crosses a sensitive boundary
+
+## Related
+
+- [`get_cross_domain`](./mcp-integration.md#get_cross_domain) — every import edge that crosses a domain
+- [`carto check`](../../README.md#cli-commands) — flags new cross-domain violations
+- [Leiden algorithm paper](https://www.nature.com/articles/s41598-019-41695-z) — the algorithm Carto uses

package/docs/concepts/import-graph.md

@@ -0,0 +1,102 @@
+# Concept: Import Graph
+
+> The foundation. Every other Carto feature is computed from this one structure.
+
+## What's in it
+
+For each source file Carto parses, it records:
+
+| Source field | Meaning |
+|--------------|---------|
+| `id`         | Stable integer (monotonic per project) |
+| `path`       | Project-relative, forward-slash-normalized |
+| `language`   | `javascript`, `typescript`, `python`, `go`, `rust`, `java`, `cpp`, `csharp`, `ruby`, `r`, `prisma` |
+| `hash`       | SHA-256 of content (mtime+size cache uses this) |
+| `is_entry_point` | True for top-level CLIs, `main.*` files, Next.js page entries |
+
+And for each import the parser finds:
+
+| Edge field | Meaning |
+|------------|---------|
+| `from_file_id` | The importer |
+| `to_file_id`   | The importee (null if unresolved — e.g. a bare module name like `express`) |
+| `to_path`      | The raw import specifier |
+| `resolved`     | Boolean — did we successfully find the target in the index? |
+
+## What parser does what
+
+Two-layer extraction:
+
+**1. Tree-sitter** (per-language plugin in `src/extractors/languages/`). Parses the file into a syntax tree, walks the tree, pulls out:
+
+- import statements (`import x from`, `from x import`, `use crate::x`, `using A.B;`, `require 'x'`, `#include "x"`)
+- exported symbols
+- type definitions (interfaces, classes, structs)
+
+For supported grammars, this is ~0.05–0.2 ms/file.
+
+**2. Babel deep-parse** (TypeScript/JavaScript only, opt-in for route+model files). Used when the framework-specific extractor needs more than tree-sitter can give — e.g. tRPC procedures, React Router routes, Zod schemas, Prisma models. Slower (~5–20 ms/file) but reserved for the small subset of files where it matters.
+
+## Import resolution
+
+Local imports (`./foo`, `../utils`) are resolved relative to the importing file's directory, then tried against the index:
+
+```
+./foo                  → src/lib/foo.ts ?
+                       → src/lib/foo.js ?
+                       → src/lib/foo/index.ts ?
+                       → src/lib/foo/index.js ?
+```
+
+The first hit wins. TypeScript `paths` aliases (`@/components/Button`) are read from `tsconfig.json` and applied before the relative search — so `@/components/Button` resolves to `src/components/Button.tsx` if that's what the alias maps to.
+
+Bare module names (`express`, `react`, `lodash`) are *intentionally* not resolved. They're external; Carto's job is to know the *internal* structure.
+
+## How it's stored
+
+SQLite tables `files` + `imports`. One row per file, one row per edge. Indexes on `(from_file_id)`, `(to_file_id)`, `(path)`.
+
+Plus a derived bitmap layer in `.carto/bitmap.bin`:
+
+- `forward[i]` = bitmap of files that file `i` imports (1-hop)
+- `reverse[i]` = bitmap of files that import file `i` (1-hop)
+- `popcountIndex` = files sorted by transitive 5-hop dependent count, desc
+
+The bitmap layer is the read path. The SQLite tables are the write path + source of truth. The bitmap is rebuilt from SQLite on first sync after upgrade.
+
+## Path-alias support
+
+Carto reads `paths` from:
+
+- `tsconfig.json` → alias resolution for `.ts/.tsx`
+- `jsconfig.json` → alias resolution for `.js/.jsx`
+
+It does *not* currently read:
+
+- `webpack.config.js` `resolve.alias` (too many variants — convert to tsconfig if you need it indexed)
+- `vite.config.{js,ts}` `resolve.alias` (same)
+- `next.config.{js,mjs}` `experimental.transpilePackages` (only affects bundling, not module resolution)
+
+If you have a path alias that's not being resolved, declare it in `tsconfig.json#paths` even if you don't use TypeScript. Carto reads it; your runtime ignores it.
+
+## Edge cases we handle
+
+- **Type-only imports** (`import type { X }`) — counted as edges. They're real dependencies; if the type changes, the importer needs to update.
+- **Dynamic imports** (`import('./foo')`) — captured when the path argument is a string literal. Computed values are not resolved (no symbolic execution).
+- **Conditional imports** (inside `if (process.env.X)`) — captured statically; we don't try to evaluate the condition.
+- **Barrel re-exports** (`export * from './foo'`) — captured as edges, but the barrel itself doesn't get flagged as a "consumer" (it's just a pass-through).
+- **CommonJS `require()`** — captured for `.js/.cjs` files.
+- **Python `from .x import y`** — relative imports resolved against the package layout. Was a bug pre-2.0.7; fixed in Spec 7.
+
+## Edge cases we don't handle
+
+- **Monkey-patching / runtime mutation** — no.
+- **Reflection-based loading** (Python `importlib`, Java `Class.forName`) — no.
+- **Cross-language imports** (TS frontend HTTP-calling a Python backend route) — handled by route extraction, not the import graph. See [`get_change_plan`](./mcp-integration.md#get_change_plan).
+
+## What you'd want to look at
+
+- `carto inspect` — shows file count, edge count, schema version
+- `get_neighbors(file, hops)` MCP tool — exact 1- or 2-hop import neighbors of a single file
+- `get_high_impact_files(n)` MCP tool — top-N by transitive dependent count
+- `.carto/anci.bin` — the binary serialization of the whole graph; consumable by any AI tool via the `loadAnci()` library