carto-md 2.0.7 → 2.0.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +290 -26
- package/docs/anci/v0.1-DRAFT.md +420 -0
- package/docs/scale.md +129 -0
- package/package.json +10 -5
- package/scripts/postinstall.js +413 -0
- package/src/acp/agent.js +5 -5
- package/src/acp/providers/index.js +2 -2
- package/src/agents/leiden.js +11 -17
- package/src/agents/scan-structure.js +1 -1
- package/src/anci/consumer.js +305 -0
- package/src/anci/deserialize.js +160 -0
- package/src/anci/emit.js +85 -0
- package/src/anci/serialize.js +264 -0
- package/src/anci/yaml.js +401 -0
- package/src/bitmap/bitset.js +190 -0
- package/src/bitmap/index.js +121 -0
- package/src/bitmap/sidecar.js +545 -0
- package/src/bitmap/tools.js +310 -0
- package/src/cli/anci.js +237 -0
- package/src/cli/check.js +57 -0
- package/src/cli/index.js +28 -2
- package/src/cli/init.js +297 -65
- package/src/cli/inspect.js +295 -0
- package/src/cli/pr-impact.js +497 -0
- package/src/cli/serve.js +1 -1
- package/src/cli/watch.js +6 -0
- package/src/engine/worker.js +24 -4
- package/src/extractors/imports.js +176 -0
- package/src/extractors/languages/html.js +4 -1
- package/src/extractors/languages/javascript.js +5 -0
- package/src/extractors/languages/prisma.js +4 -1
- package/src/extractors/languages/python.js +5 -1
- package/src/extractors/languages/r.js +4 -1
- package/src/extractors/languages/typescript.js +2 -0
- package/src/extractors/tree-sitter-parser.js +15 -0
- package/src/mcp/change-plan.js +8 -8
- package/src/mcp/diff-parser.js +246 -0
- package/src/mcp/server-v2.js +489 -8
- package/src/mcp/validate.js +304 -0
- package/src/store/config-loader.js +77 -0
- package/src/store/sqlite-store.js +389 -4
- package/src/store/sync-v2.js +472 -97
- package/BENCHMARK_RESULTS.md +0 -34
package/README.md
CHANGED
|
@@ -17,7 +17,9 @@ carto init
|
|
|
17
17
|
|
|
18
18
|
That's it. Carto auto-wires itself into every AI tool you have installed. Restart your AI tool and it now knows your codebase architecturally.
|
|
19
19
|
|
|
20
|
-
**Works with:** Cursor · Claude Code · Kiro · Claude Desktop · Windsurf · VS Code Copilot · JetBrains · Zed
|
|
20
|
+
**Works with:** Cursor · Claude Code · Codex · Kiro · Claude Desktop · Windsurf · VS Code Copilot · JetBrains · Zed
|
|
21
|
+
|
|
22
|
+
> Carto also publishes [**ANCI**](#anci--the-open-spec-for-codebases-describing-themselves-to-ai) — the open file format for any codebase to describe its architecture to AI tools. Carto is the reference implementation; the spec is open and any tool can consume it. [§ ANCI ↓](#anci--the-open-spec-for-codebases-describing-themselves-to-ai)
|
|
21
23
|
|
|
22
24
|
---
|
|
23
25
|
|
|
@@ -34,7 +36,7 @@ That's it. Carto auto-wires itself into every AI tool you have installed. Restar
|
|
|
34
36
|
|
|
35
37
|
### Claude Code (CLI)
|
|
36
38
|
|
|
37
|
-
|
|
39
|
+
`carto init` writes `<project>/.mcp.json` for you when Claude Code is detected (`claude` binary on PATH or `~/.claude/` exists). Manual:
|
|
38
40
|
```bash
|
|
39
41
|
claude mcp add carto -- carto serve
|
|
40
42
|
```
|
|
@@ -43,6 +45,21 @@ Or create `.mcp.json` at the project root:
|
|
|
43
45
|
{ "mcpServers": { "carto": { "command": "carto", "args": ["serve"] } } }
|
|
44
46
|
```
|
|
45
47
|
|
|
48
|
+
### Codex (CLI)
|
|
49
|
+
|
|
50
|
+
`carto init` writes `~/.codex/config.toml` for you when Codex is detected (`codex` binary on PATH or `~/.codex/` exists). Manual:
|
|
51
|
+
```bash
|
|
52
|
+
codex mcp add carto -- carto serve
|
|
53
|
+
```
|
|
54
|
+
Or edit `~/.codex/config.toml`:
|
|
55
|
+
```toml
|
|
56
|
+
[mcp_servers.carto]
|
|
57
|
+
command = "carto"
|
|
58
|
+
args = ["serve"]
|
|
59
|
+
cwd = "/your/project"
|
|
60
|
+
enabled = true
|
|
61
|
+
```
|
|
62
|
+
|
|
46
63
|
### Kiro
|
|
47
64
|
|
|
48
65
|
`carto init` writes this for you. Manual: `~/.kiro/settings/mcp.json`
|
|
@@ -52,21 +69,25 @@ Or create `.mcp.json` at the project root:
|
|
|
52
69
|
|
|
53
70
|
### Claude Desktop
|
|
54
71
|
|
|
55
|
-
`carto init` writes this for you. Manual:
|
|
72
|
+
`carto init` writes this for you (cross-platform). Manual paths:
|
|
73
|
+
- macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
|
|
74
|
+
- Windows: `%APPDATA%\Claude\claude_desktop_config.json`
|
|
75
|
+
- Linux: `~/.config/Claude/claude_desktop_config.json` (community Linux builds)
|
|
76
|
+
|
|
56
77
|
```json
|
|
57
78
|
{ "mcpServers": { "carto": { "command": "carto", "args": ["serve"], "cwd": "/your/project" } } }
|
|
58
79
|
```
|
|
59
80
|
|
|
60
81
|
### VS Code Copilot
|
|
61
82
|
|
|
62
|
-
`.vscode/mcp.json` in your project root, or Command Palette → `MCP: Add Server`
|
|
83
|
+
`carto init` writes `<project>/.vscode/mcp.json` for you when the `code` binary is on PATH. Manual: `.vscode/mcp.json` in your project root, or Command Palette → `MCP: Add Server`. Note: VS Code uses `servers` (not `mcpServers`) and requires `"type": "stdio"`.
|
|
63
84
|
```json
|
|
64
|
-
{ "servers": { "carto": { "
|
|
85
|
+
{ "servers": { "carto": { "type": "stdio", "command": "carto", "args": ["serve"] } } }
|
|
65
86
|
```
|
|
66
87
|
|
|
67
88
|
### Windsurf
|
|
68
89
|
|
|
69
|
-
`~/.windsurf/
|
|
90
|
+
`carto init` writes this for you when Windsurf is detected. Manual: `~/.codeium/windsurf/mcp_config.json`
|
|
70
91
|
```json
|
|
71
92
|
{ "mcpServers": { "carto": { "command": "carto", "args": ["serve"], "cwd": "/your/project" } } }
|
|
72
93
|
```
|
|
@@ -94,6 +115,30 @@ Without carto, your AI greps text and guesses. With carto, it queries structural
|
|
|
94
115
|
|
|
95
116
|
Your AI's response goes from "here's a guess" to "here's the right change with the architectural context."
|
|
96
117
|
|
|
118
|
+
### Catching bad changes before save
|
|
119
|
+
|
|
120
|
+
The harder problem isn't finding the right file — it's stopping a confident-sounding AI from writing a refactor that breaks the rest of the repo. Carto exposes a diff-shaped query the AI can call before the user accepts a change:
|
|
121
|
+
|
|
122
|
+
> *AI proposes a 12-line patch to `packages/pg-meta/src/pg-format/index.ts`. Before showing the diff, it calls `validate_diff(patch)`.*
|
|
123
|
+
|
|
124
|
+
```
|
|
125
|
+
# Diff Validation
|
|
126
|
+
|
|
127
|
+
**Risk:** 🔴 HIGH
|
|
128
|
+
**Files changed:** 1
|
|
129
|
+
**Union blast radius:** 83 transitive dependents
|
|
130
|
+
|
|
131
|
+
## Violations (1)
|
|
132
|
+
|
|
133
|
+
| Severity | Kind | File | Detail |
|
|
134
|
+
|----------|-------------|----------------------------------------|-----------------------------------------------------------------|
|
|
135
|
+
| HIGH | high_blast | `packages/pg-meta/src/pg-format/index.ts` | Modifying this file affects 83 transitive dependents (>50). |
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
The AI sees this *before* it proposes the diff. It revises its plan, splits the change, or asks the user. The bad refactor never makes it to the screen. Sub-millisecond on a 7,000-file repo — see the **Benchmarks** section below.
|
|
139
|
+
|
|
140
|
+
Every `validate_diff` call is also written to a local SQLite log, so a session that runs five hours later can ask `did_we_discuss_this("snake_case naming")` and get back the prior decision. The AI stops re-deciding settled questions.
|
|
141
|
+
|
|
97
142
|
### In the wild
|
|
98
143
|
|
|
99
144
|

|
|
@@ -116,6 +161,9 @@ Your AI's response goes from "here's a guess" to "here's the right change with t
|
|
|
116
161
|
| C / C++ | `.cpp` `.cc` `.cxx` `.h` `.hpp` |
|
|
117
162
|
| C# | `.cs` |
|
|
118
163
|
| Ruby | `.rb` |
|
|
164
|
+
| R | `.r` `.R` |
|
|
165
|
+
| Prisma schema | `.prisma` |
|
|
166
|
+
| HTML | `.html` (for `fetch()` discovery) |
|
|
119
167
|
|
|
120
168
|
### Route extraction (framework-aware)
|
|
121
169
|
|
|
@@ -192,7 +240,131 @@ Carto supports any LLM provider — configure in your editor:
|
|
|
192
240
|
|
|
193
241
|
---
|
|
194
242
|
|
|
195
|
-
##
|
|
243
|
+
## GitHub Action — PR impact reports
|
|
244
|
+
|
|
245
|
+
Drop carto onto every PR your repo gets. Posts a sticky comment on each pull request with the diff's blast radius, cross-domain violations, affected routes, and a risk badge.
|
|
246
|
+
|
|
247
|
+
`.github/workflows/carto.yml`:
|
|
248
|
+
```yaml
|
|
249
|
+
name: Carto Impact Report
|
|
250
|
+
|
|
251
|
+
on:
|
|
252
|
+
pull_request:
|
|
253
|
+
branches: [main]
|
|
254
|
+
|
|
255
|
+
permissions:
|
|
256
|
+
contents: read
|
|
257
|
+
pull-requests: write
|
|
258
|
+
|
|
259
|
+
jobs:
|
|
260
|
+
carto:
|
|
261
|
+
runs-on: ubuntu-latest
|
|
262
|
+
steps:
|
|
263
|
+
- uses: actions/checkout@v4
|
|
264
|
+
with: { fetch-depth: 0 }
|
|
265
|
+
- uses: theanshsonkar/carto@v2.0.9
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
That's the whole config. The action handles `npm install`, builds (or restores from cache) the `.carto/` index, runs `carto pr-impact`, and posts the comment via `GITHUB_TOKEN`.
|
|
269
|
+
|
|
270
|
+
### Inputs
|
|
271
|
+
|
|
272
|
+
| Input | Default | What it does |
|
|
273
|
+
|-------|---------|--------------|
|
|
274
|
+
| `carto-version` | `latest` | The `carto-md` npm version to install. Pin in production for reproducibility. |
|
|
275
|
+
| `base` | auto (`origin/$GITHUB_BASE_REF`) | Git ref the PR branched from. |
|
|
276
|
+
| `head` | auto (`$GITHUB_SHA`) | Git ref of the PR head. |
|
|
277
|
+
| `fail-on` | _(empty)_ | Fail the workflow when risk meets/exceeds this severity. One of `HIGH`, `MEDIUM`, `LOW`. Empty = comment-only. |
|
|
278
|
+
| `comment-mode` | `sticky` | `sticky` updates the existing carto comment in place. `new` posts a new comment every push. `none` skips posting (renders to stdout). |
|
|
279
|
+
| `node-version` | `20` | Node.js version on the runner. |
|
|
280
|
+
|
|
281
|
+
### Outputs
|
|
282
|
+
|
|
283
|
+
| Output | Description |
|
|
284
|
+
|--------|-------------|
|
|
285
|
+
| `risk` | Rolled-up risk: `SAFE` \| `LOW` \| `MEDIUM` \| `HIGH`. Lets downstream steps gate behavior on Carto's verdict. |
|
|
286
|
+
| `comment-url` | URL of the posted/updated PR comment. |
|
|
287
|
+
|
|
288
|
+
### What the comment looks like
|
|
289
|
+
|
|
290
|
+
```markdown
|
|
291
|
+
## 🗺️ Carto Impact Report
|
|
292
|
+
|
|
293
|
+
This PR touches AUTH and DATABASE domains.
|
|
294
|
+
|
|
295
|
+
| Metric | Value |
|
|
296
|
+
|--------|-------|
|
|
297
|
+
| Risk | 🔴 HIGH |
|
|
298
|
+
| Blast radius (union) | 23 files |
|
|
299
|
+
| Files changed | 6 |
|
|
300
|
+
| Cross-domain violations introduced | 2 |
|
|
301
|
+
| High-impact file changed | src/auth/session.ts (8 direct dependents) |
|
|
302
|
+
|
|
303
|
+
<details>
|
|
304
|
+
<summary>Affected routes (4)</summary>
|
|
305
|
+
|
|
306
|
+
- POST /auth/login — risk: HIGH
|
|
307
|
+
- GET /auth/me — risk: HIGH
|
|
308
|
+
- POST /auth/register — risk: MEDIUM
|
|
309
|
+
- POST /api/users — risk: LOW
|
|
310
|
+
|
|
311
|
+
</details>
|
|
312
|
+
|
|
313
|
+
<details>
|
|
314
|
+
<summary>Cross-domain violations (2)</summary>
|
|
315
|
+
|
|
316
|
+
- auth/login.ts now imports from payments/billing.ts (AUTH→PAYMENTS)
|
|
317
|
+
- database/user-repo.ts now imports from auth/jwt.ts (DATABASE→AUTH)
|
|
318
|
+
|
|
319
|
+
</details>
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
### Standalone CLI use
|
|
323
|
+
|
|
324
|
+
The action is a thin wrapper around `carto pr-impact`. Use it locally, in custom CI, or in pre-commit hooks:
|
|
325
|
+
|
|
326
|
+
```bash
|
|
327
|
+
carto pr-impact --base origin/main --head HEAD # markdown to stdout
|
|
328
|
+
carto pr-impact --base origin/main --head HEAD --format json
|
|
329
|
+
carto pr-impact --base origin/main --head HEAD --fail-on HIGH # exit 2 on HIGH risk
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
---
|
|
333
|
+
|
|
334
|
+
## ANCI — the open spec for codebases describing themselves to AI
|
|
335
|
+
|
|
336
|
+
Every AI coding tool today re-discovers a codebase's architecture from scratch on every session. Cursor builds its own embedding index. Cline builds its own. Continue builds its own. Same parsing, every tool, every session.
|
|
337
|
+
|
|
338
|
+
**ANCI** (Architecturally Normalized Code Index) is the file format that fixes this. Two files at `.carto/anci.{yaml,bin}` that describe the codebase's architecture in a form any AI tool can read without indexing it itself. OpenAPI did this for REST APIs. ANCI does it for codebases.
|
|
339
|
+
|
|
340
|
+
`carto sync` writes both files automatically. The header is grep-able YAML; the body is a compact binary import graph. Spec lives in [`docs/anci/v0.1-DRAFT.md`](docs/anci/v0.1-DRAFT.md). Carto is the reference implementation.
|
|
341
|
+
|
|
342
|
+
```bash
|
|
343
|
+
carto anci publish # re-emit anci.{yaml,bin} from the index
|
|
344
|
+
carto anci show # human-readable summary
|
|
345
|
+
carto anci validate ./.carto # validate a published pair
|
|
346
|
+
```
|
|
347
|
+
|
|
348
|
+
Any tool can consume an ANCI pair without depending on Carto:
|
|
349
|
+
|
|
350
|
+
```js
|
|
351
|
+
const { loadAnci } = require('carto-md/src/anci/consumer');
|
|
352
|
+
|
|
353
|
+
const reader = loadAnci('./.carto');
|
|
354
|
+
console.log(reader.domains); // [{ name: 'AUTH', file_count: 42 }, ...]
|
|
355
|
+
console.log(reader.getHighImpactFiles(5)); // top 5 by transitive dependents
|
|
356
|
+
console.log(reader.blastRadius('src/auth/session.ts'));// { count, hops, files: [...] }
|
|
357
|
+
console.log(reader.simulateChangeImpact([ // multi-file change blast radius
|
|
358
|
+
'src/auth/session.ts',
|
|
359
|
+
'src/db/connection.ts',
|
|
360
|
+
]));
|
|
361
|
+
```
|
|
362
|
+
|
|
363
|
+
> **Status:** v0.1.0-DRAFT — wire format may change up to v1.0. The reference implementation lives in this repo at `src/anci/`.
|
|
364
|
+
|
|
365
|
+
---
|
|
366
|
+
|
|
367
|
+
|
|
196
368
|
|
|
197
369
|
Once carto is wired in, your AI tool can call any of these mid-task. You don't need to memorize them — your AI will pick the right ones.
|
|
198
370
|
|
|
@@ -201,6 +373,8 @@ Once carto is wired in, your AI tool can call any of these mid-task. You don't n
|
|
|
201
373
|
| `get_architecture()` | 500-word project overview: domains, entry points, stack, key patterns. **Use this first.** |
|
|
202
374
|
| `get_change_plan(intent)` | Given "add rate limiting to /api/users" → files to touch, domains affected, blast radius, similar patterns |
|
|
203
375
|
| `get_blast_radius(file)` | All files affected by changing a given file, with hop distance |
|
|
376
|
+
| `simulate_change_impact(files)` | Union of all files transitively affected by changing **multiple** files at once. Powered by the bitmap engine — sub-millisecond on 7K-file repos. |
|
|
377
|
+
| `validate_diff(diff)` | Given a unified diff: violations (cross-domain imports, high-blast files), blast radius per file, risk level (SAFE/LOW/MEDIUM/HIGH), suggestions. Each call is recorded in the **episodic memory** log so other tools can ask "did we discuss this?". Sub-15ms p99. |
|
|
204
378
|
| `get_context(file)` | Everything about a file: domain, blast radius, neighbors, routes, models |
|
|
205
379
|
| `get_file_summary(file)` | What a file does, its role, key deps and dependents |
|
|
206
380
|
| `get_similar_patterns(file)` | Files with same domain, route shape, or shared deps — find conventions before writing new code |
|
|
@@ -214,8 +388,16 @@ Once carto is wired in, your AI tool can call any of these mid-task. You don't n
|
|
|
214
388
|
| `get_high_impact_files(n)` | Top N files by blast radius |
|
|
215
389
|
| `get_env_vars(domain?)` | All env vars with domain mapping |
|
|
216
390
|
| `get_domains_list()` | All detected domains with file, route, model counts |
|
|
391
|
+
| `get_recent_decisions(time_range, kind?)` | Recent validation decisions and architectural choices the AI has made in this project |
|
|
392
|
+
| `get_session_context(session_id?)` | Full context for an AI session: every decision and intervention, ordered chronologically |
|
|
393
|
+
| `did_we_discuss_this(topic)` | Substring search over the episodic memory log — avoid re-deciding settled questions |
|
|
394
|
+
| `get_intervention_history(file?)` | Past Carto-issued violations and suggestions, optionally filtered by file |
|
|
395
|
+
|
|
396
|
+
**Latency:** All bitmap-backed queries return in microseconds on real repos; cross-domain and `simulate_change_impact` settle under 2ms even on a 7.5K-file codebase like vscode. See [Benchmarks](#benchmarks) for the per-tool table.
|
|
397
|
+
|
|
398
|
+
## Episodic Memory
|
|
217
399
|
|
|
218
|
-
|
|
400
|
+
Carto remembers every diff it validates. The `validate_diff` tool writes one row per call into a local SQLite log (`ai_sessions`/`decisions`/`interventions` tables) — so a session that runs five hours later can still ask `did_we_discuss_this("snake_case naming")` and get back the prior decision. The log lives next to the index in `.carto/carto.db` — never sent over the network, never shared between projects.
|
|
219
401
|
|
|
220
402
|
---
|
|
221
403
|
|
|
@@ -223,6 +405,8 @@ Once carto is wired in, your AI tool can call any of these mid-task. You don't n
|
|
|
223
405
|
|
|
224
406
|
Carto uses **Leiden+CPM graph clustering** — files that import each other heavily cluster together. Domain names are inferred from path tokens, with keyword hints for well-known patterns (AUTH, PAYMENTS, DATABASE, etc.).
|
|
225
407
|
|
|
408
|
+
**Adaptive strategy:** Repos under 100 files use keyword-only clustering (avoids over-fragmentation). Larger repos with dense import graphs get graph-based clustering with a gamma that scales continuously with repo size.
|
|
409
|
+
|
|
226
410
|
Works on any repo — not just SaaS apps. vscode gets AUTH/EVENTS/DATABASE. zed (Rust) gets DATABASE/AUTH/EVENTS. A game engine would get RENDERER/PHYSICS/AUDIO.
|
|
227
411
|
|
|
228
412
|
Custom domains via `carto.config.json`:
|
|
@@ -235,55 +419,111 @@ Custom domains via `carto.config.json`:
|
|
|
235
419
|
}
|
|
236
420
|
```
|
|
237
421
|
|
|
422
|
+
Full schema with anchor pinning (forces files into a domain regardless of clustering):
|
|
423
|
+
```json
|
|
424
|
+
{
|
|
425
|
+
"domains": {
|
|
426
|
+
"AUTH": {
|
|
427
|
+
"keywords": ["auth", "login", "session"],
|
|
428
|
+
"anchor": ["src/auth/session.ts", "src/auth/middleware.ts"]
|
|
429
|
+
}
|
|
430
|
+
}
|
|
431
|
+
}
|
|
432
|
+
```
|
|
433
|
+
|
|
434
|
+
**Stability tracking:** Carto tracks domain assignments across syncs. If >5% of files change domain, `carto check` flags it as unstable. Two consecutive syncs with no code changes always produce 0% drift.
|
|
435
|
+
|
|
238
436
|
---
|
|
239
437
|
|
|
240
438
|
## CLI commands
|
|
241
439
|
|
|
242
440
|
| Command | What it does |
|
|
243
441
|
|---------|-------------|
|
|
244
|
-
| `carto init` | Detect project, index codebase, generate AGENTS.md, auto-wire MCP into every AI tool found |
|
|
245
|
-
| `carto sync` | Full re-index (skips unchanged files via mtime+size cache) |
|
|
246
|
-
| `carto
|
|
247
|
-
| `carto
|
|
442
|
+
| `carto init` | Detect project, index codebase, generate AGENTS.md, install git hooks (pre-commit, post-checkout, post-merge, post-rewrite), auto-wire MCP into every AI tool found |
|
|
443
|
+
| `carto sync` | Full re-index (skips unchanged files via mtime+size cache). Called automatically by git hooks on commit/checkout/merge/rebase. |
|
|
444
|
+
| `carto serve` | Start MCP server (called by your AI tool — usually you don't run this directly). On every file-aware query the server mtime+size-checks the file and re-parses inline if stale. |
|
|
445
|
+
| `carto watch` | **Optional.** Live re-index on every file save. Not required — git hooks + lazy MCP re-parse keep the index fresh by default. Use only for AI-heavy workflows that write 50+ files between commits. |
|
|
248
446
|
| `carto agent` | Start ACP agent mode (for Zed / JetBrains / VS Code) |
|
|
249
447
|
| `carto impact <file>` | Blast radius: risk level, affected files, routes at risk |
|
|
448
|
+
| `carto pr-impact` | Diff-shaped impact report between two git refs. Markdown (default) or JSON. Used by the [GitHub Action](#github-action--pr-impact-reports); works locally too. `--fail-on HIGH\|MEDIUM\|LOW` exits non-zero on threshold trip. |
|
|
250
449
|
| `carto check` | Cross-domain violations, high-risk uncommitted changes, domain health |
|
|
450
|
+
| `carto inspect` | Read-only diagnostic: index paths, sizes, freshness, bitmap sidecar shape, top-impact files, schema version, sync timestamps. `--json` for piping into `jq`. Never triggers a rebuild. |
|
|
251
451
|
| `carto remove` | Remove AGENTS.md and .carto/ from project |
|
|
252
452
|
|
|
253
453
|
---
|
|
254
454
|
|
|
255
455
|
## Benchmarks
|
|
256
456
|
|
|
257
|
-
Measured on real open-source repos. Apple M-series, 8 CPUs, 8GB RAM. SHAs pinned in `~/carto-test-repos`.
|
|
457
|
+
Measured on real open-source repos. Apple M-series, 8 CPUs, 8GB RAM. SHAs pinned in `~/carto-test-repos`. Reproducible via `~/carto-test-repos/run-bench.sh`.
|
|
458
|
+
|
|
459
|
+
### Indexing speed
|
|
258
460
|
|
|
259
461
|
| Repo | Language | Indexed Files | First Run | Second Run | DB Size | Import Edges |
|
|
260
462
|
|------|----------|---------------|-----------|------------|---------|--------------|
|
|
261
|
-
| [prisma/prisma](https://github.com/prisma/prisma) | TypeScript | 961 | **
|
|
262
|
-
| [supabase/supabase](https://github.com/supabase/supabase) | TypeScript | 6,
|
|
263
|
-
| [microsoft/vscode](https://github.com/microsoft/vscode) | TypeScript | 7,567 | **
|
|
264
|
-
| [zed-industries/zed](https://github.com/zed-industries/zed) | Rust | 1,752 | **
|
|
463
|
+
| [prisma/prisma](https://github.com/prisma/prisma) | TypeScript | 961 | **1.0s** | **350ms** | 1.1 MB | 1,387 |
|
|
464
|
+
| [supabase/supabase](https://github.com/supabase/supabase) | TypeScript | 6,330 | **5.4s** | **1.2s** | 4.8 MB | 5,189 |
|
|
465
|
+
| [microsoft/vscode](https://github.com/microsoft/vscode) | TypeScript | 7,567 | **8.0s** | **935ms** | 14.3 MB | 13,335 |
|
|
466
|
+
| [zed-industries/zed](https://github.com/zed-industries/zed) | Rust | 1,752 | **2.9s** | **468ms** | 4.8 MB | 2,110 |
|
|
467
|
+
|
|
468
|
+
**Indexed Files** counts what Carto actually parses — `.ts/.js/.py/.go/.rs/...` after excluding `node_modules`, build output, and per-file test patterns (`*.test.*` / `*.spec.*` / `*.stories.*` for JS/TS, `test_*.py` / `*_test.py` for Python, `test_*` / `*_test.r` for R). The on-disk file count of the repo is larger.
|
|
469
|
+
|
|
470
|
+
**Second Run** = `carto sync` after no changes. mtime+size checked before reading content — if nothing changed, nothing is re-parsed.
|
|
471
|
+
|
|
472
|
+
### MCP query latency (bitmap engine vs SQLite)
|
|
473
|
+
|
|
474
|
+
Carto's MCP query path is bitmap-backed on five tools, plus a sixth (`simulate_change_impact`) that's only feasible with bitmap OR-aggregation. Speedups measured against the same data, same DB, same machine — bitmap path vs the equivalent SQLite path.
|
|
475
|
+
|
|
476
|
+
| Tool | vscode (7,567 files) | Speedup vs SQLite |
|
|
477
|
+
|------|---------------------:|------------------:|
|
|
478
|
+
| `get_blast_radius` | sub-ms | **10.7×** |
|
|
479
|
+
| `get_cross_domain` | 2.1ms | **6.2×** |
|
|
480
|
+
| `get_high_impact_files` | sub-ms | **559×** |
|
|
481
|
+
| `get_similar_patterns` | sub-ms | **73×** |
|
|
482
|
+
| `simulate_change_impact` | sub-ms | **6.5×** (no SQLite equivalent at this latency) |
|
|
483
|
+
|
|
484
|
+
Median speedup across all five tools on vscode: **10.7×**. Smaller repos with denser graphs hit higher peaks — supabase `get_high_impact_files` clocks well over 100× on its tighter import graph. Reproducible via `npm run bench:bitmap -- --repo <path>`.
|
|
485
|
+
|
|
486
|
+
### `validate_diff` latency
|
|
487
|
+
|
|
488
|
+
The new diff-shaped query that lets the AI ask "is this patch safe?" before showing it to the user. Profiled with a representative 20-line diff against 20 random mid-blast-radius files per repo, 1000 calls each.
|
|
489
|
+
|
|
490
|
+
| Repo | Files | p50 | p99 |
|
|
491
|
+
|------|-------|----:|----:|
|
|
492
|
+
| supabase | 6,259 | **0.082ms** | **0.298ms** |
|
|
493
|
+
| vscode | 7,567 | **0.084ms** | **0.489ms** |
|
|
494
|
+
|
|
495
|
+
Budget was p50 ≤ 5ms, p99 ≤ 15ms. Both targets are cleared by 30-60×. The bitmap engine handles every blast-radius and cross-domain query in microseconds; what's left is diff parsing + result aggregation. Reproducible via `node bench/validation-perf/index.js --repo <path>`.
|
|
496
|
+
|
|
497
|
+
### Scale
|
|
265
498
|
|
|
266
|
-
|
|
499
|
+
Synthetic stress sweep + real-world corpus, dense-fan-out worst case all the way to 50K files. Headline numbers (full table in [`docs/scale.md`](docs/scale.md)):
|
|
267
500
|
|
|
268
|
-
|
|
501
|
+
- Synth 50K files: init 1.1m, `blast_radius` p50 22µs, `simulate_change_impact` p50 50µs, `high_impact_files` p50 750ns. The dense Uint32Array bitset hits 415 MB on disk and 1.35 GB peak RSS at this size — the Tier-2 Roaring upgrade per PEAK §9.6 is the next move.
|
|
502
|
+
- Real-world (vscode, 7,567 files): `blast_radius` p50 2.7µs, `cross_domain` p50 1.23ms, `similar_patterns` p50 834ns, `simulate_change_impact` p50 19µs.
|
|
503
|
+
|
|
504
|
+
Reproducible via `npm run bench:scale -- --size <N>` (synth) and `node bench/scale-test/real-world.js --repo <path>` (any local clone, including Linux kernel or Chromium).
|
|
269
505
|
|
|
270
506
|
### Domains detected
|
|
271
507
|
|
|
272
508
|
| Repo | Domains |
|
|
273
509
|
|------|---------|
|
|
274
|
-
| prisma | CORE · DATABASE ·
|
|
275
|
-
| supabase | CORE · AUTH · DATABASE · PAYMENTS ·
|
|
276
|
-
| vscode |
|
|
277
|
-
| zed (Rust) | CORE · DATABASE · AUTH · EVENTS · PAYMENTS ·
|
|
510
|
+
| prisma | CORE · DATABASE · AUTH · EVENTS |
|
|
511
|
+
| supabase | CORE · AUTH · DATABASE · PAYMENTS · NOTIFICATIONS · EVENTS · TRPC |
|
|
512
|
+
| vscode | EXTENSIONS · AUTH · EVENTS · DATABASE · EXTENSION · CLI · CORE |
|
|
513
|
+
| zed (Rust) | CORE · DATABASE · AUTH · EVENTS · PAYMENTS · TRPC · NOTIFICATIONS |
|
|
514
|
+
|
|
515
|
+
vscode at 7,567 indexed files in around 8 seconds. Rust import graph working on zed (2,110 edges from `mod` declarations and `use crate::` paths).
|
|
516
|
+
|
|
517
|
+
### Accuracy
|
|
278
518
|
|
|
279
|
-
|
|
519
|
+
12 corpus repos pass `node test/accuracy-corpus.js --samples 100` — full parity between the bitmap path and the SQLite path on `blastRadius`, `crossDomain`, `highImpactFiles`, and `simulateChangeImpact`. The bitmap layer is a speedup, never a behavior change.
|
|
280
520
|
|
|
281
521
|
---
|
|
282
522
|
|
|
283
523
|
## How it works
|
|
284
524
|
|
|
285
525
|
```
|
|
286
|
-
carto init
|
|
526
|
+
carto init
|
|
287
527
|
↓
|
|
288
528
|
Discovers all files (no cap — SQLite handles the volume)
|
|
289
529
|
mtime+size check → skip unchanged files
|
|
@@ -293,9 +533,22 @@ Leiden+CPM graph clustering → auto-detects domains
|
|
|
293
533
|
Computes reverse deps → blast radius for every file
|
|
294
534
|
Writes AGENTS.md + .carto/context/*.md (lazy, on-demand)
|
|
295
535
|
Auto-wires MCP into every AI tool found
|
|
536
|
+
Installs 4 git hooks: pre-commit, post-checkout, post-merge, post-rewrite
|
|
537
|
+
↓
|
|
538
|
+
[no daemon, no watcher, no background process]
|
|
539
|
+
↓
|
|
540
|
+
─── Freshness mechanism 1: git hooks (90% of cases) ───
|
|
541
|
+
You commit / pull / switch branches / rebase
|
|
542
|
+
→ hook runs `carto sync` quietly in <1s
|
|
543
|
+
→ only changed files re-parsed
|
|
296
544
|
↓
|
|
297
|
-
|
|
545
|
+
─── Freshness mechanism 2: lazy mtime check (the gap) ───
|
|
546
|
+
You edit files between commits, AI asks "blast radius of db.ts?"
|
|
547
|
+
→ MCP server stats db.ts (mtime+size vs DB row)
|
|
548
|
+
→ if stale, re-parses just that file inline (<50ms)
|
|
549
|
+
→ returns fresh answer
|
|
298
550
|
↓
|
|
551
|
+
─── Optional: carto watch (AI-heavy workflows only) ───
|
|
299
552
|
File saved → debounce 50ms → re-parse 1 file → SQLite write → <50ms
|
|
300
553
|
```
|
|
301
554
|
|
|
@@ -306,10 +559,21 @@ File saved → debounce 50ms → re-parse 1 file → SQLite write → <50ms
|
|
|
306
559
|
- **Sends your code anywhere.** Local only. SQLite on disk.
|
|
307
560
|
- **Writes secrets into AGENTS.md.** `.cartoignore` blocks `.env` and credential files by default.
|
|
308
561
|
- **Touches your manual notes.** Writes only between `<!-- CARTO:AUTO:START -->` and `<!-- CARTO:AUTO:END -->`.
|
|
562
|
+
- **Forces you to install a C++ toolchain.** Prebuilt native binaries ship for macOS arm64, Linux x64 (glibc + musl/Alpine), and Windows x64. Intel Macs and other platforms transparently fall back to building from source, then to regex-only extraction if that fails.
|
|
309
563
|
- **Costs money.** MIT license. Free forever.
|
|
310
564
|
|
|
311
565
|
---
|
|
312
566
|
|
|
567
|
+
## Origin
|
|
568
|
+
|
|
569
|
+
I was building [Emfirge](https://www.emfirge.cloud) — a cloud security agent that maps AWS infrastructure into a graph and simulates the blast radius of every change.
|
|
570
|
+
|
|
571
|
+
To make the AI inside Emfirge understand infrastructure, I wrote a module called `cartography.py`. It mapped AWS resources, built a graph of how they connected, and wrote it into a structured map. The AI stopped hallucinating. It worked with facts, not guesses.
|
|
572
|
+
|
|
573
|
+
Carto is the same idea, applied to source code. Same insight: AI agents stop guessing once they can query the architecture.
|
|
574
|
+
|
|
575
|
+
---
|
|
576
|
+
|
|
313
577
|
## License
|
|
314
578
|
|
|
315
579
|
MIT. Free forever.
|