carto-md 2.0.8 → 2.0.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -19,6 +19,8 @@ That's it. Carto auto-wires itself into every AI tool you have installed. Restar
19
19
 
20
20
  **Works with:** Cursor · Claude Code · Codex · Kiro · Claude Desktop · Windsurf · VS Code Copilot · JetBrains · Zed
21
21
 
22
+ > Carto also publishes [**ANCI**](#anci--the-open-spec-for-codebases-describing-themselves-to-ai) — the open file format for any codebase to describe its architecture to AI tools. Carto is the reference implementation; the spec is open and any tool can consume it. [§ ANCI ↓](#anci--the-open-spec-for-codebases-describing-themselves-to-ai)
23
+
22
24
  ---
23
25
 
24
26
  ## Use it with your AI tool
@@ -159,6 +161,9 @@ Every `validate_diff` call is also written to a local SQLite log, so a session t
159
161
  | C / C++ | `.cpp` `.cc` `.cxx` `.h` `.hpp` |
160
162
  | C# | `.cs` |
161
163
  | Ruby | `.rb` |
164
+ | R | `.r` `.R` |
165
+ | Prisma schema | `.prisma` |
166
+ | HTML | `.html` (for `fetch()` discovery) |
162
167
 
163
168
  ### Route extraction (framework-aware)
164
169
 
@@ -235,7 +240,131 @@ Carto supports any LLM provider — configure in your editor:
235
240
 
236
241
  ---
237
242
 
238
- ## The 22 MCP tools
243
+ ## GitHub Action PR impact reports
244
+
245
+ Drop carto onto every PR your repo gets. Posts a sticky comment on each pull request with the diff's blast radius, cross-domain violations, affected routes, and a risk badge.
246
+
247
+ `.github/workflows/carto.yml`:
248
+ ```yaml
249
+ name: Carto Impact Report
250
+
251
+ on:
252
+ pull_request:
253
+ branches: [main]
254
+
255
+ permissions:
256
+ contents: read
257
+ pull-requests: write
258
+
259
+ jobs:
260
+ carto:
261
+ runs-on: ubuntu-latest
262
+ steps:
263
+ - uses: actions/checkout@v4
264
+ with: { fetch-depth: 0 }
265
+ - uses: theanshsonkar/carto@v2.0.9
266
+ ```
267
+
268
+ That's the whole config. The action handles `npm install`, builds (or restores from cache) the `.carto/` index, runs `carto pr-impact`, and posts the comment via `GITHUB_TOKEN`.
269
+
270
+ ### Inputs
271
+
272
+ | Input | Default | What it does |
273
+ |-------|---------|--------------|
274
+ | `carto-version` | `latest` | The `carto-md` npm version to install. Pin in production for reproducibility. |
275
+ | `base` | auto (`origin/$GITHUB_BASE_REF`) | Git ref the PR branched from. |
276
+ | `head` | auto (`$GITHUB_SHA`) | Git ref of the PR head. |
277
+ | `fail-on` | _(empty)_ | Fail the workflow when risk meets/exceeds this severity. One of `HIGH`, `MEDIUM`, `LOW`. Empty = comment-only. |
278
+ | `comment-mode` | `sticky` | `sticky` updates the existing carto comment in place. `new` posts a new comment every push. `none` skips posting (renders to stdout). |
279
+ | `node-version` | `20` | Node.js version on the runner. |
280
+
281
+ ### Outputs
282
+
283
+ | Output | Description |
284
+ |--------|-------------|
285
+ | `risk` | Rolled-up risk: `SAFE` \| `LOW` \| `MEDIUM` \| `HIGH`. Lets downstream steps gate behavior on Carto's verdict. |
286
+ | `comment-url` | URL of the posted/updated PR comment. |
287
+
288
+ ### What the comment looks like
289
+
290
+ ```markdown
291
+ ## 🗺️ Carto Impact Report
292
+
293
+ This PR touches AUTH and DATABASE domains.
294
+
295
+ | Metric | Value |
296
+ |--------|-------|
297
+ | Risk | 🔴 HIGH |
298
+ | Blast radius (union) | 23 files |
299
+ | Files changed | 6 |
300
+ | Cross-domain violations introduced | 2 |
301
+ | High-impact file changed | src/auth/session.ts (8 direct dependents) |
302
+
303
+ <details>
304
+ <summary>Affected routes (4)</summary>
305
+
306
+ - POST /auth/login — risk: HIGH
307
+ - GET /auth/me — risk: HIGH
308
+ - POST /auth/register — risk: MEDIUM
309
+ - POST /api/users — risk: LOW
310
+
311
+ </details>
312
+
313
+ <details>
314
+ <summary>Cross-domain violations (2)</summary>
315
+
316
+ - auth/login.ts now imports from payments/billing.ts (AUTH→PAYMENTS)
317
+ - database/user-repo.ts now imports from auth/jwt.ts (DATABASE→AUTH)
318
+
319
+ </details>
320
+ ```
321
+
322
+ ### Standalone CLI use
323
+
324
+ The action is a thin wrapper around `carto pr-impact`. Use it locally, in custom CI, or in pre-commit hooks:
325
+
326
+ ```bash
327
+ carto pr-impact --base origin/main --head HEAD # markdown to stdout
328
+ carto pr-impact --base origin/main --head HEAD --format json
329
+ carto pr-impact --base origin/main --head HEAD --fail-on HIGH # exit 2 on HIGH risk
330
+ ```
331
+
332
+ ---
333
+
334
+ ## ANCI — the open spec for codebases describing themselves to AI
335
+
336
+ Every AI coding tool today re-discovers a codebase's architecture from scratch on every session. Cursor builds its own embedding index. Cline builds its own. Continue builds its own. Same parsing, every tool, every session.
337
+
338
+ **ANCI** (Architecturally Normalized Code Index) is the file format that fixes this. Two files at `.carto/anci.{yaml,bin}` that describe the codebase's architecture in a form any AI tool can read without indexing it itself. OpenAPI did this for REST APIs. ANCI does it for codebases.
339
+
340
+ `carto sync` writes both files automatically. The header is grep-able YAML; the body is a compact binary import graph. Spec lives in [`docs/anci/v0.1-DRAFT.md`](docs/anci/v0.1-DRAFT.md). Carto is the reference implementation.
341
+
342
+ ```bash
343
+ carto anci publish # re-emit anci.{yaml,bin} from the index
344
+ carto anci show # human-readable summary
345
+ carto anci validate ./.carto # validate a published pair
346
+ ```
347
+
348
+ Any tool can consume an ANCI pair without depending on Carto:
349
+
350
+ ```js
351
+ const { loadAnci } = require('carto-md/src/anci/consumer');
352
+
353
+ const reader = loadAnci('./.carto');
354
+ console.log(reader.domains); // [{ name: 'AUTH', file_count: 42 }, ...]
355
+ console.log(reader.getHighImpactFiles(5)); // top 5 by transitive dependents
356
+ console.log(reader.blastRadius('src/auth/session.ts'));// { count, hops, files: [...] }
357
+ console.log(reader.simulateChangeImpact([ // multi-file change blast radius
358
+ 'src/auth/session.ts',
359
+ 'src/db/connection.ts',
360
+ ]));
361
+ ```
362
+
363
+ > **Status:** v0.1.0-DRAFT — wire format may change up to v1.0. The reference implementation lives in this repo at `src/anci/`.
364
+
365
+ ---
366
+
367
+
239
368
 
240
369
  Once carto is wired in, your AI tool can call any of these mid-task. You don't need to memorize them — your AI will pick the right ones.
241
370
 
@@ -264,7 +393,7 @@ Once carto is wired in, your AI tool can call any of these mid-task. You don't n
264
393
  | `did_we_discuss_this(topic)` | Substring search over the episodic memory log — avoid re-deciding settled questions |
265
394
  | `get_intervention_history(file?)` | Past Carto-issued violations and suggestions, optionally filtered by file |
266
395
 
267
- **All MCP queries:** **<5ms** on every benchmarked repo.
396
+ **Latency:** All bitmap-backed queries return in microseconds on real repos; cross-domain and `simulate_change_impact` settle under 2ms even on a 7.5K-file codebase like vscode. See [Benchmarks](#benchmarks) for the per-tool table.
268
397
 
269
398
  ## Episodic Memory
270
399
 
@@ -316,6 +445,7 @@ Full schema with anchor pinning (forces files into a domain regardless of cluste
316
445
  | `carto watch` | **Optional.** Live re-index on every file save. Not required — git hooks + lazy MCP re-parse keep the index fresh by default. Use only for AI-heavy workflows that write 50+ files between commits. |
317
446
  | `carto agent` | Start ACP agent mode (for Zed / JetBrains / VS Code) |
318
447
  | `carto impact <file>` | Blast radius: risk level, affected files, routes at risk |
448
+ | `carto pr-impact` | Diff-shaped impact report between two git refs. Markdown (default) or JSON. Used by the [GitHub Action](#github-action--pr-impact-reports); works locally too. `--fail-on HIGH\|MEDIUM\|LOW` exits non-zero on threshold trip. |
319
449
  | `carto check` | Cross-domain violations, high-risk uncommitted changes, domain health |
320
450
  | `carto inspect` | Read-only diagnostic: index paths, sizes, freshness, bitmap sidecar shape, top-impact files, schema version, sync timestamps. `--json` for piping into `jq`. Never triggers a rebuild. |
321
451
  | `carto remove` | Remove AGENTS.md and .carto/ from project |
@@ -330,12 +460,12 @@ Measured on real open-source repos. Apple M-series, 8 CPUs, 8GB RAM. SHAs pinned
330
460
 
331
461
  | Repo | Language | Indexed Files | First Run | Second Run | DB Size | Import Edges |
332
462
  |------|----------|---------------|-----------|------------|---------|--------------|
333
- | [prisma/prisma](https://github.com/prisma/prisma) | TypeScript | 961 | **961ms** | **431ms** | 0.7 MB | 1,387 |
334
- | [supabase/supabase](https://github.com/supabase/supabase) | TypeScript | 6,330 | **5.4s** | **1.2s** | 4.0 MB | 5,321 |
335
- | [microsoft/vscode](https://github.com/microsoft/vscode) | TypeScript | 7,567 | **7.7s** | **1.0s** | 6.7 MB | 13,420 |
336
- | [zed-industries/zed](https://github.com/zed-industries/zed) | Rust | 1,752 | **3.0s** | **491ms** | 4.4 MB | 2,113 |
463
+ | [prisma/prisma](https://github.com/prisma/prisma) | TypeScript | 961 | **1.0s** | **350ms** | 1.1 MB | 1,387 |
464
+ | [supabase/supabase](https://github.com/supabase/supabase) | TypeScript | 6,330 | **5.4s** | **1.2s** | 4.8 MB | 5,189 |
465
+ | [microsoft/vscode](https://github.com/microsoft/vscode) | TypeScript | 7,567 | **8.0s** | **935ms** | 14.3 MB | 13,335 |
466
+ | [zed-industries/zed](https://github.com/zed-industries/zed) | Rust | 1,752 | **2.9s** | **468ms** | 4.8 MB | 2,110 |
337
467
 
338
- **Indexed Files** counts what Carto actually parses — `.ts/.js/.py/.go/.rs/...` after excluding `node_modules`, build output, and per-file `*.test.*` / `*.spec.*` / `*.stories.*` / `test_*.py` patterns. The on-disk file count of the repo is larger.
468
+ **Indexed Files** counts what Carto actually parses — `.ts/.js/.py/.go/.rs/...` after excluding `node_modules`, build output, and per-file test patterns (`*.test.*` / `*.spec.*` / `*.stories.*` for JS/TS, `test_*.py` / `*_test.py` for Python, `test_*` / `*_test.r` for R). The on-disk file count of the repo is larger.
339
469
 
340
470
  **Second Run** = `carto sync` after no changes. mtime+size checked before reading content — if nothing changed, nothing is re-parsed.
341
471
 
@@ -351,7 +481,7 @@ Carto's MCP query path is bitmap-backed on five tools, plus a sixth (`simulate_c
351
481
  | `get_similar_patterns` | sub-ms | **73×** |
352
482
  | `simulate_change_impact` | sub-ms | **6.5×** (no SQLite equivalent at this latency) |
353
483
 
354
- Median speedup across all five tools on vscode: **10.7×**. Smaller repos hit higher peaks — laravel-framework `get_high_impact_files` clocks at 155× on the smaller graph. Reproducible via `npm run bench:bitmap -- --repo <path>`.
484
+ Median speedup across all five tools on vscode: **10.7×**. Smaller repos with denser graphs hit higher peaks — supabase `get_high_impact_files` clocks well over 100× on its tighter import graph. Reproducible via `npm run bench:bitmap -- --repo <path>`.
355
485
 
356
486
  ### `validate_diff` latency
357
487
 
@@ -364,6 +494,15 @@ The new diff-shaped query that lets the AI ask "is this patch safe?" before show
364
494
 
365
495
  Budget was p50 ≤ 5ms, p99 ≤ 15ms. Both targets are cleared by 30-60×. The bitmap engine handles every blast-radius and cross-domain query in microseconds; what's left is diff parsing + result aggregation. Reproducible via `node bench/validation-perf/index.js --repo <path>`.
366
496
 
497
+ ### Scale
498
+
499
+ Synthetic stress sweep + real-world corpus, dense-fan-out worst case all the way to 50K files. Headline numbers (full table in [`docs/scale.md`](docs/scale.md)):
500
+
501
+ - Synth 50K files: init 1.1m, `blast_radius` p50 22µs, `simulate_change_impact` p50 50µs, `high_impact_files` p50 750ns. The dense Uint32Array bitset hits 415 MB on disk and 1.35 GB peak RSS at this size — the Tier-2 Roaring upgrade per PEAK §9.6 is the next move.
502
+ - Real-world (vscode, 7,567 files): `blast_radius` p50 2.7µs, `cross_domain` p50 1.23ms, `similar_patterns` p50 834ns, `simulate_change_impact` p50 19µs.
503
+
504
+ Reproducible via `npm run bench:scale -- --size <N>` (synth) and `node bench/scale-test/real-world.js --repo <path>` (any local clone, including Linux kernel or Chromium).
505
+
367
506
  ### Domains detected
368
507
 
369
508
  | Repo | Domains |
@@ -373,7 +512,7 @@ Budget was p50 ≤ 5ms, p99 ≤ 15ms. Both targets are cleared by 30-60×. The b
373
512
  | vscode | EXTENSIONS · AUTH · EVENTS · DATABASE · EXTENSION · CLI · CORE |
374
513
  | zed (Rust) | CORE · DATABASE · AUTH · EVENTS · PAYMENTS · TRPC · NOTIFICATIONS |
375
514
 
376
- vscode at 7,567 indexed files in under 8 seconds. Rust import graph working on zed (2,113 edges from `mod` declarations and `use crate::` paths).
515
+ vscode at 7,567 indexed files in around 8 seconds. Rust import graph working on zed (2,110 edges from `mod` declarations and `use crate::` paths).
377
516
 
378
517
  ### Accuracy
379
518
 
@@ -420,6 +559,7 @@ File saved → debounce 50ms → re-parse 1 file → SQLite write → <50ms
420
559
  - **Sends your code anywhere.** Local only. SQLite on disk.
421
560
  - **Writes secrets into AGENTS.md.** `.cartoignore` blocks `.env` and credential files by default.
422
561
  - **Touches your manual notes.** Writes only between `<!-- CARTO:AUTO:START -->` and `<!-- CARTO:AUTO:END -->`.
562
+ - **Forces you to install a C++ toolchain.** Prebuilt native binaries ship for macOS arm64, Linux x64 (glibc + musl/Alpine), and Windows x64. Intel Macs and other platforms transparently fall back to building from source, then to regex-only extraction if that fails.
423
563
  - **Costs money.** MIT license. Free forever.
424
564
 
425
565
  ---
@@ -0,0 +1,420 @@
1
+ # ANCI v0.1 DRAFT — Architecturally Normalized Code Index
2
+
3
+ > **Status:** DRAFT (unstable). Do not depend on this format in production
4
+ > consumers without pinning to a specific carto-md version. The wire
5
+ > format may change in any direction up to v1.0.
6
+ >
7
+ > **Spec:** v0.1.0-DRAFT
8
+ > **Reference implementation:** carto-md ≥ 2.0.9
9
+ > **Editor:** [@theanshsonkar](https://github.com/theanshsonkar)
10
+ > **License:** MIT (the spec itself)
11
+ > **Repository:** https://github.com/theanshsonkar/carto
12
+
13
+ ---
14
+
15
+ ## 1. Motivation
16
+
17
+ Every AI coding tool today re-discovers a codebase's architecture from
18
+ scratch on every session. Cursor builds its own embedding index. Cline
19
+ builds its own. Continue builds its own. The work duplicates across
20
+ tools and is lost across sessions.
21
+
22
+ ANCI fills the hole. It is a **static file format** — two files,
23
+ `anci.yaml` and `anci.bin`, sitting at a known location in a repository
24
+ — that describes that codebase's architecture in a form any AI tool can
25
+ read without indexing it itself.
26
+
27
+ ANCI is to codebases what OpenAPI is to REST APIs: a standardized way for
28
+ something to describe itself to consumers it doesn't know about.
29
+
30
+ ## 2. Goals & non-goals
31
+
32
+ ### Goals
33
+
34
+ * **Tool-neutral.** No assumption about which AI tool consumes ANCI.
35
+ * **Local-first.** ANCI files live next to the code, in `.carto/` by
36
+ convention. Never sent over a network unless the user chooses to.
37
+ * **Hybrid representation.** Human-readable header for grep / inspection;
38
+ binary body for fast queries on million-file repos.
39
+ * **Bounded size.** Target ≤ 20 MB total for a 100K-file repo.
40
+ * **Lossless round-trip.** Producer → file → consumer reproduces the
41
+ exact graph, domains, and metadata.
42
+ * **Forward-compatible.** New optional sections may be added in v0.x
43
+ releases. Consumers ignore unknown sections.
44
+
45
+ ### Non-goals
46
+
47
+ * **Not a query language.** Consumers traverse the data structures with
48
+ their own code. ANCI is just the data.
49
+ * **Not a network protocol.** No RPC. No streaming. Static files only.
50
+ * **Not a replacement for source code.** ANCI describes structure
51
+ (graph, domains, routes), not source.
52
+ * **Not yet stable.** v0.1 DRAFT may break in v0.2, v0.3, etc. Stability
53
+ begins at v1.0.
54
+
55
+ ## 3. File layout
56
+
57
+ A repository that ships ANCI places two files in `.carto/`:
58
+
59
+ ```
60
+ .carto/
61
+ ├── anci.yaml # Header. UTF-8 YAML. Required. Human-readable.
62
+ ├── anci.bin # Body. Binary. Required. Machine-readable.
63
+ ```
64
+
65
+ Producers write both. Consumers read either independently:
66
+
67
+ * If a consumer only needs domain names and route counts, parse
68
+ `anci.yaml` and ignore the body.
69
+ * If a consumer needs blast radius or graph traversal, parse the body.
70
+
71
+ The header MUST point to the body's filename and byte length. If they
72
+ disagree, the body is authoritative for graph data and the header is
73
+ authoritative for human metadata.
74
+
75
+ ## 4. Header format (`anci.yaml`)
76
+
77
+ UTF-8, YAML 1.2, **strict subset**:
78
+
79
+ * 2-space indentation
80
+ * `key: value` pairs only (no flow style `{}`/`[]`)
81
+ * All string values double-quoted (no implicit typing)
82
+ * Lists use `- ` prefix
83
+ * No multi-line strings, no anchors, no aliases, no tags
84
+
85
+ A v0.1.0-DRAFT producer MUST emit a header that conforms to this
86
+ schema. A consumer MAY accept other YAML inputs but reference
87
+ implementations only emit/parse the strict subset.
88
+
89
+ ### 4.1 Schema
90
+
91
+ ```yaml
92
+ anci:
93
+ version: "0.1.0-DRAFT" # required, string
94
+ generator: "carto-md@2.0.9" # required, string
95
+ generated_at: "2026-06-07T..." # required, ISO-8601 UTC
96
+ body:
97
+ file: "anci.bin" # required, relative to header
98
+ bytes: 12345 # required, body length on disk
99
+
100
+ project:
101
+ total_files: 7567 # required
102
+ total_routes: 86 # required
103
+ total_models: 12 # required
104
+ total_import_edges: 13420 # required
105
+
106
+ domains: # required, possibly empty
107
+ - name: "AUTH"
108
+ file_count: 42
109
+ route_count: 7
110
+ model_count: 1
111
+
112
+ high_impact: # optional, top N by transitive dep count
113
+ - file: "src/auth/session.ts"
114
+ transitive_dependents: 47
115
+
116
+ routes: # optional
117
+ - method: "POST"
118
+ path: "/auth/login"
119
+ file: "src/auth/login.ts"
120
+ framework: "express"
121
+ handler: "login"
122
+
123
+ models: # optional
124
+ - name: "User"
125
+ kind: "prisma"
126
+ file: "prisma/schema.prisma"
127
+ ```
128
+
129
+ Field names are stable for the v0.x line. Adding new optional fields is
130
+ not a breaking change. Removing or renaming a field IS breaking and
131
+ requires a version bump.
132
+
133
+ ### 4.2 Versioning
134
+
135
+ | Field | Meaning |
136
+ |-----------------|-----------------------------------------------------------|
137
+ | `anci.version` | Spec version. v0.x is unstable. v1.0+ follows semver. |
138
+ | `anci.generator`| Producer identity. Free-form string, conventionally `name@version`. |
139
+
140
+ Consumers MUST refuse to parse a header whose `anci.version` is not in
141
+ their compatibility set. Reference consumer behavior (carto-md):
142
+
143
+ * Accepts: `0.1.x-DRAFT`
144
+ * Rejects: anything else
145
+
146
+ ## 5. Body format (`anci.bin`)
147
+
148
+ Binary, little-endian, length-prefixed sections. Designed so a single
149
+ streaming pass can fully reconstruct all bitmaps, paths, and domain
150
+ mappings.
151
+
152
+ ### 5.1 Header
153
+
154
+ | Offset | Size | Field | Value |
155
+ |-------:|------:|--------------|-----------------------------------------------|
156
+ | 0 | 4 B | magic | `0x49434E41` (`'A','N','C','I'` little-endian)|
157
+ | 4 | 1 B | version | `0x01` |
158
+ | 5 | 3 B | reserved | zero |
159
+ | 8 | 4 B | size_bits | u32 — bit dimension of every bitmap (= max file id + 1) |
160
+
161
+ Total fixed-header length: **12 bytes.**
162
+
163
+ ### 5.2 Sections
164
+
165
+ After the fixed header, sections appear **in this order**. Each section
166
+ begins with a u32 count.
167
+
168
+ #### 5.2.1 `forward` — file → its imports
169
+
170
+ ```
171
+ count_u32
172
+ count × {
173
+ from_file_id_u32
174
+ words_len_u32
175
+ words_u32 × words_len // raw Uint32Array bytes, LE
176
+ }
177
+ ```
178
+
179
+ Bit `j` of the bitmap of `from_file_id` is 1 iff the file with id
180
+ `from_file_id` directly imports the file with id `j`.
181
+
182
+ #### 5.2.2 `reverse` — file → its direct dependents
183
+
184
+ Same record shape as `forward`. Bit `j` set iff file `to_file_id` is
185
+ directly imported by file `j`.
186
+
187
+ #### 5.2.3 `popcount` — pre-computed transitive dependent counts
188
+
189
+ ```
190
+ count_u32
191
+ count × {
192
+ file_id_u32
193
+ count_u32 // # of distinct dependents reachable in ≤ 5 hops
194
+ }
195
+ ```
196
+
197
+ Sorted DESC by count. Lets `getHighImpactFiles(n)` answer in O(1).
198
+ Producers MUST clamp BFS to 5 hops to match the carto-md reference.
199
+
200
+ #### 5.2.4 `paths` — file id → path string
201
+
202
+ ```
203
+ count_u32
204
+ count × {
205
+ file_id_u32
206
+ path_len_u32
207
+ path_bytes (UTF-8, no NUL terminator)
208
+ }
209
+ ```
210
+
211
+ Paths are project-relative POSIX paths (forward slashes, no leading `./`).
212
+
213
+ #### 5.2.5 `file_domain` — file id → domain id
214
+
215
+ ```
216
+ count_u32
217
+ count × {
218
+ file_id_u32
219
+ domain_id_u32
220
+ }
221
+ ```
222
+
223
+ Files absent from this section have no domain assignment. A file MAY
224
+ have at most one domain.
225
+
226
+ #### 5.2.6 `domain_names` — domain id → human name
227
+
228
+ ```
229
+ count_u32
230
+ count × {
231
+ domain_id_u32
232
+ name_len_u32
233
+ name_bytes (UTF-8)
234
+ }
235
+ ```
236
+
237
+ Domain names are uppercase by convention (`AUTH`, `PAYMENTS`,
238
+ `DATABASE`) but the spec does not constrain their contents.
239
+
240
+ ### 5.3 Total layout (concrete byte map)
241
+
242
+ ```
243
+ [ 12 B fixed header ]
244
+ [ forward section ]
245
+ [ reverse section ]
246
+ [ popcount section ]
247
+ [ paths section ]
248
+ [ file_domain section ]
249
+ [ domain_names section]
250
+ EOF
251
+ ```
252
+
253
+ A consumer that knows the spec version can stream-parse top to bottom.
254
+ There is no out-of-line index — the format is designed for one
255
+ sequential read.
256
+
257
+ ## 6. Generation algorithm (reference)
258
+
259
+ Producers SHOULD compute the body as follows:
260
+
261
+ 1. Assign each file a stable, dense integer id.
262
+ 2. For each file, build a bitmap of its forward imports (resolved
263
+ imports only; unresolved imports are excluded).
264
+ 3. Reverse the forward bitmaps into `reverse`.
265
+ 4. For each file with ≥ 1 dependent, run a 5-hop BFS over `reverse`
266
+ using the same self-loop guard as `bitmap/sidecar.js`:
267
+ a. Seed `visited` with direct dependents.
268
+ b. Expand frontier by one hop, mask `visited`, add result to
269
+ `visited`. Repeat to depth 5 or until frontier is empty.
270
+ c. Clear the seed bit (cycles must not count the file itself).
271
+ d. Record `popcount(visited)` if > 0.
272
+ 5. Sort `popcount` records DESC.
273
+ 6. Write sections in the order specified in §5.2.
274
+
275
+ The carto-md reference implementation lives in `src/anci/serialize.js`.
276
+
277
+ ## 7. Consumer responsibilities
278
+
279
+ A consumer MUST:
280
+
281
+ * Validate magic bytes and version before reading any section.
282
+ * Refuse to parse if `size_bits` differs from the bitmap dimension
283
+ derived from any section's `words_len`.
284
+ * Treat unknown sections (in future v0.x releases) as opaque and skip
285
+ them. The reference implementation enforces strict order today, but
286
+ consumers SHOULD be permissive on order to ease forward compatibility.
287
+
288
+ A consumer SHOULD:
289
+
290
+ * Cache the parsed body in memory rather than re-parsing on every
291
+ query. The format is designed for fast load, not fast random seek.
292
+ * Cross-validate the header's `body.bytes` against the actual file
293
+ size and warn (not fail) on mismatch.
294
+
295
+ ## 8. Versioning & deprecation policy
296
+
297
+ ### v0.1.0-DRAFT (this spec)
298
+
299
+ * Format may change in any direction up to v1.0.
300
+ * Consumers pin to specific producer versions.
301
+ * Reference implementation is carto-md ≥ 2.0.9.
302
+ * No backward compatibility guarantees.
303
+
304
+ ### Future v0.x (e.g. v0.2)
305
+
306
+ * MAY add optional header fields.
307
+ * MAY add optional body sections (appended after `domain_names`).
308
+ * MUST NOT remove or rename existing fields.
309
+ * MUST bump `anci.version` to a new minor.
310
+ * Reference consumer accepts the latest v0.x and the version it shipped
311
+ against.
312
+
313
+ ### v1.0 stability
314
+
315
+ When the format is judged ready (reference partner integrations exist,
316
+ plus 6 months of v0.x stability), `anci.version` graduates to `1.0.0`
317
+ and standard semver applies thereafter:
318
+
319
+ * MAJOR: incompatible breaking change.
320
+ * MINOR: backward-compatible addition.
321
+ * PATCH: clarification only.
322
+
323
+ A v1.x consumer MUST accept all v1.x bodies. v0.x compatibility at v1.0
324
+ is not guaranteed — producers re-emit at the v1.0 schema.
325
+
326
+ ## 9. Security considerations
327
+
328
+ ANCI is a passive description of code structure. It contains:
329
+
330
+ * File paths (project-relative).
331
+ * Domain names.
332
+ * Route paths and HTTP methods.
333
+ * Model names and kinds.
334
+ * Import graph as integer bitmaps.
335
+
336
+ ANCI does **not** contain source code, secrets, env values, or
337
+ user-identifying information. Producers MUST NOT include source content
338
+ in any field.
339
+
340
+ A repository that publishes its `.carto/anci.{yaml,bin}` to a remote
341
+ location is exposing its architecture. This is the same disclosure
342
+ level as making the directory tree public. Producers SHOULD respect
343
+ `.cartoignore` and SHOULD NOT include paths matching its patterns.
344
+
345
+ ## 10. Why YAML + binary (and not just YAML, or just binary)
346
+
347
+ | Property | YAML alone | Binary alone | Hybrid |
348
+ |------------------------|:----------:|:------------:|:------:|
349
+ | Human-readable | ✅ | ❌ | ✅ |
350
+ | Grep-able from CI | ✅ | ❌ | ✅ |
351
+ | Compact at 100K files | ❌ (50–500 MB) | ✅ | ✅ (5–20 MB) |
352
+ | Fast graph queries | ❌ | ✅ | ✅ |
353
+ | Easy partial parse | ✅ | ⚠️ | ✅ |
354
+
355
+ The split mirrors the structure of the data: metadata is small and
356
+ varied (good for YAML); the graph is large and uniform (good for
357
+ binary).
358
+
359
+ ## 11. Why these specific bitmap sections
360
+
361
+ The body ships exactly the data needed to answer the four MCP tools
362
+ that drive most AI-tool queries:
363
+
364
+ * `getBlastRadius(file)` → walk `reverse` BFS-style.
365
+ * `simulateChangeImpact(files)` → OR-aggregate `reverse` bitmaps.
366
+ * `getHighImpactFiles(n)` → take first N from `popcount`.
367
+ * `getDomainsList()` / domain membership → header + `file_domain` +
368
+ `domain_names`.
369
+
370
+ Everything else (cross-domain detection, similarity, change-plan
371
+ ranking) can be derived from these primitives. The format keeps the
372
+ ship surface small.
373
+
374
+ ## 12. Compatibility with carto-md `bitmap.bin`
375
+
376
+ ANCI v0.1 and carto-md's internal `bitmap.bin` (magic `0x54524243`
377
+ "CBRT") share record-level encoding but are **distinct file formats
378
+ with distinct magics**. Reasons:
379
+
380
+ * `bitmap.bin` is private cache; `anci.bin` is public export.
381
+ * `bitmap.bin` includes `crossForward` and `domainBitmaps` (internal
382
+ query optimizations); ANCI omits them. Consumers can re-derive both
383
+ in O(N) on load.
384
+ * `bitmap.bin` may evolve at carto-md's pace; ANCI evolves only at
385
+ spec-revision pace.
386
+
387
+ A consumer MUST NOT attempt to load `bitmap.bin` as ANCI or vice versa.
388
+
389
+ ## 13. Reference vs. interoperability tests
390
+
391
+ The carto-md reference implementation includes a roundtrip test suite
392
+ (`test/test.js` → `ANCI roundtrip`). Other implementations are
393
+ encouraged to:
394
+
395
+ * Run carto-md's reference suite against their own
396
+ serialize/deserialize code.
397
+ * Publish their own conformance tests for the spec to consume.
398
+
399
+ The DRAFT spec deliberately ships **without** an independent
400
+ conformance harness. v1.0 will require one before formalization.
401
+
402
+ ## 14. Open questions (RFC-stage feedback welcome)
403
+
404
+ * **Should the body be Roaring-compressed instead of dense
405
+ Uint32Array?** Roaring is smaller on sparse repos but adds a
406
+ dependency. v0.1 ships dense; v1.0 may add an opt-in compressed
407
+ encoding.
408
+ * **Should env vars be in the header?** Currently no — they leak
409
+ configuration surface. Open question for partners.
410
+ * **Should source language be part of `paths`?** Currently no — language
411
+ is derivable from extension. Open if consumers ask.
412
+ * **Cross-repo references?** Out of scope for v0.1. Tier 3 work.
413
+
414
+ ## 15. Acknowledgements
415
+
416
+ Inspired by OpenAPI's standardization play (2010-2015), Git's pack
417
+ reachability bitmaps (2013), and Lucene's Roaring posting lists (2014).
418
+
419
+ The spec is iterated alongside carto-md. File issues at
420
+ https://github.com/theanshsonkar/carto for proposed changes.