carto-md 2.0.8 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (182) hide show
  1. package/CONTRIBUTING.md +19 -15
  2. package/README.md +203 -286
  3. package/docs/anci/v0.1-DRAFT.md +420 -0
  4. package/docs/api/README.md +99 -0
  5. package/docs/api/did_we_discuss_this.md +34 -0
  6. package/docs/api/dismiss_suggestion.md +34 -0
  7. package/docs/api/explain_change_in_natural_language.md +33 -0
  8. package/docs/api/find_consumers_of_api.md +34 -0
  9. package/docs/api/get_action_patterns.md +32 -0
  10. package/docs/api/get_active_drift.md +32 -0
  11. package/docs/api/get_active_suggestions.md +25 -0
  12. package/docs/api/get_ai_cost_attribution.md +32 -0
  13. package/docs/api/get_arch_events.md +42 -0
  14. package/docs/api/get_architectural_drift.md +37 -0
  15. package/docs/api/get_architecture.md +25 -0
  16. package/docs/api/get_blast_radius.md +34 -0
  17. package/docs/api/get_canonical_pattern.md +39 -0
  18. package/docs/api/get_change_plan.md +34 -0
  19. package/docs/api/get_change_velocity.md +32 -0
  20. package/docs/api/get_churn_vs_blast_radius.md +32 -0
  21. package/docs/api/get_complexity_trend.md +39 -0
  22. package/docs/api/get_context.md +34 -0
  23. package/docs/api/get_conventions.md +32 -0
  24. package/docs/api/get_cross_domain.md +25 -0
  25. package/docs/api/get_cross_language_call_graph.md +25 -0
  26. package/docs/api/get_cross_repo_blast_radius.md +34 -0
  27. package/docs/api/get_cross_team_coupling.md +25 -0
  28. package/docs/api/get_data_flow.md +33 -0
  29. package/docs/api/get_dead_code_with_confidence.md +32 -0
  30. package/docs/api/get_decision_log.md +32 -0
  31. package/docs/api/get_dependency_surface.md +25 -0
  32. package/docs/api/get_domain.md +34 -0
  33. package/docs/api/get_domain_evolution.md +39 -0
  34. package/docs/api/get_domain_health.md +32 -0
  35. package/docs/api/get_domains_list.md +25 -0
  36. package/docs/api/get_drift_digest.md +32 -0
  37. package/docs/api/get_env_vars.md +32 -0
  38. package/docs/api/get_evolution_delta.md +36 -0
  39. package/docs/api/get_file_ownership.md +33 -0
  40. package/docs/api/get_file_summary.md +34 -0
  41. package/docs/api/get_high_impact_files.md +32 -0
  42. package/docs/api/get_hot_in_prod_no_tests.md +34 -0
  43. package/docs/api/get_hotspot_files.md +37 -0
  44. package/docs/api/get_iac_resources.md +25 -0
  45. package/docs/api/get_interface_contract.md +33 -0
  46. package/docs/api/get_intervention_history.md +32 -0
  47. package/docs/api/get_invariants.md +37 -0
  48. package/docs/api/get_llm_enrichment.md +33 -0
  49. package/docs/api/get_microservice_cut_points.md +32 -0
  50. package/docs/api/get_microservices_migration_cut_points.md +25 -0
  51. package/docs/api/get_minimal_context_for_intent.md +39 -0
  52. package/docs/api/get_models.md +32 -0
  53. package/docs/api/get_neighbors.md +39 -0
  54. package/docs/api/get_org_architecture.md +25 -0
  55. package/docs/api/get_org_domain_mapping.md +25 -0
  56. package/docs/api/get_pending_decisions.md +32 -0
  57. package/docs/api/get_predictive_risk.md +32 -0
  58. package/docs/api/get_progressive_disclosure_tree.md +25 -0
  59. package/docs/api/get_recent_decisions.md +37 -0
  60. package/docs/api/get_risk_weighted_blast_radius.md +32 -0
  61. package/docs/api/get_routes.md +25 -0
  62. package/docs/api/get_safety_checklist.md +33 -0
  63. package/docs/api/get_semantic_diff.md +34 -0
  64. package/docs/api/get_service_boundary_violations.md +25 -0
  65. package/docs/api/get_service_dependency_graph.md +25 -0
  66. package/docs/api/get_session_context.md +32 -0
  67. package/docs/api/get_similar_patterns.md +39 -0
  68. package/docs/api/get_stale_docs.md +25 -0
  69. package/docs/api/get_structure.md +25 -0
  70. package/docs/api/get_temporal_context.md +34 -0
  71. package/docs/api/get_test_coverage_map.md +25 -0
  72. package/docs/api/get_token_budget_report.md +36 -0
  73. package/docs/api/get_upgrade_risk.md +25 -0
  74. package/docs/api/get_working_memory.md +25 -0
  75. package/docs/api/ingest_otlp_traces.md +34 -0
  76. package/docs/api/scaffold_for_intent.md +34 -0
  77. package/docs/api/search_routes.md +34 -0
  78. package/docs/api/simulate_change_impact.md +37 -0
  79. package/docs/api/validate_change.md +39 -0
  80. package/docs/api/validate_diff.md +39 -0
  81. package/docs/concepts/anci.md +87 -0
  82. package/docs/concepts/blast-radius.md +66 -0
  83. package/docs/concepts/domains.md +91 -0
  84. package/docs/concepts/import-graph.md +102 -0
  85. package/docs/concepts/mcp-integration.md +148 -0
  86. package/docs/guides/adding-feature-safely.md +101 -0
  87. package/docs/guides/ci-integration.md +175 -0
  88. package/docs/guides/monorepo-setup.md +121 -0
  89. package/docs/guides/onboarding-new-engineer.md +121 -0
  90. package/docs/guides/pre-merge-review.md +139 -0
  91. package/docs/migration/v1-to-v2.md +110 -0
  92. package/docs/quickstart.md +95 -0
  93. package/docs/scale.md +129 -0
  94. package/docs/troubleshooting.md +180 -0
  95. package/package.json +12 -5
  96. package/scripts/gen-api-docs.js +170 -0
  97. package/scripts/postinstall.js +391 -24
  98. package/src/acp/agent.js +83 -11
  99. package/src/acp/config.js +64 -0
  100. package/src/acp/persistence.js +146 -0
  101. package/src/acp/providers/anthropic.js +179 -27
  102. package/src/acp/providers/index.js +15 -2
  103. package/src/acp/providers/openai.js +164 -38
  104. package/src/acp/providers/sse.js +82 -0
  105. package/src/acp/safety.js +128 -0
  106. package/src/acp/session.js +73 -0
  107. package/src/adjacent/call-graph.js +170 -0
  108. package/src/adjacent/iac.js +167 -0
  109. package/src/adjacent/llm-enrich.js +35 -0
  110. package/src/adjacent/runtime.js +216 -0
  111. package/src/adjacent/semantic-diff.js +143 -0
  112. package/src/agents/leiden.js +4 -4
  113. package/src/agents/scan-structure.js +2 -2
  114. package/src/ai/context-builder.js +215 -0
  115. package/src/ai/retrieval/lexical.js +122 -0
  116. package/src/ai/retrieval/rrf.js +121 -0
  117. package/src/ai/retrieval/semantic.js +35 -0
  118. package/src/ai/retrieval/structural.js +82 -0
  119. package/src/ai/tools.js +423 -0
  120. package/src/anci/consumer.js +305 -0
  121. package/src/anci/deserialize.js +160 -0
  122. package/src/anci/emit.js +85 -0
  123. package/src/anci/serialize.js +264 -0
  124. package/src/anci/yaml.js +401 -0
  125. package/src/bitmap/index.js +1 -1
  126. package/src/bitmap/tools.js +2 -2
  127. package/src/brain/conventions/index.js +185 -0
  128. package/src/brain/index.js +31 -0
  129. package/src/brain/invariants/index.js +252 -0
  130. package/src/brain/procedural/index.js +181 -0
  131. package/src/brain/suggestions/index.js +153 -0
  132. package/src/brain/working/index.js +170 -0
  133. package/src/cli/anci.js +237 -0
  134. package/src/cli/check.js +47 -1
  135. package/src/cli/diff.js +83 -0
  136. package/src/cli/doctor.js +270 -0
  137. package/src/cli/explain.js +61 -0
  138. package/src/cli/index.js +115 -0
  139. package/src/cli/init.js +144 -6
  140. package/src/cli/inspect.js +1 -1
  141. package/src/cli/org.js +172 -0
  142. package/src/cli/pr-impact.js +554 -0
  143. package/src/cli/serve.js +1 -1
  144. package/src/cli/status.js +211 -0
  145. package/src/cli/sync.js +2 -2
  146. package/src/cli/temporal.js +188 -0
  147. package/src/cli/validate.js +201 -0
  148. package/src/cli/watch.js +4 -4
  149. package/src/cli/why.js +101 -0
  150. package/src/extractors/frameworks.js +236 -0
  151. package/src/extractors/languages/dart.js +138 -0
  152. package/src/extractors/languages/go.js +11 -1
  153. package/src/extractors/languages/javascript.js +16 -6
  154. package/src/extractors/languages/kotlin.js +169 -0
  155. package/src/extractors/languages/php.js +195 -0
  156. package/src/extractors/languages/python.js +12 -1
  157. package/src/extractors/languages/swift.js +140 -0
  158. package/src/extractors/languages/typescript.js +15 -2
  159. package/src/extractors/plugin-api.js +102 -0
  160. package/src/mcp/change-plan.js +8 -8
  161. package/src/mcp/files-without-tests.js +285 -0
  162. package/src/mcp/middleware/index.js +451 -0
  163. package/src/mcp/server.js +2292 -0
  164. package/src/mcp/validate.js +1 -1
  165. package/src/org/detect.js +262 -0
  166. package/src/org/queries.js +144 -0
  167. package/src/org/store.js +173 -0
  168. package/src/org/sync.js +106 -0
  169. package/src/predictive/cut-points.js +83 -0
  170. package/src/predictive/drift-digest.js +88 -0
  171. package/src/predictive/ownership.js +145 -0
  172. package/src/predictive/risk-score.js +121 -0
  173. package/src/predictive/validate-change.js +55 -0
  174. package/src/store/store-adapter.js +3 -3
  175. package/src/store/{sync-v2.js → sync.js} +105 -16
  176. package/src/temporal/backfill.js +211 -0
  177. package/src/temporal/delta.js +85 -0
  178. package/src/temporal/events.js +180 -0
  179. package/src/temporal/queries.js +358 -0
  180. package/src/temporal/snapshot.js +151 -0
  181. package/src/temporal/store.js +400 -0
  182. package/src/mcp/server-v2.js +0 -986
@@ -0,0 +1,148 @@
1
+ # Concept: MCP Integration
2
+
3
+ > The protocol layer that makes Carto's index visible to your AI tool.
4
+
5
+ ## What MCP is
6
+
7
+ The [Model Context Protocol](https://modelcontextprotocol.io) is an open protocol Anthropic introduced in late 2024 for connecting LLMs to external tools and data. An MCP **server** exposes tools and resources; an MCP **client** (Cursor, Claude Desktop, Kiro, Claude Code, Cline, Continue, Windsurf, Zed) calls them on behalf of the model.
8
+
9
+ Carto runs an MCP server in stdio mode. You start it with `carto serve`, but you almost never do that by hand — your AI tool spawns it.
10
+
11
+ ## What `carto init` writes
12
+
13
+ For each AI tool it detects on your machine, init writes the appropriate MCP config:
14
+
15
+ | Tool | Config file | Format |
16
+ |------|-------------|--------|
17
+ | Cursor | `~/.cursor/mcp.json` | `{ mcpServers: { carto: ... } }` |
18
+ | Claude Code | `<project>/.mcp.json` | same |
19
+ | Claude Desktop | `~/Library/Application Support/Claude/claude_desktop_config.json` | same (cross-platform paths in README) |
20
+ | Kiro | `~/.kiro/settings/mcp.json` | same |
21
+ | Codex CLI | `~/.codex/config.toml` | `[mcp_servers.carto]` |
22
+ | VS Code Copilot | `<project>/.vscode/mcp.json` | `{ servers: { carto: { type: "stdio", ... } } }` |
23
+ | Windsurf | `~/.codeium/windsurf/mcp_config.json` | `{ mcpServers: { carto: ... } }` |
24
+
25
+ Each config tells the tool: *"when the user starts a chat, run `carto serve` as a child process and route MCP traffic to it"*.
26
+
27
+ ## The ~76 tools Carto exposes
28
+
29
+ Grouped by what they answer. Categories below match the [`docs/api/`](../api/) layout — that directory is the full per-tool reference (auto-generated from `src/mcp/server.js`). The picks below are the high-impact ones; your AI tool picks whichever one fits the question mid-task.
30
+
31
+ ### Core graph (16) — project shape, blast radius, change planning
32
+
33
+ - **`get_architecture()`** — 500-word overview. The right *first* call for any new task.
34
+ - **`get_blast_radius(file)`** — every file affected by changing this one, with hop distance.
35
+ - **`get_change_plan(intent)`** — natural-language intent → files to touch, affected domains, similar patterns.
36
+ - **`simulate_change_impact(files)`** — union of the blast radius for *multiple* files at once. The bitmap engine is what makes this feasible at scale.
37
+ - **`get_high_impact_files(n)`** — top N files by transitive dependents.
38
+ - **`get_cross_domain()`** — every import edge that crosses a domain boundary.
39
+
40
+ Plus `get_structure`, `get_context`, `get_neighbors`, `get_domain`, `get_domains_list`, `get_routes`, `search_routes`, `get_models`, `get_env_vars`, `get_file_summary`, `get_similar_patterns`.
41
+
42
+ ### Episodic memory (5) — what your AI decided last week
43
+
44
+ - **`validate_diff(diff)`** — given a unified diff: violations, blast radius, risk grade, suggestions. Sub-15ms p99. **Every call writes a row to the decision log** — the rest of this category reads from it.
45
+ - **`did_we_discuss_this(topic)`** — substring search over the decision log. Six-week recall, conversational.
46
+ - **`get_recent_decisions(time_range, kind?)`**, **`get_session_context(session_id?)`**, **`get_intervention_history(file?)`** — the underlying log surfaced different ways.
47
+
48
+ ### Temporal (9) — how the architecture changed over time
49
+
50
+ - **`get_architectural_drift()`** — per-domain growth/shrink + event count over a window.
51
+ - **`get_hotspot_files()`** — top files by churn × blast_radius (the CodeHealth heuristic).
52
+ - **`get_domain_evolution(domain)`** — time-series of one domain's file count by snapshot.
53
+
54
+ Plus `get_arch_events`, `get_temporal_context`, `get_change_velocity`, `get_complexity_trend`, `get_churn_vs_blast_radius`, `get_domain_health`.
55
+
56
+ ### Brain (10) — invariants, conventions, action patterns
57
+
58
+ Carto's "rules of the codebase" — mined from the import graph and git history, not written by hand.
59
+
60
+ - **`get_invariants(domain?, threshold?)`** — confidence-scored architectural rules: *"AUTH never imports from PAYMENTS"*.
61
+ - **`get_conventions(file?)`** — naming, export, and directory conventions that apply to a given location.
62
+ - **`get_action_patterns(intent?)`** — *"when developers add X, they also touch Y"* — procedural patterns from git history.
63
+ - **`scaffold_for_intent(intent)`** — anchor file + co-changed files + canonical pattern + conventions.
64
+
65
+ Plus `get_canonical_pattern`, `get_working_memory`, `get_active_suggestions`, `get_pending_decisions`, `get_active_drift`, `dismiss_suggestion`.
66
+
67
+ ### Predictive (7) — P(file causes the next bug)
68
+
69
+ - **`get_predictive_risk(file?)`** — 0–1 score per file combining blast radius, churn, cross-domain coupling, prior interventions, and test coverage.
70
+ - **`get_safety_checklist(file)`** — per-file pre-edit safety summary.
71
+ - **`validate_change(file, content)`** — pre-write governance: diffs `content` vs disk then runs `validate_diff`.
72
+ - **`get_drift_digest(time_range)`** — weekly architectural digest in CLI-renderable markdown.
73
+
74
+ Plus `get_microservice_cut_points`, `get_file_ownership`, `get_cross_team_coupling`, `get_ai_cost_attribution`.
75
+
76
+ ### AI-native retrieval (14) — context that fits the budget
77
+
78
+ - **`get_minimal_context_for_intent(intent, budget_tokens)`** — token-budgeted hybrid retrieval (structural + lexical + semantic, fused with RRF, boosted by same-domain and recent-churn). Usually returns 6–12 files instead of the usual 40+.
79
+ - **`get_progressive_disclosure_tree()`** — domain → top files → exports, structured for an LLM to drill into.
80
+ - **`get_token_budget_report(intent, budget)`** — diagnostic complement: which files would be picked and at what cost.
81
+
82
+ Plus `get_data_flow`, `get_interface_contract`, `get_dependency_surface`, `get_upgrade_risk`, `get_test_coverage_map`, `get_stale_docs`, `get_decision_log`, `get_evolution_delta`, `get_change_velocity`, `explain_change_in_natural_language`.
83
+
84
+ ### Adjacent (8) — runtime, IaC, cross-language, dead code
85
+
86
+ - **`get_cross_language_call_graph()`** — match frontend `fetch()` to backend route handlers across language boundaries.
87
+ - **`get_iac_resources()`** — Terraform / Helm / Pulumi / CDK resources discovered in the repo.
88
+ - **`ingest_otlp_traces(path)`** + **`get_risk_weighted_blast_radius(otlp?)`** — combine static dependents with runtime call counts.
89
+ - **`get_dead_code_with_confidence(otlp?)`** — zero static dependents AND zero runtime hits → the safe-to-delete list.
90
+ - **`get_semantic_diff(diff)`** — rename + symbol-relocation detection beyond line-level.
91
+
92
+ Plus `get_hot_in_prod_no_tests`, `get_llm_enrichment`.
93
+
94
+ ### Org / multi-repo (7) — service graphs across repos
95
+
96
+ - **`get_org_architecture()`** — org-wide summary across registered repos.
97
+ - **`get_cross_repo_blast_radius(repo)`** — *"if I break repo X, who notices?"*
98
+ - **`find_consumers_of_api(target)`** — every file importing a given npm/pypi/go/maven package across the org.
99
+ - **`get_service_dependency_graph()`**, **`get_service_boundary_violations()`** — service-shape views.
100
+
101
+ Plus `get_org_domain_mapping`, `get_microservices_migration_cut_points`.
102
+
103
+ > Full per-tool API reference at [`docs/api/`](../api/). You don't memorize these — your AI picks the right one mid-task.
104
+
105
+ ## How the AI knows when to call which tool
106
+
107
+ The model reads the tool descriptions Carto provides in the `tools/list` response. Each description is one sentence written so the model has enough context to choose. Examples:
108
+
109
+ > *"Get a 500-word project overview: domains, entry points, stack, key patterns. **Use this first.**"*
110
+
111
+ > *"Given a unified diff: violations (cross-domain imports, high-blast files), blast radius per file, risk level (SAFE/LOW/MEDIUM/HIGH), suggestions. Each call is recorded in the episodic memory log so other tools can ask 'did we discuss this?'. Sub-15ms p99."*
112
+
113
+ The "Use this first" hint matters. The model orients itself with `get_architecture()`, then narrows to specific tools based on what the user asked.
114
+
115
+ ## Lazy re-parse on stale files
116
+
117
+ When the model asks about `src/lib/db.ts` and you've edited it since the last `carto sync`, the MCP server stats the file, sees the mtime is newer than the DB row, and re-parses it inline (~50 ms). The model gets a fresh answer; you didn't think about freshness at all.
118
+
119
+ That's why Carto doesn't need a daemon. Git hooks handle the "I just committed" case; lazy re-parse handles the "I edited but didn't commit yet" case. Together they cover ~100% of real usage.
120
+
121
+ ## Wiring it manually
122
+
123
+ If `carto init` didn't detect your AI tool, the README's [Manual MCP wiring](../../README.md) section (collapsed `<details>` block right after the install snippet) has copy-paste snippets for every supported client. The shape is always:
124
+
125
+ ```json
126
+ {
127
+ "mcpServers": {
128
+ "carto": {
129
+ "command": "carto",
130
+ "args": ["serve"],
131
+ "cwd": "/path/to/your/project"
132
+ }
133
+ }
134
+ }
135
+ ```
136
+
137
+ Restart the tool, and it'll show carto in its MCP server list.
138
+
139
+ ## Debugging
140
+
141
+ - `carto doctor` runs a sanity check on Node version, native modules, the index, git hooks, and the *presence* of MCP config files (it doesn't validate their content — that varies per tool).
142
+ - Set `CARTO_DEBUG=1` before launching your AI tool to make the MCP server log every tool call to stderr.
143
+ - Read [`troubleshooting.md`](../troubleshooting.md) for known gotchas.
144
+
145
+ ## Related
146
+
147
+ - [Model Context Protocol spec](https://modelcontextprotocol.io)
148
+ - [`anci.md`](./anci.md) — the file-format Carto exports for tools that prefer reading a static file over speaking MCP
@@ -0,0 +1,101 @@
1
+ # Guide: Adding a feature safely
2
+
3
+ > The workflow Carto makes possible. Add a feature without breaking five other files you didn't know existed.
4
+
5
+ ## The before-Carto pattern
6
+
7
+ You're asked to *"add rate limiting to /api/users"*. You ask Cursor. Cursor:
8
+
9
+ 1. Greps for `users` — finds 47 matches across 12 files
10
+ 2. Picks the first one that looks like a route handler
11
+ 3. Adds rate-limiting middleware to it
12
+ 4. Confidently announces success
13
+
14
+ What got missed:
15
+
16
+ - The middleware was already declared in `src/auth/middleware.ts` — now you have two copies
17
+ - The route handler is one of three for `/api/users` (GET, POST, PATCH) — the AI updated GET only
18
+ - Three callers in tests still pass requests through the un-rate-limited path
19
+
20
+ You merge it. CI passes (tests didn't cover the new middleware). It ships. The next time someone hits POST /api/users from production they're not rate-limited and the AbusePrevention engineer wakes up at 2 AM.
21
+
22
+ ## The with-Carto pattern
23
+
24
+ Same task. Open your AI tool — which is now wired to carto MCP — and ask the same question.
25
+
26
+ The AI calls `get_change_plan("add rate limiting to /api/users")`. Carto's response:
27
+
28
+ ```
29
+ ## Plan: add rate limiting to /api/users
30
+
31
+ ### Relevant routes
32
+ - POST /api/users src/api/users.ts:42 (no rate limit)
33
+ - GET /api/users src/api/users.ts:18 (no rate limit)
34
+ - PATCH /api/users src/api/users.ts:67 (no rate limit)
35
+
36
+ ### Files to touch
37
+ - src/api/users.ts — primary (modify 3 route handlers)
38
+ - src/auth/middleware.ts — rate-limit middleware already exists here, reuse it
39
+
40
+ ### Similar patterns
41
+ - src/api/posts.ts — already uses rateLimit() middleware (12 callers)
42
+ - src/api/billing.ts — already uses rateLimit() middleware (4 callers)
43
+
44
+ ### Blast radius
45
+ - src/api/users.ts has 8 direct dependents
46
+ - src/auth/middleware.ts has 23 transitive dependents — changes here
47
+ ripple wide; do not modify the middleware signature
48
+
49
+ ### Cross-domain check
50
+ - No new cross-domain edges introduced by reusing existing middleware
51
+ ```
52
+
53
+ The AI now has every fact it needs. It writes the change correctly the first time.
54
+
55
+ ## The literal step-by-step
56
+
57
+ ```bash
58
+ # 1. Ask carto first
59
+ carto explain "add rate limiting to /api/users"
60
+ # (or do this implicitly via your AI tool — same call under the hood)
61
+
62
+ # 2. Read the existing pattern
63
+ carto why src/api/posts.ts # see how rate limiting is already used
64
+
65
+ # 3. Make the change in your AI tool
66
+ # (the AI references the plan from step 1)
67
+
68
+ # 4. Before committing, validate
69
+ git diff | carto validate --fail-on HIGH
70
+ # Exits 2 if the diff would be HIGH-risk → blocks the commit
71
+
72
+ # 5. Final check
73
+ carto diff
74
+ # Markdown impact report against HEAD
75
+ ```
76
+
77
+ If you're using the [GitHub Action](./ci-integration.md), step 4 happens automatically on the PR. The local `carto validate` step is for catching issues earlier.
78
+
79
+ ## What "safely" actually means
80
+
81
+ Three signals Carto surfaces that you wouldn't see otherwise:
82
+
83
+ 1. **The right files.** `get_change_plan` ranks files by relevance to intent (IDF token scoring + route anchoring + graph expansion). The model's hit rate jumps from "pick the most plausible from 47 grep hits" to "use the 2–3 files Carto identified".
84
+
85
+ 2. **Existing patterns.** `get_similar_patterns` finds files that already solve the same shape of problem in your codebase. The model copies your conventions instead of inventing new ones.
86
+
87
+ 3. **The boundary it's about to cross.** If reusing the middleware works, no new cross-domain edge. If the AI tries to import auth utilities into the payments domain, `validate_diff` catches it before save.
88
+
89
+ ## When to override
90
+
91
+ Carto's heuristics are statistical. Sometimes the right answer involves crossing a domain boundary, or modifying a high-blast-radius file. Don't refuse — *acknowledge*.
92
+
93
+ The MCP `did_we_discuss_this("auth-to-payments coupling")` tool lets the AI check whether the decision has already been made. If yes, it proceeds with cited context. If no, it asks you, then records the decision so future sessions don't re-litigate.
94
+
95
+ This is the episodic-memory layer in action: not a gate, but a continuity.
96
+
97
+ ## Related
98
+
99
+ - [`docs/concepts/blast-radius.md`](../concepts/blast-radius.md)
100
+ - [`docs/concepts/domains.md`](../concepts/domains.md) (the cross-domain check)
101
+ - [`docs/guides/pre-merge-review.md`](./pre-merge-review.md) (the CI-gated version)
@@ -0,0 +1,175 @@
1
+ # Guide: CI integration
2
+
3
+ > Where carto fits into your CI pipeline, beyond the GitHub Action.
4
+
5
+ ## The default — GitHub Action
6
+
7
+ [`docs/guides/pre-merge-review.md`](./pre-merge-review.md) covers the canonical case: drop the Action onto every PR, get sticky-comment impact reports automatically.
8
+
9
+ This guide is for the cases where the Action isn't the right fit: GitLab, Bitbucket, custom CI, pre-commit, monorepo-specific gates.
10
+
11
+ ## GitLab CI
12
+
13
+ ```yaml
14
+ # .gitlab-ci.yml
15
+ carto-impact:
16
+ stage: test
17
+ image: node:20
18
+ rules:
19
+ - if: $CI_PIPELINE_SOURCE == "merge_request_event"
20
+ cache:
21
+ key: carto-${CI_COMMIT_REF_SLUG}
22
+ paths:
23
+ - .carto/
24
+ script:
25
+ - npm install -g carto-md
26
+ - test -d .carto || carto init
27
+ - carto sync
28
+ - carto pr-impact --base origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME --head HEAD --format markdown > impact.md
29
+ - cat impact.md
30
+ artifacts:
31
+ paths: [impact.md]
32
+ ```
33
+
34
+ Posting to the merge-request as a comment requires `glab`:
35
+
36
+ ```yaml
37
+ - glab mr note ${CI_MERGE_REQUEST_IID} -F impact.md
38
+ ```
39
+
40
+ ## Bitbucket Pipelines
41
+
42
+ ```yaml
43
+ # bitbucket-pipelines.yml
44
+ pipelines:
45
+ pull-requests:
46
+ '**':
47
+ - step:
48
+ name: Carto Impact
49
+ image: node:20
50
+ script:
51
+ - npm install -g carto-md
52
+ - carto init || carto sync
53
+ - carto pr-impact --base $BITBUCKET_PR_DESTINATION_BRANCH --head HEAD --fail-on HIGH
54
+ ```
55
+
56
+ ## CircleCI
57
+
58
+ ```yaml
59
+ # .circleci/config.yml
60
+ version: 2.1
61
+ jobs:
62
+ carto-impact:
63
+ docker:
64
+ - image: cimg/node:20.10
65
+ steps:
66
+ - checkout
67
+ - restore_cache:
68
+ keys: [carto-{{ .Branch }}, carto-]
69
+ - run: npm install -g carto-md
70
+ - run: test -d .carto || carto init
71
+ - run: carto sync
72
+ - run: carto pr-impact --base origin/main --head HEAD --format markdown
73
+ - save_cache:
74
+ key: carto-{{ .Branch }}
75
+ paths: [.carto/]
76
+ ```
77
+
78
+ ## Custom CI / shell script
79
+
80
+ The whole thing is just three commands. Drop them into whatever runner you use:
81
+
82
+ ```bash
83
+ #!/usr/bin/env bash
84
+ set -euo pipefail
85
+
86
+ npm install -g carto-md
87
+
88
+ if [ ! -d .carto ]; then
89
+ carto init
90
+ else
91
+ carto sync
92
+ fi
93
+
94
+ carto pr-impact \
95
+ --base "${BASE_REF:-origin/main}" \
96
+ --head HEAD \
97
+ --format markdown \
98
+ --fail-on HIGH
99
+ ```
100
+
101
+ Exit codes:
102
+
103
+ - 0 — comment rendered, risk below threshold
104
+ - 1 — misuse / index missing / git failure
105
+ - 2 — `--fail-on` threshold tripped
106
+
107
+ ## Pre-commit hook
108
+
109
+ For "fail fast" before pushing. Add to `.git/hooks/pre-commit`:
110
+
111
+ ```bash
112
+ #!/bin/sh
113
+ git diff --cached | carto validate --fail-on HIGH
114
+ ```
115
+
116
+ Now a `git commit` that would produce a HIGH-risk change exits 2 and the commit is blocked. The dev sees the violation reasons in the same terminal.
117
+
118
+ Note: `carto init`'s installed pre-commit hook calls `carto sync`, not `carto validate`. If you want both, prepend the validate line:
119
+
120
+ ```bash
121
+ #!/bin/sh
122
+ # Spec 22: validate the staged diff first
123
+ git diff --cached | carto validate --fail-on HIGH || exit 2
124
+
125
+ # carto-md: keep index fresh on git events
126
+ carto sync >/dev/null 2>&1 || true
127
+ ```
128
+
129
+ ## Caching tips
130
+
131
+ Re-running `carto init` from scratch is fast but not free (~10 seconds for a medium repo). On CI, cache `.carto/`:
132
+
133
+ ```yaml
134
+ # GitHub Actions
135
+ - uses: actions/cache@v4
136
+ with:
137
+ path: .carto/
138
+ key: carto-${{ hashFiles('**/*.ts', '**/*.js', '**/*.py', '**/*.go') }}
139
+ restore-keys: carto-
140
+ ```
141
+
142
+ The cache key hashes source files — any change invalidates the cache, but the source-file hash *captures* what changed, so reuse is high across PRs.
143
+
144
+ After cache restore, run `carto sync` (not `init`). Sync is incremental — mtime+size cached, only changed files re-parsed.
145
+
146
+ ## Slack notifications (drift digest)
147
+
148
+ Not built in yet — but easy to wire from CI. Run `carto check --json` on a weekly cron, post the cross-domain section to Slack:
149
+
150
+ ```bash
151
+ carto check --json | jq '.crossDomain | length' \
152
+ | xargs -I {} curl -X POST -H 'Content-Type: application/json' \
153
+ --data "{\"text\": \"This week: {} cross-domain edges in repo X\"}" \
154
+ $SLACK_WEBHOOK_URL
155
+ ```
156
+
157
+ ## Don't gate everything
158
+
159
+ A few patterns we've learned not to recommend:
160
+
161
+ - **Gating *every* PR on `--fail-on LOW`** — too noisy. LOW is just "a file changed" — every PR trips it.
162
+ - **Running carto on `push` events** — wastes runner minutes; the impact report is only useful in PR context where there's a base branch to diff against.
163
+ - **Running on docs-only changes** — `path-filters` in your workflow to skip MD-only PRs saves minutes:
164
+
165
+ ```yaml
166
+ on:
167
+ pull_request:
168
+ paths-ignore: ['**/*.md', 'docs/**']
169
+ ```
170
+
171
+ ## Related
172
+
173
+ - [`docs/guides/pre-merge-review.md`](./pre-merge-review.md) — the canonical GitHub Action setup
174
+ - [`docs/guides/monorepo-setup.md`](./monorepo-setup.md) — caching considerations for large repos
175
+ - [`docs/troubleshooting.md`](../troubleshooting.md) — what to do when CI fails
@@ -0,0 +1,121 @@
1
+ # Guide: Monorepo setup
2
+
3
+ > The realities of indexing a 30K-file monorepo with carto. Knobs, gotchas, expected sizes.
4
+
5
+ ## The default behavior
6
+
7
+ `carto init` at the monorepo root indexes *everything*. No file cap. SQLite handles the volume; the bitmap engine handles the queries.
8
+
9
+ Real numbers from the corpus runs (see [`docs/scale.md`](../scale.md)):
10
+
11
+ | Repo | Indexed files | First run | Re-sync (no changes) | DB size |
12
+ |------|--------------:|----------:|---------------------:|--------:|
13
+ | prisma | 961 | 1.0s | 350ms | 1.1 MB |
14
+ | zed | 1,752 | 2.9s | 468ms | 4.8 MB |
15
+ | supabase | 6,330 | 5.4s | 1.2s | 4.8 MB |
16
+ | vscode | 7,567 | 8.0s | 935ms | 14.3 MB |
17
+
18
+ For a 30–50K-file monorepo, expect ~30–50s first run, ~3–5s incremental. Beyond 100K files Carto prints an ETA and runs in 1–2 minutes.
19
+
20
+ ## What gets excluded by default
21
+
22
+ `carto init` writes a `.cartoignore` if one doesn't exist. Defaults exclude:
23
+
24
+ - `node_modules/`, `vendor/`, `dist/`, `build/`, `target/`, `out/`
25
+ - `.git/`, `.svn/`, `.hg/`
26
+ - Test files matching common conventions:
27
+ - `*.test.*`, `*.spec.*`, `*.stories.*` (JS/TS)
28
+ - `test_*.py`, `*_test.py` (Python)
29
+ - `test_*`, `*_test.r` (R)
30
+ - Subdirs named `test/`, `tests/`, `__tests__/`, `spec/`
31
+ - Secrets / credentials: `.env*`, `*secret*`, `*credential*`, `*private_key*`, `*.pem`, `*.key`, SSH keys, AWS/GCP creds, kubeconfig, token files
32
+ - Binaries: images, media, archives
33
+ - Cache directories
34
+
35
+ If your monorepo has unusual layout (a separately-managed `vendor/` you do want indexed, e.g.), edit `.cartoignore` to allow it back in.
36
+
37
+ ## Submodules
38
+
39
+ If `.gitmodules` is present, `carto init` warns:
40
+
41
+ ```
42
+ [CARTO] Detected 3 git submodules. Add their paths to .cartoignore
43
+ if you don't want them indexed.
44
+ ```
45
+
46
+ Default behavior: submodule directories *are* descended into. If your submodules are external/vendored packages, you almost certainly want them excluded — add their paths to `.cartoignore`.
47
+
48
+ ## Per-workspace domain hints
49
+
50
+ Monorepos with `packages/auth/`, `packages/payments/`, `packages/db/` produce good auto-detected domains because the path tokens line up with what the Leiden+CPM clusterer finds in the graph.
51
+
52
+ If you have a flatter layout (`apps/web/src/{auth,payments,db}/`), use `carto.config.json` to pin domains:
53
+
54
+ ```json
55
+ {
56
+ "domains": {
57
+ "AUTH": {
58
+ "keywords": ["auth", "login", "session"],
59
+ "anchor": ["apps/web/src/auth/session.ts"]
60
+ },
61
+ "PAYMENTS": {
62
+ "keywords": ["payment", "billing", "checkout"],
63
+ "anchor": ["apps/web/src/payments/charge.ts"]
64
+ }
65
+ }
66
+ }
67
+ ```
68
+
69
+ The `anchor` files force those paths into the named domain regardless of clustering. See [`domains.md`](../concepts/domains.md).
70
+
71
+ ## TypeScript path aliases
72
+
73
+ Carto reads `tsconfig.json#paths` and `jsconfig.json#paths` for import resolution. In a monorepo with workspace references (e.g. `tsconfig.json#references`), each workspace's `tsconfig.json` is read independently. The aliases declared in `apps/web/tsconfig.json` apply when resolving imports *inside* `apps/web/`, etc.
74
+
75
+ If you use TypeScript project references with `"composite": true`, this just works. If you use a monorepo tool that bypasses tsconfig (some Nx setups), declare the equivalent paths in a top-level `tsconfig.json` for Carto's benefit even if the build doesn't use them.
76
+
77
+ ## Cross-package imports
78
+
79
+ When `@company/auth` is published locally (npm workspaces, pnpm workspaces, yarn workspaces), imports of `@company/auth` resolve via the workspace symlink — which Carto follows. So `import { sessionFor } from '@company/auth'` from `apps/web/src/api/users.ts` correctly resolves to `packages/auth/src/index.ts`, and the edge ends up in the graph.
80
+
81
+ What this enables:
82
+
83
+ - `get_blast_radius('packages/auth/src/session.ts')` returns dependents across *every* workspace
84
+ - Cross-domain checks fire on workspace-to-workspace edges
85
+ - The `simulate_change_impact` MCP tool correctly unions blast radius across packages
86
+
87
+ ## Excluding individual workspaces
88
+
89
+ `carto.config.json` can pin which directories to index:
90
+
91
+ ```json
92
+ {
93
+ "include": ["apps/", "packages/"],
94
+ "exclude": ["packages/legacy-thing/"]
95
+ }
96
+ ```
97
+
98
+ `include` is rare — `.cartoignore` is usually the right tool. `exclude` is the common case for "we don't index this old workspace anymore".
99
+
100
+ ## Scoped indexing (>100K files)
101
+
102
+ For *very* large monorepos (Google-scale, ~1M files), the full index gets unwieldy. The roadmap has a `--scope=packages/changed/**` flag that only indexes affected workspaces; that hasn't shipped yet. If you hit this scale, file an issue — the gating decision is "real users at this scale" rather than premature.
103
+
104
+ In the meantime, the working knob is `.cartoignore`. Exclude packages you don't change, accept a partial graph, get a usable index.
105
+
106
+ ## Performance expectations
107
+
108
+ | Files | First run | Re-sync | DB | Bitmap | RSS |
109
+ |------:|----------:|--------:|---:|-------:|----:|
110
+ | 1K | 1s | 350ms | 1 MB | 100 KB | 60 MB |
111
+ | 10K | 8s | 1s | 14 MB | 1.2 MB | 220 MB |
112
+ | 50K | 50s | 4s | 80 MB | 8 MB | 600 MB |
113
+ | 100K | ~100s | ~10s | 160 MB | 20 MB | 1.1 GB |
114
+
115
+ These are from the synthetic stress harness (`bench/scale-test/`). Real repos vary depending on edge density, but the shape is the same.
116
+
117
+ ## Related
118
+
119
+ - [`docs/concepts/domains.md`](../concepts/domains.md) — adaptive gamma + custom hints
120
+ - [`docs/concepts/import-graph.md`](../concepts/import-graph.md) — alias resolution details
121
+ - [`docs/scale.md`](../scale.md) — full benchmark table
@@ -0,0 +1,121 @@
1
+ # Guide: Onboarding a new engineer
2
+
3
+ > Day 1, hour 1, productive. Without 47 Slack DMs to teammates.
4
+
5
+ ## The cold start
6
+
7
+ New engineer joins. The repo has 50K lines, three years of history, no architecture docs. The standard onboarding pattern is:
8
+
9
+ - Day 1–3: Set up tooling. Read README. Get the app running locally.
10
+ - Day 4–10: Get assigned a "starter ticket" — a tiny bug fix in a dark corner. Realize you have no idea where to begin.
11
+ - Day 11–20: Slack DMs. *"Where does session validation live?"* *"What does this file do?"* *"Is this safe to change?"*
12
+ - Day 21+: Maybe starting to feel productive.
13
+
14
+ That's 3 weeks × $5K/week = $15K of ramp time per hire. Most of it is the new engineer trying to build the mental model of the codebase that everyone else already has.
15
+
16
+ ## What Carto does instead
17
+
18
+ `carto init` produces three things the new engineer can read on day 1:
19
+
20
+ 1. **`AGENTS.md`** at the project root. Plain markdown. Architecture overview, domain list with file counts, top high-impact files. Skim in 5 minutes.
21
+
22
+ 2. **`.carto/context/<DOMAIN>.md`** for each detected domain. Per-domain context: entry points, key files, common patterns. Read just the ones relevant to your starter ticket.
23
+
24
+ 3. **A live MCP server** their AI tool (Cursor/Claude/Kiro/etc.) is already wired to. Every question the new engineer would have asked in Slack — "where does session validation live?", "what does this file do?", "what's the blast radius of this change?" — is one MCP call away.
25
+
26
+ ## A literal day-1 walk-through
27
+
28
+ ```bash
29
+ # new engineer just cloned the repo
30
+ git clone …
31
+ cd …
32
+
33
+ # carto is probably already installed; if not:
34
+ npm install -g carto-md
35
+
36
+ # first command they run:
37
+ carto init
38
+ ```
39
+
40
+ The init banner gives the mirror moment:
41
+
42
+ ```
43
+ ┌─ Carto · indexed ─────────────────────────────────────────
44
+ │ 3,847 files · 7 domains · 142 routes · 8,201 import edges
45
+
46
+ │ Top domains:
47
+ │ CORE (1,940 files)
48
+ │ DATABASE (412 files)
49
+ │ AUTH (287 files)
50
+ │ PAYMENTS (235 files)
51
+
52
+ │ 💡 Highest-risk file: src/lib/db/client.ts
53
+ │ (94 files depend on it — try `carto why src/lib/db/client.ts`)
54
+ └───────────────────────────────────────────────────────────
55
+ ```
56
+
57
+ They now know:
58
+
59
+ - This repo has ~4K source files (real size, not the inflated package.json claim)
60
+ - It's split into 7 domains they should learn one at a time
61
+ - The single most important file is `src/lib/db/client.ts`
62
+
63
+ Five minutes in, they've replaced 30 minutes of clicking around.
64
+
65
+ ## What to ask first
66
+
67
+ Three questions every new engineer should ask their AI tool in the first hour:
68
+
69
+ 1. *"Give me the architectural overview of this project."* → calls `get_architecture()`. 500-word summary.
70
+
71
+ 2. *"What's in the AUTH domain?"* (or whichever they're working on) → calls `get_domain("AUTH")`. Routes, models, key files.
72
+
73
+ 3. *"What does `src/lib/db/client.ts` do and what depends on it?"* → calls `get_file_summary()` + `get_blast_radius()`. Now they know the danger zone.
74
+
75
+ ## When they pick up their first ticket
76
+
77
+ They start with:
78
+
79
+ ```bash
80
+ carto explain "fix the bug in user signup where email validation is skipped"
81
+ ```
82
+
83
+ This calls `get_change_plan()` — same engine the AI uses. Output:
84
+
85
+ ```
86
+ ## Plan: fix bug in user signup where email validation is skipped
87
+
88
+ ### Relevant routes
89
+ - POST /auth/signup src/auth/signup.ts:42
90
+
91
+ ### Files to touch
92
+ - src/auth/signup.ts — the route handler (15 direct dependents)
93
+ - src/auth/validators.ts — email validation lives here
94
+
95
+ ### Similar patterns
96
+ - src/auth/login.ts already validates email format using validateEmail()
97
+ - src/auth/reset-password.ts also uses validateEmail()
98
+
99
+ ### Blast radius
100
+ - src/auth/signup.ts: 15 dependent files. Test coverage in
101
+ src/auth/__tests__/signup.test.ts — re-run after change.
102
+ ```
103
+
104
+ A new engineer who's never opened `signup.ts` can read this, open the 3 files it names, find the bug, fix it correctly the first time. No Slack DM needed.
105
+
106
+ ## When they break something
107
+
108
+ Even with Carto, mistakes happen. The two tools that help most:
109
+
110
+ - **`carto diff`** before committing. Shows the architectural impact of the local change — blast radius, cross-domain violations, files-without-tests. The new engineer self-corrects.
111
+ - **The GitHub Action** ([guide](./ci-integration.md)) catches what the new engineer didn't. The PR gets an impact comment; reviewers spot risky changes before merge.
112
+
113
+ ## The team multiplier
114
+
115
+ Onboarding is the cost the *team* pays for a hire, not just the hire. Every Slack DM from a new engineer is an interruption to a senior. A 1-week ramp instead of a 3-week ramp doesn't just save the new engineer's time — it saves the senior's too.
116
+
117
+ ## Related
118
+
119
+ - [`docs/quickstart.md`](../quickstart.md) — install + init
120
+ - [`docs/concepts/domains.md`](../concepts/domains.md) — how domains are named
121
+ - [`docs/guides/adding-feature-safely.md`](./adding-feature-safely.md) — the workflow for the second week onward