codecartographer-pi 0.6.1 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +253 -217
- package/dist/core/dashboard.d.ts +36 -0
- package/dist/core/dashboard.js +526 -0
- package/dist/core/index.d.ts +1 -0
- package/dist/core/index.js +1 -0
- package/dist/core/utils.d.ts +13 -0
- package/dist/core/utils.js +27 -0
- package/dist/core/workspace.d.ts +1 -0
- package/dist/core/workspace.js +13 -1
- package/dist/extensions/codecarto/auto-runner.d.ts +80 -0
- package/dist/extensions/codecarto/auto-runner.js +399 -0
- package/dist/extensions/codecarto/dashboard-flags.d.ts +6 -0
- package/dist/extensions/codecarto/dashboard-flags.js +17 -0
- package/dist/extensions/codecarto/dashboard-narrator.d.ts +8 -0
- package/dist/extensions/codecarto/dashboard-narrator.js +182 -0
- package/dist/extensions/codecarto/dashboard-writer.d.ts +1 -0
- package/dist/extensions/codecarto/dashboard-writer.js +123 -0
- package/dist/extensions/codecarto/index.js +75 -206
- package/dist/extensions/codecarto/next-flags.d.ts +4 -0
- package/dist/extensions/codecarto/next-flags.js +19 -6
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -3,197 +3,254 @@
|
|
|
3
3
|
[](https://github.com/HuginnIndustries/CodeCartographer/actions/workflows/ci.yml)
|
|
4
4
|
[](LICENSE)
|
|
5
5
|
[](https://www.npmjs.com/package/codecartographer-pi)
|
|
6
|
+
[](package.json)
|
|
6
7
|
|
|
7
|
-
A structured reverse-engineering
|
|
8
|
+
> **A structured pipeline for reverse-engineering unfamiliar codebases with an LLM.** Drop it into any repo, point an LLM at the guide, and walk away with a layered analysis: architecture map, behavioral contracts, protocol documentation, defect report, porting bundle, and a language-agnostic reimplementation spec. Every finding is evidence-tagged. Every phase output is validated before the next one starts.
|
|
8
9
|
|
|
9
|
-
|
|
10
|
+
```text
|
|
11
|
+
● CodeCartographer
|
|
12
|
+
├─ ✓ architecture phase ⟳ 25 · 76 tool uses · 1.0M tokens · 4m28s
|
|
13
|
+
├─ ✓ defect-scan-mech. ⟳ 39 · 91 tool uses · 2.4M tokens · 7m05s
|
|
14
|
+
└─ ⠹ contracts phase ⟳ 11 · 37 tool uses · 335.1k tokens · 40.1s
|
|
15
|
+
⎿ extracting behavioral contracts from server/index.ts…
|
|
16
|
+
```
|
|
10
17
|
|
|
11
|
-
|
|
18
|
+
---
|
|
12
19
|
|
|
13
|
-
|
|
20
|
+
## At a glance
|
|
14
21
|
|
|
15
|
-
|
|
22
|
+
| What you get | Where it lives |
|
|
23
|
+
|---|---|
|
|
24
|
+
| **Layered analysis pipeline** — architecture → defect scan → behavioral contracts → protocols → porting → reimplementation spec | `.codecarto/` template |
|
|
25
|
+
| **Validation gates between phases** — no advancing past a `FAIL` output | `core/` state machine |
|
|
26
|
+
| **Three delivery surfaces** — Pi extension, MCP server, or pure template | All three share `core/` |
|
|
27
|
+
| **Live progress widget** while phase sub-agents work | Pi extension |
|
|
28
|
+
| **HTML dashboard** — single-file aggregate of progress, links, usage, narrative | `.codecarto/dashboard.html` |
|
|
29
|
+
| **Per-phase token tracking** | `/codecarto-usage` |
|
|
30
|
+
| **Opt-in LLM steering** of the next phase's seed prompt | `/codecarto-next --llm-steer` |
|
|
16
31
|
|
|
17
|
-
|
|
32
|
+
---
|
|
18
33
|
|
|
19
|
-
|
|
20
|
-
cp -r /path/to/CodeCartographer/.codecarto /path/to/your-repo/
|
|
21
|
-
```
|
|
34
|
+
## Install
|
|
22
35
|
|
|
23
|
-
|
|
36
|
+
Pick the surface that matches your tooling. All three share the same `core/` and produce byte-identical phase prompts.
|
|
24
37
|
|
|
25
|
-
|
|
26
|
-
# Edit .codecarto/workflow/status.yaml and set the pipeline field:
|
|
27
|
-
pipeline: workflow/pipeline-full-with-deep-audit.yaml # 7-phase with split defect scan (default; depth-first)
|
|
28
|
-
pipeline: workflow/pipeline-full-with-audit.yaml # 6-phase with single early defect scan
|
|
29
|
-
pipeline: workflow/pipeline.yaml # 5-phase without defect scan — remove defect-scan phases
|
|
30
|
-
pipeline: workflow/pipeline-defect-scan.yaml # 2-phase defect audit — remove contracts through reimplementation-spec
|
|
31
|
-
pipeline: workflow/pipeline-lite.yaml # 3-phase understanding — remove defect-scan phases, porting, and reimplementation-spec
|
|
32
|
-
pipeline: workflow/pipeline-architecture-only.yaml # 1-phase quick overview — keep only architecture
|
|
33
|
-
```
|
|
38
|
+
### Pi extension (recommended for interactive use)
|
|
34
39
|
|
|
35
|
-
|
|
40
|
+
[Pi](https://github.com/badlogic/pi-mono/tree/main/packages/coding-agent) is a TUI coding agent. The CodeCartographer extension adds slash commands, a live agents widget, and the dashboard.
|
|
36
41
|
|
|
37
|
-
```
|
|
38
|
-
|
|
42
|
+
```bash
|
|
43
|
+
pi install npm:codecartographer-pi # from the npm registry
|
|
44
|
+
pi install /absolute/path/to/CodeCartographer # from a local checkout
|
|
45
|
+
pi install git:github.com/your-user/CodeCartographer # from a git URL
|
|
39
46
|
```
|
|
40
47
|
|
|
41
|
-
|
|
48
|
+
> **Don't** run `npm install codecartographer-pi` for the Pi use case. Plain `npm install` puts the package on disk but doesn't register it with Pi. Use `pi install npm:...` so Pi writes the package into its own `~/.pi/agent/settings.json`.
|
|
42
49
|
|
|
43
|
-
|
|
50
|
+
For extension development, point Pi directly at the entrypoint:
|
|
44
51
|
|
|
45
|
-
|
|
52
|
+
```bash
|
|
53
|
+
pi -e /absolute/path/to/CodeCartographer/extensions/codecarto/index.ts
|
|
54
|
+
```
|
|
46
55
|
|
|
47
|
-
|
|
56
|
+
### MCP server (for Claude Code, Claude Desktop, any MCP host)
|
|
48
57
|
|
|
49
58
|
```bash
|
|
50
|
-
|
|
51
|
-
# or, from a local checkout:
|
|
52
|
-
pi install /absolute/path/to/CodeCartographer
|
|
53
|
-
# or, from a git URL:
|
|
54
|
-
pi install git:github.com/your-user/CodeCartographer
|
|
59
|
+
npm install --global codecartographer-pi
|
|
55
60
|
```
|
|
56
61
|
|
|
57
|
-
|
|
62
|
+
Add to your host config (`~/.config/claude-code/config.json`, `claude_desktop_config.json`, etc.):
|
|
63
|
+
|
|
64
|
+
```json
|
|
65
|
+
{
|
|
66
|
+
"mcpServers": {
|
|
67
|
+
"codecartographer": {
|
|
68
|
+
"command": "codecarto-mcp"
|
|
69
|
+
}
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
```
|
|
58
73
|
|
|
59
|
-
|
|
74
|
+
### Pure template (no runtime, any LLM that reads/writes files)
|
|
60
75
|
|
|
61
76
|
```bash
|
|
62
|
-
|
|
77
|
+
cp -r /path/to/CodeCartographer/.codecarto /path/to/your-repo/
|
|
63
78
|
```
|
|
64
79
|
|
|
65
|
-
|
|
80
|
+
Then in the LLM session: `Read .codecarto/GUIDE.md and begin the analysis.`
|
|
66
81
|
|
|
67
|
-
|
|
82
|
+
---
|
|
68
83
|
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
/codecarto-next
|
|
73
|
-
```
|
|
84
|
+
## How it works
|
|
85
|
+
|
|
86
|
+
The "code" is structured Markdown + YAML inside `.codecarto/`:
|
|
74
87
|
|
|
75
|
-
|
|
88
|
+
- **`GUIDE.md`** — LLM entry point. Every session reads this first.
|
|
89
|
+
- **`workflow/pipeline.yaml`** — phase definitions, dependencies, output paths.
|
|
90
|
+
- **`workflow/status.yaml`** — mutable per-project state. Single source of truth for progress.
|
|
91
|
+
- **`workflow/VALIDATE.md`** — validation protocol run after every phase.
|
|
92
|
+
- **`findings/<phase>/SKILL.md`** — detailed analysis instructions per phase.
|
|
93
|
+
- **`templates/`** — output templates that enforce consistent structure.
|
|
76
94
|
|
|
77
|
-
|
|
95
|
+
Phases form a DAG: `contracts` and `protocols` can run in parallel after `architecture`; `porting` waits for both; `reimplementation-spec` is last. The host (Pi, MCP, or your shell) reads the active pipeline, finds the next phase whose dependencies are all `complete`, hands the LLM that phase's instructions, validates the output, and advances `status.yaml`.
|
|
78
96
|
|
|
79
|
-
|
|
80
|
-
- `/codecarto-next [--llm-steer | --no-llm-steer]` to spawn the next eligible phase as a sub-agent (the optional flag opts into the LLM-rewriter for the seed prompt; see *Phase orchestration* below)
|
|
81
|
-
- `/codecarto-status` to show current phase progress
|
|
82
|
-
- `/codecarto-validate` and `/codecarto-complete` for validation-gated status updates
|
|
83
|
-
- `/codecarto-phase <id>` to force a specific phase even out of pipeline order
|
|
84
|
-
- `/codecarto-skill <name>` to run a post-pipeline skill once all phases are complete
|
|
85
|
-
- `/codecarto-usage` to show cumulative + per-phase token usage from local phase runs (0.6.0+)
|
|
86
|
-
- a footer/widget showing the active CodeCartographer phase, plus a live **Agents** widget above the editor while a phase sub-agent is running
|
|
87
|
-
- tool interception that blocks `edit` and `write` outside `.codecarto/`
|
|
88
|
-
- direct phase prompts that tell Pi exactly which `.codecarto/findings/<phase>/SKILL.md` file to read, without registering those internal files as global Pi skills
|
|
97
|
+
For multi-session work, every new session reads `.codecarto/GUIDE.md` (or the lighter `NEW_THREAD_BLURB.md`), checks `workflow/status.yaml`, and picks up where the last session left off. You don't explain what happened in previous sessions.
|
|
89
98
|
|
|
90
|
-
|
|
99
|
+
---
|
|
91
100
|
|
|
92
|
-
|
|
101
|
+
## Phases produce these artifacts
|
|
93
102
|
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
103
|
+
| Artifact | Description |
|
|
104
|
+
|---|---|
|
|
105
|
+
| **Architecture map** | Layers, dependency direction, public surfaces, runtime lifecycle, concurrency model |
|
|
106
|
+
| **Defect report** | Multi-pass scan for logic errors, security issues, concurrency bugs, API violations |
|
|
107
|
+
| **Defect fix tracker** | Remediation log mapping each fix, deferral, or acceptance back to the defect report |
|
|
108
|
+
| **Behavioral contracts** | Feature-by-feature behavior with defaults, error handling, and acceptance tests |
|
|
109
|
+
| **Protocols and state** | Event flows, state machines, persistence formats, compatibility hazards |
|
|
110
|
+
| **Porting bundle** | Everything synthesized into a porting-oriented view with priority rankings |
|
|
111
|
+
| **Reimplementation spec** | Language-agnostic build plan with modules, acceptance scenarios, and known unknowns |
|
|
99
112
|
|
|
100
|
-
|
|
113
|
+
Every finding is tagged with an evidence level: `observed fact`, `strong inference`, `portability hazard`, or `open question`. Every phase output is validated against explicit completion criteria before the pipeline advances.
|
|
101
114
|
|
|
102
|
-
|
|
103
|
-
- **0.4.0 — phase-completion summary.** When a phase finishes (completed, aborted, or errored), a Markdown closeout block is appended to the orchestrator's transcript via `pi.sendMessage(...)`. You see it in the TUI scrollback; the orchestrator's LLM picks it up as context on your next message. No auto-trigger — control of the next step stays with you.
|
|
104
|
-
- **0.5.0 — opt-in LLM-steered seed prompt.** Set `orchestrator.llm_steer_next_phase: true` in `.codecarto/workflow/config.yaml`, or pass `--llm-steer` per invocation, and the orchestrator's model will run a one-shot rewriter that reads the previous phase's closeout and customizes the next phase's seed prompt to highlight relevant prior findings. Off by default — extra orchestrator-side tokens, opt-in.
|
|
105
|
-
- **0.6.0 — local usage log.** Each phase run is appended to `.codecarto/workflow/.usage.local.yaml` (gitignored). `/codecarto-usage` reports cumulative + per-phase totals. Best-effort logging — write failures don't surface as phase errors.
|
|
115
|
+
---
|
|
106
116
|
|
|
107
|
-
|
|
117
|
+
## Pipeline variants
|
|
108
118
|
|
|
109
|
-
The
|
|
119
|
+
The default is a 7-phase run that splits the defect scan into a mechanical early pass and a semantic late pass — the reimplementation phase then designs around defects with full contracts and protocols context. Scale back if you want less:
|
|
110
120
|
|
|
111
|
-
|
|
121
|
+
| Variant | Phases | Use when |
|
|
122
|
+
|---|---|---|
|
|
123
|
+
| **Full with deep audit** (default) | 7 | Complete analysis with split defect scan; reimplementation grounded in contracts/protocols-aware defect findings |
|
|
124
|
+
| **Full with audit** | 6 | Single early defect scan; cheaper than the deep variant when defects are mostly mechanical |
|
|
125
|
+
| **Full** | 5 | Porting or reimplementation without any defect scan |
|
|
126
|
+
| **Defect scan** | 2 | Maintenance audit to surface latent problems |
|
|
127
|
+
| **Lite** | 3 | You need to understand behavior without porting plans |
|
|
128
|
+
| **Architecture only** | 1 | Quick structural overview |
|
|
112
129
|
|
|
113
|
-
|
|
130
|
+
Set the active pipeline by editing `workflow/status.yaml`'s `pipeline:` field, or pass it as the argument to `/codecarto-init`.
|
|
114
131
|
|
|
115
|
-
|
|
132
|
+
**On disk:**
|
|
116
133
|
|
|
117
|
-
|
|
134
|
+
| Variant | Pipeline file |
|
|
135
|
+
|---|---|
|
|
136
|
+
| Full with deep audit (**default**) | `workflow/pipeline-full-with-deep-audit.yaml` |
|
|
137
|
+
| Full with audit | `workflow/pipeline-full-with-audit.yaml` |
|
|
138
|
+
| Full | `workflow/pipeline.yaml` |
|
|
139
|
+
| Defect scan | `workflow/pipeline-defect-scan.yaml` |
|
|
140
|
+
| Lite | `workflow/pipeline-lite.yaml` |
|
|
141
|
+
| Architecture only | `workflow/pipeline-architecture-only.yaml` |
|
|
118
142
|
|
|
119
|
-
|
|
120
|
-
npm install --global codecartographer-pi
|
|
121
|
-
# or, in a project: npm install codecartographer-pi
|
|
122
|
-
```
|
|
143
|
+
---
|
|
123
144
|
|
|
124
|
-
|
|
145
|
+
## The dashboard
|
|
125
146
|
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
147
|
+
Every state change re-renders `.codecarto/dashboard.html` — a self-contained single-file artifact you open in any browser. Aggregates everything a human wants to see at a glance:
|
|
148
|
+
|
|
149
|
+
- Pipeline progress strip with per-phase status badges
|
|
150
|
+
- Per-phase cards with output links, open questions, carry-forward routing, owner notes, last-run usage
|
|
151
|
+
- Aggregate token usage panel + per-phase breakdown
|
|
152
|
+
- Activity timeline with session-file links
|
|
153
|
+
- Open questions roll-up grouped by source phase
|
|
154
|
+
- Closeouts list (reverse-chronological) with relative-path links
|
|
155
|
+
|
|
156
|
+
No JavaScript. No external assets. Light/dark via `prefers-color-scheme`. Works opened directly from `file://`.
|
|
157
|
+
|
|
158
|
+
**Opt-in narrative summary.** `/codecarto-dashboard --narrate` runs the orchestrator's model as a one-shot session that writes a 200–400 word executive summary citing specific findings from recent closeouts. Cached to `.codecarto/.dashboard-narration.local.md` and preserved across deterministic re-renders with a "(N runs since)" staleness note.
|
|
159
|
+
|
|
160
|
+
---
|
|
161
|
+
|
|
162
|
+
## Pi extension features
|
|
163
|
+
|
|
164
|
+
Beyond the slash commands, the Pi extension layers on:
|
|
165
|
+
|
|
166
|
+
**Phase sub-agents.** `/codecarto-next` spawns each phase as an isolated `AgentSession`. Tool calls, file reads, and reasoning live in the child's own context window — they never accumulate in the orchestrator. Your TUI stays on the orchestrator session and remains responsive while phases work in background.
|
|
167
|
+
|
|
168
|
+
**Live agents widget** above the editor showing tool count, token usage, elapsed time, and current activity.
|
|
169
|
+
|
|
170
|
+
```text
|
|
171
|
+
● CodeCartographer
|
|
172
|
+
└─ ⠹ architecture phase ⟳ 3 · 5 tool uses · 12.3k tokens · 1m32s
|
|
173
|
+
⎿ reading…
|
|
134
174
|
```
|
|
135
175
|
|
|
136
|
-
|
|
176
|
+
**File-backed phase sessions.** Phase transcripts persist to the same Pi session directory the orchestrator uses, so `/resume`, `/tree`, and `/export` browse them as first-class sessions. Each appears as `CodeCartographer phase: <id>` with lineage back to the orchestrator's session.
|
|
137
177
|
|
|
138
|
-
|
|
139
|
-
|---|---|---|
|
|
140
|
-
| `codecarto_init` | Copy `.codecarto/` into the target repo and select a pipeline | `/codecarto-init` |
|
|
141
|
-
| `codecarto_status` | Current phase, active pipeline, progress, open questions | `/codecarto-status` |
|
|
142
|
-
| `codecarto_next` | Return the next eligible phase prompt as text | `/codecarto-next` |
|
|
143
|
-
| `codecarto_phase` | Return a specific phase's prompt (forced, even out of order) | `/codecarto-phase` |
|
|
144
|
-
| `codecarto_validate` | Validate a phase output, returning structured criteria rows | `/codecarto-validate` |
|
|
145
|
-
| `codecarto_complete` | Atomically mark a phase complete after validation passes | `/codecarto-complete` |
|
|
146
|
-
| `codecarto_skill` | Return a post-pipeline skill prompt | `/codecarto-skill` |
|
|
178
|
+
**Phase-completion summary in the orchestrator transcript.** When a phase finishes, a Markdown closeout block is appended to the orchestrator's session via `pi.sendMessage(...)`. Visible in the TUI scrollback; available to the orchestrator's LLM as context on your next message. No auto-trigger — you stay in control.
|
|
147
179
|
|
|
148
|
-
|
|
180
|
+
**Opt-in LLM-steered seed prompts.** Set `orchestrator.llm_steer_next_phase: true` in `.codecarto/workflow/config.yaml` (or pass `--llm-steer` per invocation), and the orchestrator's LLM rewrites the next phase's seed prompt to highlight relevant prior findings. Off by default — extra orchestrator-side tokens, opt-in. The rewritten prompt is injected into the orchestrator transcript so you can audit what the rewriter chose to emphasize.
|
|
149
181
|
|
|
150
|
-
|
|
182
|
+
**Per-phase usage tracking.** Each phase run is appended to `.codecarto/workflow/.usage.local.yaml`. `/codecarto-usage` reports cumulative + per-phase totals.
|
|
151
183
|
|
|
152
|
-
|
|
184
|
+
**Tool interception.** `bash` is blocked outright; `edit` and `write` are confined to `.codecarto/`. Same rules apply to phase sub-agents.
|
|
185
|
+
|
|
186
|
+
### Slash commands
|
|
187
|
+
|
|
188
|
+
| Command | Purpose |
|
|
153
189
|
|---|---|
|
|
154
|
-
|
|
|
155
|
-
|
|
|
156
|
-
|
|
|
157
|
-
|
|
|
158
|
-
|
|
|
159
|
-
|
|
|
160
|
-
|
|
|
190
|
+
| `/codecarto-init [variant]` | Copy `.codecarto/` into the current repository, select pipeline variant |
|
|
191
|
+
| `/codecarto-status` | Current phase, progress, open questions |
|
|
192
|
+
| `/codecarto-next [--auto [--strict]] [--llm-steer \| --no-llm-steer]` | Spawn the next eligible phase as a sub-agent. `--auto` walks the full pipeline end-to-end (auto-validate + auto-complete + advance); `--strict` flips the `PASS WITH GAPS` rule from "advance" to "pause". |
|
|
193
|
+
| `/codecarto-phase <id>` | Force a specific phase, even out of pipeline order |
|
|
194
|
+
| `/codecarto-validate [phase]` | Validate a phase output against completion criteria |
|
|
195
|
+
| `/codecarto-complete [phase]` | Atomically mark a phase complete (validation must pass) |
|
|
196
|
+
| `/codecarto-skill <name>` | Run a post-pipeline skill once all phases are complete |
|
|
197
|
+
| `/codecarto-usage` | Cumulative + per-phase token usage |
|
|
198
|
+
| `/codecarto-dashboard [--narrate]` | Regenerate `.codecarto/dashboard.html`; `--narrate` for the LLM executive summary |
|
|
161
199
|
|
|
162
|
-
|
|
200
|
+
### End-to-end auto mode (0.8.0+)
|
|
163
201
|
|
|
164
|
-
|
|
202
|
+
`/codecarto-next --auto` walks the entire pipeline without intervention. The loop spawns each next-eligible phase, auto-validates the output, auto-marks it complete, and advances until the pipeline finishes — or until something stops it (`FAIL` / `MISSING` validation, sub-agent error, or `ctx.signal` abort). The orchestrator's TUI stays responsive throughout; per-phase summaries land in the transcript as usual, and a final `codecarto-auto-summary` block reports the outcome with cumulative tokens, wall time, and a recovery hint if the run stopped early.
|
|
165
203
|
|
|
166
|
-
|
|
204
|
+
- **Resumability** is implicit: re-running `--auto` reads `status.yaml` and picks up from `getNextEligiblePhase`.
|
|
205
|
+
- **`--strict`** (requires `--auto`) treats `PASS WITH GAPS` as a stop — useful when you want to triage gaps before advancing.
|
|
206
|
+
- **`--auto --llm-steer`** runs the rewriter on every phase transition; the per-phase steering blocks land in the orchestrator transcript so the run is auditable.
|
|
167
207
|
|
|
168
|
-
|
|
169
|
-
|---|---|---|
|
|
170
|
-
| **Full with deep audit** (default) | 7 | Complete analysis with split defect scan; reimplementation grounded in contracts/protocols-aware defect findings |
|
|
171
|
-
| **Full with audit** | 6 | Single early defect scan; cheaper than the deep variant when the defects are mostly mechanical |
|
|
172
|
-
| **Full** | 5 | Porting or reimplementation without any defect scan |
|
|
173
|
-
| **Defect scan** | 2 | Maintenance audit to surface latent problems |
|
|
174
|
-
| **Lite** | 3 | You need to understand behavior without porting plans |
|
|
175
|
-
| **Architecture only** | 1 | Quick structural overview |
|
|
208
|
+
### Version history (Pi orchestration)
|
|
176
209
|
|
|
177
|
-
|
|
210
|
+
The current parallel-sub-agent design landed in 0.2.0 and has been incrementally enriched: file-backed sessions (0.3.0), summary injection (0.4.0), opt-in LLM steering (0.5.0), usage tracking (0.6.0), HTML dashboard (0.7.0), and end-to-end auto mode (0.8.0). 0.1.x workspaces don't need migration — existing `.codecarto/` directories work unchanged. See `CHANGELOG.md` for details.
|
|
211
|
+
|
|
212
|
+
---
|
|
213
|
+
|
|
214
|
+
## MCP server
|
|
215
|
+
|
|
216
|
+
The same framework is packaged as a [Model Context Protocol](https://modelcontextprotocol.io) server. The MCP path returns prompt text for the host to dispatch and never runs sub-agents itself, so the Pi-only orchestration features (sub-agents, live widget, dashboard, usage tracking) don't apply — but phase prompts and validation are byte-identical with the Pi path because both import the same `core/`.
|
|
217
|
+
|
|
218
|
+
Implements MCP spec revision [`2025-11-25`](https://modelcontextprotocol.io/specification/2025-11-25) via `@modelcontextprotocol/sdk` ≥ 1.29.0. The negotiated `protocolVersion` reflects whatever the connecting client requests; the server accepts every revision the SDK supports (currently `2025-11-25`, `2025-06-18`, `2025-03-26`, `2024-11-05`, `2024-10-07`).
|
|
219
|
+
|
|
220
|
+
| Tool | Pi equivalent |
|
|
221
|
+
|---|---|
|
|
222
|
+
| `codecarto_init` | `/codecarto-init` |
|
|
223
|
+
| `codecarto_status` | `/codecarto-status` |
|
|
224
|
+
| `codecarto_next` | `/codecarto-next` |
|
|
225
|
+
| `codecarto_phase` | `/codecarto-phase` |
|
|
226
|
+
| `codecarto_validate` | `/codecarto-validate` |
|
|
227
|
+
| `codecarto_complete` | `/codecarto-complete` |
|
|
228
|
+
| `codecarto_skill` | `/codecarto-skill` |
|
|
178
229
|
|
|
179
|
-
|
|
230
|
+
Each tool accepts an absolute `cwd` for the target repository. `codecarto_init` requires `force: true` to overwrite an existing `.codecarto/` (instead of Pi's interactive confirmation).
|
|
231
|
+
|
|
232
|
+
---
|
|
233
|
+
|
|
234
|
+
## Compatible environments
|
|
180
235
|
|
|
181
236
|
| Environment | Notes |
|
|
182
237
|
|---|---|
|
|
183
|
-
| **
|
|
184
|
-
| **
|
|
185
|
-
| **
|
|
186
|
-
| **Aider** |
|
|
238
|
+
| **Pi** | Native — install the extension, get slash commands + widget + dashboard. |
|
|
239
|
+
| **Claude Code** | MCP server, or point it at `.codecarto/GUIDE.md` directly. |
|
|
240
|
+
| **Claude Desktop** | MCP server. |
|
|
241
|
+
| **OpenCode / Aider / Cursor / Windsurf / IDE copilots** | Open the repo, point the LLM at `.codecarto/GUIDE.md`. |
|
|
187
242
|
| **Claude.ai / ChatGPT (web chat)** | Paste file contents manually. Tedious for multi-phase runs. |
|
|
188
243
|
| **API-based agents** | Load files programmatically, pass to the model, write outputs back. |
|
|
189
244
|
|
|
190
|
-
|
|
245
|
+
---
|
|
191
246
|
|
|
192
|
-
|
|
247
|
+
## Token usage and cost
|
|
193
248
|
|
|
194
|
-
|
|
249
|
+
CodeCartographer is token-intensive — it reads source code multiple times across phases and produces thousands of words of structured output. Plan accordingly.
|
|
195
250
|
|
|
196
|
-
|
|
251
|
+
### Template overhead (fixed cost)
|
|
252
|
+
|
|
253
|
+
Every session reads the guide, pipeline definition, status, and validation protocol. Each phase additionally reads its own `SKILL.md` and output template. Fixed regardless of codebase size:
|
|
197
254
|
|
|
198
255
|
| Component | Tokens (input) |
|
|
199
256
|
|---|---|
|
|
@@ -204,17 +261,16 @@ Every session reads the guide, pipeline definition, status file, and validation
|
|
|
204
261
|
| Protocols phase instructions | ~1,200 |
|
|
205
262
|
| Porting phase instructions | ~1,200 |
|
|
206
263
|
| Reimplementation spec phase instructions | ~1,100 |
|
|
207
|
-
| **Total template overhead
|
|
208
|
-
|
|
209
|
-
### Source Code Reading (Variable Cost)
|
|
264
|
+
| **Total template overhead, 6-phase run** | **~27,000** |
|
|
265
|
+
| **Total template overhead, 7-phase deep-audit** | **~32,000** (split defect scan adds one more SKILL load) |
|
|
210
266
|
|
|
211
|
-
|
|
267
|
+
### Source code reading (variable cost)
|
|
212
268
|
|
|
213
|
-
Rough guide: **expect
|
|
269
|
+
The dominant cost. Each phase reads some or all of your source code; the architecture phase reads the most. Rough guide: **expect 1–3× your codebase size in tokens per phase**. A 50k-token codebase might consume 100–200k input tokens across a full pipeline run.
|
|
214
270
|
|
|
215
|
-
### Output
|
|
271
|
+
### Output generation
|
|
216
272
|
|
|
217
|
-
|
|
273
|
+
From a real 6-phase run (CodeCartographer analyzing itself — a small ~14k-word template):
|
|
218
274
|
|
|
219
275
|
| Phase | Output size |
|
|
220
276
|
|---|---|
|
|
@@ -228,36 +284,40 @@ Each phase produces a structured findings document. From a real 6-phase run (Cod
|
|
|
228
284
|
|
|
229
285
|
Larger codebases produce proportionally larger outputs.
|
|
230
286
|
|
|
231
|
-
### Cost
|
|
287
|
+
### Cost estimates
|
|
232
288
|
|
|
233
289
|
For a medium-sized codebase (~100k tokens of source):
|
|
234
290
|
|
|
235
|
-
| Pipeline | Estimated
|
|
291
|
+
| Pipeline | Estimated input | Estimated output | Total |
|
|
236
292
|
|---|---|---|---|
|
|
237
293
|
| Architecture only | ~130k | ~5k | ~135k tokens |
|
|
238
294
|
| Defect scan (2-phase) | ~260k | ~10k | ~270k tokens |
|
|
239
295
|
| Lite (3-phase) | ~370k | ~15k | ~385k tokens |
|
|
240
296
|
| Full (5-phase) | ~570k | ~22k | ~592k tokens |
|
|
241
297
|
| Full with audit (6-phase) | ~700k | ~27k | ~727k tokens |
|
|
298
|
+
| Full with deep audit (7-phase, default) | ~830k | ~32k | ~862k tokens |
|
|
242
299
|
|
|
243
|
-
At current API pricing (~$3/M input, ~$15/M output for Claude Sonnet), a full 5-phase run on a 100k-token codebase costs roughly **$2
|
|
300
|
+
At current API pricing (~$3/M input, ~$15/M output for Claude Sonnet), a full 5-phase run on a 100k-token codebase costs roughly **$2–4**. Larger codebases scale linearly.
|
|
244
301
|
|
|
245
|
-
### Tips to
|
|
302
|
+
### Tips to reduce token usage
|
|
246
303
|
|
|
247
|
-
- **Start with architecture-only
|
|
248
|
-
- **
|
|
249
|
-
- **For very large codebases** (500k+ tokens of source), the LLM can't read everything anyway. It
|
|
250
|
-
- **The lite pipeline (3 phases) gives 80% of the value** for understanding a codebase without
|
|
304
|
+
- **Start with `architecture-only`** to see if the output quality is useful before committing to a full run.
|
|
305
|
+
- **One LLM session per phase** — each phase gets a fresh context window so you're not paying to carry stale context.
|
|
306
|
+
- **For very large codebases** (500k+ tokens of source), the LLM can't read everything anyway. It uses the architecture map to prioritize and produces partial results. `open_questions` in `status.yaml` shows what was skipped.
|
|
307
|
+
- **The `lite` pipeline (3 phases) gives 80% of the value** for understanding a codebase without porting-specific phases.
|
|
308
|
+
- **Skip `--llm-steer`** unless you're hitting cross-phase coherence issues — the rewriter costs orchestrator-side tokens per phase.
|
|
251
309
|
|
|
252
|
-
|
|
310
|
+
---
|
|
253
311
|
|
|
254
|
-
|
|
312
|
+
## Model compatibility
|
|
255
313
|
|
|
256
|
-
|
|
314
|
+
LLM-agnostic by design, but model choice affects both what you can analyze and how good the results are. Two independent constraints: **context window size** and **model capability**.
|
|
257
315
|
|
|
258
|
-
|
|
316
|
+
### Context window
|
|
259
317
|
|
|
260
|
-
|
|
318
|
+
Each phase runs in its own session, so the context window limits how much source code can be read per phase — not across the whole pipeline. After template overhead, prior-phase findings, and output generation:
|
|
319
|
+
|
|
320
|
+
| Phase | Available for source (128k model) | Available (200k model) |
|
|
261
321
|
|---|---|---|
|
|
262
322
|
| Architecture | ~121k | ~193k |
|
|
263
323
|
| Defect scan | ~115k | ~187k |
|
|
@@ -266,79 +326,40 @@ Each phase runs in its own session, so the context window limits how much source
|
|
|
266
326
|
| Porting | ~104k | ~176k |
|
|
267
327
|
| Reimplementation spec | ~103k | ~175k |
|
|
268
328
|
|
|
269
|
-
|
|
329
|
+
Practical limits by codebase size:
|
|
270
330
|
|
|
271
|
-
| Codebase
|
|
331
|
+
| Codebase | 128k context | 200k context |
|
|
272
332
|
|---|---|---|
|
|
273
333
|
| <30k tokens | All phases comfortable | All phases comfortable |
|
|
274
|
-
| 30–60k tokens | Feasible, some PARTIAL results | Comfortable |
|
|
275
|
-
| 60–100k tokens | Marginal — heavy PARTIAL use | Feasible with prioritization |
|
|
276
|
-
| >100k tokens | Not viable | Feasible, later phases may PARTIAL |
|
|
334
|
+
| 30–60k tokens | Feasible, some `PARTIAL` results | Comfortable |
|
|
335
|
+
| 60–100k tokens | Marginal — heavy `PARTIAL` use | Feasible with prioritization |
|
|
336
|
+
| >100k tokens | Not viable | Feasible, later phases may `PARTIAL` |
|
|
277
337
|
|
|
278
|
-
The pipeline handles context exhaustion gracefully: phases
|
|
338
|
+
The pipeline handles context exhaustion gracefully: phases write `PARTIAL` validation and log remaining work in `open_questions`.
|
|
279
339
|
|
|
280
|
-
### Model
|
|
340
|
+
### Model capability
|
|
281
341
|
|
|
282
|
-
|
|
342
|
+
The harder constraint. Tasks that degrade fastest on weaker models:
|
|
283
343
|
|
|
284
|
-
1. **Evidence classification** (high risk) — distinguishing `observed fact` from `strong inference` from `open question` requires calibrated self-awareness about certainty. Weaker models
|
|
285
|
-
2. **Defect scan** (high risk) — the
|
|
286
|
-
3. **Architecture synthesis** (medium-high risk) — abstracting a coherent layer map from many files is high-order reasoning.
|
|
344
|
+
1. **Evidence classification** (high risk) — distinguishing `observed fact` from `strong inference` from `open question` requires calibrated self-awareness about certainty. Weaker models over-classify inferences as facts and skip `open question` tagging.
|
|
345
|
+
2. **Defect scan** (high risk) — the multi-pass scan demands domain-specific reasoning (concurrency, security, API contracts). Weaker models produce more false positives, miss subtle bugs, and over-report style issues as defects.
|
|
346
|
+
3. **Architecture synthesis** (medium-high risk) — abstracting a coherent layer map from many files is high-order reasoning.
|
|
287
347
|
4. **Structured output adherence** (medium risk) — filling templates correctly with all required sections and consistent formatting.
|
|
288
|
-
5. **Cross-phase coherence** (medium risk) — later phases build on earlier findings. Weak architecture
|
|
348
|
+
5. **Cross-phase coherence** (medium risk) — later phases build on earlier findings. Weak architecture compounds errors downstream.
|
|
289
349
|
|
|
290
|
-
### Recommended
|
|
350
|
+
### Recommended model tiers
|
|
291
351
|
|
|
292
|
-
|
|
|
352
|
+
| Tier | Examples | Recommended pipeline | Notes |
|
|
293
353
|
|---|---|---|---|
|
|
294
|
-
| Frontier | Claude Opus 4.6, Claude Sonnet 4.6 | Full-with-deep-audit (default)
|
|
295
|
-
| Strong mid-tier | Claude Haiku 4.5, GPT-4o | Lite (3-phase) | Architecture and contracts are solid. Skip defect scan — false
|
|
296
|
-
| Smaller / faster | GPT-4o-mini, Gemini Flash, small open-weight models | Architecture only | Fair structural overview. Multi-phase
|
|
297
|
-
|
|
298
|
-
### What to Expect Below Sonnet 4.6
|
|
299
|
-
|
|
300
|
-
- **Architecture phase**: Usually passable. The layer map and public surfaces will be present but may lack nuance in dependency direction and porting priorities.
|
|
301
|
-
- **Contracts and protocols**: Quality depends heavily on how well architecture was captured. Expect missing edge cases and less precise error-behavior documentation.
|
|
302
|
-
- **Defect scan**: Not recommended. The six specialized passes require strong domain reasoning. Weaker models produce noisy reports that cost more time to triage than they save.
|
|
303
|
-
- **Porting and reimplementation**: These synthesis phases amplify upstream quality. If earlier phases are weak, these will be too.
|
|
354
|
+
| **Frontier** | Claude Opus 4.6, Claude Sonnet 4.6 | Full-with-deep-audit (default) | Full quality on codebases up to ~100k tokens; the deep audit's semantic pass benefits most from frontier reasoning. |
|
|
355
|
+
| **Strong mid-tier** | Claude Haiku 4.5, GPT-4o | Lite (3-phase) | Architecture and contracts are solid. Skip defect scan — false-positive rate too high. |
|
|
356
|
+
| **Smaller / faster** | GPT-4o-mini, Gemini Flash, small open-weight models | Architecture only | Fair structural overview. Multi-phase runs produce significant quality loss. |
|
|
304
357
|
|
|
305
|
-
If you're testing a new model, start with `pipeline-architecture-only.yaml` on a codebase you already understand
|
|
358
|
+
If you're testing a new model, start with `pipeline-architecture-only.yaml` on a codebase you already understand and compare the output against your own knowledge. Fast signal on whether to trust the model with deeper phases.
|
|
306
359
|
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
CodeCartographer is a pure template — no CLI, no runtime, no dependencies. The "code" is structured Markdown and YAML files that tell an LLM what to analyze, in what order, and how to format the results.
|
|
310
|
-
|
|
311
|
-
The workflow is driven by flat files inside `.codecarto/`:
|
|
312
|
-
|
|
313
|
-
- **`GUIDE.md`** — the LLM entry point. Every session starts here.
|
|
314
|
-
- **`workflow/pipeline.yaml`** — phase definitions, dependencies, and output paths.
|
|
315
|
-
- **`workflow/status.yaml`** — mutable per-project state. Single source of truth for progress.
|
|
316
|
-
- **`workflow/VALIDATE.md`** — validation protocol run after every phase.
|
|
317
|
-
- **`findings/<phase>/SKILL.md`** — detailed analysis instructions per phase.
|
|
318
|
-
- **`templates/`** — output templates that enforce consistent structure.
|
|
360
|
+
---
|
|
319
361
|
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
### Multi-Session Workflows
|
|
323
|
-
|
|
324
|
-
Large codebases typically need one LLM session per phase. Start a new session and point it at `.codecarto/GUIDE.md` — it reads `status.yaml`, sees what's done, and picks up the next phase automatically. You don't need to explain what happened in previous sessions.
|
|
325
|
-
|
|
326
|
-
For follow-up sessions, you can also use `NEW_THREAD_BLURB.md` as a lighter entry point — it's a compact checklist that saves tokens by skipping the full guide.
|
|
327
|
-
|
|
328
|
-
### The Defect Scan
|
|
329
|
-
|
|
330
|
-
The defect-scan phase runs six sequential analysis passes: logic and correctness, error handling, concurrency, security, API contract violations, and configuration hazards. Each finding gets a severity (critical/high/medium/low) and a recommended action (fix before porting / port differently / leave behind).
|
|
331
|
-
|
|
332
|
-
## Design Principles
|
|
333
|
-
|
|
334
|
-
- **LLM-agnostic**: works with any model that can read/write files.
|
|
335
|
-
- **Phase-gated**: one phase per session, validated before advancing.
|
|
336
|
-
- **Single source of truth**: `status.yaml` tracks progress; no duplicated state.
|
|
337
|
-
- **Evidence-classified**: every finding is tagged as observed fact, strong inference, portability hazard, or open question.
|
|
338
|
-
- **Template-driven**: consistent output structure across projects and sessions.
|
|
339
|
-
- **Drop-in**: lives inside your repo as `.codecarto/`. No symlinking or copying source code.
|
|
340
|
-
|
|
341
|
-
## Repository Structure
|
|
362
|
+
## Repository structure
|
|
342
363
|
|
|
343
364
|
```
|
|
344
365
|
.codecarto/ # The drop-in template (Markdown + YAML).
|
|
@@ -346,30 +367,30 @@ The defect-scan phase runs six sequential analysis passes: logic and correctness
|
|
|
346
367
|
findings/
|
|
347
368
|
architecture/ # System structure, layers, dependency direction.
|
|
348
369
|
defect-scan/ # Multi-pass defect report with severity and actions.
|
|
349
|
-
passes/ # Per-category analysis instructions (6 pass files).
|
|
350
370
|
contracts/ # User-visible behavior, defaults, acceptance checks.
|
|
351
371
|
protocols/ # Event streams, state machines, persistence formats.
|
|
352
372
|
porting/ # Reverse-engineering synthesis bundle.
|
|
353
|
-
reimplementation-spec/ #
|
|
373
|
+
reimplementation-spec/ # Language-agnostic build spec.
|
|
354
374
|
scratch/ # Disposable analysis notes.
|
|
355
375
|
templates/ # Output structure templates.
|
|
356
|
-
workflow/ # Pipeline definitions, status, validation.
|
|
376
|
+
workflow/ # Pipeline definitions, status, validation, config.
|
|
377
|
+
closeouts/ # Per-session closeout files.
|
|
357
378
|
THREAD_LOG.md # Cross-session summary log.
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
379
|
+
dashboard.html # Generated; gitignored.
|
|
380
|
+
core/ # Pipeline state machine, validators, prompt assembly,
|
|
381
|
+
# dashboard renderer, usage log, orchestrator config.
|
|
382
|
+
extensions/codecarto/ # Pi extension surface (slash commands, widget,
|
|
383
|
+
# tool gating, dashboard writer + narrator).
|
|
384
|
+
mcp-server/ # MCP server surface (seven tools mirroring Pi commands).
|
|
361
385
|
tests/ # Invariant tests catching cross-wrapper drift.
|
|
362
386
|
docs/ # Roadmap, design notes.
|
|
363
|
-
CONTRIBUTING.md # How to contribute to CodeCartographer itself.
|
|
364
|
-
SECURITY.md # Security policy and reporting.
|
|
365
|
-
CHANGELOG.md # Version history.
|
|
366
387
|
```
|
|
367
388
|
|
|
368
|
-
|
|
389
|
+
The `.codecarto/.gitignore` excludes generated findings, scratch files, the dashboard, and the local usage / narration caches. Template files (workflow definitions, skills, output templates) are safe to commit so teammates can run their own analyses.
|
|
369
390
|
|
|
370
|
-
|
|
391
|
+
---
|
|
371
392
|
|
|
372
|
-
## For
|
|
393
|
+
## For automated agents
|
|
373
394
|
|
|
374
395
|
1. Load the active pipeline YAML and `workflow/status.yaml`.
|
|
375
396
|
2. Select the first phase whose status is not `complete` and whose dependencies are all `complete`.
|
|
@@ -377,9 +398,24 @@ The `.codecarto/.gitignore` excludes generated findings and scratch files by def
|
|
|
377
398
|
4. Write outputs to the declared paths. Run validation. Update status.
|
|
378
399
|
5. Repeat until all phases are complete. Set `current_phase` to `complete` when done.
|
|
379
400
|
|
|
401
|
+
The MCP server does steps 1–3 directly; the Pi extension wraps them as slash commands plus the parallel-sub-agent runner described above.
|
|
402
|
+
|
|
403
|
+
---
|
|
404
|
+
|
|
405
|
+
## Design principles
|
|
406
|
+
|
|
407
|
+
- **LLM-agnostic** — works with any model that can read and write files.
|
|
408
|
+
- **Phase-gated** — one phase per session, validated before advancing.
|
|
409
|
+
- **Single source of truth** — `status.yaml` tracks progress; no duplicated state.
|
|
410
|
+
- **Evidence-classified** — every finding tagged as observed fact, strong inference, portability hazard, or open question.
|
|
411
|
+
- **Template-driven** — consistent output structure across projects and sessions.
|
|
412
|
+
- **Drop-in** — lives inside your repo as `.codecarto/`. No symlinks, no copying source code, no runtime daemon.
|
|
413
|
+
|
|
414
|
+
---
|
|
415
|
+
|
|
380
416
|
## Contributing
|
|
381
417
|
|
|
382
|
-
Bug reports, feature requests, and pull requests are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, branch model, and the maintainer release process. All participants are expected to follow the [Code of Conduct](CODE_OF_CONDUCT.md). For security issues,
|
|
418
|
+
Bug reports, feature requests, and pull requests are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, branch model, and the maintainer release process. All participants are expected to follow the [Code of Conduct](CODE_OF_CONDUCT.md). For security issues, follow [SECURITY.md](SECURITY.md) instead of filing a public issue.
|
|
383
419
|
|
|
384
420
|
## License
|
|
385
421
|
|