codebase-snapshot 0.9.0 → 0.9.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +229 -1
- package/README.md +141 -127
- package/SECURITY.md +6 -2
- package/dist/bin.js +2544 -507
- package/dist/chunk-EWZYK3VX.js +2577 -0
- package/dist/{chunk-FJ4RQYY2.js → chunk-FCWD4LY7.js} +493 -362
- package/dist/cli.cjs +2451 -436
- package/dist/cli.d.cts +11 -1
- package/dist/cli.d.ts +11 -1
- package/dist/cli.js +68 -750
- package/dist/index.cjs +481 -360
- package/dist/index.d.cts +111 -5
- package/dist/index.d.ts +111 -5
- package/dist/index.js +1 -1
- package/dist/mcp.cjs +9398 -0
- package/dist/mcp.d.cts +399 -0
- package/dist/mcp.d.ts +399 -0
- package/dist/mcp.js +4 -0
- package/docs/agent.md +36 -0
- package/docs/cli.md +85 -14
- package/docs/codebase-token-budget.md +98 -0
- package/docs/configuration.md +12 -2
- package/docs/how-to-feed-a-repo-to-claude.md +4 -2
- package/docs/install.md +39 -11
- package/docs/mcp.md +149 -0
- package/docs/stale-context-is-your-bug.md +118 -0
- package/docs/troubleshooting.md +13 -0
- package/docs/vs-repomix.md +4 -2
- package/package.json +32 -16
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# How many tokens is a codebase?
|
|
2
|
+
|
|
3
|
+
Before choosing a part budget, know the order of magnitude. A token is roughly 3.5–4 bytes of
|
|
4
|
+
English prose; source code is denser, usually **3.3–3.7 bytes per token** because identifiers,
|
|
5
|
+
punctuation and indentation tokenise less efficiently than words.
|
|
6
|
+
|
|
7
|
+
The rule that survives contact with real repositories:
|
|
8
|
+
|
|
9
|
+
```text
|
|
10
|
+
tokens ≈ bytes / 3.5 ≈ lines × 11
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
## A measured example
|
|
14
|
+
|
|
15
|
+
This repository, measured rather than estimated:
|
|
16
|
+
|
|
17
|
+
| | |
|
|
18
|
+
| :--- | ---: |
|
|
19
|
+
| TypeScript files under `src/` | 57 |
|
|
20
|
+
| Lines | 14,626 |
|
|
21
|
+
| Bytes | 603,420 |
|
|
22
|
+
| **Estimated tokens** | **≈ 165,000** |
|
|
23
|
+
|
|
24
|
+
That single number decides the whole conversation. 165k tokens does not fit in an 8k window, it
|
|
25
|
+
almost exactly fills a 200k window leaving nothing for the question or the answer, and it fits
|
|
26
|
+
in a 1M window without effort.
|
|
27
|
+
|
|
28
|
+
## What the default part budget means
|
|
29
|
+
|
|
30
|
+
`maxPartBytes` defaults to **307,200 bytes (300 KB)** per part, which is roughly **84,000
|
|
31
|
+
tokens**. If your model's window is 128k, one part plus a prompt and a reply fits, and the second
|
|
32
|
+
part has to go in a separate conversation — which is exactly what the parts are for.
|
|
33
|
+
|
|
34
|
+
| `maxPartBytes` | ≈ tokens per part | Fits comfortably in |
|
|
35
|
+
| :--- | ---: | :--- |
|
|
36
|
+
| `60k` | ≈ 16k | 32k windows, small chat prompts |
|
|
37
|
+
| `120k` | ≈ 33k | 128k windows with room for a long answer |
|
|
38
|
+
| `300k` (default) | ≈ 84k | 200k windows, one part per session |
|
|
39
|
+
| `1mb` | ≈ 280k | 1M windows, or a summarising pass first |
|
|
40
|
+
|
|
41
|
+
Set it from the destination, not from taste:
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
codebase-snapshot generate --max-part-bytes 120k
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
## Sizing the window, not just the code
|
|
48
|
+
|
|
49
|
+
A window has to hold three things, and only the middle one is your snapshot:
|
|
50
|
+
|
|
51
|
+
```text
|
|
52
|
+
[ system prompt + instructions ] + [ snapshot parts ] + [ the answer ]
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
A rough allocation that works: **never spend more than 70% of the window on the snapshot**, and
|
|
56
|
+
leave at least 8k tokens for the reply. On a 128k window that caps you at ~82k tokens of
|
|
57
|
+
context — about one 300k-byte part.
|
|
58
|
+
|
|
59
|
+
If you are re-reading several parts in one conversation, the sum is what matters, not the
|
|
60
|
+
budget of any single part.
|
|
61
|
+
|
|
62
|
+
## When the repository does not fit
|
|
63
|
+
|
|
64
|
+
It usually does not. 165k tokens is a small library; a real application is five to twenty times
|
|
65
|
+
that. In order of preference:
|
|
66
|
+
|
|
67
|
+
1. **Scope the run, not the budget.** `include`/`exclude` patterns, or the wizard's "only
|
|
68
|
+
specific folders" option, cut more than any budget change. Packing the thing you are asking
|
|
69
|
+
about beats packing the repository.
|
|
70
|
+
2. **Let the layers do the work.** The detected preset groups files into layers, and
|
|
71
|
+
`reports/navigation.md` maps every file to the part it landed in — read the index, open one
|
|
72
|
+
part, instead of feeding everything.
|
|
73
|
+
3. **Raise the budget and lower the count.** One 300k part is easier to reason about than six
|
|
74
|
+
60k parts, and cheaper in prompt overhead.
|
|
75
|
+
4. **Summarise once, in a separate session.** Ask the model for a map of the parts, then use that
|
|
76
|
+
map to choose which parts to read in detail.
|
|
77
|
+
|
|
78
|
+
What not to do: silently truncate. A snapshot never exceeds its budget — a file too large for
|
|
79
|
+
the remaining space is cut with an explicit omission marker that names the budget, and above
|
|
80
|
+
`largeFileBytes` it becomes a head/tail excerpt rather than a full body.
|
|
81
|
+
|
|
82
|
+
## Checking your own number
|
|
83
|
+
|
|
84
|
+
The manifest records what was actually written, so you never have to guess:
|
|
85
|
+
|
|
86
|
+
```bash
|
|
87
|
+
codebase-snapshot report --format json \
|
|
88
|
+
| jq '{files: .manifest.counts.totalFiles, bytes: .manifest.counts.totalSizeBytes, parts: .manifest.counts.partsCount}'
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
`totalSizeBytes / 3.5` is your token estimate for the run. If it is far from the window you
|
|
92
|
+
intended to fill, the budget was not the problem — the selection was.
|
|
93
|
+
|
|
94
|
+
## Related
|
|
95
|
+
|
|
96
|
+
- [Budgeting LLM context](how-to-context-budget.md) — choosing ceilings per layer
|
|
97
|
+
- [Feeding a repo to an LLM](how-to-feed-a-repo-to-claude.md) — the end-to-end recipe
|
|
98
|
+
- [Configuration](configuration.md) — `maxPartBytes`, `largeFileBytes`, `maxMonolithBytes`
|
package/docs/configuration.md
CHANGED
|
@@ -139,6 +139,12 @@ not mistaken for a Node library.
|
|
|
139
139
|
| `docs` | 22 | ≥ 3 markdown files and no `package.json` | 100k | `.docusaurus`, `.vitepress/dist`, `site` |
|
|
140
140
|
| `generic` | 0 | anything else | 120k | — |
|
|
141
141
|
|
|
142
|
+
Each detector reports **the signal that fired**, not just a verdict: `next.config.ts`, `depends on
|
|
143
|
+
express`, `django in pyproject.toml`. The first run prints it — `Detected nextjs` / `Because
|
|
144
|
+
next.config.ts` — so a detection can be checked rather than taken on faith. `generic` is the
|
|
145
|
+
fallback, reached only when no detector matched, and the first run says so instead of pretending it
|
|
146
|
+
recognised the project.
|
|
147
|
+
|
|
142
148
|
A preset contributes layers, kinds, ignores and a recommended budget — what `init` writes as
|
|
143
149
|
`maxPartBytes`. Force one when detection is wrong:
|
|
144
150
|
|
|
@@ -403,8 +409,12 @@ preset's figure only takes effect once `init` has written it into a config.
|
|
|
403
409
|
npx codebase-snapshot init
|
|
404
410
|
```
|
|
405
411
|
|
|
406
|
-
Walks every tunable parameter, showing the detected value as the default
|
|
407
|
-
|
|
412
|
+
Walks every tunable parameter, showing the detected value as the default — press `enter` through
|
|
413
|
+
the lot and you get the recommended configuration.
|
|
414
|
+
|
|
415
|
+
The bare `snapshot` command starts the same wizard when the project has no config, but asks one
|
|
416
|
+
question first: take the recommended setup in a single keystroke, walk through each setting, or pick
|
|
417
|
+
a different preset. Every prompt has a flag, so the same setup works non-interactively:
|
|
408
418
|
|
|
409
419
|
```bash
|
|
410
420
|
npx codebase-snapshot init --yes
|
|
@@ -27,12 +27,14 @@ Everything below runs locally.
|
|
|
27
27
|
|
|
28
28
|
```bash
|
|
29
29
|
cd /path/to/your/project
|
|
30
|
-
|
|
30
|
+
snapshot # installed globally: one word, and it asks once
|
|
31
|
+
npx codebase-snapshot setup # or from a checkout with no global install
|
|
31
32
|
```
|
|
32
33
|
|
|
33
34
|
`setup` detects the project type from the repository, writes a commented `snapshot.config.mjs`, adds
|
|
34
35
|
the npm scripts and the devDependency, installs it, and generates. It does not overwrite a config you
|
|
35
|
-
already have. Use `--no-install` if you would rather not touch the lockfile.
|
|
36
|
+
already have. Use `--no-install` if you would rather not touch the lockfile. `npx codebase-snapshot
|
|
37
|
+
--autopilot` does the same thing with no questions at all, which is what you want in CI.
|
|
36
38
|
|
|
37
39
|
If you prefer to see each stage:
|
|
38
40
|
|
package/docs/install.md
CHANGED
|
@@ -8,31 +8,55 @@ Requires **Node.js 18 or newer**.
|
|
|
8
8
|
## One command
|
|
9
9
|
|
|
10
10
|
```bash
|
|
11
|
-
npx codebase-snapshot
|
|
11
|
+
npx codebase-snapshot --autopilot
|
|
12
12
|
```
|
|
13
13
|
|
|
14
14
|
Detects the project, writes the config (an existing one is kept), adds the npm scripts and the
|
|
15
15
|
devDependency, installs them with the package manager your lockfile names, and generates the first
|
|
16
|
-
snapshot. `--no-install` skips the install; `--dry-run` writes nothing.
|
|
17
|
-
|
|
16
|
+
snapshot. `--no-install` skips the install; `--dry-run` writes nothing. `setup` does the same as a
|
|
17
|
+
command word, and the bare `snapshot` (or `npx codebase-snapshot` with no arguments) does the
|
|
18
|
+
same after one question — see [the first run](#the-first-run). The sections below are the same
|
|
19
|
+
steps taken one at a time.
|
|
20
|
+
|
|
21
|
+
## The first run
|
|
22
|
+
|
|
23
|
+
With the package installed globally, the whole thing is one word:
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
snapshot
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
In a project with no configuration it names what it recognised and what it keyed on
|
|
30
|
+
(`Detected nextjs` / `Because next.config.ts`), then offers the recommended setup in one
|
|
31
|
+
keystroke or a walk through each setting with the defaults pre-filled. It writes the config,
|
|
32
|
+
adds the npm scripts and devDependency, installs, and generates.
|
|
33
|
+
|
|
34
|
+
In a project that already has a config — or one it has generated before — the same command just
|
|
35
|
+
generates, so nobody is walked through setup twice.
|
|
36
|
+
|
|
37
|
+
**It never guesses.** With no terminal (CI, a container, an agent) a first run exits `2` and
|
|
38
|
+
names the three ways forward instead of generating with settings nobody chose. `--autopilot` is
|
|
39
|
+
the answer for unattended use; naming the command (`npx codebase-snapshot generate`) is the
|
|
40
|
+
other, and it runs on the documented defaults exactly as before.
|
|
41
|
+
|
|
18
42
|
## One-off use
|
|
19
43
|
|
|
20
44
|
```bash
|
|
21
45
|
npx codebase-snapshot generate
|
|
22
46
|
```
|
|
23
47
|
|
|
24
|
-
This is the shortest path and needs no install.
|
|
25
|
-
`codebase-snapshot` and, once installed, as the short alias `snap`.
|
|
48
|
+
This is the shortest path and needs no install.
|
|
26
49
|
|
|
27
|
-
Pin the version when you need the same build on every machine
|
|
50
|
+
Pin the version when you need the same build on every machine. `--version` prints the exact
|
|
51
|
+
string to pin:
|
|
28
52
|
|
|
29
53
|
```bash
|
|
30
54
|
npx codebase-snapshot@latest generate
|
|
31
|
-
npx codebase-snapshot
|
|
55
|
+
npx codebase-snapshot --version
|
|
32
56
|
```
|
|
33
57
|
|
|
34
58
|
**Why the bare name works.** npm resolves which binary to run from the package's `bin`
|
|
35
|
-
entries. This package declares `codebase-snapshot` and `
|
|
59
|
+
entries. This package declares `codebase-snapshot` and `snapshot` pointing at the *same file*, so
|
|
36
60
|
npm treats them as aliases and can always resolve one. Keep them aliases: two *different*
|
|
37
61
|
executables under a scoped package name would make the bare form unresolvable.
|
|
38
62
|
|
|
@@ -57,8 +81,9 @@ Then run it through your client so the binary resolves from the project:
|
|
|
57
81
|
| pnpm | `pnpm exec codebase-snapshot generate` |
|
|
58
82
|
| bun | `bun run codebase-snapshot generate` |
|
|
59
83
|
|
|
60
|
-
`
|
|
61
|
-
|
|
84
|
+
`snapshot` works everywhere `codebase-snapshot` does — `npx snapshot` from a project dependency,
|
|
85
|
+
`pnpm exec snapshot`, `yarn snapshot`, and the bare word after a global install. Adding the script
|
|
86
|
+
to `package.json` is the most reproducible option:
|
|
62
87
|
|
|
63
88
|
```json
|
|
64
89
|
{
|
|
@@ -129,6 +154,8 @@ preset, and JSON configs are parsed rather than executed. See
|
|
|
129
154
|
npm install -g codebase-snapshot
|
|
130
155
|
```
|
|
131
156
|
|
|
157
|
+
This is what puts the short `snapshot` command on your `PATH`.
|
|
158
|
+
|
|
132
159
|
Convenient, but prefer pinning per project: snapshots are only reproducible if the generator
|
|
133
160
|
version is. A global install also shadows `npx` for this package — npm will run your global
|
|
134
161
|
copy instead of the published one, which is confusing when the two differ. That is exactly
|
|
@@ -156,7 +183,8 @@ A snapshot written by **3.0.0 or earlier** is refused rather than replaced: its
|
|
|
156
183
|
the directory. Delete the old snapshot directory and generate again.
|
|
157
184
|
|
|
158
185
|
See [CHANGELOG](../CHANGELOG.md) for breaking changes — 3.0.0 removed nothing, but it briefly
|
|
159
|
-
did in error; 4.0.0 restored the `snap` alias.
|
|
186
|
+
did in error; 4.0.0 restored the `snap` alias, and 0.9.2 replaced that alias with `snapshot`
|
|
187
|
+
when the version line was restarted.
|
|
160
188
|
|
|
161
189
|
## Uninstalling
|
|
162
190
|
|
package/docs/mcp.md
ADDED
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
# Run codebase-snapshot as an MCP server
|
|
2
|
+
|
|
3
|
+
`codebase-snapshot mcp` serves a repository to any [Model Context Protocol](https://modelcontextprotocol.io)
|
|
4
|
+
client over stdio. It is one command, no configuration, and it is the snapshot — not a directory
|
|
5
|
+
walk — that the client sees.
|
|
6
|
+
|
|
7
|
+
> Available from **0.9.4**. Run `npx codebase-snapshot --version` before wiring a client; anything
|
|
8
|
+
> older does not have the `mcp` command.
|
|
9
|
+
|
|
10
|
+
## Start it
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
npx -y codebase-snapshot@0.9.4 mcp
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
The server serves the directory it was started in. `--cwd <path>` points it elsewhere, and that
|
|
17
|
+
directory is the only one it will read.
|
|
18
|
+
|
|
19
|
+
A client needs two things: the command `npx` and the arguments above. The surrounding
|
|
20
|
+
configuration shape belongs to the client — the example below is the `mcpServers` form used by
|
|
21
|
+
Claude Desktop and Cursor:
|
|
22
|
+
|
|
23
|
+
```json
|
|
24
|
+
{
|
|
25
|
+
"mcpServers": {
|
|
26
|
+
"codebase-snapshot": {
|
|
27
|
+
"command": "npx",
|
|
28
|
+
"args": ["-y", "codebase-snapshot@0.9.4", "mcp", "--cwd", "/absolute/path/to/repo"]
|
|
29
|
+
}
|
|
30
|
+
}
|
|
31
|
+
}
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Claude Code registers the same thing from a shell:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
claude mcp add codebase-snapshot -- npx -y codebase-snapshot@0.9.4 mcp --cwd /absolute/path/to/repo
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
On Windows, wrap the command as `cmd /c npx` with the same arguments — `npx` is a `.cmd` shim
|
|
41
|
+
there, and clients that spawn without a shell cannot execute it directly.
|
|
42
|
+
|
|
43
|
+
## The seven tools
|
|
44
|
+
|
|
45
|
+
| Tool | Answers | Writes |
|
|
46
|
+
| :-- | :-- | :-- |
|
|
47
|
+
| `snapshot_status` | Is there a snapshot, when was it built, what does it cover? | no |
|
|
48
|
+
| `snapshot_plan` | What would a generate at this budget include, and what would it leave out? | no |
|
|
49
|
+
| `snapshot_generate` | Build it. | yes |
|
|
50
|
+
| `snapshot_validate` | Is the context the agent was given still current? | no |
|
|
51
|
+
| `snapshot_locate` | Which part holds this symbol, route or file? | no |
|
|
52
|
+
| `snapshot_read` | A window of one file, as the snapshot contains it. | no |
|
|
53
|
+
| `snapshot_explain` | Why is this path not in the snapshot? | no |
|
|
54
|
+
|
|
55
|
+
Six are declared `readOnlyHint`, which most clients use to run them without asking. Only
|
|
56
|
+
`snapshot_generate` prompts.
|
|
57
|
+
|
|
58
|
+
`snapshot_locate` is the one that makes the others usable: a twelve-part snapshot is hundreds of
|
|
59
|
+
kilobytes, and asking "which part holds authentication?" is cheaper than reading all twelve. It
|
|
60
|
+
searches the manifest's index — file paths, exported symbols, component names, routes, layers and
|
|
61
|
+
kinds — and returns the part each match lives in.
|
|
62
|
+
|
|
63
|
+
`snapshot_read` is windowed on purpose. It defaults to 200 lines and reports `nextFromLine` when
|
|
64
|
+
more remains, so an agent continues deliberately instead of pulling a whole part into a context
|
|
65
|
+
window it has not budgeted.
|
|
66
|
+
|
|
67
|
+
## Resources
|
|
68
|
+
|
|
69
|
+
Resources are the browse surface; tools are the act surface.
|
|
70
|
+
|
|
71
|
+
| URI | Contents |
|
|
72
|
+
| :-- | :-- |
|
|
73
|
+
| `snapshot://manifest` | Every included file with layer, kind, part and hash, plus every skip reason |
|
|
74
|
+
| `snapshot://parts/{index}` | One budgeted part, by 1-based index |
|
|
75
|
+
| `snapshot://reports/audit` | Secret findings, large files, parse diagnostics |
|
|
76
|
+
| `snapshot://reports/tree` | The directory tree of included files |
|
|
77
|
+
| `snapshot://reports/navigation` | File-to-part lookup |
|
|
78
|
+
| `snapshot://contract` | The machine-readable CLI contract |
|
|
79
|
+
| `snapshot://skill` | The agent skill, as one read |
|
|
80
|
+
|
|
81
|
+
Part resources carry their size in the description, so a client choosing between them can see that
|
|
82
|
+
one is 300 kB before attaching it.
|
|
83
|
+
|
|
84
|
+
## Prompts
|
|
85
|
+
|
|
86
|
+
Two, both about order of operations rather than features: `orient` maps the repository before
|
|
87
|
+
anything is read, and `keep-current` validates before trusting earlier context and again after
|
|
88
|
+
making changes.
|
|
89
|
+
|
|
90
|
+
## What it will not do
|
|
91
|
+
|
|
92
|
+
- **Read outside its root.** `--cwd` is the only directory that is readable. A `root` argument can
|
|
93
|
+
name a sub-directory of it and nothing else; symlinks are resolved before the check.
|
|
94
|
+
- **Emit a sensitive file.** `.env`, `*.pem` and the rest stay out, and `includeSensitiveFiles` is
|
|
95
|
+
not a tool argument — a prompt cannot ask for it. Reads come from the emitted snapshot, so
|
|
96
|
+
redaction applies to everything the client sees.
|
|
97
|
+
- **Install or configure.** `snapshot_generate` writes the snapshot directory and nothing else. It
|
|
98
|
+
never runs a package manager and never writes a config file.
|
|
99
|
+
- **Touch the network.** The server makes no outbound connections, and there is no HTTP transport.
|
|
100
|
+
|
|
101
|
+
## Both protocol eras
|
|
102
|
+
|
|
103
|
+
Revision `2026-07-28` removed the handshake that every client in use today still performs. The
|
|
104
|
+
server speaks both and answers whichever arrives first.
|
|
105
|
+
|
|
106
|
+
| Era | Revisions | Opening move |
|
|
107
|
+
| :-- | :-- | :-- |
|
|
108
|
+
| Legacy | `2025-11-25`, `2025-06-18` | `initialize`, then `notifications/initialized` |
|
|
109
|
+
| Modern | `2026-07-28` | `server/discover`, or `_meta` on every request |
|
|
110
|
+
|
|
111
|
+
The two disagree about an unsupported version, and the difference is observable: a legacy client is
|
|
112
|
+
negotiated *down* to a revision the server speaks, while a modern client receives `-32022` with the
|
|
113
|
+
list of supported revisions so it can retry.
|
|
114
|
+
|
|
115
|
+
## Why there is no SDK
|
|
116
|
+
|
|
117
|
+
`@modelcontextprotocol/sdk` is 4.5 MB unpacked and brings Express, Hono, Ajv, JOSE and Zod with it.
|
|
118
|
+
The transport here is newline-delimited JSON-RPC, which is roughly 400 lines of readable code, and
|
|
119
|
+
the package's only runtime dependency remains `typescript`. The parity test — a live `tools/list`
|
|
120
|
+
compared against `describe --format json` — is what keeps the protocol implementation honest, not
|
|
121
|
+
the presence of a library.
|
|
122
|
+
|
|
123
|
+
## Performance
|
|
124
|
+
|
|
125
|
+
Generation is synchronous, so a tool call blocks the server for its duration. Both numbers are
|
|
126
|
+
measured, not estimated:
|
|
127
|
+
|
|
128
|
+
| Repository | Files | `snapshot_generate` |
|
|
129
|
+
| :-- | --: | --: |
|
|
130
|
+
| This project | 127 | 1.1 s |
|
|
131
|
+
| A synthetic tree | 3,000 | 7.5 s |
|
|
132
|
+
|
|
133
|
+
Both are under any client's request timeout. If a repository large enough to exceed yours shows up,
|
|
134
|
+
`snapshot_plan` gives the same numbers without writing, and the fix — moving generation to a worker
|
|
135
|
+
thread — is tracked rather than assumed.
|
|
136
|
+
|
|
137
|
+
## Troubleshooting
|
|
138
|
+
|
|
139
|
+
**The client lists no tools.** The server writes JSON-RPC to stdout and nothing else; every note goes
|
|
140
|
+
to stderr. Check the client's server log — the first line says which directory is being served.
|
|
141
|
+
|
|
142
|
+
**`Unknown tool` or a missing command.** The published version is older than 0.9.4. Pin the exact
|
|
143
|
+
version in the client arguments.
|
|
144
|
+
|
|
145
|
+
**`Unknown resource` for a part.** Resources are computed from the manifest, so nothing but the
|
|
146
|
+
static set exists before the first generate. Call `snapshot_status`, then `snapshot_generate`.
|
|
147
|
+
|
|
148
|
+
**A file is missing from `snapshot_locate`.** It is excluded, and `snapshot_explain` will name the
|
|
149
|
+
reason — an ignore rule, the preset, a sensitive-file pattern, or your own config.
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
# Stale LLM context is a bug
|
|
2
|
+
|
|
3
|
+
A snapshot is a copy. The code keeps moving. Nothing in the copy says so, and nothing in the
|
|
4
|
+
model can notice.
|
|
5
|
+
|
|
6
|
+
That is the whole failure mode: you hand a model a faithful description of your repository on
|
|
7
|
+
Monday, change four files on Wednesday, and on Thursday ask about a function you have already
|
|
8
|
+
deleted. The answer is confident, plausible and about a repository that no longer exists.
|
|
9
|
+
|
|
10
|
+
```text
|
|
11
|
+
you ──▶ snapshot (Monday) ──▶ model ──▶ answer about Monday's code
|
|
12
|
+
code moves (Wednesday) ────┘
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
There is no error message, because nothing failed. The model did exactly what it was given.
|
|
16
|
+
|
|
17
|
+
## Why "just regenerate it" is not a control
|
|
18
|
+
|
|
19
|
+
Regenerating is cheap — it takes a second. The problem is that you have to *know* to do it, and
|
|
20
|
+
nobody notices a stale snapshot until an answer is wrong. A habit is not a guard, and a README
|
|
21
|
+
line that says "re-run this before asking" is a habit.
|
|
22
|
+
|
|
23
|
+
What you need is not more discipline. It is a comparison that can fail.
|
|
24
|
+
|
|
25
|
+
## Three things that make drift visible
|
|
26
|
+
|
|
27
|
+
### 1. An identity for the copy
|
|
28
|
+
|
|
29
|
+
Every run writes `manifest.fingerprint`, a hash of what was included, derived from the content
|
|
30
|
+
of every file. Two repositories that differ by one byte have different fingerprints; two runs
|
|
31
|
+
over the same tree on different machines have the same one.
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
codebase-snapshot report --format json | jq -r .manifest.fingerprint
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Quote that value wherever the snapshot is used — in the prompt, in the PR description, in the
|
|
38
|
+
ticket. It turns "the snapshot we sent" into a value that can be compared later.
|
|
39
|
+
|
|
40
|
+
### 2. A comparison you can run
|
|
41
|
+
|
|
42
|
+
`validate` re-scans the repository, re-hashes each file the snapshot claims to contain, and
|
|
43
|
+
compares. It exits `1` when the code has moved, and `0` when the snapshot still describes what
|
|
44
|
+
is on disk.
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
codebase-snapshot validate
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
```text
|
|
51
|
+
✖ Snapshot does not match the repository:
|
|
52
|
+
- modified: src/parser/partition.ts
|
|
53
|
+
- added: src/presets/evidence.ts
|
|
54
|
+
- removed: src/legacy/old-partitioner.ts
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
That is the difference between a copy and an artefact. A copy is trusted; an artefact is
|
|
58
|
+
checked.
|
|
59
|
+
|
|
60
|
+
### 3. A record of what was left out
|
|
61
|
+
|
|
62
|
+
`manifest.skipped[]` names every path the snapshot did not include, with a reason for each:
|
|
63
|
+
|
|
64
|
+
| Reason | Meaning |
|
|
65
|
+
| :--- | :--- |
|
|
66
|
+
| `gitignore` | the repository's own rules excluded it |
|
|
67
|
+
| `default-ignore` | `node_modules`, build output, VCS metadata |
|
|
68
|
+
| `sensitive-file` | `.env`, `*.pem`, `.npmrc` — excluded unless explicitly requested |
|
|
69
|
+
| `binary` | not text, so not useful as context |
|
|
70
|
+
| `outside-root` | a symlink pointing out of the repository |
|
|
71
|
+
| `nested-repository` | a submodule or nested checkout |
|
|
72
|
+
| `read-error` | it could not be read, and the run says so instead of dropping it quietly |
|
|
73
|
+
|
|
74
|
+
When an answer is wrong because the model never saw a file, this is the list that tells you
|
|
75
|
+
whether the file was excluded on purpose or by accident.
|
|
76
|
+
|
|
77
|
+
## Gate it
|
|
78
|
+
|
|
79
|
+
The comparison is only a guard if something runs it. In CI, after whatever changes the tree:
|
|
80
|
+
|
|
81
|
+
```yaml
|
|
82
|
+
- name: Snapshot is still current
|
|
83
|
+
run: npx codebase-snapshot validate
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
A failure means the context you are about to rely on describes a different repository. Whether
|
|
87
|
+
that blocks the pipeline or just warns is a policy question; having it at all is not.
|
|
88
|
+
|
|
89
|
+
The same gate works for a long-running agent: run `validate` before a session that will read
|
|
90
|
+
the snapshot, and regenerate when it fails.
|
|
91
|
+
|
|
92
|
+
## What this does not tell you
|
|
93
|
+
|
|
94
|
+
Worth being precise about the limits, because "validate" sounds like more than it is:
|
|
95
|
+
|
|
96
|
+
- **It compares files, not meaning.** A refactor that preserves every byte proves nothing is
|
|
97
|
+
wrong — correctly, because nothing is.
|
|
98
|
+
- **It cannot see what the model did with the text.** A current snapshot can still be
|
|
99
|
+
misunderstood, summarised badly, or ignored.
|
|
100
|
+
- **It says nothing about whether the snapshot was a good selection.** Use `explain <path>` to
|
|
101
|
+
see why one path was included or skipped, and `report` for the shape of the whole run.
|
|
102
|
+
- **It does not follow history.** If you keep three snapshots from three branches, each one
|
|
103
|
+
validates against the branch it was generated from.
|
|
104
|
+
|
|
105
|
+
## When this is overkill
|
|
106
|
+
|
|
107
|
+
If you generate a snapshot, paste it into a chat, and throw it away in the same hour, you do not
|
|
108
|
+
need any of this — `snapshot` alone is the right amount of ceremony.
|
|
109
|
+
|
|
110
|
+
The controls start to pay for themselves the moment a snapshot outlives the session that made
|
|
111
|
+
it: a checked-in context directory, a scheduled job, a review bot, an agent that runs for days,
|
|
112
|
+
or a team where the person who generated the snapshot is not the person who reads it.
|
|
113
|
+
|
|
114
|
+
## Related
|
|
115
|
+
|
|
116
|
+
- [Budgeting LLM context](how-to-context-budget.md) — choosing a part size that fits the window
|
|
117
|
+
- [Snapshot output](output.md) — the manifest fields this page refers to
|
|
118
|
+
- [CLI reference](cli.md#validate--check) — `validate` in full
|
package/docs/troubleshooting.md
CHANGED
|
@@ -49,6 +49,9 @@ column is in the [CLI reference](cli.md#options).
|
|
|
49
49
|
| Message | Fix |
|
|
50
50
|
| :--- | :--- |
|
|
51
51
|
| `Unknown command "frobnicate". Expected one of: …` | Use one of the commands in the [CLI reference](cli.md). |
|
|
52
|
+
| `No configuration found in … and this session is not interactive, so nothing was generated.` | A first run and no terminal to ask its question. Nothing is wrong with the repository: say which setup you want — `--autopilot` for the full setup, `init --yes` then `generate` for the steps, or `generate` for the documented defaults. See [the first run](cli.md#the-first-run). |
|
|
53
|
+
| `--autopilot sets a project up and writes a config; --no-config ignores config files. Pass one or the other.` | The two flags contradict each other. Drop one. |
|
|
54
|
+
| `--no-install only applies to \`setup\`, or to \`generate --autopilot\`.` | The install only happens during a setup. Drop the flag, or add `--autopilot`. |
|
|
52
55
|
| `--preset does not apply to \`generate\` and was not applied. It is accepted by: \`init\`. To scope a run, set include/excludePatterns/preset in snapshot.config.ts (or run \`codebase-snapshot init\` to generate one).` | `--preset`, `--include`, `--exclude`, `--yes` and `--force` configure `init`, which writes them into the config. Set them at generation time in `snapshot.config.ts` instead. |
|
|
53
56
|
| `--install does not apply to \`generate\` and was not applied. It is accepted by: \`skill\`.` | `--install` and `--target` configure `skill` only. |
|
|
54
57
|
| `--cwd directory does not exist: …` / `--cwd is not a directory: …` | Point `--cwd` at an existing repository directory. |
|
|
@@ -171,6 +174,16 @@ The output paths are unsafe: `outDir` or `tmpDir` is the working directory, an a
|
|
|
171
174
|
filesystem root, or they contain one another. The message prints the resolved paths so you
|
|
172
175
|
can see why. Use a subdirectory.
|
|
173
176
|
|
|
177
|
+
### `Nothing to snapshot: … has no files.`
|
|
178
|
+
|
|
179
|
+
```
|
|
180
|
+
❌ Nothing to snapshot: this directory has no files. Run this inside the repository you want to snapshot, or pass --cwd.
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
The directory is empty apart from `.git` (or `.hg`, `.svn`, `.DS_Store`, `Thumbs.db`). Setting the
|
|
184
|
+
project up would install a devDependency and add scripts for a repository with nothing in it, so
|
|
185
|
+
`--autopilot` refuses too. Run the command from the repository, or point `--cwd` at it.
|
|
186
|
+
|
|
174
187
|
### Budget errors
|
|
175
188
|
|
|
176
189
|
A budget set in a **config file** exits `1`, and names the setting as you wrote it:
|
package/docs/vs-repomix.md
CHANGED
|
@@ -29,7 +29,7 @@ If you want the most popular, most integrated tool: **use repomix.**
|
|
|
29
29
|
| Reproducible output | Parts are byte-identical across runs; a fingerprint covers them | Content is stable; no published determinism guarantee |
|
|
30
30
|
| Staleness check | `validate` re-hashes the repository and exits non-zero on drift | Not a feature |
|
|
31
31
|
| Remote repositories | No — operates on a local checkout | **Yes** — packs a GitHub URL directly |
|
|
32
|
-
| MCP server |
|
|
32
|
+
| MCP server | **Yes** — seven tools over stdio, and it serves the validated snapshot rather than the working tree | **Yes** — packs a repository, including by URL |
|
|
33
33
|
| Secret handling | 30 default rules, value-span masking, per-line audit table | Secretlint integration |
|
|
34
34
|
| Language awareness | 25 presets, AST or structural summaries labelled by confidence | Tree-sitter parsing with a compression mode |
|
|
35
35
|
| Adoption | New | **Mature** — large user base, many integrations |
|
|
@@ -41,7 +41,9 @@ If you want the most popular, most integrated tool: **use repomix.**
|
|
|
41
41
|
repomix is built around that workflow.
|
|
42
42
|
- **You need a remote URL.** `repomix --remote https://github.com/owner/repo` works without a clone.
|
|
43
43
|
This project has no equivalent.
|
|
44
|
-
- **You want MCP
|
|
44
|
+
- **You want MCP against a remote URL.** repomix's server can pack a repository by URL. This one
|
|
45
|
+
serves a local checkout, and refuses to read outside it. Both ship a server; ours is narrower on
|
|
46
|
+
purpose, and it can tell the agent whether the snapshot it is reading is still current.
|
|
45
47
|
- **You want the ecosystem.** More stars, more tutorials, more people who have already hit your
|
|
46
48
|
problem.
|
|
47
49
|
- **You are packing something enormous and prefer compression to budgets.** repomix's tree-sitter
|
package/package.json
CHANGED
|
@@ -1,28 +1,31 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "codebase-snapshot",
|
|
3
|
-
"version": "0.9.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "0.9.4",
|
|
4
|
+
"description": "Pack a codebase for an LLM into budgeted Markdown parts, with a manifest of every file and skip, and validate to prove the snapshot is current.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"codebase",
|
|
7
7
|
"snapshot",
|
|
8
8
|
"llm",
|
|
9
|
-
"
|
|
10
|
-
"
|
|
9
|
+
"claude",
|
|
10
|
+
"chatgpt",
|
|
11
|
+
"cursor",
|
|
11
12
|
"context-window",
|
|
12
|
-
"context-engineering",
|
|
13
|
-
"codebase-snapshot",
|
|
14
|
-
"codebase-to-prompt",
|
|
15
13
|
"repo-to-llm",
|
|
14
|
+
"codebase-to-prompt",
|
|
16
15
|
"markdown",
|
|
17
|
-
"
|
|
16
|
+
"token-efficiency",
|
|
17
|
+
"context-engineering",
|
|
18
18
|
"ai-agents",
|
|
19
|
-
"
|
|
20
|
-
"
|
|
19
|
+
"mcp",
|
|
20
|
+
"model-context-protocol",
|
|
21
|
+
"mcp-server",
|
|
22
|
+
"prompt-engineering",
|
|
21
23
|
"partitioner",
|
|
22
24
|
"ast",
|
|
25
|
+
"codebase-packer",
|
|
23
26
|
"code-analysis",
|
|
24
|
-
"
|
|
25
|
-
"codebase-
|
|
27
|
+
"ai",
|
|
28
|
+
"codebase-snapshot"
|
|
26
29
|
],
|
|
27
30
|
"author": "Ankur Halder <ankur.halder12345@gmail.com>",
|
|
28
31
|
"license": "MIT",
|
|
@@ -44,7 +47,7 @@
|
|
|
44
47
|
"sideEffects": false,
|
|
45
48
|
"bin": {
|
|
46
49
|
"codebase-snapshot": "./dist/bin.js",
|
|
47
|
-
"
|
|
50
|
+
"snapshot": "./dist/bin.js"
|
|
48
51
|
},
|
|
49
52
|
"exports": {
|
|
50
53
|
".": {
|
|
@@ -67,6 +70,16 @@
|
|
|
67
70
|
"default": "./dist/cli.cjs"
|
|
68
71
|
}
|
|
69
72
|
},
|
|
73
|
+
"./mcp": {
|
|
74
|
+
"import": {
|
|
75
|
+
"types": "./dist/mcp.d.ts",
|
|
76
|
+
"default": "./dist/mcp.js"
|
|
77
|
+
},
|
|
78
|
+
"require": {
|
|
79
|
+
"types": "./dist/mcp.d.cts",
|
|
80
|
+
"default": "./dist/mcp.cjs"
|
|
81
|
+
}
|
|
82
|
+
},
|
|
70
83
|
"./package.json": "./package.json"
|
|
71
84
|
},
|
|
72
85
|
"files": [
|
|
@@ -91,11 +104,14 @@
|
|
|
91
104
|
"test:coverage": "node --test --experimental-test-coverage tests/*.test.js",
|
|
92
105
|
"prepublishOnly": "npm run build",
|
|
93
106
|
"publish:create": "npm publish packages/create --access public",
|
|
94
|
-
"prepack": "npm run build",
|
|
107
|
+
"prepack": "npm run sync-docs && npm run build",
|
|
95
108
|
"pack-check": "node scripts/pack-check.mjs",
|
|
96
|
-
"
|
|
109
|
+
"sync-docs": "node scripts/sync-docs.mjs",
|
|
110
|
+
"site": "npm run sync-docs && node scripts/build-site.mjs",
|
|
111
|
+
"audit-site": "node scripts/audit-site.mjs",
|
|
112
|
+
"readme-lint": "node scripts/readme-lint.mjs",
|
|
97
113
|
"seo-rank": "node scripts/seo-rank.mjs",
|
|
98
|
-
"verify": "npm run check && npm run pack-check"
|
|
114
|
+
"verify": "npm run check && npm run sync-docs -- --check && npm run site && npm run audit-site && npm run readme-lint && npm run pack-check"
|
|
99
115
|
},
|
|
100
116
|
"dependencies": {
|
|
101
117
|
"typescript": "^5.0.0"
|