codebase-snapshot 0.0.0-stage → 4.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +1242 -0
- package/LICENSE +21 -0
- package/README.md +276 -2
- package/SECURITY.md +110 -0
- package/dist/bin.js +9840 -0
- package/dist/chunk-SSCFLILW.js +7885 -0
- package/dist/cli.cjs +9822 -0
- package/dist/cli.d.cts +124 -0
- package/dist/cli.d.ts +124 -0
- package/dist/cli.js +1931 -0
- package/dist/index.cjs +7973 -0
- package/dist/index.d.cts +710 -0
- package/dist/index.d.ts +710 -0
- package/dist/index.js +3 -0
- package/dist/types-Ca4PmkiQ.d.cts +491 -0
- package/dist/types-Ca4PmkiQ.d.ts +491 -0
- package/docs/agent.md +135 -0
- package/docs/cli.md +437 -0
- package/docs/configuration.md +424 -0
- package/docs/how-to-context-budget.md +90 -0
- package/docs/how-to-feed-a-repo-to-claude.md +122 -0
- package/docs/install.md +196 -0
- package/docs/output.md +264 -0
- package/docs/troubleshooting.md +384 -0
- package/docs/vs-code2prompt.md +66 -0
- package/docs/vs-gitingest.md +69 -0
- package/docs/vs-repomix.md +83 -0
- package/package.json +112 -4
|
@@ -0,0 +1,424 @@
|
|
|
1
|
+
# Configuration reference
|
|
2
|
+
|
|
3
|
+
Every setting, every default, and what each one changes. For commands and options see the
|
|
4
|
+
[CLI reference](cli.md).
|
|
5
|
+
|
|
6
|
+
Nothing here is required: with no config at all the tool runs on the defaults below.
|
|
7
|
+
|
|
8
|
+
## Contents
|
|
9
|
+
|
|
10
|
+
- [Config files](#config-files)
|
|
11
|
+
- [Scoping: which files get in](#scoping-which-files-get-in)
|
|
12
|
+
- [Project presets](#project-presets)
|
|
13
|
+
- [Classification: layers and kinds](#classification-layers-and-kinds)
|
|
14
|
+
- [Sizing: byte budgets](#sizing-byte-budgets)
|
|
15
|
+
- [Artifacts: what gets written](#artifacts-what-gets-written)
|
|
16
|
+
- [Secrets: redaction](#secrets-redaction)
|
|
17
|
+
- [Output directories](#output-directories)
|
|
18
|
+
- [Hooks](#hooks)
|
|
19
|
+
- [Complete example](#complete-example)
|
|
20
|
+
- [Interactive setup](#interactive-setup)
|
|
21
|
+
|
|
22
|
+
## Config files
|
|
23
|
+
|
|
24
|
+
The first match wins:
|
|
25
|
+
|
|
26
|
+
```
|
|
27
|
+
snapshot.config.ts
|
|
28
|
+
snapshot.config.js
|
|
29
|
+
snapshot.config.mjs
|
|
30
|
+
snapshot.config.cjs
|
|
31
|
+
snapshot.config.json
|
|
32
|
+
.snapshotrc / .snapshotrc.json
|
|
33
|
+
package.json → "snapshot"
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
`snapshot.config.json` accepts comments. `package.json`'s `"snapshot"` key is read only when
|
|
37
|
+
no dedicated config file exists, and it must be an object (a wrong type is a warning, since
|
|
38
|
+
other tools use that key too).
|
|
39
|
+
|
|
40
|
+
The extension decides how the file is loaded. `.cjs` is always CommonJS and must use
|
|
41
|
+
`module.exports`; `.mjs` and `.ts` are always ESM and must use `export default`. `.js` follows
|
|
42
|
+
the package's `type` field — `module` means ESM, `commonjs` means CommonJS; with no `type`
|
|
43
|
+
Node decides by detecting the syntax, so `export default` works on current Node but is not
|
|
44
|
+
portable to versions without detection. Getting it wrong is the `Could not load …` error
|
|
45
|
+
above, not a silent fallback.
|
|
46
|
+
|
|
47
|
+
A config file that exists but cannot be parsed is a **hard error** (exit `1`, nothing
|
|
48
|
+
written). Silently ignoring it would produce a snapshot with none of your settings.
|
|
49
|
+
|
|
50
|
+
A `.ts` config needs Node ≥ 22.6 **and** an ES-module project, because Node applies the
|
|
51
|
+
package's `type` field to a `.ts` file. `codebase-snapshot init` picks the right filename for
|
|
52
|
+
the project — `.ts` when both conditions hold, `.mjs` otherwise — and writes a plain object
|
|
53
|
+
with no imports so it loads even when the package is only used through `npx`.
|
|
54
|
+
|
|
55
|
+
The snippets below show an `export default { … }` object for brevity. `init` writes it as
|
|
56
|
+
`const config = { … }; export default config;` because the anonymous form is reported by
|
|
57
|
+
`import/no-anonymous-default-export`, which `eslint-config-next` enables by default — both load
|
|
58
|
+
identically, so use whichever keeps your linter quiet.
|
|
59
|
+
|
|
60
|
+
CLI flags override the config for a single run (`--max-part-bytes`, `--out-dir`, …).
|
|
61
|
+
`--no-config` goes further and ignores every config file, so the run uses the built-in
|
|
62
|
+
defaults and the project's detected preset. A code config is executed when it is loaded, so
|
|
63
|
+
`--no-config` is also the way to snapshot a repository whose config you do not want to run —
|
|
64
|
+
see [the trust boundary](../SECURITY.md#trust-boundary).
|
|
65
|
+
|
|
66
|
+
## Scoping: which files get in
|
|
67
|
+
|
|
68
|
+
```js
|
|
69
|
+
export default {
|
|
70
|
+
include: [], // allow-list; empty means "everything"
|
|
71
|
+
excludePatterns: [], // extra excludes, applied on top of the preset's
|
|
72
|
+
discovery: "auto", // "auto" | "git" | "walk" — how files are found
|
|
73
|
+
};
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
- `include` is an allow-list of globs. When non-empty, only matching files are considered.
|
|
77
|
+
- `exclude` is accepted as an alias of `excludePatterns` and means the same thing; the resolved
|
|
78
|
+
config always records it under `excludePatterns`.
|
|
79
|
+
- `excludePatterns` removes files everywhere. Presets contribute their own ecosystem ignores
|
|
80
|
+
on top; those are not removable, only addable to.
|
|
81
|
+
- `discovery` chooses how the file list is produced, and is recorded in `manifest.config`, so
|
|
82
|
+
changing it shows up as config drift:
|
|
83
|
+
- `"auto"` (default) — list through `git ls-files` when git can do it; if a `.git` directory
|
|
84
|
+
exists but git cannot list the files, refuse rather than silently switch methods.
|
|
85
|
+
- `"git"` — require git; a directory without a usable repository is an error.
|
|
86
|
+
- `"walk"` — scan the tree and read `.gitignore` files directly (nested ones included), still
|
|
87
|
+
skipping credential files. Use it when a `.git` directory cannot be listed. It cannot
|
|
88
|
+
reproduce git's own ignore sources (`.git/info/exclude`, your global `core.excludesFile`),
|
|
89
|
+
git's inclusion of files that are tracked although a `.gitignore` now matches them, or its
|
|
90
|
+
treatment of a submodule as a single entry. The same setting is available per run as
|
|
91
|
+
[`--discovery`](cli.md#options).
|
|
92
|
+
|
|
93
|
+
Built-in ignores always apply: `.git`, `node_modules`, `.next`, `dist`, `build`, lockfiles,
|
|
94
|
+
minified assets and binaries. Sensitive files are handled separately (see
|
|
95
|
+
[Secrets](#secrets-redaction)).
|
|
96
|
+
|
|
97
|
+
Every path the scanner discovers is recorded in `manifest.skipped[]` with the reason and the
|
|
98
|
+
rule that matched — nothing it sees is dropped silently, including a `.gitignore`d path in a
|
|
99
|
+
repository where git does the filtering. See the [`skipped[]`
|
|
100
|
+
reasons](output.md#skipped). To debug one path:
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
npx codebase-snapshot explain src/some/file.ts
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
## Project presets
|
|
107
|
+
|
|
108
|
+
Detection reads the repository **once** and picks the highest-priority match, so a Next.js
|
|
109
|
+
app inside a pnpm workspace is still `nextjs`, not `monorepo`. `peerDependencies` and
|
|
110
|
+
`optionalDependencies` count as dependencies. A bare `package.json` ranks below any
|
|
111
|
+
definitive language manifest, so a Go or Rust repository with a `package.json` for tooling is
|
|
112
|
+
not mistaken for a Node library.
|
|
113
|
+
|
|
114
|
+
| Preset | Priority | Detected from | Budget | Ecosystem ignores |
|
|
115
|
+
| :--- | ---: | :--- | :--- | :--- |
|
|
116
|
+
| `nextjs` | 100 | `next.config.*`, or `next` | 300k | `.svelte-kit`, `.astro`, `.vite`, `.output` |
|
|
117
|
+
| `nuxt` | 98 | `nuxt.config.*`, or `nuxt` | 200k | `.nuxt`, `.output` |
|
|
118
|
+
| `sveltekit` | 96 | `svelte.config.*`, `@sveltejs/kit` | 200k | `.svelte-kit`, `.vercel` |
|
|
119
|
+
| `remix` | 94 | `@remix-run/*` | 200k | `.cache`, `public/build` |
|
|
120
|
+
| `astro` | 92 | `astro.config.*`, `astro` | 150k | `.astro`, `dist` |
|
|
121
|
+
| `angular` | 90 | `angular.json`, `@angular/core` | 200k | `.angular`, `dist` |
|
|
122
|
+
| `solid` | 88 | `solid-js`, `solid-start` | 120k | `.solid`, `dist` |
|
|
123
|
+
| `vue-vite` | 86 | `vue` + `vite`/`vite.config.*` | 120k | `.vite`, `dist` |
|
|
124
|
+
| `react-vite` | 84 | `react` + `vite`/`vite.config.*` | 120k | `.vite`, `dist` |
|
|
125
|
+
| `nest` | 80 | `@nestjs/core` | 200k | `dist`, `node_modules` |
|
|
126
|
+
| `node-express` | 78 | `express`, `fastify`, `koa`, `@hapi/hapi` | 200k | `dist` |
|
|
127
|
+
| `monorepo` | 70 | `pnpm-workspace.yaml`, `nx.json`, `turbo.json`, `lerna.json`, `"workspaces"` | 200k | `.turbo`, `.nx` |
|
|
128
|
+
| `python-django` | 56 | `manage.py`, django in `pyproject.toml`/`requirements.txt` | 150k | `__pycache__`, `.venv`, `venv`, `.mypy_cache`, `.pytest_cache`, `.tox`, `.eggs`, `staticfiles`, `media` |
|
|
129
|
+
| `python-fastapi` | 54 | fastapi or flask in `pyproject.toml`/`requirements.txt` | 150k | `__pycache__`, `.venv`, `venv`, `.mypy_cache`, `.pytest_cache`, `.tox`, `.eggs` |
|
|
130
|
+
| `go` | 50 | `go.mod` | 200k | `vendor`, `testdata` |
|
|
131
|
+
| `rust` | 48 | `Cargo.toml` | 200k | `target` |
|
|
132
|
+
| `java-spring` | 46 | `pom.xml`, `build.gradle`, `build.gradle.kts` | 200k | `.gradle`, `out`, `target` |
|
|
133
|
+
| `node-lib` | 45 | any `package.json` | 120k | `dist` |
|
|
134
|
+
| `dotnet` | 44 | `*.csproj`, `*.sln`, `global.json` | 200k | `bin`, `obj`, `.vs` |
|
|
135
|
+
| `php-laravel` | 42 | `artisan`, laravel in `composer.json`, `composer.json` | 150k | `vendor`, `storage`, `bootstrap/cache`, `public/build` |
|
|
136
|
+
| `ruby-rails` | 40 | `Gemfile`, `config.ru`, `Rakefile` | 150k | `.bundle`, `log`, `tmp`, `vendor/bundle`, `public/assets` |
|
|
137
|
+
| `python-lib` | 30 | `pyproject.toml`, `setup.py`, `requirements.txt`, `.py` files | 120k | `__pycache__`, `.venv`, `venv`, `.mypy_cache`, `.pytest_cache`, `.tox`, `.eggs` |
|
|
138
|
+
| `infra` | 24 | `*.tf`, `Chart.yaml`, `kustomization.yaml`, `docker-compose.yml` | 100k | `.terraform`, `.terragrunt-cache` |
|
|
139
|
+
| `docs` | 22 | ≥ 3 markdown files and no `package.json` | 100k | `.docusaurus`, `.vitepress/dist`, `site` |
|
|
140
|
+
| `generic` | 0 | anything else | 120k | — |
|
|
141
|
+
|
|
142
|
+
A preset contributes layers, kinds, ignores and a recommended budget — what `init` writes as
|
|
143
|
+
`maxPartBytes`. Force one when detection is wrong:
|
|
144
|
+
|
|
145
|
+
```js
|
|
146
|
+
export default { preset: "react-vite" };
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
An unknown name is an error listing every valid preset, never a silent fallback. The resolved
|
|
150
|
+
preset is recorded in `manifest.config.preset`, so changing it shows up as config drift.
|
|
151
|
+
|
|
152
|
+
## Classification: layers and kinds
|
|
153
|
+
|
|
154
|
+
Layers group files (roughly directories), kinds label their role. Both are first-match-wins
|
|
155
|
+
with `other` as the fallback, and both accept a glob string, a `RegExp`, or a function.
|
|
156
|
+
|
|
157
|
+
```js
|
|
158
|
+
export default {
|
|
159
|
+
layersMode: "merge", // "merge" (default) or "replace"
|
|
160
|
+
layers: [
|
|
161
|
+
{ name: "api", match: "src/api/**" },
|
|
162
|
+
{ name: "ui", match: /\.(tsx|vue|svelte)$/ },
|
|
163
|
+
],
|
|
164
|
+
kinds: [
|
|
165
|
+
{ kind: "route", match: /(^|\/)route\.tsx?$/ },
|
|
166
|
+
{ kind: "hook", match: "src/hooks/**" },
|
|
167
|
+
],
|
|
168
|
+
};
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
A layer rule is `{ name, match }` and a kind rule is `{ kind, match }`. The label key is
|
|
172
|
+
different in each, and a row that gets it wrong is rejected rather than silently ignored, so
|
|
173
|
+
copy the shapes above.
|
|
174
|
+
|
|
175
|
+
- `layersMode: "merge"` keeps the preset's rules and lets yours win by being evaluated first.
|
|
176
|
+
- `layersMode: "replace"` discards the preset's rules entirely.
|
|
177
|
+
|
|
178
|
+
Classification drives per-layer budgets and the `Layers:` breakdown in `report`.
|
|
179
|
+
|
|
180
|
+
## Sizing: byte budgets
|
|
181
|
+
|
|
182
|
+
```js
|
|
183
|
+
export default {
|
|
184
|
+
maxPartBytes: "300k", // hard ceiling per part; default 307200, minimum 4096
|
|
185
|
+
largeFileBytes: "200k", // above this a file becomes a head/tail preview
|
|
186
|
+
previewHeadLines: 80,
|
|
187
|
+
previewTailLines: 40,
|
|
188
|
+
budgets: { // optional per-layer overrides
|
|
189
|
+
default: "120k",
|
|
190
|
+
byLayer: { docs: 40_000, core: 300_000 },
|
|
191
|
+
},
|
|
192
|
+
};
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Sizes accept bytes (`307200`), `"120k"`, `"1.5mb"`, and spaces (`"120 k"`). Zero and negative
|
|
196
|
+
values are errors; so is anything below 4096 — a part budget that small cannot hold a header,
|
|
197
|
+
let alone code.
|
|
198
|
+
|
|
199
|
+
The budget is a hard ceiling: a part never exceeds it. A single file larger than the budget is
|
|
200
|
+
not split mid-file — it becomes a head/tail preview, marked `isPreviewTruncated`.
|
|
201
|
+
|
|
202
|
+
`budgets.byLayer` lets expensive layers get more room than generated docs, for example.
|
|
203
|
+
|
|
204
|
+
## Artifacts: what gets written
|
|
205
|
+
|
|
206
|
+
```js
|
|
207
|
+
export default {
|
|
208
|
+
artifacts: {
|
|
209
|
+
readme: true, // snapshot/README.md
|
|
210
|
+
manifest: true, // snapshot/manifest.json
|
|
211
|
+
parts: true, // snapshot/parts/*.md
|
|
212
|
+
navigation: true, // snapshot/reports/navigation.md
|
|
213
|
+
projectTree: true, // snapshot/reports/project-tree.md
|
|
214
|
+
audit: true, // snapshot/reports/audit.md
|
|
215
|
+
monolith: false, // full-codebase.json + full-codebase.md
|
|
216
|
+
},
|
|
217
|
+
};
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
Everything except `monolith` is on by default; `monolith` follows `includeMonolith`. Turn
|
|
221
|
+
artifacts off with the config, `--skip-artifacts a,b`, or `--only-artifacts a,b` (the two
|
|
222
|
+
cannot be combined).
|
|
223
|
+
|
|
224
|
+
Two combinations warn because they produce a snapshot that contradicts itself: `manifest` off
|
|
225
|
+
means `validate` and `report` cannot inspect the snapshot, and `parts` off with `navigation`
|
|
226
|
+
on leaves links pointing at files that were never written. Both are allowed.
|
|
227
|
+
|
|
228
|
+
The `.codebase-snapshot` ownership marker is always written, whatever the selection.
|
|
229
|
+
|
|
230
|
+
See [Snapshot output](output.md) for the layout and the manifest schema.
|
|
231
|
+
|
|
232
|
+
## Secrets: redaction
|
|
233
|
+
|
|
234
|
+
```js
|
|
235
|
+
export default {
|
|
236
|
+
secrets: {
|
|
237
|
+
customPatterns: [
|
|
238
|
+
{ rule: "github-pat", pattern: /ghp_[A-Za-z0-9]{36}/ },
|
|
239
|
+
{ rule: "internal-token", pattern: /CORP-[A-Z0-9]{32}/ },
|
|
240
|
+
],
|
|
241
|
+
disableDefaultPatterns: false,
|
|
242
|
+
},
|
|
243
|
+
includeSensitiveFiles: false, // .env, *.pem, .npmrc, .ssh/, .tfstate …
|
|
244
|
+
};
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
Each entry is `{ rule, pattern }`, where `rule` is the name that appears in the audit and
|
|
248
|
+
`pattern` is a `RegExp`, or a string in a `.json` config. A bare `RegExp` in the array works
|
|
249
|
+
too — it comes out as `custom-1`, `custom-2`, … in the findings — but naming the rule is what
|
|
250
|
+
makes an audit entry mean something.
|
|
251
|
+
|
|
252
|
+
- Credential-bearing **files** are skipped by default and listed in `skipped[]` with reason
|
|
253
|
+
`sensitive-file`. `includeSensitiveFiles: true` emits them.
|
|
254
|
+
- Secret-shaped **strings** inside ordinary files are redacted in the emitted parts. Only the
|
|
255
|
+
matched span is masked, never the whole line, and no excerpt of a matched line reaches any
|
|
256
|
+
artifact. Multi-line PEM keys are masked as a block.
|
|
257
|
+
- `customPatterns` add rules; `disableDefaultPatterns` removes the built-in set — it does not
|
|
258
|
+
disable redaction itself.
|
|
259
|
+
|
|
260
|
+
Redaction is **fail-closed**, so a placeholder is indistinguishable from a secret: a
|
|
261
|
+
`.env.example` whose line reads `JWT_SECRET=replace-with-base64url-48-bytes` is masked like a
|
|
262
|
+
live key would be, and a test fixture literal such as `process.env.X_SECRET = "test-value"` is
|
|
263
|
+
masked too. The same holds for an environment reference written as a quoted value —
|
|
264
|
+
`password: "${DB_PASSWORD}"` is masked exactly like a literal password, because the rule sees a
|
|
265
|
+
credential-shaped key followed by a string and a `${…}` reference cannot be told apart from a
|
|
266
|
+
secret by shape alone. That is the intended trade-off — the alternative is guessing which values
|
|
267
|
+
are throwaway — and it is worth knowing before you hand the snapshot to a model and find the
|
|
268
|
+
template less readable than the original. To narrow it, remove a rule by name with
|
|
269
|
+
`disableDefaultPatterns: true` plus your own `customPatterns`.
|
|
270
|
+
|
|
271
|
+
Findings are summarised in `reports/audit.md` (one row per line, naming every rule that matched
|
|
272
|
+
it), counted in `manifest.totals.redactedLines`, and shown in the `generate` summary as a
|
|
273
|
+
`Redacted:` line separate from `Warnings:`.
|
|
274
|
+
|
|
275
|
+
## Output directories
|
|
276
|
+
|
|
277
|
+
```js
|
|
278
|
+
export default {
|
|
279
|
+
outDir: "snapshot",
|
|
280
|
+
tmpDir: "snapshot.tmp",
|
|
281
|
+
};
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
`outDir` may not be the working directory, an ancestor, or the filesystem root, and
|
|
285
|
+
`outDir`/`tmpDir` may not contain one another — the message prints the resolved paths. Writes
|
|
286
|
+
are atomic: the previous snapshot survives until the new one is complete.
|
|
287
|
+
|
|
288
|
+
Both directories are also **refused if they already exist and this generator did not write
|
|
289
|
+
them**, so `outDir: "src"` cannot replace your source tree. Ownership is proved by the
|
|
290
|
+
`.codebase-snapshot` marker and nothing else: a JSON object naming this generator, with a
|
|
291
|
+
schema version this build understands, a per-snapshot `id`, and the list of entries it wrote.
|
|
292
|
+
A file with that name but no valid marker, or a directory holding a manifest from another
|
|
293
|
+
tool, is refused. The proof is never a filename, so an existing `docs/README.md` is safe.
|
|
294
|
+
|
|
295
|
+
Files your hook writes into the output directory are recorded in the marker's entry list, so
|
|
296
|
+
they do not block later runs; a file you add by hand afterwards does, because it is not in the
|
|
297
|
+
snapshot the marker describes.
|
|
298
|
+
|
|
299
|
+
## Hooks
|
|
300
|
+
|
|
301
|
+
```js
|
|
302
|
+
export default {
|
|
303
|
+
hooks: {
|
|
304
|
+
afterGenerate: (manifest) => {
|
|
305
|
+
// runs after a real generate; a dry run does not call it
|
|
306
|
+
},
|
|
307
|
+
},
|
|
308
|
+
};
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
A throwing hook is downgraded to a warning on the manifest — it never loses a snapshot that
|
|
312
|
+
was already written. A hook may return a promise: it is not awaited (this API is synchronous),
|
|
313
|
+
and a rejection is recorded as a warning on the re-shipped manifest and marker. Anything the
|
|
314
|
+
hook writes to stdout is redirected to stderr for the duration of the call, so `--format json`
|
|
315
|
+
stays parseable.
|
|
316
|
+
|
|
317
|
+
"Not awaited" does not mean "free": Node keeps the process alive while the hook's own timers,
|
|
318
|
+
sockets or pending I/O are outstanding, so a hook that schedules a long job delays the command by
|
|
319
|
+
that long even though the snapshot is already written. Keep `afterGenerate` short — hand long work
|
|
320
|
+
to a background process — or use the programmatic `generateSnapshot`, whose promise you can `await`
|
|
321
|
+
alongside your own work.
|
|
322
|
+
|
|
323
|
+
A hook may write its own files into the output directory; that is safe across reruns because
|
|
324
|
+
ownership is established by the marker rather than by the filenames present.
|
|
325
|
+
|
|
326
|
+
A hook runs in the **process** working directory, so with `--cwd elsewhere` a relative path
|
|
327
|
+
resolves against your shell, not the target repository. Use the absolute paths on `manifest`
|
|
328
|
+
(or resolve against `outDir` yourself) when passing `--cwd`.
|
|
329
|
+
|
|
330
|
+
## Complete example
|
|
331
|
+
|
|
332
|
+
Every setting at its built-in default. Nothing here has to be written — the tool runs on these
|
|
333
|
+
values with no config at all.
|
|
334
|
+
|
|
335
|
+
```js
|
|
336
|
+
const config = {
|
|
337
|
+
// ── Project ──────────────────────────────────────────────
|
|
338
|
+
preset: "detected", // "detected" auto-detects; or name one of the 25
|
|
339
|
+
branch: "main", // recorded when git cannot report one
|
|
340
|
+
|
|
341
|
+
// ── Output ───────────────────────────────────────────────
|
|
342
|
+
outDir: "snapshot",
|
|
343
|
+
tmpDir: "snapshot.tmp",
|
|
344
|
+
|
|
345
|
+
// ── Size ─────────────────────────────────────────────────
|
|
346
|
+
maxPartBytes: "300k",
|
|
347
|
+
largeFileBytes: "200k",
|
|
348
|
+
previewHeadLines: 80,
|
|
349
|
+
previewTailLines: 40,
|
|
350
|
+
budgets: { default: "300k", byLayer: {} },
|
|
351
|
+
|
|
352
|
+
// ── Scope ────────────────────────────────────────────────
|
|
353
|
+
include: [],
|
|
354
|
+
excludePatterns: [],
|
|
355
|
+
discovery: "auto", // "auto" | "git" | "walk"
|
|
356
|
+
|
|
357
|
+
// ── Artifacts ────────────────────────────────────────────
|
|
358
|
+
artifacts: {
|
|
359
|
+
readme: true,
|
|
360
|
+
manifest: true,
|
|
361
|
+
parts: true,
|
|
362
|
+
navigation: true,
|
|
363
|
+
projectTree: true,
|
|
364
|
+
audit: true,
|
|
365
|
+
monolith: false,
|
|
366
|
+
},
|
|
367
|
+
|
|
368
|
+
// ── Safety ───────────────────────────────────────────────
|
|
369
|
+
includeSensitiveFiles: false,
|
|
370
|
+
secrets: { customPatterns: [], disableDefaultPatterns: false },
|
|
371
|
+
|
|
372
|
+
// ── Classification ───────────────────────────────────────
|
|
373
|
+
layersMode: "merge",
|
|
374
|
+
layers: [],
|
|
375
|
+
kinds: [],
|
|
376
|
+
|
|
377
|
+
// ── Advanced ─────────────────────────────────────────────
|
|
378
|
+
includeMonolith: false,
|
|
379
|
+
maxMonolithBytes: null,
|
|
380
|
+
hooks: {},
|
|
381
|
+
};
|
|
382
|
+
|
|
383
|
+
export default config;
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
Deleting any line is safe — the defaults are real, and each one is what the comment says.
|
|
387
|
+
|
|
388
|
+
The object is exported through a name because `export default { … }` is reported by
|
|
389
|
+
`import/no-anonymous-default-export`, which `eslint-config-next` enables by default: the inline
|
|
390
|
+
form made the generated file a lint error on first save in a Next.js project. A named export is
|
|
391
|
+
also what lets you reference the object (from a test, or from `--dry-run`) without a wrapper.
|
|
392
|
+
|
|
393
|
+
This is **not** byte-identical to a generated file. `codebase-snapshot init` starts from these
|
|
394
|
+
values and substitutes two things: `preset` becomes the detected name (`node-lib` for a plain
|
|
395
|
+
npm package), and `maxPartBytes` with `budgets.default` become that preset's
|
|
396
|
+
`recommendedBudget` (`"120k"` in the same project). `generate` on its own applies no
|
|
397
|
+
recommended budget — without a config it uses `--max-part-bytes`, default 307200 — so the
|
|
398
|
+
preset's figure only takes effect once `init` has written it into a config.
|
|
399
|
+
|
|
400
|
+
## Interactive setup
|
|
401
|
+
|
|
402
|
+
```bash
|
|
403
|
+
npx codebase-snapshot init
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
Walks every tunable parameter, showing the detected value as the default. Every prompt has a
|
|
407
|
+
flag, so the same setup works non-interactively:
|
|
408
|
+
|
|
409
|
+
```bash
|
|
410
|
+
npx codebase-snapshot init --yes
|
|
411
|
+
npx codebase-snapshot init --preset go --max-part-bytes 200k \
|
|
412
|
+
--include "cmd/**,internal/**" --exclude "**/*_test.go" --yes
|
|
413
|
+
```
|
|
414
|
+
|
|
415
|
+
`--dry-run` prints the config and writes nothing. `--force` replaces an existing ES-module
|
|
416
|
+
config (`.ts`, `.js`, `.mjs`) in place; without it, an existing config is left alone and the
|
|
417
|
+
command exits `1`.
|
|
418
|
+
|
|
419
|
+
It will not replace a JSON-family config — `snapshot.config.json`, `.snapshotrc`,
|
|
420
|
+
`.snapshotrc.json`, or the `"snapshot"` key in `package.json` — because this wizard writes an
|
|
421
|
+
ES-module file and a new file beside the old one would silently shadow it. Those cases exit `1`
|
|
422
|
+
with a message naming the file to delete or rename. Note that this means `--force` writing
|
|
423
|
+
`snapshot.config.ts` does shadow a `"snapshot"` key in `package.json`, since a dedicated config
|
|
424
|
+
file always wins.
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
# How to budget LLM context files for a codebase
|
|
2
|
+
|
|
3
|
+
Context is a budget, not a bin. Every token spent on a lockfile is a token not spent on the function
|
|
4
|
+
being asked about, and the failure is silent — the model simply gets vaguer. This page is about
|
|
5
|
+
choosing the numbers and knowing what they cost.
|
|
6
|
+
|
|
7
|
+
## Start from the destination, not the repository
|
|
8
|
+
|
|
9
|
+
Work backwards. The constraint is whatever consumes the output:
|
|
10
|
+
|
|
11
|
+
| Consumer | What actually limits you |
|
|
12
|
+
| :--- | :--- |
|
|
13
|
+
| A chat window | The window, minus room for the conversation and the answer |
|
|
14
|
+
| A RAG pipeline | Embedding chunk size, and how many chunks get retrieved per query |
|
|
15
|
+
| An agent loop | The window, minus tool schemas and minus the reasoning it has left to do |
|
|
16
|
+
| A review artifact | Human attention — usually far smaller than any window |
|
|
17
|
+
|
|
18
|
+
An agent needs room to *think*, so it wants the smallest context that answers the question. A chat
|
|
19
|
+
window can take the largest.
|
|
20
|
+
|
|
21
|
+
## The numbers that matter
|
|
22
|
+
|
|
23
|
+
- **`maxPartBytes`** — the hard ceiling for one part. This is a promise, not an average: no part
|
|
24
|
+
exceeds it. Set it to the destination's limit and stop worrying about the largest file.
|
|
25
|
+
- **`largeFileBytes`** — above this, a file becomes a head/tail excerpt rather than its full body.
|
|
26
|
+
This is what stops one generated 40 MB file from consuming an entire snapshot.
|
|
27
|
+
- **`budgets.byLayer`** — different ceilings per layer. Documentation is usually worth less per byte
|
|
28
|
+
than source; `{ docs: 40_000 }` is a common shape.
|
|
29
|
+
- **`maxMonolithBytes`** — the single-file artifact roughly doubles output, so it is off by default
|
|
30
|
+
and capped when on.
|
|
31
|
+
|
|
32
|
+
```js
|
|
33
|
+
const config = {
|
|
34
|
+
maxPartBytes: "120k",
|
|
35
|
+
largeFileBytes: "200k",
|
|
36
|
+
budgets: { default: "120k", byLayer: { docs: 40_000, config: 20_000 } },
|
|
37
|
+
artifacts: { monolith: false },
|
|
38
|
+
};
|
|
39
|
+
export default config;
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## What each byte buys
|
|
43
|
+
|
|
44
|
+
Measured on this repository's own snapshot, as a rough guide:
|
|
45
|
+
|
|
46
|
+
| Layer | Share of bytes | Worth it? |
|
|
47
|
+
| :--- | :--- | :--- |
|
|
48
|
+
| Source | largest | yes — this is the question |
|
|
49
|
+
| Docs and comments | large | partly; often the best value per byte, but not when duplicated |
|
|
50
|
+
| Config | small | yes — it explains how the source runs |
|
|
51
|
+
| Tests | often 30–40% | rarely, for a code question |
|
|
52
|
+
| Generated and vendored | varies | almost never |
|
|
53
|
+
|
|
54
|
+
The cheapest large win in most repositories is excluding tests and build output. The second cheapest
|
|
55
|
+
is excluding duplicated documentation (the same content in a wiki, a `docs/` tree and a README).
|
|
56
|
+
|
|
57
|
+
## Do not trust a size estimate you have not measured
|
|
58
|
+
|
|
59
|
+
A snapshot is auditable: `manifest.json` records `includedBytes` and `includedLines` **measured from
|
|
60
|
+
the block actually emitted**, not from the file on disk. That distinction matters — a truncated file
|
|
61
|
+
does not report its on-disk size.
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
npx codebase-snapshot report
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
`report` prints totals, per-layer metrics and the secret audit, so you can see where the budget went
|
|
68
|
+
before you spend it on a model call.
|
|
69
|
+
|
|
70
|
+
## Tuning loop
|
|
71
|
+
|
|
72
|
+
1. Generate with a ceiling that matches the destination.
|
|
73
|
+
2. Run `report`; look at the per-layer breakdown.
|
|
74
|
+
3. Cut the largest layer that does not answer your question.
|
|
75
|
+
4. Re-generate and compare `manifest.totals.events` and the part count.
|
|
76
|
+
5. Keep the settings that produced a snapshot you actually used — the config is the durable artifact,
|
|
77
|
+
not the output.
|
|
78
|
+
|
|
79
|
+
## The failure this prevents
|
|
80
|
+
|
|
81
|
+
Without a ceiling, the boundary is wherever the tool stopped. Two runs can cut differently, so a
|
|
82
|
+
worse answer is indistinguishable from a different cut. With `maxPartBytes` fixed and output
|
|
83
|
+
deterministic, a changed answer means changed code — which is the only interpretation that lets you
|
|
84
|
+
debug it.
|
|
85
|
+
|
|
86
|
+
## Next
|
|
87
|
+
|
|
88
|
+
- [Configuration](configuration.md) — every setting, all 25 presets
|
|
89
|
+
- [Snapshot output](output.md) — the full `manifest.json` schema
|
|
90
|
+
- [How to feed a codebase to an LLM](how-to-feed-a-repo-to-claude.md)
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
# How to feed a whole codebase to ChatGPT, Claude or Cursor
|
|
2
|
+
|
|
3
|
+
A model cannot clone your repository. Everything it knows about your code has to arrive as text, and
|
|
4
|
+
the text has to fit. This is the practical, repeated recipe — including the parts most guides skip,
|
|
5
|
+
which are the ones that decide whether the answer is any good.
|
|
6
|
+
|
|
7
|
+
## The problem, stated properly
|
|
8
|
+
|
|
9
|
+
There are three separate failures, and they need different fixes:
|
|
10
|
+
|
|
11
|
+
1. **It does not fit.** A real repository is far larger than any context window once you include
|
|
12
|
+
node_modules, lockfiles and build output. Naively concatenating files produces something that is
|
|
13
|
+
rejected, truncated by an unknown amount, or billed at a size you did not expect.
|
|
14
|
+
2. **It does not fit *predictably*.** If the boundary moves between runs, you cannot tell whether a
|
|
15
|
+
changed answer came from changed code or a different cut of the same code.
|
|
16
|
+
3. **It is not current.** The document you pasted was generated yesterday. Nobody notices until the
|
|
17
|
+
model confidently describes a function you deleted.
|
|
18
|
+
|
|
19
|
+
Most "paste your repo into an LLM" tooling solves only the first.
|
|
20
|
+
|
|
21
|
+
## The recipe
|
|
22
|
+
|
|
23
|
+
### 1. Generate from the checkout, not from a website
|
|
24
|
+
|
|
25
|
+
If the code is private, it should not be uploaded to a third-party service to be turned into text.
|
|
26
|
+
Everything below runs locally.
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
cd /path/to/your/project
|
|
30
|
+
npx codebase-snapshot setup
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
`setup` detects the project type from the repository, writes a commented `snapshot.config.mjs`, adds
|
|
34
|
+
the npm scripts and the devDependency, installs it, and generates. It does not overwrite a config you
|
|
35
|
+
already have. Use `--no-install` if you would rather not touch the lockfile.
|
|
36
|
+
|
|
37
|
+
If you prefer to see each stage:
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
npx codebase-snapshot init --yes # config + scripts + dependency + .gitignore
|
|
41
|
+
npm install
|
|
42
|
+
npm run snapshot # generate
|
|
43
|
+
npm run snapshot:validate # exit 0 = still matches the disk
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
### 2. Choose the budget, not the file list
|
|
47
|
+
|
|
48
|
+
The setting that matters is `maxPartBytes` — the hard ceiling for one part. Pick it from the
|
|
49
|
+
destination, not from taste:
|
|
50
|
+
|
|
51
|
+
| Destination | Practical budget |
|
|
52
|
+
| :--- | :--- |
|
|
53
|
+
| A chat window | 40–120 KB per part; paste one or two |
|
|
54
|
+
| A RAG index | 120–300 KB per part, chunked by heading |
|
|
55
|
+
| An agent's context | 60–150 KB; smaller parts leave room to reason |
|
|
56
|
+
|
|
57
|
+
```js
|
|
58
|
+
// snapshot.config.mjs
|
|
59
|
+
const config = {
|
|
60
|
+
maxPartBytes: "120k",
|
|
61
|
+
// Per-layer ceilings, when docs and code deserve different treatment.
|
|
62
|
+
budgets: { default: "120k", byLayer: { docs: 40_000 } },
|
|
63
|
+
};
|
|
64
|
+
export default config;
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
A file larger than the budget is not silently dropped: it is truncated with an explicit marker
|
|
68
|
+
naming the budget, and files above `largeFileBytes` become a head/tail excerpt instead of a body.
|
|
69
|
+
|
|
70
|
+
### 3. Tell the model what it is looking at
|
|
71
|
+
|
|
72
|
+
The snapshot is self-describing, and saying so up front measurably improves answers:
|
|
73
|
+
|
|
74
|
+
> The context below is a generated snapshot of a repository. `manifest.json` lists every included
|
|
75
|
+
> file with its language, part and hash, and every excluded file with the reason. Reads start with
|
|
76
|
+
> `parts/part-001.md`. If something you need is missing, ask me to check `explain <path>` rather than
|
|
77
|
+
> assuming it does not exist.
|
|
78
|
+
|
|
79
|
+
That last sentence matters more than it looks: models fill gaps by invention, and telling them the
|
|
80
|
+
gap is documented turns a hallucination into a question.
|
|
81
|
+
|
|
82
|
+
### 4. Include the index, always
|
|
83
|
+
|
|
84
|
+
Paste `manifest.json` alongside the parts, or at least `reports/navigation.md`. Without the index the
|
|
85
|
+
model sees text with no map and guesses at structure. With it, it can say "this file is in part 3,
|
|
86
|
+
which I have not been given" instead of inventing the file.
|
|
87
|
+
|
|
88
|
+
### 5. Re-validate before every session that matters
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
npm run snapshot:validate || npm run snapshot
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
`validate` compares each file's hash *and* a digest of the settings used to generate the snapshot. A
|
|
95
|
+
changed budget, layer rule or secret pattern is drift, not silence — so the check catches both code
|
|
96
|
+
changes and configuration changes.
|
|
97
|
+
|
|
98
|
+
## What to exclude
|
|
99
|
+
|
|
100
|
+
The defaults already skip binaries, credential files (`.env`, `*.pem`, `.ssh/`, `.npmrc`) and
|
|
101
|
+
anything gitignored. Beyond that, the highest-value exclusions are usually:
|
|
102
|
+
|
|
103
|
+
```js
|
|
104
|
+
excludePatterns: ["**/*.test.ts", "**/__snapshots__/**", "public/**", "*.min.js", "vendor/**"],
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Tests and fixtures are often 40% of a repository and rarely part of the question. Removing them buys
|
|
108
|
+
context for the code that is being asked about.
|
|
109
|
+
|
|
110
|
+
## What this does not solve
|
|
111
|
+
|
|
112
|
+
- **The model still has a fixed window.** If the code genuinely exceeds it, you are choosing what to
|
|
113
|
+
omit — this tool makes that choice explicit and recorded rather than accidental.
|
|
114
|
+
- **Understanding beats volume.** A smaller snapshot with the right 20 files outperforms a large one
|
|
115
|
+
containing everything. Use `include` when you know the area.
|
|
116
|
+
- **Drift is only detected when you run the check.** It is not a watcher, unless you use `--watch`.
|
|
117
|
+
|
|
118
|
+
## Next
|
|
119
|
+
|
|
120
|
+
- [Configuration](configuration.md) — every setting and its default
|
|
121
|
+
- [Snapshot output](output.md) — what each artifact contains
|
|
122
|
+
- [Comparing with other tools](vs-repomix.md)
|