codebase-snapshot 0.0.0-stage → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,424 @@
1
+ # Configuration reference
2
+
3
+ Every setting, every default, and what each one changes. For commands and options see the
4
+ [CLI reference](cli.md).
5
+
6
+ Nothing here is required: with no config at all the tool runs on the defaults below.
7
+
8
+ ## Contents
9
+
10
+ - [Config files](#config-files)
11
+ - [Scoping: which files get in](#scoping-which-files-get-in)
12
+ - [Project presets](#project-presets)
13
+ - [Classification: layers and kinds](#classification-layers-and-kinds)
14
+ - [Sizing: byte budgets](#sizing-byte-budgets)
15
+ - [Artifacts: what gets written](#artifacts-what-gets-written)
16
+ - [Secrets: redaction](#secrets-redaction)
17
+ - [Output directories](#output-directories)
18
+ - [Hooks](#hooks)
19
+ - [Complete example](#complete-example)
20
+ - [Interactive setup](#interactive-setup)
21
+
22
+ ## Config files
23
+
24
+ The first match wins:
25
+
26
+ ```
27
+ snapshot.config.ts
28
+ snapshot.config.js
29
+ snapshot.config.mjs
30
+ snapshot.config.cjs
31
+ snapshot.config.json
32
+ .snapshotrc / .snapshotrc.json
33
+ package.json → "snapshot"
34
+ ```
35
+
36
+ `snapshot.config.json` accepts comments. `package.json`'s `"snapshot"` key is read only when
37
+ no dedicated config file exists, and it must be an object (a wrong type is a warning, since
38
+ other tools use that key too).
39
+
40
+ The extension decides how the file is loaded. `.cjs` is always CommonJS and must use
41
+ `module.exports`; `.mjs` and `.ts` are always ESM and must use `export default`. `.js` follows
42
+ the package's `type` field — `module` means ESM, `commonjs` means CommonJS; with no `type`
43
+ Node decides by detecting the syntax, so `export default` works on current Node but is not
44
+ portable to versions without detection. Getting it wrong is the `Could not load …` error
45
+ above, not a silent fallback.
46
+
47
+ A config file that exists but cannot be parsed is a **hard error** (exit `1`, nothing
48
+ written). Silently ignoring it would produce a snapshot with none of your settings.
49
+
50
+ A `.ts` config needs Node ≥ 22.6 **and** an ES-module project, because Node applies the
51
+ package's `type` field to a `.ts` file. `codebase-snapshot init` picks the right filename for
52
+ the project — `.ts` when both conditions hold, `.mjs` otherwise — and writes a plain object
53
+ with no imports so it loads even when the package is only used through `npx`.
54
+
55
+ The snippets below show an `export default { … }` object for brevity. `init` writes it as
56
+ `const config = { … }; export default config;` because the anonymous form is reported by
57
+ `import/no-anonymous-default-export`, which `eslint-config-next` enables by default — both load
58
+ identically, so use whichever keeps your linter quiet.
59
+
60
+ CLI flags override the config for a single run (`--max-part-bytes`, `--out-dir`, …).
61
+ `--no-config` goes further and ignores every config file, so the run uses the built-in
62
+ defaults and the project's detected preset. A code config is executed when it is loaded, so
63
+ `--no-config` is also the way to snapshot a repository whose config you do not want to run —
64
+ see [the trust boundary](../SECURITY.md#trust-boundary).
65
+
66
+ ## Scoping: which files get in
67
+
68
+ ```js
69
+ export default {
70
+ include: [], // allow-list; empty means "everything"
71
+ excludePatterns: [], // extra excludes, applied on top of the preset's
72
+ discovery: "auto", // "auto" | "git" | "walk" — how files are found
73
+ };
74
+ ```
75
+
76
+ - `include` is an allow-list of globs. When non-empty, only matching files are considered.
77
+ - `exclude` is accepted as an alias of `excludePatterns` and means the same thing; the resolved
78
+ config always records it under `excludePatterns`.
79
+ - `excludePatterns` removes files everywhere. Presets contribute their own ecosystem ignores
80
+ on top; those are not removable, only addable to.
81
+ - `discovery` chooses how the file list is produced, and is recorded in `manifest.config`, so
82
+ changing it shows up as config drift:
83
+ - `"auto"` (default) — list through `git ls-files` when git can do it; if a `.git` directory
84
+ exists but git cannot list the files, refuse rather than silently switch methods.
85
+ - `"git"` — require git; a directory without a usable repository is an error.
86
+ - `"walk"` — scan the tree and read `.gitignore` files directly (nested ones included), still
87
+ skipping credential files. Use it when a `.git` directory cannot be listed. It cannot
88
+ reproduce git's own ignore sources (`.git/info/exclude`, your global `core.excludesFile`),
89
+ git's inclusion of files that are tracked although a `.gitignore` now matches them, or its
90
+ treatment of a submodule as a single entry. The same setting is available per run as
91
+ [`--discovery`](cli.md#options).
92
+
93
+ Built-in ignores always apply: `.git`, `node_modules`, `.next`, `dist`, `build`, lockfiles,
94
+ minified assets and binaries. Sensitive files are handled separately (see
95
+ [Secrets](#secrets-redaction)).
96
+
97
+ Every path the scanner discovers is recorded in `manifest.skipped[]` with the reason and the
98
+ rule that matched — nothing it sees is dropped silently, including a `.gitignore`d path in a
99
+ repository where git does the filtering. See the [`skipped[]`
100
+ reasons](output.md#skipped). To debug one path:
101
+
102
+ ```bash
103
+ npx codebase-snapshot explain src/some/file.ts
104
+ ```
105
+
106
+ ## Project presets
107
+
108
+ Detection reads the repository **once** and picks the highest-priority match, so a Next.js
109
+ app inside a pnpm workspace is still `nextjs`, not `monorepo`. `peerDependencies` and
110
+ `optionalDependencies` count as dependencies. A bare `package.json` ranks below any
111
+ definitive language manifest, so a Go or Rust repository with a `package.json` for tooling is
112
+ not mistaken for a Node library.
113
+
114
+ | Preset | Priority | Detected from | Budget | Ecosystem ignores |
115
+ | :--- | ---: | :--- | :--- | :--- |
116
+ | `nextjs` | 100 | `next.config.*`, or `next` | 300k | `.svelte-kit`, `.astro`, `.vite`, `.output` |
117
+ | `nuxt` | 98 | `nuxt.config.*`, or `nuxt` | 200k | `.nuxt`, `.output` |
118
+ | `sveltekit` | 96 | `svelte.config.*`, `@sveltejs/kit` | 200k | `.svelte-kit`, `.vercel` |
119
+ | `remix` | 94 | `@remix-run/*` | 200k | `.cache`, `public/build` |
120
+ | `astro` | 92 | `astro.config.*`, `astro` | 150k | `.astro`, `dist` |
121
+ | `angular` | 90 | `angular.json`, `@angular/core` | 200k | `.angular`, `dist` |
122
+ | `solid` | 88 | `solid-js`, `solid-start` | 120k | `.solid`, `dist` |
123
+ | `vue-vite` | 86 | `vue` + `vite`/`vite.config.*` | 120k | `.vite`, `dist` |
124
+ | `react-vite` | 84 | `react` + `vite`/`vite.config.*` | 120k | `.vite`, `dist` |
125
+ | `nest` | 80 | `@nestjs/core` | 200k | `dist`, `node_modules` |
126
+ | `node-express` | 78 | `express`, `fastify`, `koa`, `@hapi/hapi` | 200k | `dist` |
127
+ | `monorepo` | 70 | `pnpm-workspace.yaml`, `nx.json`, `turbo.json`, `lerna.json`, `"workspaces"` | 200k | `.turbo`, `.nx` |
128
+ | `python-django` | 56 | `manage.py`, django in `pyproject.toml`/`requirements.txt` | 150k | `__pycache__`, `.venv`, `venv`, `.mypy_cache`, `.pytest_cache`, `.tox`, `.eggs`, `staticfiles`, `media` |
129
+ | `python-fastapi` | 54 | fastapi or flask in `pyproject.toml`/`requirements.txt` | 150k | `__pycache__`, `.venv`, `venv`, `.mypy_cache`, `.pytest_cache`, `.tox`, `.eggs` |
130
+ | `go` | 50 | `go.mod` | 200k | `vendor`, `testdata` |
131
+ | `rust` | 48 | `Cargo.toml` | 200k | `target` |
132
+ | `java-spring` | 46 | `pom.xml`, `build.gradle`, `build.gradle.kts` | 200k | `.gradle`, `out`, `target` |
133
+ | `node-lib` | 45 | any `package.json` | 120k | `dist` |
134
+ | `dotnet` | 44 | `*.csproj`, `*.sln`, `global.json` | 200k | `bin`, `obj`, `.vs` |
135
+ | `php-laravel` | 42 | `artisan`, laravel in `composer.json`, `composer.json` | 150k | `vendor`, `storage`, `bootstrap/cache`, `public/build` |
136
+ | `ruby-rails` | 40 | `Gemfile`, `config.ru`, `Rakefile` | 150k | `.bundle`, `log`, `tmp`, `vendor/bundle`, `public/assets` |
137
+ | `python-lib` | 30 | `pyproject.toml`, `setup.py`, `requirements.txt`, `.py` files | 120k | `__pycache__`, `.venv`, `venv`, `.mypy_cache`, `.pytest_cache`, `.tox`, `.eggs` |
138
+ | `infra` | 24 | `*.tf`, `Chart.yaml`, `kustomization.yaml`, `docker-compose.yml` | 100k | `.terraform`, `.terragrunt-cache` |
139
+ | `docs` | 22 | ≥ 3 markdown files and no `package.json` | 100k | `.docusaurus`, `.vitepress/dist`, `site` |
140
+ | `generic` | 0 | anything else | 120k | — |
141
+
142
+ A preset contributes layers, kinds, ignores and a recommended budget — what `init` writes as
143
+ `maxPartBytes`. Force one when detection is wrong:
144
+
145
+ ```js
146
+ export default { preset: "react-vite" };
147
+ ```
148
+
149
+ An unknown name is an error listing every valid preset, never a silent fallback. The resolved
150
+ preset is recorded in `manifest.config.preset`, so changing it shows up as config drift.
151
+
152
+ ## Classification: layers and kinds
153
+
154
+ Layers group files (roughly directories), kinds label their role. Both are first-match-wins
155
+ with `other` as the fallback, and both accept a glob string, a `RegExp`, or a function.
156
+
157
+ ```js
158
+ export default {
159
+ layersMode: "merge", // "merge" (default) or "replace"
160
+ layers: [
161
+ { name: "api", match: "src/api/**" },
162
+ { name: "ui", match: /\.(tsx|vue|svelte)$/ },
163
+ ],
164
+ kinds: [
165
+ { kind: "route", match: /(^|\/)route\.tsx?$/ },
166
+ { kind: "hook", match: "src/hooks/**" },
167
+ ],
168
+ };
169
+ ```
170
+
171
+ A layer rule is `{ name, match }` and a kind rule is `{ kind, match }`. The label key is
172
+ different in each, and a row that gets it wrong is rejected rather than silently ignored, so
173
+ copy the shapes above.
174
+
175
+ - `layersMode: "merge"` keeps the preset's rules and lets yours win by being evaluated first.
176
+ - `layersMode: "replace"` discards the preset's rules entirely.
177
+
178
+ Classification drives per-layer budgets and the `Layers:` breakdown in `report`.
179
+
180
+ ## Sizing: byte budgets
181
+
182
+ ```js
183
+ export default {
184
+ maxPartBytes: "300k", // hard ceiling per part; default 307200, minimum 4096
185
+ largeFileBytes: "200k", // above this a file becomes a head/tail preview
186
+ previewHeadLines: 80,
187
+ previewTailLines: 40,
188
+ budgets: { // optional per-layer overrides
189
+ default: "120k",
190
+ byLayer: { docs: 40_000, core: 300_000 },
191
+ },
192
+ };
193
+ ```
194
+
195
+ Sizes accept bytes (`307200`), `"120k"`, `"1.5mb"`, and spaces (`"120 k"`). Zero and negative
196
+ values are errors; so is anything below 4096 — a part budget that small cannot hold a header,
197
+ let alone code.
198
+
199
+ The budget is a hard ceiling: a part never exceeds it. A single file larger than the budget is
200
+ not split mid-file — it becomes a head/tail preview, marked `isPreviewTruncated`.
201
+
202
+ `budgets.byLayer` lets expensive layers get more room than generated docs, for example.
203
+
204
+ ## Artifacts: what gets written
205
+
206
+ ```js
207
+ export default {
208
+ artifacts: {
209
+ readme: true, // snapshot/README.md
210
+ manifest: true, // snapshot/manifest.json
211
+ parts: true, // snapshot/parts/*.md
212
+ navigation: true, // snapshot/reports/navigation.md
213
+ projectTree: true, // snapshot/reports/project-tree.md
214
+ audit: true, // snapshot/reports/audit.md
215
+ monolith: false, // full-codebase.json + full-codebase.md
216
+ },
217
+ };
218
+ ```
219
+
220
+ Everything except `monolith` is on by default; `monolith` follows `includeMonolith`. Turn
221
+ artifacts off with the config, `--skip-artifacts a,b`, or `--only-artifacts a,b` (the two
222
+ cannot be combined).
223
+
224
+ Two combinations warn because they produce a snapshot that contradicts itself: `manifest` off
225
+ means `validate` and `report` cannot inspect the snapshot, and `parts` off with `navigation`
226
+ on leaves links pointing at files that were never written. Both are allowed.
227
+
228
+ The `.codebase-snapshot` ownership marker is always written, whatever the selection.
229
+
230
+ See [Snapshot output](output.md) for the layout and the manifest schema.
231
+
232
+ ## Secrets: redaction
233
+
234
+ ```js
235
+ export default {
236
+ secrets: {
237
+ customPatterns: [
238
+ { rule: "github-pat", pattern: /ghp_[A-Za-z0-9]{36}/ },
239
+ { rule: "internal-token", pattern: /CORP-[A-Z0-9]{32}/ },
240
+ ],
241
+ disableDefaultPatterns: false,
242
+ },
243
+ includeSensitiveFiles: false, // .env, *.pem, .npmrc, .ssh/, .tfstate …
244
+ };
245
+ ```
246
+
247
+ Each entry is `{ rule, pattern }`, where `rule` is the name that appears in the audit and
248
+ `pattern` is a `RegExp`, or a string in a `.json` config. A bare `RegExp` in the array works
249
+ too — it comes out as `custom-1`, `custom-2`, … in the findings — but naming the rule is what
250
+ makes an audit entry mean something.
251
+
252
+ - Credential-bearing **files** are skipped by default and listed in `skipped[]` with reason
253
+ `sensitive-file`. `includeSensitiveFiles: true` emits them.
254
+ - Secret-shaped **strings** inside ordinary files are redacted in the emitted parts. Only the
255
+ matched span is masked, never the whole line, and no excerpt of a matched line reaches any
256
+ artifact. Multi-line PEM keys are masked as a block.
257
+ - `customPatterns` add rules; `disableDefaultPatterns` removes the built-in set — it does not
258
+ disable redaction itself.
259
+
260
+ Redaction is **fail-closed**, so a placeholder is indistinguishable from a secret: a
261
+ `.env.example` whose line reads `JWT_SECRET=replace-with-base64url-48-bytes` is masked like a
262
+ live key would be, and a test fixture literal such as `process.env.X_SECRET = "test-value"` is
263
+ masked too. The same holds for an environment reference written as a quoted value —
264
+ `password: "${DB_PASSWORD}"` is masked exactly like a literal password, because the rule sees a
265
+ credential-shaped key followed by a string and a `${…}` reference cannot be told apart from a
266
+ secret by shape alone. That is the intended trade-off — the alternative is guessing which values
267
+ are throwaway — and it is worth knowing before you hand the snapshot to a model and find the
268
+ template less readable than the original. To narrow it, remove a rule by name with
269
+ `disableDefaultPatterns: true` plus your own `customPatterns`.
270
+
271
+ Findings are summarised in `reports/audit.md` (one row per line, naming every rule that matched
272
+ it), counted in `manifest.totals.redactedLines`, and shown in the `generate` summary as a
273
+ `Redacted:` line separate from `Warnings:`.
274
+
275
+ ## Output directories
276
+
277
+ ```js
278
+ export default {
279
+ outDir: "snapshot",
280
+ tmpDir: "snapshot.tmp",
281
+ };
282
+ ```
283
+
284
+ `outDir` may not be the working directory, an ancestor, or the filesystem root, and
285
+ `outDir`/`tmpDir` may not contain one another — the message prints the resolved paths. Writes
286
+ are atomic: the previous snapshot survives until the new one is complete.
287
+
288
+ Both directories are also **refused if they already exist and this generator did not write
289
+ them**, so `outDir: "src"` cannot replace your source tree. Ownership is proved by the
290
+ `.codebase-snapshot` marker and nothing else: a JSON object naming this generator, with a
291
+ schema version this build understands, a per-snapshot `id`, and the list of entries it wrote.
292
+ A file with that name but no valid marker, or a directory holding a manifest from another
293
+ tool, is refused. The proof is never a filename, so an existing `docs/README.md` is safe.
294
+
295
+ Files your hook writes into the output directory are recorded in the marker's entry list, so
296
+ they do not block later runs; a file you add by hand afterwards does, because it is not in the
297
+ snapshot the marker describes.
298
+
299
+ ## Hooks
300
+
301
+ ```js
302
+ export default {
303
+ hooks: {
304
+ afterGenerate: (manifest) => {
305
+ // runs after a real generate; a dry run does not call it
306
+ },
307
+ },
308
+ };
309
+ ```
310
+
311
+ A throwing hook is downgraded to a warning on the manifest — it never loses a snapshot that
312
+ was already written. A hook may return a promise: it is not awaited (this API is synchronous),
313
+ and a rejection is recorded as a warning on the re-shipped manifest and marker. Anything the
314
+ hook writes to stdout is redirected to stderr for the duration of the call, so `--format json`
315
+ stays parseable.
316
+
317
+ "Not awaited" does not mean "free": Node keeps the process alive while the hook's own timers,
318
+ sockets or pending I/O are outstanding, so a hook that schedules a long job delays the command by
319
+ that long even though the snapshot is already written. Keep `afterGenerate` short — hand long work
320
+ to a background process — or use the programmatic `generateSnapshot`, whose promise you can `await`
321
+ alongside your own work.
322
+
323
+ A hook may write its own files into the output directory; that is safe across reruns because
324
+ ownership is established by the marker rather than by the filenames present.
325
+
326
+ A hook runs in the **process** working directory, so with `--cwd elsewhere` a relative path
327
+ resolves against your shell, not the target repository. Use the absolute paths on `manifest`
328
+ (or resolve against `outDir` yourself) when passing `--cwd`.
329
+
330
+ ## Complete example
331
+
332
+ Every setting at its built-in default. Nothing here has to be written — the tool runs on these
333
+ values with no config at all.
334
+
335
+ ```js
336
+ const config = {
337
+ // ── Project ──────────────────────────────────────────────
338
+ preset: "detected", // "detected" auto-detects; or name one of the 25
339
+ branch: "main", // recorded when git cannot report one
340
+
341
+ // ── Output ───────────────────────────────────────────────
342
+ outDir: "snapshot",
343
+ tmpDir: "snapshot.tmp",
344
+
345
+ // ── Size ─────────────────────────────────────────────────
346
+ maxPartBytes: "300k",
347
+ largeFileBytes: "200k",
348
+ previewHeadLines: 80,
349
+ previewTailLines: 40,
350
+ budgets: { default: "300k", byLayer: {} },
351
+
352
+ // ── Scope ────────────────────────────────────────────────
353
+ include: [],
354
+ excludePatterns: [],
355
+ discovery: "auto", // "auto" | "git" | "walk"
356
+
357
+ // ── Artifacts ────────────────────────────────────────────
358
+ artifacts: {
359
+ readme: true,
360
+ manifest: true,
361
+ parts: true,
362
+ navigation: true,
363
+ projectTree: true,
364
+ audit: true,
365
+ monolith: false,
366
+ },
367
+
368
+ // ── Safety ───────────────────────────────────────────────
369
+ includeSensitiveFiles: false,
370
+ secrets: { customPatterns: [], disableDefaultPatterns: false },
371
+
372
+ // ── Classification ───────────────────────────────────────
373
+ layersMode: "merge",
374
+ layers: [],
375
+ kinds: [],
376
+
377
+ // ── Advanced ─────────────────────────────────────────────
378
+ includeMonolith: false,
379
+ maxMonolithBytes: null,
380
+ hooks: {},
381
+ };
382
+
383
+ export default config;
384
+ ```
385
+
386
+ Deleting any line is safe — the defaults are real, and each one is what the comment says.
387
+
388
+ The object is exported through a name because `export default { … }` is reported by
389
+ `import/no-anonymous-default-export`, which `eslint-config-next` enables by default: the inline
390
+ form made the generated file a lint error on first save in a Next.js project. A named export is
391
+ also what lets you reference the object (from a test, or from `--dry-run`) without a wrapper.
392
+
393
+ This is **not** byte-identical to a generated file. `codebase-snapshot init` starts from these
394
+ values and substitutes two things: `preset` becomes the detected name (`node-lib` for a plain
395
+ npm package), and `maxPartBytes` with `budgets.default` become that preset's
396
+ `recommendedBudget` (`"120k"` in the same project). `generate` on its own applies no
397
+ recommended budget — without a config it uses `--max-part-bytes`, default 307200 — so the
398
+ preset's figure only takes effect once `init` has written it into a config.
399
+
400
+ ## Interactive setup
401
+
402
+ ```bash
403
+ npx codebase-snapshot init
404
+ ```
405
+
406
+ Walks every tunable parameter, showing the detected value as the default. Every prompt has a
407
+ flag, so the same setup works non-interactively:
408
+
409
+ ```bash
410
+ npx codebase-snapshot init --yes
411
+ npx codebase-snapshot init --preset go --max-part-bytes 200k \
412
+ --include "cmd/**,internal/**" --exclude "**/*_test.go" --yes
413
+ ```
414
+
415
+ `--dry-run` prints the config and writes nothing. `--force` replaces an existing ES-module
416
+ config (`.ts`, `.js`, `.mjs`) in place; without it, an existing config is left alone and the
417
+ command exits `1`.
418
+
419
+ It will not replace a JSON-family config — `snapshot.config.json`, `.snapshotrc`,
420
+ `.snapshotrc.json`, or the `"snapshot"` key in `package.json` — because this wizard writes an
421
+ ES-module file and a new file beside the old one would silently shadow it. Those cases exit `1`
422
+ with a message naming the file to delete or rename. Note that this means `--force` writing
423
+ `snapshot.config.ts` does shadow a `"snapshot"` key in `package.json`, since a dedicated config
424
+ file always wins.
@@ -0,0 +1,90 @@
1
+ # How to budget LLM context files for a codebase
2
+
3
+ Context is a budget, not a bin. Every token spent on a lockfile is a token not spent on the function
4
+ being asked about, and the failure is silent — the model simply gets vaguer. This page is about
5
+ choosing the numbers and knowing what they cost.
6
+
7
+ ## Start from the destination, not the repository
8
+
9
+ Work backwards. The constraint is whatever consumes the output:
10
+
11
+ | Consumer | What actually limits you |
12
+ | :--- | :--- |
13
+ | A chat window | The window, minus room for the conversation and the answer |
14
+ | A RAG pipeline | Embedding chunk size, and how many chunks get retrieved per query |
15
+ | An agent loop | The window, minus tool schemas and minus the reasoning it has left to do |
16
+ | A review artifact | Human attention — usually far smaller than any window |
17
+
18
+ An agent needs room to *think*, so it wants the smallest context that answers the question. A chat
19
+ window can take the largest.
20
+
21
+ ## The numbers that matter
22
+
23
+ - **`maxPartBytes`** — the hard ceiling for one part. This is a promise, not an average: no part
24
+ exceeds it. Set it to the destination's limit and stop worrying about the largest file.
25
+ - **`largeFileBytes`** — above this, a file becomes a head/tail excerpt rather than its full body.
26
+ This is what stops one generated 40 MB file from consuming an entire snapshot.
27
+ - **`budgets.byLayer`** — different ceilings per layer. Documentation is usually worth less per byte
28
+ than source; `{ docs: 40_000 }` is a common shape.
29
+ - **`maxMonolithBytes`** — the single-file artifact roughly doubles output, so it is off by default
30
+ and capped when on.
31
+
32
+ ```js
33
+ const config = {
34
+ maxPartBytes: "120k",
35
+ largeFileBytes: "200k",
36
+ budgets: { default: "120k", byLayer: { docs: 40_000, config: 20_000 } },
37
+ artifacts: { monolith: false },
38
+ };
39
+ export default config;
40
+ ```
41
+
42
+ ## What each byte buys
43
+
44
+ Measured on this repository's own snapshot, as a rough guide:
45
+
46
+ | Layer | Share of bytes | Worth it? |
47
+ | :--- | :--- | :--- |
48
+ | Source | largest | yes — this is the question |
49
+ | Docs and comments | large | partly; often the best value per byte, but not when duplicated |
50
+ | Config | small | yes — it explains how the source runs |
51
+ | Tests | often 30–40% | rarely, for a code question |
52
+ | Generated and vendored | varies | almost never |
53
+
54
+ The cheapest large win in most repositories is excluding tests and build output. The second cheapest
55
+ is excluding duplicated documentation (the same content in a wiki, a `docs/` tree and a README).
56
+
57
+ ## Do not trust a size estimate you have not measured
58
+
59
+ A snapshot is auditable: `manifest.json` records `includedBytes` and `includedLines` **measured from
60
+ the block actually emitted**, not from the file on disk. That distinction matters — a truncated file
61
+ does not report its on-disk size.
62
+
63
+ ```bash
64
+ npx codebase-snapshot report
65
+ ```
66
+
67
+ `report` prints totals, per-layer metrics and the secret audit, so you can see where the budget went
68
+ before you spend it on a model call.
69
+
70
+ ## Tuning loop
71
+
72
+ 1. Generate with a ceiling that matches the destination.
73
+ 2. Run `report`; look at the per-layer breakdown.
74
+ 3. Cut the largest layer that does not answer your question.
75
+ 4. Re-generate and compare `manifest.totals.events` and the part count.
76
+ 5. Keep the settings that produced a snapshot you actually used — the config is the durable artifact,
77
+ not the output.
78
+
79
+ ## The failure this prevents
80
+
81
+ Without a ceiling, the boundary is wherever the tool stopped. Two runs can cut differently, so a
82
+ worse answer is indistinguishable from a different cut. With `maxPartBytes` fixed and output
83
+ deterministic, a changed answer means changed code — which is the only interpretation that lets you
84
+ debug it.
85
+
86
+ ## Next
87
+
88
+ - [Configuration](configuration.md) — every setting, all 25 presets
89
+ - [Snapshot output](output.md) — the full `manifest.json` schema
90
+ - [How to feed a codebase to an LLM](how-to-feed-a-repo-to-claude.md)
@@ -0,0 +1,122 @@
1
+ # How to feed a whole codebase to ChatGPT, Claude or Cursor
2
+
3
+ A model cannot clone your repository. Everything it knows about your code has to arrive as text, and
4
+ the text has to fit. This is the practical, repeated recipe — including the parts most guides skip,
5
+ which are the ones that decide whether the answer is any good.
6
+
7
+ ## The problem, stated properly
8
+
9
+ There are three separate failures, and they need different fixes:
10
+
11
+ 1. **It does not fit.** A real repository is far larger than any context window once you include
12
+ node_modules, lockfiles and build output. Naively concatenating files produces something that is
13
+ rejected, truncated by an unknown amount, or billed at a size you did not expect.
14
+ 2. **It does not fit *predictably*.** If the boundary moves between runs, you cannot tell whether a
15
+ changed answer came from changed code or a different cut of the same code.
16
+ 3. **It is not current.** The document you pasted was generated yesterday. Nobody notices until the
17
+ model confidently describes a function you deleted.
18
+
19
+ Most "paste your repo into an LLM" tooling solves only the first.
20
+
21
+ ## The recipe
22
+
23
+ ### 1. Generate from the checkout, not from a website
24
+
25
+ If the code is private, it should not be uploaded to a third-party service to be turned into text.
26
+ Everything below runs locally.
27
+
28
+ ```bash
29
+ cd /path/to/your/project
30
+ npx codebase-snapshot setup
31
+ ```
32
+
33
+ `setup` detects the project type from the repository, writes a commented `snapshot.config.mjs`, adds
34
+ the npm scripts and the devDependency, installs it, and generates. It does not overwrite a config you
35
+ already have. Use `--no-install` if you would rather not touch the lockfile.
36
+
37
+ If you prefer to see each stage:
38
+
39
+ ```bash
40
+ npx codebase-snapshot init --yes # config + scripts + dependency + .gitignore
41
+ npm install
42
+ npm run snapshot # generate
43
+ npm run snapshot:validate # exit 0 = still matches the disk
44
+ ```
45
+
46
+ ### 2. Choose the budget, not the file list
47
+
48
+ The setting that matters is `maxPartBytes` — the hard ceiling for one part. Pick it from the
49
+ destination, not from taste:
50
+
51
+ | Destination | Practical budget |
52
+ | :--- | :--- |
53
+ | A chat window | 40–120 KB per part; paste one or two |
54
+ | A RAG index | 120–300 KB per part, chunked by heading |
55
+ | An agent's context | 60–150 KB; smaller parts leave room to reason |
56
+
57
+ ```js
58
+ // snapshot.config.mjs
59
+ const config = {
60
+ maxPartBytes: "120k",
61
+ // Per-layer ceilings, when docs and code deserve different treatment.
62
+ budgets: { default: "120k", byLayer: { docs: 40_000 } },
63
+ };
64
+ export default config;
65
+ ```
66
+
67
+ A file larger than the budget is not silently dropped: it is truncated with an explicit marker
68
+ naming the budget, and files above `largeFileBytes` become a head/tail excerpt instead of a body.
69
+
70
+ ### 3. Tell the model what it is looking at
71
+
72
+ The snapshot is self-describing, and saying so up front measurably improves answers:
73
+
74
+ > The context below is a generated snapshot of a repository. `manifest.json` lists every included
75
+ > file with its language, part and hash, and every excluded file with the reason. Reads start with
76
+ > `parts/part-001.md`. If something you need is missing, ask me to check `explain <path>` rather than
77
+ > assuming it does not exist.
78
+
79
+ That last sentence matters more than it looks: models fill gaps by invention, and telling them the
80
+ gap is documented turns a hallucination into a question.
81
+
82
+ ### 4. Include the index, always
83
+
84
+ Paste `manifest.json` alongside the parts, or at least `reports/navigation.md`. Without the index the
85
+ model sees text with no map and guesses at structure. With it, it can say "this file is in part 3,
86
+ which I have not been given" instead of inventing the file.
87
+
88
+ ### 5. Re-validate before every session that matters
89
+
90
+ ```bash
91
+ npm run snapshot:validate || npm run snapshot
92
+ ```
93
+
94
+ `validate` compares each file's hash *and* a digest of the settings used to generate the snapshot. A
95
+ changed budget, layer rule or secret pattern is drift, not silence — so the check catches both code
96
+ changes and configuration changes.
97
+
98
+ ## What to exclude
99
+
100
+ The defaults already skip binaries, credential files (`.env`, `*.pem`, `.ssh/`, `.npmrc`) and
101
+ anything gitignored. Beyond that, the highest-value exclusions are usually:
102
+
103
+ ```js
104
+ excludePatterns: ["**/*.test.ts", "**/__snapshots__/**", "public/**", "*.min.js", "vendor/**"],
105
+ ```
106
+
107
+ Tests and fixtures are often 40% of a repository and rarely part of the question. Removing them buys
108
+ context for the code that is being asked about.
109
+
110
+ ## What this does not solve
111
+
112
+ - **The model still has a fixed window.** If the code genuinely exceeds it, you are choosing what to
113
+ omit — this tool makes that choice explicit and recorded rather than accidental.
114
+ - **Understanding beats volume.** A smaller snapshot with the right 20 files outperforms a large one
115
+ containing everything. Use `include` when you know the area.
116
+ - **Drift is only detected when you run the check.** It is not a watcher, unless you use `--watch`.
117
+
118
+ ## Next
119
+
120
+ - [Configuration](configuration.md) — every setting and its default
121
+ - [Snapshot output](output.md) — what each artifact contains
122
+ - [Comparing with other tools](vs-repomix.md)