@ankurhalder/codebase-snapshot 2.2.0 → 4.0.0

package/docs/install.md

@@ -1,39 +1,33 @@
 # Installation
 
-`@ankurhalder/codebase-snapshot` is published to the public npm registry, so it installs
-with **any** npm-compatible client. There is nothing extra to configure and no second
-registry to add.
+`@ankurhalder/codebase-snapshot` is published to the public npm registry, so it installs with
+**any** npm-compatible client. There is no second registry to add and nothing to configure.
 
 Requires **Node.js 18 or newer**.
 
-## One-off use (no install)
+## One-off use
 
 ```bash
 npx @ankurhalder/codebase-snapshot generate
 ```
 
-If that exits with **no output and no error** instead of generating, your npm has
-resolved the bare spec to a placeholder manifest that carries no executable — npm creates
-a `0.0.0-stage` placeholder version during staged publishing, and some npm builds pick it
-when no version is given. Pin the version or the tag and it works:
+This is the shortest path and needs no install. The command is also available as
+`codebase-snapshot` and, once installed, as the short alias `snap`.
+
+Pin the version when you need the same build on every machine:
 
 ```bash
 npx @ankurhalder/codebase-snapshot@latest generate
-npx @ankurhalder/codebase-snapshot@2 generate
+npx @ankurhalder/codebase-snapshot@4 generate
 ```
 
-`npx -p @ankurhalder/codebase-snapshot codebase-snapshot generate`, `pnpm dlx`,
-`yarn dlx` and `bunx` are unaffected. A normal `npm install -D` followed by `npx snap`
-is unaffected too, and is the more reproducible choice for a project.
-
-Equivalent one-liners for other clients:
+**Why the bare name works.** npm resolves which binary to run from the package's `bin`
+entries. This package declares `codebase-snapshot` and `snap` pointing at the *same file*, so
+npm treats them as aliases and can always resolve one. Keep them aliases: two *different*
+executables under a scoped package name would make the bare form unresolvable.
 
-```bash
-yarn dlx -p @ankurhalder/codebase-snapshot snap generate   # yarn Berry
-yarn global add @ankurhalder/codebase-snapshot             # yarn classic (installs globally)
-pnpm dlx @ankurhalder/codebase-snapshot generate
-bunx @ankurhalder/codebase-snapshot generate
-```
+If a command exits with no output at all, a stale global install is the usual cause — see
+[Troubleshooting](#troubleshooting).
 
 ## Add to a project
 
@@ -44,18 +38,17 @@ pnpm add -D @ankurhalder/codebase-snapshot
 bun add -d @ankurhalder/codebase-snapshot
 ```
 
-Then run it through your client so it resolves from the project's `node_modules`:
+Then run it through your client so the binary resolves from the project:
 
-```bash
-npx snap generate
-yarn snap generate
-pnpm exec snap generate
-bun run snap generate
-```
+| Client | Run |
+| :--- | :--- |
+| npm | `npx codebase-snapshot generate` |
+| yarn (classic and Berry) | `yarn codebase-snapshot generate` |
+| pnpm | `pnpm exec codebase-snapshot generate` |
+| bun | `bun run codebase-snapshot generate` |
 
-Both `snap` and `codebase-snapshot` are exposed as binaries — pick whichever you prefer.
-
-Adding it to `package.json` scripts works the same everywhere:
+`snap` works everywhere `codebase-snapshot` does. Adding it to `package.json` scripts is the
+most reproducible option:
 
 ```json
 {
@@ -66,85 +59,123 @@ Adding it to `package.json` scripts works the same everywhere:
 }
 ```
 
-## Global install
+`codebase-snapshot init` writes these entries for you.
+
+## One-off runners for other clients
 
 ```bash
-npm install -g @ankurhalder/codebase-snapshot
-pnpm add -g @ankurhalder/codebase-snapshot
-bun add -g @ankurhalder/codebase-snapshot
+yarn dlx -p @ankurhalder/codebase-snapshot codebase-snapshot generate   # yarn Berry
+pnpm dlx @ankurhalder/codebase-snapshot generate
+bunx @ankurhalder/codebase-snapshot generate
 ```
 
-A global install exists for convenience, but pinning the version per project (the
-`-D`/`--save-dev` examples above) is the better default: snapshots are only reproducible
-if the generator version is reproducible.
+## Agent skill
 
-## Version pinning
+To make a repository discoverable by an AI agent — so it finds this tool without being told
+it exists — run:
 
 ```bash
-npx @ankurhalder/codebase-snapshot@2 generate
-pnpm dlx @ankurhalder/codebase-snapshot@2.2.0 generate
+npx @ankurhalder/codebase-snapshot skill --install
 ```
 
+That writes an `AGENTS.md` section, `.claude/skills/codebase-snapshot/SKILL.md` and
+`.cursor/rules/codebase-snapshot.mdc`. Re-running only replaces its own marked block. See
+[Using this with an AI agent](agent.md).
+
 ## Verified clients
 
-Every client below was used to install the published package and run
-`snap --version` successfully. This matrix is enforced in CI
-(`.github/workflows/ci.yml` → `client-install`), so a change that breaks a client fails
-the build rather than a user's terminal.
+Every client below installs the published package and runs it successfully; the release check
+enforces this on every pull request, so a change that breaks one fails the build rather than
+a user's terminal.
 
-| Client | Version tested | Run installed binary with |
-| :--- | :--- | :--- |
-| npm | 10.x | `npx snap` |
-| yarn (classic) | 1.22.22 | `yarn snap` |
-| yarn (Berry, `nodeLinker: pnp`) | 4.6.0 | `yarn snap` |
-| pnpm | 10.28.0 | `pnpm exec snap` |
-| bun | 1.4.4 | `bun run snap` |
+| Client | Run installed binary with |
+| :--- | :--- |
+| npm 10–11 | `npx codebase-snapshot` |
+| yarn classic 1.22 | `yarn codebase-snapshot` |
+| yarn Berry 4.6 (`nodeLinker: pnp`) | `yarn codebase-snapshot` |
+| pnpm 10 | `pnpm exec codebase-snapshot` |
+| bun 1.4 | `bun run codebase-snapshot` |
 
-Yarn Berry's default **strict PnP** mode works: there is no `node_modules` directory, and
-the runtime `typescript` dependency still resolves through yarn's `.pnp.cjs` loader.
+Yarn Berry's default **strict PnP** works: there is no `node_modules` directory, and the
+runtime `typescript` dependency still resolves through yarn's `.pnp.cjs` loader.
 
-## Package manager workspaces
+## Before scanning a repository you do not own
 
-The binary is resolved from the workspace it is installed in. In a monorepo, install it
-in the package that runs the snapshot (usually the root), or use any client's one-off
-runner (`npx` / `pnpm dlx` / `yarn dlx` / `bunx`) which works from any subdirectory.
+A `snapshot.config.ts`, `.js`, `.mjs` or `.cjs` in the target repository is **imported and
+executed** when the tool runs there — that is what lets a config express a `RegExp` or a
+`match` function. Pointing `--cwd` at a checkout you do not trust therefore runs its code.
+`--no-config` ignores every config file and uses the built-in defaults plus the detected
+preset, and JSON configs are parsed rather than executed. See
+[the trust boundary](../SECURITY.md#trust-boundary).
 
-## Uninstalling
+## Global install
 
-Standard for your client (`npm uninstall`, `yarn remove`, `pnpm remove`, `bun remove`).
-Snapshots already written to `snapshot/` are ordinary files and are left alone.
+```bash
+npm install -g @ankurhalder/codebase-snapshot
+```
 
-## Not supported yet
+Convenient, but prefer pinning per project: snapshots are only reproducible if the generator
+version is. A global install also shadows `npx` for this package — npm will run your global
+copy instead of the published one, which is confusing when the two differ. That is exactly
+the situation described under [Troubleshooting](#troubleshooting).
 
-- **`pip`, `brew`, `scoop`, `docker`, and standalone binaries.** These require shipping a
-  prebuilt executable rather than JavaScript, and are tracked as future work in
-  [docs/plans/distribution](../plans/distribution/README.md). Until then, any machine
-  running the tool needs Node 18+.
-- **Deno.** The API can be imported under Deno's npm compatibility, but running the CLI
-  needs a `./bin` subpath export that is not published yet.
+## Workspaces
 
-## Troubleshooting
+The binary resolves from the workspace it is installed in. In a monorepo, install it in the
+package that runs the snapshot (usually the root), or use a one-off runner
+(`npx` / `pnpm dlx` / `yarn dlx` / `bunx`) which works from any subdirectory.
 
-**`snap: command not found`** — when installed locally (not globally), the binary lives
-in `node_modules/.bin` and is only on your `PATH` inside a package script or a client
-runner. Use `npx snap`, `pnpm exec snap`, `yarn snap`, or `bun run snap`.
+## Upgrading
 
-**`Cannot find module 'typescript'`** — the package declares `typescript` as a runtime
-dependency for AST analysis. If your client prunes or externalizes it (some bundlers and
-`--ignore-scripts` CI setups do), rerun a clean install, or pin `typescript@^5` yourself.
+```bash
+npm install -D @ankurhalder/codebase-snapshot@latest
+npx codebase-snapshot generate     # rewrite the snapshot with the new version
+```
+
+`manifest.generator.version` records which version produced a snapshot. Re-run `generate`
+after an upgrade so the snapshot, and the version it records, match the build you are now
+running.
 
-**CI has no Node** — that is not supported yet; this package is JavaScript and needs a
-Node runtime. See the future-work note above.
+A snapshot written by **3.0.0 or earlier** is refused rather than replaced: its
+`.codebase-snapshot` marker has no per-snapshot id, so the current build cannot prove it wrote
+the directory. Delete the old snapshot directory and generate again.
 
-**`yarn snap` prints extra lines, or breaks a script that parses output** — yarn classic
-wraps everything: it echoes `yarn run 1.22.22`, the resolved command, your tool's output,
-and a `Done in 0.37s.` trailer. For a human that is harmless; for anything that parses
-`snap`'s stdout it is not. In scripts and CI, call the binary directly:
+See [CHANGELOG](../CHANGELOG.md) for breaking changes — 3.0.0 removed nothing, but it briefly
+did in error; 4.0.0 restored the `snap` alias.
+
+## Uninstalling
+
+Standard for your client (`npm uninstall`, `yarn remove`, `pnpm remove`, `bun remove`).
+Snapshots already written to the output directory are ordinary files and are left alone.
+If you installed globally, `npm rm -g @ankurhalder/codebase-snapshot` also removes the
+shim that shadows `npx`.
+
+## Troubleshooting
+
+**`codebase-snapshot: command not found`** — when installed locally (not globally) the binary
+lives in `node_modules/.bin` and is only on `PATH` inside a package script or a client
+runner. Use `npx codebase-snapshot`, `pnpm exec codebase-snapshot`,
+`yarn codebase-snapshot` or `bun run codebase-snapshot`.
+
+**A command exits 0 and prints nothing** — almost always a stale **global** install
+shadowing `npx`. Check and remove it:
 
 ```bash
-./node_modules/.bin/snap generate
-npx snap generate          # same, without the yarn wrapper
+npm ls -g --depth=0 | grep codebase-snapshot
+npm rm -g @ankurhalder/codebase-snapshot
 ```
 
-This is why the CI job in this repository reads the version from a line that *is* the
-version rather than taking the first or last line of output.
+Then re-run. If the package is not installed globally, clear the npx cache
+(`npm cache clean --force`) and retry.
+
+**`Cannot find module 'typescript'`** — the package declares `typescript` as a runtime
+dependency for AST analysis. If your client prunes it (some `--ignore-scripts` CI setups do),
+rerun a clean install or pin `typescript@^5` yourself.
+
+**`yarn codebase-snapshot` prints extra lines / breaks a script** — yarn classic wraps
+everything: it echoes `yarn run 1.22.x`, the resolved command, your output, then
+`Done in 0.37s.`. For anything that parses the output, call the binary directly
+(`./node_modules/.bin/codebase-snapshot` or `npx codebase-snapshot`).
+
+More symptoms — refusals, config errors, drift, missing files — are in
+[Troubleshooting](troubleshooting.md).

package/docs/output.md

@@ -0,0 +1,213 @@
+# Snapshot output
+
+A run writes one directory (default `./snapshot`) that is meant to be handed to a model as
+context. It is self-describing: `manifest.json` indexes everything the parts contain and
+explains everything they omit.
+
+## Layout
+
+```
+snapshot/
+├── .codebase-snapshot          ownership marker (see Safety)
+├── README.md                   human summary of this snapshot
+├── manifest.json               machine index: files, parts, config, hashes, skips
+├── parts/
+│   ├── part-001.md             the code, grouped and size-budgeted
+│   ├── part-002.md
+│   └── …
+└── reports/
+    ├── project-tree.md         directory tree of included files
+    ├── navigation.md           links between layers, kinds and parts
+    └── audit.md                secrets, large files, parse diagnostics, git state
+```
+
+With `--include-monolith` you also get `full-codebase.json` and `full-codebase.md` — the
+whole repository in one file. They are off by default because they grow without bound;
+`--max-monolith-bytes` skips them when they would exceed a size you set.
+
+Every artifact can be turned off. `--skip-artifacts parts` omits the parts,
+`--only-artifacts manifest` writes nothing but the manifest, and `config.artifacts` does the
+same persistently. The ownership marker is always written.
+
+## Parts
+
+Each `part-NNN.md` holds whole files under the byte budget (`maxPartBytes`, default 300 KB).
+A file larger than the budget is not split mid-file: it is emitted as a head/tail preview and
+marked as truncated. Each file section is tagged with its layer, kind, language and a
+structural summary (imports, exports, classes, functions) so a reader can navigate without
+opening every file.
+
+Parts are packed in a stable order, so the same input produces the same output — which is
+what makes `manifest.fingerprint` meaningful.
+
+## Manifest
+
+`manifest.json` is the index and the contract between runs. Top-level keys:
+
+| Key | Contents |
+| :--- | :--- |
+| `schemaVersion` | Manifest format version (`3.0.0`). |
+| `generatedAt` | ISO timestamp of the run. |
+| `generator` | `{ name, version }` of the tool that wrote it. |
+| `git` | `{ branch, commit, isDirty }`. `branch` falls back to the configured branch on a detached HEAD and outside a repository; `commit` is `"unknown"` and `isDirty` is `false` when git cannot report. A `false` there means "not known to be dirty", not "clean tree" — check `commit` to tell the two apart. |
+| `budgets` | The effective byte limits for the run: `{ maxPartBytes, largeFileBytes }`. The per-layer overrides live at `config.budgets`, which is `{ default, byLayer }`. |
+| `config` | The resolved configuration actually used (preset, artifacts, sizing, scoping). |
+| `counts` | `{ totalFiles, totalSizeBytes, totalLines, partsCount, skippedCount, largeFileCount, truncatedPreviewCount }`. |
+| `totals` | Bookkeeping: `{ candidateFiles, includedFiles, fullFiles, previewFiles, skippedFiles, totalSourceBytes, totalContentBytes, partCount, warningCount }`. |
+| `fingerprint` | Stable hash of the snapshot content. Two runs over unchanged input produce the same value. |
+| `validation` | `{ status: "passed" \| "failed", errors: [] }` for the run's own integrity check. |
+| `artifacts` | The optional monolith files: `{ fullJson, fullMarkdown }`. Each value is `null` when that file was not written, otherwise `{ byteSize, sha256 }`. The object itself is always present. |
+| `layers`, `kinds` | File counts per layer and per kind. |
+| `indexes` | Cross-references: `routes`, `components`, `handlers`, `features`, `imports`, `directives`. |
+| `parts` | One entry per part (below). |
+| `files` | One entry per included file (below). |
+| `skipped` | One entry per excluded file (below). |
+| `warnings` | Non-fatal observations, including a throwing `hooks.afterGenerate`. |
+
+### `parts[]`
+
+```json
+{
+  "index": 1,
+  "filename": "part-001.md",
+  "byteSize": 1432,
+  "sha256": "3472b9edcd504403f2416606e4222f326ec207d7916bf928ade4ec7de1406d2d",
+  "lines": 91,
+  "fileCount": 2,
+  "layersCovered": ["root", "src"],
+  "includedFiles": [
+    { "relativePath": "package.json", "sourceBytes": 35, "includedBytes": 35, "lines": 1, "layer": "root", "kind": "data" },
+    { "relativePath": "src/math.ts", "sourceBytes": 63, "includedBytes": 63, "lines": 3, "layer": "src", "kind": "other" }
+  ]
+}
+```
+
+`byteSize` and `sha256` are what `validate` checks against the file on disk. Both are full
+values — the hash is the whole 64-character digest, never elided.
+
+### `files[]`
+
+```json
+{
+  "relativePath": "src/math.ts",
+  "sizeBytes": 63,
+  "lines": 3,
+  "hashSha256": "aa229260647049a42a0446f48f1b34f19657921b7a0a27a41c0b2a294a25c919",
+  "part": "part-001.md",
+  "layer": "src",
+  "kind": "other",
+  "language": "typescript",
+  "inclusionMode": "full",
+  "isLarge": false,
+  "isPreviewTruncated": false,
+  "originalLines": 3,
+  "includedLines": 3,
+  "includedBytes": 63,
+  "summary": {
+    "directives": [], "imports": [], "exports": ["add"], "reExports": [],
+    "functions": ["add"], "classes": [], "interfaces": [], "types": [],
+    "enums": [], "variables": [], "reactComponents": [], "hooks": [],
+    "routeHandlers": []
+  },
+  "summaryQuality": "ast"
+}
+```
+
+- `isLarge` is a fact about the file: it is bigger than `largeFileBytes` (default 200 KB).
+  A large file is emitted as a head/tail excerpt rather than inlined whole.
+- `isPreviewTruncated` is a fact about what was written: the body in this snapshot is **not**
+  the whole file, because lines were dropped.
+- `inclusionMode` is `preview` exactly when `isPreviewTruncated` is `true`, and there are two
+  routes to it:
+  - **Over `largeFileBytes`** — the head/tail excerpt drops the middle, so
+    `isLarge: true`, `isPreviewTruncated: true`. If the head and tail happen to cover every
+    line, nothing is dropped and the entry stays `isLarge: true`, `inclusionMode: "full"`.
+  - **Under `largeFileBytes` but too big for its part** — the partitioner byte-truncates the
+    section to keep the part inside its budget, so `isLarge: false`,
+    `isPreviewTruncated: true`. This is the one that surprises people: the file is not "large"
+    by the configured threshold, yet its body is incomplete.
+- Only `isPreviewTruncated` tells you content is missing; `isLarge` alone does not.
+- `hashSha256` is the hash of the source file on disk — `validate` recomputes it to detect
+  drift. It is the full 64-character digest.
+- `summary` and `summaryQuality` are **conditional**. Both keys are present for a language
+  with an analyser and both are absent for a shapeless one — a `.gitignore`, a `.txt`. Treat a
+  missing `summary` as "no structural information", not as an empty one.
+- `summary` has one of three shapes. Code files get the thirteen keys above (`directives`,
+  `imports`, `exports`, `reExports`, `functions`, `classes`, `interfaces`, `types`, `enums`,
+  `variables`, `reactComponents`, `hooks`, `routeHandlers`), plus `parseErrors` when the parser
+  reported syntax errors. JSON gets `{ shape, itemCount, keys, sampleKeys }` and Markdown gets
+  `{ headings, frontmatterKeys }` — a shape description rather than a code summary.
+- `summaryOmitted: true` means the file's summary was dropped because the summary alone would
+  not fit the part budget. The entry then has `summaryQuality` but no `summary`, and the
+  file's body is still emitted in full. It is only ever present when `true`.
+- `summaryQuality` is `ast` (parsed by the TypeScript compiler), `structural` (JSON or
+  Markdown shape) or `heuristic` (pattern-matched). It tells you how much to trust the
+  summary: `heuristic` means a line scan, not a parse.
+
+Querying this array is usually faster and cheaper than re-reading the parts.
+
+### `skipped[]`
+
+```json
+{ "path": ".env", "reason": "sensitive-file", "detail": "credential-bearing file; enable includeSensitiveFiles to emit it" }
+```
+
+Reasons, and what each one means:
+
+| `reason` | The file was dropped because |
+| :--- | :--- |
+| `excluded` | It matched your own `excludePatterns`, or fell outside a non-empty `include` allow-list. |
+| `default-ignore` | A built-in rule caught it, the preset's ecosystem ignores did (`node_modules`, `__pycache__`, `target`, …), or it sat inside an output directory. |
+| `sensitive-file` | Its name is credential-bearing (`.env`, `*.pem`, `.npmrc`, …). |
+| `sensitive-path` | The path itself matched a secret rule. The file is skipped and the path is masked in every artifact. |
+| `binary` | It is not text. |
+| `symlink` | It is a link, and links are not followed. A directory link names its target in `detail`. |
+| `outside-root` | It is a link whose target resolves outside the scanned root. |
+| `gitignore` | A `.gitignore` excluded it, in a scan that is not reading git's index. |
+| `read-error` | It could not be read. |
+| `not-file` | It is a directory, socket or FIFO. |
+
+Every path the scanner discovers is recorded — nothing is dropped silently, in either discovery
+mode. Inside a git repository the candidate list comes from git, which applies `.gitignore`
+before the scanner runs; the ignored paths are then listed separately and recorded here too, so
+a `.gitignore`d file or directory appears in `skipped[]` with reason `gitignore` whether or not
+git is present.
+
+`explain` reports two further reasons that never appear here, because they describe a path
+rather than a decision about a candidate file: `missing` (nothing exists at that path) and
+`not-discovered` (the path exists but was never a candidate — inside `node_modules`, for
+example). See [Troubleshooting](troubleshooting.md#a-file-is-missing-from-the-snapshot).
+
+## Safety
+
+- **Sensitive files are excluded by default.** `.env`, `*.pem`, `*.key`, `.npmrc`, `.netrc`,
+  `.git-credentials`, `id_rsa`, `.kube/config`, `*.tfstate` and similar are skipped and
+  listed in `skipped[]`. `--include-sensitive-files` opts in.
+- **Secret-shaped strings are redacted** inside ordinary source files. Only the matched span
+  is masked, never the whole line, and no excerpt of a matched line is written to any
+  artifact.
+- **The output directory is owned by the tool.** Each run writes `.codebase-snapshot` into it
+  first, then swaps the new snapshot in atomically. A previous snapshot survives until the
+  replacement is complete.
+- **The tool refuses to replace a directory it did not write.** Pointing `--out-dir` at
+  `src/` is refused with exit `1` and the directory is left untouched — a mistyped output
+  path cannot delete source code. Empty directories, directories containing only
+  `.gitkeep`/`.gitignore`/`.DS_Store`, and directories holding a complete snapshot this build
+  wrote are accepted.
+- **Ownership is the marker, not a filename.** `.codebase-snapshot` is a JSON record carrying
+  this generator's name and `version`, the marker's `schemaVersion`, a `createdAt` timestamp,
+  a per-snapshot `id`, the run `state` (`staging` while it is being written, `complete` once it
+  is whole), the `manifestSha256` digest of the manifest it shipped, and `entries` — the
+  inventory of everything it wrote. A zero-byte marker, a marker that is not JSON, a marker
+  written by 3.0.0 (no `id`), a bare `manifest.json`, and a directory with files you added
+  after the run are all refused.
+  See [Troubleshooting](troubleshooting.md#refusing-to-replace-outdir-).
+
+The marker also makes hook-written artifacts safe: the entries a `hooks.afterGenerate` adds to
+the output are recorded in the marker, so they do not make the next run refuse the directory.
+
+## Committing the snapshot
+
+Usually you do not. Add the output directory to `.gitignore` — `codebase-snapshot init` does
+it for you. If you do commit snapshots (for example, to diff context across releases),
+`manifest.json` is the file that matters: it is compact and `diff` can compare two of them.