codebase-snapshot 0.0.0-stage → 0.9.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Ankur Halder
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,277 @@
-# Temporary Holding Version
+# codebase-snapshot
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+**Turn any repository into an LLM-ready snapshot: code packed into size-budgeted markdown
+parts, indexed by a JSON manifest, with a secret audit — in one command.**
+
+[![npm version](https://img.shields.io/npm/v/codebase-snapshot.svg)](https://www.npmjs.com/package/codebase-snapshot)
+[![npm downloads](https://img.shields.io/npm/dw/codebase-snapshot.svg)](https://www.npmjs.com/package/codebase-snapshot)
+[![license](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/ankurhalder/codebase-snapshot/blob/main/LICENSE)
+[![node](https://img.shields.io/badge/node-%3E%3D18-brightgreen.svg)](https://nodejs.org)
+
+```bash
+npx codebase-snapshot generate
+```
+
+That writes `./snapshot/` — a directory you can drop into a chat, a prompt, a RAG pipeline or
+a repository review. It works on TypeScript and JavaScript projects, and on Python, Go, Rust,
+Java/Kotlin, C#/.NET, PHP, Ruby, C/C++, Vue, Svelte and Astro, without configuration.
+
+![How it works](https://raw.githubusercontent.com/ankurhalder/codebase-snapshot/main/docs/how-it-works.svg)
+
+---
+
+## Why
+
+Feeding a real repository to a model is awkward: files are too big, context windows are too
+small, and picking what matters by hand does not scale or repeat. `codebase-snapshot` does it
+deterministically:
+
+- **One file per part, never over budget.** `maxPartBytes` is a hard ceiling. A file bigger
+  than the budget is truncated with an explicit omission marker naming the budget rather than
+  blowing the limit — and above `largeFileBytes` it becomes a head/tail excerpt instead.
+- **An index, not just text.** `manifest.json` lists every included file with its layer, kind,
+  language, part, hash and a structural summary — and every *excluded* file with the reason.
+- **Provably current.** `validate` re-hashes the repository against the snapshot and exits
+  non-zero on any drift, so a pipeline can gate on context being fresh.
+- **Deterministic where it matters.** Two runs over unchanged input give byte-identical
+  `parts/`, `reports/navigation.md` and `reports/project-tree.md`, and the same
+  `manifest.fingerprint`. Only `manifest.json`, the snapshot's `README.md`,
+  `reports/audit.md` and `.codebase-snapshot` change between runs, because they carry
+  timestamps, a run id and a pid.
+- **Safe by default.** Credential files are skipped unless asked for, secret-shaped strings are
+  redacted, and a mistyped output directory cannot delete source code.
+- **Usable by agents, not just humans.** A machine-readable contract and an installable skill
+  let an agent with no prior knowledge drive the whole thing.
+
+## Install
+
+```bash
+npx codebase-snapshot generate      # no install, one-off
+npm install -D codebase-snapshot    # or add it to a project
+```
+
+This is a **0.x** release. Version numbers follow semver's 0.x conventions, so the CLI surface may change in a minor release until 1.0.0 — pin an exact version if you need a fixed surface today.`n`nAny npm-compatible client works — npm, yarn (classic and Berry with strict PnP), pnpm and bun —
+each verified against the packed tarball before release. Requires **Node.js 18+**.
+
+Published from GitHub Actions with [build provenance](https://docs.npmjs.com/generating-provenance-statements): npm shows the tarball was built from this repository at a tagged commit, and `npm audit signatures` verifies it.
+
+→ [Installation guide](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/install.md)
+
+## Quick start
+
+One command does the whole first run — detects the project, writes a commented config, adds the
+npm scripts and the devDependency, installs it, and generates the snapshot:
+
+```bash
+npx codebase-snapshot setup
+```
+
+```bash
+# Or step by step, if you want to see each stage:
+npx codebase-snapshot init --yes   # config + scripts + devDependency + .gitignore
+npm install                                     # the package, into node_modules
+npm run snapshot                                # generate
+npm run snapshot:validate                       # exit 0 = current with disk
+```
+
+`setup` keeps a config the repository already has, and `setup --no-install` skips the install
+(CI, offline, or a lockfile you would rather not touch).
+
+```
+📸 Generating snapshot for: /Users/you/project
+   Branch: main | Part budget: 300 KB
+
+✅ Snapshot generated in 0.14s:
+   - Tracked files: 128
+   - Total lines:   9,431
+   - Total size:    0.41 MB
+   - Parts written: 3 under snapshot/parts/
+   - Reports:       snapshot/reports/ (project-tree, navigation, audit)
+   - Manifest:      snapshot/manifest.json
+   - Fingerprint:   e4607b570268c8def0911babbace97e4a0196d9eaf2cac22c8847500883d70aa
+   - Integrity:     passed
+```
+
+The fingerprint is printed in full — all 64 hex characters, never elided — and it is the same
+value as `manifest.fingerprint`.
+
+## What you get
+
+```
+snapshot/
+├── README.md                  human summary of this snapshot
+├── manifest.json              machine index: files, parts, hashes, config, skips
+├── parts/part-001.md …        the code, grouped and size-budgeted
+└── reports/
+    ├── project-tree.md        directory tree of included files
+    ├── navigation.md          links between layers, kinds and parts
+    └── audit.md               secrets, large files, parse diagnostics, git state
+```
+
+Every artifact can be turned off (`--skip-artifacts`, `--only-artifacts`, or the config).
+
+→ [Snapshot output and manifest schema](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/output.md)
+
+## Built to be trusted with a whole repository
+
+A snapshot is only useful if you can believe it, so the tool is explicit about everything it does
+*not* contain:
+
+- **Nothing is dropped silently.** Every excluded path is recorded in `manifest.skipped[]` with a
+  reason — `binary`, `gitignore`, `default-ignore`, `sensitive-file`, `sensitive-path`, `symlink`,
+  `outside-root`, `read-error`, `not-file`, `nested-repository`, `excluded` — and `explain <path>`
+  answers the same question for any single path.
+- **Encoding is stated, not guessed.** UTF-16 files are transcoded instead of vanishing as
+  "binary"; a file that is not valid UTF-8 is kept but labelled `utf-8-lossy` with the count of
+  undecodable bytes, so you never mistake mojibake for source.
+- **Counts describe what was emitted.** `includedBytes` and `includedLines` are measured from the
+  block in the part, not from the file on disk.
+- **Drift is checked, not assumed.** `validate` compares every file's hash *and* a digest of the
+  settings the snapshot was generated with, so a changed budget, layer rule or secret pattern is
+  drift rather than silence.
+- **Memory is bounded.** Files above 2 MB are not parsed for a structural summary (that is where a 50 MB file
+  used to reach 3 GB of resident memory), and 130 000 files in one directory or a 3 000-level path
+  scan without crashing.
+- **`diff` compares payloads, not just paths.** Two snapshots of identical sources with different
+  settings say so instead of reporting "0 changed".
+- **When git cannot list files** — an interrupted clone, a dehydrated `.git`, `safe.directory` —
+  the run refuses rather than quietly scanning something different, and `--discovery walk` is the
+  documented escape hatch.
+
+## Supported projects
+
+25 presets, detected automatically from the repository — each contributing layers, kinds,
+ecosystem ignore patterns and a recommended part budget.
+
+| | | | |
+| :--- | :--- | :--- | :--- |
+| Next.js | Nuxt | SvelteKit | Remix |
+| Astro | Angular | Solid | Vue + Vite |
+| React + Vite | NestJS | Express / Fastify / Koa | Node library |
+| Monorepo | Django | FastAPI / Flask | Python library |
+| Go | Rust | Java / Spring | .NET |
+| Laravel (PHP) | Rails (Ruby) | Terraform / Docker infra | Docs-only |
+| generic fallback | | | |
+
+Non-TypeScript languages get import/export/class/function summaries rather than being treated
+as opaque text. Vue, Svelte and Astro script blocks go through the real TypeScript compiler.
+Every summary is labelled `ast`, `structural` or `heuristic` so you know how much to trust it —
+Python, for example, is a line-scan `heuristic`, while JSON and Markdown get `structural`
+shapes.
+
+→ [Preset table and detection rules](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/configuration.md#project-presets)
+
+## Commands
+
+| Command | Does |
+| :--- | :--- |
+| `generate` *(default)* | Write the snapshot. |
+| `validate` / `check` | Compare the snapshot with the repository; exit `1` on drift. |
+| `report` | Snapshot health, layer metrics, secret audit. |
+| `init` | Interactive (or `--yes`) setup that writes a self-documenting config. |
+| `explain <path>` | Why one path is included, which part it lands in, or why it was skipped. |
+| `diff <a> <b>` | Compare two snapshots. |
+| `describe` | The machine-readable contract (`--format json`). |
+| `skill` | Print or install the agent skill. |
+
+Exit codes: `0` success · `1` drift or refused generation · `2` usage error.
+
+→ [Full CLI reference](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/cli.md)
+
+## For AI agents
+
+Everything an agent needs is available as data, and the tool can install its own skill:
+
+```bash
+# The contract: workflow, every option and default, exit codes, presets, safety rules
+npx codebase-snapshot describe --format json
+
+# Install the skill into the registries agent tools read: .claude/skills/, .cursor/rules/
+# (it adds those files to .gitignore, and never touches your own AGENTS.md)
+npx codebase-snapshot skill --install
+```
+
+An agent told only "use `npx codebase-snapshot`" can read the contract, follow its
+ordered workflow, and produce a validated snapshot unaided — that path is covered by a test.
+
+→ [Using this with an AI agent](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/agent.md) · [AGENTS.md](https://github.com/ankurhalder/codebase-snapshot/blob/main/AGENTS.md) · [llms.txt](https://github.com/ankurhalder/codebase-snapshot/blob/main/llms.txt)
+
+## Configuration
+
+`snapshot.config.ts` (or `.mjs`) is written by `init` with **every** setting at its current
+value, commented and grouped, so it doubles as the reference. Deleting any line is safe: the
+defaults are real. The object is exported through a name — `export default { … }` is
+`import/no-anonymous-default-export`, on by default in `eslint-config-next`.
+
+```js
+const config = {
+  maxPartBytes: "120k",                                   // hard budget per part
+  excludePatterns: ["vendor/**", "**/*.min.js"],
+  artifacts: { monolith: false },                         // single-file output stays off
+  budgets: { default: "120k", byLayer: { docs: 40_000 } },
+  includeSensitiveFiles: false,                           // .env, *.pem … stay out
+};
+
+export default config;
+```
+
+→ [Configuration reference](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/configuration.md)
+
+## Programmatic API
+
+```js
+import { generateSnapshot, resolveConfig, validateSnapshot, loadConfigFile } from "codebase-snapshot";
+
+// Pass the repository explicitly: loadConfigFile returns only the config it found, so a
+// directory with no config file would leave resolveConfig scanning process.cwd() instead.
+const dir = "/path/to/repo";
+const config = resolveConfig(await loadConfigFile(dir), { cwd: dir });
+const manifest = generateSnapshot(config);
+const validation = validateSnapshot(config);
+```
+
+CommonJS works too (`require("codebase-snapshot")`), with type definitions for
+both. The CLI and the API share one implementation, so anything the CLI does is available here.
+
+## Documentation
+
+| | |
+| :--- | :--- |
+| [Installation](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/install.md) | Every client, version pinning, agent skill install |
+| [CLI reference](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/cli.md) | Commands, options, exit codes, examples |
+| [Configuration](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/configuration.md) | Every setting, all 25 presets, budgets, hooks, secrets |
+| [Snapshot output](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/output.md) | Directory layout and the full manifest schema |
+| [AI agents](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/agent.md) | The contract, the skill, what an agent should avoid |
+| [Troubleshooting](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/troubleshooting.md) | Every message and what to do about it |
+| [Changelog](https://github.com/ankurhalder/codebase-snapshot/blob/main/CHANGELOG.md) | Release history and breaking changes |
+
+## Guides
+
+| | |
+| :--- | :--- |
+| [Feeding a codebase to ChatGPT, Claude or Cursor](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/how-to-feed-a-repo-to-claude.md) | The full recipe, including the parts most guides skip |
+| [Budgeting LLM context](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/how-to-context-budget.md) | Choosing the numbers from the destination, not from taste |
+| [vs repomix](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/vs-repomix.md) | Including where repomix is the better choice |
+| [vs gitingest](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/vs-gitingest.md) | Including where gitingest is the better choice |
+| [vs code2prompt](https://github.com/ankurhalder/codebase-snapshot/blob/main/docs/vs-code2prompt.md) | Including where code2prompt is the better choice |
+
+## Safety
+
+- **Credential files are excluded by default** — `.env`, `*.pem`, `*.key`, `.npmrc`, `.netrc`,
+  `.git-credentials`, `id_rsa`, `.kube/config`, `*.tfstate` — and listed in
+  `manifest.skipped[]` with the reason.
+- **Secrets are redacted** inside ordinary files: only the matched span is masked, never the
+  whole line, and no excerpt reaches any artifact.
+- **The output directory is owned by the tool.** Each run is written to a scratch directory and
+  swapped in atomically; the previous snapshot survives a failure.
+- **It refuses to replace a directory it did not create**, so `--out-dir src` cannot destroy
+  source code.
+
+## Contributing
+
+Issues and pull requests are welcome — see [CONTRIBUTING.md](https://github.com/ankurhalder/codebase-snapshot/blob/main/CONTRIBUTING.md) and the
+[code of conduct](https://github.com/ankurhalder/codebase-snapshot/blob/main/CODE_OF_CONDUCT.md). Security reports: [SECURITY.md](https://github.com/ankurhalder/codebase-snapshot/blob/main/SECURITY.md).
+
+## License
+
+[MIT](https://github.com/ankurhalder/codebase-snapshot/blob/main/LICENSE) © Ankur Halder

package/SECURITY.md

@@ -0,0 +1,110 @@
+# Security policy
+
+## Supported versions
+
+Only the latest published release receives security fixes.
+
+| Version | Supported |
+| :--- | :--- |
+| 4.x | ✅ |
+| 3.x and older | ❌ |
+
+## Reporting a vulnerability
+
+**Do not open a public issue for a security problem.**
+
+Email **ankur.halder12345@gmail.com** with:
+
+- what the issue is and which version you reproduced it on,
+- the smallest reproduction you can construct,
+- the impact you believe it has.
+
+You can expect an acknowledgement within a few days. If the report is valid, a fix and a
+release will follow, and you will be credited in the release notes unless you ask otherwise.
+
+## Scope
+
+`codebase-snapshot` reads a repository and writes files. The security-relevant surfaces are:
+
+**Secret handling.** Credential-bearing files (`.env`, `*.pem`, `*.key`, `.npmrc`, `.netrc`,
+`.git-credentials`, `id_rsa`, `.kube/config`, `*.tfstate` and similar) are excluded by default.
+Secret-shaped strings inside ordinary files are redacted in the emitted parts; only the matched
+span is masked and no excerpt of a matched line is written to any artifact.
+
+Detection is **pattern-based**, so it recognises the shapes it knows and nothing else. A
+secret deliberately split across two string literals, joined with a backslash line
+continuation, or assembled by template-literal concatenation can be missed, because the
+pattern never sees the whole value on one line.
+
+In YAML and similar `key: value` files, an **unquoted** value that reads like an expression is
+treated as code rather than as a literal and is not masked: `password: opts.db.password` and
+`password: my.secret.value` are emitted as written, and so is a value that starts with
+punctuation, such as `token: !hunter2hunter2`. Quote the value — `password: "…"` — to have it
+redacted; a plain unquoted value with no dots or leading punctuation is masked.
+
+A clean audit is therefore evidence that the known patterns did not fire — not a guarantee
+that the repository holds no credentials.
+
+Reports we want to hear about: a pattern that is *not* detected, a redaction that leaves the
+secret readable elsewhere (a report file, the manifest, a navigation index), or a way to make
+the tool emit a sensitive file without `--include-sensitive-files`.
+
+**Filesystem safety.** The output directory is written to a scratch directory and swapped in
+atomically, and it is only replaced when a previous run wrote it. Ownership is proved by the
+`.codebase-snapshot` marker — a JSON object naming this generator, with a schema version, a
+per-snapshot id and the entries it wrote — not by a filename and not by a manifest alone.
+`outDir`/`tmpDir` may not be the working directory, an ancestor, or the filesystem root, and
+may not contain one another.
+
+The marker is **evidence, not authentication**. It is unsigned and carries no secret, so anyone
+who can write a marker and a stub manifest with a matching digest can satisfy the check — but
+anyone who can write those files can also delete the directory by hand. The guard exists to stop
+the tool from deleting a directory *it* did not create; it is not a defence against an attacker
+who already has write access to the output path.
+
+Reports we want to hear about: any path configuration that deletes or overwrites files the tool
+did not create.
+
+**Supply chain.** The package has exactly one runtime dependency (`typescript`), publishes with
+provenance from a pinned GitHub Actions workflow, and every third-party action in that workflow is pinned
+to a full commit SHA.
+
+## Trust boundary
+
+A configuration file is **code, and it runs**.
+
+`snapshot.config.ts`, `.js`, `.mjs` and `.cjs` are imported by the CLI and executed in your
+Node process. That is deliberate and cannot be avoided: a config that can express a `RegExp`
+or a `match` function — and both are documented — has to be evaluable code. Only
+`snapshot.config.json`, `.snapshotrc`, `.snapshotrc.json` and the `"snapshot"` key of
+`package.json` are inert: they are parsed as data, never executed.
+
+So running the tool inside a repository loads the code that repository's config contains. The
+repository you scan is not necessarily a repository you own — a vendor drop, a dependency
+checkout, a pull request under review, a CI job that scans contributed code.
+
+- **`--no-config` avoids it.** The flag ignores every config file and runs on the built-in
+  defaults plus the detected preset, so nothing from the repository is imported.
+- **Do not run the tool in a repository you do not trust** without `--no-config` — and do not
+  point `--cwd` at one. Review the config first, or run the scan somewhere with no credentials
+  in reach.
+- **`hooks.afterGenerate` is the same surface.** It is a function in the config, called after
+  a successful run, in the process working directory, with the manifest as its argument.
+
+This is a property of configuration-as-code, not a bug to be reported. What *is* in scope is
+anything that widens it: a config format that executes when the documentation says it is
+inert, a config loaded from a location the docs do not name, or a path where config code runs
+when `--no-config` was passed.
+
+## Not in scope
+
+- The declared behaviour of `--include-sensitive-files` or a `customPatterns` redaction rule the
+  user configured themselves.
+- Obfuscated forms: a credential packed inside an archive, compressed, or otherwise not
+  recognisable by shape is not something the redaction rules can find, and a report that one
+  survived this way is out of scope rather than a defect. One obfuscation *is* covered: a
+  base64-encoded PEM file, whose encoded form begins `LS0tLS1CRUdJTi` (`-----BEGIN `), is matched
+  by the `base64-private-key` rule without decoding it.
+- Secrets that are already committed to a public repository, unless the tool emits them into a
+  snapshot they would otherwise not appear in.
+- Anything that requires an attacker to already control the machine running the tool.