@ajdev0/token-shrink 2.0.0

package/README.md

@@ -0,0 +1,345 @@
+# token-shrink
+
+A **local-first, framework-aware token reduction engine** — a polyglot AST semantic proxy and MCP server. It prunes implementation bodies out of **dependency files** while preserving every type signature, interface, and module export, so LLM agents see the full shape of the code at a fraction of the tokens.
+
+> **The 80–90% reduction target**: full type information, no implementation noise. Ring 0 (your active file) stays complete; Ring 1 (its direct imports) is delivered as pruned skeletons.
+
+---
+
+## How it works
+
+```
+       active file                       imports (Ring 1)
+  ┌──────────────────┐       ┌──────────────────────┐
+  │  src/page.ts     │  ──►  │  src/util.ts         │
+  └──────────────────┘       └──────────────────────┘
+          ▾                              ▾
+  tree-sitter (WASM) ─────────────► prune impl blocks
+       parse & query               keep interfaces · types ·
+                                   signatures · exports
+                                          ▾
+                                  pruned skeleton (Ring 0 full source)
+                                          ▾
+                          Compressed Code Context (Markdown)
+                                  │               │
+                            via MCP (stdio)   via HTTP (Fastify)
+                      get_compressed_code_context  POST /v1/context
+```
+
+Pipeline stages:
+
+1. **Parse** — `web-tree-sitter` loads a `.wasm` grammar per language (auto-downloaded on first run).
+2. **Prune** — an S-expression query matches implementation blocks (`statement_block`, `block`, `compound_statement`…), which are replaced with a short token (`/* ... */`, or `pass` for Python) using **descending-order splicing** so offsets stay valid.
+3. **Watch** — `chokidar` watches the repo, `sha1`-hashes file contents, and refreshes the cache only on change.
+4. **Assemble** — the active file's imports are resolved and merged into a Markdown context payload (Ring 0 + Ring 1).
+
+---
+
+
+
+## Install
+
+Requires **Node.js 18+**.
+
+```bash
+# run anywhere without installing
+#   --root   project root   --port   http port   --host   bind address
+npx @ajdev0/token-shrink --root /path/to/project
+
+# or install locally
+npm install @ajdev0/token-shrink
+```
+
+
+
+### Build from source
+
+```bash
+# install deps
+npm install
+
+# compile (tsup -> dist/), typecheck, and run tests
+npm run build
+npm run typecheck
+npm test
+```
+
+The build produces three binaries:
+
+
+| Binary               | Entry           | Purpose                                      |
+| -------------------- | --------------- | -------------------------------------------- |
+| `@ajdev0/token-shrink`       | `dist/cli.cjs`   | Fastify HTTP server (`POST /v1/context`)     |
+| `@ajdev0/token-shrink-mcp`   | `dist/mcp.cjs`   | MCP stdio server for AI agents               |
+| library              | `dist/index.js` | `prune()`, `assemble()`, `createWatcher()` … |
+
+### Publish to npm
+
+```bash
+npm login
+npm run build && npm test
+npm pack --dry-run    # preview tarball contents
+npm publish
+```
+
+The package name on npm is **`@ajdev0/token-shrink`**. After publishing, users can run:
+
+```bash
+npx @ajdev0/token-shrink-mcp --root /path/to/project
+```
+
+
+---
+
+
+
+## WASM grammars (auto-download)
+
+Grammars are fetched from the official tree-sitter GitHub releases on **first use** and cached in `wasm/`:
+
+```text
+wasm/
+├── tree-sitter-typescript.wasm
+├── tree-sitter-javascript.wasm
+├── tree-sitter-tsx.wasm
+├── tree-sitter-python.wasm
+├── tree-sitter-go.wasm
+├── ...
+```
+
+- First run requires network access; afterwards everything is offline and fast.
+- Files are written atomically (`*.tmp` → rename) with an in-flight lock, so concurrent first-run parses never corrupt the cache.
+
+---
+
+
+
+## Usage
+
+
+
+### 1. MCP server (AI agents — Cursor, Claude, etc.)
+
+Run the stdio MCP server and expose the `get_compressed_code_context` tool:
+
+```bash
+# point it at your project
+token-shrink-mcp --root /path/to/project
+
+# root also works via env or cwd
+ROOT=/path/to/project token-shrink-mcp
+cd /path/to/project && token-shrink-mcp
+```
+
+**Cursor MCP config** (`.cursor/mcp.json`):
+
+```json
+{
+  "mcpServers": {
+    "token-shrink": {
+      "command": "token-shrink-mcp",
+      "args": ["--root", "/absolute/path/to/your/project"]
+    }
+  }
+}
+```
+
+**Claude Code MCP config** — add it to the project's `.mcp.json`, or register with the Claude CLI:
+
+```bash
+# register the server for this project
+claude mcp add token-shrink -- token-shrink-mcp --root /path/to/project
+# persistent flag: -- transport stdio
+claude mcp add token-shrink --transport stdio -- token-shrink-mcp --root /path/to/project
+```
+
+or place in `.claude/settings.json` / project `.mcp.json`:
+
+```json
+{
+  "mcpServers": {
+    "token-shrink": {
+      "command": "token-shrink-mcp",
+      "args": ["--root", "/path/to/project"]
+    }
+  }
+}
+```
+
+**Auto rule**: by default the server writes agent integration rules so the tool is used automatically on every prompt:
+
+- **Cursor**: `.cursor/rules/token-shrink.mdc`
+- **Claude Code**: `.claude/rules/token-shrink.md`
+
+Both are sentinel-tagged and never rewrite a user-authored file at the same path. Repeated starts are no-ops. Choose the target(s) with `--rule-target=cursor|claude|all` (default `all`, comma-separated values allowed):
+
+```bash
+# only Claude Code
+token-shrink-mcp --root /path/to/project --rule-target=claude
+
+# completely disable auto-rules
+token-shrink-mcp --root /path/to/project --no-create-rule
+```
+
+Opt out also via `--create-rule=false` or `TOKEN_SHRINK_CREATE_RULE=0`.
+
+**Tool:** `get_compressed_code_context`
+
+
+| Argument         | Type      | Required | Description                                   |
+| ---------------- | --------- | -------- | --------------------------------------------- |
+| `activeFilePath` | `string`  | yes      | The file the agent is working on              |
+| `maxSkeletons`   | `number`  | no       | Cap on Ring-1 files (default `50`, max `200`) |
+| `includeStats`   | `boolean` | no       | Append approximate token counts               |
+
+
+Returns a Markdown payload with the active file fully inlined (Ring 0) and the pruned skeletons of its direct imports (Ring 1).
+
+### 2. HTTP server (Fastify)
+
+```bash
+token-shrink --root /path/to/project --port 3000
+# env equivalents: ROOT=… PORT=… HOST=…
+```
+
+
+| Route         | Method | Body                                               | Returns                          |
+| ------------- | ------ | -------------------------------------------------- | -------------------------------- |
+| `/health`     | `GET`  | —                                                  | status, root, indexed file count |
+| `/v1/context` | `POST` | `{ activeFilePath, maxSkeletons?, includeStats? }` | assembled Markdown + deps        |
+
+
+```bash
+curl -s http://localhost:3000/health
+# {"status":"ok","service":"token-shrink","version":"2.0.0","root":".","indexed":182}
+
+curl -s -X POST http://localhost:3000/v1/context \
+  -H 'Content-Type: application/json' \
+  -d '{"activeFilePath":"./src/page.ts","includeStats":true}'
+```
+
+
+
+### 3. Library API
+
+```ts
+import { prune, assemble, createWatcher } from 'token-shrink';
+
+// prune a single file -> skeleton (keeps signatures, strips bodies)
+const { code, removed } = await prune('src/util.ts', sourceText);
+
+// assemble context for an active file from a warm cache
+const { markdown } = assemble('src/page.ts', watcher.cache.entries, {
+  includeStats: true,
+});
+
+// incremental watcher
+const watcher = createWatcher({ root: process.cwd(), ignored: ['node_modules'] });
+await watcher.indexAll();
+```
+
+---
+
+
+
+## Supported languages
+
+S-expression queries match implementation blocks; interfaces, signatures, and exports are never touched. The `Block node` column shows the AST node that gets collapsed during pruning.
+
+
+| Language        | Extensions                              | Grammar wasm                  | Block node           |
+| --------------- | --------------------------------------- | ----------------------------- | -------------------- |
+| TypeScript      | `.ts` `.cts` `.mts`                     | `tree-sitter-typescript.wasm` | `statement_block`    |
+| JavaScript      | `.js` `.cjs` `.mjs`                     | `tree-sitter-javascript.wasm` | `statement_block`    |
+| React / Next.js | `.tsx`                                  | `tree-sitter-tsx.wasm`        | `statement_block`¹   |
+| React (JSX)     | `.jsx`                                  | `tree-sitter-javascript.wasm` | `statement_block`¹   |
+| Python          | `.py` `.pyi`                            | `tree-sitter-python.wasm`     | `block` → `pass`     |
+| Dart / Flutter  | `.dart`                                 | `tree-sitter-dart.wasm`       | `block`              |
+| Swift / SwiftUI | `.swift`                                | `tree-sitter-swift.wasm`      | `statements`         |
+| Go              | `.go`                                   | `tree-sitter-go.wasm`         | `block`              |
+| Rust            | `.rs`                                   | `tree-sitter-rust.wasm`       | `block`              |
+| Java            | `.java`                                 | `tree-sitter-java.wasm`       | `block`              |
+| Kotlin          | `.kt` `.kts`                            | `tree-sitter-kotlin.wasm`     | `block`              |
+| C               | `.c` `.h`                               | `tree-sitter-c.wasm`          | `compound_statement` |
+| C++             | `.cc` `.cpp` `.cxx` `.hpp` `.hh` `.hxx` | `tree-sitter-cpp.wasm`        | `compound_statement` |
+| PHP             | `.php`                                  | `tree-sitter-php.wasm`        | `compound_statement` |
+
+
+> ¹ TSX/JSX also preserve `'use client'` / `'use server'` directive lines inside otherwise-pruned bodies (framework-aware).
+
+Language IDs: `typescript · javascript · tsx · jsx · python · dart · swift · go · rust · java · kotlin · c · cpp · php`.
+
+---
+
+
+
+## Example
+
+**Input** `src/util.ts`
+
+```ts
+export interface User {
+  id: number;
+  name: string;
+}
+export function buildGreeting(u: User) {
+  const parts = [u.name, u.email];
+  return parts.join(' | ');
+}
+export const formatEmail = (u: User) => {
+  return u.email.toLowerCase().trim();
+};
+```
+
+**Pruned skeleton (Ring 1)** — signatures and the interface intact, bodies collapsed:
+
+```ts
+export interface User {
+  id: number;
+  name: string;
+}
+export function buildGreeting(u: User) /* ... */
+export const formatEmail = (u: User) => /* ... */;
+```
+
+---
+
+
+
+## Design notes
+
+- **Bottom-up splicing** — ranges are sorted by start index descending and replaced in place, so earlier offsets never shift and the output stays a valid, parseable file.
+- **Regex-based import extraction** — resilient across languages; resolves relative imports (`./x`, `../y`), aliases (`@/`, `~`), and skips bare package specifiers.
+- **Incremental hashing** — files are re-pruned only when their `sha1` hash changes; the watcher is debounced (100 ms) and zero-CPU while idle.
+- **Ram-safe watchers** — sockets / non-regular files are never opened with `fs.watch`, so stray unix sockets in the tree can't crash the server.
+
+---
+
+
+
+## Project layout
+
+```
+token-shrink/
+├── package.json / tsconfig.json / tsup.config.ts / vitest.config.ts
+├── src/
+│   ├── index.ts             # library entry (exports)
+│   ├── cli.ts               # Fastify HTTP server
+│   ├── mcp.ts               # MCP stdio server
+│   ├── parser/
+│   │   ├── registry.ts      # extension → language spec + S-queries
+│   │   ├── wasm.ts          # auto-download + cache of .wasm files
+│   │   └── pruner.ts        # prune(filePath, source) → skeleton
+│   ├── watcher/
+│   │   └── sync.ts          # chokidar watch + hash cache + import graph
+│   └── server/
+│       └── assembler.ts     # Ring 0 + Ring 1 Markdown payload
+├── tests/                   # pruning integrity + token-reduction tests
+└── wasm/                    # auto-downloaded grammars (gitignored)
+```
+
+
+
+## License
+
+MIT

package/dist/chunk-7H6PGILN.js

@@ -0,0 +1,86 @@
+#!/usr/bin/env node
+import {
+  assemble,
+  createWatcher,
+  warmGrammars
+} from "./chunk-HRF3BIOV.js";
+
+// src/cli.ts
+import Fastify from "fastify";
+import picocolors from "picocolors";
+import path from "path";
+function parseArgv(argv = process.argv) {
+  const out = {};
+  for (let i = 0; i < argv.length; i++) {
+    const cur = argv[i];
+    if (cur?.startsWith("--")) {
+      const key = cur.slice(2);
+      const next = argv[i + 1];
+      if (next && !next.startsWith("--")) {
+        out[key] = next;
+        i++;
+      } else {
+        out[key] = "true";
+      }
+    }
+  }
+  return out;
+}
+async function startServer(opts = {}) {
+  const args = parseArgv();
+  const port = opts.port ?? Number(args.port ?? process.env.PORT ?? 3e3);
+  const host = opts.host ?? args.host ?? "0.0.0.0";
+  const root = path.resolve(opts.root ?? args.root ?? process.cwd());
+  const silent = opts.silent ?? args.silent === "true";
+  const log = (msg) => {
+    if (!silent) console.log(picocolors.dim(msg));
+  };
+  void warmGrammars().catch(() => {
+  });
+  const watcher = createWatcher({ root, ignored: opts.ignored });
+  log(`Indexing ${root} in the background\u2026`);
+  void watcher.indexAll().then((n) => log(`Indexed ${n} files.`));
+  const app = Fastify({ logger: !silent });
+  app.get("/health", async () => ({
+    status: "ok",
+    service: "token-shrink",
+    version: "2.0.0",
+    root,
+    indexed: watcher.cache.entries.size
+  }));
+  app.post("/v1/context", async (req, reply) => {
+    const body = req.body ?? {};
+    if (!body.activeFilePath) {
+      return reply.status(400).send({ error: "`activeFilePath` is required" });
+    }
+    const result = assemble(body.activeFilePath, watcher.cache.entries, {
+      maxSkeletons: body.maxSkeletons,
+      includeStats: body.includeStats
+    });
+    return {
+      markdown: result.markdown,
+      activeFilePath: result.activeFilePath,
+      dependencies: result.included.map((i) => i.filePath),
+      unresolved: result.unresolved
+    };
+  });
+  app.setNotFoundHandler(async (req, reply) => {
+    void req;
+    return reply.status(404).send({ error: "Not found" });
+  });
+  await app.listen({ port, host });
+  log(`Listening on http://${host}:${port}`);
+  return { app, watcher };
+}
+var argv1 = process.argv[1] ? path.basename(process.argv[1]) : "";
+if (argv1 === "cli.js" || argv1 === "cli.mjs" || argv1 === "cli.cjs" || argv1 === "cli.ts") {
+  startServer().catch((err) => {
+    console.error(picocolors.red(`[token-shrink] ${err.message}`));
+    process.exitCode = 1;
+  });
+}
+
+export {
+  startServer
+};
+//# sourceMappingURL=chunk-7H6PGILN.js.map

package/dist/chunk-7H6PGILN.js.map

@@ -0,0 +1 @@
+{"version":3,"sources":["../src/cli.ts"],"sourcesContent":["/**\n * HTTP/CLI server. Exposes the context assembler as a small Fastify service:\n *\n *   GET  /health      -> liveness\n *   POST /v1/context  -> { activeFilePath, maxSkeletons?, includeStats? } => markdown\n *\n * Run as `token-shrink` or `node dist/cli.js`. The server watches `--root`\n * (defaults to cwd) and keeps the import graph + skeletons warm.\n */\n\nimport Fastify from 'fastify';\nimport picocolors from 'picocolors';\nimport path from 'node:path';\n\nimport { assemble } from './server/assembler.js';\nimport { createWatcher, type Matcher } from './watcher/sync.js';\nimport { warmGrammars } from './parser/wasm.js';\n\nexport interface CliOptions {\n  port?: number;\n  host?: string;\n  root?: string;\n  ignored?: Matcher[];\n  silent?: boolean;\n}\n\nfunction parseArgv(argv = process.argv): Record<string, string> {\n  const out: Record<string, string> = {};\n  for (let i = 0; i < argv.length; i++) {\n    const cur = argv[i];\n    if (cur?.startsWith('--')) {\n      const key = cur.slice(2);\n      const next = argv[i + 1];\n      if (next && !next.startsWith('--')) {\n        out[key] = next;\n        i++;\n      } else {\n        out[key] = 'true';\n      }\n    }\n  }\n  return out;\n}\n\n/** Start the Fastify server; returns the running instance + watcher handle. */\nexport async function startServer(opts: CliOptions = {}) {\n  const args = parseArgv();\n  const port = opts.port ?? Number(args.port ?? process.env.PORT ?? 3000);\n  const host = opts.host ?? args.host ?? '0.0.0.0';\n  const root = path.resolve(opts.root ?? args.root ?? process.cwd());\n  const silent = opts.silent ?? args.silent === 'true';\n\n  const log = (msg: string) => {\n    if (!silent) console.log(picocolors.dim(msg));\n  };\n\n  // Warm grammars in the background so the first prune isn't slow.\n  void warmGrammars().catch(() => {});\n\n  const watcher = createWatcher({ root, ignored: opts.ignored });\n  log(`Indexing ${root} in the background…`);\n  void watcher.indexAll().then((n) => log(`Indexed ${n} files.`));\n\n  const app = Fastify({ logger: !silent });\n\n  app.get('/health', async () => ({\n    status: 'ok',\n    service: 'token-shrink',\n    version: '2.0.0',\n    root,\n    indexed: watcher.cache.entries.size,\n  }));\n\n  app.post('/v1/context', async (req, reply) => {\n    const body = (req.body ?? {}) as {\n      activeFilePath?: string;\n      maxSkeletons?: number;\n      includeStats?: boolean;\n    };\n    if (!body.activeFilePath) {\n      return reply.status(400).send({ error: '`activeFilePath` is required' });\n    }\n    const result = assemble(body.activeFilePath, watcher.cache.entries, {\n      maxSkeletons: body.maxSkeletons,\n      includeStats: body.includeStats,\n    });\n    return {\n      markdown: result.markdown,\n      activeFilePath: result.activeFilePath,\n      dependencies: result.included.map((i) => i.filePath),\n      unresolved: result.unresolved,\n    };\n  });\n\n  app.setNotFoundHandler(async (req, reply) => {\n    void req;\n    return reply.status(404).send({ error: 'Not found' });\n  });\n\n  await app.listen({ port, host });\n  log(`Listening on http://${host}:${port}`);\n\n  return { app, watcher };\n}\n\n// Start when invoked directly (`node dist/cli.cjs` / `token-shrink`).\nconst argv1 = process.argv[1] ? path.basename(process.argv[1]) : '';\nif (\n  argv1 === 'cli.js' || argv1 === 'cli.mjs' ||\n  argv1 === 'cli.cjs' || argv1 === 'cli.ts'\n) {\n  startServer().catch((err) => {\n    console.error(picocolors.red(`[token-shrink] ${err.message}`));\n    process.exitCode = 1;\n  });\n}\n"],"mappings":";;;;;;;;AAUA,OAAO,aAAa;AACpB,OAAO,gBAAgB;AACvB,OAAO,UAAU;AAcjB,SAAS,UAAU,OAAO,QAAQ,MAA8B;AAC9D,QAAM,MAA8B,CAAC;AACrC,WAAS,IAAI,GAAG,IAAI,KAAK,QAAQ,KAAK;AACpC,UAAM,MAAM,KAAK,CAAC;AAClB,QAAI,KAAK,WAAW,IAAI,GAAG;AACzB,YAAM,MAAM,IAAI,MAAM,CAAC;AACvB,YAAM,OAAO,KAAK,IAAI,CAAC;AACvB,UAAI,QAAQ,CAAC,KAAK,WAAW,IAAI,GAAG;AAClC,YAAI,GAAG,IAAI;AACX;AAAA,MACF,OAAO;AACL,YAAI,GAAG,IAAI;AAAA,MACb;AAAA,IACF;AAAA,EACF;AACA,SAAO;AACT;AAGA,eAAsB,YAAY,OAAmB,CAAC,GAAG;AACvD,QAAM,OAAO,UAAU;AACvB,QAAM,OAAO,KAAK,QAAQ,OAAO,KAAK,QAAQ,QAAQ,IAAI,QAAQ,GAAI;AACtE,QAAM,OAAO,KAAK,QAAQ,KAAK,QAAQ;AACvC,QAAM,OAAO,KAAK,QAAQ,KAAK,QAAQ,KAAK,QAAQ,QAAQ,IAAI,CAAC;AACjE,QAAM,SAAS,KAAK,UAAU,KAAK,WAAW;AAE9C,QAAM,MAAM,CAAC,QAAgB;AAC3B,QAAI,CAAC,OAAQ,SAAQ,IAAI,WAAW,IAAI,GAAG,CAAC;AAAA,EAC9C;AAGA,OAAK,aAAa,EAAE,MAAM,MAAM;AAAA,EAAC,CAAC;AAElC,QAAM,UAAU,cAAc,EAAE,MAAM,SAAS,KAAK,QAAQ,CAAC;AAC7D,MAAI,YAAY,IAAI,0BAAqB;AACzC,OAAK,QAAQ,SAAS,EAAE,KAAK,CAAC,MAAM,IAAI,WAAW,CAAC,SAAS,CAAC;AAE9D,QAAM,MAAM,QAAQ,EAAE,QAAQ,CAAC,OAAO,CAAC;AAEvC,MAAI,IAAI,WAAW,aAAa;AAAA,IAC9B,QAAQ;AAAA,IACR,SAAS;AAAA,IACT,SAAS;AAAA,IACT;AAAA,IACA,SAAS,QAAQ,MAAM,QAAQ;AAAA,EACjC,EAAE;AAEF,MAAI,KAAK,eAAe,OAAO,KAAK,UAAU;AAC5C,UAAM,OAAQ,IAAI,QAAQ,CAAC;AAK3B,QAAI,CAAC,KAAK,gBAAgB;AACxB,aAAO,MAAM,OAAO,GAAG,EAAE,KAAK,EAAE,OAAO,+BAA+B,CAAC;AAAA,IACzE;AACA,UAAM,SAAS,SAAS,KAAK,gBAAgB,QAAQ,MAAM,SAAS;AAAA,MAClE,cAAc,KAAK;AAAA,MACnB,cAAc,KAAK;AAAA,IACrB,CAAC;AACD,WAAO;AAAA,MACL,UAAU,OAAO;AAAA,MACjB,gBAAgB,OAAO;AAAA,MACvB,cAAc,OAAO,SAAS,IAAI,CAAC,MAAM,EAAE,QAAQ;AAAA,MACnD,YAAY,OAAO;AAAA,IACrB;AAAA,EACF,CAAC;AAED,MAAI,mBAAmB,OAAO,KAAK,UAAU;AAC3C,SAAK;AACL,WAAO,MAAM,OAAO,GAAG,EAAE,KAAK,EAAE,OAAO,YAAY,CAAC;AAAA,EACtD,CAAC;AAED,QAAM,IAAI,OAAO,EAAE,MAAM,KAAK,CAAC;AAC/B,MAAI,uBAAuB,IAAI,IAAI,IAAI,EAAE;AAEzC,SAAO,EAAE,KAAK,QAAQ;AACxB;AAGA,IAAM,QAAQ,QAAQ,KAAK,CAAC,IAAI,KAAK,SAAS,QAAQ,KAAK,CAAC,CAAC,IAAI;AACjE,IACE,UAAU,YAAY,UAAU,aAChC,UAAU,aAAa,UAAU,UACjC;AACA,cAAY,EAAE,MAAM,CAAC,QAAQ;AAC3B,YAAQ,MAAM,WAAW,IAAI,kBAAkB,IAAI,OAAO,EAAE,CAAC;AAC7D,YAAQ,WAAW;AAAA,EACrB,CAAC;AACH;","names":[]}

package/dist/chunk-7SQ6HMWM.js

@@ -0,0 +1,145 @@
+#!/usr/bin/env node
+
+// src/parser/registry.ts
+import path from "path";
+var BLOCK = { query: "(statement_block) @block", replacement: { token: "/* ... */" } };
+var SWIFT_BLOCK = { query: "(statements) @block", replacement: { token: "/* ... */" } };
+var GO_BLOCK = { query: "(block) @block", replacement: { token: "/* ... */" } };
+var RUST_BLOCK = { query: "(block) @block", replacement: { token: "/* ... */" } };
+var PY_BLOCK = { query: "(block) @block", replacement: { token: "pass" } };
+var C_BLOCK = { query: "(compound_statement) @block", replacement: { token: "/* ... */" } };
+var JAVA_BLOCK = {
+  query: "(block) @block",
+  replacement: { token: "/* ... */" }
+};
+var PHP_BLOCK = {
+  query: "(compound_statement) @block",
+  replacement: { token: "/* ... */" }
+};
+var KOTLIN_BLOCK = { query: "(block) @block", replacement: { token: "/* ... */" } };
+var DART_BLOCK = { query: "(block) @block", replacement: { token: "/* ... */" } };
+var DIRECTIVE_KEEP = {
+  token: "/* ... */",
+  keepIf: /^\s*'use (client|server)'/
+};
+var registry = {
+  typescript: {
+    name: "TypeScript",
+    wasm: "tree-sitter-typescript.wasm",
+    url: "https://github.com/tree-sitter/tree-sitter-typescript/releases/latest/download/tree-sitter-typescript.wasm",
+    extensions: [".ts", ".cts", ".mts"],
+    rules: [BLOCK]
+  },
+  javascript: {
+    name: "JavaScript",
+    wasm: "tree-sitter-javascript.wasm",
+    url: "https://github.com/tree-sitter/tree-sitter-javascript/releases/latest/download/tree-sitter-javascript.wasm",
+    extensions: [".js", ".cjs", ".mjs"],
+    rules: [BLOCK]
+  },
+  tsx: {
+    name: "React / Next.js (TSX)",
+    wasm: "tree-sitter-tsx.wasm",
+    url: "https://github.com/tree-sitter/tree-sitter-typescript/releases/latest/download/tree-sitter-tsx.wasm",
+    extensions: [".tsx"],
+    rules: [{ query: "(statement_block) @block", replacement: DIRECTIVE_KEEP }]
+  },
+  jsx: {
+    name: "React (JSX)",
+    wasm: "tree-sitter-javascript.wasm",
+    url: "https://github.com/tree-sitter/tree-sitter-javascript/releases/latest/download/tree-sitter-javascript.wasm",
+    extensions: [".jsx"],
+    rules: [{ query: "(statement_block) @block", replacement: DIRECTIVE_KEEP }]
+  },
+  python: {
+    name: "Python",
+    wasm: "tree-sitter-python.wasm",
+    url: "https://github.com/tree-sitter/tree-sitter-python/releases/latest/download/tree-sitter-python.wasm",
+    extensions: [".py", ".pyi"],
+    rules: [PY_BLOCK]
+  },
+  dart: {
+    name: "Dart / Flutter",
+    wasm: "tree-sitter-dart.wasm",
+    url: "https://github.com/UserNobody14/tree-sitter-dart.wasm/releases/latest/download/tree-sitter-dart.wasm",
+    extensions: [".dart"],
+    rules: [DART_BLOCK]
+  },
+  swift: {
+    name: "Swift / SwiftUI",
+    wasm: "tree-sitter-swift.wasm",
+    url: "https://github.com/alex-pinkus/tree-sitter-swift/releases/latest/download/tree-sitter-swift.wasm",
+    extensions: [".swift"],
+    rules: [SWIFT_BLOCK]
+  },
+  go: {
+    name: "Go",
+    wasm: "tree-sitter-go.wasm",
+    url: "https://github.com/tree-sitter/tree-sitter-go/releases/latest/download/tree-sitter-go.wasm",
+    extensions: [".go"],
+    rules: [GO_BLOCK]
+  },
+  rust: {
+    name: "Rust",
+    wasm: "tree-sitter-rust.wasm",
+    url: "https://github.com/tree-sitter/tree-sitter-rust/releases/latest/download/tree-sitter-rust.wasm",
+    extensions: [".rs"],
+    rules: [RUST_BLOCK]
+  },
+  java: {
+    name: "Java",
+    wasm: "tree-sitter-java.wasm",
+    url: "https://github.com/tree-sitter/tree-sitter-java/releases/latest/download/tree-sitter-java.wasm",
+    extensions: [".java"],
+    rules: [JAVA_BLOCK]
+  },
+  kotlin: {
+    name: "Kotlin",
+    wasm: "tree-sitter-kotlin.wasm",
+    url: "https://github.com/fwcd/tree-sitter-kotlin/releases/latest/download/tree-sitter-kotlin.wasm",
+    extensions: [".kt", ".kts"],
+    rules: [KOTLIN_BLOCK]
+  },
+  c: {
+    name: "C",
+    wasm: "tree-sitter-c.wasm",
+    url: "https://github.com/tree-sitter/tree-sitter-c/releases/latest/download/tree-sitter-c.wasm",
+    extensions: [".c", ".h"],
+    rules: [C_BLOCK]
+  },
+  cpp: {
+    name: "C++",
+    wasm: "tree-sitter-cpp.wasm",
+    url: "https://github.com/tree-sitter/tree-sitter-cpp/releases/latest/download/tree-sitter-cpp.wasm",
+    extensions: [".cc", ".cpp", ".cxx", ".hpp", ".hh", ".hxx"],
+    rules: [C_BLOCK]
+  },
+  php: {
+    name: "PHP",
+    wasm: "tree-sitter-php.wasm",
+    url: "https://github.com/tree-sitter/tree-sitter-php/releases/latest/download/tree-sitter-php.wasm",
+    extensions: [".php"],
+    rules: [PHP_BLOCK]
+  }
+};
+var extensionIndex = /* @__PURE__ */ new Map();
+for (const [id, spec] of Object.entries(registry)) {
+  for (const ext of spec.extensions) {
+    extensionIndex.set(ext, id);
+  }
+}
+function languageForFile(filePath) {
+  const langId = extensionIndex.get(path.extname(filePath).toLowerCase());
+  if (!langId) return null;
+  return registry[langId];
+}
+var supportedLanguageIds = Object.keys(registry);
+var allExtensions = [...extensionIndex.keys()];
+
+export {
+  registry,
+  languageForFile,
+  supportedLanguageIds,
+  allExtensions
+};
+//# sourceMappingURL=chunk-7SQ6HMWM.js.map

package/dist/chunk-7SQ6HMWM.js.map

@@ -0,0 +1 @@
+{"version":3,"sources":["../src/parser/registry.ts"],"sourcesContent":["/**\n * Language registry mapping file extensions to tree-sitter grammars and\n * pruning strategies. Each spec defines:\n *   - the web-tree-sitter language name\n *   - the `.wasm` file and the authoritative download URL\n *   - one or more S-expression queries (the \"prune query\") that match the\n *     implementation blocks to remove, plus the replacement token.\n *\n * The empty-query default of '' matches nothing, so the pruner leaves the\n * file untouched when a language has no registered prune rules yet.\n */\n\nimport path from 'node:path';\n\n/** Source of truth mirror in the PRD language matrix. */\nexport type Replacement = {\n  /** Token used to replace a pruned block. */\n  token: string;\n  /**\n   * Optional regex, run against the raw block text. If it matches, the block\n   * is kept instead of pruned. Used to preserve directive/header lines\n   * ('use client/server') even when they live inside an otherwise-pruned body.\n   */\n  keepIf?: RegExp;\n};\n\nexport interface LanguageSpec {\n  /** Friendly name, used in logs and the report header. */\n  name: string;\n  /** web-tree-sitter `Language.load` file name. */\n  wasm: string;\n  /** Where to download the `.wasm` binary if not cached locally. */\n  url: string;\n  /** Extensions that map to this language: `.ts`, `.tsx`, etc. */\n  extensions: string[];\n  /**\n   * One or more (query, replacement) pairs. Blocks matched by each query are\n   * pruned bottom-up. Queries are combined; a block is pruned if it matches\n   * any. This keeps small, frequently repeated constructs cheap to configure.\n   */\n  rules: { query: string; replacement: Replacement }[];\n}\n\n// ---------------------------------------------------------------------------\n// Pruning workhorse: prune every `statement_block` / `block` node.\n// ---------------------------------------------------------------------------\n\nconst BLOCK = { query: '(statement_block) @block', replacement: { token: '/* ... */' } };\nconst SWIFT_BLOCK = { query: '(statements) @block', replacement: { token: '/* ... */' } };\nconst GO_BLOCK = { query: '(block) @block', replacement: { token: '/* ... */' } };\nconst RUST_BLOCK = { query: '(block) @block', replacement: { token: '/* ... */' } };\nconst PY_BLOCK = { query: '(block) @block', replacement: { token: 'pass' } };\nconst C_BLOCK = { query: '(compound_statement) @block', replacement: { token: '/* ... */' } };\nconst JAVA_BLOCK = {\n  query: '(block) @block',\n  replacement: { token: '/* ... */' },\n};\nconst PHP_BLOCK = {\n  query: '(compound_statement) @block',\n  replacement: { token: '/* ... */' },\n};\nconst KOTLIN_BLOCK = { query: '(block) @block', replacement: { token: '/* ... */' } };\nconst DART_BLOCK = { query: '(block) @block', replacement: { token: '/* ... */' } };\n\n// Keeps `'use client'`/`'use server'` and top-level directive strings.\nconst DIRECTIVE_KEEP: Replacement = {\n  token: '/* ... */',\n  keepIf: /^\\s*'use (client|server)'/,\n};\n\nexport const registry: Record<string, LanguageSpec> = {\n  typescript: {\n    name: 'TypeScript',\n    wasm: 'tree-sitter-typescript.wasm',\n    url: 'https://github.com/tree-sitter/tree-sitter-typescript/releases/latest/download/tree-sitter-typescript.wasm',\n    extensions: ['.ts', '.cts', '.mts'],\n    rules: [BLOCK],\n  },\n  javascript: {\n    name: 'JavaScript',\n    wasm: 'tree-sitter-javascript.wasm',\n    url: 'https://github.com/tree-sitter/tree-sitter-javascript/releases/latest/download/tree-sitter-javascript.wasm',\n    extensions: ['.js', '.cjs', '.mjs'],\n    rules: [BLOCK],\n  },\n  tsx: {\n    name: 'React / Next.js (TSX)',\n    wasm: 'tree-sitter-tsx.wasm',\n    url: 'https://github.com/tree-sitter/tree-sitter-typescript/releases/latest/download/tree-sitter-tsx.wasm',\n    extensions: ['.tsx'],\n    rules: [{ query: '(statement_block) @block', replacement: DIRECTIVE_KEEP }],\n  },\n  jsx: {\n    name: 'React (JSX)',\n    wasm: 'tree-sitter-javascript.wasm',\n    url: 'https://github.com/tree-sitter/tree-sitter-javascript/releases/latest/download/tree-sitter-javascript.wasm',\n    extensions: ['.jsx'],\n    rules: [{ query: '(statement_block) @block', replacement: DIRECTIVE_KEEP }],\n  },\n  python: {\n    name: 'Python',\n    wasm: 'tree-sitter-python.wasm',\n    url: 'https://github.com/tree-sitter/tree-sitter-python/releases/latest/download/tree-sitter-python.wasm',\n    extensions: ['.py', '.pyi'],\n    rules: [PY_BLOCK],\n  },\n  dart: {\n    name: 'Dart / Flutter',\n    wasm: 'tree-sitter-dart.wasm',\n    url: 'https://github.com/UserNobody14/tree-sitter-dart.wasm/releases/latest/download/tree-sitter-dart.wasm',\n    extensions: ['.dart'],\n    rules: [DART_BLOCK],\n  },\n  swift: {\n    name: 'Swift / SwiftUI',\n    wasm: 'tree-sitter-swift.wasm',\n    url: 'https://github.com/alex-pinkus/tree-sitter-swift/releases/latest/download/tree-sitter-swift.wasm',\n    extensions: ['.swift'],\n    rules: [SWIFT_BLOCK],\n  },\n  go: {\n    name: 'Go',\n    wasm: 'tree-sitter-go.wasm',\n    url: 'https://github.com/tree-sitter/tree-sitter-go/releases/latest/download/tree-sitter-go.wasm',\n    extensions: ['.go'],\n    rules: [GO_BLOCK],\n  },\n  rust: {\n    name: 'Rust',\n    wasm: 'tree-sitter-rust.wasm',\n    url: 'https://github.com/tree-sitter/tree-sitter-rust/releases/latest/download/tree-sitter-rust.wasm',\n    extensions: ['.rs'],\n    rules: [RUST_BLOCK],\n  },\n  java: {\n    name: 'Java',\n    wasm: 'tree-sitter-java.wasm',\n    url: 'https://github.com/tree-sitter/tree-sitter-java/releases/latest/download/tree-sitter-java.wasm',\n    extensions: ['.java'],\n    rules: [JAVA_BLOCK],\n  },\n  kotlin: {\n    name: 'Kotlin',\n    wasm: 'tree-sitter-kotlin.wasm',\n    url: 'https://github.com/fwcd/tree-sitter-kotlin/releases/latest/download/tree-sitter-kotlin.wasm',\n    extensions: ['.kt', '.kts'],\n    rules: [KOTLIN_BLOCK],\n  },\n  c: {\n    name: 'C',\n    wasm: 'tree-sitter-c.wasm',\n    url: 'https://github.com/tree-sitter/tree-sitter-c/releases/latest/download/tree-sitter-c.wasm',\n    extensions: ['.c', '.h'],\n    rules: [C_BLOCK],\n  },\n  cpp: {\n    name: 'C++',\n    wasm: 'tree-sitter-cpp.wasm',\n    url: 'https://github.com/tree-sitter/tree-sitter-cpp/releases/latest/download/tree-sitter-cpp.wasm',\n    extensions: ['.cc', '.cpp', '.cxx', '.hpp', '.hh', '.hxx'],\n    rules: [C_BLOCK],\n  },\n  php: {\n    name: 'PHP',\n    wasm: 'tree-sitter-php.wasm',\n    url: 'https://github.com/tree-sitter/tree-sitter-php/releases/latest/download/tree-sitter-php.wasm',\n    extensions: ['.php'],\n    rules: [PHP_BLOCK],\n  },\n} satisfies Record<string, LanguageSpec>;\n\n/** Map of extension -> language spec id, built once. */\nconst extensionIndex = new Map<string, string>();\nfor (const [id, spec] of Object.entries(registry)) {\n  for (const ext of spec.extensions) {\n    extensionIndex.set(ext, id);\n  }\n}\n\nexport type LanguageId = keyof typeof registry;\n\n/** Return the language spec for a file path, or null if unsupported. */\nexport function languageForFile(filePath: string): LanguageSpec | null {\n  const langId = extensionIndex.get(path.extname(filePath).toLowerCase());\n  if (!langId) return null;\n  return registry[langId];\n}\n\n/** All languages that have at least one prune rule. */\nexport const supportedLanguageIds = Object.keys(registry) as LanguageId[];\n\n/** Convenience: list every known extension. */\nexport const allExtensions = [...extensionIndex.keys()];\n"],"mappings":";;;AAYA,OAAO,UAAU;AAmCjB,IAAM,QAAQ,EAAE,OAAO,4BAA4B,aAAa,EAAE,OAAO,YAAY,EAAE;AACvF,IAAM,cAAc,EAAE,OAAO,uBAAuB,aAAa,EAAE,OAAO,YAAY,EAAE;AACxF,IAAM,WAAW,EAAE,OAAO,kBAAkB,aAAa,EAAE,OAAO,YAAY,EAAE;AAChF,IAAM,aAAa,EAAE,OAAO,kBAAkB,aAAa,EAAE,OAAO,YAAY,EAAE;AAClF,IAAM,WAAW,EAAE,OAAO,kBAAkB,aAAa,EAAE,OAAO,OAAO,EAAE;AAC3E,IAAM,UAAU,EAAE,OAAO,+BAA+B,aAAa,EAAE,OAAO,YAAY,EAAE;AAC5F,IAAM,aAAa;AAAA,EACjB,OAAO;AAAA,EACP,aAAa,EAAE,OAAO,YAAY;AACpC;AACA,IAAM,YAAY;AAAA,EAChB,OAAO;AAAA,EACP,aAAa,EAAE,OAAO,YAAY;AACpC;AACA,IAAM,eAAe,EAAE,OAAO,kBAAkB,aAAa,EAAE,OAAO,YAAY,EAAE;AACpF,IAAM,aAAa,EAAE,OAAO,kBAAkB,aAAa,EAAE,OAAO,YAAY,EAAE;AAGlF,IAAM,iBAA8B;AAAA,EAClC,OAAO;AAAA,EACP,QAAQ;AACV;AAEO,IAAM,WAAyC;AAAA,EACpD,YAAY;AAAA,IACV,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,OAAO,QAAQ,MAAM;AAAA,IAClC,OAAO,CAAC,KAAK;AAAA,EACf;AAAA,EACA,YAAY;AAAA,IACV,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,OAAO,QAAQ,MAAM;AAAA,IAClC,OAAO,CAAC,KAAK;AAAA,EACf;AAAA,EACA,KAAK;AAAA,IACH,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,MAAM;AAAA,IACnB,OAAO,CAAC,EAAE,OAAO,4BAA4B,aAAa,eAAe,CAAC;AAAA,EAC5E;AAAA,EACA,KAAK;AAAA,IACH,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,MAAM;AAAA,IACnB,OAAO,CAAC,EAAE,OAAO,4BAA4B,aAAa,eAAe,CAAC;AAAA,EAC5E;AAAA,EACA,QAAQ;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,OAAO,MAAM;AAAA,IAC1B,OAAO,CAAC,QAAQ;AAAA,EAClB;AAAA,EACA,MAAM;AAAA,IACJ,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,OAAO;AAAA,IACpB,OAAO,CAAC,UAAU;AAAA,EACpB;AAAA,EACA,OAAO;AAAA,IACL,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,QAAQ;AAAA,IACrB,OAAO,CAAC,WAAW;AAAA,EACrB;AAAA,EACA,IAAI;AAAA,IACF,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,KAAK;AAAA,IAClB,OAAO,CAAC,QAAQ;AAAA,EAClB;AAAA,EACA,MAAM;AAAA,IACJ,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,KAAK;AAAA,IAClB,OAAO,CAAC,UAAU;AAAA,EACpB;AAAA,EACA,MAAM;AAAA,IACJ,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,OAAO;AAAA,IACpB,OAAO,CAAC,UAAU;AAAA,EACpB;AAAA,EACA,QAAQ;AAAA,IACN,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,OAAO,MAAM;AAAA,IAC1B,OAAO,CAAC,YAAY;AAAA,EACtB;AAAA,EACA,GAAG;AAAA,IACD,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,MAAM,IAAI;AAAA,IACvB,OAAO,CAAC,OAAO;AAAA,EACjB;AAAA,EACA,KAAK;AAAA,IACH,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,OAAO,QAAQ,QAAQ,QAAQ,OAAO,MAAM;AAAA,IACzD,OAAO,CAAC,OAAO;AAAA,EACjB;AAAA,EACA,KAAK;AAAA,IACH,MAAM;AAAA,IACN,MAAM;AAAA,IACN,KAAK;AAAA,IACL,YAAY,CAAC,MAAM;AAAA,IACnB,OAAO,CAAC,SAAS;AAAA,EACnB;AACF;AAGA,IAAM,iBAAiB,oBAAI,IAAoB;AAC/C,WAAW,CAAC,IAAI,IAAI,KAAK,OAAO,QAAQ,QAAQ,GAAG;AACjD,aAAW,OAAO,KAAK,YAAY;AACjC,mBAAe,IAAI,KAAK,EAAE;AAAA,EAC5B;AACF;AAKO,SAAS,gBAAgB,UAAuC;AACrE,QAAM,SAAS,eAAe,IAAI,KAAK,QAAQ,QAAQ,EAAE,YAAY,CAAC;AACtE,MAAI,CAAC,OAAQ,QAAO;AACpB,SAAO,SAAS,MAAM;AACxB;AAGO,IAAM,uBAAuB,OAAO,KAAK,QAAQ;AAGjD,IAAM,gBAAgB,CAAC,GAAG,eAAe,KAAK,CAAC;","names":[]}