ast-mcp-server 0.3.0

package/CHANGELOG.md

@@ -0,0 +1,27 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.3.0] - 2026-08-03
+
+Initial public release.
+
+### Added
+
+- Bounded structural reads for project files, symbols, outlines, references, exact declaration source, and TypeScript diagnostics.
+- Hash-bound `prepare → review → apply` operations for project-wide symbol renames and body replacements.
+- Persistent CLI plans with workspace/config fingerprints, cooperative locks, exact postimages, conservative rollback, and idempotent receipts.
+- Declarative batch CLI with bounded fan-out, deterministic aggregation, compact `emit`, and persisted prepare plans.
+- Bundled `structural-code-editing` skill for Claude Code and Hermes.
+- Interactive and non-interactive agent setup that detects clients, installs the skill, registers the MCP server, verifies its tools, and replays safely.
+- Yarn 4 package workflow with dependency lifecycle scripts disabled and isolated installed-tarball smokes.
+
+### Security
+
+- New TypeScript diagnostics fail a prepared mutation closed unless explicitly reviewed with `allow_new_errors`.
+- Apply rejects stale workspaces, changed configs/sources, mismatched plan hashes, unsafe plan files, conflicting MCP registrations, and conflicting skill content.
+- Inputs, outputs, operation stores, subprocesses, batch fan-out, plan lifetime, and filesystem access are bounded.
+
+[0.3.0]: https://github.com/yailPeralta/ast-mcp-server/releases/tag/v0.3.0

package/LICENSE

@@ -0,0 +1,7 @@
+ISC License
+
+Copyright (c) 2026 Yail Peralta
+
+Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
+
+THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

package/README.md

@@ -0,0 +1,382 @@
+# ast-mcp-server
+
+[![CI](https://github.com/yailPeralta/ast-mcp-server/actions/workflows/ci.yml/badge.svg)](https://github.com/yailPeralta/ast-mcp-server/actions/workflows/ci.yml)
+[![npm version](https://img.shields.io/npm/v/ast-mcp-server.svg)](https://www.npmjs.com/package/ast-mcp-server)
+[![Node.js 20.19+](https://img.shields.io/badge/Node.js-20.19%2B-339933?logo=node.js&logoColor=white)](https://nodejs.org/)
+[![License: ISC](https://img.shields.io/badge/License-ISC-blue.svg)](LICENSE)
+
+`ast-mcp-server` gives coding agents compact, type-aware access to TypeScript and JavaScript projects. It uses the real compiler project model through `ts-morph`, so declarations, references, rename locations, and diagnostics come from the AST instead of text-search guesses.
+
+Reads are bounded and structured. Writes follow an explicit `prepare → review → apply` protocol with immutable hashes, workspace freshness checks, diagnostic guards, and idempotent receipts.
+
+## The problem
+
+Coding agents often fall back to two generic operations: read files as plain text and write text patches. That works, but it has three predictable costs:
+
+1. **Too much context.** The agent may load hundreds of lines when it only needs one signature or method body. That consumes model context and tokens without improving the answer.
+2. **Fragile edits.** Text patches do not inherently understand declarations, scopes, overloads, or TypeScript diagnostics. A plausible-looking edit can target the wrong construct or introduce a new compiler error.
+3. **Weak cross-file reasoning.** Text search can find matching words, but it cannot reliably distinguish two unrelated symbols with the same name. Project-wide references and renames need the compiler's understanding of the program.
+
+## What this tool does instead
+
+This MCP server gives the agent structural code tools in addition to generic file reads and writes. Under the hood, `ts-morph` uses the TypeScript compiler project model, so the server can reason about declarations and references as code rather than undifferentiated text.
+
+| Need                           | Structural operation      | Returned scope                                        |
+| ------------------------------ | ------------------------- | ----------------------------------------------------- |
+| Understand a file              | `ast_get_outline`         | Signatures without implementation bodies              |
+| Inspect one declaration        | `ast_get_symbol_source`   | Exact source for one function, method, class, or type |
+| Find usages across the project | `ast_find_references`     | Compiler-resolved reference locations                 |
+| Rename a symbol everywhere     | `ast_rename_symbol`       | A reviewed project-wide rename plan                   |
+| Change one implementation      | `ast_replace_symbol_body` | A body-only plan that preserves the declaration       |
+
+Reads can start with a compact outline and fetch exact source only when needed. Mutations are prepared in memory first, compared against baseline diagnostics, and returned as immutable, hash-bound plans. Nothing is written until the caller reviews and explicitly applies the plan.
+
+## Why this helps
+
+- **Less context:** the agent retrieves the smallest structural unit that answers the question instead of loading the complete file by default.
+- **Safer changes:** exact symbol selection, diagnostic deltas, workspace freshness checks, and `prepare → review → apply` reduce the failure modes of ad hoc text editing.
+- **Accurate project-wide operations:** references and renames use compiler resolution rather than matching identifier text with grep.
+
+AST-aware editing is not a proof that a change is semantically correct. The safety comes from combining structural selection with diagnostics, exact previews, reviewed hashes, freshness checks, and fail-closed apply semantics.
+
+The included batch benchmark records a 50% reduction in model round-trips and a 95.21% reduction in serialized context for its search-to-source scenario. These are reproducible scenario measurements, not universal token or latency claims.
+
+## Requirements
+
+- Node.js 20.19 or newer
+- Corepack with Yarn 4.15.0 (pinned by `packageManager`)
+- A target project with a `tsconfig.json`
+
+## Install
+
+Install the published CLI globally while keeping dependency lifecycle scripts disabled:
+
+```bash
+npm install --global ast-mcp-server --ignore-scripts
+ast-tool setup
+```
+
+`--ignore-scripts` prevents dependencies from running `preinstall`, `install`, or `postinstall` hooks. The package and its current runtime dependencies do not require those hooks.
+
+### Install from source
+
+To build the current source instead:
+
+```bash
+git clone https://github.com/yailPeralta/ast-mcp-server.git
+cd ast-mcp-server
+corepack enable
+yarn install --immutable
+yarn build
+```
+
+The repository pins Yarn 4 and commits `enableScripts: false` in `.yarnrc.yml`. Dependency lifecycle scripts are therefore disabled during installation; switching from npm without this setting would merely change logos while preserving the risk.
+
+The package exposes two executables when installed:
+
+- `ast-mcp-server`: MCP stdio server.
+- `ast-tool`: batch, skill-installation, and agent-setup CLI.
+
+## Guided agent setup
+
+The installed package opens the interactive wizard with:
+
+```bash
+ast-tool setup
+```
+
+From a source checkout, use the Yarn script; it builds first and then opens the same wizard:
+
+```bash
+yarn setup
+```
+
+The wizard detects Claude Code and Hermes from `PATH`, shows their executable and version, selects every detected agent by default, and lets you deselect agents before confirmation. For each selected agent it:
+
+1. preflights any existing `ast` MCP registration;
+2. installs the bundled `structural-code-editing` skill;
+3. registers this package's MCP server through the agent's official CLI;
+4. reconnects and verifies the expected tools.
+
+Existing matching registrations and skill files are unchanged. Conflicting MCP registrations fail before any write; remove or rename them explicitly instead of letting a setup script guess. Conflicting skill content also fails closed unless `--force-skill` is explicit.
+
+For automation, make the target set and confirmation explicit:
+
+```bash
+ast-tool setup --agents all --yes
+ast-tool setup --agents claude --yes
+```
+
+From a source checkout, replace `ast-tool` with `yarn setup` in those commands.
+
+## Install the agent skill
+
+The package bundles a `structural-code-editing` skill that teaches an agent when to use the AST tools, how to minimize context, and how to review mutations safely. Install it for both Claude Code and Hermes with one command:
+
+```bash
+ast-tool install-skill all
+```
+
+Or install one target at a time:
+
+```bash
+ast-tool install-skill claude
+ast-tool install-skill hermes
+```
+
+The default is user scope. It writes to Claude Code's personal skill directory and to the active `HERMES_HOME`:
+
+| Target      | Destination                                                                                                |
+| ----------- | ---------------------------------------------------------------------------------------------------------- |
+| Claude Code | `$CLAUDE_CONFIG_DIR/skills/structural-code-editing/SKILL.md`, or `~/.claude/skills/...` by default         |
+| Hermes      | `$HERMES_HOME/skills/software-development/structural-code-editing/SKILL.md`, or `~/.hermes/...` by default |
+
+To commit the skill into one project for Claude Code, use project scope:
+
+```bash
+ast-tool install-skill claude --scope project --project-root /absolute/project
+```
+
+This writes `.claude/skills/structural-code-editing/SKILL.md` below that project. Project scope is intentionally rejected for Hermes because Hermes skills belong to a profile, not a source repository.
+
+Installation is idempotent. Existing identical content is left untouched; different content fails closed unless `--force` is explicit. From an unlinked source checkout, replace `ast-tool` with `yarn node /absolute/path/to/ast-mcp-server/dist/cli.js`.
+
+Claude Code detects changes in an existing skill directory live; restart it if the top-level skills directory did not exist when the session started. In Hermes, run `/reload-skills` or start a new session, then verify with `hermes skills list`.
+
+`install-skill` only installs the skill; it does not configure the MCP transport. Use the guided `setup` command to do both, or complete the client-specific MCP setup below—the instructions are useful, but they have not yet learned to open a stdio socket through positive thinking.
+
+## Use with Claude Code
+
+Claude Code supports local stdio MCP servers. After building this repository, register the server with an absolute entrypoint:
+
+```bash
+AST_MCP_DIR="$(pwd)"
+claude mcp add --scope user --transport stdio ast -- \
+  node "$AST_MCP_DIR/dist/index.js"
+claude mcp get ast
+```
+
+`claude mcp get ast` should report `Status: ✔ Connected`. The `--` separator is required: everything after it is the server command, not a Claude Code option.
+
+The example uses `--scope user`, which makes the server available in all your projects. Use `--scope local` instead to register it only for the project from which you run the command. Avoid committing a project-scoped `.mcp.json` that contains another developer's absolute checkout path.
+
+Start Claude Code inside any TypeScript project with a `tsconfig.json`:
+
+```bash
+cd /absolute/path/to/your-typescript-project
+claude
+```
+
+Then ask Claude to use the `ast` tools. For example:
+
+```text
+Use the ast MCP server to inspect this project.
+First search for UserService, then fetch only the exact source of its create method.
+```
+
+For a reviewed rename:
+
+```text
+Use ast_rename_symbol to prepare renaming UserService.create to createUser.
+Do not apply it yet. Show me the affected files, diagnostic delta, plan hash,
+and the complete operation preview.
+```
+
+After reviewing the preview:
+
+```text
+Apply that operation with ast_apply_operation using the exact operation_id and
+plan_hash returned by the prepare step.
+```
+
+Project-scoped read and prepare tools require `project_root`. Claude should pass the current project directory or its explicit `tsconfig.json` path. Preview and apply calls instead use the prepared operation coordinates; the MCP server itself contains no repository-specific paths.
+
+### MCP or batch CLI?
+
+| Workflow                                         | Recommended interface                     |
+| ------------------------------------------------ | ----------------------------------------- |
+| Interactive exploration or one reviewed mutation | Claude Code MCP tools                     |
+| A known multi-step read pipeline                 | `ast-tool run pipeline.json` through Bash |
+| Prepare now and apply in a later process         | `ast-tool run`, then `ast-tool apply`     |
+
+Use `/mcp` inside Claude Code to inspect server status and tools. Outside the session, use `claude mcp list`, `claude mcp get ast`, or `claude mcp remove ast -s user`.
+
+## Other MCP clients
+
+Hermes Agent:
+
+```bash
+hermes mcp add ast --command node --args /absolute/path/to/ast-mcp-server/dist/index.js
+hermes mcp test ast
+```
+
+Project-scoped tools accept `project_root`, either the project directory or an explicit `tsconfig.json` path. The server contains no repository-specific paths.
+
+## MCP tools
+
+| Tool                        | Purpose                                                              | Mutates files |
+| --------------------------- | -------------------------------------------------------------------- | ------------- |
+| `ast_list_files`            | Paginated, project-relative source file inventory                    | No            |
+| `ast_get_outline`           | Body-free declaration signatures; detailed symbol metadata is opt-in | No            |
+| `ast_get_symbol_source`     | Exact source for one declaration                                     | No            |
+| `ast_search_symbols`        | Paginated structural symbol discovery                                | No            |
+| `ast_find_references`       | Compiler-resolved references with bounded context                    | No            |
+| `ast_get_diagnostics`       | Project- or file-scoped TypeScript diagnostics                       | No            |
+| `ast_rename_symbol`         | Prepare a project-wide rename                                        | No            |
+| `ast_replace_symbol_body`   | Prepare a body-only replacement while preserving the signature       | No            |
+| `ast_get_operation_preview` | Retrieve the complete retained diff for a prepared plan              | No            |
+| `ast_apply_operation`       | Apply one reviewed, hash-bound plan                                  | Yes           |
+
+Read results use project-relative paths, deterministic ordering, structured MCP output, and pagination where result sets can grow with the project.
+
+## Batch CLI
+
+`ast-tool` lets Claude Code and other Bash-capable clients collapse a known structural pipeline into one shell call:
+
+```bash
+ast-tool validate pipeline.json
+ast-tool run pipeline.json
+cat pipeline.json | ast-tool run -
+```
+
+Example search-to-source pipeline:
+
+```json
+{
+  "version": 1,
+  "project_root": "/absolute/project",
+  "steps": [
+    {
+      "id": "search",
+      "tool": "ast_search_symbols",
+      "input": { "query": "UserService", "limit": 20 }
+    },
+    {
+      "id": "source",
+      "tool": "ast_get_symbol_source",
+      "input": {
+        "file_path": { "$ref": "#/steps/search/symbols/0/file" },
+        "symbol_path": { "$ref": "#/steps/search/symbols/0/symbol_path" }
+      }
+    }
+  ],
+  "emit": { "$ref": "#/steps/source" }
+}
+```
+
+A `$ref` is an RFC 6901 JSON Pointer rooted at prior step results. References cannot point forward. If `emit` is omitted, only the final step result is returned; intermediate results remain inside the process.
+
+### Bounded foreach
+
+```json
+{
+  "version": 1,
+  "project_root": "/absolute/project",
+  "limits": { "concurrency": 4 },
+  "steps": [
+    { "id": "files", "tool": "ast_list_files", "input": { "limit": 20 } },
+    {
+      "id": "outlines",
+      "tool": "ast_get_outline",
+      "foreach": { "$ref": "#/steps/files/files" },
+      "input": { "file_path": { "$item": "" } }
+    }
+  ]
+}
+```
+
+`$item` accepts an empty pointer for the complete item or `/field` for one field. Foreach is read-only, order-preserving, fail-fast, and concurrency-bounded.
+
+### Batch limits
+
+- Input document: 1 MiB.
+- Steps: 50.
+- Total tool invocations: 500.
+- Foreach items per step: 200.
+- Read concurrency: default 4, maximum 16.
+- Each retained step result and final serialized output: 10 MiB.
+- Total retained intermediate context: 50 MiB.
+- One project root per pipeline.
+- No branches, eval, embedded JavaScript, while loops, or arbitrary transformations.
+
+Success is one JSON value on stdout. Errors are structured JSON on stderr. Exit code 0 is success, 1 is execution/apply failure, and 2 is usage or schema failure.
+
+## Reviewed mutations
+
+### MCP process
+
+Rename and body replacement never write directly:
+
+1. Call `ast_rename_symbol` or `ast_replace_symbol_body`.
+2. Review the diagnostic delta, affected files, `blocked`, and `plan_hash`.
+3. Fetch complete diffs with `ast_get_operation_preview` when needed.
+4. Call `ast_apply_operation` with both `operation_id` and `plan_hash`.
+
+MCP plans live in a bounded in-memory store and do not survive a server restart.
+
+### CLI process boundary
+
+A batch may contain at most one prepare operation. It must be the final step and cannot use foreach. `ast_apply_operation` and arbitrary preview calls are forbidden inside batch documents.
+
+A CLI prepare writes an exact private plan and returns top-level `operation_id`, `plan_hash`, and `plan_file` even when `emit` omits them:
+
+```bash
+ast-tool run prepare-rename.json
+ast-tool apply /path/from/plan_file.astplan --plan-hash <reviewed-sha256>
+```
+
+The default plan directory is:
+
+```text
+${XDG_STATE_HOME:-~/.local/state}/ast-tool/plans
+```
+
+Set `AST_TOOL_STATE_DIR` to isolate it. Directories are mode `0700`; plans are mode `0600`, atomically replaced, size-bounded, versioned, and expire with the prepared operation. Plan files contain exact proposed source bytes and must be treated as private code.
+
+Apply loads the exact retained postimages, requires the separately supplied reviewed hash, validates serialized byte hashes and contained paths, rechecks the complete source/config workspace, stages writes, verifies postimages, and persists an applied receipt inside the same cooperative workspace lock. A later CLI invocation can replay that receipt idempotently, including after the preparation TTL.
+
+### Guarantee boundary
+
+The server does **not** claim a filesystem-wide transaction:
+
+- Replacement is atomic per file where the local filesystem provides atomic rename.
+- A multi-file apply has a short interval in which some replacements may already be visible.
+- Rollback is best effort and refuses to overwrite a file changed by another writer after replacement.
+- MCP and CLI apply share a fail-closed filesystem lock keyed by canonical `tsconfig.json` when they use the same state directory. It does not coordinate editors, NFS writers, or hostile external processes.
+- Receipt persistence runs before that lock is released. If receipt storage fails after source replacement, apply exits non-zero and reports that verified postimages may be present; retry recovers the receipt only when the complete workspace exactly matches the reviewed post-workspace fingerprint.
+- A hard process crash can leave a stale lock. Remove it only after inspecting its metadata and proving no apply is running; exact complete postimages can then recover the receipt, while partial or divergent state remains a conflict.
+- Source encoding support is UTF-8, with or without BOM. Unsupported encodings are rejected.
+
+## Development gates
+
+```bash
+yarn format:check
+yarn lint
+yarn typecheck
+yarn test
+yarn build
+yarn test:mcp
+yarn test:cli
+yarn test:package
+yarn npm audit --all --recursive
+yarn pack --dry-run
+```
+
+`test:mcp` exercises the built stdio server. `test:cli` runs a read pipeline and a prepare/apply/replay workflow across separate Node processes.
+
+## Benchmarks
+
+```bash
+yarn benchmark /absolute/project --sample 20 --output benchmark/results/project.json
+yarn benchmark:corpus benchmark/task-corpus.json --output benchmark/results/self-corpus.json
+yarn benchmark:batch --iterations 5 --output benchmark/results/self-batch.json
+```
+
+The batch benchmark compares two separate client calls with one batch invocation in fresh Node processes, recording model round-trips, actual tool invocations, wall time, maximum RSS, and serialized character counts. Character counts are not model-specific token estimates. See `benchmark/README.md` for methodology and limitations.
+
+## Scope
+
+- TypeScript and JavaScript projects understood by the TypeScript compiler.
+- Structural rename and callable-body replacement.
+- Declarative DAG-like pipelines with prior-result references and bounded foreach.
+- No arbitrary signature migration, file creation/deletion plans, cross-language refactors, or general-purpose scripting language.

package/benchmark/README.md

@@ -0,0 +1,86 @@
+# Benchmark methodology
+
+The benchmark suite measures two separate properties. It deliberately avoids converting characters into tokens because tokenizer choice changes the answer.
+
+## Project payload and latency
+
+`scripts/benchmark.mjs`:
+
+1. Loads a real TypeScript project from disk.
+2. Selects the lexicographically first non-declaration source files after an optional filter.
+3. Serializes the default compact outline payload (`file` plus `outline`).
+4. Also records the larger opt-in payload containing detailed symbol metadata.
+5. Reports fresh project load, cold cached-session, warm cached-session, and per-file outline latency.
+
+Run:
+
+```bash
+yarn benchmark /absolute/project/path --sample 20 --output benchmark/results/project.json
+```
+
+The selection rule is deterministic but not statistically representative of every repository. Use `--filter` and multiple samples when comparing particular modules.
+
+## Compact-first task corpus
+
+`task-corpus.json` defines twelve questions over this repository. Each task declares:
+
+- files a full-file workflow would load;
+- declarations a compact-first workflow would request after inspecting outlines;
+- evidence strings necessary to answer that task.
+
+`scripts/benchmark-corpus.mjs` compares:
+
+- baseline: complete text of every declared file;
+- compact: default outline payloads plus only the selected declaration sources.
+
+It fails unless every predefined evidence string is present in the compact payload. This makes the check reproducible, but it does not prove that every possible reviewer would reach an equivalent conclusion.
+
+Run:
+
+```bash
+yarn benchmark:corpus benchmark/task-corpus.json --output benchmark/results/self-corpus.json
+```
+
+## Interpreting results
+
+- Character reduction measures serialized payload size, not billed model tokens.
+- Negative reduction on a tiny file is valid: protocol structure can cost more than reading that file directly.
+- Project-load latency and warm-session latency should be considered separately.
+- Detailed symbol metadata is opt-in because it repeats signature text and can dominate small files.
+- Benchmark scripts record absolute local paths and timestamps. Checked reports replace local paths with descriptive placeholders; regenerate them when comparing another machine or revision.
+
+## Batch orchestration
+
+`scripts/benchmark-batch.mjs` compares a two-operation search-to-source workflow in fresh Node processes:
+
+- separate: the client receives the broad symbol-search payload, selects the first match, then requests its source;
+- batch: `ast-tool` performs the same two MCP tool invocations in process and emits only the final source result.
+
+Run:
+
+```bash
+yarn benchmark:batch --iterations 5 --output benchmark/results/self-batch.json
+```
+
+The report records logical model round-trips, actual tool invocations, median wall time, process maximum RSS, final-result characters, and total characters exposed to the model. It does not include model inference latency, so the practical wall-time benefit of removing a model round-trip will be larger than this local process-only comparison. Maximum RSS includes the Node runtime and TypeScript project in each fresh worker.
+
+## Checked results
+
+Results generated on 2026-08-03 with Node.js v24.16.0:
+
+| Workload        |   Sample | Full source chars | Compact chars | Reduction | Fresh load | Warm session |
+| --------------- | -------: | ----------------: | ------------: | --------: | ---------: | -----------: |
+| This repository | 20 files |            77,982 |        12,512 |    83.96% |     543 ms |      1.86 ms |
+| `x-scraper`     | 20 files |           178,110 |        17,036 |    90.44% |   3,951 ms |    180.16 ms |
+| Task corpus     | 12 tasks |            83,289 |        28,372 |    65.94% |        n/a |          n/a |
+
+The task corpus passed all 12 predefined evidence checks. Its compact count includes the selected declaration bodies, not just outlines. Exact raw reports are under `benchmark/results/`.
+
+The checked broad search-to-source batch result used five fresh-process samples:
+
+| Mode     | Model round-trips | Tool invocations | Context chars | Median wall time | Median max RSS |
+| -------- | ----------------: | ---------------: | ------------: | ---------------: | -------------: |
+| Separate |                 2 |                2 |        11,514 |        536.19 ms |      377.54 MB |
+| Batch    |                 1 |                2 |           552 |        535.24 ms |      378.35 MB |
+
+That run reduced model round-trips by 50% and serialized context by 95.21%. Local execution latency decreased by 0.18% and RSS increased by 0.21%, both small enough to treat as noise rather than a performance claim. The batch value is orchestration/context reduction; it does not make TypeScript analysis free.